In general, you should try to pass all the parameters that the repro line mentions as these may affect the result. I understand this is problematic if you want to re-launch from intellij, in which case I'd suggest two options -
1) run the test in intellij via gradle, setting all the options that the repro line mentioned; this should work most of the time, 2) add @Repeat on the test you wish to trigger to fail and run that from intellij, then try to pick the seed that failed and add @Seed on the test, once you've identified it as the problem. Hope this helps, On Thu, Sep 25, 2025 at 11:35 AM Alan Woodward <[email protected]> wrote: > Aha, yes, I need to set -Ptests.nightly=true as it changes the behaviour > of `rarely()`. Thanks, this drove me absolutely nuts for about three hours > yesterday. > > > On 24 Sep 2025, at 16:23, Michael Sokolov <[email protected]> wrote: > > > > did you also set the other test parameters like -Ptests.nightly=true ? > > > > On Wed, Sep 24, 2025 at 7:01 AM Alan Woodward <[email protected]> > wrote: > >> > >> Hi all, > >> > >> I’m seeing some strange behaviour trying to reproduce a test failure in > IntelliJ, compared to what happens when I run the test on the command line > via gradle. > >> > >> The following failed in the elasticsearch CI this morning, and > reproduces when I run it on my laptop: > >> > >> ./gradlew :lucene:test-framework:test --tests > "org.apache.lucene.tests.store.TestMockDirectoryWrapper.testIsLoadedOnSlice" > -Ptests.jvms=12 -Ptests.jvmargs= -Ptests.seed=7AA64BB2BA5EE192 > -Ptests.useSecurityManager=true -Ptests.nightly=true -Ptests.gui=true > -Ptests.file.encoding=US-ASCII -Ptests.vectorsize=128 > -Ptests.forceintegervectors=true > >> > >> However, if I take that seed and fix it on TestMockDirectoryWrapper, > the same test passes when run inside IntelliJ. Adding some debug println > statements indicates that the seed is indeed the same, but the random code > paths taken seem to be different. I made the following changes to the code: > >> > >> --- > >> > >> $ git diff > >> diff --git > a/lucene/test-framework/src/java/org/apache/lucene/tests/store/BaseDirectoryTestCase.java > b/lucene/test-framework/src/java/org/apache/lucene/tests/store/BaseDirectoryTestCase.java > >> index 27bfa3c72f1..e522168b43c 100644 > >> --- > a/lucene/test-framework/src/java/org/apache/lucene/tests/store/BaseDirectoryTestCase.java > >> +++ > b/lucene/test-framework/src/java/org/apache/lucene/tests/store/BaseDirectoryTestCase.java > >> @@ -22,6 +22,7 @@ import static org.hamcrest.Matchers.containsString; > >> import static org.hamcrest.Matchers.hasItemInArray; > >> import static org.hamcrest.Matchers.not; > >> > >> +import com.carrotsearch.randomizedtesting.RandomizedContext; > >> import com.carrotsearch.randomizedtesting.RandomizedTest; > >> import com.carrotsearch.randomizedtesting.generators.RandomBytes; > >> import com.carrotsearch.randomizedtesting.generators.RandomNumbers; > >> @@ -1647,6 +1648,10 @@ public abstract class BaseDirectoryTestCase > extends LuceneTestCase { > >> } > >> var loaded = in.isLoaded(); > >> > >> + > System.out.println(RandomizedContext.current().getRunnerSeedAsString()); > >> + System.out.println(dir.toString()); > >> + System.out.println(dir.getClass().getName()); > >> + > >> if (Constants.WINDOWS) { > >> // On Windows, we temporarily don't care until this is fixed: > #14050 > >> } else if (FilterDirectory.unwrap(dir) instanceof MMapDirectory > >> diff --git > a/lucene/test-framework/src/test/org/apache/lucene/tests/store/TestMockDirectoryWrapper.java > b/lucene/test-framework/src/test/org/apache/lucene/tests/store/TestMockDirectoryWrapper.java > >> index b5ca29dcbb0..d06e32b4ba1 100644 > >> --- > a/lucene/test-framework/src/test/org/apache/lucene/tests/store/TestMockDirectoryWrapper.java > >> +++ > b/lucene/test-framework/src/test/org/apache/lucene/tests/store/TestMockDirectoryWrapper.java > >> @@ -21,6 +21,7 @@ import java.io.FileNotFoundException; > >> import java.io.IOException; > >> import java.nio.file.NoSuchFileException; > >> import java.nio.file.Path; > >> +import com.carrotsearch.randomizedtesting.annotations.Seed; > >> import org.apache.lucene.document.Document; > >> import org.apache.lucene.store.ByteArrayDataInput; > >> import org.apache.lucene.store.ByteBuffersDirectory; > >> @@ -33,6 +34,7 @@ import > org.apache.lucene.tests.index.RandomIndexWriter; > >> > >> // See: https://issues.apache.org/jira/browse/SOLR-12028 Tests cannot > remove files on Windows > >> // machines occasionally > >> +@Seed("7AA64BB2BA5EE192") > >> public class TestMockDirectoryWrapper extends BaseDirectoryTestCase { > >> > >> @Override > >> > >> — > >> > >> Running from gradle I get the following output: > >> > >> 1> 7AA64BB2BA5EE192 > >> 1> > >> MockDirectoryWrapper(NRTCachingDirectory(MMapDirectory@/Users/romseygeek/projects/lucene/lucene/test-framework/build/tests-tmp/lucene.tests.store.TestMockDirectoryWrapper_7AA64BB2BA5EE192-008/index-MMapDirectory-001 > lockFactory=org.apache.lucene.store.NativeFSLockFactory@2bb5ff68; > maxCacheMB=0.9546899795532227 maxMergeSizeMB=0.9689407348632812)) > >> 1> org.apache.lucene.tests.store.MockDirectoryWrapper > >> > >> And running via IntelliJ: > >> > >> 7AA64BB2BA5EE192 > >> MockDirectoryWrapper(ByteBuffersDirectory@7e97aee2 > lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@dfeade5) > >> org.apache.lucene.tests.store.MockDirectoryWrapper > >> > >> Same seed, different Directory implementations. This happens with both > the IntelliJ and gradle test runners, and I’ve checked that both situations > use the same JDK. I am pretty stumped at the moment. Any ideas, anyone? > >> > >> - Alan > >> > >> > >> > >> > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: [email protected] > >> For additional commands, e-mail: [email protected] > >> > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
