I think it's no longer reproducible because I added a synchronized around the test logging method. It should be resolved now?
On Tue, Apr 14, 2026, 8:55 PM Dawid Weiss <[email protected]> wrote: > > I can't reproduce this, even when looping, but your analysis seems > correct, neoremind. I filed this - > > https://github.com/apache/lucene/pull/15957 > > Dawid > > On Wed, Apr 8, 2026 at 7:17 AM xu <[email protected]> wrote: > >> I think the merge itself isn't actually getting skipped, it's a >> thread-safety issue in the test. >> The test collects InfoStream messages into a ArrayList<String>, but with >> ConcurrentMergeScheduler, the "merged new segment" message gets written by >> the merge background thread. One thread writes to a non-thread-safe list, >> the other test thread reads, oops, visibility problem in JMM. To address, >> simply change: >> >> List<String> infoStream = new ArrayList<>(); >> >> to: >> >> List<String> infoStream = Collections.synchronizedList(new ArrayList<>()); >> >> I can reproduce this by putting the test in a for-loop running 10k times, >> it shows roughly 0.3% chance of failure. >> >> To verify the theory, I quickly create a JCStress test (a dummy one >> though, but it illustrates the point). It shows the write from one thread >> is completely or partially invisible to the reader thread: >> >> @JCStressTest >> @State >> @Outcome(id = "1, 42", expect = Expect.ACCEPTABLE, desc = "Fully visible >> write.") >> @Outcome(id = "0, -1", expect = Expect.ACCEPTABLE, desc = "Write not >> visible at all.") >> @Outcome( expect = Expect.ACCEPTABLE_INTERESTING, desc = >> "Partial / stale visibility.") >> public class ArrayListVisibilityTest { >> >> final List<Integer> list = new ArrayList<>(); >> >> @Actor >> public void writer() { >> list.add(42); >> } >> >> @Actor >> public void reader(II_Result r) { >> int size = list.size(); >> r.r1 = size; >> try { >> r.r2 = size > 0 ? list.get(0) : -1; >> } catch (Exception e) { >> // ArrayIndexOutOfBoundsException is possible under races >> r.r2 = -2; >> } >> } >> } >> >> ``` >> RESULT SAMPLES FREQ EXPECT DESCRIPTION >> 0, -1 1,838,722,183 67.78% Acceptable Write not visible at >> all. >> 1, -2 106,178 <0.01% Interesting Partial / stale >> visibility. >> 1, 42 873,770,813 32.21% Acceptable Fully visible write. >> ``` >> In practice, a lock (or a memory barrier) would guarantee visibility, in >> essence, the merge thread's CPU flushes writes to store buffer, and the >> test thread's CPU is able to snoop the invalid to get the freshly added >> message (at least this should be how things work on x86). >> >> Thanks, >> neoremind >> >> Michael Sokolov <[email protected]> 于2026年4月6日周一 09:52写道： >> >>> I just added this TestInfoStream.testMergeSmallIndex test so I thought >>> I'd try to reproduce, but it didn't repro for me. The failure >>> indicates that no segment merge was logged. I'm not sure how it could >>> get skipped though. >>> >>> On Sun, Apr 5, 2026 at 1:57 PM Policeman Jenkins Server via builds >>> <[email protected]> wrote: >>> > >>> > Build: https://jenkins.thetaphi.de/job/Lucene-MMAPv2-Linux/5230/ >>> > Java: 64bit/hotspot/jdk-27-ea+14 -XX:-UseCompressedOops -XX:+UseZGC >>> > >>> > 3 tests failed. >>> > FAILED: org.apache.lucene.index.TestInfoStream.testMergeSmallIndex >>> > >>> > Error Message: >>> > java.lang.AssertionError: expected:<1> but was:<0> >>> > >>> > Stack Trace: >>> > java.lang.AssertionError: expected:<1> but was:<0> >>> > at >>> __randomizedtesting.SeedInfo.seed([F77FD74A101B3B0B:45303098296E3A19]:0) >>> > at org.junit.Assert.fail(Assert.java:89) >>> > at org.junit.Assert.failNotEquals(Assert.java:835) >>> > at org.junit.Assert.assertEquals(Assert.java:647) >>> > at org.junit.Assert.assertEquals(Assert.java:633) >>> > at >>> org.apache.lucene.index.TestInfoStream.testMergeSmallIndex(TestInfoStream.java:151) >>> > at >>> java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104) >>> > at java.base/java.lang.reflect.Method.invoke(Method.java:565) >>> > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1763) >>> > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946) >>> > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982) >>> > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996) >>> > at >>> org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48) >>> > at >>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) >>> > at >>> org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45) >>> > at >>> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) >>> > at >>> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) >>> > at org.junit.rules.RunRules.evaluate(RunRules.java:20) >>> > at >>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) >>> > at >>> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) >>> > at >>> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843) >>> > at >>> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490) >>> > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955) >>> > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840) >>> > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891) >>> > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902) >>> > at >>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) >>> > at >>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) >>> > at >>> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38) >>> > at >>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) >>> > at >>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) >>> > at >>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) >>> > at >>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) >>> > at >>> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:52) >>> > at >>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) >>> > at >>> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) >>> > at >>> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) >>> > at >>> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47) >>> > at org.junit.rules.RunRules.evaluate(RunRules.java:20) >>> > at >>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) >>> > at >>> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) >>> > at >>> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850) >>> > at java.base/java.lang.Thread.run(Thread.java:1527) >>> > >>> > >>> > FAILED: >>> org.apache.lucene.tests.store.TestMockDirectoryWrapper.testIsLoaded >>> > >>> > Error Message: >>> > java.lang.AssertionError >>> > >>> > Stack Trace: >>> > java.lang.AssertionError >>> > at >>> __randomizedtesting.SeedInfo.seed([F77FD74A101B3B0B:D95E8D3E0CAB8C7C]:0) >>> > at org.junit.Assert.fail(Assert.java:87) >>> > at org.junit.Assert.assertTrue(Assert.java:42) >>> > at org.junit.Assert.assertTrue(Assert.java:53) >>> > at >>> org.apache.lucene.tests.store.BaseDirectoryTestCase.testIsLoaded(BaseDirectoryTestCase.java:1663) >>> > at >>> org.apache.lucene.tests.store.BaseDirectoryTestCase.testIsLoaded(BaseDirectoryTestCase.java:1623) >>> > at >>> java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104) >>> > at java.base/java.lang.reflect.Method.invoke(Method.java:565) >>> > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1763) >>> > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946) >>> > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982) >>> > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996) >>> > at >>> org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48) >>> > at >>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) >>> > at >>> org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45) >>> > at >>> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) >>> > at >>> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) >>> > at org.junit.rules.RunRules.evaluate(RunRules.java:20) >>> > at >>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) >>> > at >>> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) >>> > at >>> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843) >>> > at >>> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490) >>> > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955) >>> > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840) >>> > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891) >>> > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902) >>> > at >>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) >>> > at >>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) >>> > at >>> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38) >>> > at >>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) >>> > at >>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) >>> > at >>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) >>> > at >>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) >>> > at >>> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:52) >>> > at >>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) >>> > at >>> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) >>> > at >>> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) >>> > at >>> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47) >>> > at org.junit.rules.RunRules.evaluate(RunRules.java:20) >>> > at >>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) >>> > at >>> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) >>> > at >>> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850) >>> > at java.base/java.lang.Thread.run(Thread.java:1527) >>> > >>> > >>> > FAILED: >>> org.apache.lucene.tests.store.TestSerializedIOCountingDirectory.testIsLoaded >>> > >>> > Error Message: >>> > java.lang.AssertionError >>> > >>> > Stack Trace: >>> > java.lang.AssertionError >>> > at >>> __randomizedtesting.SeedInfo.seed([F77FD74A101B3B0B:D95E8D3E0CAB8C7C]:0) >>> > at org.junit.Assert.fail(Assert.java:87) >>> > at org.junit.Assert.assertTrue(Assert.java:42) >>> > at org.junit.Assert.assertTrue(Assert.java:53) >>> > at >>> org.apache.lucene.tests.store.BaseDirectoryTestCase.testIsLoaded(BaseDirectoryTestCase.java:1663) >>> > at >>> org.apache.lucene.tests.store.BaseDirectoryTestCase.testIsLoaded(BaseDirectoryTestCase.java:1623) >>> > at >>> java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104) >>> > at java.base/java.lang.reflect.Method.invoke(Method.java:565) >>> > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1763) >>> > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946) >>> > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982) >>> > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996) >>> > at >>> org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48) >>> > at >>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) >>> > at >>> org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45) >>> > at >>> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) >>> > at >>> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) >>> > at org.junit.rules.RunRules.evaluate(RunRules.java:20) >>> > at >>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) >>> > at >>> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) >>> > at >>> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843) >>> > at >>> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490) >>> > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955) >>> > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840) >>> > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891) >>> > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902) >>> > at >>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) >>> > at >>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) >>> > at >>> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38) >>> > at >>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) >>> > at >>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) >>> > at >>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) >>> > at >>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) >>> > at >>> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:52) >>> > at >>> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) >>> > at >>> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) >>> > at >>> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) >>> > at >>> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47) >>> > at org.junit.rules.RunRules.evaluate(RunRules.java:20) >>> > at >>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) >>> > at >>> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) >>> > at >>> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850) >>> > at java.base/java.lang.Thread.run(Thread.java:1527) >>> > >>> > --------------------------------------------------------------------- >>> > To unsubscribe, e-mail: [email protected] >>> > For additional commands, e-mail: [email protected] >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: [email protected] >>> For additional commands, e-mail: [email protected] >>> >>> >> >> -- >> >> Best regards, >> >> Xu >> >
