jpountz opened a new issue, #13127:
URL: https://github.com/apache/lucene/issues/13127
### Description
This failure is not reproducible, which is maybe not too surprising given
that the test involves concurrency. A few things are interesting:
- This is the same failure I had got on
https://github.com/apache/lucene/pull/12205 that made me nervous about merging.
- The seed provided for the recreation configures
`checkPendingFlushOnUpdate=false` on the `IndexWriterConfig`, and we recently
merged a related optimization: https://github.com/apache/lucene/pull/13115.
This makes me wonder if we have a longstanding problem, which is becoming
apparent as we reduce contention on the indexing path.
```
org.apache.lucene.sandbox.codecs.idversion.TestIDVersionPostingsFormat >
testGlobalVersions FAILED
java.lang.AssertionError: maxSeqNo must be greater or equal to 7442 but
was 7441
at
__randomizedtesting.SeedInfo.seed([B97A2BDBC7E40BF6:B4D76006D5101E6]:0)
at
[email protected]/org.apache.lucene.index.DocumentsWriterDeleteQueue.close(DocumentsWriterDeleteQueue.java:325)
at
[email protected]/org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:659)
at
[email protected]/org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:576)
at
[email protected]/org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:381)
at
[email protected]/org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:355)
at
[email protected]/org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:345)
at
[email protected]/org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:170)
at
[email protected]/org.apache.lucene.search.SearcherManager.refreshIfNeeded(SearcherManager.java:144)
at
[email protected]/org.apache.lucene.search.SearcherManager.refreshIfNeeded(SearcherManager.java:52)
at
[email protected]/org.apache.lucene.search.ReferenceManager.doMaybeRefresh(ReferenceManager.java:167)
at
[email protected]/org.apache.lucene.search.ReferenceManager.maybeRefresh(ReferenceManager.java:213)
at
org.apache.lucene.sandbox.codecs.idversion.TestIDVersionPostingsFormat.testGlobalVersions(TestIDVersionPostingsFormat.java:907)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at
[email protected]/com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
at
[email protected]/com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
at
[email protected]/com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
at
[email protected]/com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
at
[email protected]/org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
at
[email protected]/org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
at
[email protected]/org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
at
[email protected]/org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
at
[email protected]/org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
at [email protected]/org.junit.rules.RunRules.evaluate(RunRules.java:20)
at
[email protected]/com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
[email protected]/com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
at
[email protected]/com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
at
[email protected]/com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
at
[email protected]/com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
at
[email protected]/com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
at
[email protected]/com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
at
[email protected]/com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
at
[email protected]/org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
at
[email protected]/com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
[email protected]/org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
at
[email protected]/com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at
[email protected]/com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at
[email protected]/com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
[email protected]/com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
[email protected]/org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at
[email protected]/org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
at
[email protected]/org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
at
[email protected]/org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
at
[email protected]/org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
at [email protected]/org.junit.rules.RunRules.evaluate(RunRules.java:20)
at
[email protected]/com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at
[email protected]/com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
at
[email protected]/com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
at java.base/java.lang.Thread.run(Thread.java:833)
org.apache.lucene.sandbox.codecs.idversion.TestIDVersionPostingsFormat >
test suite's output saved to
/opt/buildkite-agent/builds/bk-agent-prod-gcp-1708533031230287464/elastic/apache-lucene-periodic/lucene/sandbox/build/test-results/test/outputs/OUTPUT-org.apache.lucene.sandbox.codecs.idversion.TestIDVersionPostingsFormat.txt,
copied below:
> java.lang.AssertionError: maxSeqNo must be greater or equal to 7442
but was 7441
> at
__randomizedtesting.SeedInfo.seed([B97A2BDBC7E40BF6:B4D76006D5101E6]:0)
> at
[email protected]/org.apache.lucene.index.DocumentsWriterDeleteQueue.close(DocumentsWriterDeleteQueue.java:325)
> at
[email protected]/org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:659)
> at
[email protected]/org.apache.lucene.index.IndexWriter.getReader(IndexWriter.java:576)
> at
[email protected]/org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:381)
> at
[email protected]/org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:355)
> at
[email protected]/org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:345)
> at
[email protected]/org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:170)
> at
[email protected]/org.apache.lucene.search.SearcherManager.refreshIfNeeded(SearcherManager.java:144)
> at
[email protected]/org.apache.lucene.search.SearcherManager.refreshIfNeeded(SearcherManager.java:52)
> at
[email protected]/org.apache.lucene.search.ReferenceManager.doMaybeRefresh(ReferenceManager.java:167)
> at
[email protected]/org.apache.lucene.search.ReferenceManager.maybeRefresh(ReferenceManager.java:213)
> at
org.apache.lucene.sandbox.codecs.idversion.TestIDVersionPostingsFormat.testGlobalVersions(TestIDVersionPostingsFormat.java:907)
> at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
> at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:568)
> at
[email protected]/com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
> at
[email protected]/com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
> at
[email protected]/com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
> at
[email protected]/com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
> at
[email protected]/org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
> at
[email protected]/org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> at
[email protected]/org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
> at
[email protected]/org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> at
[email protected]/org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> at
[email protected]/org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at
[email protected]/com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
[email protected]/com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
> at
[email protected]/com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
> at
[email protected]/com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
> at
[email protected]/com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
> at
[email protected]/com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
> at
[email protected]/com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
> at
[email protected]/com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
> at
[email protected]/org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> at
[email protected]/com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
[email protected]/org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
> at
[email protected]/com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
[email protected]/com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
[email protected]/com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
[email protected]/com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
[email protected]/org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> at
[email protected]/org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> at
[email protected]/org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> at
[email protected]/org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> at
[email protected]/org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
> at
[email protected]/org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at
[email protected]/com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
[email protected]/com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
> at
[email protected]/com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
> at java.base/java.lang.Thread.run(Thread.java:833)
2> NOTE: reproduce with: gradlew test --tests
TestIDVersionPostingsFormat.testGlobalVersions -Dtests.seed=B97A2BDBC7E40BF6
-Dtests.locale=yo-NG -Dtests.timezone=Australia/Broken_Hill
-Dtests.asserts=true -Dtests.file.encoding=UTF-8
2> Feb 21, 2024 4:39:34 PM
com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
2> NOTE: test params are: codec=CheapBastard,
sim=Asserting(RandomSimilarity(queryNorm=false): {id=LM
Jelinek-Mercer(0.700000)}), locale=yo-NG, timezone=Australia/Broken_Hill
2> NOTE: Linux 5.15.0-1051-gcp amd64/Oracle Corporation 17.0.2
(64-bit)/cpus=32,threads=1,free=118531608,total=270532608
2> NOTE: All tests run in this JVM: [TestIDVersionPostingsFormat]
```
### Gradle command to reproduce
```
gradlew :lucene:sandbox:test --tests
"org.apache.lucene.sandbox.codecs.idversion.TestIDVersionPostingsFormat.testGlobalVersions"
-Ptests.jvms=12 -Ptests.jvmargs= -Ptests.seed=B97A2BDBC7E40BF6
-Ptests.gui=true -Ptests.file.encoding=US-ASCII -Ptests.vectorsize=128
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
