[
https://issues.apache.org/jira/browse/LUCENE-8639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Simon Willnauer resolved LUCENE-8639.
-------------------------------------
Resolution: Fixed
Fix Version/s: master (9.0)
7.7
8.0
> SeqNo accounting in IW is broken if many threads start indexing while we
> flush.
> -------------------------------------------------------------------------------
>
> Key: LUCENE-8639
> URL: https://issues.apache.org/jira/browse/LUCENE-8639
> Project: Lucene - Core
> Issue Type: Improvement
> Reporter: Simon Willnauer
> Priority: Major
> Fix For: 8.0, 7.7, master (9.0)
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> While this is rare in the wild we have a test failure that shows that our
> seqNo accounting is broken when we carry over seqNo to a new delete queue.
> We had this test-failure:
> {noformat}
> 6:06:08 [junit4] Suite: org.apache.lucene.index.TestIndexTooManyDocs
> 16:06:08 [junit4] 2> ??? 14, 2019 9:05:46 ?
> com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler
> uncaughtException
> 16:06:08 [junit4] 2> WARNING: Uncaught exception in thread:
> Thread[Thread-8,5,TGRP-TestIndexTooManyDocs]
> 16:06:08 [junit4] 2> java.lang.AssertionError: seqNo=7 vs maxSeqNo=6
> 16:06:08 [junit4] 2> at
> __randomizedtesting.SeedInfo.seed([43B7C75B765AFEBD]:0)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.DocumentsWriterDeleteQueue.getNextSequenceNumber(DocumentsWriterDeleteQueue.java:482)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.DocumentsWriterDeleteQueue.add(DocumentsWriterDeleteQueue.java:168)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.DocumentsWriterDeleteQueue.add(DocumentsWriterDeleteQueue.java:146)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.DocumentsWriterPerThread.finishDocument(DocumentsWriterPerThread.java:362)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:264)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:494)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1594)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1586)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.TestIndexTooManyDocs.lambda$testIndexTooManyDocs$0(TestIndexTooManyDocs.java:70)
> 16:06:08 [junit4] 2> at java.lang.Thread.run(Thread.java:748)
> 16:06:08 [junit4] 2>
> 16:06:08 [junit4] 2> ??? 14, 2019 9:05:46 ?
> com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler
> uncaughtException
> 16:06:08 [junit4] 2> WARNING: Uncaught exception in thread:
> Thread[Thread-9,5,TGRP-TestIndexTooManyDocs]
> 16:06:08 [junit4] 2> java.lang.AssertionError: seqNo=6 vs maxSeqNo=6
> 16:06:08 [junit4] 2> at
> __randomizedtesting.SeedInfo.seed([43B7C75B765AFEBD]:0)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.DocumentsWriterDeleteQueue.getNextSequenceNumber(DocumentsWriterDeleteQueue.java:482)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.DocumentsWriterDeleteQueue.add(DocumentsWriterDeleteQueue.java:168)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.DocumentsWriterDeleteQueue.add(DocumentsWriterDeleteQueue.java:146)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.DocumentsWriterPerThread.finishDocument(DocumentsWriterPerThread.java:362)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:264)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:494)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1594)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1586)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.TestIndexTooManyDocs.lambda$testIndexTooManyDocs$0(TestIndexTooManyDocs.java:70)
> 16:06:08 [junit4] 2> at java.lang.Thread.run(Thread.java:748)
> 16:06:08 [junit4] 2>
> 16:06:08 [junit4] 2> ??? 14, 2019 11:05:45 ?
> com.carrotsearch.randomizedtesting.ThreadLeakControl$2 evaluate
> 16:06:08 [junit4] 2> WARNING: Suite execution timed out:
> org.apache.lucene.index.TestIndexTooManyDocs
> 16:06:08 [junit4] 2> 1) Thread[id=20,
> name=SUITE-TestIndexTooManyDocs-seed#[43B7C75B765AFEBD], state=RUNNABLE,
> group=TGRP-TestIndexTooManyDocs]
> 16:06:08 [junit4] 2> at
> java.lang.Thread.getStackTrace(Thread.java:1559)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:696)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$4.run(ThreadLeakControl.java:693)
> 16:06:08 [junit4] 2> at
> java.security.AccessController.doPrivileged(Native Method)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.getStackTrace(ThreadLeakControl.java:693)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.getThreadsWithTraces(ThreadLeakControl.java:709)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.formatThreadStacksFull(ThreadLeakControl.java:689)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.access$1000(ThreadLeakControl.java:65)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$2.evaluate(ThreadLeakControl.java:415)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:708)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.access$200(RandomizedRunner.java:138)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:629)
> 16:06:08 [junit4] 2> 2) Thread[id=23, name=Thread-2, state=BLOCKED,
> group=TGRP-TestIndexTooManyDocs]
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.IndexWriter.nrtIsCurrent(IndexWriter.java:4942)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:290)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:269)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:259)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:140)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.TestIndexTooManyDocs.lambda$testIndexTooManyDocs$1(TestIndexTooManyDocs.java:86)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.TestIndexTooManyDocs$$Lambda$8/1612361439.run(Unknown
> Source)
> 16:06:08 [junit4] 2> at java.lang.Thread.run(Thread.java:748)
> 16:06:08 [junit4] 2> 3) Thread[id=18, name=JUnit4-serializer-daemon,
> state=TIMED_WAITING, group=main]
> 16:06:08 [junit4] 2> at java.lang.Thread.sleep(Native Method)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.ant.tasks.junit4.events.Serializer$1.run(Serializer.java:50)
> 16:06:08 [junit4] 2> 4) Thread[id=22, name=Thread-1, state=RUNNABLE,
> group=TGRP-TestIndexTooManyDocs]
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.ReaderPool.anyDocValuesChanges(ReaderPool.java:354)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.IndexWriter.nrtIsCurrent(IndexWriter.java:4946)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:290)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:269)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:259)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:140)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.TestIndexTooManyDocs.lambda$testIndexTooManyDocs$1(TestIndexTooManyDocs.java:86)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.TestIndexTooManyDocs$$Lambda$8/1612361439.run(Unknown
> Source)
> 16:06:08 [junit4] 2> at java.lang.Thread.run(Thread.java:748)
> 16:06:08 [junit4] 2> 5) Thread[id=1, name=main, state=WAITING,
> group=main]
> 16:06:08 [junit4] 2> at java.lang.Object.wait(Native Method)
> 16:06:08 [junit4] 2> at java.lang.Thread.join(Thread.java:1252)
> 16:06:08 [junit4] 2> at java.lang.Thread.join(Thread.java:1326)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:639)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.run(RandomizedRunner.java:496)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:269)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:394)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:13)
> 16:06:08 [junit4] 2> 6) Thread[id=21,
> name=TEST-TestIndexTooManyDocs.testIndexTooManyDocs-seed#[43B7C75B765AFEBD],
> state=WAITING, group=TGRP-TestIndexTooManyDocs]
> 16:06:08 [junit4] 2> at sun.misc.Unsafe.park(Native Method)
> 16:06:08 [junit4] 2> at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> 16:06:08 [junit4] 2> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> 16:06:08 [junit4] 2> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997)
> 16:06:08 [junit4] 2> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
> 16:06:08 [junit4] 2> at
> java.util.concurrent.CountDownLatch.await(CountDownLatch.java:231)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.TestIndexTooManyDocs.testIndexTooManyDocs(TestIndexTooManyDocs.java:101)
> 16:06:08 [junit4] 2> at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 16:06:08 [junit4] 2> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 16:06:08 [junit4] 2> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 16:06:08 [junit4] 2> at
> java.lang.reflect.Method.invoke(Method.java:498)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
> 16:06:08 [junit4] 2> at java.lang.Thread.run(Thread.java:748)
> 16:06:08 [junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=TestIndexTooManyDocs -Dtests.method=testIndexTooManyDocs
> -Dtests.seed=43B7C75B765AFEBD -Dtests.slow=true -Dtests.badapples=true
> -Dtests.locale=ar-MA -Dtests.timezone=America/Jamaica -Dtests.asserts=true
> -Dtests.file.encoding=US-ASCII
> 16:06:08 [junit4] ERROR 7200s J3 |
> TestIndexTooManyDocs.testIndexTooManyDocs <<<
> 16:06:08 [junit4] > Throwable #1: java.lang.Exception: Test abandoned
> because suite timeout was reached.
> 16:06:08 [junit4] > at
> __randomizedtesting.SeedInfo.seed([43B7C75B765AFEBD]:0)
> 16:06:08 [junit4] 2> ??? 14, 2019 11:05:45 ?
> com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
> 16:06:08 [junit4] 2> WARNING: Will linger awaiting termination of 3
> leaked thread(s).
> 16:06:08 [junit4] 2> ??? 14, 2019 11:06:05 ?
> com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
> 16:06:08 [junit4] 2> SEVERE: 3 threads leaked from SUITE scope at
> org.apache.lucene.index.TestIndexTooManyDocs:
> 16:06:08 [junit4] 2> 1) Thread[id=23, name=Thread-2, state=BLOCKED,
> group=TGRP-TestIndexTooManyDocs]
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.IndexWriter.nrtIsCurrent(IndexWriter.java:4942)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:290)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:269)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:259)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:140)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.TestIndexTooManyDocs.lambda$testIndexTooManyDocs$1(TestIndexTooManyDocs.java:86)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.TestIndexTooManyDocs$$Lambda$8/1612361439.run(Unknown
> Source)
> 16:06:08 [junit4] 2> at java.lang.Thread.run(Thread.java:748)
> 16:06:08 [junit4] 2> 2) Thread[id=22, name=Thread-1, state=BLOCKED,
> group=TGRP-TestIndexTooManyDocs]
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.IndexWriter.nrtIsCurrent(IndexWriter.java:4942)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:290)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:269)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:259)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:140)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.TestIndexTooManyDocs.lambda$testIndexTooManyDocs$1(TestIndexTooManyDocs.java:86)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.TestIndexTooManyDocs$$Lambda$8/1612361439.run(Unknown
> Source)
> 16:06:08 [junit4] 2> at java.lang.Thread.run(Thread.java:748)
> 16:06:08 [junit4] 2> 3) Thread[id=21,
> name=TEST-TestIndexTooManyDocs.testIndexTooManyDocs-seed#[43B7C75B765AFEBD],
> state=WAITING, group=TGRP-TestIndexTooManyDocs]
> 16:06:08 [junit4] 2> at sun.misc.Unsafe.park(Native Method)
> 16:06:08 [junit4] 2> at
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
> 16:06:08 [junit4] 2> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
> 16:06:08 [junit4] 2> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997)
> 16:06:08 [junit4] 2> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
> 16:06:08 [junit4] 2> at
> java.util.concurrent.CountDownLatch.await(CountDownLatch.java:231)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.TestIndexTooManyDocs.testIndexTooManyDocs(TestIndexTooManyDocs.java:101)
> 16:06:08 [junit4] 2> at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 16:06:08 [junit4] 2> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 16:06:08 [junit4] 2> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 16:06:08 [junit4] 2> at
> java.lang.reflect.Method.invoke(Method.java:498)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> 16:06:08 [junit4] 2> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
> 16:06:08 [junit4] 2> at java.lang.Thread.run(Thread.java:748)
> 16:06:08 [junit4] 2> ??? 14, 2019 11:06:05 ?
> com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
> 16:06:08 [junit4] 2> INFO: Starting to interrupt leaked threads:
> 16:06:08 [junit4] 2> 1) Thread[id=23, name=Thread-2, state=BLOCKED,
> group=TGRP-TestIndexTooManyDocs]
> 16:06:08 [junit4] 2> 2) Thread[id=22, name=Thread-1, state=RUNNABLE,
> group=TGRP-TestIndexTooManyDocs]
> 16:06:08 [junit4] 2> 3) Thread[id=21,
> name=TEST-TestIndexTooManyDocs.testIndexTooManyDocs-seed#[43B7C75B765AFEBD],
> state=WAITING, group=TGRP-TestIndexTooManyDocs]
> 16:06:08 [junit4] 2> Jan 14, 2019 4:06:08 PM
> com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
> 16:06:08 [junit4] 2> SEVERE: There are still zombie threads that
> couldn't be terminated:
> 16:06:08 [junit4] 2> 1) Thread[id=23, name=Thread-2, state=RUNNABLE,
> group=TGRP-TestIndexTooManyDocs]
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.IndexWriter.nrtIsCurrent(IndexWriter.java:4942)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:290)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:269)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:259)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:140)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.TestIndexTooManyDocs.lambda$testIndexTooManyDocs$1(TestIndexTooManyDocs.java:86)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.TestIndexTooManyDocs$$Lambda$8/1612361439.run(Unknown
> Source)
> 16:06:08 [junit4] 2> at java.lang.Thread.run(Thread.java:748)
> 16:06:08 [junit4] 2> 2) Thread[id=22, name=Thread-1, state=BLOCKED,
> group=TGRP-TestIndexTooManyDocs]
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.IndexWriter.nrtIsCurrent(IndexWriter.java:4942)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.StandardDirectoryReader.doOpenFromWriter(StandardDirectoryReader.java:290)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:269)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.StandardDirectoryReader.doOpenIfChanged(StandardDirectoryReader.java:259)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.DirectoryReader.openIfChanged(DirectoryReader.java:140)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.TestIndexTooManyDocs.lambda$testIndexTooManyDocs$1(TestIndexTooManyDocs.java:86)
> 16:06:08 [junit4] 2> at
> org.apache.lucene.index.TestIndexTooManyDocs$$Lambda$8/1612361439.run(Unknown
> Source)
> 16:06:08 [junit4] 2> at java.lang.Thread.run(Thread.java:748)
> 16:06:08 [junit4] 2> NOTE: test params are: codec=Asserting(Lucene80):
> {id=PostingsFormat(name=Direct)}, docValues:{}, maxPointsInLeafNode=848,
> maxMBSortInHeap=7.086705758151636,
> sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@41eeb709),
> locale=ar-MA, timezone=America/Jamaica
> 16:06:08 [junit4] 2> NOTE: Linux 4.15.0-1026-gcp amd64/Oracle
> Corporation 1.8.0_191
> (64-bit)/cpus=16,threads=3,free=456394640,total=506462208
> 16:06:08 [junit4] 2> NOTE: All tests run in this JVM:
> [TestIndexTooManyDocs]
> 16:06:08 [junit4] 2> NOTE: reproduce with: ant test
> -Dtestcase=TestIndexTooManyDocs -Dtests.seed=43B7C75B765AFEBD
> -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=ar-MA
> -Dtests.timezone=America/Jamaica -Dtests.asserts=true
> -Dtests.file.encoding=US-ASCII
> 16:06:08 [junit4] ERROR 0.00s J3 | TestIndexTooManyDocs (suite) <<<
> 16:06:08 [junit4] > Throwable #1: java.lang.Exception: Suite timeout
> exceeded (>= 7200000 msec).
> 16:06:08 [junit4] > at
> __randomizedtesting.SeedInfo.seed([43B7C75B765AFEBD]:0)Throwable #2:
> com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an
> uncaught exception in thread: Thread[id=30, name=Thread-9, state=RUNNABLE,
> group=TGRP-TestIndexTooManyDocs]
> 16:06:08 [junit4] > Caused by: java.lang.AssertionError: seqNo=6 vs
> maxSeqNo=6
> 16:06:08 [junit4] > at
> __randomizedtesting.SeedInfo.seed([43B7C75B765AFEBD]:0)
> 16:06:08 [junit4] > at
> org.apache.lucene.index.DocumentsWriterDeleteQueue.getNextSequenceNumber(DocumentsWriterDeleteQueue.java:482)
> 16:06:08 [junit4] > at
> org.apache.lucene.index.DocumentsWriterDeleteQueue.add(DocumentsWriterDeleteQueue.java:168)
> 16:06:08 [junit4] > at
> org.apache.lucene.index.DocumentsWriterDeleteQueue.add(DocumentsWriterDeleteQueue.java:146)
> 16:06:08 [junit4] > at
> org.apache.lucene.index.DocumentsWriterPerThread.finishDocument(DocumentsWriterPerThread.java:362)
> 16:06:08 [junit4] > at
> org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:264)
> 16:06:08 [junit4] > at
> org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:494)
> 16:06:08 [junit4] > at
> org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1594)
> 16:06:08 [junit4] > at
> org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1586)
> 16:06:08 [junit4] > at
> org.apache.lucene.index.TestIndexTooManyDocs.lambda$testIndexTooManyDocs$0(TestIndexTooManyDocs.java:70)
> 16:06:08 [junit4] > at
> java.lang.Thread.run(Thread.java:748)Throwable #3:
> com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an
> uncaught exception in thread: Thread[id=29, name=Thread-8, state=RUNNABLE,
> group=TGRP-TestIndexTooManyDocs]
> 16:06:08 [junit4] > Caused by: java.lang.AssertionError: seqNo=7 vs
> maxSeqNo=6
> 16:06:08 [junit4] > at
> __randomizedtesting.SeedInfo.seed([43B7C75B765AFEBD]:0)
> 16:06:08 [junit4] > at
> org.apache.lucene.index.DocumentsWriterDeleteQueue.getNextSequenceNumber(DocumentsWriterDeleteQueue.java:482)
> 16:06:08 [junit4] > at
> org.apache.lucene.index.DocumentsWriterDeleteQueue.add(DocumentsWriterDeleteQueue.java:168)
> 16:06:08 [junit4] > at
> org.apache.lucene.index.DocumentsWriterDeleteQueue.add(DocumentsWriterDeleteQueue.java:146)
> 16:06:08 [junit4] > at
> org.apache.lucene.index.DocumentsWriterPerThread.finishDocument(DocumentsWriterPerThread.java:362)
> 16:06:08 [junit4] > at
> org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:264)
> 16:06:08 [junit4] > at
> org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:494)
> 16:06:08 [junit4] > at
> org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1594)
> 16:06:08 [junit4] > at
> org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1586)
> 16:06:08 [junit4] > at
> org.apache.lucene.index.TestIndexTooManyDocs.lambda$testIndexTooManyDocs$0(TestIndexTooManyDocs.java:70)
> {noformat}
> The reason seems to be that we don't prevent new threadstates from being
> created while we move over to a new delete queue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
