It was a thread scheduling issue. The test opened an IndexWriter, launched indexing threads, then opened the first NRT reader.
The failure would happen (as best I can figure out...) if a merge completed before the NRT reader was opened. Mike McCandless http://blog.mikemccandless.com On Wed, Jun 13, 2012 at 11:17 AM, Dawid Weiss <[email protected]> wrote: > Was it a race condition of some sort? Why do you think it wasn't > reproducible from the same seed? > > Dawid > > On Wed, Jun 13, 2012 at 5:04 PM, Michael McCandless > <[email protected]> wrote: >> I could never repro this but I think I found the cause (just committed >> the fix)... >> >> Mike McCandless >> >> http://blog.mikemccandless.com >> >> On Wed, Jun 13, 2012 at 6:45 AM, <[email protected]> wrote: >>> Build: >>> http://jenkins.sd-datasolutions.de/job/Lucene-Solr-4.x-Linux-Java6-64/101/ >>> >>> 1 tests failed. >>> REGRESSION: org.apache.lucene.index.TestNRTThreads.testNRTThreads >>> >>> Error Message: >>> Thread threw an uncaught exception, thread: Thread[Thread-923,5,] >>> >>> Stack Trace: >>> java.lang.RuntimeException: Thread threw an uncaught exception, thread: >>> Thread[Thread-923,5,] >>> at >>> com.carrotsearch.randomizedtesting.RunnerThreadGroup.processUncaught(RunnerThreadGroup.java:96) >>> at >>> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:857) >>> at >>> com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(RandomizedRunner.java:132) >>> at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(RandomizedRunner.java:669) >>> at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:695) >>> at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:734) >>> at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:745) >>> at >>> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) >>> at >>> org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68) >>> at >>> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38) >>> at >>> org.apache.lucene.util.TestRuleIcuHack$1.evaluate(TestRuleIcuHack.java:51) >>> at >>> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) >>> at >>> org.apache.lucene.util.TestRuleNoInstanceHooksOverrides$1.evaluate(TestRuleNoInstanceHooksOverrides.java:53) >>> at >>> org.apache.lucene.util.TestRuleNoStaticHooksShadowing$1.evaluate(TestRuleNoStaticHooksShadowing.java:52) >>> at >>> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:36) >>> at >>> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) >>> at >>> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:56) >>> at >>> com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:605) >>> at >>> com.carrotsearch.randomizedtesting.RandomizedRunner.access$400(RandomizedRunner.java:132) >>> at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:551) >>> Caused by: java.lang.RuntimeException: java.lang.AssertionError: sub reader >>> _5(4.0):c6 wasn't warmed: {SegmentCoreReader(owner=_j(4.0):C17)=true} >>> at __randomizedtesting.SeedInfo.seed([E1A5D8EF6222FB8E]:0) >>> at >>> org.apache.lucene.index.ThreadedIndexingAndSearchingTestCase$2.run(ThreadedIndexingAndSearchingTestCase.java:395) >>> Caused by: java.lang.AssertionError: sub reader _5(4.0):c6 wasn't warmed: >>> {SegmentCoreReader(owner=_j(4.0):C17)=true} >>> at org.junit.Assert.fail(Assert.java:93) >>> at org.junit.Assert.assertTrue(Assert.java:43) >>> at >>> org.apache.lucene.index.ThreadedIndexingAndSearchingTestCase$2.run(ThreadedIndexingAndSearchingTestCase.java:345) >>> >>> >>> >>> >>> Build Log: >>> [...truncated 1297 lines...] >>> [junit4] > at >>> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) >>> [junit4] > at >>> org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68) >>> [junit4] > at >>> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38) >>> [junit4] > at >>> org.apache.lucene.util.TestRuleIcuHack$1.evaluate(TestRuleIcuHack.java:51) >>> [junit4] > at >>> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) >>> [junit4] > at >>> org.apache.lucene.util.TestRuleNoInstanceHooksOverrides$1.evaluate(TestRuleNoInstanceHooksOverrides.java:53) >>> [junit4] > at >>> org.apache.lucene.util.TestRuleNoStaticHooksShadowing$1.evaluate(TestRuleNoStaticHooksShadowing.java:52) >>> [junit4] > at >>> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:36) >>> [junit4] > at >>> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) >>> [junit4] > at >>> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:56) >>> [junit4] > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:605) >>> [junit4] > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner.access$400(RandomizedRunner.java:132) >>> [junit4] > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:551) >>> [junit4] > Caused by: java.lang.RuntimeException: >>> java.lang.AssertionError: sub reader _5(4.0):c6 wasn't warmed: >>> {SegmentCoreReader(owner=_j(4.0):C17)=true} >>> [junit4] > at >>> __randomizedtesting.SeedInfo.seed([E1A5D8EF6222FB8E]:0) >>> [junit4] > at >>> org.apache.lucene.index.ThreadedIndexingAndSearchingTestCase$2.run(ThreadedIndexingAndSearchingTestCase.java:395) >>> [junit4] > Caused by: java.lang.AssertionError: sub reader _5(4.0):c6 >>> wasn't warmed: {SegmentCoreReader(owner=_j(4.0):C17)=true} >>> [junit4] > at org.junit.Assert.fail(Assert.java:93) >>> [junit4] > at org.junit.Assert.assertTrue(Assert.java:43) >>> [junit4] > at >>> org.apache.lucene.index.ThreadedIndexingAndSearchingTestCase$2.run(ThreadedIndexingAndSearchingTestCase.java:345) >>> [junit4] > Throwable #3: java.lang.RuntimeException: Thread threw an >>> uncaught exception, thread: Thread[Thread-924,5,] >>> [junit4] > at >>> com.carrotsearch.randomizedtesting.RunnerThreadGroup.processUncaught(RunnerThreadGroup.java:96) >>> [junit4] > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:857) >>> [junit4] > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(RandomizedRunner.java:132) >>> [junit4] > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(RandomizedRunner.java:669) >>> [junit4] > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:695) >>> [junit4] > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:734) >>> [junit4] > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:745) >>> [junit4] > at >>> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) >>> [junit4] > at >>> org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68) >>> [junit4] > at >>> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38) >>> [junit4] > at >>> org.apache.lucene.util.TestRuleIcuHack$1.evaluate(TestRuleIcuHack.java:51) >>> [junit4] > at >>> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) >>> [junit4] > at >>> org.apache.lucene.util.TestRuleNoInstanceHooksOverrides$1.evaluate(TestRuleNoInstanceHooksOverrides.java:53) >>> [junit4] > at >>> org.apache.lucene.util.TestRuleNoStaticHooksShadowing$1.evaluate(TestRuleNoStaticHooksShadowing.java:52) >>> [junit4] > at >>> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:36) >>> [junit4] > at >>> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) >>> [junit4] > at >>> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:56) >>> [junit4] > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:605) >>> [junit4] > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner.access$400(RandomizedRunner.java:132) >>> [junit4] > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:551) >>> [junit4] > Caused by: java.lang.RuntimeException: >>> java.lang.AssertionError: sub reader _5(4.0):c6 wasn't warmed: >>> {SegmentCoreReader(owner=_j(4.0):C17)=true} >>> [junit4] > at >>> __randomizedtesting.SeedInfo.seed([E1A5D8EF6222FB8E]:0) >>> [junit4] > at >>> org.apache.lucene.index.ThreadedIndexingAndSearchingTestCase$2.run(ThreadedIndexingAndSearchingTestCase.java:395) >>> [junit4] > Caused by: java.lang.AssertionError: sub reader _5(4.0):c6 >>> wasn't warmed: {SegmentCoreReader(owner=_j(4.0):C17)=true} >>> [junit4] > at org.junit.Assert.fail(Assert.java:93) >>> [junit4] > at org.junit.Assert.assertTrue(Assert.java:43) >>> [junit4] > at >>> org.apache.lucene.index.ThreadedIndexingAndSearchingTestCase$2.run(ThreadedIndexingAndSearchingTestCase.java:345) >>> [junit4] > Throwable #4: java.lang.RuntimeException: Thread threw an >>> uncaught exception, thread: Thread[Thread-925,5,] >>> [junit4] > at >>> com.carrotsearch.randomizedtesting.RunnerThreadGroup.processUncaught(RunnerThreadGroup.java:96) >>> [junit4] > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:857) >>> [junit4] > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(RandomizedRunner.java:132) >>> [junit4] > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(RandomizedRunner.java:669) >>> [junit4] > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:695) >>> [junit4] > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:734) >>> [junit4] > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:745) >>> [junit4] > at >>> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) >>> [junit4] > at >>> org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRuleReportUncaughtExceptions.java:68) >>> [junit4] > at >>> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38) >>> [junit4] > at >>> org.apache.lucene.util.TestRuleIcuHack$1.evaluate(TestRuleIcuHack.java:51) >>> [junit4] > at >>> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) >>> [junit4] > at >>> org.apache.lucene.util.TestRuleNoInstanceHooksOverrides$1.evaluate(TestRuleNoInstanceHooksOverrides.java:53) >>> [junit4] > at >>> org.apache.lucene.util.TestRuleNoStaticHooksShadowing$1.evaluate(TestRuleNoStaticHooksShadowing.java:52) >>> [junit4] > at >>> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:36) >>> [junit4] > at >>> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) >>> [junit4] > at >>> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:56) >>> [junit4] > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner.runSuite(RandomizedRunner.java:605) >>> [junit4] > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner.access$400(RandomizedRunner.java:132) >>> [junit4] > at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$2.run(RandomizedRunner.java:551) >>> [junit4] > Caused by: java.lang.RuntimeException: >>> java.lang.AssertionError: sub reader _5(4.0):c6 wasn't warmed: >>> {SegmentCoreReader(owner=_j(4.0):C17)=true} >>> [junit4] > at >>> __randomizedtesting.SeedInfo.seed([E1A5D8EF6222FB8E]:0) >>> [junit4] > at >>> org.apache.lucene.index.ThreadedIndexingAndSearchingTestCase$2.run(ThreadedIndexingAndSearchingTestCase.java:395) >>> [junit4] > Caused by: java.lang.AssertionError: sub reader _5(4.0):c6 >>> wasn't warmed: {SegmentCoreReader(owner=_j(4.0):C17)=true} >>> [junit4] > at org.junit.Assert.fail(Assert.java:93) >>> [junit4] > at org.junit.Assert.assertTrue(Assert.java:43) >>> [junit4] > at >>> org.apache.lucene.index.ThreadedIndexingAndSearchingTestCase$2.run(ThreadedIndexingAndSearchingTestCase.java:345) >>> [junit4] > >>> [junit4] 1> Thread-924: hit exc >>> [junit4] 1> java.lang.AssertionError: sub reader _5(4.0):c6 wasn't >>> warmed: {SegmentCoreReader(owner=_j(4.0):C17)=true} >>> [junit4] 1> at org.junit.Assert.fail(Assert.java:93) >>> [junit4] 1> at org.junit.Assert.assertTrue(Assert.java:43) >>> [junit4] 1> at >>> org.apache.lucene.index.ThreadedIndexingAndSearchingTestCase$2.run(ThreadedIndexingAndSearchingTestCase.java:345) >>> [junit4] 1> Thread-923: hit exc >>> [junit4] 1> java.lang.AssertionError: sub reader _5(4.0):c6 wasn't >>> warmed: {SegmentCoreReader(owner=_j(4.0):C17)=true} >>> [junit4] 1> at org.junit.Assert.fail(Assert.java:93) >>> [junit4] 1> at org.junit.Assert.assertTrue(Assert.java:43) >>> [junit4] 1> at >>> org.apache.lucene.index.ThreadedIndexingAndSearchingTestCase$2.run(ThreadedIndexingAndSearchingTestCase.java:345) >>> [junit4] 1> Thread-925: hit exc >>> [junit4] 1> java.lang.AssertionError: sub reader _5(4.0):c6 wasn't >>> warmed: {SegmentCoreReader(owner=_j(4.0):C17)=true} >>> [junit4] 1> at org.junit.Assert.fail(Assert.java:93) >>> [junit4] 1> at org.junit.Assert.assertTrue(Assert.java:43) >>> [junit4] 1> at >>> org.apache.lucene.index.ThreadedIndexingAndSearchingTestCase$2.run(ThreadedIndexingAndSearchingTestCase.java:345) >>> [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestNRTThreads >>> -Dtests.method=testNRTThreads -Dtests.seed=E1A5D8EF6222FB8E >>> -Dtests.multiplier=3 -Dtests.locale=sr -Dtests.timezone=Europe/Berlin >>> -Dargs="-Dfile.encoding=UTF-8" >>> [junit4] 1> >>> [junit4] 2> >>> [junit4] > (@AfterClass output) >>> [junit4] 2> NOTE: test params are: codec=Lucene40: >>> {extra29=PostingsFormat(name=MockRandom), body=Lucene40(minBlockSize=82 >>> maxBlockSize=202), extra26=Pulsing40(freqCutoff=11 minBlockSize=82 >>> maxBlockSize=202), extra25=Pulsing40(freqCutoff=14 minBlockSize=82 >>> maxBlockSize=202), extra23=MockVariableIntBlock(baseBlockSize=86), >>> extra22=Lucene40(minBlockSize=82 maxBlockSize=202), >>> packID=PostingsFormat(name=NestedPulsing), date=Pulsing40(freqCutoff=11 >>> minBlockSize=82 maxBlockSize=202), docid=PostingsFormat(name=MockRandom), >>> title=Pulsing40(freqCutoff=14 minBlockSize=82 maxBlockSize=202), >>> extra8=Lucene40(minBlockSize=82 maxBlockSize=202), >>> extra9=MockVariableIntBlock(baseBlockSize=86), >>> extra11=PostingsFormat(name=NestedPulsing), >>> extra13=PostingsFormat(name=MockSep), >>> extra39=MockFixedIntBlock(blockSize=1027), >>> extra16=MockFixedIntBlock(blockSize=1027), >>> extra34=PostingsFormat(name=NestedPulsing), >>> extra15=MockVariableIntBlock(baseBlockSize=86), >>> extra35=PostingsFormat(name=MockRandom), extra17=Pulsing40(freqCutoff=14 >>> minBlockSize=82 maxBlockSize=202), >>> extra0=MockVariableIntBlock(baseBlockSize=86), >>> extra19=PostingsFormat(name=Lucene40WithOrds), >>> extra1=MockFixedIntBlock(blockSize=1027), >>> extra7=PostingsFormat(name=MockSep), >>> titleTokenized=PostingsFormat(name=MockRandom), >>> extra30=MockFixedIntBlock(blockSize=1027), extra32=Pulsing40(freqCutoff=11 >>> minBlockSize=82 maxBlockSize=202), >>> extra10=PostingsFormat(name=Lucene40WithOrds)}, sim=DefaultSimilarity, >>> locale=sr, timezone=Europe/Berlin >>> [junit4] 2> NOTE: Linux 2.6.32-41-server amd64/Sun Microsystems Inc. >>> 1.6.0_32 (64-bit)/cpus=8,threads=1,free=217941008,total=393347072 >>> [junit4] 2> NOTE: All tests run in this JVM: [TestRandomStoredFields, >>> TestDocumentsWriterDeleteQueue, TestTimeLimitingCollector, >>> TestOmitPositions, Test2BPostings, TestPrefixCodedTerms, TestAtomicUpdate, >>> TestMatchAllDocsQuery, TestExplanations, TestIndexWriterWithThreads, >>> TestNamedSPILoader, TestQueryWrapperFilter, ThrowInUncaught, >>> TestSpecialOperations, Nested1, Nested2, Nested3, TestWindowsMMap, >>> TestRecyclingByteBlockAllocator, TestDocumentWriter, TestCompoundFile, >>> TestFieldCacheRewriteMethod, TestSegmentReader, TestParallelAtomicReader, >>> TestByteArrayDataInput, TestIndexWriterLockRelease, TestPrefixRandom, >>> TestReaderClosed, TestFileSwitchDirectory, TestNoMergeScheduler, >>> TestMultiMMap, TestPositionIncrement, TestIdentityHashSet, TestSurrogates, >>> TestFieldValueFilter, TestTopDocsMerge, TestParallelReaderEmptyIndex, >>> TestOpenBitSet, TestWildcard, TestDeterminism, TestTermVectorsReader, >>> TestImpersonation, TestCustomNorms, TestCustomSearcherSort, >>> TestMergeSchedulerExternal, TestTermsEnum2, TestDocTermOrds, >>> Test10KPulsings, TestPackedInts, TestUnicodeUtil, TestSimilarity, >>> TestPostingsOffsets, TestLazyProxSkipping, TestMultiThreadTermVectors, >>> TestIndexWriterNRTIsCurrent, TestIndexWriterConfig, TestMockAnalyzer, >>> TestPrefixFilter, TestIndexWriterUnicode, TestRollingBuffer, >>> TestDocValuesScoring, TestFieldsReader, TestStressIndexing, >>> TestSpansAdvanced2, TestSimpleAttributeImpl, TestIndexableField, >>> TestSimpleExplanationsOfNonMatches, TestDateSort, TestTermInfosReaderIndex, >>> TestSimilarityBase, TestConsistentFieldNumbers, TestNumericUtils, >>> TestGraphTokenizers, TestBitVector, Nested, Nested, >>> TestNRTReaderWithThreads, TestByteBlockPool, TestIndexWriterCommit, >>> TestSearcherManager, TestConcurrentMergeScheduler, TestByteSlices, >>> TestCachingCollector, TestMixedCodecs, TestCopyBytes, Before3, Before3, >>> TestLockFactory, TestLongPostings, TestExternalCodecs, >>> TestLookaheadTokenFilter, TestDirectoryReader, TestPagedBytes, TestSetOnce, >>> TestSizeBoundedForceMerge, TestDateFilter, TestRegexpQuery, TestLock, >>> TestForTooMuchCloning, TestBasics, TestElevationComparator, >>> TestRegexpRandom, TestDateTools, TestNRTThreads] >>> [junit4] 2> >>> [junit4] Completed on J0 in 1.74s, 1 test, 1 error <<< FAILURES! >>> [...truncated 277 lines...] >>> >>> [...truncated 1675 lines...] >>> >>> [...truncated 1675 lines...] >>> >>> [...truncated 1675 lines...] >>> >>> [...truncated 1675 lines...] >>> >>> [...truncated 1675 lines...] >>> >>> >>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: [email protected] >>> For additional commands, e-mail: [email protected] >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
