Thanks Mike! I was looking at the info stream to see what happens, and merging seems to work fine, it just seems to be unable to keep up with segment creations. If someone wants to have a look at the info stream, I uploaded it at http://people.apache.org/~jpountz/org.apache.lucene.index.TestIndexWriterReader.testTooManySegments.log
On Thu, Nov 21, 2013 at 12:17 PM, Michael McCandless <luc...@mikemccandless.com> wrote: > I think the problem is that it got a LogByteSizeMergePolicy w/ > mergeFactor=50; not sure why it would fail under JRockit but not other > JVMs; maybe a thread scheduling difference? > > I committed a fix to not use newIndexWriterConfig, so we always get a > sane (the default) merge policy. > > Mike McCandless > > http://blog.mikemccandless.com > > > On Thu, Nov 21, 2013 at 5:49 AM, Adrien Grand <jpou...@gmail.com> wrote: >> This one is easily reproducible with a JRockit JDK but unreproducible >> with an Oracle JDK or OpenJDK. >> >> On Thu, Nov 21, 2013 at 7:03 AM, Policeman Jenkins Server >> <jenk...@thetaphi.de> wrote: >>> Build: http://jenkins.thetaphi.de/job/Lucene-Solr-4.x-Linux/8305/ >>> Java: 64bit/jrockit-jdk1.6.0_45-R28.2.7-4.1.0 -XnoOpt >>> >>> 1 tests failed. >>> REGRESSION: >>> org.apache.lucene.index.TestIndexWriterReader.testTooManySegments >>> >>> Error Message: >>> >>> >>> Stack Trace: >>> java.lang.AssertionError >>> at >>> __randomizedtesting.SeedInfo.seed([2694FC274A8D4839:6007370D936AF0EA]:0) >>> at org.junit.Assert.fail(Assert.java:92) >>> at org.junit.Assert.assertTrue(Assert.java:43) >>> at org.junit.Assert.assertTrue(Assert.java:54) >>> at >>> org.apache.lucene.index.TestIndexWriterReader.testTooManySegments(TestIndexWriterReader.java:1160) >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>> at >>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) >>> at >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >>> at java.lang.reflect.Method.invoke(Method.java:597) >>> at >>> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559) >>> at >>> com.carrotsearch.randomizedtesting.RandomizedRunner.access$600(RandomizedRunner.java:79) >>> at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:738) >>> at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:774) >>> at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:787) >>> at >>> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) >>> at >>> org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCacheSanity.java:51) >>> at >>> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) >>> at >>> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) >>> at >>> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) >>> at >>> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70) >>> at >>> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) >>> at >>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) >>> at >>> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358) >>> at >>> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:782) >>> at >>> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:442) >>> at >>> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:746) >>> at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:648) >>> at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:683) >>> at >>> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:693) >>> at >>> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) >>> at >>> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) >>> at >>> com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55) >>> at >>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) >>> at >>> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) >>> at >>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) >>> at >>> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:44) >>> at >>> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) >>> at >>> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:70) >>> at >>> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:56) >>> at >>> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) >>> at >>> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:358) >>> at java.lang.Thread.run(Thread.java:662) >>> >>> >>> >>> >>> Build Log: >>> [...truncated 995 lines...] >>> [junit4] Suite: org.apache.lucene.index.TestIndexWriterReader >>> [junit4] 2> NOTE: reproduce with: ant test >>> -Dtestcase=TestIndexWriterReader -Dtests.method=testTooManySegments >>> -Dtests.seed=2694FC274A8D4839 -Dtests.multiplier=3 -Dtests.slow=true >>> -Dtests.locale=pl -Dtests.timezone=Asia/Gaza -Dtests.file.encoding=UTF-8 >>> [junit4] FAILURE 16.1s J1 | TestIndexWriterReader.testTooManySegments <<< >>> [junit4] > Throwable #1: java.lang.AssertionError >>> [junit4] > at >>> __randomizedtesting.SeedInfo.seed([2694FC274A8D4839:6007370D936AF0EA]:0) >>> [junit4] > at >>> org.apache.lucene.index.TestIndexWriterReader.testTooManySegments(TestIndexWriterReader.java:1160) >>> [junit4] > at java.lang.Thread.run(Thread.java:662) >>> [junit4] 2> NOTE: test params are: codec=Lucene46: >>> {f=Pulsing41(freqCutoff=18 minBlockSize=88 maxBlockSize=246), >>> field5=Lucene41(blocksize=128), id=Lucene41(blocksize=128), >>> field=Pulsing41(freqCutoff=18 minBlockSize=88 maxBlockSize=246), >>> field4=Pulsing41(freqCutoff=18 minBlockSize=88 maxBlockSize=246), >>> field3=PostingsFormat(name=Asserting), field2=Pulsing41(freqCutoff=18 >>> minBlockSize=88 maxBlockSize=246), foo=Pulsing41(freqCutoff=18 >>> minBlockSize=88 maxBlockSize=246), field1=Lucene41(blocksize=128), >>> indexname=PostingsFormat(name=Asserting)}, docValues:{}, >>> sim=DefaultSimilarity, locale=pl, timezone=Asia/Gaza >>> [junit4] 2> NOTE: Linux 3.8.0-32-generic amd64/Oracle Corporation >>> 1.6.0_45 (64-bit)/cpus=8,threads=1,free=183934704,total=536870912 >>> [junit4] 2> NOTE: All tests run in this JVM: [TestCopyBytes, >>> TestBasics, TestBinaryTerms, TestMockDirectoryWrapper, >>> TestCachingTokenFilter, TestIndexWriterUnicode, TestCodecs, TestAssertions, >>> TestToken, Nested, TestNot, TestMultiDocValues, TestSegmentMerger, >>> TestBufferedIndexInput, Before3, Before3, TestSloppyPhraseQuery2, >>> FuzzyTermOnShortTermsTest, TestDuelingCodecs, TestPayloadNearQuery, >>> TestIndexSearcher, TestAttributeSource, TestPrefixQuery, TestIntsRef, >>> TestQueryWrapperFilter, TestRegexpRandom2, TestOpenBitSet, TestMinimize, >>> TestDateFilter, Test2BDocs, TestLongPostings, MultiCollectorTest, >>> TestPerFieldDocValuesFormat, TestPerFieldPostingsFormat2, >>> TestDocValuesIndexing, TestIndexWriterNRTIsCurrent, >>> TestCodecHoldsOpenFiles, TestIndexWriterWithThreads, >>> TestBackwardsCompatibility3x, TestIdentityHashSet, >>> TestSimpleSearchEquivalence, TestElevationComparator, TestHugeRamFile, >>> TestIndexWriterDelete, TestDocValuesFormat, TestBooleanQuery, >>> TestSnapshotDeletionPolicy, TestIndexWriterOnJRECrash, >>> TestBackwardsCompatibility, TestDoc, TestIndexInput, TestWAH8DocIdSet, >>> TestBooleanQueryVisitSubscorers, TestGraphTokenizers, NestedSetupChain, >>> NestedTeardownChain, TestIntBlockPool, TestDocCount, >>> TestControlledRealTimeReopenThread, TestCachingWrapperFilter, >>> TestTermRangeFilter, TestNamedSPILoader, TestConjunctions, >>> Test4GBStoredFields, TestDocument, TestThreadedForceMerge, TestFieldCache, >>> TestRateLimiter, TestLucene3xTermVectorsFormat, TestSizeBoundedForceMerge, >>> TestPositionIncrement, TestRecyclingIntBlockAllocator, >>> TestFieldMaskingSpanQuery, TestCharsRef, TestSloppyMath, >>> TestTermVectorsReader, TestSimilarityBase, TestRAMDirectory, Nested1, >>> TestSortDocValues, TestTransactions, TestTotalHitCountCollector, >>> TestSimpleExplanationsOfNonMatches, TestFileSwitchDirectory, TestFSTs, >>> TestTerm, TestNumericTokenStream, TestMultiMMap, Nested, >>> TestSpanSearchEquivalence, TestDocsAndPositions, TestIntroSorter, >>> TestSpanExplanations, TestOmitNorms, TestPayloadExplanations, >>> TestEarlyTermination, TestIndexWriterReader] >>> [junit4] Completed on J1 in 38.24s, 23 tests, 1 failure <<< FAILURES! >>> >>> [...truncated 676 lines...] >>> BUILD FAILED >>> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:426: The >>> following error occurred while executing this line: >>> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:406: The >>> following error occurred while executing this line: >>> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/build.xml:39: The >>> following error occurred while executing this line: >>> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/extra-targets.xml:37: The >>> following error occurred while executing this line: >>> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/build.xml:49: The >>> following error occurred while executing this line: >>> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:1276: >>> The following error occurred while executing this line: >>> /mnt/ssd/jenkins/workspace/Lucene-Solr-4.x-Linux/lucene/common-build.xml:912: >>> There were test failures: 379 suites, 2758 tests, 1 failure, 105 ignored >>> (92 assumptions) >>> >>> Total time: 21 minutes 14 seconds >>> Build step 'Invoke Ant' marked build as failure >>> Description set: Java: 64bit/jrockit-jdk1.6.0_45-R28.2.7-4.1.0 -XnoOpt >>> Archiving artifacts >>> Recording test results >>> Email was triggered for: Failure >>> Sending email for trigger: Failure >>> >>> >>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >>> For additional commands, e-mail: dev-h...@lucene.apache.org >> >> >> >> -- >> Adrien >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >> For additional commands, e-mail: dev-h...@lucene.apache.org >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > -- Adrien --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
