Mike: Because heap dumps were disabled 2 weeks ago on Jenkins. I cannot reenable them, as it fills my disk, because currently some Lucene/Solr test (not sure which) *always* produces a heap dump. So it looks like some test it forcefully OOM (maybe for test reasons), although all tests fail.
I can try with JRockit JVM, it has an -XX option to crash and thread-dump JVM on OOM. ----- Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -----Original Message----- > From: Michael McCandless [mailto:luc...@mikemccandless.com] > Sent: Tuesday, July 31, 2012 12:48 PM > To: dev@lucene.apache.org > Subject: Re: [JENKINS] Lucene-Solr-tests-only-4.x-java7 - Build # 228 - Still > Failing > > Thanks Robert, I'll dig. > > Separately: how come we didn't get a heap dump from the failure...? > > Mike McCandless > > http://blog.mikemccandless.com > > On Mon, Jul 30, 2012 at 10:15 PM, Robert Muir <rcm...@gmail.com> wrote: > > This seed OOMs here too. > > > > I added these prints: > > System.out.println(RamUsageEstimator.sizeOf(fields)); > > System.out.println(RamUsageEstimator.sizeOf(fieldsLive)); > > > > I see: > > [junit4:junit4] 1> 367701000 > > [junit4:junit4] 1> 222881224 > > > > (I am sure they are likely sharing some ram). But this is a massive > > amount of ram for this test. > > > > I will temporarily disable the multiplier, until the test can be made > > more ram efficient. it grows linearly with the multiplier, i think > > because of all the lists within lists within maps etc :) > > > > [junit4:junit4] 1> 136660616 > > [junit4:junit4] 1> 89595032 > > > > > > On Mon, Jul 30, 2012 at 8:30 PM, Apache Jenkins Server > > <jenk...@builds.apache.org> wrote: > >> Build: > >> https://builds.apache.org/job/Lucene-Solr-tests-only-4.x-java7/228/ > >> > >> 1 tests failed. > >> REGRESSION: org.apache.lucene.index.TestPostingsFormat.test > >> > >> Error Message: > >> Java heap space > >> > >> Stack Trace: > >> java.lang.OutOfMemoryError: Java heap space > >> at > __randomizedtesting.SeedInfo.seed([E8AAFF6FA7F41694:60FEC0B509087B6C]: > 0) > >> at org.apache.lucene.util.BytesRef.<init>(BytesRef.java:75) > >> at > org.apache.lucene.util.fst.ByteSequenceOutputs.read(ByteSequenceOutputs.jav > a:124) > >> at > org.apache.lucene.util.fst.ByteSequenceOutputs.read(ByteSequenceOutputs.jav > a:33) > >> at org.apache.lucene.util.fst.FST.readNextRealArc(FST.java:960) > >> at org.apache.lucene.util.fst.NodeHash.hash(NodeHash.java:106) > >> at org.apache.lucene.util.fst.NodeHash.add(NodeHash.java:124) > >> at org.apache.lucene.util.fst.Builder.compileNode(Builder.java:195) > >> at org.apache.lucene.util.fst.Builder.freezeTail(Builder.java:287) > >> at org.apache.lucene.util.fst.Builder.add(Builder.java:392) > >> at > org.apache.lucene.codecs.memory.MemoryPostingsFormat$TermsWriter.finish > Term(MemoryPostingsFormat.java:257) > >> at > org.apache.lucene.index.TestPostingsFormat.buildIndex(TestPostingsFormat.jav > a:441) > >> at > org.apache.lucene.index.TestPostingsFormat.test(TestPostingsFormat.java:896) > >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > >> at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java: > 57) > >> at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorI > mpl.java:43) > >> at java.lang.reflect.Method.invoke(Method.java:601) > >> at > com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRu > nner.java:1995) > >> at > com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(Random > izedRunner.java:132) > >> at > com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(Randomiz > edRunner.java:818) > >> at > com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(Randomiz > edRunner.java:877) > >> at > com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(Randomiz > edRunner.java:891) > >> at > org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSet > upTeardownChained.java:50) > >> at > org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCach > eSanity.java:32) > >> at > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfte > rRule.java:45) > >> at > com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.ev > aluate(SystemPropertiesInvariantRule.java:55) > >> at > org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRule > ReportUncaughtExceptions.java:68) > >> at > org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThrea > dAndTestName.java:48) > >> at > org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgn > oreAfterMaxFailures.java:70) > >> at > org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.ja > va:48) > >> at > com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(Rando > mizedRunner.java:825) > >> at > com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(Randomiz > edRunner.java:132) > >> at > >> > com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(Randomize > >> dRunner.java:671) > >> > >> > >> > >> > >> Build Log: > >> [...truncated 1197 lines...] > >> [junit4:junit4] Suite: org.apache.lucene.index.TestPostingsFormat > >> [junit4:junit4] ERROR 4.61s J0 | TestPostingsFormat.test > >> [junit4:junit4] > Throwable #1: java.lang.OutOfMemoryError: Java heap > space > >> [junit4:junit4] > at > __randomizedtesting.SeedInfo.seed([E8AAFF6FA7F41694:60FEC0B509087B6C]: > 0) > >> [junit4:junit4] > at > org.apache.lucene.util.BytesRef.<init>(BytesRef.java:75) > >> [junit4:junit4] > at > org.apache.lucene.util.fst.ByteSequenceOutputs.read(ByteSequenceOutputs.jav > a:124) > >> [junit4:junit4] > at > org.apache.lucene.util.fst.ByteSequenceOutputs.read(ByteSequenceOutputs.jav > a:33) > >> [junit4:junit4] > at > org.apache.lucene.util.fst.FST.readNextRealArc(FST.java:960) > >> [junit4:junit4] > at > org.apache.lucene.util.fst.NodeHash.hash(NodeHash.java:106) > >> [junit4:junit4] > at > org.apache.lucene.util.fst.NodeHash.add(NodeHash.java:124) > >> [junit4:junit4] > at > org.apache.lucene.util.fst.Builder.compileNode(Builder.java:195) > >> [junit4:junit4] > at > org.apache.lucene.util.fst.Builder.freezeTail(Builder.java:287) > >> [junit4:junit4] > at > org.apache.lucene.util.fst.Builder.add(Builder.java:392) > >> [junit4:junit4] > at > org.apache.lucene.codecs.memory.MemoryPostingsFormat$TermsWriter.finish > Term(MemoryPostingsFormat.java:257) > >> [junit4:junit4] > at > org.apache.lucene.index.TestPostingsFormat.buildIndex(TestPostingsFormat.jav > a:441) > >> [junit4:junit4] > at > org.apache.lucene.index.TestPostingsFormat.test(TestPostingsFormat.java:896) > >> [junit4:junit4] > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native > Method) > >> [junit4:junit4] > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java: > 57) > >> [junit4:junit4] > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorI > mpl.java:43) > >> [junit4:junit4] > at java.lang.reflect.Method.invoke(Method.java:601) > >> [junit4:junit4] > at > com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRu > nner.java:1995) > >> [junit4:junit4] > at > com.carrotsearch.randomizedtesting.RandomizedRunner.access$1100(Random > izedRunner.java:132) > >> [junit4:junit4] > at > com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(Randomiz > edRunner.java:818) > >> [junit4:junit4] > at > com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(Randomiz > edRunner.java:877) > >> [junit4:junit4] > at > com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(Randomiz > edRunner.java:891) > >> [junit4:junit4] > at > org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSet > upTeardownChained.java:50) > >> [junit4:junit4] > at > org.apache.lucene.util.TestRuleFieldCacheSanity$1.evaluate(TestRuleFieldCach > eSanity.java:32) > >> [junit4:junit4] > at > org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfte > rRule.java:45) > >> [junit4:junit4] > at > com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.ev > aluate(SystemPropertiesInvariantRule.java:55) > >> [junit4:junit4] > at > org.apache.lucene.util.TestRuleReportUncaughtExceptions$1.evaluate(TestRule > ReportUncaughtExceptions.java:68) > >> [junit4:junit4] > at > org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThrea > dAndTestName.java:48) > >> [junit4:junit4] > at > org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgn > oreAfterMaxFailures.java:70) > >> [junit4:junit4] > at > org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.ja > va:48) > >> [junit4:junit4] > at > com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(Rando > mizedRunner.java:825) > >> [junit4:junit4] > at > com.carrotsearch.randomizedtesting.RandomizedRunner.access$700(Randomiz > edRunner.java:132) > >> [junit4:junit4] > at > com.carrotsearch.randomizedtesting.RandomizedRunner$3$1.run(Randomized > Runner.java:671) > >> [junit4:junit4] > > >> [junit4:junit4] 2> NOTE: reproduce with: ant test - > Dtestcase=TestPostingsFormat -Dtests.method=test - > Dtests.seed=E8AAFF6FA7F41694 -Dtests.multiplier=3 -Dtests.slow=true - > Dtests.locale=ja -Dtests.timezone=America/Fort_Wayne - > Dtests.file.encoding=US-ASCII > >> [junit4:junit4] 2> > >> [junit4:junit4] > (@AfterClass output) > >> [junit4:junit4] 2> NOTE: test params are: codec=Lucene40: > {=PostingsFormat(name=MockSep), rl=Lucene40(minBlockSize=82 > maxBlockSize=257), xgeplm=PostingsFormat(name=Memory doPackFST= false), > kojsjftf=PostingsFormat(name=NestedPulsing)}, sim=DefaultSimilarity, > locale=ja, timezone=America/Fort_Wayne > >> [junit4:junit4] 2> NOTE: FreeBSD 9.0-RELEASE amd64/Oracle Corporation > 1.7.0_04 (64-bit)/cpus=16,threads=1,free=33223808,total=536870912 > >> [junit4:junit4] 2> NOTE: All tests run in this JVM: [TestSearchAfter, > TestDocValuesIndexing, TestWeakIdentityMap, > TestSimpleExplanationsOfNonMatches, TestFilteredQuery, > TestFieldCacheTermsFilter, Nested, Nested, Nested, Nested, Nested, Nested, > Nested, Nested, Nested, Nested, Nested, Nested, Nested, Nested, Nested, > Nested, Nested, Nested, Nested, Nested, Nested, TestSpanExplanations, > TestIndexWriterWithThreads, TestMaxTermFrequency, TestStressIndexing2, > TestIndexableBinaryStringTools, TestCheckIndex, TestCharsRef, > TestTermdocPerf, TestDateSort, TestSetOnce, TestMixedCodecs, > TestSloppyPhraseQuery2, TestMultiFields, TestSnapshotDeletionPolicy, > TestTimeLimitingCollector, TestNoDeletionPolicy, TestLazyProxSkipping, > Test2BTerms, TestIndexWriterForceMerge, TestRollingUpdates, > TestTermVectorsReader, TestAssertions, TestSpecialOperations, > TestFilterAtomicReader, TestElevationComparator, TestMultiMMap, > TestByteSlices, TestPayloadTermQuery, TestDocumentWriter, > TestSizeBoundedForceMerge, TestFuzzyQuery, TestDeletionPolicy, > TestShardSearching, TestNumericUtils, TestSpanSearchEquivalence, > TestDisjunctionMaxQuery, TestByteArrayDataInput, TestMultiPhraseQuery, > TestTopDocsCollector, TestFixedBitSet, TestSearch, TestTransactionRollback, > TestTermVectors, TestIndexWriter, TestDirectoryReaderReopen, TestOmitTf, > TestCopyBytes, TestRegexpQuery, TestSmallFloat, > TestIndexWriterNRTIsCurrent, TestMockCharFilter, TestPrefixCodedTerms, > TestLevenshteinAutomata, Nested, Nested, TestMultiTermConstantScore, > TestIndexWriterReader, TestWindowsMMap, TestDemo, > TestParallelCompositeReader, TestScoreCachingWrappingScorer, > TestFileSwitchDirectory, TestArrayUtil, TestPagedBytes, > TestIndexWriterLockRelease, InBeforeClass, InAfterClass, InTestMethod, > NonStringProperties, TestBasicOperations, > TestSpanExplanationsOfNonMatches, TestSortedVIntList, TestScorerPerf, > TestPrefixInBooleanQuery, TestRegexpRandom, TestNorms, > TestIndexWriterExceptions, TestBasics, TestDuelingCodecs, > TestIndexWriterMergePolicy, TestNeverDelete, > TestFlushByRamOrCountsPolicy, TestTopDocsMerge, TestCollectionUtil, > TestCompoundFile, TestSimpleExplanations, TestCustomNorms, > TestStressIndexing, TestDeterminizeLexicon, TestDocumentsWriterStallControl, > TestRollingCharBuffer, TestRegexpRandom2, TestConsistentFieldNumbers, > TestDoubleBarrelLRUCache, TestCrashCausesCorruptIndex, TestDocValues, > TestUTF32ToUTF8, TestTransactions, TestThreadedForceMerge, > TestSegmentReader, TestPerFieldPostingsFormat, TestFieldCache, > TestPayloadNearQuery, TestPrefixRandom, TestDocsAndPositions, TestFlex, > TestIndexableField, TestSimilarity2, TestAllFilesHaveCodecHeader, > TestMultiLevelSkipList, TestExternalCodecs, Nested2, Nested3, Nested1, > TestTermRangeQuery, TestSpansAdvanced2, > TestSpanMultiTermQueryWrapper, TestComplexExplanationsOfNonMatches, > TestSumDocFreq, TestPerSegmentDeletes, TestParallelReaderEmptyIndex, > TestIndexFileDeleter, Before3, Before3, TestDocValuesScoring, TestDocIdSet, > TestFieldValueFilter, TestAppendingCodec, TestSpansAdvanced, > TestConstantScoreQuery, TestPositionIncrement, Nested, TestReaderClosed, > TestNot, TestAutomatonQueryUnicode, TestIndexWriterOnJRECrash, > TestBinaryDocument, TestSpanFirstQuery, TestPriorityQueue, TestDocBoost, > TestIntBlockCodec, TestIsCurrent, MultiCollectorTest, TestPrefixFilter, > TestVersion, TestNoMergeScheduler, TestCachingTokenFilter, TestIOUtils, > TestCharFilter, TestPostingsFormat] > >> [junit4:junit4] 2> > >> [junit4:junit4] Completed on J0 in 13.67s, 1 test, 1 error <<< FAILURES! > >> > >> [...truncated 5 lines...] > >> BUILD FAILED > >> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x- > java7/checkout/build.xml:29: The following error occurred while executing this > line: > >> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x- > java7/checkout/lucene/build.xml:50: The following error occurred while > executing this line: > >> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x- > java7/checkout/lucene/common-build.xml:1095: The following error occurred > while executing this line: > >> /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-tests-only-4.x-ja > >> va7/checkout/lucene/common-build.xml:790: There were test failures: > >> 309 suites, 1736 tests, 1 error, 14 ignored (12 assumptions) > >> > >> Total time: 6 minutes 14 seconds > >> Build step 'Execute shell' marked build as failure Archiving > >> artifacts Recording test results Email was triggered for: Failure > >> Sending email for trigger: Failure > >> > >> > >> > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For > >> additional commands, e-mail: dev-h...@lucene.apache.org > > > > > > > > -- > > lucidimagination.com > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For > > additional commands, e-mail: dev-h...@lucene.apache.org > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional > commands, e-mail: dev-h...@lucene.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
