OK I found a problem (fixes the OOM here). RandomIndexWriter was making pretty enormous BYTES_VAR_SORTED values for the random dv fields it adds. (max 800 bytes).
I committed a fix. On Fri, Mar 23, 2012 at 6:07 AM, Robert Muir <[email protected]> wrote: > Right but the point of TestMixedCodecs (I think) is to write different > segments with different codecs (so they are using different filenames > and implementations for all parts of the index), and then test things > like forceMerge'ing them all, so we don't have bugs where we somehow > use the wrong codec or whatever. > > So SimpleText is important here: it has a totally different impl for > every codec api, I dont want to disable it. I think it might be a case > of RandomIndexWriter making too many docvalues fields... but I'm not > sure. > > At least it reproduces > > On Fri, Mar 23, 2012 at 6:00 AM, Uwe Schindler <[email protected]> wrote: >> It was something like @UseNoMemoryExpensiveCodec, right? >> >> ----- >> Uwe Schindler >> H.-H.-Meier-Allee 63, D-28213 Bremen >> http://www.thetaphi.de >> eMail: [email protected] >> >> >>> -----Original Message----- >>> From: Robert Muir [mailto:[email protected]] >>> Sent: Friday, March 23, 2012 10:37 AM >>> To: [email protected] >>> Subject: Re: [JENKINS] Lucene-trunk - Build # 1869 - Still Failing >>> >>> I think maybe this test needs to be toned down after simpletext >>> docvalues... i've >>> noticed with nightly builds (-Dtests.nightly, >>> -Dtests.multiplier=5) it oom's often. >>> >>> i dont think we want to disable simpletext for this test, because its the >>> only >>> other real docvalues we have. >>> >>> instead we probably just want to tone it down... i'll see if i can reproduce >>> locally. >>> >>> On Fri, Mar 23, 2012 at 1:04 AM, Apache Jenkins Server >>> <[email protected]> wrote: >>> > Build: https://builds.apache.org/job/Lucene-trunk/1869/ >>> > >>> > 1 tests failed. >>> > REGRESSION: org.apache.lucene.index.TestMixedCodecs.test >>> > >>> > Error Message: >>> > GC overhead limit exceeded >>> > >>> > Stack Trace: >>> > java.lang.OutOfMemoryError: GC overhead limit exceeded >>> > at java.nio.ByteBuffer.wrap(ByteBuffer.java:367) >>> > at >>> > java.lang.StringCoding$StringDecoder.decode(StringCoding.java:155) >>> > at java.lang.StringCoding.decode(StringCoding.java:196) >>> > at java.lang.String.<init>(String.java:491) >>> > at >>> > org.apache.lucene.codecs.simpletext.SimpleTextPerDocProducer.readStrin >>> > g(SimpleTextPerDocProducer.java:210) >>> > at >>> > org.apache.lucene.codecs.simpletext.SimpleTextPerDocProducer$ValueRead >>> > er.fromString(SimpleTextPerDocProducer.java:309) >>> > at >>> > org.apache.lucene.codecs.simpletext.SimpleTextPerDocProducer$SimpleTex >>> > tDocValues.read(SimpleTextPerDocProducer.java:185) >>> > at >>> > org.apache.lucene.codecs.simpletext.SimpleTextPerDocProducer$SimpleTex >>> > tDocValues.load(SimpleTextPerDocProducer.java:150) >>> > at >>> > org.apache.lucene.index.DocValues$SourceCache$DirectSourceCache.load(D >>> > ocValues.java:680) >>> > at >>> > org.apache.lucene.index.DocValues.getSource(DocValues.java:81) >>> > at >>> > org.apache.lucene.codecs.simpletext.SimpleTextPerDocProducer$SimpleTex >>> > tDocValues.getDirectSource(SimpleTextPerDocProducer.java:194) >>> > at >>> > >>> org.apache.lucene.codecs.DocValuesConsumer.merge(DocValuesConsumer.jav >>> > a:112) >>> > at >>> > >>> org.apache.lucene.codecs.DocValuesConsumer.merge(DocValuesConsumer.jav >>> > a:90) >>> > at >>> > org.apache.lucene.codecs.PerDocConsumer.merge(PerDocConsumer.java:60) >>> > at >>> > >>> org.apache.lucene.index.SegmentMerger.mergePerDoc(SegmentMerger.java:3 >>> > 79) >>> > at >>> > org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:116) >>> > at >>> > org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3382) >>> > at >>> > org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3004) >>> > at >>> > org.apache.lucene.index.SerialMergeScheduler.merge(SerialMergeSchedule >>> > r.java:37) >>> > at >>> > org.apache.lucene.index.IndexWriter.maybeMerge(IndexWriter.java:1644) >>> > at >>> > org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1458) >>> > at >>> > org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:1412) >>> > at >>> > >>> org.apache.lucene.index.RandomIndexWriter.doRandomForceMerge(RandomI >>> nd >>> > exWriter.java:361) >>> > at >>> > org.apache.lucene.index.RandomIndexWriter.close(RandomIndexWriter.java >>> > :406) >>> > at >>> > org.apache.lucene.index.TestMixedCodecs.test(TestMixedCodecs.java:57) >>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>> > at >>> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.j >>> > ava:57) >>> > at >>> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccess >>> > orImpl.java:43) >>> > at java.lang.reflect.Method.invoke(Method.java:616) >>> > at >>> > >>> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkM >>> > ethod.java:45) >>> > at >>> > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCall >>> > able.java:15) >>> > at >>> > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMet >>> > hod.java:42) >>> > >>> > >>> > >>> > >>> > Build Log (for compile errors): >>> > [...truncated 13394 lines...] >>> > >>> > >>> > >>> > >>> > --------------------------------------------------------------------- >>> > To unsubscribe, e-mail: [email protected] For >>> > additional commands, e-mail: [email protected] >>> >>> >>> >>> -- >>> lucidimagination.com >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: [email protected] For additional >>> commands, e-mail: [email protected] >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> > > > > -- > lucidimagination.com -- lucidimagination.com --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
