I pushed a fix ... it was because memory and direct DV formats were illegally using an IndexInput from one thread while cloning it from another thread.
I also tried to improve RAMInputStream's EOFE exception messages to differentiate the two cases. Mike McCandless http://blog.mikemccandless.com On Sun, Jan 24, 2016 at 4:31 AM, Michael McCandless <luc...@mikemccandless.com> wrote: > I'll dig. > > Mike McCandless > > http://blog.mikemccandless.com > > > On Sat, Jan 23, 2016 at 7:57 PM, Policeman Jenkins Server > <jenk...@thetaphi.de> wrote: >> Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Linux/15631/ >> Java: 64bit/jdk-9-ea+95 -XX:+UseCompressedOops -XX:+UseParallelGC >> -XX:-CompactStrings >> >> 1 tests failed. >> FAILED: >> org.apache.lucene.index.TestMixedDocValuesUpdates.testManyReopensAndFields >> >> Error Message: >> this IndexWriter is closed >> >> Stack Trace: >> org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed >> at >> org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:714) >> at >> org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:728) >> at >> org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1459) >> at >> org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1242) >> at >> org.apache.lucene.index.TestMixedDocValuesUpdates.testManyReopensAndFields(TestMixedDocValuesUpdates.java:87) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:520) >> at >> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764) >> at >> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871) >> at >> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907) >> at >> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921) >> at >> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50) >> at >> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) >> at >> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49) >> at >> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) >> at >> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) >> at >> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) >> at >> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367) >> at >> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:809) >> at >> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:460) >> at >> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:880) >> at >> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:781) >> at >> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:816) >> at >> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:827) >> at >> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46) >> at >> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) >> at >> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42) >> at >> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) >> at >> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) >> at >> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) >> at >> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) >> at >> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) >> at >> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54) >> at >> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48) >> at >> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65) >> at >> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55) >> at >> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) >> at >> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:367) >> at java.lang.Thread.run(Thread.java:747) >> Caused by: java.io.EOFException: read past EOF: pos=3135 vs length=3135: >> RAMInputStream(name=_2r_3_Direct_0.dvdd) >> at >> org.apache.lucene.store.RAMInputStream.nextBuffer(RAMInputStream.java:116) >> at >> org.apache.lucene.store.RAMInputStream.readBytes(RAMInputStream.java:77) >> at >> org.apache.lucene.store.MockIndexInputWrapper.readBytes(MockIndexInputWrapper.java:138) >> at >> org.apache.lucene.store.BufferedChecksumIndexInput.readBytes(BufferedChecksumIndexInput.java:49) >> at org.apache.lucene.store.DataInput.readBytes(DataInput.java:87) >> at org.apache.lucene.store.DataInput.skipBytes(DataInput.java:350) >> at >> org.apache.lucene.store.ChecksumIndexInput.seek(ChecksumIndexInput.java:52) >> at >> org.apache.lucene.codecs.CodecUtil.checksumEntireFile(CodecUtil.java:436) >> at >> org.apache.lucene.codecs.memory.DirectDocValuesProducer.checkIntegrity(DirectDocValuesProducer.java:281) >> at >> org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsReader.checkIntegrity(PerFieldDocValuesFormat.java:341) >> at >> org.apache.lucene.index.SegmentDocValuesProducer.checkIntegrity(SegmentDocValuesProducer.java:141) >> at >> org.apache.lucene.codecs.DocValuesConsumer.merge(DocValuesConsumer.java:135) >> at >> org.apache.lucene.index.SegmentMerger.mergeDocValues(SegmentMerger.java:162) >> at >> org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:106) >> at >> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4062) >> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3642) >> at >> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588) >> at >> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626) >> >> >> >> >> Build Log: >> [...truncated 709 lines...] >> [junit4] Suite: org.apache.lucene.index.TestMixedDocValuesUpdates >> [junit4] 2> NOTE: reproduce with: ant test >> -Dtestcase=TestMixedDocValuesUpdates -Dtests.method=testManyReopensAndFields >> -Dtests.seed=1F8ED644ABF66A01 -Dtests.multiplier=3 -Dtests.slow=true >> -Dtests.locale=am-ET -Dtests.timezone=America/Tegucigalpa >> -Dtests.asserts=true -Dtests.file.encoding=UTF-8 >> [junit4] ERROR 1.40s J2 | >> TestMixedDocValuesUpdates.testManyReopensAndFields <<< >> [junit4] > Throwable #1: >> org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed >> [junit4] > at >> org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:714) >> [junit4] > at >> org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:728) >> [junit4] > at >> org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1459) >> [junit4] > at >> org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1242) >> [junit4] > at >> org.apache.lucene.index.TestMixedDocValuesUpdates.testManyReopensAndFields(TestMixedDocValuesUpdates.java:87) >> [junit4] > at java.lang.Thread.run(Thread.java:747) >> [junit4] > Caused by: java.io.EOFException: read past EOF: pos=3135 vs >> length=3135: RAMInputStream(name=_2r_3_Direct_0.dvdd) >> [junit4] > at >> org.apache.lucene.store.RAMInputStream.nextBuffer(RAMInputStream.java:116) >> [junit4] > at >> org.apache.lucene.store.RAMInputStream.readBytes(RAMInputStream.java:77) >> [junit4] > at >> org.apache.lucene.store.MockIndexInputWrapper.readBytes(MockIndexInputWrapper.java:138) >> [junit4] > at >> org.apache.lucene.store.BufferedChecksumIndexInput.readBytes(BufferedChecksumIndexInput.java:49) >> [junit4] > at >> org.apache.lucene.store.DataInput.readBytes(DataInput.java:87) >> [junit4] > at >> org.apache.lucene.store.DataInput.skipBytes(DataInput.java:350) >> [junit4] > at >> org.apache.lucene.store.ChecksumIndexInput.seek(ChecksumIndexInput.java:52) >> [junit4] > at >> org.apache.lucene.codecs.CodecUtil.checksumEntireFile(CodecUtil.java:436) >> [junit4] > at >> org.apache.lucene.codecs.memory.DirectDocValuesProducer.checkIntegrity(DirectDocValuesProducer.java:281) >> [junit4] > at >> org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsReader.checkIntegrity(PerFieldDocValuesFormat.java:341) >> [junit4] > at >> org.apache.lucene.index.SegmentDocValuesProducer.checkIntegrity(SegmentDocValuesProducer.java:141) >> [junit4] > at >> org.apache.lucene.codecs.DocValuesConsumer.merge(DocValuesConsumer.java:135) >> [junit4] > at >> org.apache.lucene.index.SegmentMerger.mergeDocValues(SegmentMerger.java:162) >> [junit4] > at >> org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:106) >> [junit4] > at >> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4062) >> [junit4] > at >> org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3642) >> [junit4] > at >> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588) >> [junit4] > at >> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626)Throwable >> #2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an >> uncaught exception in thread: Thread[id=749, name=Lucene Merge Thread #33, >> state=RUNNABLE, group=TGRP-TestMixedDocValuesUpdates] >> [junit4] > Caused by: >> org.apache.lucene.index.MergePolicy$MergeException: java.io.EOFException: >> read past EOF: pos=3135 vs length=3135: >> RAMInputStream(name=_2r_3_Direct_0.dvdd) >> [junit4] > at >> __randomizedtesting.SeedInfo.seed([1F8ED644ABF66A01]:0) >> [junit4] > at >> org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:668) >> [junit4] > at >> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:648) >> [junit4] > Caused by: java.io.EOFException: read past EOF: pos=3135 vs >> length=3135: RAMInputStream(name=_2r_3_Direct_0.dvdd) >> [junit4] > at >> org.apache.lucene.store.RAMInputStream.nextBuffer(RAMInputStream.java:116) >> [junit4] > at >> org.apache.lucene.store.RAMInputStream.readBytes(RAMInputStream.java:77) >> [junit4] > at >> org.apache.lucene.store.MockIndexInputWrapper.readBytes(MockIndexInputWrapper.java:138) >> [junit4] > at >> org.apache.lucene.store.BufferedChecksumIndexInput.readBytes(BufferedChecksumIndexInput.java:49) >> [junit4] > at >> org.apache.lucene.store.DataInput.readBytes(DataInput.java:87) >> [junit4] > at >> org.apache.lucene.store.DataInput.skipBytes(DataInput.java:350) >> [junit4] > at >> org.apache.lucene.store.ChecksumIndexInput.seek(ChecksumIndexInput.java:52) >> [junit4] > at >> org.apache.lucene.codecs.CodecUtil.checksumEntireFile(CodecUtil.java:436) >> [junit4] > at >> org.apache.lucene.codecs.memory.DirectDocValuesProducer.checkIntegrity(DirectDocValuesProducer.java:281) >> [junit4] > at >> org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsReader.checkIntegrity(PerFieldDocValuesFormat.java:341) >> [junit4] > at >> org.apache.lucene.index.SegmentDocValuesProducer.checkIntegrity(SegmentDocValuesProducer.java:141) >> [junit4] > at >> org.apache.lucene.codecs.DocValuesConsumer.merge(DocValuesConsumer.java:135) >> [junit4] > at >> org.apache.lucene.index.SegmentMerger.mergeDocValues(SegmentMerger.java:162) >> [junit4] > at >> org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:106) >> [junit4] > at >> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4062) >> [junit4] > at >> org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3642) >> [junit4] > at >> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588) >> [junit4] > at >> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626) >> [junit4] 2> ጃንዩ 23, 2016 6:53:38 ከሰዓት >> com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler >> uncaughtException >> [junit4] 2> WARNING: Uncaught exception in thread: Thread[Lucene Merge >> Thread #33,5,TGRP-TestMixedDocValuesUpdates] >> [junit4] 2> org.apache.lucene.index.MergePolicy$MergeException: >> java.io.EOFException: read past EOF: pos=3135 vs length=3135: >> RAMInputStream(name=_2r_3_Direct_0.dvdd) >> [junit4] 2> at >> __randomizedtesting.SeedInfo.seed([1F8ED644ABF66A01]:0) >> [junit4] 2> at >> org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:668) >> [junit4] 2> at >> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:648) >> [junit4] 2> Caused by: java.io.EOFException: read past EOF: pos=3135 vs >> length=3135: RAMInputStream(name=_2r_3_Direct_0.dvdd) >> [junit4] 2> at >> org.apache.lucene.store.RAMInputStream.nextBuffer(RAMInputStream.java:116) >> [junit4] 2> at >> org.apache.lucene.store.RAMInputStream.readBytes(RAMInputStream.java:77) >> [junit4] 2> at >> org.apache.lucene.store.MockIndexInputWrapper.readBytes(MockIndexInputWrapper.java:138) >> [junit4] 2> at >> org.apache.lucene.store.BufferedChecksumIndexInput.readBytes(BufferedChecksumIndexInput.java:49) >> [junit4] 2> at >> org.apache.lucene.store.DataInput.readBytes(DataInput.java:87) >> [junit4] 2> at >> org.apache.lucene.store.DataInput.skipBytes(DataInput.java:350) >> [junit4] 2> at >> org.apache.lucene.store.ChecksumIndexInput.seek(ChecksumIndexInput.java:52) >> [junit4] 2> at >> org.apache.lucene.codecs.CodecUtil.checksumEntireFile(CodecUtil.java:436) >> [junit4] 2> at >> org.apache.lucene.codecs.memory.DirectDocValuesProducer.checkIntegrity(DirectDocValuesProducer.java:281) >> [junit4] 2> at >> org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsReader.checkIntegrity(PerFieldDocValuesFormat.java:341) >> [junit4] 2> at >> org.apache.lucene.index.SegmentDocValuesProducer.checkIntegrity(SegmentDocValuesProducer.java:141) >> [junit4] 2> at >> org.apache.lucene.codecs.DocValuesConsumer.merge(DocValuesConsumer.java:135) >> [junit4] 2> at >> org.apache.lucene.index.SegmentMerger.mergeDocValues(SegmentMerger.java:162) >> [junit4] 2> at >> org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:106) >> [junit4] 2> at >> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4062) >> [junit4] 2> at >> org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3642) >> [junit4] 2> at >> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:588) >> [junit4] 2> at >> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:626) >> [junit4] 2> >> [junit4] IGNOR/A 0.00s J2 | TestMixedDocValuesUpdates.testTonsOfUpdates >> [junit4] > Assumption #1: 'nightly' test group is disabled (@Nightly()) >> [junit4] 2> NOTE: test params are: codec=Asserting(Lucene60): >> {updKey=BlockTreeOrds(blocksize=128), id=PostingsFormat(name=Memory >> doPackFST= true), key=PostingsFormat(name=Memory doPackFST= true)}, >> docValues:{cf3=DocValuesFormat(name=Direct), >> cf=DocValuesFormat(name=SimpleText), f=DocValuesFormat(name=Memory), >> f0=DocValuesFormat(name=Memory), f1=DocValuesFormat(name=SimpleText), >> f2=DocValuesFormat(name=Direct), cf0=DocValuesFormat(name=Asserting), >> f3=DocValuesFormat(name=Asserting), f4=DocValuesFormat(name=Memory), >> cf2=DocValuesFormat(name=SimpleText), f5=DocValuesFormat(name=SimpleText), >> cf1=DocValuesFormat(name=Memory)}, sim=ClassicSimilarity, locale=am-ET, >> timezone=America/Tegucigalpa >> [junit4] 2> NOTE: Linux 3.19.0-47-generic amd64/Oracle Corporation 9-ea >> (64-bit)/cpus=12,threads=1,free=379815216,total=523239424 >> [junit4] 2> NOTE: All tests run in this JVM: [TestAutomaton, >> TestWildcardRandom, TestMultiThreadTermVectors, TestCharsRefBuilder, >> TestFastCompressionMode, TestForUtil, TestIndexWriterMerging, >> Test2BPostingsBytes, TestDemoParallelLeafReader, >> TestControlledRealTimeReopenThread, TestPackedTokenAttributeImpl, >> TestFieldType, TestUsageTrackingFilterCachingPolicy, TestBooleanRewrites, >> TestTopDocsMerge, TestDateSort, TestExternalCodecs, TestReadOnlyIndex, >> TestRecyclingIntBlockAllocator, TestMultiMMap, TestAtomicUpdate, >> TestTotalHitCountCollector, TestCharTermAttributeImpl, TestStressIndexing2, >> TestIsCurrent, TestMatchNoDocsQuery, Test2BPositions, TestArrayUtil, >> TestNearSpansOrdered, TestSearchWithThreads, TestReqExclBulkScorer, >> TestIndexWriterOnDiskFull, TestConcurrentMergeScheduler, TestSort, >> Test2BTerms, TestNorms, TestBufferedChecksum, TestNIOFSDirectory, >> TestTieredMergePolicy, TestIndexFileDeleter, TestSimpleAttributeImpl, >> TestParallelReaderEmptyIndex, TestPrefixCodedTerms, TestFuzzyQuery, >> TestIndexInput, TestMathUtil, TestForceMergeForever, TestDirectPacked, >> TestSparseFixedBitSet, TestMaxTermFrequency, TestCachingCollector, >> TestSimpleSearchEquivalence, TestCrashCausesCorruptIndex, TestLongPostings, >> TestTopDocsCollector, TestIndexableField, TestSortedSetSelector, >> TestMixedDocValuesUpdates] >> [junit4] Completed [166/406 (1!)] on J2 in 7.48s, 4 tests, 1 error, 1 >> skipped <<< FAILURES! >> >> [...truncated 805 lines...] >> BUILD FAILED >> /home/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:675: The following >> error occurred while executing this line: >> /home/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:619: The following >> error occurred while executing this line: >> /home/jenkins/workspace/Lucene-Solr-trunk-Linux/build.xml:57: The following >> error occurred while executing this line: >> /home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/build.xml:50: The >> following error occurred while executing this line: >> /home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:1424: >> The following error occurred while executing this line: >> /home/jenkins/workspace/Lucene-Solr-trunk-Linux/lucene/common-build.xml:981: >> There were test failures: 406 suites (1 ignored), 3349 tests, 1 error, 55 >> ignored (51 assumptions) [seed: 1F8ED644ABF66A01] >> >> Total time: 5 minutes 42 seconds >> Build step 'Invoke Ant' marked build as failure >> Archiving artifacts >> [WARNINGS] Skipping publisher since build result is FAILURE >> Recording test results >> Email was triggered for: Failure - Any >> Sending email for trigger: Failure - Any >> >> >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >> For additional commands, e-mail: dev-h...@lucene.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
