[
https://issues.apache.org/jira/browse/LUCENE-3147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13040708#comment-13040708
]
Michael McCandless commented on LUCENE-3147:
--------------------------------------------
I'm still hitting an intermittent fail:
{noformat}
NOTE: reproduce with: ant test -Dtestcase=TestIndexWriterWithThreads
-Dtestmethod=testIOExceptionDuringWriteSegmentWithThreads
-Dtests.seed=-3173666621914391759:573268695851991855
java.lang.RuntimeException: MockDirectoryWrapper: cannot close: there are still
open files: {_h.fdt=1, _h.tvd=1, _h.tvf=1, _h.fdx=1, _h.tvx=1}
at
org.apache.lucene.store.MockDirectoryWrapper.close(MockDirectoryWrapper.java:454)
at
org.apache.lucene.index.TestIndexWriterWithThreads._testMultipleThreadsFailure(TestIndexWriterWithThreads.java:279)
at
org.apache.lucene.index.TestIndexWriterWithThreads.testIOExceptionDuringWriteSegmentWithThreads(TestIndexWriterWithThreads.java:407)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
at org.junit.rules.TestWatchman$1.evaluate(TestWatchman.java:48)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:76)
at
org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1334)
at
org.apache.lucene.util.LuceneTestCase$LuceneTestCaseRunner.runChild(LuceneTestCase.java:1252)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
at org.junit.runners.Suite.runChild(Suite.java:128)
at org.junit.runners.Suite.runChild(Suite.java:24)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
at org.junit.runner.JUnitCore.run(JUnitCore.java:136)
at org.junit.runner.JUnitCore.run(JUnitCore.java:117)
at org.junit.runner.JUnitCore.runMain(JUnitCore.java:98)
at org.junit.runner.JUnitCore.runMainAndExit(JUnitCore.java:53)
at org.junit.runner.JUnitCore.main(JUnitCore.java:45)
Caused by: java.lang.RuntimeException: unclosed IndexOutput: _h.tvf
at
org.apache.lucene.store.MockDirectoryWrapper.addFileHandle(MockDirectoryWrapper.java:393)
at
org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:369)
at
org.apache.lucene.index.TermVectorsTermsWriter.initTermVectorsWriter(TermVectorsTermsWriter.java:98)
at
org.apache.lucene.index.TermVectorsTermsWriter.finishDocument(TermVectorsTermsWriter.java:123)
at org.apache.lucene.index.TermsHash.finishDocument(TermsHash.java:138)
at
org.apache.lucene.index.DocInverter.finishDocument(DocInverter.java:95)
at
org.apache.lucene.index.DocFieldProcessor.finishDocument(DocFieldProcessor.java:293)
at
org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:246)
at
org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:368)
at
org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1469)
at
org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1441)
at
org.apache.lucene.index.TestIndexWriterWithThreads$IndexerThread.run(TestIndexWriterWithThreads.java:65)
{noformat}
The seed doesn't repro for me; I have to run in a while(1), but it repros
fairly quickly with -Dtest.iters=10.
I'm not sure how/why but somehow there is a DWPT that successfully indexed 1
doc, opened the doc stores, and then is never heard from again... when it fails
it always still has the 5 doc stores open. Somehow, when we flushAll at close
we fail to visit this DWPT. I don't know where it's getting lost...
OK, this has something to do w/ DocsWriterFlushControl.abortFullFlushes -- that
method visits the DWPT in question, to abort its flush, but, this does not
close the opened doc store files.
But: why do we ever "abort" a full flush...? It looks like it happens because
one of the DWPTs hits an exc while flushing...?
> MockDirectoryWrapper should track open file handles of IndexOutput too
> ----------------------------------------------------------------------
>
> Key: LUCENE-3147
> URL: https://issues.apache.org/jira/browse/LUCENE-3147
> Project: Lucene - Java
> Issue Type: Test
> Components: general/test
> Reporter: Shai Erera
> Assignee: Shai Erera
> Priority: Minor
> Fix For: 3.2, 4.0
>
> Attachments: LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch,
> LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch, LUCENE-3147.patch,
> LUCENE-3147.patch, LUCENE-3147.patch
>
>
> MockDirectoryWrapper currently tracks open file handles of IndexInput only.
> Therefore IO files that are not closed do not fail our tests, which can then
> lead to test directories fail to delete on Windows. We should make sure all
> open files are tracked and if they are left open, fail the test. I'll attach
> a patch shortly.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
