[ http://issues.apache.org/jira/browse/LUCENE-669?page=comments#action_12453819 ] Michael Busch commented on LUCENE-669: --------------------------------------
Mike, you are right, it is very weird that 1) this problem happens and 2) my patch fixes it. It took me quite a while to figure out what the real problem is but I think I have at least a guess now. The problem again only happens on an IBM JVM. It happens rarely with your unmodified TestStressIndexing. If I change this testcase so that is does not perform the stress test on a RAMDirectory but only on a FSDirectory it happens almost always. This is VERY weird, because I did not change the stress test at all... I just commented the lines // // First in a RAM directory: // Directory directory = new RAMDirectory(); // runStressTest(directory); // directory.close(); and suddenly every test fails with the IOException "The handle is invalid". So that makes me think it has something to do with garbage collection and the finalize() methods. Now if I just comment out the finalize() method of FSDirectory.FSIndexOutput the test passes. This is even stranger, because the IOException occurred in FSDirectory.FSIndexInput.close(), not in FSIndexOutput, so in a different class which uses hence a different file descriptor. So I checked how java.io.FileDescriptor is implemented: It simply wraps an int value. I can't see how these int values are computed, because the open() method of RandomAccessFile is native. So I believe the following happens: (1) FSIndexOutput uses a FileDescriptor with value x (2) FSIndexOutput.close() is called, so the underlying file is being closed (3) A new FSIndexInput instance is created by a searcher thread. This opens a RandomAccessFile. Because FileDescriptor x is not in use anymore, x is used as the value for the new FileDescriptor. (4) Now garbage collection kicks in. It removes the old instance of FSIndexOutput for which close() has been called already. So the garbage collector calls finalize() which calls RandomAccessFile.close() again which still uses the descriptor with value x. So this call of close() actually closes the file used by the IndexInput instance created in (3). (5) FSIndexInput.close() is called and tries to close the file which has been closed already in (4) and thus the IOException occurs. So it seems to me that the IBM JVM makes file descriptor values available after a file has been closed, whereas Sun waits until the FileDescriptor instance is destroyed. This might be a bug in the JVM, but since this patch is very simple we could just use it to be on the safe side. Do you think this makes sense? Or does anybody have a better idea why commenting out the finalize() method in FSIndexOutput prevents FSIndexInput.close() from throwing the IOException? > finalize()-methods of FSDirectory.FSIndexInput and FSDirectory.FSIndexOutput > try to close already closed file > ------------------------------------------------------------------------------------------------------------- > > Key: LUCENE-669 > URL: http://issues.apache.org/jira/browse/LUCENE-669 > Project: Lucene - Java > Issue Type: Bug > Components: Store > Reporter: Michael Busch > Assigned To: Michael Busch > Priority: Trivial > Attachments: FSDirectory_close_file2.patch > > > Hi all, > I found a small problem in FSDirectory: The finalize()-methods of > FSDirectory.FSIndexInput and FSDirectory.FSIndexOutput try to close the > underlying file. This is not a problem unless the file has been closed before > by calling the close() method. If it has been closed before, the finalize > method throws an IOException saying that the file is already closed. Usually > this IOException would go unnoticed, because the GarbageCollector, which > calls finalize(), just eats it. However, if I use the Eclipse debugger the > execution of my code will always be suspended when this exception is thrown. > Even though this exception probably won't cause problems during normal > execution of Lucene, the code becomes cleaner if we apply this small patch. > Might this IOException also have a performance impact, if it is thrown very > frequently? > I attached the patch which applies cleanly on the current svn HEAD. All > testcases pass and I verfied with the Eclipse debugger that the IOException > is not longer thrown. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
