On Saturday 02 April 2005 10:29, Kristian Ottosen wrote: > Hello, > > I have really been pulling out hair for a while over this. The problem only > occur using FSDirectory on some Linux system (some Debian, Suse and Redhat - > but not all) and never under Windows, Solaris or Mac OS X. > > What happens is that Lucene for some reason tries to read a segment which > was just merged into another segment and therefore deleted and fails with a > FileNotFoundException. > > A few facts: Only a single thread is involved. If we use a RAM based Store > (RAMDirectory) there is no problem at all. Also there seems to a problem > related to locking using the lock files. We get timeouts even though there > is only a single thread working on the index. Switching the compound file > format on/off makes no difference. The problem may not appear immediately - > but only after several thousand documents have been indexed in a row using a > combination of IndexSearcher (check for duplicates) and IndexWriter (add > document) operations. > > I wonder if there is general problem running Lucene on top of some of the > journaling file systems like ReiserFS or Ext3?
The only reason I can think of for a FileNotFoundException is that the corresponding close() of the file has not terminated yet, or was not done at all. I don't know the precise file system semantics in relation to a single Linux process, but if these semantics allow for a file closed by a process to be (temporarily) invisible to that same process, one could end up with a FileNotFoundException. It's the first time AFAIK something like this has been reported, so another source of the problem could be your hard disk, but I would not expect a journalling file system to ignore errors like this, ie. a journalling file system should provide some indication of a error situation for a hardware problem. Could you might check the system log for file system messages? Going back to the file system semantics allowing a temporarily invisible file, you might change the Lucene java code to perform a file system sync (via a shell) in case of the FileNotFoundException, and then retry opening the file? If that helps, it's the file system semantics. In case there is such a thing as a file system sync in the java.io package, this would have to be added to Lucene. Otherwise it would probably be useful to inform the maintainers of the JVM. Regards, Paul Elschot. --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
