Hes Siemelink wrote:
> It happens from time to time... but I don't know how to reproduce it.
>
> Rebuilding this particular index unfortunately takes about 10 hrs, so
it's
> not feasable to delete the index and rebuild it when this happens...
our
> users would be missing a lot of search results then!
>
> There are a couple of workaround strategies I could follow (using
separate
> indexes for updating and querying for example), but I would like to
know
> what's causing this so I can make a fix with less impact on my code.
This exception typically means that the segments file is referring to
a segment that doesn't exist in the filesytem (eg was deleted or lost
or something).
ok, that is what I thought...
Now the question is how this file gets deleted. Should I look into Lucene
(maybe there something goes wrong while segmenting?) or into the
environment
(maybe the temp directory that contains the locks is wiped, making a
critical lock disappear?)
One helpful thing to do is call IndexWriter.setInfoStream(...) and
save the resulting output. This prints details about which segments
were merged, and what the merged segment name is. This might provide
some useful details for example was your deleted segments file one
that was just merged away, or, was it a segment that was just created
by merging.
What kind of filesystem is your index stored on (eg, local or NFS)?
And how about the lock directory (normally defaults to the JVMs
java.io.tmpdir property)?
local file system, NOT NFS.
The lock directory is default. I believe Tomcat makes it point to its own
temp directory. I'll test the possibility of putting the locks somewhere
else.
Given that you are using a local filesystem, that rules out file
system caching issues which we see on NFS (whereby the filesystem is
presenting a "stale" copy of the segments file). I don't think local
filesystems typically have cache coherence issues.
When this exception happens, does it eventually go away on its own or
is the index permanantly corrupt (and you have to build another one)?
The index is permanently corrupt. Since the application is hosted by an
external party, I have not had the chance to inspect the files before
overwriting them.
Alas, given that that exception is "permanent" when this happens, I
don't think it's a locking issue (even if the tmp dir sweeps away lock
files). Locking issues usually result in transient exceptions (like
yours) when opening the reader, but once the writer is done
committing, then readers open fine without exceptions.
You are certain that only one writer is open at a time against the
index right? Lucene's write lock tries to ensure that, but still good
to verify.
When the exception occurs, is it possible to get a listing of all
files in the directory?
Are there any interesting customizations in your usage of Lucene?
Mike
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
