Rob Staveley (Tom) wrote:
It looks like my segments file only contains information for the .cfs
segments. So this approach doesn't work. I was wondering if I could use
IndexWriter.addIndexes(IndexReader[]) instead. Can I open an IndexReader
without a corresponding segments file? In notice that IndexReader.open(...)
always operates on directories, which I guess means that it uses the
segments file.
Is a segments file something that can be easily bodged for a bunch of index
files which aren't referenced by the segments file?
This probably all seems like a foolish errand, but my two indexes are > 300G
each and regenerating them is something I'd like to avoid.
1. do a backup first !!!
3. separate the segment data which is accounted for in the current
"segments" file from all other data, and move that to a "healthy" index.
Move the rest of the info to a "corrupt" index. Make sure that the
"healthy" index is still healthy after this operation .. ;)
2. look at the source of
org.apache.lucene.index.SegmentInfos.read(Directory) and
write(Directory). You can see how the new "segments" file is created
based on the SegmentInfo information. So, the only challenge is to
create a bunch of SegmentInfo instances corresponding to your segment
names in the "corrupt" index, and write them out to a new "segments"
file according to this format.
3. you can easily discover the number of documents in each segment. This
is equivalent to the length (in bytes) of each <segment_name>.f<number>
file, which are storing lengthNorm info per document and per field.
Once you have written the new "segments" file, try to open it in
IndexReader and iterate over documents to see if the index is basically
"healthy". You could also do an IndexWriter.optimize() to make sure that
all parts of the index correctly fit together.
A note: if a cfs file is corrupted, you can try to explode it into
individual files - you can use
org.apache.lucene.index.CompoundFileReader, iterate over file names,
open streams and save them to regular files; and then do the trick with
re-generating the segment file.
Then you may consider this "corrupt" index to be restored. Now you can
merge your "healthy" and "restored" indexes using IndexWriter.addIndexes().
BTW. I strongly recommend doing frequent backups at different stages.
Also, I recommend using BeanShell - you will save a lot of time you
would otherwise spend on editing and compilation of these steps.
--
Best regards,
Andrzej Bialecki <><
___. ___ ___ ___ _ _ __________________________________
[__ || __|__/|__||\/| Information Retrieval, Semantic Web
___|||__|| \| || | Embedded Unix, System Integration
http://www.sigram.com Contact: info at sigram dot com
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
