Yesterday, when typing a reply I noticed an error ... the
CheckIndex.check
was not checking a good directory.. so the segments not found error
was not
correct. I apologize. I made the correction and this is the
correct error
that occurs when an i/o error is thrown. I ran the CheckIndex.check
method
after the i/o exception was caught:
Segments file=segments_296 numSegments=1
version=FORMAT_SHARED_DOC_STORE
[Lucene 2.3]
1 of 1: name=+1dq docCount=1788454
compund=true
numFiles=1
size (MB) = 331.125
no deletions
test: open reader....... OK
test: fields, norms...... OK [4 fields]
test: terms, freq, prox...2008-11-05 15:12:33,357 INFO Cache:195 -
Purging 0 of 4 Cached results.
2008-11-05 15:12:33,358 INFO Cache: 196 - 956018816 of 1073676288
bytes of
memory are currently free
OK [3409802 terms; 19311204 terms/docs pairs; 19842525 tokens]
test: stored fields....... Wed Nov 05 15:12:59 UTC 2008: Purging
Query
Cache 0 of 9 query cached results.
Wed Nov 05 15:12:59 UTC 2008: 955521656 of 1073676288 bytes of
memory are
currently free.
FAILED
WARNING: would remove reference to this segment (-fix was not
specified); full exception:
java.io.IOException: Input/Output error
at java.io.RandomAccessfile.readBytes(Native Method)
at java.io.RandomAccessFile.read(RandomAccessfile.java:315)
at
org.apache.lucene.store.FSDirectory
$FSIndexInput.readInternal(FSDirectory.java:550)
at
org
.apache
.lucene.store.BufferedIndexInput.readBytes(BufferedIndexInput.java:
131)
at
org.apache.lucene.index.CompoundFileReader
$CSIndexInput.readInternal(CompoundFileReader.java:240)
at
org
.apache
.lucene.store.BufferedIndexInput.refill(bufferedIndexInput.java:152)
at
org
.apache
.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:38)
at org.apache.lucene.store.IndexInput.readChars(IndexInput.java:126)
at org.apache.lucene.store.IndexInput.readString(IndexInput.java:112)
at org.apache.lucene.index.FieldsReader.addfield(FieldsReader.java:
324)
at org.apache.lucene.index.FieldsReader.doc(FieldsReader.java:166)
at
org.apache.lucene.index.SegmentReader.document(SegmentReader.java:659)
at org.apache.lucene.index.IndexReader.document(IndexReader.java:525)
at org.apache.lucene.index.CheckIndex.check(checkIndex.java:234)
at
gminer
.rest
.content
.RestContentSearchServlet.doGet(RestContentSearchServlet.java:269)
.................. continues on and then
WARNING: 1 broken segments detected
WARNING: 1788454 documents would be lost if -fix were specified
NOTE: would write new segments file [-fix was not specified]
************************************************
Then I wrote out the return status from the check method call and it
was "
false".
The Normal Input/Output error follows...
NOW, I am leaving the IndexSearcher open for the current day and if I
resubmit the query... the query will most likely work the 2nd
time... Is
there something to be done to the index so that there are not the
errors
above?
I appreciate all your help.
Julie
JulieSoko wrote:
I am narrowing down this problem that I have had for a week
now... I am
using lucene version 2.3.1 and 64 bit java versio 1.5.0-12-b04
running on
Linux box. We are merging indexes every 60 seconds and there are
1..*
searches occuring at anytime on the indexes. The problem is that
we will
get an Input /Output error trying to read the index for a search
randomly... say every 5th search. I have posted the error before,
but
have narrowed it down, I believe, to a merge issue.
This is the error that a searcher will output at random times:
java.io.IOException: Input/output error
java.io.RandomAccessFile.readBytes(Native Method)
java.io.RandomAccessFile.read(RandomAccessFile.java:315)
at
org.apache.lucene.store.FSDirecotry
$FSIndexInput.readInternal(FSDirectory.java:550)
at
org
.apache
.lucene.store.BufferedIndexInput.readBytes(BufferedInputInput.java:
131)
at
org.apache.lucene.index.CompoundFileReader
$CSIndexInput.readInternal(CompoundFileReader.java:240)
at
org
.apache
.lucene.instoreBufferedIndexInput.refill(BufferedIndexInput.java:
152)
at
org
.apache
.lucene.instoreBufferedIndexInput.readByte(BufferedIndexInput.java:
152)
at org.lucene.store.IndexInput.readVInt(IndexInput.java:76)
at org.apache.lucene.index.TermBuffer.read(TermBuffer.java:63)
at
org.apache.lucene.index.SegmentTermEnum.next(SegmentTermEnum.java:
123)
at
org.apache.lucene.index.SegmentTermEnum.scanTo(SegmentTermEnum.java:
154)
at
org
.apache.lucene.index.TermInfosReader.scanEnum(TermInfosReader.java:
223)
at org.apache.lucene.index.TermInfosReader.get(TermInfosReader.java:
217)
at org.apache.lucene.index.SegmentReader.docFreq(SegmentReader.java:
678)
at
org.apache.lucene.search.IndexSearcher.docFreq(IndexSearcher.java:87)
at org.apache.lucene.search.Searcher.docFreqs(searcher.java:118)
at
org
.apache.lucene.search.MultiSearcher.createWeight(MultiSearcher.java:
311)
at org.apache.lucene.search.Searcher.search(Searcher.java:178)
******************************************************************************
NOW, when I get the above Exception, I check the index using the
CheckIndex.check method... As part of the check , this exception is
thrown:
Error: could not read any segments file in directory
java.io.FileNotFoundException: no segments* file found in
[EMAIL PROTECTED]/rt10/jetty/20081103
at
org.apache.lucene.index.SegmentInfos
$findSegmentsFile.run(SegementInfos.java:587)
.....
Is there any point in the merging of indexes that the segment files
are
removed? If I rerun the search, right after this error occurs, the
search is ok... I do open a new IndexSearcher...
The IndexWriter code is this:
IndexWriter combinedWriter = new IndexWriter(currentMergeDir, new
StandardAnalyzer());
combinedWriter.addIndexes(dirToMerge);
combinedWriter.flush();
combinedWriter.close();
As you can see above, each time there is a merge a new IndexWriter is
created, indexes added, flushed and closed.
I know you are not supposed to have synchronization issues between
writing
and flushing , but could there be an issue when you are creating a
new
searcher at the instant where the files are merged and there are no
segments in a dir???
Thanks,
Julie
--
View this message in context:
http://www.nabble.com/No-segment-files-found--Searcher-error-tp20305354p20343211.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
