[jira] Created: (LUCENE-1328) FileNotFoundException in

2008-07-07 Thread Yajun Liu (JIRA)
FileNotFoundException in - Key: LUCENE-1328 URL: https://issues.apache.org/jira/browse/LUCENE-1328 Project: Lucene - Java Issue Type: Bug Components: Index Affects Versions: 2.1 Environment: OS

Re: [jira] Created: (LUCENE-1328) FileNotFoundException in

2008-07-07 Thread robert engels
If your "automatic recycle" means a restart/reboot, the /tmp directory is probably being cleared by the OS and you might have a startup race condition. On Jul 7, 2008, at 2:17 AM, Yajun Liu (JIRA) wrote: FileNotFoundException in - Key: LUCENE-1328

Re: [jira] Created: (LUCENE-1328) FileNotFoundException in

2008-07-07 Thread Michael McCandless
Yajun Liu (JIRA) <[EMAIL PROTECTED]> wrote: Can you double check which underlying version of Lucene you are using? Those source file/line numbers don't line up to a stock 2.1 release as far as I can tell. This part is odd: > When this happen, the program automatically tried to reopen the most >

Re: [jira] Created: (LUCENE-1328) FileNotFoundException in

2008-07-07 Thread Yajun
My bad, we don't use /tmp explicitly. We use /var/tmp/snapshot_timestamp which is not deleted by OS when reboot. --Yajun Robert Engels wrote: > > If your "automatic recycle" means a restart/reboot, the /tmp > directory is probably being cleared by the OS and you might have a > startup race

Re: [jira] Created: (LUCENE-1328) FileNotFoundException in

2008-07-07 Thread Yajun
My answer is inline. Michael McCandless-2 wrote: > > Yajun Liu (JIRA) <[EMAIL PROTECTED]> wrote: > > Can you double check which underlying version of Lucene you are > using? Those source file/line numbers don't line up to a stock 2.1 > release as far as I can tell. > > YL>> The lucene libra

Re: [jira] Created: (LUCENE-1328) FileNotFoundException in

2008-07-07 Thread Michael McCandless
Yajun wrote: YL>> The lucene library comes with Solr. The jar file is lucene-core-2007-05-20_00-04-53.jar. I compared the source code of IndexReader which is close to Lucene 2.2, not 2.1 Hmmm OK thanks. This part is odd: When this happen, the program automatically tried to reopen the most

Re: [jira] Created: (LUCENE-1328) FileNotFoundException in

2008-07-07 Thread Yajun
I don't use deleteDocument nor setNorm with IndexReader. I tried both using hard links (cp -lr) and copy (cp -r) to create snapshot, both has the same problem. It seems that segment file segments_xxx has segment that does not exist in the index directory anymore. I used to delete all the "inval

Re: [jira] Created: (LUCENE-1328) FileNotFoundException in

2008-07-07 Thread Michael McCandless
One more question: are you sure that when you copy out your snapshot, the IndexWriter was closed? If IndexWriter is open when the copy is done, it's possible to get a corrupted copy. Mike Yajun wrote: I don't use deleteDocument nor setNorm with IndexReader. I tried both using hard l

Re: [jira] Created: (LUCENE-1328) FileNotFoundException in

2008-07-07 Thread Yajun
Mike, Not very sure about whether IndexWriter is closed. The index update goes through Solr, I'll debug it tonight. Even if IndexWriter is not closed, since I "validate" the snapshot, at least that the snapshot should keep being "validated". :-) --Yajun Michael McCandless-2 wrote: > > > On

Re: [jira] Created: (LUCENE-1328) FileNotFoundException in

2008-07-07 Thread Michael McCandless
Ahh right your validation would catch the IndexWriter-still-open case. It seems like something external to Lucene is messing up your index. It's odd. Mike Yajun wrote: Mike, Not very sure about whether IndexWriter is closed. The index update goes through Solr, I'll debug it tonight.

Re: [jira] Created: (LUCENE-1328) FileNotFoundException in

2008-07-07 Thread Yajun
I'm adding tons of logging, hopefully it will give me some information. --Yajun Michael McCandless-2 wrote: > > > Ahh right your validation would catch the IndexWriter-still-open case. > > It seems like something external to Lucene is messing up your index. > It's odd. > > Mike > > Yaju

Re: [jira] Created: (LUCENE-1328) FileNotFoundException in

2008-07-07 Thread Yonik Seeley
On Mon, Jul 7, 2008 at 5:03 PM, Yajun <[EMAIL PROTECTED]> wrote: > > I'm adding tons of logging, hopefully it will give me some information. Try capturing the directory contents before you take a snapshot... something like ls -l index > index/ls.txt Then if a missing file turns up, you can compar

Re: [jira] Created: (LUCENE-1328) FileNotFoundException in

2008-07-08 Thread robert engels
On most system, /tmp is a link to /var/tmp, so it is cleared on reboot. On Jul 7, 2008, at 12:44 PM, Yajun wrote: My bad, we don't use /tmp explicitly. We use /var/tmp/ snapshot_timestamp which is not deleted by OS when reboot. --Yajun Robert Engels wrote: If your "automatic recycle" me