Re: Problems with IndexWriter#commit() on Linux

2010-02-10 Thread Michael McCandless
the document count and the max doc ID should >> >>>> > tell you something... >> >>>> > >> >>>> > Is it possible that you are updating existing docs >> >>>> > rather than adding new ones? >> >>>>

Re: Problems with IndexWriter#commit() on Linux

2010-02-10 Thread Naama Kraus
;> >> Thanks dor the input. > >>>> >> > >>>> >> 1. While the process is running, I do see the index files growing > on > >>>> disk > >>>> >> and the time stamps changing. Should I see a change in size right > after &g

Re: Problems with IndexWriter#commit() on Linux

2010-02-08 Thread Michael McCandless
right after >>>> >> killing the process, is that what you mean ? >>>> >> 2. Yes, same directory is being used for indexing and search. >>>> >> 3. Didn't try Luke, good idea. Though I wonder, the same code runs well >>>> on >>>

Re: Problems with IndexWriter#commit() on Linux

2010-02-08 Thread Naama Kraus
e time stamps changing. Should I see a change in size right > after > >>> >> killing the process, is that what you mean ? > >>> >> 2. Yes, same directory is being used for indexing and search. > >>> >> 3. Didn't try Luke, good idea. T

Re: Problems with IndexWriter#commit() on Linux

2010-02-08 Thread Michael McCandless
same directory is being used for indexing and search. >>> >> 3. Didn't try Luke, good idea. Though I wonder, the same code runs well >>> on >>> >> Windows. >>> >> >>> >> Naama >>> >> >>> >> On Thu, Jan 7,

Re: Problems with IndexWriter#commit() on Linux

2010-02-07 Thread Naama Kraus
gt;> >> >> > Several questions: >> >> > 1> are the index files larger after you kill your process? >> >> >Or have the timestamps changed? >> >> > 2> are you absolutely sure that your indexer, when you >> >> >

Re: Problems with IndexWriter#commit() on Linux

2010-01-07 Thread Naama Kraus
rick Erickson >> >wrote: > >> > >> > Several questions: > >> > 1> are the index files larger after you kill your process? > >> >Or have the timestamps changed? > >> > 2> are you absolutely sure that your indexer, when

Re: Problems with IndexWriter#commit() on Linux

2010-01-07 Thread Michael McCandless
; >    Or have the timestamps changed? >> > 2> are you absolutely sure that your indexer, when you >> >     add documents, is pointing at the same directory your >> >     search is pointing to? >> > 3> Have you gotten a copy of Luke and examined your

Re: Problems with IndexWriter#commit() on Linux

2010-01-07 Thread Erick Erickson
e directory your > > search is pointing to? > > 3> Have you gotten a copy of Luke and examined your index > > to see if, perhaps, your documents aren't being added the > > way you think they are? > > > > Erick > > > > On Thu, Jan 7, 2

Re: Problems with IndexWriter#commit() on Linux

2010-01-07 Thread Naama Kraus
> Erick > > On Thu, Jan 7, 2010 at 7:13 AM, Naama Kraus wrote: > > > Hi, > > > > I am using IndexWriter#commit() methods in my program to commit document > > additions to the index. I do that once in a while, after a bunch of > > documents were added. Since m

Re: Problems with IndexWriter#commit() on Linux

2010-01-07 Thread Erick Erickson
Luke and examined your index to see if, perhaps, your documents aren't being added the way you think they are? Erick On Thu, Jan 7, 2010 at 7:13 AM, Naama Kraus wrote: > Hi, > > I am using IndexWriter#commit() methods in my program to commit document > additions to the

Problems with IndexWriter#commit() on Linux

2010-01-07 Thread Naama Kraus
Hi, I am using IndexWriter#commit() methods in my program to commit document additions to the index. I do that once in a while, after a bunch of documents were added. Since my indexing process is long, I want to make sure I don't loose too many additions in case of a crash. When runni

Re: IndexWriter commit

2008-09-15 Thread Cam Bazz
Hello Dipen, I think what he meant is that if power is off the last transaction is trashed, but your index is not. Best. On Mon, Sep 15, 2008 at 10:55 PM, Dipen <[EMAIL PROTECTED]> wrote: > hi michael, > this is rather hard for me to understand, if a system loses power > (electricity), how can

Re: IndexWriter commit

2008-09-15 Thread Michael McCandless
It's only if power is lost *after* the call to IndexWriter.commit() has successfully returned, that the guarantee holds. commit() does not return until all newly written and referenced files in the index have been successfully fsync'd (and the OS does not return from fsync until all bytes

Re: IndexWriter commit

2008-09-15 Thread Dipen
hi michael, this is rather hard for me to understand, if a system loses power (electricity), how can it be ensured that fsync() call will happen at all, this commit function relies on fsync() but what if OS doesnt have time or power in this case to actually call fsync() and synchronize. I read ab

Re: IndexWriter commit

2008-09-15 Thread Cam Bazz
Hello, Thanks a bunch Michael. Its been a long time I wanted to upgrade to 2.4. It seems major change has been done. Best. On Mon, Sep 15, 2008 at 9:49 PM, Michael McCandless <[EMAIL PROTECTED]> wrote: > > Oh and I just committed a fix to IndexWriter's javadocs -- commit(long) is a > private met

Re: IndexWriter commit

2008-09-15 Thread Michael McCandless
Oh and I just committed a fix to IndexWriter's javadocs -- commit(long) is a private method that should never have been in the javadocs. Thanks for raising this! Mike Cam Bazz wrote: Hello, What is the difference between flush in <2.4 and commit? Also I have been looking over docs, an

Re: IndexWriter commit

2008-09-15 Thread Michael McCandless
There is no difference, unless your computer/OS crashes or loses power shortly after you had call the method. In that case, there's a big difference: commit() guarantees your index will be intact (assuming the storage system holding your index was not damaged) but with flush(), which does

IndexWriter commit

2008-09-15 Thread Cam Bazz
Hello, What is the difference between flush in <2.4 and commit? Also I have been looking over docs, and they mention commit(long) but there is no commit(long) method but only commit() Best. - To unsubscribe, e-mail: [EMAIL PROT