RE: Reopen IndexWriter after delete?
I suggest checking the list archive. Doug has explained the reasons behind the current design several times. Otis --- "Wilton, Reece" <[EMAIL PROTECTED]> wrote: > I agree it's a bit of a strange design. > > It seems that there should be one class that handles all > modifications > of the index. Usually you'd only have one instance of this so you > wouldn't need to open and close it all the time (I'm basically > writing > one of these classes myself to simplify my code. I'm sure other > people > have written a similar class). There should be another class that is > responsible for searching. You may have multiple instances of this > so > you can have multiple headends searching the index. > > The IndexWriter and IndexReader almost do this separation. It seems > that if the IndexWriter had the delete functionality, instead of the > IndexReader, things would be a lot simplier (from a synchronization > standpoint). Maybe Otis, Erik or Doug could suggest why this may or > may > not be a good idea. > > -Reece > > -Original Message- > From: Dror Matalon [mailto:[EMAIL PROTECTED] > Sent: Wednesday, November 12, 2003 12:06 PM > To: Lucene Users List > Subject: Re: Reopen IndexWriter after delete? > > Which begs the question: why do you need to use an IndexReader rather > than an IndexWriter to delete an item? > > On Tue, Nov 11, 2003 at 02:46:37PM -0800, Otis Gospodnetic wrote: > > > 1). If I delete a term using an IndexReader, can I use an > existing > > > IndexWriter to write to the index? Or do I need to close and > reopen > > > the IndexWriter? > > > > No. You should close IndexWriter first, then open IndexReader, > then > > call delete, then close IndexReader, and then open a new > IndexWriter. > > > > > 2). Is it safe to call IndexReader.delete(term) while an > IndexWriter > > > is > > > writing? Or should I be synchronizing these two tasks so only > one > > > occurs at a time? > > > > No, it is not safe. You should close the IndexWriter, then delete > the > > document and close IndexReader, and then get a new IndexWriter and > > continue writing. > > > > Incidentally, I just wrote a section about concurrency issues and > about > > locking in Lucene for the upcoming Lucene book. > > > > Otis > > > > > > __ > > Do you Yahoo!? > > Protect your identity with Yahoo! Mail AddressGuard > > http://antispam.yahoo.com/whatsnewfree > > > > > - > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > For additional commands, e-mail: > [EMAIL PROTECTED] > > > > -- > Dror Matalon > Zapatec Inc > 1700 MLK Way > Berkeley, CA 94709 > http://www.fastbuzz.com > http://www.zapatec.com > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > __ Do you Yahoo!? Protect your identity with Yahoo! Mail AddressGuard http://antispam.yahoo.com/whatsnewfree - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Reopen IndexWriter after delete?
Because Lucene has to first find the segment that the specified document is in, and this is done via IndexReaders, not IndexWriters. More about this in the Lucene book. Otis --- Dror Matalon <[EMAIL PROTECTED]> wrote: > Which begs the question: why do you need to use an IndexReader rather > than an IndexWriter to delete an item? > > On Tue, Nov 11, 2003 at 02:46:37PM -0800, Otis Gospodnetic wrote: > > > 1). If I delete a term using an IndexReader, can I use an > existing > > > IndexWriter to write to the index? Or do I need to close and > reopen > > > the IndexWriter? > > > > No. You should close IndexWriter first, then open IndexReader, > then > > call delete, then close IndexReader, and then open a new > IndexWriter. > > > > > 2). Is it safe to call IndexReader.delete(term) while an > IndexWriter > > > is > > > writing? Or should I be synchronizing these two tasks so only > one > > > occurs at a time? > > > > No, it is not safe. You should close the IndexWriter, then delete > the > > document and close IndexReader, and then get a new IndexWriter and > > continue writing. > > > > Incidentally, I just wrote a section about concurrency issues and > about > > locking in Lucene for the upcoming Lucene book. > > > > Otis > > > > > > __ > > Do you Yahoo!? > > Protect your identity with Yahoo! Mail AddressGuard > > http://antispam.yahoo.com/whatsnewfree > > > > > - > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > For additional commands, e-mail: > [EMAIL PROTECTED] > > > > -- > Dror Matalon > Zapatec Inc > 1700 MLK Way > Berkeley, CA 94709 > http://www.fastbuzz.com > http://www.zapatec.com > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > __ Do you Yahoo!? Protect your identity with Yahoo! Mail AddressGuard http://antispam.yahoo.com/whatsnewfree - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Reopen IndexWriter after delete?
I agree it's a bit of a strange design. It seems that there should be one class that handles all modifications of the index. Usually you'd only have one instance of this so you wouldn't need to open and close it all the time (I'm basically writing one of these classes myself to simplify my code. I'm sure other people have written a similar class). There should be another class that is responsible for searching. You may have multiple instances of this so you can have multiple headends searching the index. The IndexWriter and IndexReader almost do this separation. It seems that if the IndexWriter had the delete functionality, instead of the IndexReader, things would be a lot simplier (from a synchronization standpoint). Maybe Otis, Erik or Doug could suggest why this may or may not be a good idea. -Reece -Original Message- From: Dror Matalon [mailto:[EMAIL PROTECTED] Sent: Wednesday, November 12, 2003 12:06 PM To: Lucene Users List Subject: Re: Reopen IndexWriter after delete? Which begs the question: why do you need to use an IndexReader rather than an IndexWriter to delete an item? On Tue, Nov 11, 2003 at 02:46:37PM -0800, Otis Gospodnetic wrote: > > 1). If I delete a term using an IndexReader, can I use an existing > > IndexWriter to write to the index? Or do I need to close and reopen > > the IndexWriter? > > No. You should close IndexWriter first, then open IndexReader, then > call delete, then close IndexReader, and then open a new IndexWriter. > > > 2). Is it safe to call IndexReader.delete(term) while an IndexWriter > > is > > writing? Or should I be synchronizing these two tasks so only one > > occurs at a time? > > No, it is not safe. You should close the IndexWriter, then delete the > document and close IndexReader, and then get a new IndexWriter and > continue writing. > > Incidentally, I just wrote a section about concurrency issues and about > locking in Lucene for the upcoming Lucene book. > > Otis > > > __ > Do you Yahoo!? > Protect your identity with Yahoo! Mail AddressGuard > http://antispam.yahoo.com/whatsnewfree > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > -- Dror Matalon Zapatec Inc 1700 MLK Way Berkeley, CA 94709 http://www.fastbuzz.com http://www.zapatec.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Reopen IndexWriter after delete?
Which begs the question: why do you need to use an IndexReader rather than an IndexWriter to delete an item? On Tue, Nov 11, 2003 at 02:46:37PM -0800, Otis Gospodnetic wrote: > > 1). If I delete a term using an IndexReader, can I use an existing > > IndexWriter to write to the index? Or do I need to close and reopen > > the IndexWriter? > > No. You should close IndexWriter first, then open IndexReader, then > call delete, then close IndexReader, and then open a new IndexWriter. > > > 2). Is it safe to call IndexReader.delete(term) while an IndexWriter > > is > > writing? Or should I be synchronizing these two tasks so only one > > occurs at a time? > > No, it is not safe. You should close the IndexWriter, then delete the > document and close IndexReader, and then get a new IndexWriter and > continue writing. > > Incidentally, I just wrote a section about concurrency issues and about > locking in Lucene for the upcoming Lucene book. > > Otis > > > __ > Do you Yahoo!? > Protect your identity with Yahoo! Mail AddressGuard > http://antispam.yahoo.com/whatsnewfree > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > -- Dror Matalon Zapatec Inc 1700 MLK Way Berkeley, CA 94709 http://www.fastbuzz.com http://www.zapatec.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Reopen IndexWriter after delete?
Correct. write.lock is used for that. Otis --- Morus Walter <[EMAIL PROTECTED]> wrote: > Otis Gospodnetic writes: > > > > No, it is not safe. You should close the IndexWriter, then delete > the > > document and close IndexReader, and then get a new IndexWriter and > > continue writing. > > > IIRC lucene takes care that you do so. > Locking prevents you from having an open IndexWriter and > modify the index with an IndexReader (and vice verse). > > Morus > > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > __ Do you Yahoo!? Protect your identity with Yahoo! Mail AddressGuard http://antispam.yahoo.com/whatsnewfree - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Reopen IndexWriter after delete?
Otis Gospodnetic writes: > > No, it is not safe. You should close the IndexWriter, then delete the > document and close IndexReader, and then get a new IndexWriter and > continue writing. > IIRC lucene takes care that you do so. Locking prevents you from having an open IndexWriter and modify the index with an IndexReader (and vice verse). Morus - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Reopen IndexWriter after delete?
> 1). If I delete a term using an IndexReader, can I use an existing > IndexWriter to write to the index? Or do I need to close and reopen > the IndexWriter? No. You should close IndexWriter first, then open IndexReader, then call delete, then close IndexReader, and then open a new IndexWriter. > 2). Is it safe to call IndexReader.delete(term) while an IndexWriter > is > writing? Or should I be synchronizing these two tasks so only one > occurs at a time? No, it is not safe. You should close the IndexWriter, then delete the document and close IndexReader, and then get a new IndexWriter and continue writing. Incidentally, I just wrote a section about concurrency issues and about locking in Lucene for the upcoming Lucene book. Otis __ Do you Yahoo!? Protect your identity with Yahoo! Mail AddressGuard http://antispam.yahoo.com/whatsnewfree - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Reopen IndexWriter after delete?
Hi, A couple questions... 1). If I delete a term using an IndexReader, can I use an existing IndexWriter to write to the index? Or do I need to close and reopen the IndexWriter? 2). Is it safe to call IndexReader.delete(term) while an IndexWriter is writing? Or should I be synchronizing these two tasks so only one occurs at a time? Any help is appreciated! -Reece - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
