Re: Unique doc ids

2008-01-24 Thread Michael McCandless
Yonik Seeley wrote: On Jan 24, 2008 5:47 AM, Michael McCandless <[EMAIL PROTECTED]> wrote: Yonik Seeley wrote: On Jan 23, 2008 6:34 AM, Michael McCandless <[EMAIL PROTECTED]> wrote: writer.freezeDocIDs(); try { get docIDs from somewhere & call writer.deleteByDocID } finally

Re: Unique doc ids

2008-01-24 Thread Yonik Seeley
On Jan 24, 2008 5:47 AM, Michael McCandless <[EMAIL PROTECTED]> wrote: > > Yonik Seeley wrote: > > > On Jan 23, 2008 6:34 AM, Michael McCandless > > <[EMAIL PROTECTED]> wrote: > >>writer.freezeDocIDs(); > >>try { > >> get docIDs from somewhere & call writer.deleteByDocID > >>} fina

Re: Unique doc ids

2008-01-24 Thread Michael McCandless
Yonik Seeley wrote: On Jan 23, 2008 6:34 AM, Michael McCandless <[EMAIL PROTECTED]> wrote: writer.freezeDocIDs(); try { get docIDs from somewhere & call writer.deleteByDocID } finally { writer.unfreezeDocIDs(); } Interesting idea, but would require the IndexWriter to f

Re: Unique doc ids

2008-01-23 Thread Nadav Har'El
Hi Michael, On Tue, Jan 22, 2008, Michael Busch wrote about "Unique doc ids": > the question of how to delete with IndexWriter using doc ids is >... > mapping from the dynamic doc ids to the new unique ones. We would also > have to store a reverse mapping (UID -> ID) in

Re: Unique doc ids

2008-01-23 Thread Yonik Seeley
On Jan 23, 2008 6:34 AM, Michael McCandless <[EMAIL PROTECTED]> wrote: >writer.freezeDocIDs(); >try { > get docIDs from somewhere & call writer.deleteByDocID >} finally { > writer.unfreezeDocIDs(); >} Interesting idea, but would require the IndexWriter to flush the buffer

Re: Unique doc ids

2008-01-23 Thread Grant Ingersoll
On Jan 23, 2008, at 6:34 AM, Michael McCandless wrote: At first it might be optional, +1 There are still applications that don't require a UID, or are static for long enough periods of time that the Lucene internal id is sufficient, so I would hate to impose this on those apps. I thin

Re: Unique doc ids

2008-01-23 Thread Michael McCandless
its presence. At first it might be optional, but I could see us over time making more and more functionality that require UID to be present, to the point where it's eventually not really optional... Mike Michael Busch wrote: Paul Elschot wrote: Michael, How would IndexWriter.addIn

Re: Unique doc ids

2008-01-23 Thread Michael Busch
Paul Elschot wrote: > Michael, > > How would IndexWriter.addIndexes() work with unique doc ids? Hi Paul, it would probably be a limitation of this design. The only way I can think of right now to ensure that during an addIndexes() the UIDs don't change is an API in IndexWriter

Re: Unique doc ids

2008-01-23 Thread Michael Busch
Terry Yang wrote: > Hi,Michael, > You idea is good! But i have a question and thanks for your help! > Hi Terry, > Can u explain more about how you store a reverse UID-->ID? How u guarantee > UID > can be mapped to the correct dynamic ID. I mean if a docid =5 and then for > some reason changed t

Re: Unique doc ids

2008-01-23 Thread Paul Elschot
Michael, How would IndexWriter.addIndexes() work with unique doc ids? Regards, Paul Elschot Op Tuesday 22 January 2008 12:07:16 schreef Michael Busch: > Hi Team, > > the question of how to delete with IndexWriter using doc ids is > currently being discussed on java-user > (http

Re: Unique doc ids

2008-01-22 Thread Terry Yang
Hi,Michael, You idea is good! But i have a question and thanks for your help! How you plan to store a unique ID for each doc? My understanding will be adding a field(i.e uniqueid) for each doc and the field has one identical token value. We can add unique ID as payload for that token before indexi

Unique doc ids

2008-01-22 Thread Michael Busch
Hi Team, the question of how to delete with IndexWriter using doc ids is currently being discussed on java-user (http://www.gossamer-threads.com/lists/lucene/java-user/57228), so I thought this is a good time to mention an idea that I recently had. I'm planning to work on column-stored fields soon