Re: Optimize and internal document order

2007-09-03 Thread Andrzej Bialecki
Yonik Seeley wrote: On 8/31/07, Doug Cutting <[EMAIL PROTECTED]> wrote: If each document has an indexed id field in both indexes, then you could simply use a FieldCache of that id field in each index to determine the mapping. I've thought about this approach, but I think it has some scalabilit

Re: Optimize and internal document order

2007-09-03 Thread Yonik Seeley
On 8/31/07, Doug Cutting <[EMAIL PROTECTED]> wrote: > If each document has an indexed id field in both indexes, then you could > simply use a FieldCache of that id field in each index to determine the > mapping. I've thought about this approach, but I think it has some scalability issues... It see

Re: Optimize and internal document order

2007-08-31 Thread Doug Cutting
Andrzej Bialecki wrote: If so, then in the trunk/ version of Lucene, is there any way to figure out (predictably) how internal document numbers are reassigned after calling optimize() ? If each document has an indexed id field in both indexes, then you could simply use a FieldCache of that id

Re: Optimize and internal document order

2007-08-31 Thread Karl Wettin
31 aug 2007 kl. 10.04 skrev Andrzej Bialecki: Karl Wettin wrote: 30 aug 2007 kl. 22.50 skrev Andrzej Bialecki: I think this is possible to achieve by using a FilterIndexReader, which keeps a map of updated documents, and re-maps old doc ids to the new ones on the fly. From time to time I

Re: Optimize and internal document order

2007-08-31 Thread Andrzej Bialecki
Karl Wettin wrote: 30 aug 2007 kl. 22.50 skrev Andrzej Bialecki: I think this is possible to achieve by using a FilterIndexReader, which keeps a map of updated documents, and re-maps old doc ids to the new ones on the fly. From time to time I'd like to optimize the "aux" index to get rid of

Re: Optimize and internal document order

2007-08-30 Thread Karl Wettin
30 aug 2007 kl. 22.50 skrev Andrzej Bialecki: I think this is possible to achieve by using a FilterIndexReader, which keeps a map of updated documents, and re-maps old doc ids to the new ones on the fly. From time to time I'd like to optimize the "aux" index to get rid of deleted docs. A

Optimize and internal document order

2007-08-30 Thread Andrzej Bialecki
Hi all, I have the following scenario: I want to use ParallelReader to maintain parts of the index that are changing quickly, and where changes are limited to specific fields only. Let's say I have a "main" index (many fields, slowly changing, large updates), and an "aux" index (fast changin