Re: building custom cache - using lucene docids

2013-11-23 Thread Erick Erickson
bq: But can I assume that docids in other segments (other than the last one) will be relatively stable? Kinda. Maybe. Maybe not. It depends on how you define "other than the last one". The key is that the internal doc IDs may change when segments are merged. And old segments get merged. Doc IDs w

Re: building custom cache - using lucene docids

2013-11-23 Thread Roman Chyla
Hi Erick, Many thanks for the info. An additional question: Do i understand you correctly that when two segmets get merged, the docids (of the original segments) remain the same? (unless, perhaps in situation, they were merged using the last index segment which was opened for writing and where th

Re: building custom cache - using lucene docids

2013-11-24 Thread Erick Erickson
bq: Do i understand you correctly that when two segmets get merged, the docids (of the original segments) remain the same? The original segments are unchanged, segments are _never_ changed after they're closed. But they'll be thrown away. Say you have segment1 and segment2 that get merged into seg

Re: building custom cache - using lucene docids

2013-11-24 Thread Jack Krupansky
- Jack Krupansky -Original Message- From: Erick Erickson Sent: Sunday, November 24, 2013 8:31 AM To: solr-user@lucene.apache.org Subject: Re: building custom cache - using lucene docids bq: Do i understand you correctly that when two segmets get merged, the docids (of the original segme

Re: building custom cache - using lucene docids

2013-11-24 Thread Mikhail Khludnev
Roman, I don't fully understand your question. After segment is flushed it's never changed, hence segment-local docids are always the same. Due to merge segment can gone, its' docs become new ones in another segment. This is true for 'global' (Solr-style) docnums, which can flip after merge is ha

Re: building custom cache - using lucene docids

2013-11-25 Thread Roman Chyla
On Sun, Nov 24, 2013 at 8:31 AM, Erick Erickson wrote: > bq: Do i understand you correctly that when two segmets get merged, the > docids > (of the original segments) remain the same? > > The original segments are unchanged, segments are _never_ changed after > they're closed. But they'll be throw

Re: building custom cache - using lucene docids

2013-11-25 Thread Roman Chyla
segments to understand what was going on there, thanks for any possible pointers roman > > -- Jack Krupansky > > -Original Message- From: Erick Erickson > Sent: Sunday, November 24, 2013 8:31 AM > To: solr-user@lucene.apache.org > Subject: Re: building custom

Re: building custom cache - using lucene docids

2013-11-25 Thread Roman Chyla
On Mon, Nov 25, 2013 at 12:54 AM, Mikhail Khludnev < mkhlud...@griddynamics.com> wrote: > Roman, > > I don't fully understand your question. After segment is flushed it's never > changed, hence segment-local docids are always the same. Due to merge > segment can gone, its' docs become new ones in

Re: building custom cache - using lucene docids

2013-11-25 Thread Roman Chyla
OK, I've spent some time reading the solr/lucene4x classes, and this is myunderstanding (feel free to correct me ;-)) DirectoryReader holds the opened segments -- each segment has its own reader, the BaseCompositeReader (or extended classes thereof) store the offsets per each segment; eg. [0, 5, 2