On Fri, Nov 20, 2009 at 12:35 PM, Yonik Seeley <yo...@lucidimagination.com> wrote: > On Fri, Nov 20, 2009 at 12:24 PM, Michael <solrco...@gmail.com> wrote: >> So -- I thought I understood you to mean that if I frequently merge, >> it's basically the same as an optimize, and cruft will get purged. Am >> I misunderstanding you? > > That only applies to the segments involved in the merge. The deleted > documents are left behind when old segments are merged into a new > segment.
Your statement is leading me to believe that I have misunderstood the merge process. I thought that every time there are 10 segments, they get merged down to 1. Therefore, every time a merge happens, every single segment in my entire index is "involved in the merge". 9 segments later, we're back to 10 segments, and they're merged into 1. 9 segments later, we're back to 10 segments once again, and they're merged into 1. Maybe I have misunderstood the mergeFactor docs. Maybe instead it's like this? 1. Segment A1 fills with N docs, and a new segment A2 is created. 2. A2 fills with N docs, and A3 is created; A3 fills with N docs, etc. 3. A9 fills with N docs, and merging occurs: Segment B1 is created with 10*N docs, segments A1-A9 are deleted. 4. A new segment A1 fills with N docs, and a new segment A2 is created; B1 is still sitting with 10*N docs. 5. Eventually A1 through A9 each have N docs, and then merging occurs: Segment B2 is created, with 10*N docs. 6. Eventually Segments B1 through B9 each have 10*N docs, and merging occurs: Segment C1 is created, with 100*N docs. Segments B1-B9 are deleted. 7. A new A1 starts filling again. Some time down the line I might have 4 D segments with 1000*N docs each, 6 C segments with 100*N docs each, 8 B segments with 10*N docs each, 2 A segments with N docs each, and an open A3 segment filling up. If this is right, then your statement above means that yes, each merge of many As into 1 B purges all the deleted docs in A1-A9, but All my Ds, Cs, and Bs aren't updated to purge deleted docs yet. Only when B1-B9 merge into a new C do their deleted docs get purged; only when C1-C9 merge into a new D do their deleted docs get purged; etc. Is this right? Sorry it was so verbose! Michael
