On Tue, Jul 21, 2009 at 7:26 PM, Jason Rutherglen<[email protected]> wrote: >> EG you could imagine an addIndexes* call getting started, >> completing a few merges. Then, concurrently, CMS picks & chooses >> some of those added external segments to merge with some of the >> original segments. Then addIndexes hits an exception. What do we >> do? > > An exception due to an IO error from the external dir?
Or OOME or a Lucene bug or something. > We can > abandon the merge and remove the external segments that failed > from segmentInfos? But that won't work in this case because some of the external segments have been merged with "real" segments in the index. >> If IndexWriter maintained "branches" of the segmentInfos we >> could actually rollback all changes, ie, remove what was done by >> addIndexes but retroactively preserve any segments created by >> other methods (flushing, other addIndexes calls, etc.). > > Branches stored in a TreeMap? With the keys being the > method+randomId that initiated them? (i.e. addIndexes12) Actually I meant keeping track of the "genealogy" of how the merges were done, plus holding an extra refCount against "real" segments that had been merged with external-but-not-yet-committed segments. So that if the addIndexes fails, yet we had merged [say] "real" segments 1, 2, 3 with "external" segments 4 and 5, we could on failure of addIndexes go "undo" that merge and put back segments 1, 2, 3. Mike --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
