> EG you could imagine an addIndexes* call getting started, > completing a few merges. Then, concurrently, CMS picks & chooses > some of those added external segments to merge with some of the > original segments. Then addIndexes hits an exception. What do we > do?
An exception due to an IO error from the external dir? We can abandon the merge and remove the external segments that failed from segmentInfos? > If IndexWriter maintained "branches" of the segmentInfos we > could actually rollback all changes, ie, remove what was done by > addIndexes but retroactively preserve any segments created by > other methods (flushing, other addIndexes calls, etc.). Branches stored in a TreeMap? With the keys being the method+randomId that initiated them? (i.e. addIndexes12) On Tue, Jul 14, 2009 at 12:56 PM, Michael McCandless<luc...@mikemccandless.com> wrote: > I agree, there's no real reason why addIndexes can run concurrently > with other things. It's just software ;) > > One challenge is the transactional guarantee that addIndexes provide, > ie, it's all or none. If there's an exception while adding, then > nothing was added. > > But, that was added before autoCommit=false. So perhaps we could > relax that and expect the app to instead rely on the "global" > transactional semantics provided by autoCommit=false. > > EG you could imagine an addIndexes* call getting started, completing a > few merges. Then, concurrently, CMS picks & chooses some of those > added external segments to merge with some of the original segments. > Then addIndexes hits an exception. What do we do? > > If IndexWriter maintained "branches" of the segmentInfos we could > actually rollback all changes, ie, remove what was done by addIndexes > but retroactively preserve any segments created by other methods > (flushing, other addIndexes calls, etc.). > > Mike > > On Tue, Jul 14, 2009 at 3:08 PM, Jason > Rutherglen<jason.rutherg...@gmail.com> wrote: >> For replicating and general system performance, it would be good to offer a >> way to addIndexes* without blocking the addition of more docs. This seems >> doable somehow? > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-dev-h...@lucene.apache.org > > --------------------------------------------------------------------- To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org For additional commands, e-mail: java-dev-h...@lucene.apache.org
