I think that when LUCENE-1750 is finished, you will be able to: 1) Create a MergePolicy that limits the segments size it's about to merge to a certain size. 2) Then have a daemon or something that runs on "idle" times and call optimize(maxNumSegments), or even open a new writer w/ the default merge policy and allow it to merge?
Shai On Thu, Jul 30, 2009 at 5:48 PM, Grant Ingersoll <gsing...@apache.org>wrote: > Note also response from Mike that talks a little bit about something along > these lines: > http://www.lucidimagination.com/search/document/fa990adba4d2572b/is_there_a_way_to_control_when_merges_happen#f6f0bfeef4bf9a39 > > -Grant > > > On Jul 30, 2009, at 10:35 AM, Grant Ingersoll wrote: > > Given a large segment and a bunch of small segments, how does the >> ConcurrentMergeScheduler (CMS) work? Does it always merge the smaller >> segments into the bigger one, or does it merge the smaller segments >> together? >> >> Something I've been thinking about: Given a high update environment (and >> near real time, less than 1 minute, search constraints) and/or a very bursty >> environment, we've always said to keep the merge factor small for search >> reasons, at least in the high-update case. However, I've seen a couple of >> times where this causes problems because merges can take over and cause >> pauses, even with CMS, so I am wonder if it makes sense to have a larger >> merge factor (>10), knowing that I may have a few large segments and then a >> bunch of small ones and that the CMS will, in the background, be able to >> keep merging the smaller segments together and in most cases avoid ever >> having to merge into the large segments (b/c maybe I can just optimize down >> at slower times or even merge larger segments later. ) Seems like this >> would allow one to make sure larger merges need not take place, or at least >> reduce the chances of that happening. >> >> Not sure if I worded that correctly. >> >> Thanks, >> Grant >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org >> For additional commands, e-mail: java-dev-h...@lucene.apache.org >> >> > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-dev-h...@lucene.apache.org > >
