We just need an Executor-based MS, then we can throw all other out of the window, as threading concerns are now resolved by a proper choice of Executor supplied to constructor. Also an application has much more control over threading in multiple-index situations, as single Executor can be reused for multiple MSs.
On Fri, May 28, 2010 at 18:36, Michael McCandless <luc...@mikemccandless.com> wrote: > Hmm... so I think the questions really are "how many merges are > allowed to run concurrently?" (SMS is 1 and CMS is N), and "do I spawn > my own threads for merging or do I steal the app's threads" (SMS > steals app threads and CMS spawns new ones). > > Of course if you steal app threads you can only make use of as much > concurrency as the app's threads... > > Both SMS and CMS will allow other indexing ops to proceed, to a point, > but if the other indexing ops spawn too many merges, then those > threads will be blocked by both SMS and CMS. > > So I'm not sure blocking/non-blocking is a good first split -- even > SMS isn't blocking other app indexing threads. > > Mike > > On Thu, May 27, 2010 at 3:58 AM, Shai Erera <ser...@gmail.com> wrote: >> Hi >> >> I've been thinking recently why are these two named like they are ... with a >> MS we're basically asking two questions: (1) should it block other merges >> from happening (or app thread from continuing) and (2) should it do its >> merges concurrently? >> >> SMS answers 'true' to (1) and 'false' to (2), while CMS answers the >> opposite. >> >> BUT, there's really no reason why these two are coupled. E.g. someone who >> wants to block other merges from running, or hold the app thread until >> merges are finished, does not necessarily want the merges to run in >> sequence. Those are two different decisions. Even if you want to block the >> application thread and other merges, you can still benefit form having the >> merges run concurrently. >> >> So, I was thinking that what we really want is a BlockingMS and >> NonBlockingMS that can be used according to the answer you look for in (1), >> and then we can have variants for both that execute the merges concurrently >> or not. I think that serial merging should be supported w/ BlockingMS only, >> but am interested in other opinions. One of the scenarios for serial merging >> is if the application wants to ensure no additional threads are spawned >> other than what it decided to spawn, and therefore it can only be used w/ >> the BlockingMS.Another scenario is to control IO, but in this case a >> NonBlockingSerialMS may fit as well (depends if you think other merges may >> start while this one is running). >> >> In fact, w/o changing much, we can have CMS optionally block other merges / >> app thread by waiting until all merges are finished. We may even stick w/ >> the current SMS/CMS names, just documenting that CMS can be used to block >> threads, only merges will be done concurrently. >> >> What do you think? >> >> Shai >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > > -- Kirill Zakharenko/Кирилл Захаренко (ear...@gmail.com) Phone: +7 (495) 683-567-4 ICQ: 104465785 --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
