Re: Throttling merges
Yep, pretty much like ionice for Linux. At the Directory level makes the most sense with the context passed in. then somehow add throttling in there. Yeah I'm not sure how exactly that will work yet. On Sat, Jul 18, 2009 at 9:12 AM, Michael McCandlessluc...@mikemccandless.com wrote: The goal is to be like ionice right? Meaning, lower the priority of IO caused by merging? I agree that makes sense. I wonder if we could implement it in the Directory level, so that when openInput/createOutput is called we can optionally specify the context (reader, merging, writer, etc.), and then somehow add throttling in there. Mike On Sat, Jul 18, 2009 at 10:37 AM, Jason Rutherglenjason.rutherg...@gmail.com wrote: It may be useful to allow users to throttle merges. A callback that IW passes into SegmentMerger would suffice where individual SM methods make use of the callback. I suppose this could slow down overall merging by adding a potentially useless method call. However if merging typically consumes IO resources for an extended period of time, this offers a way for the user to tune IO consumption and at preferred times free up IO for other tasks. - To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org For additional commands, e-mail: java-dev-h...@lucene.apache.org - To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org For additional commands, e-mail: java-dev-h...@lucene.apache.org - To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org For additional commands, e-mail: java-dev-h...@lucene.apache.org
Throttling merges
It may be useful to allow users to throttle merges. A callback that IW passes into SegmentMerger would suffice where individual SM methods make use of the callback. I suppose this could slow down overall merging by adding a potentially useless method call. However if merging typically consumes IO resources for an extended period of time, this offers a way for the user to tune IO consumption and at preferred times free up IO for other tasks. - To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org For additional commands, e-mail: java-dev-h...@lucene.apache.org
Re: Throttling merges
The goal is to be like ionice right? Meaning, lower the priority of IO caused by merging? I agree that makes sense. I wonder if we could implement it in the Directory level, so that when openInput/createOutput is called we can optionally specify the context (reader, merging, writer, etc.), and then somehow add throttling in there. Mike On Sat, Jul 18, 2009 at 10:37 AM, Jason Rutherglenjason.rutherg...@gmail.com wrote: It may be useful to allow users to throttle merges. A callback that IW passes into SegmentMerger would suffice where individual SM methods make use of the callback. I suppose this could slow down overall merging by adding a potentially useless method call. However if merging typically consumes IO resources for an extended period of time, this offers a way for the user to tune IO consumption and at preferred times free up IO for other tasks. - To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org For additional commands, e-mail: java-dev-h...@lucene.apache.org - To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org For additional commands, e-mail: java-dev-h...@lucene.apache.org
