Re: Throttling merges

2009-07-20 Thread Jason Rutherglen
Yep, pretty much like "ionice" for Linux. At the Directory level makes the most sense with the context passed in. > then somehow add throttling in there. Yeah I'm not sure how exactly that will work yet. On Sat, Jul 18, 2009 at 9:12 AM, Michael McCandless wrote: > The goal is to be like "ionic

Re: Throttling merges

2009-07-18 Thread Michael McCandless
The goal is to be like "ionice" right? Meaning, lower the priority of IO caused by merging? I agree that makes sense. I wonder if we could implement it in the Directory level, so that when openInput/createOutput is called we can optionally specify the "context" (reader, merging, writer, etc.), a

Throttling merges

2009-07-18 Thread Jason Rutherglen
It may be useful to allow users to throttle merges. A callback that IW passes into SegmentMerger would suffice where individual SM methods make use of the callback. I suppose this could slow down overall merging by adding a potentially useless method call. However if merging typically consumes IO r