Re: Block device throttling [Re: Distributed storage.]

2007-08-14 Thread Daniel Phillips
On Tuesday 14 August 2007 05:46, Evgeniy Polyakov wrote: > > The throttling of the virtual device must begin in > > generic_make_request and last to ->endio. You release the throttle > > of the virtual device at the point you remap the bio to an > > underlying device, which you have convinced your

Re: Block device throttling [Re: Distributed storage.]

2007-08-14 Thread Evgeniy Polyakov
On Tue, Aug 14, 2007 at 05:32:29AM -0700, Daniel Phillips ([EMAIL PROTECTED]) wrote: > On Tuesday 14 August 2007 04:50, Evgeniy Polyakov wrote: > > On Tue, Aug 14, 2007 at 04:35:43AM -0700, Daniel Phillips > ([EMAIL PROTECTED]) wrote: > > > On Tuesday 14 August 2007 04:30, Evgeniy Polyakov wrote:

Re: Block device throttling [Re: Distributed storage.]

2007-08-14 Thread Daniel Phillips
On Tuesday 14 August 2007 04:50, Evgeniy Polyakov wrote: > On Tue, Aug 14, 2007 at 04:35:43AM -0700, Daniel Phillips ([EMAIL PROTECTED]) wrote: > > On Tuesday 14 August 2007 04:30, Evgeniy Polyakov wrote: > > > > And it will not solve the deadlock problem in general. (Maybe > > > > it works for y

Re: Block device throttling [Re: Distributed storage.]

2007-08-14 Thread Evgeniy Polyakov
On Tue, Aug 14, 2007 at 04:35:43AM -0700, Daniel Phillips ([EMAIL PROTECTED]) wrote: > On Tuesday 14 August 2007 04:30, Evgeniy Polyakov wrote: > > > And it will not solve the deadlock problem in general. (Maybe it > > > works for your virtual device, but I wonder...) If the virtual > > > device

Re: Block device throttling [Re: Distributed storage.]

2007-08-14 Thread Daniel Phillips
On Tuesday 14 August 2007 04:30, Evgeniy Polyakov wrote: > > And it will not solve the deadlock problem in general. (Maybe it > > works for your virtual device, but I wonder...) If the virtual > > device allocates memory during generic_make_request then the memory > > needs to be throttled. > > D

Re: Block device throttling [Re: Distributed storage.]

2007-08-14 Thread Evgeniy Polyakov
On Tue, Aug 14, 2007 at 04:13:10AM -0700, Daniel Phillips ([EMAIL PROTECTED]) wrote: > On Tuesday 14 August 2007 01:46, Evgeniy Polyakov wrote: > > On Mon, Aug 13, 2007 at 06:04:06AM -0700, Daniel Phillips > ([EMAIL PROTECTED]) wrote: > > > Perhaps you never worried about the resources that the d

Re: Block device throttling [Re: Distributed storage.]

2007-08-14 Thread Daniel Phillips
On Tuesday 14 August 2007 01:46, Evgeniy Polyakov wrote: > On Mon, Aug 13, 2007 at 06:04:06AM -0700, Daniel Phillips ([EMAIL PROTECTED]) wrote: > > Perhaps you never worried about the resources that the device > > mapper mapping function allocates to handle each bio and so did not > > consider thi

Re: Block device throttling [Re: Distributed storage.]

2007-08-14 Thread Evgeniy Polyakov
On Mon, Aug 13, 2007 at 06:04:06AM -0700, Daniel Phillips ([EMAIL PROTECTED]) wrote: > Perhaps you never worried about the resources that the device mapper > mapping function allocates to handle each bio and so did not consider > this hole significant. These resources can be significant, as is

Re: Block device throttling [Re: Distributed storage.]

2007-08-13 Thread Daniel Phillips
On Monday 13 August 2007 05:18, Evgeniy Polyakov wrote: > > Say you have a device mapper device with some physical device > > sitting underneath, the classic use case for this throttle code. > > Say 8,000 threads each submit an IO in parallel. The device mapper > > mapping function will be called

Re: Block device throttling [Re: Distributed storage.]

2007-08-13 Thread Evgeniy Polyakov
On Mon, Aug 13, 2007 at 05:18:14AM -0700, Daniel Phillips ([EMAIL PROTECTED]) wrote: > > If limit is for > > 1gb of pending block io, and system has for example 2gbs of ram (or > > any other resonable parameters), then there is no way we can deadlock > > in allocation, since it will not force pag

Re: Block device throttling [Re: Distributed storage.]

2007-08-13 Thread Daniel Phillips
On Monday 13 August 2007 05:04, Evgeniy Polyakov wrote: > On Mon, Aug 13, 2007 at 04:04:26AM -0700, Daniel Phillips ([EMAIL PROTECTED]) wrote: > > On Monday 13 August 2007 01:14, Evgeniy Polyakov wrote: > > > > Oops, and there is also: > > > > > > > > 3) The bio throttle, which is supposed to prev

Re: Block device throttling [Re: Distributed storage.]

2007-08-13 Thread Evgeniy Polyakov
On Mon, Aug 13, 2007 at 04:18:03AM -0700, Daniel Phillips ([EMAIL PROTECTED]) wrote: > > No. Since all requests for virtual device end up in physical devices, > > which have limits, this mechanism works. Virtual device will > > essentially call either generic_make_request() for new physical > > de

Re: Block device throttling [Re: Distributed storage.]

2007-08-13 Thread Evgeniy Polyakov
On Mon, Aug 13, 2007 at 04:04:26AM -0700, Daniel Phillips ([EMAIL PROTECTED]) wrote: > On Monday 13 August 2007 01:14, Evgeniy Polyakov wrote: > > > Oops, and there is also: > > > > > > 3) The bio throttle, which is supposed to prevent deadlock, can > > > itself deadlock. Let me see if I can reme

Re: Block device throttling [Re: Distributed storage.]

2007-08-13 Thread Daniel Phillips
On Monday 13 August 2007 01:23, Evgeniy Polyakov wrote: > On Sun, Aug 12, 2007 at 10:36:23PM -0700, Daniel Phillips ([EMAIL PROTECTED]) wrote: > > (previous incomplete message sent accidentally) > > > > On Wednesday 08 August 2007 02:54, Evgeniy Polyakov wrote: > > > On Tue, Aug 07, 2007 at 10:55:

Re: Block device throttling [Re: Distributed storage.]

2007-08-13 Thread Daniel Phillips
On Monday 13 August 2007 01:14, Evgeniy Polyakov wrote: > > Oops, and there is also: > > > > 3) The bio throttle, which is supposed to prevent deadlock, can > > itself deadlock. Let me see if I can remember how it goes. > > > > * generic_make_request puts a bio in flight > > * the bio gets pas

Re: Block device throttling [Re: Distributed storage.]

2007-08-13 Thread Evgeniy Polyakov
On Sun, Aug 12, 2007 at 10:36:23PM -0700, Daniel Phillips ([EMAIL PROTECTED]) wrote: > (previous incomplete message sent accidentally) > > On Wednesday 08 August 2007 02:54, Evgeniy Polyakov wrote: > > On Tue, Aug 07, 2007 at 10:55:38PM +0200, Jens Axboe wrote: > > > > So, what did we decide? To

Re: Block device throttling [Re: Distributed storage.]

2007-08-13 Thread Evgeniy Polyakov
On Sun, Aug 12, 2007 at 11:44:00PM -0700, Daniel Phillips ([EMAIL PROTECTED]) wrote: > On Sunday 12 August 2007 22:36, I wrote: > > Note! There are two more issues I forgot to mention earlier. > > Oops, and there is also: > > 3) The bio throttle, which is supposed to prevent deadlock, can itsel

Re: Block device throttling [Re: Distributed storage.]

2007-08-13 Thread Daniel Phillips
On Sunday 12 August 2007 22:36, I wrote: > Note! There are two more issues I forgot to mention earlier. Oops, and there is also: 3) The bio throttle, which is supposed to prevent deadlock, can itself deadlock. Let me see if I can remember how it goes. * generic_make_request puts a bio in fl

Re: Block device throttling [Re: Distributed storage.]

2007-08-12 Thread Daniel Phillips
(previous incomplete message sent accidentally) On Wednesday 08 August 2007 02:54, Evgeniy Polyakov wrote: > On Tue, Aug 07, 2007 at 10:55:38PM +0200, Jens Axboe wrote: > > So, what did we decide? To bloat bio a bit (add a queue pointer) or > to use physical device limits? The latter requires to r

Re: Block device throttling [Re: Distributed storage.]

2007-08-12 Thread Daniel Phillips
On Wednesday 08 August 2007 02:54, Evgeniy Polyakov wrote: > On Tue, Aug 07, 2007 at 10:55:38PM +0200, Jens Axboe ([EMAIL PROTECTED]) wrote: > > So, what did we decide? To bloat bio a bit (add a queue pointer) or > to use physical device limits? The latter requires to replace all > occurence of bi