Re: [PATCH 00/14] introduce the BFQ-v0 I/O scheduler as an extra scheduler

2016-10-30 Thread Manuel Krause
Dear blk-mq maintainers, Since years now I use the BFQ disk IO scheduler by default, always fetching the newest release. Now a reality story of mine: For a clean BUG hunt, I was forced to leave out BFQ for a week recently. Result was an unusable experience with CFQ. Long time pauses of deskt

Re: [PATCH 00/14] introduce the BFQ-v0 I/O scheduler as an extra scheduler

2016-10-29 Thread Paolo Valente
> Il giorno 29 ott 2016, alle ore 16:12, Jens Axboe ha > scritto: > > On 10/28/2016 11:38 PM, Paolo Valente wrote: >> >>> Il giorno 26 ott 2016, alle ore 18:12, Jens Axboe ha >>> scritto: >>> >>> On 10/26/2016 10:04 AM, Paolo Valente wrote: > Il giorno 26 ott 2016, alle ore 17:32,

Re: [PATCH 00/14] introduce the BFQ-v0 I/O scheduler as an extra scheduler

2016-10-29 Thread Manuel Krause
Hey, people, don't you annoy yourselves all the time? The BFQ patches provide a useful alternative for the code called "legacy" by you, while you're not maintaining the base any more, and just about to invent something new, again. ?! When blk-mq has no scheduler -> work on it. When you want to

Re: [PATCH 00/14] introduce the BFQ-v0 I/O scheduler as an extra scheduler

2016-10-29 Thread Jens Axboe
On 10/28/2016 11:38 PM, Paolo Valente wrote: Il giorno 26 ott 2016, alle ore 18:12, Jens Axboe ha scritto: On 10/26/2016 10:04 AM, Paolo Valente wrote: Il giorno 26 ott 2016, alle ore 17:32, Jens Axboe ha scritto: On 10/26/2016 09:29 AM, Christoph Hellwig wrote: On Wed, Oct 26, 2016 at

Re: [PATCH 00/14] introduce the BFQ-v0 I/O scheduler as an extra scheduler

2016-10-29 Thread Bart Van Assche
On 10/28/16 22:38, Paolo Valente wrote: > Then, assuming that this solution may be of general interest, and that > BFQ benefits convinced you a little bit too, may I get significant > collaboration/help on implementing this infrastructure? If so, Jens > and all possibly interested parties, could w

Re: [PATCH 00/14] introduce the BFQ-v0 I/O scheduler as an extra scheduler

2016-10-28 Thread Paolo Valente
> Il giorno 26 ott 2016, alle ore 18:12, Jens Axboe ha > scritto: > > On 10/26/2016 10:04 AM, Paolo Valente wrote: >> >>> Il giorno 26 ott 2016, alle ore 17:32, Jens Axboe ha >>> scritto: >>> >>> On 10/26/2016 09:29 AM, Christoph Hellwig wrote: On Wed, Oct 26, 2016 at 05:13:07PM +0200,

Re: [PATCH 00/14] introduce the BFQ-v0 I/O scheduler as an extra scheduler

2016-10-28 Thread Linus Walleij
On Fri, Oct 28, 2016 at 5:29 PM, Christoph Hellwig wrote: > On Fri, Oct 28, 2016 at 11:32:21AM +0200, Linus Walleij wrote: >> So I'm not just complaining by the way, I'm trying to fix this. Also >> Bartlomiej from Samsung has done some stabs at switching MMC/SD >> to blk-mq. I just rebased my late

Re: [PATCH 00/14] introduce the BFQ-v0 I/O scheduler as an extra scheduler

2016-10-28 Thread Linus Walleij
On Fri, Oct 28, 2016 at 4:22 PM, Jens Axboe wrote: > On 10/28/2016 03:32 AM, Linus Walleij wrote: >> >> This is without using Bartlomiej's clever hack to pretend we have >> 2 elements in the HW queue though. His early tests indicate that >> it doesn't help much: the performance regression we see i

Re: [PATCH 00/14] introduce the BFQ-v0 I/O scheduler as an extra scheduler

2016-10-28 Thread Mark Brown
On Fri, Oct 28, 2016 at 06:05:35PM +0200, Arnd Bergmann wrote: > On Friday, October 28, 2016 9:30:07 AM CEST Jens Axboe wrote: > > Also, 4.8 and newer have support for BLK_MQ_F_BLOCKING, if you need to > > block in ->queue_rq(). That could eliminate the need to offload to a > > kthread manually.

Re: [PATCH 00/14] introduce the BFQ-v0 I/O scheduler as an extra scheduler

2016-10-28 Thread Mark Brown
On Fri, Oct 28, 2016 at 08:17:01AM -0600, Jens Axboe wrote: > On 10/28/2016 12:36 AM, Ulf Hansson wrote: > > You have been pushing Paolo in different directions throughout the > > years with his work in BFQ, wasting lots of his time/effort. > I have not. Various entities have advised Paolo approa

Re: [PATCH 00/14] introduce the BFQ-v0 I/O scheduler as an extra scheduler

2016-10-28 Thread Arnd Bergmann
On Friday, October 28, 2016 9:30:07 AM CEST Jens Axboe wrote: > On 10/28/2016 03:32 AM, Linus Walleij wrote: > > The patch to enable MQ looks like this: > > https://git.kernel.org/cgit/linux/kernel/git/linusw/linux-stericsson.git/commit/?h=mmc-mq&id=8f79b527e2e854071d8da019451da68d4753f71d > > BTW

Re: [PATCH 00/14] introduce the BFQ-v0 I/O scheduler as an extra scheduler

2016-10-28 Thread Bartlomiej Zolnierkiewicz
Hi, On Friday, October 28, 2016 09:30:07 AM Jens Axboe wrote: > On 10/28/2016 03:32 AM, Linus Walleij wrote: > > The patch to enable MQ looks like this: > > https://git.kernel.org/cgit/linux/kernel/git/linusw/linux-stericsson.git/commit/?h=mmc-mq&id=8f79b527e2e854071d8da019451da68d4753f71d > > B

Re: [PATCH 00/14] introduce the BFQ-v0 I/O scheduler as an extra scheduler

2016-10-28 Thread Jens Axboe
On 10/28/2016 03:32 AM, Linus Walleij wrote: The patch to enable MQ looks like this: https://git.kernel.org/cgit/linux/kernel/git/linusw/linux-stericsson.git/commit/?h=mmc-mq&id=8f79b527e2e854071d8da019451da68d4753f71d BTW, another viable "hack" for the depth issue would be to expose more than

Re: [PATCH 00/14] introduce the BFQ-v0 I/O scheduler as an extra scheduler

2016-10-28 Thread Christoph Hellwig
On Fri, Oct 28, 2016 at 11:32:21AM +0200, Linus Walleij wrote: > So I'm not just complaining by the way, I'm trying to fix this. Also > Bartlomiej from Samsung has done some stabs at switching MMC/SD > to blk-mq. I just rebased my latest stab at a naïve switch to blk-mq > to v4.9-rc2 with these res

Re: [PATCH 00/14] introduce the BFQ-v0 I/O scheduler as an extra scheduler

2016-10-28 Thread Jens Axboe
On 10/28/2016 03:32 AM, Linus Walleij wrote: On Fri, Oct 28, 2016 at 12:27 AM, Linus Walleij wrote: On Thu, Oct 27, 2016 at 11:08 PM, Jens Axboe wrote: blk-mq has evolved to support a variety of devices, there's nothing special about mmc that can't work well within that framework. There is

Re: [PATCH 00/14] introduce the BFQ-v0 I/O scheduler as an extra scheduler

2016-10-28 Thread Jens Axboe
On 10/28/2016 12:36 AM, Ulf Hansson wrote: [...] Moreover, I am still trying to understand what's the big deal to why you say no to BFQ as a legacy scheduler. Ideally it shouldn't cause you any maintenance burden and it doesn't make the removal of the legacy blk layer any more difficult, righ

Re: [PATCH 00/14] introduce the BFQ-v0 I/O scheduler as an extra scheduler

2016-10-28 Thread Jens Axboe
On 10/28/2016 01:59 AM, Jan Kara wrote: On Thu 27-10-16 10:26:18, Jens Axboe wrote: On 10/27/2016 03:26 AM, Jan Kara wrote: On Wed 26-10-16 10:12:38, Jens Axboe wrote: On 10/26/2016 10:04 AM, Paolo Valente wrote: Il giorno 26 ott 2016, alle ore 17:32, Jens Axboe ha scritto: On 10/26/2016

Re: [PATCH 00/14] introduce the BFQ-v0 I/O scheduler as an extra scheduler

2016-10-28 Thread Jens Axboe
On 10/27/2016 04:27 PM, Linus Walleij wrote: On Thu, Oct 27, 2016 at 11:08 PM, Jens Axboe wrote: blk-mq has evolved to support a variety of devices, there's nothing special about mmc that can't work well within that framework. There is. Read mmc_queue_thread() in drivers/mmc/card/queue.c Th

Re: [PATCH 00/14] introduce the BFQ-v0 I/O scheduler as an extra scheduler

2016-10-28 Thread Richard Weinberger
On Fri, Oct 28, 2016 at 2:07 PM, Arnd Bergmann wrote: >> > I don't think that's an accurate statement. In terms of coverage, most >> > drivers do support blk-mq. Anything SCSI, nvme, virtio-blk, SATA runs on >> > (or can run on) top of blk-mq. >> >> Well, I just used "git grep" and found that many

Re: [PATCH 00/14] introduce the BFQ-v0 I/O scheduler as an extra scheduler

2016-10-28 Thread Arnd Bergmann
On Thursday, October 27, 2016 8:13:08 PM CEST Ulf Hansson wrote: > On 27 October 2016 at 19:43, Jens Axboe wrote: > > On 10/27/2016 11:32 AM, Ulf Hansson wrote: > >> > >> [...] > >> > >>> > >>> I'm hesistant to add a new scheduler because it's very easy to add, very > >>> difficult to get rid of.

Re: [PATCH 00/14] introduce the BFQ-v0 I/O scheduler as an extra scheduler

2016-10-28 Thread Linus Walleij
On Fri, Oct 28, 2016 at 12:27 AM, Linus Walleij wrote: > On Thu, Oct 27, 2016 at 11:08 PM, Jens Axboe wrote: > >> blk-mq has evolved to support a variety of devices, there's nothing >> special about mmc that can't work well within that framework. > > There is. Read mmc_queue_thread() in drivers/m

Re: [PATCH 00/14] introduce the BFQ-v0 I/O scheduler as an extra scheduler

2016-10-28 Thread Jan Kara
On Thu 27-10-16 10:26:18, Jens Axboe wrote: > On 10/27/2016 03:26 AM, Jan Kara wrote: > >On Wed 26-10-16 10:12:38, Jens Axboe wrote: > >>On 10/26/2016 10:04 AM, Paolo Valente wrote: > >>> > Il giorno 26 ott 2016, alle ore 17:32, Jens Axboe ha > scritto: > > On 10/26/2016 09:29 AM

Re: [PATCH 00/14] introduce the BFQ-v0 I/O scheduler as an extra scheduler

2016-10-27 Thread Ulf Hansson
[...] > >> Moreover, I am still trying to understand what's the big deal to why >> you say no to BFQ as a legacy scheduler. Ideally it shouldn't cause >> you any maintenance burden and it doesn't make the removal of the >> legacy blk layer any more difficult, right? > > > Not sure I can state it m

Re: [PATCH 00/14] introduce the BFQ-v0 I/O scheduler as an extra scheduler

2016-10-27 Thread Linus Walleij
On Thu, Oct 27, 2016 at 11:08 PM, Jens Axboe wrote: > blk-mq has evolved to support a variety of devices, there's nothing > special about mmc that can't work well within that framework. There is. Read mmc_queue_thread() in drivers/mmc/card/queue.c This repeatedly calls req = blk_fetch_request(q

Re: [PATCH 00/14] introduce the BFQ-v0 I/O scheduler as an extra scheduler

2016-10-27 Thread Mark Brown
On Thu, Oct 27, 2016 at 12:45:48PM -0700, Christoph Hellwig wrote: > On Thu, Oct 27, 2016 at 08:41:27PM +0100, Mark Brown wrote: > > Plus the benchmarking to verify that it works well of course, especially > > initially where it'll also be a new queue infrastructure as well as the > > blk-mq conve

Re: [PATCH 00/14] introduce the BFQ-v0 I/O scheduler as an extra scheduler

2016-10-27 Thread Jens Axboe
On 10/27/2016 01:34 PM, Ulf Hansson wrote: [...] Instead, what I can tell, as we have been looking into converting mmc (which I maintains) and that is indeed a significant amount of work. We will need to rip out all of the mmc request management, and most likely we also need to extend the blkmq

Re: [PATCH 00/14] introduce the BFQ-v0 I/O scheduler as an extra scheduler

2016-10-27 Thread Christoph Hellwig
On Thu, Oct 27, 2016 at 08:41:27PM +0100, Mark Brown wrote: > Plus the benchmarking to verify that it works well of course, especially > initially where it'll also be a new queue infrastructure as well as the > blk-mq conversion itself. It does feel like something that's going to > take at least a

Re: [PATCH 00/14] introduce the BFQ-v0 I/O scheduler as an extra scheduler

2016-10-27 Thread Mark Brown
On Thu, Oct 27, 2016 at 12:21:06PM -0600, Jens Axboe wrote: > On 10/27/2016 12:13 PM, Ulf Hansson wrote: > > I can imagine, that it's not always a straight forward "convert to blk > > mq" patch for every block device driver. > Well, I've actually done a few conversions, and it's not difficult at

Re: [PATCH 00/14] introduce the BFQ-v0 I/O scheduler as an extra scheduler

2016-10-27 Thread Ulf Hansson
[...] >> Instead, what I can tell, as we have been looking into converting mmc >> (which I maintains) and that is indeed a significant amount of work. >> We will need to rip out all of the mmc request management, and most >> likely we also need to extend the blkmq interface - as to be able to >> d

Re: [PATCH 00/14] introduce the BFQ-v0 I/O scheduler as an extra scheduler

2016-10-27 Thread Jens Axboe
On 10/27/2016 12:13 PM, Ulf Hansson wrote: 2) While we work on evolving blkmq and convert block device drivers to it, BFQ could as a separate legacy scheduler, help *lots* of Linux users to get a significant improved experience. Should we really prevent them from that? I think you block maintaine

Re: [PATCH 00/14] introduce the BFQ-v0 I/O scheduler as an extra scheduler

2016-10-27 Thread Ulf Hansson
On 27 October 2016 at 19:43, Jens Axboe wrote: > On 10/27/2016 11:32 AM, Ulf Hansson wrote: >> >> [...] >> >>> >>> I'm hesistant to add a new scheduler because it's very easy to add, very >>> difficult to get rid of. If we do add BFQ as a legacy scheduler now, >>> it'll take us years and years to

Re: [PATCH 00/14] introduce the BFQ-v0 I/O scheduler as an extra scheduler

2016-10-27 Thread Jens Axboe
On 10/27/2016 11:32 AM, Ulf Hansson wrote: [...] I'm hesistant to add a new scheduler because it's very easy to add, very difficult to get rid of. If we do add BFQ as a legacy scheduler now, it'll take us years and years to get rid of it again. We should be moving towards LESS moving parts in

Re: [PATCH 00/14] introduce the BFQ-v0 I/O scheduler as an extra scheduler

2016-10-27 Thread Ulf Hansson
[...] > > I'm hesistant to add a new scheduler because it's very easy to add, very > difficult to get rid of. If we do add BFQ as a legacy scheduler now, > it'll take us years and years to get rid of it again. We should be > moving towards LESS moving parts in the legacy path, not more. Jens, I t

Re: [PATCH 00/14] introduce the BFQ-v0 I/O scheduler as an extra scheduler

2016-10-27 Thread Jens Axboe
On 10/27/2016 03:26 AM, Jan Kara wrote: On Wed 26-10-16 10:12:38, Jens Axboe wrote: On 10/26/2016 10:04 AM, Paolo Valente wrote: Il giorno 26 ott 2016, alle ore 17:32, Jens Axboe ha scritto: On 10/26/2016 09:29 AM, Christoph Hellwig wrote: On Wed, Oct 26, 2016 at 05:13:07PM +0200, Arnd Ber

Re: [PATCH 00/14] introduce the BFQ-v0 I/O scheduler as an extra scheduler

2016-10-27 Thread Jens Axboe
On 10/27/2016 08:34 AM, Grozdan wrote: On Thu, Oct 27, 2016 at 11:26 AM, Jan Kara wrote: On Wed 26-10-16 10:12:38, Jens Axboe wrote: On 10/26/2016 10:04 AM, Paolo Valente wrote: Il giorno 26 ott 2016, alle ore 17:32, Jens Axboe ha scritto: On 10/26/2016 09:29 AM, Christoph Hellwig wrote:

Re: [PATCH 00/14] introduce the BFQ-v0 I/O scheduler as an extra scheduler

2016-10-27 Thread Heinz Diehl
On 27.10.2016, Grozdan wrote: > So in the end, I'm here to support the inclusion of BFQ. Paolo has put > too much energy, time, and sleepless nights into this so people like > me can have a working, responsive system during heavy disk operations. > From a normal user's perspective, I do not want

Re: [PATCH 00/14] introduce the BFQ-v0 I/O scheduler as an extra scheduler

2016-10-27 Thread Grozdan
On Thu, Oct 27, 2016 at 11:26 AM, Jan Kara wrote: > On Wed 26-10-16 10:12:38, Jens Axboe wrote: >> On 10/26/2016 10:04 AM, Paolo Valente wrote: >> > >> >>Il giorno 26 ott 2016, alle ore 17:32, Jens Axboe ha >> >>scritto: >> >> >> >>On 10/26/2016 09:29 AM, Christoph Hellwig wrote: >> >>>On Wed, O

Re: [PATCH 00/14] introduce the BFQ-v0 I/O scheduler as an extra scheduler

2016-10-27 Thread Jan Kara
On Wed 26-10-16 10:12:38, Jens Axboe wrote: > On 10/26/2016 10:04 AM, Paolo Valente wrote: > > > >>Il giorno 26 ott 2016, alle ore 17:32, Jens Axboe ha > >>scritto: > >> > >>On 10/26/2016 09:29 AM, Christoph Hellwig wrote: > >>>On Wed, Oct 26, 2016 at 05:13:07PM +0200, Arnd Bergmann wrote: >

Re: [PATCH 00/14] introduce the BFQ-v0 I/O scheduler as an extra scheduler

2016-10-26 Thread Jens Axboe
On 10/26/2016 10:04 AM, Paolo Valente wrote: Il giorno 26 ott 2016, alle ore 17:32, Jens Axboe ha scritto: On 10/26/2016 09:29 AM, Christoph Hellwig wrote: On Wed, Oct 26, 2016 at 05:13:07PM +0200, Arnd Bergmann wrote: The question to ask first is whether to actually have pluggable schedule

Re: [PATCH 00/14] introduce the BFQ-v0 I/O scheduler as an extra scheduler

2016-10-26 Thread Paolo Valente
> Il giorno 26 ott 2016, alle ore 17:32, Jens Axboe ha > scritto: > > On 10/26/2016 09:29 AM, Christoph Hellwig wrote: >> On Wed, Oct 26, 2016 at 05:13:07PM +0200, Arnd Bergmann wrote: >>> The question to ask first is whether to actually have pluggable >>> schedulers on blk-mq at all, or just h

Re: [PATCH 00/14] introduce the BFQ-v0 I/O scheduler as an extra scheduler

2016-10-26 Thread Jens Axboe
On 10/26/2016 09:29 AM, Christoph Hellwig wrote: On Wed, Oct 26, 2016 at 05:13:07PM +0200, Arnd Bergmann wrote: The question to ask first is whether to actually have pluggable schedulers on blk-mq at all, or just have one that is meant to do the right thing in every case (and possibly can be byp

Re: [PATCH 00/14] introduce the BFQ-v0 I/O scheduler as an extra scheduler

2016-10-26 Thread Christoph Hellwig
On Wed, Oct 26, 2016 at 05:13:07PM +0200, Arnd Bergmann wrote: > The question to ask first is whether to actually have pluggable > schedulers on blk-mq at all, or just have one that is meant to > do the right thing in every case (and possibly can be bypassed > completely). That would be my prefere

Re: [PATCH 00/14] introduce the BFQ-v0 I/O scheduler as an extra scheduler

2016-10-26 Thread Arnd Bergmann
On Wednesday, October 26, 2016 8:05:11 AM CEST Bart Van Assche wrote: > On 10/26/2016 04:34 AM, Jan Kara wrote: > > On Wed 26-10-16 03:19:03, Christoph Hellwig wrote: > >> Just as last time: > >> > >> big NAK for introducing giant new infrastructure like a new I/O scheduler > >> for the legacy requ

Re: [PATCH 00/14] introduce the BFQ-v0 I/O scheduler as an extra scheduler

2016-10-26 Thread Bart Van Assche
On 10/26/2016 04:34 AM, Jan Kara wrote: On Wed 26-10-16 03:19:03, Christoph Hellwig wrote: Just as last time: big NAK for introducing giant new infrastructure like a new I/O scheduler for the legacy request structure. Please direct your engergy towards blk-mq instead. Christoph, we will prob

Re: [PATCH 00/14] introduce the BFQ-v0 I/O scheduler as an extra scheduler

2016-10-26 Thread Paolo Valente
> Il giorno 26 ott 2016, alle ore 12:19, Christoph Hellwig > ha scritto: > > Just as last time: > > big NAK for introducing giant new infrastructure like a new I/O scheduler > for the legacy request structure. > I would fully agree, if there weren't important problems involved. But there are

Re: [PATCH 00/14] introduce the BFQ-v0 I/O scheduler as an extra scheduler

2016-10-26 Thread Jan Kara
On Wed 26-10-16 03:19:03, Christoph Hellwig wrote: > Just as last time: > > big NAK for introducing giant new infrastructure like a new I/O scheduler > for the legacy request structure. > > Please direct your engergy towards blk-mq instead. Christoph, we will probably talk about this next week b

Re: [PATCH 00/14] introduce the BFQ-v0 I/O scheduler as an extra scheduler

2016-10-26 Thread Christoph Hellwig
Just as last time: big NAK for introducing giant new infrastructure like a new I/O scheduler for the legacy request structure. Please direct your engergy towards blk-mq instead.