On Tue, Mar 31, 2015 at 10:27:26AM -0400, Rik van Riel wrote: > CPUs with nohz_full do not want disruption from timer interrupts, > or other random system things. This includes block mq work. > > There is another issue with block mq vs. realtime tasks that run > 100% of the time, which is not uncommon on systems that have CPUs > dedicated to real time use with isolcpus= and nohz_full= > > Specifically, on systems like that, a block work item may never > get to run, which could lead to filesystems getting stuck forever. > > We can avoid both issues by not scheduling blk-mq workqueues on > cpus in nohz_full mode. > > Question for Jens: should we try to spread out the load for > currently offline and nohz CPUs across the remaining CPUs in > the system, to get the full benefit of blk-mq in these situations? > > If so, do you have any preference on how I should implement that? > > Cc: Frederic Weisbecker <fweis...@redhat.com> > Cc: Ingo Molnar <mi...@kernel.org> > Cc: Jens Axboe <ax...@kernel.org> > Signed-off-by: Rik van Riel <r...@redhat.com> > --- > block/blk-mq.c | 5 +++++ > 1 file changed, 5 insertions(+) > > diff --git a/block/blk-mq.c b/block/blk-mq.c > index 4f4bea21052e..1004d6817fa4 100644 > --- a/block/blk-mq.c > +++ b/block/blk-mq.c > @@ -21,6 +21,7 @@ > #include <linux/sched/sysctl.h> > #include <linux/delay.h> > #include <linux/crash_dump.h> > +#include <linux/tick.h> > > #include <trace/events/block.h> > > @@ -1760,6 +1761,10 @@ static void blk_mq_init_cpu_queues(struct > request_queue *q, > if (!cpu_online(i)) > continue; > > + /* Do not schedule work on nohz full dedicated CPUs. */ > + if (tick_nohz_full_cpu(i)) > + continue; > + > hctx = q->mq_ops->map_queue(q, i); > cpumask_set_cpu(i, hctx->cpumask); > hctx->nr_ctx++;
Rik, I suppose any bound workqueue queued on isolated CPUs should be moved at queue time to other CPUs (sacrifficing performance). So that by doing "queue_work" on an isolated CPU would move that work somewhere else. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
