Re: [PATCH 3/4] workqueue: Create low-level unbound workqueues cpumask
Frederic Weisbecker writes: > On Mon, Mar 16, 2015 at 10:12:12AM -0700, Kevin Hilman wrote: >> Lai Jiangshan writes: >> >> > On 03/14/2015 07:49 AM, Kevin Hilman wrote: [...] >> >> >> >> As I mentioned in an earlier discussion[1], I still think this could >> >> default too the housekeeping CPUs in the NO_HZ_FULL case: >> >> >> >> #ifdef CONFIG_NO_HZ_FULL >> >> cpumask_complement(wq_unbound_cpumask, tick_nohz_full_mask); >> > >> > >> > No, the default/booted wq_unbound_cpumask should be cpu_possible_mask. >> > >> >> Even for NO_HZ_FULL? >> >> IMO, for NO_HZ_FULL, we want the unbound workqueues to be on the >> housekeeping CPU(s). > > If it should be the default on NO_HZ_FULL, maybe we should do this from the > tick nohz code. Some late or fs initcall that will do the workqueue affinity, > timer affinity, etc... Sure, I'd be fine with that too. Kevin -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 3/4] workqueue: Create low-level unbound workqueues cpumask
On Mon, Mar 16, 2015 at 10:12:12AM -0700, Kevin Hilman wrote: > Lai Jiangshan writes: > > > On 03/14/2015 07:49 AM, Kevin Hilman wrote: > >> Lai Jiangshan writes: > >> > >>> From: Frederic Weisbecker > >>> > >>> Create a cpumask that limit the affinity of all unbound workqueues. > >>> This cpumask is controlled though a file at the root of the workqueue > >>> sysfs directory. > >>> > >>> It works on a lower-level than the per WQ_SYSFS workqueues cpumask files > >>> such that the effective cpumask applied for a given unbound workqueue is > >>> the intersection of /sys/devices/virtual/workqueue/$WORKQUEUE/cpumask and > >>> the new /sys/devices/virtual/workqueue/cpumask_unbounds file. > >>> > >>> This patch implements the basic infrastructure and the read interface. > >>> cpumask_unbounds is initially set to cpu_possible_mask. > >>> > >>> Cc: Christoph Lameter > >>> Cc: Kevin Hilman > >>> Cc: Lai Jiangshan > >>> Cc: Mike Galbraith > >>> Cc: Paul E. McKenney > >>> Cc: Tejun Heo > >>> Cc: Viresh Kumar > >>> Signed-off-by: Frederic Weisbecker > >>> Signed-off-by: Lai Jiangshan > >> > >> [...] > >> > >>> @@ -5094,6 +5116,9 @@ static int __init init_workqueues(void) > >>> > >>> WARN_ON(__alignof__(struct pool_workqueue) < __alignof__(long long)); > >>> > >>> + BUG_ON(!alloc_cpumask_var(&wq_unbound_cpumask, GFP_KERNEL)); > >>> + cpumask_copy(wq_unbound_cpumask, cpu_possible_mask); > >>> + > >> > >> As I mentioned in an earlier discussion[1], I still think this could > >> default too the housekeeping CPUs in the NO_HZ_FULL case: > >> > >> #ifdef CONFIG_NO_HZ_FULL > >>cpumask_complement(wq_unbound_cpumask, tick_nohz_full_mask); > > > > > > No, the default/booted wq_unbound_cpumask should be cpu_possible_mask. > > > > Even for NO_HZ_FULL? > > IMO, for NO_HZ_FULL, we want the unbound workqueues to be on the > housekeeping CPU(s). If it should be the default on NO_HZ_FULL, maybe we should do this from the tick nohz code. Some late or fs initcall that will do the workqueue affinity, timer affinity, etc... -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 3/4] workqueue: Create low-level unbound workqueues cpumask
Lai Jiangshan writes: > On 03/14/2015 07:49 AM, Kevin Hilman wrote: >> Lai Jiangshan writes: >> >>> From: Frederic Weisbecker >>> >>> Create a cpumask that limit the affinity of all unbound workqueues. >>> This cpumask is controlled though a file at the root of the workqueue >>> sysfs directory. >>> >>> It works on a lower-level than the per WQ_SYSFS workqueues cpumask files >>> such that the effective cpumask applied for a given unbound workqueue is >>> the intersection of /sys/devices/virtual/workqueue/$WORKQUEUE/cpumask and >>> the new /sys/devices/virtual/workqueue/cpumask_unbounds file. >>> >>> This patch implements the basic infrastructure and the read interface. >>> cpumask_unbounds is initially set to cpu_possible_mask. >>> >>> Cc: Christoph Lameter >>> Cc: Kevin Hilman >>> Cc: Lai Jiangshan >>> Cc: Mike Galbraith >>> Cc: Paul E. McKenney >>> Cc: Tejun Heo >>> Cc: Viresh Kumar >>> Signed-off-by: Frederic Weisbecker >>> Signed-off-by: Lai Jiangshan >> >> [...] >> >>> @@ -5094,6 +5116,9 @@ static int __init init_workqueues(void) >>> >>> WARN_ON(__alignof__(struct pool_workqueue) < __alignof__(long long)); >>> >>> + BUG_ON(!alloc_cpumask_var(&wq_unbound_cpumask, GFP_KERNEL)); >>> + cpumask_copy(wq_unbound_cpumask, cpu_possible_mask); >>> + >> >> As I mentioned in an earlier discussion[1], I still think this could >> default too the housekeeping CPUs in the NO_HZ_FULL case: >> >> #ifdef CONFIG_NO_HZ_FULL >> cpumask_complement(wq_unbound_cpumask, tick_nohz_full_mask); > > > No, the default/booted wq_unbound_cpumask should be cpu_possible_mask. > Even for NO_HZ_FULL? IMO, for NO_HZ_FULL, we want the unbound workqueues to be on the housekeeping CPU(s). Kevin -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 3/4] workqueue: Create low-level unbound workqueues cpumask
On 03/14/2015 07:49 AM, Kevin Hilman wrote: > Lai Jiangshan writes: > >> From: Frederic Weisbecker >> >> Create a cpumask that limit the affinity of all unbound workqueues. >> This cpumask is controlled though a file at the root of the workqueue >> sysfs directory. >> >> It works on a lower-level than the per WQ_SYSFS workqueues cpumask files >> such that the effective cpumask applied for a given unbound workqueue is >> the intersection of /sys/devices/virtual/workqueue/$WORKQUEUE/cpumask and >> the new /sys/devices/virtual/workqueue/cpumask_unbounds file. >> >> This patch implements the basic infrastructure and the read interface. >> cpumask_unbounds is initially set to cpu_possible_mask. >> >> Cc: Christoph Lameter >> Cc: Kevin Hilman >> Cc: Lai Jiangshan >> Cc: Mike Galbraith >> Cc: Paul E. McKenney >> Cc: Tejun Heo >> Cc: Viresh Kumar >> Signed-off-by: Frederic Weisbecker >> Signed-off-by: Lai Jiangshan > > [...] > >> @@ -5094,6 +5116,9 @@ static int __init init_workqueues(void) >> >> WARN_ON(__alignof__(struct pool_workqueue) < __alignof__(long long)); >> >> +BUG_ON(!alloc_cpumask_var(&wq_unbound_cpumask, GFP_KERNEL)); >> +cpumask_copy(wq_unbound_cpumask, cpu_possible_mask); >> + > > As I mentioned in an earlier discussion[1], I still think this could > default too the housekeeping CPUs in the NO_HZ_FULL case: > > #ifdef CONFIG_NO_HZ_FULL > cpumask_complement(wq_unbound_cpumask, tick_nohz_full_mask); No, the default/booted wq_unbound_cpumask should be cpu_possible_mask. > #else > cpumask_copy(wq_unbound_cpumask, cpu_possible_mask); > #endif > > But that could also be left to a future optimization as well. > > Kevin > > [1] https://lkml.org/lkml/2014/2/14/666 > . > -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 3/4] workqueue: Create low-level unbound workqueues cpumask
Kevin Hilman writes: > Lai Jiangshan writes: > >> From: Frederic Weisbecker >> >> Create a cpumask that limit the affinity of all unbound workqueues. >> This cpumask is controlled though a file at the root of the workqueue >> sysfs directory. >> >> It works on a lower-level than the per WQ_SYSFS workqueues cpumask files >> such that the effective cpumask applied for a given unbound workqueue is >> the intersection of /sys/devices/virtual/workqueue/$WORKQUEUE/cpumask and >> the new /sys/devices/virtual/workqueue/cpumask_unbounds file. >> >> This patch implements the basic infrastructure and the read interface. >> cpumask_unbounds is initially set to cpu_possible_mask. >> >> Cc: Christoph Lameter >> Cc: Kevin Hilman >> Cc: Lai Jiangshan >> Cc: Mike Galbraith >> Cc: Paul E. McKenney >> Cc: Tejun Heo >> Cc: Viresh Kumar >> Signed-off-by: Frederic Weisbecker >> Signed-off-by: Lai Jiangshan > > [...] > >> @@ -5094,6 +5116,9 @@ static int __init init_workqueues(void) >> >> WARN_ON(__alignof__(struct pool_workqueue) < __alignof__(long long)); >> >> +BUG_ON(!alloc_cpumask_var(&wq_unbound_cpumask, GFP_KERNEL)); >> +cpumask_copy(wq_unbound_cpumask, cpu_possible_mask); >> + > > As I mentioned in an earlier discussion[1], I still think this could > default too the housekeeping CPUs in the NO_HZ_FULL case: > > #ifdef CONFIG_NO_HZ_FULL > cpumask_complement(wq_unbound_cpumask, tick_nohz_full_mask); > #else > cpumask_copy(wq_unbound_cpumask, cpu_possible_mask); > #endif > > But that could also be left to a future optimization as well. I meant to add: Acked-by: Kevin Hilman as well, as the NO_HZ_FULL bit could be added as an add-on patch. Kevin -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 3/4] workqueue: Create low-level unbound workqueues cpumask
Lai Jiangshan writes: > From: Frederic Weisbecker > > Create a cpumask that limit the affinity of all unbound workqueues. > This cpumask is controlled though a file at the root of the workqueue > sysfs directory. > > It works on a lower-level than the per WQ_SYSFS workqueues cpumask files > such that the effective cpumask applied for a given unbound workqueue is > the intersection of /sys/devices/virtual/workqueue/$WORKQUEUE/cpumask and > the new /sys/devices/virtual/workqueue/cpumask_unbounds file. > > This patch implements the basic infrastructure and the read interface. > cpumask_unbounds is initially set to cpu_possible_mask. > > Cc: Christoph Lameter > Cc: Kevin Hilman > Cc: Lai Jiangshan > Cc: Mike Galbraith > Cc: Paul E. McKenney > Cc: Tejun Heo > Cc: Viresh Kumar > Signed-off-by: Frederic Weisbecker > Signed-off-by: Lai Jiangshan [...] > @@ -5094,6 +5116,9 @@ static int __init init_workqueues(void) > > WARN_ON(__alignof__(struct pool_workqueue) < __alignof__(long long)); > > + BUG_ON(!alloc_cpumask_var(&wq_unbound_cpumask, GFP_KERNEL)); > + cpumask_copy(wq_unbound_cpumask, cpu_possible_mask); > + As I mentioned in an earlier discussion[1], I still think this could default too the housekeeping CPUs in the NO_HZ_FULL case: #ifdef CONFIG_NO_HZ_FULL cpumask_complement(wq_unbound_cpumask, tick_nohz_full_mask); #else cpumask_copy(wq_unbound_cpumask, cpu_possible_mask); #endif But that could also be left to a future optimization as well. Kevin [1] https://lkml.org/lkml/2014/2/14/666 -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 3/4] workqueue: Create low-level unbound workqueues cpumask
On Thu, 12 Mar 2015, Lai Jiangshan wrote: > This patch implements the basic infrastructure and the read interface. > cpumask_unbounds is initially set to cpu_possible_mask. Reviewed-by: Christoph Lameter -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[PATCH 3/4] workqueue: Create low-level unbound workqueues cpumask
From: Frederic Weisbecker
Create a cpumask that limit the affinity of all unbound workqueues.
This cpumask is controlled though a file at the root of the workqueue
sysfs directory.
It works on a lower-level than the per WQ_SYSFS workqueues cpumask files
such that the effective cpumask applied for a given unbound workqueue is
the intersection of /sys/devices/virtual/workqueue/$WORKQUEUE/cpumask and
the new /sys/devices/virtual/workqueue/cpumask_unbounds file.
This patch implements the basic infrastructure and the read interface.
cpumask_unbounds is initially set to cpu_possible_mask.
Cc: Christoph Lameter
Cc: Kevin Hilman
Cc: Lai Jiangshan
Cc: Mike Galbraith
Cc: Paul E. McKenney
Cc: Tejun Heo
Cc: Viresh Kumar
Signed-off-by: Frederic Weisbecker
Signed-off-by: Lai Jiangshan
---
kernel/workqueue.c | 29 +++--
1 file changed, 27 insertions(+), 2 deletions(-)
diff --git a/kernel/workqueue.c b/kernel/workqueue.c
index 957b244..61b5bfa 100644
--- a/kernel/workqueue.c
+++ b/kernel/workqueue.c
@@ -299,6 +299,8 @@ static DEFINE_SPINLOCK(wq_mayday_lock); /* protects
wq->maydays list */
static LIST_HEAD(workqueues); /* PR: list of all workqueues */
static bool workqueue_freezing;/* PL: have wqs started
freezing? */
+static cpumask_var_t wq_unbound_cpumask;
+
/* the per-cpu worker pools */
static DEFINE_PER_CPU_SHARED_ALIGNED(struct worker_pool [NR_STD_WORKER_POOLS],
cpu_worker_pools);
@@ -3529,7 +3531,7 @@ wq_unbound_install_ctx_prepare(struct workqueue_struct
*wq,
/* make a copy of @attrs and sanitize it */
copy_workqueue_attrs(new_attrs, attrs);
- cpumask_and(new_attrs->cpumask, new_attrs->cpumask, cpu_possible_mask);
+ cpumask_and(new_attrs->cpumask, new_attrs->cpumask, wq_unbound_cpumask);
/*
* We may create multiple pwqs with differing cpumasks. Make a
@@ -3959,9 +3961,29 @@ static struct bus_type wq_subsys = {
.dev_groups = wq_sysfs_groups,
};
+static ssize_t unbounds_cpumask_show(struct device *dev,
+struct device_attribute *attr, char *buf)
+{
+ int written;
+
+ written = scnprintf(buf, PAGE_SIZE, "%*pb\n",
+ cpumask_pr_args(wq_unbound_cpumask));
+
+ return written;
+}
+
+static struct device_attribute wq_sysfs_cpumask_attr =
+ __ATTR(cpumask, 0444, unbounds_cpumask_show, NULL);
+
static int __init wq_sysfs_init(void)
{
- return subsys_virtual_register(&wq_subsys, NULL);
+ int err;
+
+ err = subsys_virtual_register(&wq_subsys, NULL);
+ if (err)
+ return err;
+
+ return device_create_file(wq_subsys.dev_root, &wq_sysfs_cpumask_attr);
}
core_initcall(wq_sysfs_init);
@@ -5094,6 +5116,9 @@ static int __init init_workqueues(void)
WARN_ON(__alignof__(struct pool_workqueue) < __alignof__(long long));
+ BUG_ON(!alloc_cpumask_var(&wq_unbound_cpumask, GFP_KERNEL));
+ cpumask_copy(wq_unbound_cpumask, cpu_possible_mask);
+
pwq_cache = KMEM_CACHE(pool_workqueue, SLAB_PANIC);
cpu_notifier(workqueue_cpu_up_callback, CPU_PRI_WORKQUEUE_UP);
--
2.1.0
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
