On 03/25/2015 01:31 AM, Tejun Heo wrote: > On Wed, Mar 18, 2015 at 12:40:17PM +0800, Lai Jiangshan wrote: >> The oreder-workquue is ignore from the low level unbound workqueue cpumask, >> it will be handled in near future. > > Ugh, right, ordered workqueues are tricky. Maybe we should change how > ordered workqueues are implemented. Just gate work items at the > workqueue layer instead of fiddling with max_active and the number of > pwqs. > >> static struct wq_unbound_install_ctx * >> wq_unbound_install_ctx_prepare(struct workqueue_struct *wq, >> - const struct workqueue_attrs *attrs) >> + const struct workqueue_attrs *attrs, >> + cpumask_var_t unbound_cpumask) >> { > ... >> /* make a copy of @attrs and sanitize it */ >> copy_workqueue_attrs(new_attrs, attrs); >> - cpumask_and(new_attrs->cpumask, new_attrs->cpumask, wq_unbound_cpumask); >> + copy_workqueue_attrs(pwq_attrs, attrs); >> + cpumask_and(new_attrs->cpumask, new_attrs->cpumask, cpu_possible_mask); >> + cpumask_and(pwq_attrs->cpumask, pwq_attrs->cpumask, unbound_cpumask); > > Hmmm... we weren't checking whether the intersection becomes null > before.
Di you refer to the unquoted following code "cpumask_empty(pwq_attrs->cpumask)"? It is explained in the changelog and the comments. > Why are we doing it now? Note that this doesn't really make > things water-tight as cpu on/offlining can still leave the mask w/o > any online cpus. Shouldn't we just let the scheduler handle it as > before? Did you refer to "cpumask_and(new_attrs->cpumask, new_attrs->cpumask, cpu_possible_mask);"? new_attrs will be copied to wq->unbound_attrs, so we hope it is sanity. the same code before this patchset did the same work. And it maybe be used for default pwq, and it can reduce the pool creation: cpu_possible_mask = 0-7 wq_unbound_cpumask = 0-3 user1 try to set wq1: attrs->cpumask = 4-9 user2 try to set wq2: attrs->cpumask = 4-11 thus both wq1 and wq2's default pwq's pool is the same pool. (pool's cpumask = 4-7) > >> @@ -3712,6 +3726,9 @@ static void wq_update_unbound_numa(struct >> workqueue_struct *wq, int cpu, >> * wq's, the default pwq should be used. >> */ >> if (wq_calc_node_cpumask(wq->unbound_attrs, node, cpu_off, cpumask)) { >> + cpumask_and(cpumask, cpumask, wq_unbound_cpumask); >> + if (cpumask_empty(cpumask)) >> + goto use_dfl_pwq; > > So, this special handling is necessary only because we did special in > the above for dfl_pwq. Why do we need these? wq->unbound_attrs is user setting attrs, its cpumask is not controlled by wq_unbound_cpumask. so we need these cpumask_and(). Another question: Why wq->unbound_attrs' cpumask is not controlled by wq_unbound_cpumask? I hope the wq->unbound_attrs is always as the same as the user's last setting, regardless how much times the wq_unbound_cpumask is changed. > >> +static int unbounds_cpumask_apply(cpumask_var_t cpumask) >> +{ > .. >> + list_for_each_entry_safe(ctx, n, &ctxs, list) { >> + if (ret >= 0) > > Let's do !ret. > >> + wq_unbound_install_ctx_commit(ctx); >> + wq_unbound_install_ctx_free(ctx); >> + } > ... >> +/** >> + * workqueue_unbounds_cpumask_set - Set the low-level unbound cpumask >> + * @cpumask: the cpumask to set >> + * >> + * The low-level workqueues cpumask is a global cpumask that limits >> + * the affinity of all unbound workqueues. This function check the >> @cpumask >> + * and apply it to all unbound workqueues and updates all pwqs of them. >> + * When all succeed, it saves @cpumask to the global low-level unbound >> + * cpumask. >> + * >> + * Retun: 0 - Success >> + * -EINVAL - No online cpu in the @cpumask >> + * -ENOMEM - Failed to allocate memory for attrs or pwqs. >> + */ >> +int workqueue_unbounds_cpumask_set(cpumask_var_t cpumask) >> +{ >> + int ret = -EINVAL; >> + >> + get_online_cpus(); >> + cpumask_and(cpumask, cpumask, cpu_possible_mask); >> + if (cpumask_intersects(cpumask, cpu_online_mask)) { > > Does this make sense? We can't prevent cpus going down right after > the mask is set. What's the point of preventing empty config if we > can't prevent transitions into it and have to handle it anyway? Like set_cpus_allowed_ptr(). The cpumask must be valid when setting, although it can be transited into non-intersection later. This code is originated from Frederic. Maybe he has some stronger reason. > >> +static ssize_t unbounds_cpumask_store(struct device *dev, >> + struct device_attribute *attr, >> + const char *buf, size_t count) > > Naming is too confusing. Please pick a name which clearly > distinguishes per-wq and global masking. What about these names? wq_unbound_cpumask ==> wq_unbound_global_cpumask workqueue_unbounds_cpumask_set() ==> workqueue_set_unbound_global_cpumask(). (public API) unbounds_cpumask_store() ==> wq_store_unbound_global_cpumask() (static function for sysfs) > > Thanks. > -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/