On 03/07/2016 01:24 PM, Christoph Lameter wrote:
On Fri, 4 Mar 2016, Waiman Long wrote:This patch provides a mechanism to selectively degenerate per-cpu counters to global counters at per-cpu counter initialization time. The following new API is added: percpu_counter_set_limit(struct percpu_counter *fbc, u32 percpu_limit) The function should be called after percpu_counter_set(). It will compare the total limit (nr_cpu * percpu_limit) against the current counter value. If the limit is not smaller, it will disable per-cpu counter and use only the global counter instead. At run time, when the counter value grows past the total limit, per-cpu counter will be enabled again.Hmmm... That is requiring manual setting of a limit. Would it not be possible to completely automatize the switch over? F.e. one could keep a cpumask of processors that use the per cpu counters. Then in the fastpath if the current cpu is a member increment the per cpu counter. If not do the spinlock thing. If there is contention add the cpu to the cpumask and use the per cpu counters. Thus automatically scaling for the processors on which frequent increments are operating. Then regularly (once per minute or so) degenerate the counter by folding the per cpu diffs into the global count and zapping the cpumask. If the cpumask is empty you can use the global count. Otherwise you just need to add up the counters of the cpus set in the cpumask.
I have modified the patch to try that out. However, that doesn't yield that much of improvement in term of performance and it slows down the percpu fast path a bit. So I am going to focus on my existing patch first and think about that later.
Cheers, Longman
