** Also affects: linux (Ubuntu)
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1328088
Title:
Kernel network namespace performance regression during rcu development
on kernels above 3.8
Status in The Linux Kernel:
In Progress
Status in “linux” package in Ubuntu:
New
Bug description:
Please, follow this in:
http://people.canonical.com/~inaddy/lp1328088/. Same description on
daily-basis updated text.
--
It was brought to my attention that "fake router creation" scalability
was affected during kernel development.
The following script was used for all the tests and charts generation:
http://people.canonical.com/~inaddy/lp1328088/make_fake_routers.sh
http://people.canonical.com/~inaddy/lp1328088/parse.py
I measured how many "fake routers" (above script) could be added per
second from 0 to 4000 created routers mark. Using this script and a
git bisect on kernel tree I was led to one specific commit causing
regression: #911af505 "rcu: Provide compile-time control for no-CBs
CPUs".
It appeared that rcu, rcu callbacks and no-cb cpus were causing the
issue so every commit that changed any of this files:
"kernel/rcutree.c kernel/rcutree.h kernel/rcutree_plugin.h
include/trace/events/rcu.h include/linux/rcupdate.h" was tested. The
idea was to check performance regression during rcu development. In
the worst case I would have data for performance regression during
kernel development (since we have rcu commits from 3.8 to 3.14).
All text below this refer to 2 groups of charts, generated during the
study:
1) Kernel git tags from 3.8 to 3.14.
http://people.canonical.com/~inaddy/lp1328088/charts/250-tag.html
2) Kernel git commits for rcu development (111 commits).
http://people.canonical.com/~inaddy/lp1328088/charts/250.html
Since there was difference in results depending on how many cpus or
how the no-cb cpus were configured, 3 kernel config options were used
on every measure:
- CONFIG_RCU_NOCB_CPU (disabled: nocbno)
- CONFIG_RCU_NOCB_CPU_ALL (enabled: nocball)
- CONFIG_RCU_NOCB_CPU_NONE (enabled: nocbnone)
After charts generation and study it was clear that NOCB_CPU_ALL (4
cpus) affected the "fake routers" creation process performance and
this regression continues up to upstream version. It was also clear
that, after this commit, there is no scalability executing this test
with more than 1 cpu.
Comparing standing out points (see charts):
#81e5949 - good
#911af50 - bad
#6faf728 - not good enough
I was able to see that from the script above the following lines were
affected:
1) ip netns add -> huge performance regression
2) ip netns exec -> some performance regression
#
# Assumption
#
rcu callbacks being offloaded to other cpus caused regression in
unshare(CLONE_NEWNET) code.
# Specific kernel entry being investigated:
unshare(CLONE_NEWNET)
To manage notifications about this bug go to:
https://bugs.launchpad.net/linux/+bug/1328088/+subscriptions
--
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help : https://help.launchpad.net/ListHelp
