[Bug 1183135] Re: On running network and I/O traffic on SM15000-XE, BUG: soft lockup -- CPU#0 stuck for 22s! seen on dmesg
Will a fixed version for Ubuntu 12.04 LTS be available? -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to irqbalance in Ubuntu. https://bugs.launchpad.net/bugs/1183135 Title: On running network and I/O traffic on SM15000-XE, BUG: soft lockup -- CPU#0 stuck for 22s! seen on dmesg To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/irqbalance/+bug/1183135/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1183135] Re: On running network and I/O traffic on SM15000-XE, BUG: soft lockup -- CPU#0 stuck for 22s! seen on dmesg
Looks like that Ubuntu 10.04 LTS is also affected. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to irqbalance in Ubuntu. https://bugs.launchpad.net/bugs/1183135 Title: On running network and I/O traffic on SM15000-XE, BUG: soft lockup -- CPU#0 stuck for 22s! seen on dmesg To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/irqbalance/+bug/1183135/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1183135] Re: On running network and I/O traffic on SM15000-XE, BUG: soft lockup -- CPU#0 stuck for 22s! seen on dmesg
Hi, In order to figure out what broke or fixed this, it would be great if we could bisect. Unfortunately the irqbalance svn and git repos don't seem to have any version numbers prior to 1.0.0, so it's a bit tough to tell which versions we should be checking. Fortunately, the package seems to build quite easily from git. Unfortunately (again), I don't have any hardware I would be able to reproduce this on. So I will place here the basic recipe to build a package from a particular git version. I would recommend trying the following to versions as baselines: commit 0b917bd98c684878b2730f3a57e15d7c6149daff (Sep 24 16:26:44 2010) commit d7ef8fca9edf24a543f88073a01edb476c295256 (Fri Dec 2 11:06:14 2011 -0500) Hopefully the first will reproduce the bug, and the second not. If that is the case, then you can proceed with a bisect between those versions. (Please ask here if you need more instructions on that) There are only 85 commits so that should take about 8 steps to bisect. You will need to grab a irqbalance git tree, and grab the packaging tree. git clone https://code.google.com/p/irqbalance pull-lp-source irqbalance quantal Now, clone a new git tree in which you'll do the build: git clone irqbalance build cd build Go into the build directory, and checkout the version to test: git checkout -b test git reset --hard 0b917bd98c684878b2730f3a57e15d7c6149daff Run autoconf: sh autogen.sh Copy the packaging, cp -a ../irqbalance-1.0.3/debian . This is optional, but you may want to edit the changelog to give yourself an easily recognizable version. Run dch -i which pulls the changelog up in your $EDITOR, with the first lines looking like: irqbalance (1.0.3-1ubuntu3) UNRELEASED; urgency=low * -- Serge Hallyn serge.hal...@ubuntu.com Tue, 24 Sep 2013 15:00:23 -0500 Edit the version number and release, and add a message, so that it looks like: irqbalance (1.0.3-1ubuntu3bisect1) precise; urgency=low * Build from commit 0b917bd98c684878b2730f3a57e15d7c6149daff -- Serge Hallyn serge.hal...@ubuntu.com Tue, 24 Sep 2013 15:00:23 -0500 Save that and exit the editor. Finally build the package: debian/rules build fakeroot debian/rules binary Now you can install the packages and test: cd .. dpkg -i irqbalance_1.0.3-1ubuntu3bisect1_amd64.deb -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to irqbalance in Ubuntu. https://bugs.launchpad.net/bugs/1183135 Title: On running network and I/O traffic on SM15000-XE, BUG: soft lockup -- CPU#0 stuck for 22s! seen on dmesg To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/irqbalance/+bug/1183135/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1183135] Re: On running network and I/O traffic on SM15000-XE, BUG: soft lockup -- CPU#0 stuck for 22s! seen on dmesg
I'm assuming this bug is in the irqbalance, and not the kernel? But have we tried one of the newer point releases (like 12.04.2 LTS) which is based on the Q HWE kernel? -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to irqbalance in Ubuntu. https://bugs.launchpad.net/bugs/1183135 Title: On running network and I/O traffic on SM15000-XE, BUG: soft lockup -- CPU#0 stuck for 22s! seen on dmesg To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/irqbalance/+bug/1183135/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1183135] Re: On running network and I/O traffic on SM15000-XE, BUG: soft lockup -- CPU#0 stuck for 22s! seen on dmesg
@Scott, What would be the next step to fix the irqbalance in 12.04? Samantha -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to irqbalance in Ubuntu. https://bugs.launchpad.net/bugs/1183135 Title: On running network and I/O traffic on SM15000-XE, BUG: soft lockup -- CPU#0 stuck for 22s! seen on dmesg To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/irqbalance/+bug/1183135/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1183135] Re: On running network and I/O traffic on SM15000-XE, BUG: soft lockup -- CPU#0 stuck for 22s! seen on dmesg
Thanks vikrant. So it appears that the bug is only in 12.04 released version of irqbalance (mentioned fixed in 0.55+20091017-3ubuntu2 of oneiric and 1.0.3-1ubuntu2 of quantal). ** Also affects: irqbalance (Ubuntu Precise) Importance: Undecided Status: New ** Also affects: irqbalance (Ubuntu Quantal) Importance: Undecided Status: New ** Changed in: irqbalance (Ubuntu Quantal) Status: New = Fix Released ** Changed in: irqbalance (Ubuntu Quantal) Importance: Undecided = Medium ** Changed in: irqbalance (Ubuntu Precise) Status: New = Confirmed ** Changed in: irqbalance (Ubuntu Precise) Importance: Undecided = High ** Changed in: irqbalance (Ubuntu) Status: Confirmed = Fix Released ** Changed in: irqbalance (Ubuntu) Importance: Undecided = High -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to irqbalance in Ubuntu. https://bugs.launchpad.net/bugs/1183135 Title: On running network and I/O traffic on SM15000-XE, BUG: soft lockup -- CPU#0 stuck for 22s! seen on dmesg To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/irqbalance/+bug/1183135/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1183135] Re: On running network and I/O traffic on SM15000-XE, BUG: soft lockup -- CPU#0 stuck for 22s! seen on dmesg
Ran the network test for over 72 hours with the latest Ubuntu irqbalance package v 1.0.3 (quantal)and no tx unit hang was seen. Line rate was obtained on all interfaces. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to irqbalance in Ubuntu. https://bugs.launchpad.net/bugs/1183135 Title: On running network and I/O traffic on SM15000-XE, BUG: soft lockup -- CPU#0 stuck for 22s! seen on dmesg To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/irqbalance/+bug/1183135/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1183135] Re: On running network and I/O traffic on SM15000-XE, BUG: soft lockup -- CPU#0 stuck for 22s! seen on dmesg
NIC: Intel 82540EM Gigabit Ethernet Controller e1000: Intel(R) PRO/1000 Network Driver - version 7.3.21-k8-NAPI (from attached dmesg) -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to irqbalance in Ubuntu. https://bugs.launchpad.net/bugs/1183135 Title: On running network and I/O traffic on SM15000-XE, BUG: soft lockup -- CPU#0 stuck for 22s! seen on dmesg To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/irqbalance/+bug/1183135/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1183135] Re: On running network and I/O traffic on SM15000-XE, BUG: soft lockup -- CPU#0 stuck for 22s! seen on dmesg
Hi Vikrant, Please let us know if you can reproduce the issue with irqbalance 1.0.3-1ubuntu2 in the 12.10 (quantal) repo. Thanks, Samantha -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to irqbalance in Ubuntu. https://bugs.launchpad.net/bugs/1183135 Title: On running network and I/O traffic on SM15000-XE, BUG: soft lockup -- CPU#0 stuck for 22s! seen on dmesg To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/irqbalance/+bug/1183135/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1183135] Re: On running network and I/O traffic on SM15000-XE, BUG: soft lockup -- CPU#0 stuck for 22s! seen on dmesg
Comment from Vikrant: With Ubuntu 12.04, we see TX hang and exceptions on running network traffic. This is seen particularly on Intel platforms. The issue seems to arise because of the fact that irqbalance does not balance the interrupts evenly across all available cores. Ubuntu 12.04 uses 0.56-1ubuntu4 irqbalance package. As a workaround, we tried to use the irqbalance package from an older version of Ubuntu, namely 0.55+20091017-3ubuntu2, and it worked very well. We see the full expected performance and there are no TX hangs or exceptions. The interrupts seem to be spread evenly across all cores. ** Changed in: ubuntu Assignee: (unassigned) = Samantha Jian-Pielak (samantha-jian) ** Package changed: ubuntu = irqbalance (Ubuntu) ** Changed in: irqbalance (Ubuntu) Assignee: Samantha Jian-Pielak (samantha-jian) = (unassigned) -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to irqbalance in Ubuntu. https://bugs.launchpad.net/bugs/1183135 Title: On running network and I/O traffic on SM15000-XE, BUG: soft lockup -- CPU#0 stuck for 22s! seen on dmesg To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/irqbalance/+bug/1183135/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
