[Bug 666828] Re: task jbd2/cciss!c0d0:258 blocked for more than 120 seconds
Just saw this one as well today, on a 12.04.5 with a 3.13 kernel Ssh, and serial console were unresponsive. Console displayed message about jbd2 being blocked for more than 120 seconds, and then loads of other processes (apache2) blocked for 120 seconds as well (but jbd2 was the first such message chronologically) Scrolling up and down virtual console worked. Changing virtual consoles worked as well, login prompts were displayed on the other virtual consoles, but after entering user name and pressing return there would just hang (no output, apart user name itself) After reboot (by killing the kvm process handling this virtual machine), logfiles showed a bunch of NUL characters (^@) before the reboot messages, and last message before these NULs was approx 2 hours before the crash. We've been running this virtual machine for a while, and this is the first time this happened. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/666828 Title: task jbd2/cciss!c0d0:258 blocked for more than 120 seconds To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/666828/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 666828] Re: task jbd2/cciss!c0d0:258 blocked for more than 120 seconds
Ok, seems like this issue is still present... I'm running Ubuntu 14.04.2 LTS Kernel: 3.16.0-41-generic x86_64 Hardware: HP Proliant DL380 ...but instead I get: task Not tainted 3.16.0-41-generic #55~14.04.1-Ubuntu task bash:17087 blocked for more than 120 seconds task jbd2/cciss!c0d0:156 blocked for more than 120 seconds task rs:main Q:Reg:633 blocked for more than 120 seconds ... Any fixes for Trysty? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/666828 Title: task jbd2/cciss!c0d0:258 blocked for more than 120 seconds To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/666828/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 666828] Re: task jbd2/cciss!c0d0:258 blocked for more than 120 seconds
** Changed in: linux (Ubuntu Natty) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/666828 Title: task jbd2/cciss!c0d0:258 blocked for more than 120 seconds To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/666828/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 666828] Re: task jbd2/cciss!c0d0:258 blocked for more than 120 seconds
Fair enough, I think I found some related files on the HP site, been reading through those. What I don't understand is this: why is it suddenly a firmware issue now on Ubuntu 12.04, when it wasn't an issue on VMWare ESX? The hardware has not changed. If there's a firmware bug, fine, but why was it handled before and not now? I managed to get an oops out of the current running kernel (3.2.0-40-generic): [69240.280022] INFO: task rs:main Q:Reg:405 blocked for more than 120 seconds. [69240.280037] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [69240.280050] rs:main Q:Reg D 0001 0 405 1 0x [69240.280057] 880036fbb9d8 0082 0001 0010 [69240.280065] 880036fbbfd8 880036fbbfd8 880036fbbfd8 000137c0 [69240.280073] 8800785a1700 88007a904500 880036fbb9b8 88007fc54080 [69240.280081] Call Trace: [69240.280093] [] ? __lock_page+0x70/0x70 [69240.280102] [] schedule+0x3f/0x60 [69240.280107] [] io_schedule+0x8f/0xd0 [69240.280113] [] sleep_on_page+0xe/0x20 [69240.280118] [] __wait_on_bit+0x5f/0x90 [69240.280122] [] wait_on_page_bit+0x78/0x80 [69240.280129] [] ? autoremove_wake_function+0x40/0x40 [69240.280134] [] grab_cache_page_write_begin+0x92/0xe0 [69240.280140] [] ? lock_timer_base.isra.29+0x38/0x70 [69240.280176] [] ? xfs_get_blocks_direct+0x20/0x20 [xfs] [69240.280183] [] block_write_begin+0x38/0xa0 [69240.280201] [] xfs_vm_write_begin+0x43/0x70 [xfs] [69240.280207] [] generic_perform_write+0xca/0x210 [69240.280213] [] generic_file_buffered_write+0x5d/0x90 [69240.280232] [] xfs_file_buffered_aio_write+0xfc/0x1c0 [xfs] [69240.280238] [] ? down_write+0x16/0x40 [69240.280257] [] xfs_file_aio_write+0x17c/0x2a0 [xfs] [69240.280263] [] ? unqueue_me+0x52/0x80 [69240.280267] [] ? futex_wait+0x108/0x210 [69240.280273] [] do_sync_write+0xda/0x120 [69240.280279] [] ? apparmor_file_permission+0x18/0x20 [69240.280285] [] ? security_file_permission+0x2c/0xb0 [69240.280289] [] ? rw_verify_area+0x61/0xf0 [69240.280294] [] vfs_write+0xb3/0x180 [69240.280298] [] sys_write+0x4a/0x90 [69240.280303] [] system_call_fastpath+0x16/0x1b -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/666828 Title: task jbd2/cciss!c0d0:258 blocked for more than 120 seconds To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/666828/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
Re: [Bug 666828] Re: task jbd2/cciss!c0d0:258 blocked for more than 120 seconds
* Stuart Longland <666...@bugs.launchpad.net>: > [1.097409] HP CISS Driver (v 3.6.26) > [1.098444] cciss :07:01.0: PCI IRQ 50 -> rerouted to legacy IRQ 18 > [1.098524] cciss :07:01.0: PCI INT A -> GSI 18 (level, low) -> IRQ 18 > [1.098694] cciss :07:01.0: Controller reports max supported commands > of 0, an obvious lie. Using 16. Ensure that firmware is up to date. > [1.216111] cciss :07:01.0: cciss0: <0x46> at PCI :07:01.0 IRQ 18 > using DAC > > I'm not sure what the firmware update procedure is on these things. That's easy: You boot into a HP firmware update CD, which is basically a linux boot disk. The filename's FW1010.2012_0530.49.iso I got a copy if you can't find the download... -- Ralf Hildebrandt Charite Universitätsmedizin Berlin ralf.hildebra...@charite.deCampus Benjamin Franklin http://www.charite.de Hindenburgdamm 30, 12203 Berlin Geschäftsbereich IT, Abt. Netzwerk fon: +49-30-450.570.155 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/666828 Title: task jbd2/cciss!c0d0:258 blocked for more than 120 seconds To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/666828/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 666828] Re: task jbd2/cciss!c0d0:258 blocked for more than 120 seconds
Just re-loaded, lspci reports the affected controller as being the following: 07:01.0 RAID bus controller: Compaq Computer Corporation Smart Array 64xx (rev 01) Subsystem: Compaq Computer Corporation Smart Array 642 Flags: bus master, 66MHz, medium devsel, latency 64, IRQ 18 Memory at fdff (64-bit, non-prefetchable) [size=8K] I/O ports at 5000 [size=256] Memory at fdf8 (64-bit, non-prefetchable) [size=256K] [virtual] Expansion ROM at 8030 [disabled] [size=256K] Capabilities: [d0] Power Management version 2 Capabilities: [dc] PCI-X non-bridge device Capabilities: [f0] Vital Product Data Kernel driver in use: cciss Kernel modules: cciss During boot, I see in `dmesg`: [1.097409] HP CISS Driver (v 3.6.26) [1.098444] cciss :07:01.0: PCI IRQ 50 -> rerouted to legacy IRQ 18 [1.098524] cciss :07:01.0: PCI INT A -> GSI 18 (level, low) -> IRQ 18 [1.098694] cciss :07:01.0: Controller reports max supported commands of 0, an obvious lie. Using 16. Ensure that firmware is up to date. [1.216111] cciss :07:01.0: cciss0: <0x46> at PCI :07:01.0 IRQ 18 using DAC I'm not sure what the firmware update procedure is on these things. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/666828 Title: task jbd2/cciss!c0d0:258 blocked for more than 120 seconds To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/666828/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 666828] Re: task jbd2/cciss!c0d0:258 blocked for more than 120 seconds
Not sure what the official status is, but I'm getting this problem with a HP ProLiant box here running Ubuntu 12.04 LTS and linux- image-3.5.0-25-generic as well as the 3.2-series kernels. For me the machine will run fine for about a day, then everything locks up. If I'm lucky, I can SSH in, but then `dmesg` hangs. `fold -w 80 /dev/vcs1` dumps the following output: message. [158400.224328] INFO: task oned:3106 blocked for more than 120 seconds. [158400.224337] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [158400.224491] INFO: task mm_sched:3116 blocked for more than 120 seconds. [158400.224501] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [158520.224030] INFO: task rs:main Q:Reg:888 blocked for more than 120 seconds. [158520.224046] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [158520.224327] INFO: task oned:3106 blocked for more than 120 seconds. [158520.224336] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [158520.224503] INFO: task mm_sched:3116 blocked for more than 120 seconds. [158520.224524] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [158640.224028] INFO: task rs:main Q:Reg:888 blocked for more than 120 seconds. [158640.224056] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [158640.224362] INFO: task oned:3106 blocked for more than 120 seconds. [158640.224382] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. The box previously worked reliably running VMWare ESXi. The link to the Red Hat bug database seems to require a log in to see the bug, so at time of writing, the bug is inaccessible to me and I'm unable to follow any advice there. (I get an error message "You are not authorized to access bug #615543.") I'm trying a complete reload (I install using a PXE boot image) to see if a newer kernel has been released that fixes the issue -- also to clean up the last vestiges of an OpenNebula installation that's no longer in use. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/666828 Title: task jbd2/cciss!c0d0:258 blocked for more than 120 seconds To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/666828/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 666828] Re: task jbd2/cciss!c0d0:258 blocked for more than 120 seconds
I'm also having the same issue with this. Why is the importance set to "medium"? I think this should be a priority. Server becomes unresponsive on a random basis. Sometimes daily, weekly, or anytime it likes to. I'm using - Ubuntu 11.10 Server 64bit - Linux MyServer 3.0.0-12-server #20-Ubuntu SMP Fri Oct 7 16:36:30 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux - HP Proliant ML 370 G3 X3. How come kernel 3.0 is still affected? Please confirm patch or fix release for Oneiric Servers. Thank you very much -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/666828 Title: task jbd2/cciss!c0d0:258 blocked for more than 120 seconds To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/666828/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 666828] Re: task jbd2/cciss!c0d0:258 blocked for more than 120 seconds
For me, upgrading the kernel to 2.6.38-10 did not solve the problem, but upgrading the SmartArray firmware as suggested in the Red Hat forum listed above did work. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/666828 Title: task jbd2/cciss!c0d0:258 blocked for more than 120 seconds To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/666828/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 666828] Re: task jbd2/cciss!c0d0:258 blocked for more than 120 seconds
For Maverick, this should be fixed now with 2.6.35-30.54. Please reopen the Maverick task if you still experience this issue. ** Changed in: linux (Ubuntu Maverick) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/666828 Title: task jbd2/cciss!c0d0:258 blocked for more than 120 seconds To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/666828/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 666828] Re: task jbd2/cciss!c0d0:258 blocked for more than 120 seconds
Patch made it upstream into 2.6.39-rc1, it was picked up by stable and is included in - 2.6.32-32.62 (10.04 Lucid) - 2.6.35-29.51 (10.10 Maverick) [this is currently in proposed only] - 2.6.38-9.43 (11.04 Natty) [this is currently in proposed only] ** Also affects: linux (Ubuntu Lucid) Importance: Undecided Status: New ** Also affects: linux (Ubuntu Maverick) Importance: Undecided Status: New ** Also affects: linux (Ubuntu Natty) Importance: Undecided Status: New ** Changed in: linux (Ubuntu Lucid) Importance: Undecided => Medium ** Changed in: linux (Ubuntu Lucid) Status: New => Fix Released ** Changed in: linux (Ubuntu) Importance: Undecided => Medium ** Changed in: linux (Ubuntu) Status: Confirmed => Fix Released ** Changed in: linux (Ubuntu Maverick) Importance: Undecided => Medium ** Changed in: linux (Ubuntu Maverick) Status: New => Fix Committed ** Changed in: linux (Ubuntu Natty) Importance: Undecided => Medium ** Changed in: linux (Ubuntu Natty) Status: New => Fix Committed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/666828 Title: task jbd2/cciss!c0d0:258 blocked for more than 120 seconds -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 666828] Re: task jbd2/cciss!c0d0:258 blocked for more than 120 seconds
This is a patch wich supposedly fixes the problem. ** Attachment added: "Patch by Greg KH" https://bugs.launchpad.net/ubuntu/+source/linux/+bug/666828/+attachment/2142133/+files/patch.txt -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/666828 Title: task jbd2/cciss!c0d0:258 blocked for more than 120 seconds -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 666828] Re: task jbd2/cciss!c0d0:258 blocked for more than 120 seconds
Is anybody using arrayprobe, the hp-health tools or similar programs to monitor the raid arrays health? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/666828 Title: task jbd2/cciss!c0d0:258 blocked for more than 120 seconds -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 666828] Re: task jbd2/cciss!c0d0:258 blocked for more than 120 seconds
Sounds like https://bugzilla.redhat.com/show_bug.cgi?id=615543 ** Bug watch added: Red Hat Bugzilla #615543 https://bugzilla.redhat.com/show_bug.cgi?id=615543 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/666828 Title: task jbd2/cciss!c0d0:258 blocked for more than 120 seconds -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 666828] Re: task jbd2/cciss!c0d0:258 blocked for more than 120 seconds
** Tags removed: regression-potential -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/666828 Title: task jbd2/cciss!c0d0:258 blocked for more than 120 seconds -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 666828] Re: task jbd2/cciss!c0d0:258 blocked for more than 120 seconds
I think I am having the same problem. I have Ubuntu Server 10.10 (kernel 2.6.35-28) running on an HP ProLiant DL380 G4. I posted some more specifics here: http://ubuntuforums.org/showthread.php?p=10722640#post10722640 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/666828 Title: task jbd2/cciss!c0d0:258 blocked for more than 120 seconds -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 666828] Re: task jbd2/cciss!c0d0:258 blocked for more than 120 seconds
** Changed in: linux (Ubuntu) Status: New => Confirmed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/666828 Title: task jbd2/cciss!c0d0:258 blocked for more than 120 seconds -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 666828] Re: task jbd2/cciss!c0d0:258 blocked for more than 120 seconds
Ubuntu Natty 2.6.38-7-server AMD64 - same issue, same hardware. Tried updating to the Alpha build just to see, but no difference. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/666828 Title: task jbd2/cciss!c0d0:258 blocked for more than 120 seconds -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 666828] Re: task jbd2/cciss!c0d0:258 blocked for more than 120 seconds
This bug also affects me: Ubuntu Server 10:10 - kernel 2.6.35-22 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/666828 Title: task jbd2/cciss!c0d0:258 blocked for more than 120 seconds -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 666828] Re: task jbd2/cciss!c0d0:258 blocked for more than 120 seconds
Same issue here "task jbd2/cciss!c0d0 blocked for more than 120 seconds" on a HP Proliant ML350 G3 (32-bits Xeon's). Cannot log in via console. Ubuntu 10.10. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/666828 Title: task jbd2/cciss!c0d0:258 blocked for more than 120 seconds -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 666828] Re: task jbd2/cciss!c0d0:258 blocked for more than 120 seconds
I'm experiencing this issue on 10.10, 2.6.35-24-server on the same hardware (HP ProLiant DL360 G4). -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/666828 Title: task jbd2/cciss!c0d0:258 blocked for more than 120 seconds -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 666828] Re: task jbd2/cciss!c0d0:258 blocked for more than 120 seconds
** Attachment added: "Screen immediately after the lockup" https://bugs.launchpad.net/bugs/666828/+attachment/1711874/+files/syslog-netz.png -- task jbd2/cciss!c0d0:258 blocked for more than 120 seconds https://bugs.launchpad.net/bugs/666828 You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
