[Kernel-packages] [Bug 1495442] Re: XFS: kernel hangs multiple times in xlog_cil_force_lsn
[Expired for linux (Ubuntu) because there has been no activity for 60 days.] ** Changed in: linux (Ubuntu) Status: Incomplete => Expired -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1495442 Title: XFS: kernel hangs multiple times in xlog_cil_force_lsn Status in linux package in Ubuntu: Expired Bug description: Ubuntu 14.04.3 LTS with Kernel 3.19.0-26-generic One of our servers running rsnapshot and nfs-kernel-server on XFS partitions crashed or hanged two sundays in a raw. Kernel 3.19.0-26-generic hangs multiple times (same time) in xlog_cil_force_lsn (XFS). I assume this bug is not related with bug report 979498. I had also this issue with kernel 3.16.0-46-generic or 3.13.0-57-generic but I have no log during the crash to determine which of the above kernel was running. I switched to kernel 3.19.0-26-generic because 3.16.0-48-generic was not working well with Intel Gigabit -- tx hang (bug 1492146) INFO: task kthreadd:2 blocked for more than 120 seconds. Tainted: G C 3.19.0-26-generic #28~14.04.1-Ubuntu "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. kthreaddD 8802362eb3a8 0 2 0 0x 8802362eb3a8 8802362c09d0 00013e80 8802362ebfd8 00013e80 880235c28000 8802362c09d0 88020001 8802362eb500 7fff 8802362eb4f8 8802362c09d0 Call Trace: [] schedule+0x29/0x70 [] schedule_timeout+0x20c/0x280 [] ? update_curr+0xe4/0x180 [] wait_for_completion+0xa4/0x170 [] ? wake_up_state+0x20/0x20 [] flush_work+0xed/0x1c0 [] ? destroy_worker+0x90/0x90 [] xlog_cil_force_lsn+0x7e/0x1f0 [xfs] [] ? __free_pages+0x27/0x30 [] ? __free_slab+0xd0/0x1f0 [] _xfs_log_force_lsn+0x5e/0x2d0 [xfs] [] xfs_log_force_lsn+0x2e/0x90 [xfs] [] ? xfs_iunpin_wait+0x19/0x20 [xfs] [] __xfs_iunpin_wait+0x8d/0x120 [xfs] [] ? autoremove_wake_function+0x40/0x40 [] xfs_iunpin_wait+0x19/0x20 [xfs] [] xfs_reclaim_inode+0x7c/0x340 [xfs] [] xfs_reclaim_inodes_ag+0x257/0x370 [xfs] [] xfs_reclaim_inodes_nr+0x33/0x40 [xfs] [] xfs_fs_free_cached_objects+0x15/0x20 [xfs] [] super_cache_scan+0x169/0x170 [] shrink_node_slabs+0x1d6/0x370 [] shrink_zone+0x20a/0x240 [] do_try_to_free_pages+0x155/0x440 [] ? zone_watermark_ok+0x1f/0x30 [] try_to_free_pages+0xba/0x150 [] __alloc_pages_nodemask+0x61b/0xa60 [] alloc_kmem_pages_node+0x6a/0x130 [] ? copy_process.part.26+0xf3/0x1c00 [] copy_process.part.26+0x113/0x1c00 [] ? dequeue_task_fair+0x44e/0x660 [] ? put_prev_entity+0x31/0x3f0 [] ? pick_next_task_fair+0x19c/0x880 [] ? kthread_create_on_node+0x1c0/0x1c0 [] ? put_prev_task_fair+0x2f/0x50 [] do_fork+0xd5/0x340 [] kernel_thread+0x26/0x30 [] kthreadd+0x15a/0x1c0 [] ? kthread_create_on_cpu+0x60/0x60 [] ret_from_fork+0x58/0x90 [] ? kthread_create_on_cpu+0x60/0x60 10:17:01 xyz CRON[59020]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly) 11:17:01 xyz CRON[59177]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly) 12:00:01 xyz CRON[59282]: (root) CMD (/usr/bin/rsnapshot -c /etc/rsnapshot.conf hourly) 12:03:05 xyz kernel: [616881.456247] INFO: task kthreadd:2 blocked for more than 120 seconds. 12:03:05 xyz kernel: [616881.456587] Tainted: G C 3.19.0-26-generic #28~14.04.1-Ubuntu ... 12:03:05 xyz kernel: [616881.457934] INFO: task kswapd0:57 blocked for more than 120 seconds. 12:03:05 xyz kernel: [616881.458271] Tainted: G C 3.19.0-26-generic #28~14.04.1-Ubuntu ... 12:03:05 xyz kernel: [616881.485629] INFO: task nfsd:2002 blocked for more than 120 seconds. 12:03:05 xyz kernel: [616881.492761] Tainted: G C 3.19.0-26-generic #28~14.04.1-Ubuntu ... 12:03:05 xyz kernel: [616881.523050] INFO: task snmpd:2155 blocked for more than 120 seconds. 12:03:05 xyz kernel: [616881.530783] Tainted: G C 3.19.0-26-generic #28~14.04.1-Ubuntu ... 12:03:05 xyz kernel: [616881.562789] INFO: task kworker/1:0:59030 blocked for more than 120 seconds. 12:03:05 xyz kernel: [616881.571082] Tainted: G C 3.19.0-26-generic #28~14.04.1-Ubuntu ... 12:03:05 xyz kernel: [616881.605747] INFO: task kworker/3:1H:59110 blocked for more than 120 seconds. 12:03:05 xyz kernel: [616881.615112] Tainted: G C 3.19.0-26-generic #28~14.04.1-Ubuntu ... 12:03:05 xyz kernel: [616881.652107] INFO: task rm:59284 blocked for more than 120 seconds. 12:03:05 xyz kernel: [616881.661441] Tainted: G C 3.19.0-26-generic #28~14.04.1-Ubuntu ... 12:05:05 xyz kernel: [617001.782228] INFO: task kswapd0:57 blocked for more than 120 seconds. 12:05:05 xyz kernel:
[Kernel-packages] [Bug 1495442] Re: XFS: kernel hangs multiple times in xlog_cil_force_lsn
https://bugs.launchpad.net/ubuntu/+source/linux-lts-utopic/+bug/1492146 has been fixed in 3.16 so we have downgraded until we are confident the XFS bugs have been resolved. I know a bunch of work has been done in 3.19 for XFS and more work has been done in 4.1, so may just have to will for next stable. I did try to run xfstest on a dev machine but didn't hit the bug we are talking about, so any idea on how to reproduce would be good. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1495442 Title: XFS: kernel hangs multiple times in xlog_cil_force_lsn Status in linux package in Ubuntu: Incomplete Bug description: Ubuntu 14.04.3 LTS with Kernel 3.19.0-26-generic One of our servers running rsnapshot and nfs-kernel-server on XFS partitions crashed or hanged two sundays in a raw. Kernel 3.19.0-26-generic hangs multiple times (same time) in xlog_cil_force_lsn (XFS). I assume this bug is not related with bug report 979498. I had also this issue with kernel 3.16.0-46-generic or 3.13.0-57-generic but I have no log during the crash to determine which of the above kernel was running. I switched to kernel 3.19.0-26-generic because 3.16.0-48-generic was not working well with Intel Gigabit -- tx hang (bug 1492146) INFO: task kthreadd:2 blocked for more than 120 seconds. Tainted: G C 3.19.0-26-generic #28~14.04.1-Ubuntu "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. kthreaddD 8802362eb3a8 0 2 0 0x 8802362eb3a8 8802362c09d0 00013e80 8802362ebfd8 00013e80 880235c28000 8802362c09d0 88020001 8802362eb500 7fff 8802362eb4f8 8802362c09d0 Call Trace: [] schedule+0x29/0x70 [] schedule_timeout+0x20c/0x280 [] ? update_curr+0xe4/0x180 [] wait_for_completion+0xa4/0x170 [] ? wake_up_state+0x20/0x20 [] flush_work+0xed/0x1c0 [] ? destroy_worker+0x90/0x90 [] xlog_cil_force_lsn+0x7e/0x1f0 [xfs] [] ? __free_pages+0x27/0x30 [] ? __free_slab+0xd0/0x1f0 [] _xfs_log_force_lsn+0x5e/0x2d0 [xfs] [] xfs_log_force_lsn+0x2e/0x90 [xfs] [] ? xfs_iunpin_wait+0x19/0x20 [xfs] [] __xfs_iunpin_wait+0x8d/0x120 [xfs] [] ? autoremove_wake_function+0x40/0x40 [] xfs_iunpin_wait+0x19/0x20 [xfs] [] xfs_reclaim_inode+0x7c/0x340 [xfs] [] xfs_reclaim_inodes_ag+0x257/0x370 [xfs] [] xfs_reclaim_inodes_nr+0x33/0x40 [xfs] [] xfs_fs_free_cached_objects+0x15/0x20 [xfs] [] super_cache_scan+0x169/0x170 [] shrink_node_slabs+0x1d6/0x370 [] shrink_zone+0x20a/0x240 [] do_try_to_free_pages+0x155/0x440 [] ? zone_watermark_ok+0x1f/0x30 [] try_to_free_pages+0xba/0x150 [] __alloc_pages_nodemask+0x61b/0xa60 [] alloc_kmem_pages_node+0x6a/0x130 [] ? copy_process.part.26+0xf3/0x1c00 [] copy_process.part.26+0x113/0x1c00 [] ? dequeue_task_fair+0x44e/0x660 [] ? put_prev_entity+0x31/0x3f0 [] ? pick_next_task_fair+0x19c/0x880 [] ? kthread_create_on_node+0x1c0/0x1c0 [] ? put_prev_task_fair+0x2f/0x50 [] do_fork+0xd5/0x340 [] kernel_thread+0x26/0x30 [] kthreadd+0x15a/0x1c0 [] ? kthread_create_on_cpu+0x60/0x60 [] ret_from_fork+0x58/0x90 [] ? kthread_create_on_cpu+0x60/0x60 10:17:01 xyz CRON[59020]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly) 11:17:01 xyz CRON[59177]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly) 12:00:01 xyz CRON[59282]: (root) CMD (/usr/bin/rsnapshot -c /etc/rsnapshot.conf hourly) 12:03:05 xyz kernel: [616881.456247] INFO: task kthreadd:2 blocked for more than 120 seconds. 12:03:05 xyz kernel: [616881.456587] Tainted: G C 3.19.0-26-generic #28~14.04.1-Ubuntu ... 12:03:05 xyz kernel: [616881.457934] INFO: task kswapd0:57 blocked for more than 120 seconds. 12:03:05 xyz kernel: [616881.458271] Tainted: G C 3.19.0-26-generic #28~14.04.1-Ubuntu ... 12:03:05 xyz kernel: [616881.485629] INFO: task nfsd:2002 blocked for more than 120 seconds. 12:03:05 xyz kernel: [616881.492761] Tainted: G C 3.19.0-26-generic #28~14.04.1-Ubuntu ... 12:03:05 xyz kernel: [616881.523050] INFO: task snmpd:2155 blocked for more than 120 seconds. 12:03:05 xyz kernel: [616881.530783] Tainted: G C 3.19.0-26-generic #28~14.04.1-Ubuntu ... 12:03:05 xyz kernel: [616881.562789] INFO: task kworker/1:0:59030 blocked for more than 120 seconds. 12:03:05 xyz kernel: [616881.571082] Tainted: G C 3.19.0-26-generic #28~14.04.1-Ubuntu ... 12:03:05 xyz kernel: [616881.605747] INFO: task kworker/3:1H:59110 blocked for more than 120 seconds. 12:03:05 xyz kernel: [616881.615112] Tainted: G C 3.19.0-26-generic #28~14.04.1-Ubuntu ... 12:03:05 xyz kernel:
[Kernel-packages] [Bug 1495442] Re: XFS: kernel hangs multiple times in xlog_cil_force_lsn
The bug doesn't seem to impact 3.13.0-63-generic. (the server is in production we don't really want to test 4.2 unstable) -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1495442 Title: XFS: kernel hangs multiple times in xlog_cil_force_lsn Status in linux package in Ubuntu: Incomplete Bug description: Ubuntu 14.04.3 LTS with Kernel 3.19.0-26-generic One of our servers running rsnapshot and nfs-kernel-server on XFS partitions crashed or hanged two sundays in a raw. Kernel 3.19.0-26-generic hangs multiple times (same time) in xlog_cil_force_lsn (XFS). I assume this bug is not related with bug report 979498. I had also this issue with kernel 3.16.0-46-generic or 3.13.0-57-generic but I have no log during the crash to determine which of the above kernel was running. I switched to kernel 3.19.0-26-generic because 3.16.0-48-generic was not working well with Intel Gigabit -- tx hang (bug 1492146) INFO: task kthreadd:2 blocked for more than 120 seconds. Tainted: G C 3.19.0-26-generic #28~14.04.1-Ubuntu "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. kthreaddD 8802362eb3a8 0 2 0 0x 8802362eb3a8 8802362c09d0 00013e80 8802362ebfd8 00013e80 880235c28000 8802362c09d0 88020001 8802362eb500 7fff 8802362eb4f8 8802362c09d0 Call Trace: [] schedule+0x29/0x70 [] schedule_timeout+0x20c/0x280 [] ? update_curr+0xe4/0x180 [] wait_for_completion+0xa4/0x170 [] ? wake_up_state+0x20/0x20 [] flush_work+0xed/0x1c0 [] ? destroy_worker+0x90/0x90 [] xlog_cil_force_lsn+0x7e/0x1f0 [xfs] [] ? __free_pages+0x27/0x30 [] ? __free_slab+0xd0/0x1f0 [] _xfs_log_force_lsn+0x5e/0x2d0 [xfs] [] xfs_log_force_lsn+0x2e/0x90 [xfs] [] ? xfs_iunpin_wait+0x19/0x20 [xfs] [] __xfs_iunpin_wait+0x8d/0x120 [xfs] [] ? autoremove_wake_function+0x40/0x40 [] xfs_iunpin_wait+0x19/0x20 [xfs] [] xfs_reclaim_inode+0x7c/0x340 [xfs] [] xfs_reclaim_inodes_ag+0x257/0x370 [xfs] [] xfs_reclaim_inodes_nr+0x33/0x40 [xfs] [] xfs_fs_free_cached_objects+0x15/0x20 [xfs] [] super_cache_scan+0x169/0x170 [] shrink_node_slabs+0x1d6/0x370 [] shrink_zone+0x20a/0x240 [] do_try_to_free_pages+0x155/0x440 [] ? zone_watermark_ok+0x1f/0x30 [] try_to_free_pages+0xba/0x150 [] __alloc_pages_nodemask+0x61b/0xa60 [] alloc_kmem_pages_node+0x6a/0x130 [] ? copy_process.part.26+0xf3/0x1c00 [] copy_process.part.26+0x113/0x1c00 [] ? dequeue_task_fair+0x44e/0x660 [] ? put_prev_entity+0x31/0x3f0 [] ? pick_next_task_fair+0x19c/0x880 [] ? kthread_create_on_node+0x1c0/0x1c0 [] ? put_prev_task_fair+0x2f/0x50 [] do_fork+0xd5/0x340 [] kernel_thread+0x26/0x30 [] kthreadd+0x15a/0x1c0 [] ? kthread_create_on_cpu+0x60/0x60 [] ret_from_fork+0x58/0x90 [] ? kthread_create_on_cpu+0x60/0x60 10:17:01 xyz CRON[59020]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly) 11:17:01 xyz CRON[59177]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly) 12:00:01 xyz CRON[59282]: (root) CMD (/usr/bin/rsnapshot -c /etc/rsnapshot.conf hourly) 12:03:05 xyz kernel: [616881.456247] INFO: task kthreadd:2 blocked for more than 120 seconds. 12:03:05 xyz kernel: [616881.456587] Tainted: G C 3.19.0-26-generic #28~14.04.1-Ubuntu ... 12:03:05 xyz kernel: [616881.457934] INFO: task kswapd0:57 blocked for more than 120 seconds. 12:03:05 xyz kernel: [616881.458271] Tainted: G C 3.19.0-26-generic #28~14.04.1-Ubuntu ... 12:03:05 xyz kernel: [616881.485629] INFO: task nfsd:2002 blocked for more than 120 seconds. 12:03:05 xyz kernel: [616881.492761] Tainted: G C 3.19.0-26-generic #28~14.04.1-Ubuntu ... 12:03:05 xyz kernel: [616881.523050] INFO: task snmpd:2155 blocked for more than 120 seconds. 12:03:05 xyz kernel: [616881.530783] Tainted: G C 3.19.0-26-generic #28~14.04.1-Ubuntu ... 12:03:05 xyz kernel: [616881.562789] INFO: task kworker/1:0:59030 blocked for more than 120 seconds. 12:03:05 xyz kernel: [616881.571082] Tainted: G C 3.19.0-26-generic #28~14.04.1-Ubuntu ... 12:03:05 xyz kernel: [616881.605747] INFO: task kworker/3:1H:59110 blocked for more than 120 seconds. 12:03:05 xyz kernel: [616881.615112] Tainted: G C 3.19.0-26-generic #28~14.04.1-Ubuntu ... 12:03:05 xyz kernel: [616881.652107] INFO: task rm:59284 blocked for more than 120 seconds. 12:03:05 xyz kernel: [616881.661441] Tainted: G C 3.19.0-26-generic #28~14.04.1-Ubuntu ... 12:05:05 xyz kernel: [617001.782228] INFO: task kswapd0:57 blocked for more than 120 seconds. 12:05:05 xyz kernel: [617001.791580]
[Kernel-packages] [Bug 1495442] Re: XFS: kernel hangs multiple times in xlog_cil_force_lsn
apport information ** Tags added: apport-collected staging trusty ** Description changed: Ubuntu 14.04.3 LTS with Kernel 3.19.0-26-generic One of our servers running rsnapshot and nfs-kernel-server on XFS partitions crashed or hanged two sundays in a raw. Kernel 3.19.0-26-generic hangs multiple times (same time) in xlog_cil_force_lsn (XFS). I assume this bug is not related with bug report 979498. I had also this issue with kernel 3.16.0-46-generic or 3.13.0-57-generic but I have no log during the crash to determine which of the above kernel was running. I switched to kernel 3.19.0-26-generic because 3.16.0-48-generic was not working well with Intel Gigabit -- tx hang (bug 1492146) INFO: task kthreadd:2 blocked for more than 120 seconds. Tainted: G C 3.19.0-26-generic #28~14.04.1-Ubuntu "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. kthreaddD 8802362eb3a8 0 2 0 0x 8802362eb3a8 8802362c09d0 00013e80 8802362ebfd8 00013e80 880235c28000 8802362c09d0 88020001 8802362eb500 7fff 8802362eb4f8 8802362c09d0 Call Trace: [] schedule+0x29/0x70 [] schedule_timeout+0x20c/0x280 [] ? update_curr+0xe4/0x180 [] wait_for_completion+0xa4/0x170 [] ? wake_up_state+0x20/0x20 [] flush_work+0xed/0x1c0 [] ? destroy_worker+0x90/0x90 [] xlog_cil_force_lsn+0x7e/0x1f0 [xfs] [] ? __free_pages+0x27/0x30 [] ? __free_slab+0xd0/0x1f0 [] _xfs_log_force_lsn+0x5e/0x2d0 [xfs] [] xfs_log_force_lsn+0x2e/0x90 [xfs] [] ? xfs_iunpin_wait+0x19/0x20 [xfs] [] __xfs_iunpin_wait+0x8d/0x120 [xfs] [] ? autoremove_wake_function+0x40/0x40 [] xfs_iunpin_wait+0x19/0x20 [xfs] [] xfs_reclaim_inode+0x7c/0x340 [xfs] [] xfs_reclaim_inodes_ag+0x257/0x370 [xfs] [] xfs_reclaim_inodes_nr+0x33/0x40 [xfs] [] xfs_fs_free_cached_objects+0x15/0x20 [xfs] [] super_cache_scan+0x169/0x170 [] shrink_node_slabs+0x1d6/0x370 [] shrink_zone+0x20a/0x240 [] do_try_to_free_pages+0x155/0x440 [] ? zone_watermark_ok+0x1f/0x30 [] try_to_free_pages+0xba/0x150 [] __alloc_pages_nodemask+0x61b/0xa60 [] alloc_kmem_pages_node+0x6a/0x130 [] ? copy_process.part.26+0xf3/0x1c00 [] copy_process.part.26+0x113/0x1c00 [] ? dequeue_task_fair+0x44e/0x660 [] ? put_prev_entity+0x31/0x3f0 [] ? pick_next_task_fair+0x19c/0x880 [] ? kthread_create_on_node+0x1c0/0x1c0 [] ? put_prev_task_fair+0x2f/0x50 [] do_fork+0xd5/0x340 [] kernel_thread+0x26/0x30 [] kthreadd+0x15a/0x1c0 [] ? kthread_create_on_cpu+0x60/0x60 [] ret_from_fork+0x58/0x90 [] ? kthread_create_on_cpu+0x60/0x60 10:17:01 xyz CRON[59020]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly) 11:17:01 xyz CRON[59177]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly) 12:00:01 xyz CRON[59282]: (root) CMD (/usr/bin/rsnapshot -c /etc/rsnapshot.conf hourly) 12:03:05 xyz kernel: [616881.456247] INFO: task kthreadd:2 blocked for more than 120 seconds. 12:03:05 xyz kernel: [616881.456587] Tainted: G C 3.19.0-26-generic #28~14.04.1-Ubuntu ... 12:03:05 xyz kernel: [616881.457934] INFO: task kswapd0:57 blocked for more than 120 seconds. 12:03:05 xyz kernel: [616881.458271] Tainted: G C 3.19.0-26-generic #28~14.04.1-Ubuntu ... 12:03:05 xyz kernel: [616881.485629] INFO: task nfsd:2002 blocked for more than 120 seconds. 12:03:05 xyz kernel: [616881.492761] Tainted: G C 3.19.0-26-generic #28~14.04.1-Ubuntu ... 12:03:05 xyz kernel: [616881.523050] INFO: task snmpd:2155 blocked for more than 120 seconds. 12:03:05 xyz kernel: [616881.530783] Tainted: G C 3.19.0-26-generic #28~14.04.1-Ubuntu ... 12:03:05 xyz kernel: [616881.562789] INFO: task kworker/1:0:59030 blocked for more than 120 seconds. 12:03:05 xyz kernel: [616881.571082] Tainted: G C 3.19.0-26-generic #28~14.04.1-Ubuntu ... 12:03:05 xyz kernel: [616881.605747] INFO: task kworker/3:1H:59110 blocked for more than 120 seconds. 12:03:05 xyz kernel: [616881.615112] Tainted: G C 3.19.0-26-generic #28~14.04.1-Ubuntu ... 12:03:05 xyz kernel: [616881.652107] INFO: task rm:59284 blocked for more than 120 seconds. 12:03:05 xyz kernel: [616881.661441] Tainted: G C 3.19.0-26-generic #28~14.04.1-Ubuntu ... 12:05:05 xyz kernel: [617001.782228] INFO: task kswapd0:57 blocked for more than 120 seconds. 12:05:05 xyz kernel: [617001.791580] Tainted: G C 3.19.0-26-generic #28~14.04.1-Ubuntu ... 12:05:05 xyz kernel: [617001.828636] INFO: task irqbalance:1353 blocked for more than 120 seconds. 12:05:05 xyz kernel: [617001.838019] Tainted: G C 3.19.0-26-generic #28~14.04.1-Ubuntu ... 12:05:05 xyz kernel:
[Kernel-packages] [Bug 1495442] Re: XFS: kernel hangs multiple times in xlog_cil_force_lsn
Would it be possible for you to test the latest upstream kernel? Refer to https://wiki.ubuntu.com/KernelMainlineBuilds . Please test the latest v4.2 kernel[0]. If this bug is fixed in the mainline kernel, please add the following tag 'kernel-fixed-upstream'. If the mainline kernel does not fix this bug, please add the tag: 'kernel-bug-exists-upstream'. Once testing of the upstream kernel is complete, please mark this bug as "Confirmed". Thanks in advance. [0] http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.2-unstable/ ** Changed in: linux (Ubuntu) Importance: Undecided => High -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1495442 Title: XFS: kernel hangs multiple times in xlog_cil_force_lsn Status in linux package in Ubuntu: Incomplete Bug description: Ubuntu 14.04.3 LTS with Kernel 3.19.0-26-generic One of our servers running rsnapshot and nfs-kernel-server on XFS partitions crashed or hanged two sundays in a raw. Kernel 3.19.0-26-generic hangs multiple times (same time) in xlog_cil_force_lsn (XFS). I assume this bug is not related with bug report 979498. I had also this issue with kernel 3.16.0-46-generic or 3.13.0-57-generic but I have no log during the crash to determine which of the above kernel was running. I switched to kernel 3.19.0-26-generic because 3.16.0-48-generic was not working well with Intel Gigabit -- tx hang (bug 1492146) INFO: task kthreadd:2 blocked for more than 120 seconds. Tainted: G C 3.19.0-26-generic #28~14.04.1-Ubuntu "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. kthreaddD 8802362eb3a8 0 2 0 0x 8802362eb3a8 8802362c09d0 00013e80 8802362ebfd8 00013e80 880235c28000 8802362c09d0 88020001 8802362eb500 7fff 8802362eb4f8 8802362c09d0 Call Trace: [] schedule+0x29/0x70 [] schedule_timeout+0x20c/0x280 [] ? update_curr+0xe4/0x180 [] wait_for_completion+0xa4/0x170 [] ? wake_up_state+0x20/0x20 [] flush_work+0xed/0x1c0 [] ? destroy_worker+0x90/0x90 [] xlog_cil_force_lsn+0x7e/0x1f0 [xfs] [] ? __free_pages+0x27/0x30 [] ? __free_slab+0xd0/0x1f0 [] _xfs_log_force_lsn+0x5e/0x2d0 [xfs] [] xfs_log_force_lsn+0x2e/0x90 [xfs] [] ? xfs_iunpin_wait+0x19/0x20 [xfs] [] __xfs_iunpin_wait+0x8d/0x120 [xfs] [] ? autoremove_wake_function+0x40/0x40 [] xfs_iunpin_wait+0x19/0x20 [xfs] [] xfs_reclaim_inode+0x7c/0x340 [xfs] [] xfs_reclaim_inodes_ag+0x257/0x370 [xfs] [] xfs_reclaim_inodes_nr+0x33/0x40 [xfs] [] xfs_fs_free_cached_objects+0x15/0x20 [xfs] [] super_cache_scan+0x169/0x170 [] shrink_node_slabs+0x1d6/0x370 [] shrink_zone+0x20a/0x240 [] do_try_to_free_pages+0x155/0x440 [] ? zone_watermark_ok+0x1f/0x30 [] try_to_free_pages+0xba/0x150 [] __alloc_pages_nodemask+0x61b/0xa60 [] alloc_kmem_pages_node+0x6a/0x130 [] ? copy_process.part.26+0xf3/0x1c00 [] copy_process.part.26+0x113/0x1c00 [] ? dequeue_task_fair+0x44e/0x660 [] ? put_prev_entity+0x31/0x3f0 [] ? pick_next_task_fair+0x19c/0x880 [] ? kthread_create_on_node+0x1c0/0x1c0 [] ? put_prev_task_fair+0x2f/0x50 [] do_fork+0xd5/0x340 [] kernel_thread+0x26/0x30 [] kthreadd+0x15a/0x1c0 [] ? kthread_create_on_cpu+0x60/0x60 [] ret_from_fork+0x58/0x90 [] ? kthread_create_on_cpu+0x60/0x60 10:17:01 xyz CRON[59020]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly) 11:17:01 xyz CRON[59177]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly) 12:00:01 xyz CRON[59282]: (root) CMD (/usr/bin/rsnapshot -c /etc/rsnapshot.conf hourly) 12:03:05 xyz kernel: [616881.456247] INFO: task kthreadd:2 blocked for more than 120 seconds. 12:03:05 xyz kernel: [616881.456587] Tainted: G C 3.19.0-26-generic #28~14.04.1-Ubuntu ... 12:03:05 xyz kernel: [616881.457934] INFO: task kswapd0:57 blocked for more than 120 seconds. 12:03:05 xyz kernel: [616881.458271] Tainted: G C 3.19.0-26-generic #28~14.04.1-Ubuntu ... 12:03:05 xyz kernel: [616881.485629] INFO: task nfsd:2002 blocked for more than 120 seconds. 12:03:05 xyz kernel: [616881.492761] Tainted: G C 3.19.0-26-generic #28~14.04.1-Ubuntu ... 12:03:05 xyz kernel: [616881.523050] INFO: task snmpd:2155 blocked for more than 120 seconds. 12:03:05 xyz kernel: [616881.530783] Tainted: G C 3.19.0-26-generic #28~14.04.1-Ubuntu ... 12:03:05 xyz kernel: [616881.562789] INFO: task kworker/1:0:59030 blocked for more than 120 seconds. 12:03:05 xyz kernel: [616881.571082] Tainted: G C 3.19.0-26-generic #28~14.04.1-Ubuntu ... 12:03:05 xyz kernel: [616881.605747] INFO: task kworker/3:1H:59110 blocked for more than 120
[Kernel-packages] [Bug 1495442] Re: XFS: kernel hangs multiple times in xlog_cil_force_lsn
We are also running into this bug on a couple of machines as well, which is causing a DOS on our machine since it blocks all IO and requires a reboot. We are running 3.19.0-28-generic also had the tx hang bug so we can't use the 3.16 kernel. I am not sure what triggers it, since it happens randomly and unpredictably. These are production machines so I can't do much debugging on it but want to confirm something seems to break. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/1495442 Title: XFS: kernel hangs multiple times in xlog_cil_force_lsn Status in linux package in Ubuntu: Incomplete Bug description: Ubuntu 14.04.3 LTS with Kernel 3.19.0-26-generic One of our servers running rsnapshot and nfs-kernel-server on XFS partitions crashed or hanged two sundays in a raw. Kernel 3.19.0-26-generic hangs multiple times (same time) in xlog_cil_force_lsn (XFS). I assume this bug is not related with bug report 979498. I had also this issue with kernel 3.16.0-46-generic or 3.13.0-57-generic but I have no log during the crash to determine which of the above kernel was running. I switched to kernel 3.19.0-26-generic because 3.16.0-48-generic was not working well with Intel Gigabit -- tx hang (bug 1492146) INFO: task kthreadd:2 blocked for more than 120 seconds. Tainted: G C 3.19.0-26-generic #28~14.04.1-Ubuntu "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. kthreaddD 8802362eb3a8 0 2 0 0x 8802362eb3a8 8802362c09d0 00013e80 8802362ebfd8 00013e80 880235c28000 8802362c09d0 88020001 8802362eb500 7fff 8802362eb4f8 8802362c09d0 Call Trace: [] schedule+0x29/0x70 [] schedule_timeout+0x20c/0x280 [] ? update_curr+0xe4/0x180 [] wait_for_completion+0xa4/0x170 [] ? wake_up_state+0x20/0x20 [] flush_work+0xed/0x1c0 [] ? destroy_worker+0x90/0x90 [] xlog_cil_force_lsn+0x7e/0x1f0 [xfs] [] ? __free_pages+0x27/0x30 [] ? __free_slab+0xd0/0x1f0 [] _xfs_log_force_lsn+0x5e/0x2d0 [xfs] [] xfs_log_force_lsn+0x2e/0x90 [xfs] [] ? xfs_iunpin_wait+0x19/0x20 [xfs] [] __xfs_iunpin_wait+0x8d/0x120 [xfs] [] ? autoremove_wake_function+0x40/0x40 [] xfs_iunpin_wait+0x19/0x20 [xfs] [] xfs_reclaim_inode+0x7c/0x340 [xfs] [] xfs_reclaim_inodes_ag+0x257/0x370 [xfs] [] xfs_reclaim_inodes_nr+0x33/0x40 [xfs] [] xfs_fs_free_cached_objects+0x15/0x20 [xfs] [] super_cache_scan+0x169/0x170 [] shrink_node_slabs+0x1d6/0x370 [] shrink_zone+0x20a/0x240 [] do_try_to_free_pages+0x155/0x440 [] ? zone_watermark_ok+0x1f/0x30 [] try_to_free_pages+0xba/0x150 [] __alloc_pages_nodemask+0x61b/0xa60 [] alloc_kmem_pages_node+0x6a/0x130 [] ? copy_process.part.26+0xf3/0x1c00 [] copy_process.part.26+0x113/0x1c00 [] ? dequeue_task_fair+0x44e/0x660 [] ? put_prev_entity+0x31/0x3f0 [] ? pick_next_task_fair+0x19c/0x880 [] ? kthread_create_on_node+0x1c0/0x1c0 [] ? put_prev_task_fair+0x2f/0x50 [] do_fork+0xd5/0x340 [] kernel_thread+0x26/0x30 [] kthreadd+0x15a/0x1c0 [] ? kthread_create_on_cpu+0x60/0x60 [] ret_from_fork+0x58/0x90 [] ? kthread_create_on_cpu+0x60/0x60 10:17:01 xyz CRON[59020]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly) 11:17:01 xyz CRON[59177]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly) 12:00:01 xyz CRON[59282]: (root) CMD (/usr/bin/rsnapshot -c /etc/rsnapshot.conf hourly) 12:03:05 xyz kernel: [616881.456247] INFO: task kthreadd:2 blocked for more than 120 seconds. 12:03:05 xyz kernel: [616881.456587] Tainted: G C 3.19.0-26-generic #28~14.04.1-Ubuntu ... 12:03:05 xyz kernel: [616881.457934] INFO: task kswapd0:57 blocked for more than 120 seconds. 12:03:05 xyz kernel: [616881.458271] Tainted: G C 3.19.0-26-generic #28~14.04.1-Ubuntu ... 12:03:05 xyz kernel: [616881.485629] INFO: task nfsd:2002 blocked for more than 120 seconds. 12:03:05 xyz kernel: [616881.492761] Tainted: G C 3.19.0-26-generic #28~14.04.1-Ubuntu ... 12:03:05 xyz kernel: [616881.523050] INFO: task snmpd:2155 blocked for more than 120 seconds. 12:03:05 xyz kernel: [616881.530783] Tainted: G C 3.19.0-26-generic #28~14.04.1-Ubuntu ... 12:03:05 xyz kernel: [616881.562789] INFO: task kworker/1:0:59030 blocked for more than 120 seconds. 12:03:05 xyz kernel: [616881.571082] Tainted: G C 3.19.0-26-generic #28~14.04.1-Ubuntu ... 12:03:05 xyz kernel: [616881.605747] INFO: task kworker/3:1H:59110 blocked for more than 120 seconds. 12:03:05 xyz kernel: [616881.615112] Tainted: G C 3.19.0-26-generic #28~14.04.1-Ubuntu ... 12:03:05 xyz kernel: [616881.652107]
