** Changed in: linux (Ubuntu)
Assignee: Tim Gardner (timg-tpi) => (unassigned)
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1659111
Title:
UbuntuKVM guest crashed while running I/O stress test with Ubuntu
kernel 4.4.0-47-generic
Status in linux package in Ubuntu:
In Progress
Status in linux source package in Xenial:
Fix Released
Status in linux source package in Yakkety:
Won't Fix
Status in linux source package in Zesty:
Incomplete
Bug description:
Attn. Canonical: For your awareness only at this time.
== Comment: #0 - LEKSHMI C. PILLAI - 2016-11-22 03:49:38 ==
Machine INFO
KVM HOST: luckyv1
Guest :lucky05
lucky05 crashed while running the I/O stress test for SAN disks.
Installed lucky05 and enabled the xmon on that.After that started the
RAW disk test on around 50 disks.After 6-7 hours after running,Now
machine dropped into xmon.
Logs:
[25023.224182] Unable to handle kernel paging request for data at address
0x00000000
[25023.224257] Faulting instruction address: 0xc000000000324c60
cpu 0x3: Vector: 300 (Data Access) at [c0000000fffc3620]
pc: c000000000324c60: locked_inode_to_wb_and_lock_list+0x50/0x290
lr: c00000000032831c: writeback_sb_inodes+0x30c/0x590
sp: c0000000fffc38a0
msr: 8000000100009033
dar: 0
dsisr: 40000000
current = 0xc0000000ff99e470
paca = 0xc00000000fb41c80 softe: 0 irq_happened: 0x01
pid = 14736, comm = kworker/u16:8
enter ? for help
[c0000000fffc3900] c00000000032831c writeback_sb_inodes+0x30c/0x590
[c0000000fffc3a10] c000000000328684 __writeback_inodes_wb+0xe4/0x150
[c0000000fffc3a70] c000000000328aec wb_writeback+0x30c/0x450
[c0000000fffc3b40] c0000000003296b4 wb_workfn+0x264/0x570
[c0000000fffc3c50] c0000000000dd930 process_one_work+0x1e0/0x5a0
[c0000000fffc3ce0] c0000000000dde84 worker_thread+0x194/0x680
[c0000000fffc3d80] c0000000000e6980 kthread+0x110/0x130
[c0000000fffc3e30] c000000000009538 ret_from_kernel_thread+0x5c/0xa4
3:mon> f
3:mon> th
[c0000000fffc3900] c00000000032831c writeback_sb_inodes+0x30c/0x590
[c0000000fffc3a10] c000000000328684 __writeback_inodes_wb+0xe4/0x150
[c0000000fffc3a70] c000000000328aec wb_writeback+0x30c/0x450
[c0000000fffc3b40] c0000000003296b4 wb_workfn+0x264/0x570
[c0000000fffc3c50] c0000000000dd930 process_one_work+0x1e0/0x5a0
[c0000000fffc3ce0] c0000000000dde84 worker_thread+0x194/0x680
[c0000000fffc3d80] c0000000000e6980 kthread+0x110/0x130
[c0000000fffc3e30] c000000000009538 ret_from_kernel_thread+0x5c/0xa4
3:mon> sh
[27384.651055] INFO: rcu_sched detected stalls on CPUs/tasks:
[27384.651220] (detected by 4, t=40598 jiffies, g=2849830, c=2849829, q=992)
[27384.651286] All QSes seen, last rcu_sched kthread activity 40596
(4301188714-4301148118), jiffies_till_next_fqs=1, root ->qsmask 0x0
[27384.651501] rcu_sched kthread starved for 40596 jiffies! g2849830 c2849829
f0x2 s3 ->state=0x0
[27384.651747] INFO: rcu_sched detected stalls on CPUs/tasks:
[27384.651905] (detected by 4, t=590354 jiffies, g=2849830, c=2849829,
q=1285)
[27384.652012] All QSes seen, last rcu_sched kthread activity 590352
(4301738470-4301148118), jiffies_till_next_fqs=1, root ->qsmask 0x0
[27384.652191] rcu_sched kthread starved for 590352 jiffies! g2849830
c2849829 f0x2 s3 ->state=0x0
[27384.730645] Unable to handle kernel paging request for data at address
0xffffffffffffffd8
[27384.730781] Faulting instruction address: 0xc0000000000e7258
cpu 0x3: Vector: 300 (Data Access) at [c0000000fffc3000]
pc: c0000000000e7258: kthread_data+0x28/0x40
lr: c0000000000de940: wq_worker_sleeping+0x30/0x110
sp: c0000000fffc3280
msr: 8000000100009033
dar: ffffffffffffffd8
dsisr: 40000000
current = 0xc0000000ff99e470
paca = 0xc00000000fb41c80 softe: 0 irq_happened: 0x01
pid = 14736, comm = kworker/u16:8
enter ? for help
== Comment: #1 - LEKSHMI C. PILLAI - 2016-11-22 04:05:41 ==
3:mon> th
[c0000000fffc32b0] c0000000000de940 wq_worker_sleeping+0x30/0x110
[c0000000fffc32f0] c000000000af31bc __schedule+0x6ec/0x990
[c0000000fffc33c0] c000000000af34a8 schedule+0x48/0xc0
[c0000000fffc33f0] c0000000000bd3d0 do_exit+0x760/0xc30
[c0000000fffc34b0] c000000000020bf4 die+0x314/0x470
[c0000000fffc3540] c000000000050d98 bad_page_fault+0xd8/0x150
[c0000000fffc35b0] c000000000008680 handle_page_fault+0x2c/0x30
--- Exception: 300 (Data Access) at c000000000324c60
locked_inode_to_wb_and_lock_list+0x50/0x290
[c0000000fffc3900] c00000000032831c writeback_sb_inodes+0x30c/0x590
[c0000000fffc3a10] c000000000328684 __writeback_inodes_wb+0xe4/0x150
[c0000000fffc3a70] c000000000328aec wb_writeback+0x30c/0x450
[c0000000fffc3b40] c0000000003296b4 wb_workfn+0x264/0x570
[c0000000fffc3c50] c0000000000dd930 process_one_work+0x1e0/0x5a0
[c0000000fffc3ce0] c0000000000dde84 worker_thread+0x194/0x680
[c0000000fffc3d80] c0000000000e6980 kthread+0x110/0x130
[c0000000fffc3e30] c000000000009538 ret_from_kernel_thread+0x5c/0xa4
3:mon>
== Comment: #6 - Laurent Dufour - 2016-11-23 03:00:16 ==
Logged in luckyv1, found a lot of ipr issue on this node:
[525973.896624] qla2xxx 0005:09:00.0: vpd r/w failed. This is likely a
firmware bug on this device. Contact the card vendor for a firmware update
[525973.956619] qla2xxx 0005:09:00.1: vpd r/w failed. This is likely a
firmware bug on this device. Contact the card vendor for a firmware update
[529433.834853] ipr 0001:04:00.0: FFFE: Soft device bus error recovered by
the IOA
[529433.834867] ipr: -----Failing Device Information-----
[529433.834870] ipr: World Wide Unique ID: 500507605EC10C000000000000000000
[529433.834873] ipr: Device Resource Path: FF
[529433.834875] ipr: Primary Problem Description: Command Timeout
[529433.834878] ipr: Secondary Problem Description: Command timeout expired
[529433.834880] ipr: SCSI Sense Data:
[529433.834882] ipr: 00000000: 00000000 00000000 00000000 00000000
[529433.834884] ipr: 00000010: 00000000 00000000 00000000 00000000
[529433.834886] ipr: SCSI Command Descriptor Block:
[529433.834889] ipr: 00000000: 9E120004 0F000000 00000000 0020AD00
[529433.834891] ipr: Additional IOA Data:
[529433.834893] ipr: 00000000: 4646001C 44010007 00050000 04700002
[529433.834895] ipr: 00000010: 3B894A49 1EE620CC 04700002 49574631
[529433.834897] ipr: 00000020: 455300CC 06B00027 00000020 84000000
[529433.834899] ipr: 00000030: 00000000 05801000 0B29A7C0 00000000
[529433.834901] ipr: 00000040: 00000000 00000000 00000000 00000000
[529433.834904] ipr: 00000050: 00000000 00000000 00000000 00000000
[529433.834906] ipr: 00000060: 00000000 00000000 00000000 00000000
[529433.834908] ipr: 00000070: 00000000 00000000 00000000 00000000
[529433.834910] ipr: 00000080: 00000000 00000000 00000000 00000000
[529433.834912] ipr: 00000090: 00000000 00000000 00000000 00000000
[529433.834914] ipr: 000000A0: 00000000 D4000018 80000000 FFFFFFFF
[529433.834917] ipr: 000000B0: FFFFFFFF 00000000 0980EC21 00000000
[529433.834919] ipr: 000000C0: 00000000 00000000 01769A24 00000000
[529433.834921] ipr: 000000D0: 01D3C300 E0050000 FFFFFFFE 0B5A0000
[529433.834923] ipr: 000000E0: 00000000 9E120004 0F000000 00000000
[529433.834926] ipr: 000000F0: 43440010 9E120004 0F000000 00000000
[529433.834928] ipr: 00000100: 0020AD00 45480010 0100E038 9E12FFFF
[529433.834930] ipr: 00000110: 01080002 00000000 45540004 00001463
In addition there are some NFS issue reported:
[563034.817901] nfs: server 10.33.11.31 not responding, timed out
[563405.504308] nfs: server 10.33.11.31 not responding, timed out
This said, chig5 enter xmon due to a bad pointer in the kernel:
3:mon> e
cpu 0x3: Vector: 300 (Data Access) at [c0000000fffc3000]
pc: c0000000000e7258: kthread_data+0x28/0x40
lr: c0000000000de940: wq_worker_sleeping+0x30/0x110
sp: c0000000fffc3280
msr: 8000000100009033
dar: ffffffffffffffd8
dsisr: 40000000
current = 0xc0000000ff99e470
paca = 0xc00000000fb41c80 softe: 0 irq_happened: 0x01
pid = 14736, comm = kworker/u16:8
3:mon> th
[c0000000fffc32b0] c0000000000de940 wq_worker_sleeping+0x30/0x110
[c0000000fffc32f0] c000000000af31bc __schedule+0x6ec/0x990
[c0000000fffc33c0] c000000000af34a8 schedule+0x48/0xc0
[c0000000fffc33f0] c0000000000bd3d0 do_exit+0x760/0xc30
[c0000000fffc34b0] c000000000020bf4 die+0x314/0x470
[c0000000fffc3540] c000000000050d98 bad_page_fault+0xd8/0x150
[c0000000fffc35b0] c000000000008680 handle_page_fault+0x2c/0x30
--- Exception: 300 (Data Access) at c000000000324c60
locked_inode_to_wb_and_lock_list+0x50/0x290
[c0000000fffc3900] c00000000032831c writeback_sb_inodes+0x30c/0x590
[c0000000fffc3a10] c000000000328684 __writeback_inodes_wb+0xe4/0x150
[c0000000fffc3a70] c000000000328aec wb_writeback+0x30c/0x450
[c0000000fffc3b40] c0000000003296b4 wb_workfn+0x264/0x570
[c0000000fffc3c50] c0000000000dd930 process_one_work+0x1e0/0x5a0
[c0000000fffc3ce0] c0000000000dde84 worker_thread+0x194/0x680
[c0000000fffc3d80] c0000000000e6980 kthread+0x110/0x130
[c0000000fffc3e30] c000000000009538 ret_from_kernel_thread+0x5c/0xa4
Looking at the other guest as Lekshmi mentioned that all the guests
are crashing.
== Comment: #7 - Laurent Dufour - 2016-11-23 03:24:34 ==
The guest lucky01 (4.4.0-47-generic) is fine :
root@lucky01:/Blast# date
Wed Nov 23 03:04:23 CST 2016
The guest lucky02 (4.4.0-47generic) has entered xmon due to the same issue as
lukcy05:
7:mon> e
cpu 0x7: Vector: 300 (Data Access) at [c0000001f265b620]
pc: c000000000324c60: locked_inode_to_wb_and_lock_list+0x50/0x290
lr: c00000000032831c: writeback_sb_inodes+0x30c/0x590
sp: c0000001f265b8a0
msr: 8000000100009033
dar: 0
dsisr: 40000000
current = 0xc0000001f222fcc0
paca = 0xc00000000fb44280 softe: 0 irq_happened: 0x01
pid = 12062, comm = kworker/u16:3
7:mon> t
[c0000001f265b900] c00000000032831c writeback_sb_inodes+0x30c/0x590
[c0000001f265ba10] c000000000328684 __writeback_inodes_wb+0xe4/0x150
[c0000001f265ba70] c000000000328aec wb_writeback+0x30c/0x450
[c0000001f265bb40] c0000000003296b4 wb_workfn+0x264/0x570
[c0000001f265bc50] c0000000000dd930 process_one_work+0x1e0/0x5a0
[c0000001f265bce0] c0000000000dde84 worker_thread+0x194/0x680
[c0000001f265bd80] c0000000000e6980 kthread+0x110/0x130
[c0000001f265be30] c000000000009538 ret_from_kernel_thread+0x5c/0xa4
--- Exception: 0 at 0000000000000000
The guest lucky03 didn't enter xmon but is not responding any more.
Unfornately sysrq is not enabled on this guest. There are still some activity
on this guest.
root@luckyv1:~# virsh qemu-monitor-command --hmp lucky03 'info cpus'
* CPU #0: nip=0xc0000000001035e0 thread_id=76434
CPU #1: nip=0xc0000000000863dc thread_id=76435
CPU #2: nip=0xc0000000000863dc thread_id=76436
CPU #3: nip=0xc0000000000863dc thread_id=76437
CPU #4: nip=0xc0000000000863dc thread_id=76439
CPU #5: nip=0xc0000000000863dc thread_id=76440
CPU #6: nip=0x0000000010072f68 thread_id=76441
CPU #7: nip=0xc0000000000863dc thread_id=76442
The guest lucky04 is not responding but neither enter xmon, but sysrq are not
enabled on this node.
But the node seems to be still active:
root@luckyv1:~# virsh qemu-monitor-command --hmp lucky04 'info cpus'
* CPU #0: nip=0xc000000000af8834 thread_id=68201
CPU #1: nip=0xc0000000000863dc thread_id=68202
CPU #2: nip=0xc0000000000645ac thread_id=68203
CPU #3: nip=0xc0000000000863dc thread_id=68204
CPU #4: nip=0xc0000000000863dc thread_id=68205
CPU #5: nip=0xc0000000000863dc thread_id=68206
CPU #6: nip=0xc000000000064590 thread_id=68207
CPU #7: nip=0xc000000000af8904 thread_id=68208
The guest lucky06 is alive:
root@lucky06:/# cat /proc/version; date
Linux version 4.4.0-47-generic (buildd@bos01-ppc64el-008) (gcc version 5.4.0
20160609 (Ubuntu/IBM 5.4.0-6ubuntu1~16.04.2) ) #68-Ubuntu SMP Wed Oct 26
19:38:24 UTC 2016
Wed Nov 23 03:20:19 CST 2016
To summarize:
lucky01 good
lucky02 panic in locked_inode_to_wb_and_lock_list()
lucky03 not responding but still active
lucky04 not responding but still active
lucky05 panic in locked_inode_to_wb_and_lock_list()
lucky06 good
== Comment: #10 - Laurent Dufour - 2016-11-24 10:27:52 ==
Here the data I captured on lucky02 which did panic the way lucky05 did.
CPU 7 panic due to a data access error:
7:mon> e
cpu 0x7: Vector: 300 (Data Access) at [c0000001f265b620]
pc: c000000000324c60: locked_inode_to_wb_and_lock_list+0x50/0x290
lr: c00000000032831c: writeback_sb_inodes+0x30c/0x590
sp: c0000001f265b8a0
msr: 8000000100009033
dar: 0
dsisr: 40000000
current = 0xc0000001f222fcc0
paca = 0xc00000000fb44280 softe: 0 irq_happened: 0x01
pid = 12062, comm = kworker/u16:3
7:mon> r
R00 = c00000000032831c R16 = c0000001fc972ef8
R01 = c0000001f265b8a0 R17 = c0000001fc972e70
R02 = c0000000015c6a00 R18 = c0000001fc972f60
R03 = c0000001fc972e70 R19 = 0000000000000000
R04 = c0000001f2230700 R20 = 0000000000000000
R05 = 0000000000000000 R21 = c0000001f2658000
R06 = 00000001fef30000 R22 = c0000001f35d5c88
R07 = 000108f684c40713 R23 = c0000001f35d5c68
R08 = 0000000000000000 R24 = 0000000000000000
R09 = 0000000000000000 R25 = c0000001fc972ef8
R10 = 0000000080000007 R26 = 0000000000000000
R11 = 00000000030883ec R27 = 0000000000000000
R12 = 0000000000000000 R28 = 0000000000000001
R13 = c00000000fb44280 R29 = c0000001fc972e70
R14 = c0000000000e6878 R30 = c0000001f265bba0
R15 = 0000000000000000 R31 = 0000000000000000
pc = c000000000324c60 locked_inode_to_wb_and_lock_list+0x50/0x290
cfar= 00003fff9647a5a8
lr = c00000000032831c writeback_sb_inodes+0x30c/0x590
msr = 8000000100009033 cr = 24652882
ctr = c000000000110b50 xer = 0000000020000000 trap = 300
dar = 0000000000000000 dsisr = 40000000
7:mon> t
[c0000001f265b900] c00000000032831c writeback_sb_inodes+0x30c/0x590
[c0000001f265ba10] c000000000328684 __writeback_inodes_wb+0xe4/0x150
[c0000001f265ba70] c000000000328aec wb_writeback+0x30c/0x450
[c0000001f265bb40] c0000000003296b4 wb_workfn+0x264/0x570
[c0000001f265bc50] c0000000000dd930 process_one_work+0x1e0/0x5a0
[c0000001f265bce0] c0000000000dde84 worker_thread+0x194/0x680
[c0000001f265bd80] c0000000000e6980 kthread+0x110/0x130
[c0000001f265be30] c000000000009538 ret_from_kernel_thread+0x5c/0xa4
The system tried to access data pointed by r31 which contains data retrieved
from the inode address stored in r29.
The panic happened during the inline call to wb_get when inode->i_wb is used.
So here inode->i_wb is null which is not expeted to happen.
At this time, CPU 6 is waiting for the same inode's spinlock inode->i_lock to
be released here:
6:mon> t
[link register ] c000000000064624 __spin_yield+0xb4/0xc0
[c0000000fdb93900] c0000000fdb93940 (unreliable)
[c0000000fdb93970] c000000000af8968 _raw_spin_lock+0xd8/0xe0
[c0000000fdb939a0] c000000000327330 __mark_inode_dirty+0xd0/0x4a0
[c0000000fdb93a20] c0000000003326f0 mark_buffer_dirty+0x1f0/0x210
[c0000000fdb93a60] c000000000334ff0 __block_commit_write.isra.7+0xf0/0x170
[c0000000fdb93ad0] c00000000033513c block_write_end+0x7c/0x100
[c0000000fdb93b20] c00000000033a340 blkdev_write_end+0x60/0xa0
[c0000000fdb93b80] c00000000022d340 generic_perform_write+0x180/0x280
[c0000000fdb93c20] c00000000022f568 __generic_file_write_iter+0x208/0x250
[c0000000fdb93c80] c00000000033b498 blkdev_write_iter+0x98/0x160
[c0000000fdb93cf0] c0000000002e24a4 new_sync_write+0xc4/0x120
[c0000000fdb93d90] c0000000002e32a0 vfs_write+0xc0/0x230
[c0000000fdb93de0] c0000000002e42dc SyS_write+0x6c/0x110
[c0000000fdb93e30] c000000000009204 system_call+0x38/0xb4
--- Exception: c01 (System Call) at 00003fff944c6728
SP (3ffef9ffe0c0) is in userspace
The CPU 6 hold the inode->i_lock in the call to inode_to_wb_and_lock_list().
Why inode->i_wb is null ?
== Comment: #11 - Laurent Dufour - 2016-11-25 11:57:50 ==
I found that lucky03 hit the panic also.
I took a closer look and it seems that there is a lock / memory barrier issue
around between the code run in locked_inode_to_wb_and_lock_list() and another
CPU. I found that the CPU 5 was running 'latest_blast' at the time the CPU 0
hit the panic. The same applied on lucky02.
== Comment: #13 - Laurent Dufour - 2016-12-05 07:32:30 ==
I did some test on luckyv05 and I was able to recreate it on 4.8 vanilla
kernel:
[113031.075540] Unable to handle kernel paging request for data at address
0x00000000
[113031.075614] Faulting instruction address: 0xc0000000003692e0
0:mon> t
[c0000000fb65f900] c00000000036cb6c writeback_sb_inodes+0x30c/0x590
[c0000000fb65fa10] c00000000036ced4 __writeback_inodes_wb+0xe4/0x150
[c0000000fb65fa70] c00000000036d33c wb_writeback+0x30c/0x450
[c0000000fb65fb40] c00000000036e198 wb_workfn+0x268/0x580
[c0000000fb65fc50] c0000000000f3470 process_one_work+0x1e0/0x590
[c0000000fb65fce0] c0000000000f38c8 worker_thread+0xa8/0x660
[c0000000fb65fd80] c0000000000fc4b0 kthread+0x110/0x130
[c0000000fb65fe30] c0000000000098f0 ret_from_kernel_thread+0x5c/0x6c
--- Exception: 0 at 0000000000000000
0:mon> e
cpu 0x0: Vector: 300 (Data Access) at [c0000000fb65f620]
pc: c0000000003692e0: locked_inode_to_wb_and_lock_list+0x50/0x290
lr: c00000000036cb6c: writeback_sb_inodes+0x30c/0x590
sp: c0000000fb65f8a0
msr: 800000010280b033
dar: 0
dsisr: 40000000
current = 0xc0000001d69be400
paca = 0xc000000003480000 softe: 0 irq_happened: 0x01
pid = 18689, comm = kworker/u16:10
Linux version 4.8.0 (laurent@lucky05) (gcc version 5.4.0 20160609 (Ubuntu/IBM
5.4.0-6ubuntu1~16.04.4) ) #1 SMP Thu Dec 1 09:25:13 CST 2016
So this is not a Ubuntu's issue but a more global one which is not fixed by
the patch
https://patchwork.kernel.org/patch/9247955/
as expected while investigating the bug 142781.
== Comment: #17 - Laurent Dufour - 2016-12-07 03:22:05 ==
For the record, I also hit the bug with 4.9-rc8:
4:mon> t
[c000000012a7f900] c0000000003787cc writeback_sb_inodes+0x30c/0x590
[c000000012a7fa10] c000000000378b34 __writeback_inodes_wb+0xe4/0x150
[c000000012a7fa70] c000000000378f9c wb_writeback+0x30c/0x450
[c000000012a7fb40] c000000000379df8 wb_workfn+0x268/0x580
[c000000012a7fc50] c0000000000f8c20 process_one_work+0x1e0/0x590
[c000000012a7fce0] c0000000000f9078 worker_thread+0xa8/0x650
[c000000012a7fd80] c000000000101a30 kthread+0x110/0x130
[c000000012a7fe30] c00000000000c0e8 ret_from_kernel_thread+0x5c/0x74
4:mon> e
cpu 0x4: Vector: 300 (Data Access) at [c000000012a7f620]
pc: c000000000374f40: locked_inode_to_wb_and_lock_list+0x50/0x290
lr: c0000000003787cc: writeback_sb_inodes+0x30c/0x590
sp: c000000012a7f8a0
msr: 800000010280b033
dar: 0
dsisr: 40000000
current = 0xc000000011540000
paca = 0xc000000003482400 softe: 0 irq_happened: 0x01
pid = 8357, comm = kworker/u16:3
Linux version 4.9.0-rc8 (root@lucky05) (gcc version 5.4.0 20160609
(Ubuntu/IBM 5.4.0-6ubuntu1~16.04.4) ) #2 SMP Tue Dec 6 05:17:47 CST 2016
== Comment: #24 - Thiago Jung Bauermann - 2017-01-11 16:09:45 ==
Dan Willians posted on 01/06 a patch series which aims to solve this bug:
https://www.spinics.net/lists/linux-fsdevel/msg106092.html
Unfortunately, the kernel test robot found problems with it:
http://lkml.iu.edu/hypermail/linux/kernel/1701.1/00239.html
Still, I think it's useful to perform tests to confirm that:
1. v4.10 is still affected by the problem and
2. Dan's patches fix this bug.
Therefore, could you please reproduce this bug on the unmodified
v4.10-rc3 build below?
http://kernel.stglabs.ibm.com/~bauermann/bug149014/v4.10-rc3/
This will allow us to confirm point 1.
Then, can you please try to reproduce it with the build below?
http://kernel.stglabs.ibm.com/~bauermann/bug149014/fix-
backing_dev_info-lifetime-v2/
This one is v4.10-rc3 plus Dan Willian's two patches from my link
above applied to it.
== Comment: #28 - Lata Kuntal - 2017-01-16 01:34:05 ==
I am seeing the same crash issue on one of UbuntuKVM 16.04.02 guest gusg8.
Pasting the console logs below :
root@guskvm:~# virsh console gusg8 --force
Connected to domain gusg8
Escape character is ^]
0:mon>
0:mon>
0:mon> t
[c00000023d1ab900] c00000000036a41c writeback_sb_inodes+0x30c/0x590
[c00000023d1aba10] c00000000036a784 __writeback_inodes_wb+0xe4/0x150
[c00000023d1aba70] c00000000036abfc wb_writeback+0x30c/0x450
[c00000023d1abb40] c00000000036ba38 wb_workfn+0x268/0x580
[c00000023d1abc50] c0000000000ef5e8 process_one_work+0x1e8/0x5b0
[c00000023d1abce0] c0000000000efa58 worker_thread+0xa8/0x650
[c00000023d1abd80] c0000000000f8224 kthread+0x114/0x140
[c00000023d1abe30] c0000000000098f0 ret_from_kernel_thread+0x5c/0x6c
--- Exception: 0 at 0000000000000000
0:mon>
0:mon>
0:mon> d
0000000000000000 **************** **************** | |
0:mon> r
R00 = c00000000036a41c R16 = c00000027ca0e868
R01 = c00000023d1ab8a0 R17 = c00000027ca0e7e0
R02 = c0000000014a6600 R18 = c00000027ca0e8d0
R03 = c00000027ca0e7e0 R19 = 0000000000000000
R04 = c0000001b092e710 R20 = 0000000000000000
R05 = 0000000000000000 R21 = c00000023d1a8000
R06 = 000000027ee30000 R22 = c000000273aace50
R07 = 00001d0c11165f1a R23 = c000000273aace30
R08 = 0000000000000000 R24 = 0000000000000000
R09 = 0000000000000000 R25 = 0000000000000000
R10 = 0000000080000000 R26 = c00000027ca0e868
R11 = c0000000014daae0 R27 = 0000000000000000
R12 = 0000000000005500 R28 = 0000000000000001
R13 = c00000000fb80000 R29 = c00000027ca0e7e0
R14 = c0000000000f8118 R30 = c00000023d1abba0
R15 = 0000000000000000 R31 = 0000000000000000
pc = c000000000366be4 locked_inode_to_wb_and_lock_list+0x54/0x290
cfar= d000000004bbf2e4 xfs_buf_delwri_submit_buffers+0x1e4/0x2b0 [xfs]
lr = c00000000036a41c writeback_sb_inodes+0x30c/0x590
msr = 800000010280b033 cr = 24aa2882
ctr = c000000000122210 xer = 0000000020000000 trap = 300
dar = 0000000000000000 dsisr = 40000000
0:mon> c
cpus stopped: 0x0-0x3
0:mon> e
cpu 0x0: Vector: 300 (Data Access) at [c00000023d1ab620]
pc: c000000000366be4: locked_inode_to_wb_and_lock_list+0x54/0x290
lr: c00000000036a41c: writeback_sb_inodes+0x30c/0x590
sp: c00000023d1ab8a0
msr: 800000010280b033
dar: 0
dsisr: 40000000
current = 0xc0000001b092dc00
paca = 0xc00000000fb80000 softe: 0 irq_happened: 0x01
pid = 774, comm = kworker/u8:3
Linux version 4.8.0-34-generic (buildd@bos01-ppc64el-026) (gcc version 5.4.0
20160609 (Ubuntu/IBM 5.4.0-6ubuntu1~16.04.4) ) #36~16.04.1-Ubuntu SMP Wed Dec
21 18:53:20 UTC 2016 (Ubuntu 4.8.0-34.36~16.04.1-generic 4.8.11)
0:mon>
== Comment: #33 - Thiago Jung Bauermann - 2017-01-23 15:31:24 ==
Lekshmi mentioned that she wasn't able to reproduce this bug with kernel
4.10.0-rc3fixlifetime+, so I replied to Dan's patch series mentioning that it
fixes this bug:
https://www.spinics.net/lists/linux-fsdevel/msg106830.html
Let's see if he answers back with a status or thoughts regarding the
patch series.
== Comment: #34 - LEKSHMI C. PILLAI - 2017-01-24 00:26:22 ==
Hi
The fix worked with 4.10.0-rc3fixlifetime+ kernel.Need to know which
kernel the fix is going to be.and whether able to get the workaround
for 16.04.02 ie; kernel 4.8
Thanks
Lekshmi
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1659111/+subscriptions
--
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help : https://help.launchpad.net/ListHelp
