[Bug 1852122] Re: ocfs2-tools is causing kernel panics in Ubuntu Focal (Ubuntu-5.4.0-9.12)
Last good test: https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac /autopkgtest-eoan/eoan/amd64/o/ocfs2-tools/20190910_143817_4312f@/log.gz Kernel: 5.2.0-15-generic First failure: https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac /autopkgtest-eoan/eoan/amd64/o/ocfs2-tools/20190918_060126_bd321@/log.gz Kernel: 5.3.0-10-generic -- You received this bug notification because you are a member of Ubuntu Server, which is subscribed to the bug report. https://bugs.launchpad.net/bugs/1852122 Title: ocfs2-tools is causing kernel panics in Ubuntu Focal (Ubuntu-5.4.0-9.12) To manage notifications about this bug go to: https://bugs.launchpad.net/ocfs2-tools/+bug/1852122/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1852122] Re: ocfs2-tools is causing kernel panics in Ubuntu Focal (Ubuntu-5.4.0-9.12)
Waiting on upstream bug. -- You received this bug notification because you are a member of Ubuntu Server, which is subscribed to the bug report. https://bugs.launchpad.net/bugs/1852122 Title: ocfs2-tools is causing kernel panics in Ubuntu Focal (Ubuntu-5.4.0-9.12) To manage notifications about this bug go to: https://bugs.launchpad.net/ocfs2-tools/+bug/1852122/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1852122] Re: ocfs2-tools is causing kernel panics in Ubuntu Focal (Ubuntu-5.4.0-9.12)
** Tags added: update-excuse -- You received this bug notification because you are a member of Ubuntu Server, which is subscribed to the bug report. https://bugs.launchpad.net/bugs/1852122 Title: ocfs2-tools is causing kernel panics in Ubuntu Focal (Ubuntu-5.4.0-9.12) To manage notifications about this bug go to: https://bugs.launchpad.net/ocfs2-tools/+bug/1852122/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1852122] Re: ocfs2-tools is causing kernel panics in Ubuntu Focal (Ubuntu-5.4.0-9.12)
Thanks for checking this Andreas! Looks like upstream got a fix for the issue and Debian has confirmed it worked. With that, I'll check if it fixes the issue indeed and suggest the SRU to the kernel-team so we can unblock ocfs2-tools. -- You received this bug notification because you are a member of Ubuntu Server, which is subscribed to the bug report. https://bugs.launchpad.net/bugs/1852122 Title: ocfs2-tools is causing kernel panics in Ubuntu Focal (Ubuntu-5.4.0-9.12) To manage notifications about this bug go to: https://bugs.launchpad.net/ocfs2-tools/+bug/1852122/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1852122] Re: ocfs2-tools is causing kernel panics in Ubuntu Focal (Ubuntu-5.4.0-9.12)
The upstream fix is likely this:
>From b73eba2a867e10b9b4477738677341f3307c07bb Mon Sep 17 00:00:00 2001
From: Gang He
Date: Sat, 4 Jan 2020 13:00:22 -0800
Subject: [PATCH] ocfs2: fix the crash due to call ocfs2_get_dlm_debug once
less
Because ocfs2_get_dlm_debug() function is called once less here, ocfs2
file system will trigger the system crash, usually after ocfs2 file
system is unmounted.
This system crash is caused by a generic memory corruption, these crash
backtraces are not always the same, for exapmle,
ocfs2: Unmounting device (253,16) on (node 172167785)
general protection fault: [#1] SMP PTI
CPU: 3 PID: 14107 Comm: fence_legacy Kdump:
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996)
RIP: 0010:__kmalloc+0xa5/0x2a0
Code: 00 00 4d 8b 07 65 4d 8b
RSP: 0018:aa1fc094bbe8 EFLAGS: 00010286
RAX: RBX: d310a8800d7a3faf RCX:
RDX: RSI: 0dc0 RDI: 96e68fc036c0
RBP: d310a8800d7a3faf R08: 96e6ffdb10a0 R09: 752e7079
R10: 0001c513 R11: 04091041 R12: 0dc0
R13: 0039 R14: 96e68fc036c0 R15: 96e68fc036c0
FS: 7f699dfba540() GS:96e6ffd8() knlGS:0
CS: 0010 DS: ES: CR0: 80050033
CR2: 55f3a9d9b768 CR3: 2cd1c000 CR4: 06e0
Call Trace:
ext4_htree_store_dirent+0x35/0x100 [ext4]
htree_dirblock_to_tree+0xea/0x290 [ext4]
ext4_htree_fill_tree+0x1c1/0x2d0 [ext4]
ext4_readdir+0x67c/0x9d0 [ext4]
iterate_dir+0x8d/0x1a0
__x64_sys_getdents+0xab/0x130
do_syscall_64+0x60/0x1f0
entry_SYSCALL_64_after_hwframe+0x49/0xbe
RIP: 0033:0x7f699d33a9fb
This regression problem was introduced by commit e581595ea29c ("ocfs: no
need to check return value of debugfs_create functions").
Link: http://lkml.kernel.org/r/20191225061501.13587-1-...@suse.com
Fixes: e581595ea29c ("ocfs: no need to check return value of debugfs_create
functions")
Signed-off-by: Gang He
Acked-by: Joseph Qi
Cc: Mark Fasheh
Cc: Joel Becker
Cc: Junxiao Bi
Cc: Changwei Ge
Cc: Gang He
Cc: Jun Piao
Cc: [5.3+]
Signed-off-by: Andrew Morton
Signed-off-by: Linus Torvalds
as reported in upstream bug. Giving it a try to finally suggest as a SRU
to the kernel team.
** Changed in: ocfs2-tools (Ubuntu Eoan)
Status: In Progress => Invalid
** Changed in: ocfs2-tools (Ubuntu Focal)
Status: In Progress => Invalid
--
You received this bug notification because you are a member of Ubuntu
Server, which is subscribed to the bug report.
https://bugs.launchpad.net/bugs/1852122
Title:
ocfs2-tools is causing kernel panics in Ubuntu Focal
(Ubuntu-5.4.0-9.12)
To manage notifications about this bug go to:
https://bugs.launchpad.net/ocfs2-tools/+bug/1852122/+subscriptions
--
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at:
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1852122] Re: ocfs2-tools is causing kernel panics in Ubuntu Focal (Ubuntu-5.4.0-9.12)
Yep, fixes the issue: [ 25.249831] ocfs2: Registered cluster interface o2cb [ 25.257661] OCFS2 User DLM kernel interface loaded [ 25.265196] o2hb: Heartbeat mode set to local [ 34.231435] o2dlm: Joining domain F67405C564CB4A7CAEBA6F6ACCA2C82F [ 34.231436] ( [ 34.231437] 0 [ 34.231438] ) 1 nodes [ 34.231825] JBD2: Ignoring recovery information on journal [ 34.233058] ocfs2: Mounting device (7,0) on (node 0, slot 0) with ordered data mode. [ 38.246001] o2dlm: Leaving domain F67405C564CB4A7CAEBA6F6ACCA2C82F [ 38.247583] ocfs2: Unmounting device (7,0) on (node 0) [ 50.998435] ocfs2: Unregistered cluster interface o2cb [ 51.117395] ocfs2: Registered cluster interface o2cb [ 51.124916] OCFS2 User DLM kernel interface loaded [ 51.131192] o2hb: Heartbeat mode set to local [ 59.999672] o2dlm: Joining domain 1D1FFAA94E654FE6B94AA0E44029CE9E [ 59.999674] ( [ 59.999675] 0 [ 59.999676] ) 1 nodes [ 60.000252] JBD2: Ignoring recovery information on journal [ 60.001543] ocfs2: Mounting device (7,0) on (node 0, slot 0) with ordered data mode. [ 64.043638] o2dlm: Leaving domain 1D1FFAA94E654FE6B94AA0E44029CE9E [ 64.045484] ocfs2: Unmounting device (7,0) on (node 0) [ 70.866574] random: crng init done [ 70.866586] random: 7 urandom warning(s) missed due to ratelimiting I'll propose patch to kernel team. -- You received this bug notification because you are a member of Ubuntu Server, which is subscribed to the bug report. https://bugs.launchpad.net/bugs/1852122 Title: ocfs2-tools is causing kernel panics in Ubuntu Focal (Ubuntu-5.4.0-9.12) To manage notifications about this bug go to: https://bugs.launchpad.net/ocfs2-tools/+bug/1852122/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1852122] Re: ocfs2-tools is causing kernel panics in Ubuntu Focal (Ubuntu-5.4.0-9.12)
** Description changed: + + [Impact] + + * Umounts to OCFS2 filesystems will cause a kernel crash in any kernel + containing commit e581595ea29c (v5.3-rc1) and not containing its fix + b73eba2a867e (v5.5-rc5). + + [Test Case] + + * ocfs2_reproducer.sh (attached to this bug) + + [Regression Potential] + + * it could cause ocfs2 issues, so I guess its a low impact considering amount of users. + * Its a straightforward fix identified by upstream developer and a clean cherry-pick for us. + + [Other Info] + + * Original description: + I noticed the tests for ocfs2-tools/1.8.6-1ubuntu1 were constantly retrying themselves. It's a feature we have so that transient / occasional failures are auto-retried, but it's misfiring here because we're not detecting that it's a consistent failure. That particular bug is fixed, but it means that ocfs2-tools is failing on ppc64el. Here's the important part of the log, full output attached. [ 85.605738] BUG: Unable to handle kernel data access at 0x01744098 - [ 85.605850] Faulting instruction address: 0xc0e81168 - [ 85.605901] Oops: Kernel access of bad area, sig: 11 [#1] - [ 85.605970] LE PAGE_SIZE=64K MMU=Hash SMP NR_CPUS=2048 NUMA pSeries - [ 85.606029] Modules linked in: ocfs2 quota_tree ocfs2_dlmfs ocfs2_stack_o2cb ocfs2_dlm ocfs2_nodemanager ocfs2_stackglue iptable_mangle xt_TCPMSS xt_tcpudp bpfilter dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua vmx_crypto crct10dif_vpmsum sch_fq_codel ip_tables x_tables autofs4 btrfs xor zstd_compress raid6_pq libcrc32c crc32c_vpmsum virtio_net virtio_blk net_failover failover - [ 85.606291] CPU: 0 PID: 1 Comm: systemd Not tainted 5.3.0-18-generic #19-Ubuntu - [ 85.606350] NIP: c0e81168 LR: c054f240 CTR: - [ 85.606410] REGS: c0005a3e3700 TRAP: 0300 Not tainted (5.3.0-18-generic) - [ 85.606469] MSR: 80009033 CR: 28024448 XER: - [ 85.606531] CFAR: 701f9806f638 DAR: 01744098 DSISR: 4000 IRQMASK: 0 - [ 85.606531] GPR00: 7374 c0005a3e3990 c19c9100 c0004fe462a8 - [ 85.606531] GPR04: c0005856d840 000e 74656772 c0004fe4a568 - [ 85.606531] GPR08: c00058568004 01744090 - [ 85.606531] GPR12: e8086002 c1d6 7fffddd522d0 - [ 85.606531] GPR16: c755e07c - [ 85.606531] GPR20: c000598caca8 c0005a3e3a58 c00058292f00 - [ 85.606531] GPR24: c0eea710 c0005856d840 c755e074 - [ 85.606531] GPR28: 6518907d c0005a3e3a68 c0004fe4b160 027c47b6 - [ 85.607079] NIP [c0e81168] rb_insert_color+0x18/0x1c0 -
[Bug 1852122] Re: ocfs2-tools is causing kernel panics in Ubuntu Focal (Ubuntu-5.4.0-9.12)
Patches for Focal and SRU were sent to kernel-t...@lists.ubuntu.com: https://lists.ubuntu.com/archives/kernel-team/2020-January/106845.html https://lists.ubuntu.com/archives/kernel-team/2020-January/106846.html -- You received this bug notification because you are a member of Ubuntu Server, which is subscribed to the bug report. https://bugs.launchpad.net/bugs/1852122 Title: ocfs2-tools is causing kernel panics in Ubuntu Focal (Ubuntu-5.4.0-9.12) To manage notifications about this bug go to: https://bugs.launchpad.net/ocfs2-tools/+bug/1852122/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1852122] Re: ocfs2-tools is causing kernel panics in Ubuntu Focal (Ubuntu-5.4.0-9.12)
Eoan patch was acked. Focal patch is already in latest kernel version. ** Changed in: linux (Ubuntu Focal) Status: In Progress => Fix Released -- You received this bug notification because you are a member of Ubuntu Server, which is subscribed to the bug report. https://bugs.launchpad.net/bugs/1852122 Title: ocfs2-tools is causing kernel panics in Ubuntu Focal (Ubuntu-5.4.0-9.12) To manage notifications about this bug go to: https://bugs.launchpad.net/ocfs2-tools/+bug/1852122/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1852122] Re: ocfs2-tools is causing kernel panics in Ubuntu Focal (Ubuntu-5.4.0-9.12)
VERIFICATION: (k)rafaeldtinoco@ocfs2:~$ apt-cache policy linux-image-5.4.0-14-generic linux-image-5.4.0-14-generic: Installed: 5.4.0-14.17 Candidate: 5.4.0-14.17 Version table: *** 5.4.0-14.17 500 500 http://br.archive.ubuntu.com/ubuntu focal-proposed/main amd64 Packages 100 /var/lib/dpkg/status (k)rafaeldtinoco@ocfs2:~/scripts$ sudo ./ocfs2_reproducer.sh === dlmfs === ocfs2_dlmfs /dlm ocfs2_dlmfs rw,relatime 0 0 === lsmod === ocfs2_stack_o2cb 16384 0 ocfs2_dlm 192512 1 ocfs2_stack_o2cb ocfs2_nodemanager 196608 7 ocfs2_stack_o2cb,ocfs2_dlm,ocfs2_dlmfs ocfs2_stackglue20480 2 ocfs2_stack_o2cb,ocfs2_dlmfs === o2hbmonitor === 1026 o2hbmonitor === o2cluster === o2cb,ocfs2,local === o2cb_ctl === cluster: name = ocfs2 node_count = 1 status = configured === losetup === 200+0 records in 200+0 records out 209715200 bytes (210 MB, 200 MiB) copied, 0.192249 s, 1.1 GB/s === mkfs === mkfs.ocfs2 1.8.6 Cluster stack: o2cb Cluster name: ocfs2 Stack Flags: 0x0 NOTE: Feature extended slot map may be enabled Label: Features: sparse extended-slotmap backup-super unwritten inline-data strict-journal-super xattr indexed-dirs refcount discontig-bg append-dio Block size: 1024 (10 bits) Cluster size: 4096 (12 bits) Volume size: 209715200 (51200 clusters) (204800 blocks) Cluster groups: 7 (tail covers 5120 clusters, rest cover 7680 clusters) Extent allocator size: 2097152 (1 groups) Journal size: 4194304 Node slots: 2 Creating bitmaps: done Initializing superblock: done Writing system files: done Writing superblock: done Writing backup superblock: 0 block(s) Formatting Journals: done Growing extent allocator: done Formatting slot map: done Formatting quota files: done Writing lost+found: done mkfs.ocfs2 successful === mount === Filesystem 1K-blocks Used Available Use% Mounted on /dev/loop0204800 14904189896 8% /mnt cleanup ** Tags removed: verification-needed-eoan ** Tags added: verification-done verification-done-eoan -- You received this bug notification because you are a member of Ubuntu Server, which is subscribed to the bug report. https://bugs.launchpad.net/bugs/1852122 Title: ocfs2-tools is causing kernel panics in Ubuntu Focal (Ubuntu-5.4.0-9.12) To manage notifications about this bug go to: https://bugs.launchpad.net/ocfs2-tools/+bug/1852122/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
