Hi all, I recently migrated an existing ext4 fs using btrfs-convert (setting nodesize to 32k and enabling optional features `extref`, `skinny-metadata` and `no-holes` - the first two of which I believe are now the default in any case?), but I’m subsequently seeing very frequent BUGs being output by the kernel:
[ 821.843637] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:281 [ 821.843641] in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 28214, name: podman [ 821.843644] CPU: 3 PID: 28214 Comm: podman Tainted: G W 5.11.6 #15 [ 821.843646] Hardware name: Dell Inc. PowerEdge R330/084XW4, BIOS 2.11.0 12/08/2020 [ 821.843647] Call Trace: [ 821.843650] dump_stack+0xa1/0xfb [ 821.843656] ___might_sleep+0x144/0x160 [ 821.843659] mutex_lock+0x17/0x40 [ 821.843662] kernfs_remove_by_name_ns+0x1f/0x80 [ 821.843666] sysfs_remove_group+0x7d/0xe0 [ 821.843668] sysfs_remove_groups+0x28/0x40 [ 821.843670] kobject_del+0x2a/0x80 [ 821.843672] btrfs_sysfs_del_one_qgroup+0x2b/0x40 [btrfs] [ 821.843685] __del_qgroup_rb+0x12/0x150 [btrfs] [ 821.843696] btrfs_remove_qgroup+0x288/0x2a0 [btrfs] [ 821.843707] btrfs_ioctl+0x3129/0x36a0 [btrfs] [ 821.843717] ? __mod_lruvec_page_state+0x5e/0xb0 [ 821.843719] ? page_add_new_anon_rmap+0xbc/0x150 [ 821.843723] ? kfree+0x1b4/0x300 [ 821.843725] ? mntput_no_expire+0x55/0x330 [ 821.843728] __x64_sys_ioctl+0x5a/0xa0 [ 821.843731] do_syscall_64+0x33/0x70 [ 821.843733] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [ 821.843736] RIP: 0033:0x4cd3fb [ 821.843739] Code: fa ff eb bd e8 86 8b fa ff e9 61 ff ff ff cc e8 fb 55 fa ff 48 8b 7c 24 10 48 8b 74 24 18 48 8b 54 24 20 48 8b 44 24 08 0f 05 <48> 3d 01 f0 ff ff 76 20 48 c7 44 24 28 ff ff ff ff 48 c7 44 24 30 [ 821.843741] RSP: 002b:000000c000906b20 EFLAGS: 00000206 ORIG_RAX: 0000000000000010 [ 821.843744] RAX: ffffffffffffffda RBX: 000000c000050000 RCX: 00000000004cd3fb [ 821.843745] RDX: 000000c000906b98 RSI: 000000004010942a RDI: 000000000000000f [ 821.843747] RBP: 000000c000907cd0 R08: 000000c000622901 R09: 0000000000000000 [ 821.843748] R10: 000000c000d992c0 R11: 0000000000000206 R12: 000000000000012d [ 821.843749] R13: 000000000000012c R14: 0000000000000200 R15: 0000000000000049 The system starts 24 containers on boot via `podman`, and by the time this process is complete there were (on the last power-cycle) 10 such BUG reports logged. Is this a recognised issue? Support information: uname: Linux dellr330 5.11.6 #15 SMP Wed Mar 17 15:18:52 GMT 2021 x86_64 Intel(R) Xeon(R) CPU E3-1240L v5 @ 2.10GHz GenuineIntel GNU/Linux version: btrfs-progs v5.10.1 btrfs fi: Label: 'space' uuid: 94cc0dca-4a1f-4d18-bdf8-943982d1b6ff Total devices 1 FS bytes used 163.44GiB devid 1 size 1.56TiB used 231.24GiB path /dev/mapper/storage-space btrfs df: Data, single: total=221.16GiB, used=154.74GiB System, single: total=4.00MiB, used=384.00KiB Metadata, single: total=10.08GiB, used=8.70GiB GlobalReserve, single: total=512.00MiB, used=0.00B fstab entry: LABEL=space /space btrfs noatime,compress-force=zstd:2,user_subvol_rm_allowed,nofail 0 2 Other dmesg entries: [ 61.973985] Btrfs loaded, crc32c=crc32c-intel, zoned=yes [ 63.310454] BTRFS: device label space devid 1 transid 24453 /dev/mapper/storage-space scanned by btrfs (6546) [ 64.471111] BTRFS info (device dm-1): force zstd compression, level 2 [ 64.471126] BTRFS info (device dm-1): disk space caching is enabled [ 64.471130] BTRFS info (device dm-1): has skinny extents [ 81.247002] BTRFS info (device dm-1): checking UUID tree [ 104.987371] BTRFS error (device dm-1): qgroup scan failed with -4 [ 106.615043] BTRFS error (device dm-1): qgroup scan failed with -4 [ 107.258435] BTRFS error (device dm-1): qgroup scan failed with -4 [ 107.962191] BTRFS error (device dm-1): qgroup scan failed with -4 [ 118.289293] BUG: sleeping function called from invalid context at kernel/locking/mutex.c:281 [ 118.289296] in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 9003, name: podman [ 118.289298] CPU: 4 PID: 9003 Comm: podman Not tainted 5.11.6 #15 [ 118.289301] Hardware name: Dell Inc. PowerEdge R330/084XW4, BIOS 2.11.0 12/08/2020 [ 118.289301] Call Trace: [ 118.289303] dump_stack+0xa1/0xfb [ 118.289308] ___might_sleep+0x144/0x160 [ 118.289310] mutex_lock+0x17/0x40 [ 118.289313] kernfs_remove_by_name_ns+0x1f/0x80 [ 118.289317] sysfs_remove_group+0x7d/0xe0 [ 118.289319] sysfs_remove_groups+0x28/0x40 [ 118.289320] kobject_del+0x2a/0x80 [ 118.289322] btrfs_sysfs_del_one_qgroup+0x2b/0x40 [btrfs] [ 118.289334] __del_qgroup_rb+0x12/0x150 [btrfs] [ 118.289343] btrfs_remove_qgroup+0x288/0x2a0 [btrfs] [ 118.289352] btrfs_ioctl+0x3129/0x36a0 [btrfs] [ 118.289361] ? __mod_lruvec_page_state+0x5e/0xb0 [ 118.289363] ? page_add_new_anon_rmap+0xbc/0x150 [ 118.289366] ? kfree+0x1b4/0x300 [ 118.289368] ? mntput_no_expire+0x55/0x330 [ 118.289371] __x64_sys_ioctl+0x5a/0xa0 [ 118.289374] do_syscall_64+0x33/0x70 [ 118.289375] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [ 118.289378] RIP: 0033:0x4cd3fb [ 118.289380] Code: fa ff eb bd e8 86 8b fa ff e9 61 ff ff ff cc e8 fb 55 fa ff 48 8b 7c 24 10 48 8b 74 24 18 48 8b 54 24 20 48 8b 44 24 08 0f 05 <48> 3d 01 f0 ff ff 76 20 48 c7 44 24 28 ff ff ff ff 48 c7 44 24 30 [ 118.289382] RSP: 002b:000000c0005e2b20 EFLAGS: 00000206 ORIG_RAX: 0000000000000010 [ 118.289384] RAX: ffffffffffffffda RBX: 000000c000050000 RCX: 00000000004cd3fb [ 118.289385] RDX: 000000c0005e2b98 RSI: 000000004010942a RDI: 0000000000000012 [ 118.289386] RBP: 000000c0005e3cd0 R08: 000000c000582c01 R09: 0000000000000000 [ 118.289387] R10: 000000c000708b70 R11: 0000000000000206 R12: 00000000000000b8 [ 118.289388] R13: 00000000000000b7 R14: 0000000000000200 R15: 0000000000000049 [ 498.003691] BTRFS info (device dm-1): qgroup scan completed (inconsistency flag cleared) [ 499.522376] BTRFS error (device dm-1): qgroup scan failed with -4 [ 499.975886] BTRFS error (device dm-1): qgroup scan failed with -4