[Devel] WARNING at mm/slub.c
Hi guys, with the kernel rh7-3.10.0-327.36.1.vz7.18.7 we're seeing the following WARNING while running LTP test suite: [11796.576981] WARNING: at mm/slub.c:1252 slab_pre_alloc_hook.isra.42.part.43+0x15/0x17() [11796.591008] Call Trace: [11796.592065] [] dump_stack+0x19/0x1b [11796.593076] [] warn_slowpath_common+0x70/0xb0 [11796.594228] [] warn_slowpath_null+0x1a/0x20 [11796.595442] [] slab_pre_alloc_hook.isra.42.part.43+0x15/0x17 [11796.596686] [] kmem_cache_alloc_trace+0x58/0x230 [11796.597965] [] ? kmapset_new+0x1e/0x50 [11796.599224] [] kmapset_new+0x1e/0x50 [11796.600433] [] __sysfs_add_one+0x4a/0xb0 [11796.601431] [] sysfs_add_one+0x1b/0xd0 [11796.602451] [] sysfs_add_file_mode+0xb7/0x100 [11796.603449] [] sysfs_create_file+0x2a/0x30 [11796.604461] [] kobject_add_internal+0x16c/0x2f0 [11796.605503] [] kobject_add+0x75/0xd0 [11796.606627] [] ? kmem_cache_alloc_trace+0x207/0x230 [11796.607655] [] __link_block_group+0xe1/0x120 [btrfs] [11796.608634] [] btrfs_make_block_group+0x150/0x270 [btrfs] [11796.609701] [] __btrfs_alloc_chunk+0x67f/0x8a0 [btrfs] [11796.610756] [] btrfs_alloc_chunk+0x34/0x40 [btrfs] [11796.611800] [] do_chunk_alloc+0x23f/0x410 [btrfs] [11796.612954] [] btrfs_check_data_free_space+0xea/0x280 [btrfs] [11796.614008] [] __btrfs_buffered_write+0x151/0x5c0 [btrfs] [11796.615153] [] btrfs_file_aio_write+0x246/0x560 [btrfs] [11796.616141] [] ? __mem_cgroup_commit_charge+0x152/0x350 [11796.617220] [] do_sync_write+0x90/0xe0 [11796.618253] [] vfs_write+0xbd/0x1e0 [11796.619224] [] SyS_write+0x7f/0xe0 [11796.620185] [] system_call_fastpath+0x16/0x1b [11796.621145] ---[ end trace 1437311f89b9e3c6 ]--- Thanks! ___ Devel mailing list Devel@openvz.org https://lists.openvz.org/mailman/listinfo/devel
Re: [Devel] WARNING at mm/slub.c
On 3/16/17, Denis Kirjanov wrote: > Hi guys, > > with the kernel rh7-3.10.0-327.36.1.vz7.18.7 we're seeing the > following WARNING while running LTP test suite: > > [11796.576981] WARNING: at mm/slub.c:1252 > slab_pre_alloc_hook.isra.42.part.43+0x15/0x17() > > [11796.591008] Call Trace: > [11796.592065] [] dump_stack+0x19/0x1b > [11796.593076] [] warn_slowpath_common+0x70/0xb0 > [11796.594228] [] warn_slowpath_null+0x1a/0x20 > [11796.595442] [] > slab_pre_alloc_hook.isra.42.part.43+0x15/0x17 > [11796.596686] [] kmem_cache_alloc_trace+0x58/0x230 > [11796.597965] [] ? kmapset_new+0x1e/0x50 > [11796.599224] [] kmapset_new+0x1e/0x50 > [11796.600433] [] __sysfs_add_one+0x4a/0xb0 > [11796.601431] [] sysfs_add_one+0x1b/0xd0 > [11796.602451] [] sysfs_add_file_mode+0xb7/0x100 > [11796.603449] [] sysfs_create_file+0x2a/0x30 > [11796.604461] [] kobject_add_internal+0x16c/0x2f0 > [11796.605503] [] kobject_add+0x75/0xd0 > [11796.606627] [] ? kmem_cache_alloc_trace+0x207/0x230 > [11796.607655] [] __link_block_group+0xe1/0x120 [btrfs] > [11796.608634] [] btrfs_make_block_group+0x150/0x270 > [btrfs] > [11796.609701] [] __btrfs_alloc_chunk+0x67f/0x8a0 > [btrfs] > [11796.610756] [] btrfs_alloc_chunk+0x34/0x40 [btrfs] > [11796.611800] [] do_chunk_alloc+0x23f/0x410 [btrfs] > [11796.612954] [] > btrfs_check_data_free_space+0xea/0x280 [btrfs] > [11796.614008] [] __btrfs_buffered_write+0x151/0x5c0 > [btrfs] > [11796.615153] [] btrfs_file_aio_write+0x246/0x560 > [btrfs] > [11796.616141] [] ? > __mem_cgroup_commit_charge+0x152/0x350 > [11796.617220] [] do_sync_write+0x90/0xe0 > [11796.618253] [] vfs_write+0xbd/0x1e0 > [11796.619224] [] SyS_write+0x7f/0xe0 > [11796.620185] [] system_call_fastpath+0x16/0x1b > [11796.621145] ---[ end trace 1437311f89b9e3c6 ]--- > Guys, I've found your commit: commit 149819fef38230c95f4d6c644061bc8b0dcdd51d Author: Vladimir Davydov Date: Fri Jun 5 13:20:02 2015 +0400 mm/fs: Port diff-mm-debug-memallocation-caused-fs-reentrance Enable the debug once again, as the issue it found has been fixed: https://jira.sw.ru/browse/PSBM-34112 Previous commit: 255427905323ac97a3c9b2d5acb2bf21ea2b31f6. Author: Dmitry Monakhov Email: dmonak...@openvz.org Subject: mm: debug memallocation caused fs reentrance Date: Sun, 9 Nov 2014 11:53:14 +0400 But I can't open a link to figure out the original reason for the patch. > Thanks! > ___ Devel mailing list Devel@openvz.org https://lists.openvz.org/mailman/listinfo/devel
[Devel] BUG with __destroy_inode. dirtied_ub is not NULL
Hi, with the kernel 2.6.32-042stab123.3 we're seeing a following bug: PID: 126TASK: 880816f2d260 CPU: 1 COMMAND: "kswapd0" #0 [880816f33870] machine_kexec at 8103e3c9 #1 [880816f338d0] crash_kexec at 810e42d2 #2 [880816f339a0] oops_end at 8154c0a0 #3 [880816f339d0] die at 810116bb #4 [880816f33a00] do_trap at 8154b874 #5 [880816f33a60] do_invalid_op at 8100ced5 #6 [880816f33b00] invalid_op at 8100c15b [exception RIP: __destroy_inode+152] RIP: 811d89b8 RSP: 880816f33bb0 RFLAGS: 00010206 RAX: RBX: 88068f677140 RCX: 0034 RDX: 0007 RSI: 88068f6773a0 RDI: 88068f677140 RBP: 880816f33bc0 R8: 4038 R9: f9529a3039c0e807 R10: 88068f6ee8f0 R11: R12: 880816f33c50 R13: 006f R14: 82400940 R15: 88068f677150 ORIG_RAX: CS: 0010 SS: 0018 #7 [880816f33bc8] destroy_inode at 811d89e6 #8 [880816f33be8] dispose_list at 811d8ef7 #9 [880816f33c28] shrink_icache_memory at 811d9257 #10 [880816f33c98] shrink_slab at 8115f928 #11 [880816f33d08] balance_pgdat at 81162be8 #12 [880816f33e28] kswapd at 8116305f #13 [880816f33ee8] kthread at 810ab8ae #14 [880816f33f48] kernel_thread at 8100c3ca crash shows that the the dirtied_ub = 0x4 ___ Devel mailing list Devel@openvz.org https://lists.openvz.org/mailman/listinfo/devel
[Devel] double faults in Virtuozzo KVM
Hi, we're seeing double faults in async_page_fault. _Some_ of them related to the fact that during the faults RSP points to userspace and it leads to double-fault scenario. Is it known problem? Thanks! [11587.895394] Hardware name: Virtuozzo KVM, BIOS 1.9.1-5.3.2.vz7.6 04/01/2014 [11587.895394] task: 88020bee ti: 880204b6 task.ti: 880204b6 [11587.895394] RIP: 0010:[] [] async_page_fault+0xd/0x30 [11587.895394] RSP: 002b:880234f61fd8 EFLAGS: 00010096 [11587.895394] RAX: 816a192c RBX: 0001 RCX: 816a192c [11587.895394] RDX: 88023fc03fc0 RSI: RDI: 880234f62098 [11587.895394] RBP: 880234f62088 R08: 88023fbfffc0 R09: 88003642af00 [11587.895394] R10: 8000 R11: R12: 88023fc04f58 [11587.895394] R13: 0028 R14: R15: [11587.895394] FS: 7ff80ffc1880() GS:88023fc0() knlGS: [11587.895394] CS: 0010 DS: ES: CR0: 8005003b [11587.895394] CR2: 880234f61fc8 CR3: b9436000 CR4: 07f0 [11587.895394] DR0: DR1: DR2: [11587.895394] DR3: DR6: 0ff0 DR7: 0400 [11587.895394] Stack: [11587.895394] c7e9e11c7f44 270f05836600 9090fb02 be0001b9d231 [11587.895394] e8df8948 000231a6fba8 00010008 [11587.895394] 0002 0003 [11587.895394] Call Trace: [11587.895394] Code: 48 89 e7 48 8b 74 24 78 48 c7 44 24 78 ff ff ff ff e8 78 3d 00 00 e9 33 02 00 00 0f 1f 00 66 66 90 66 66 90 66 66 90 48 83 ec 78 7e 01 00 00 48 89 e7 48 8b 74 24 78 48 c7 44 24 78 ff ff ff [11587.895394] RIP [] async_page_fault+0xd/0x30 [11587.895394] RSP ___ Devel mailing list Devel@openvz.org https://lists.openvz.org/mailman/listinfo/devel
Re: [Devel] double faults in Virtuozzo KVM
On Thursday, September 28, 2017, Roman Kagan wrote: > On Thu, Sep 28, 2017 at 05:55:51PM +0300, Denis Kirjanov wrote: > > Hi, we're seeing double faults in async_page_fault. > > async_page_fault is the #PF handler in KVM guests. It filters out > specially crafted #PF's from the host; the rest fall through to the > regular #PF handler. So most likely you're seeing genuine #PFs, > unrelated to virtualization. > > > _Some_ of them related to the fact that during the faults RSP points > > to userspace and it leads to double-fault scenario. > > The postmortem you quote doesn't support that. I'll post a relevant trace > > > Is it known problem? > > There used to be a bug in async pagefault machinery which caused L0 > hypervisor to inject async pagefaults into L2 guest instead of L1. This > must've been fixed in sufficiently recent Yep, I saw the patch and it's imho about the different thing. The patch fixes the wrong PF injected to an unrelated guest and thus a guest ends up with the 'CPU stuck' messages since it can't get the requested page I'd guess the problem is with your kernel. Doesn't it reproduce on bare > metal? > > > > [11587.895394] Hardware name: Virtuozzo KVM, BIOS 1.9.1-5.3.2.vz7.6 > 04/01/2014 > > [11587.895394] task: 88020bee ti: 880204b6 task.ti: > > 880204b6 > > [11587.895394] RIP: 0010:[] [] > > async_page_fault+0xd/0x30 > > [11587.895394] RSP: 002b:880234f61fd8 EFLAGS: 00010096 > > [11587.895394] RAX: 816a192c RBX: 0001 RCX: > 816a192c > > [11587.895394] RDX: 88023fc03fc0 RSI: RDI: > 880234f62098 > > [11587.895394] RBP: 880234f62088 R08: 88023fbfffc0 R09: > 88003642af00 > > [11587.895394] R10: 8000 R11: R12: > 88023fc04f58 > > [11587.895394] R13: 0028 R14: R15: > > > [11587.895394] FS: 7ff80ffc1880() GS:88023fc0() > > knlGS: > > [11587.895394] CS: 0010 DS: ES: CR0: 8005003b > > [11587.895394] CR2: 880234f61fc8 CR3: b9436000 CR4: > 07f0 > > [11587.895394] DR0: DR1: DR2: > > > [11587.895394] DR3: DR6: 0ff0 DR7: > 0400 > > [11587.895394] Stack: > > [11587.895394] c7e9e11c7f44 270f05836600 9090fb02 > > be0001b9d231 > > [11587.895394] e8df8948 000231a6fba8 00010008 > > > > [11587.895394] 0002 0003 > > > > [11587.895394] Call Trace: > > [11587.895394] Code: 48 89 e7 48 8b 74 24 78 48 c7 44 24 78 ff ff ff > > ff e8 78 3d 00 00 e9 33 02 00 00 0f 1f 00 66 66 90 66 66 90 66 66 90 > > 48 83 ec 78 7e 01 00 00 48 89 e7 48 8b 74 24 78 48 c7 44 24 78 ff > > ff ff > > [11587.895394] RIP [] async_page_fault+0xd/0x30 > > [11587.895394] RSP > > Roman. > ___ Devel mailing list Devel@openvz.org https://lists.openvz.org/mailman/listinfo/devel
Re: [Devel] double faults in Virtuozzo KVM
>> > > _Some_ of them related to the fact that during the faults RSP points >> > > to userspace and it leads to double-fault scenario. >> > >> > The postmortem you quote doesn't support that. >> >> >> I'll post a relevant trace Here it is: [32065.459255] double fault: [#1] SMP [32065.459975] Modules linked in: dm_mod hcpdriver(POE) kmodlve(OE) vzdev ppdev pcspkr sg i 2c_piix4 parport_pc parport ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic crct10dif_common virtio_console virtio_scsi virtio_net sr_mod cdrom ata_generic pata_acpi bochs_drm drm_kms_helper ttm drm serio_raw virtio_pci ata_piix virtio_ring virtio i2c_core libata floppy [32065.460041] CPU: 0 PID: 22951 Comm: cdp-2-6 ve: 0 Tainted: P OE 3.10.0-714.10.2.lve1.4.61.el7.x86_64 #1 29.2 [32065.460041] Hardware name: Virtuozzo KVM, BIOS 1.9.1-5.3.2.vz7.6 04/01/2014 [32065.460041] task: 8801e9ab8ff0 ti: 8800ab598000 task.ti: 8800ab598000 [32065.460041] RIP: 0010:[] [] async_page_fault+0xd/0x 30 [32065.460041] RSP: 002b:7f1a1290afe8 EFLAGS: 00010016 [32065.460041] RAX: 816a192c RBX: 0001 RCX: 816a192c [32065.460041] RDX: 0008 RSI: RDI: 7f1a1290b0a8 [32065.460041] RBP: 7f1a1290b098 R08: 0001 R09: [32065.460041] R10: R11: R12: 7f1a12919960 [32065.460041] R13: 0028 R14: R15: 011f3f20 [32065.460041] FS: 7f1a1291e700() GS:88023fc0() knlGS: [32065.460041] CS: 0010 DS: ES: CR0: 8005003b [32065.460041] CR2: 7f1a1290afd8 CR3: 36794000 CR4: 07f0 [32065.460041] DR0: DR1: DR2: [32065.460041] DR3: DR6: 0ff0 DR7: 0400 [32065.460041] Stack: [32065.460041] BUG: unable to handle kernel paging request at 7f1a1290afe8 [32065.460041] IP: [] show_stack_log_lvl+0x109/0x180 [32065.460041] PGD 36799067 PUD 3679a067 PMD 220382067 PTE 0 [32065.460041] Oops: [#2] SMP [32065.460041] Modules linked in: dm_mod hcpdriver(POE) kmodlve(OE) vzdev ppdev pcspkr sg i2c_piix4 parport_pc parport ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic crct10dif_common virtio_console virtio_scsi virtio_net sr_mod cdrom ata_generic pata_acpi bochs_drm drm_kms_helper ttm drm serio_raw virtio_pci ata_piix virtio_ring virtio i2c_core libata floppy [32065.460041] CPU: 0 PID: 22951 Comm: cdp-2-6 ve: 0 Tainted: P OE 3.10.0-714.10.2.lve1.4.61.el7.x86_64 #1 29.2 [32065.460041] Hardware name: Virtuozzo KVM, BIOS 1.9.1-5.3.2.vz7.6 04/01/2014 [32065.460041] task: 8801e9ab8ff0 ti: 8800ab598000 task.ti: 8800ab598000 [32065.460041] RIP: 0010:[] [] show_stack_log_lvl+0x109/0x180 [32065.460041] RSP: 002b:88023fc04e18 EFLAGS: 00010046 [32065.460041] RAX: 7f1a1290aff0 RBX: 7f1a1290afe8 RCX: [32065.460041] RDX: 88023fc03fc0 RSI: 88023fc04f58 RDI: [32065.460041] RBP: 88023fc04e68 R08: 88023fbfffc0 R09: 8800369f5900 [32065.460041] R10: 0001 R11: R12: 88023fc04f58 [32065.460041] R13: R14: 818d31b8 R15: [32065.460041] FS: 7f1a1291e700() GS:88023fc0() knlGS: [32065.460041] CS: 0010 DS: ES: CR0: 8005003b [32065.460041] CR2: 7f1a1290afe8 CR3: 36794000 CR4: 07f0 [32065.460041] DR0: DR1: DR2: [32065.460041] DR3: DR6: 0ff0 DR7: 0400 [32065.460041] Stack: [32065.460041] 88020008 88023fc04e78 88023fc04e38 7ddc4235 [32065.460041] 7f1a1290afe8 88023fc04f58 7f1a1290afe8 88023fc04f58 [32065.460041] 002b 011f3f20 88023fc04ec8 8102d7f6 [32065.460041] Call Trace: [32065.460041] <#DF> [32065.460041] [32065.460041] [] show_regs+0xb6/0x240 [32065.460041] [] __die+0x9f/0xf0 [32065.460041] [] die+0x38/0x70 [32065.460041] [] do_double_fault+0x72/0x80 [32065.460041] [] double_fault+0x28/0x30 [32065.460041] [] ? restore_args+0x30/0x30 [32065.460041] [] ? async_page_fault+0xd/0x30 [32065.460041] <> [32065.460041] Code: [32065.460041] 4d b8 4c 89 45 c0 48 89 55 c8 48 8b 5b f8 e8 37 6b 66 00 48 8b 55 c8 4c 8b 45 c0 8b 4d b8 85 c9 74 05 f6 c1 03 74 4c 48 8d 43 08 <48> 8b 33 48 c7 c7 b0 31 8d 81 89 4d b4 4c 89 45 b8 48 89 45 c8 [32065.460041] RIP [] show_stack_log_lvl+0x109/0x180 [32065.460041] RSP [32065.460041] CR2: 7f1a1290afe8 >> > >> > > Is it known problem? >> > >> > There used to be a bug in async pagefault machinery which caused L0 >> > hypervisor to inject async pagefaults into L2 guest instead of L1. >> > This >> > must've been fixed in sufficiently recent >> >> >> Yep, I saw the patch and it'
[Devel] lockdep_assert_held is triggering with the shrinker counter
Hi guys, the following commit triggers the splat in dmesg: The commit is pushed to "branch-rh7-3.10.0-514.26.1.vz7.35.x-ovz" and will appear at https://src.openvz.org/scm/ovz/vzkernel.git after rh7-3.10.0-514.26.1.vz7.35.5 --> commit 6fd774dbf6fd05eca8cfa192753bf35dac694368 Author: Kirill Tkhai Date: Thu Aug 31 18:25:20 2017 +0300 your commit added rcu read side enter instead of spinlock but looks like you forgot to update the assert statement in the list_lru_from_memcg_idx Thanks! ___ Devel mailing list Devel@openvz.org https://lists.openvz.org/mailman/listinfo/devel
Re: [Devel] lockdep_assert_held is triggering with the shrinker counter
On 1/9/18, Kirill Tkhai wrote: > Hi, Denis, > > Feel free to CC commit author in the future ;) > > On 06.01.2018 21:44, Denis Kirjanov wrote: >> Hi guys, >> >> the following commit triggers the splat in dmesg: >> >> >> The commit is pushed to "branch-rh7-3.10.0-514.26.1.vz7.35.x-ovz" and >> will appear at https://src.openvz.org/scm/ovz/vzkernel.git >> after rh7-3.10.0-514.26.1.vz7.35.5 >> --> >> commit 6fd774dbf6fd05eca8cfa192753bf35dac694368 >> Author: Kirill Tkhai >> Date: Thu Aug 31 18:25:20 2017 +0300 >> >> your commit added rcu read side enter instead of spinlock but looks >> like you forgot to update the assert statement in the >> list_lru_from_memcg_idx > > There is no assert statement in list_lru_from_memcg_idx() anymore. > I've removed it in: > > 5db3da0bf711 "mm: Make list_lru_node::memcg_lrus RCU protected", > one commit before 6fd774dbf6fd. > ah, I see. Missed it. Thx! > Do you use modified vzkernel? I assume you use a custom kernel > with your own modifications, and you don't have ported the commit > I pointed above. > > Kirill > ___ Devel mailing list Devel@openvz.org https://lists.openvz.org/mailman/listinfo/devel
[Devel] invalid victim pointer in oom context
Hi guys,
We're seeing kernel crashes while dumping kernel stack durring OOM timeout.
Here we see that the victim pointer is not valid.
Is it known issue?
[2725959.095523] RIP: 0010:[] []
dump_trace+0x1df/0x2d0
[2725959.096079] RSP: :8803020abcb0 EFLAGS: 00010283
[2725959.096741] RAX: RBX: 816ba7a0 RCX:
0002
[2725959.097340] RDX: 0002 RSI: RDI:
88022c838000
[2725959.097909] RBP: 8803020abd20 R08: 88086fc8 R09:
81917c27
[2725959.098465] R10: R11: 8803020aba1e R12:
88022c838000
[2725959.099026] R13: 8803020abd20 R14: R15:
[2725959.099596] FS: 7f16a6c46840() GS:88086fc8()
knlGS:
[2725959.100214] CS: 0010 DS: ES: CR0: 80050033
[2725959.100813] CR2: CR3: 0004e3c52000 CR4:
003607e0
[2725959.101427] DR0: DR1: DR2:
[2725959.102044] DR3: DR6: fffe0ff0 DR7:
0400
[2725959.102609] Call Trace:
[2725959.103165] [] ? vprintk_default+0x36/0x50
[2725959.103719] [] show_trace_log_lvl+0x4d/0x60
[2725959.104285] [] show_stack+0x34/0x70
[2725959.104843] [] oom_trylock+0x1d1/0x1e0
[2725959.105380] [] mem_cgroup_oom_synchronize+0xe1/0x4e0
[2725959.105954] [] ? put_page+0x4f/0x60
[2725959.106502] [] ? handle_mm_fault+0x4d7/0x14c0
[2725959.107088] [] ? __account_cfs_rq_runtime+0x100/0x160
[2725959.107654] [] pagefault_out_of_memory+0x13/0x50
[2725959.108252] [] mm_fault_error+0x68/0x12b
[2725959.108856] [] __do_page_fault+0x395/0x450
[2725959.109438] [] do_page_fault+0x35/0x90
[2725959.110051] [] page_fault+0x28/0x30
[2725959.110634] Code: 8b b7 e0 06 00 00 4d 85 ed 0f 85 bf fe ff ff 65
48 8b 04 25 00 0e 01 00 49 89 ed 48 39 c7 0f 84 aa fe ff ff 48 8b 87
e0 06 00 00 <4c> 8b 28 e9 9b fe ff ff 66 0f 1f 84 00 00 00 00 00 8b 45
9c 0f
[2725959.111960] RIP [] dump_trace+0x1df/0x2d0
[2725959.112555] RSP
[2725959.113172] CR2:
struct mem_cgroup -h 0x880765f09800 | less
oom_ctx = {
owner = 0x0,
victim = 0x88022c838000,
marked = 0x1,
oom_start = 0x19a3c0875,
oom_end = 0x19a3c0876,
overdraft = 0xfa0,
rage = 0x14,
waitq = {
lock = {
{
rlock = {
raw_lock = {
{
head_tail = 0x99c099c,
tickets =
{ head = 0x99c, tail = 0x99c }
}
Thanks!
___
Devel mailing list
Devel@openvz.org
https://lists.openvz.org/mailman/listinfo/devel
