Public bug reported:
We observed a kernel trace on a KVM hypervisor servers during live
migrating an instance:
```
[...]
Jan 23 10:58:53 fra-az1-comp-22 kernel: ------------[ cut here ]------------
Jan 23 10:58:53 fra-az1-comp-22 kernel: WARNING: CPU: 75 PID: 1082578 at
arch/x86/kvm/mmu/mmu.c:6362 kvm_shadow_mmu_try_split_huge_pages+0x2e3/0x6d0
[kvm]
Jan 23 10:58:53 fra-az1-comp-22 kernel: Modules linked in: authenc echainiv
esp4 cpuid xt_policy nf_conntrack_netlink xt_multiport xt_REDIRECT xt_nat
xt_connmark xt_mark ebt_arp nft_meta_bridge ip6_tables xt_CT xt_mac xt_set
xt_state ip_set_hash_net ip_set vhost_net vhost vhost_iotlb tap vxlan
ip6_udp_tunnel udp_tunnel xt_comment xt_physdev veth xt_CHECKSUM xt_MASQUERADE
xt_con>
Jan 23 10:58:53 fra-az1-comp-22 kernel: linear hid_generic cdc_ether usbhid
usbnet hid mii ast i2c_algo_bit drm_shmem_helper drm_kms_helper syscopyarea
crct10dif_pclmul sysfillrect crc32_pclmul polyval_clmulni polyval_generic
sysimgblt raid1 ghash_clmulni_intel sha512_ssse3 aesni_intel crypto_simd cryptd
ice ahci drm nvme i40e libahci xhci_pci i2c_piix4 xhci_pci_renesas nvme_c>
Jan 23 10:58:53 fra-az1-comp-22 kernel: CPU: 75 PID: 1082578 Comm:
live_migration Not tainted 6.2.0-37-generic #38~22.04.1-Ubuntu
Jan 23 10:58:53 fra-az1-comp-22 kernel: Hardware name: ASUSTeK COMPUTER INC.
RS720A-E11-RS24U/KMPP-D32 Series, BIOS 1501 08/23/2023
Jan 23 10:58:53 fra-az1-comp-22 kernel: RIP:
0010:kvm_shadow_mmu_try_split_huge_pages+0x2e3/0x6d0 [kvm]
Jan 23 10:58:53 fra-az1-comp-22 kernel: Code: 0a 04 00 00 48 81 c4 e0 00 00 00
5b 41 5c 41 5d 41 5e 41 5f 5d 31 c0 31 d2 31 c9 31 f6 31 ff 45 31 c0 45 31 c9
e9 3d 56 5e e5 <0f> 0b 48 8d bd 68 ff ff ff e8 8f 16 ff ff 49 89 c5 48 85 c0 0f
85
Jan 23 10:58:53 fra-az1-comp-22 kernel: RSP: 0018:ffffbe80b395bba0 EFLAGS:
00010246
Jan 23 10:58:53 fra-az1-comp-22 kernel: RAX: fffff4f620773340 RBX:
0000000000000001 RCX: 0000000000000000
Jan 23 10:58:53 fra-az1-comp-22 kernel: RDX: ffff9a2e5dccdfa8 RSI:
0000000000000000 RDI: 0000000000000000
Jan 23 10:58:53 fra-az1-comp-22 kernel: RBP: ffffbe80b395bca8 R08:
0000000000000001 R09: 00000000001d483f
Jan 23 10:58:53 fra-az1-comp-22 kernel: R10: 0000000000000000 R11:
0000000000000000 R12: ffffbe80af5c1000
Jan 23 10:58:53 fra-az1-comp-22 kernel: R13: ffff9a2dddccdfa8 R14:
ffff99c51046d2b0 R15: 0000000000000000
Jan 23 10:58:53 fra-az1-comp-22 kernel: FS: 00007f8ec4a8a640(0000)
GS:ffff9ab61eac0000(0000) knlGS:0000000000000000
Jan 23 10:58:53 fra-az1-comp-22 kernel: CS: 0010 DS: 0000 ES: 0000 CR0:
0000000080050033
Jan 23 10:58:53 fra-az1-comp-22 kernel: CR2: 00007f9053dff000 CR3:
000000a83eaa8001 CR4: 0000000000770ee0
Jan 23 10:58:53 fra-az1-comp-22 kernel: PKRU: 55555554
Jan 23 10:58:53 fra-az1-comp-22 kernel: Call Trace:
Jan 23 10:58:53 fra-az1-comp-22 kernel: <TASK>
Jan 23 10:58:53 fra-az1-comp-22 kernel: ? show_regs+0x72/0x90
Jan 23 10:58:53 fra-az1-comp-22 kernel: ?
kvm_shadow_mmu_try_split_huge_pages+0x2e3/0x6d0 [kvm]
Jan 23 10:58:53 fra-az1-comp-22 kernel: ? __warn+0x8d/0x160
Jan 23 10:58:53 fra-az1-comp-22 kernel: ?
kvm_shadow_mmu_try_split_huge_pages+0x2e3/0x6d0 [kvm]
Jan 23 10:58:53 fra-az1-comp-22 kernel: ? report_bug+0x1bb/0x1d0
Jan 23 10:58:53 fra-az1-comp-22 kernel: ? handle_bug+0x46/0x90
Jan 23 10:58:53 fra-az1-comp-22 kernel: ? exc_invalid_op+0x19/0x80
Jan 23 10:58:53 fra-az1-comp-22 kernel: ? asm_exc_invalid_op+0x1b/0x20
Jan 23 10:58:53 fra-az1-comp-22 kernel: ?
kvm_shadow_mmu_try_split_huge_pages+0x2e3/0x6d0 [kvm]
Jan 23 10:58:53 fra-az1-comp-22 kernel: ? srso_alias_return_thunk+0x5/0x7f
Jan 23 10:58:53 fra-az1-comp-22 kernel: ? tdp_iter_start+0x5c/0xa0 [kvm]
Jan 23 10:58:53 fra-az1-comp-22 kernel: ? srso_alias_return_thunk+0x5/0x7f
Jan 23 10:58:53 fra-az1-comp-22 kernel: ? clear_dirty_pt_masked+0x155/0x1c0
[kvm]
Jan 23 10:58:53 fra-az1-comp-22 kernel: kvm_mmu_try_split_huge_pages+0x89/0xd0
[kvm]
Jan 23 10:58:53 fra-az1-comp-22 kernel:
kvm_arch_mmu_enable_log_dirty_pt_masked+0x1e2/0x270 [kvm]
Jan 23 10:58:53 fra-az1-comp-22 kernel: ? __check_object_size.part.0+0x72/0x150
Jan 23 10:58:53 fra-az1-comp-22 kernel:
kvm_clear_dirty_log_protect+0x228/0x2f0 [kvm]
Jan 23 10:58:53 fra-az1-comp-22 kernel: kvm_vm_ioctl+0x171/0x8a0 [kvm]
Jan 23 10:58:53 fra-az1-comp-22 kernel: ? __handle_mm_fault+0x3c0/0x720
Jan 23 10:58:53 fra-az1-comp-22 kernel: ? srso_alias_return_thunk+0x5/0x7f
Jan 23 10:58:53 fra-az1-comp-22 kernel: ? __fget_light+0xb5/0x160
Jan 23 10:58:53 fra-az1-comp-22 kernel: __x64_sys_ioctl+0x9d/0xe0
Jan 23 10:58:53 fra-az1-comp-22 kernel: do_syscall_64+0x5c/0x90
Jan 23 10:58:53 fra-az1-comp-22 kernel: ? srso_alias_return_thunk+0x5/0x7f
Jan 23 10:58:53 fra-az1-comp-22 kernel: ? do_user_addr_fault+0x1d0/0x640
Jan 23 10:58:53 fra-az1-comp-22 kernel: ? srso_alias_return_thunk+0x5/0x7f
Jan 23 10:58:53 fra-az1-comp-22 kernel: ? exit_to_user_mode_prepare+0x3b/0xd0
Jan 23 10:58:53 fra-az1-comp-22 kernel: ? srso_alias_return_thunk+0x5/0x7f
Jan 23 10:58:53 fra-az1-comp-22 kernel: ? irqentry_exit_to_user_mode+0x17/0x20
Jan 23 10:58:53 fra-az1-comp-22 kernel: ? srso_alias_return_thunk+0x5/0x7f
Jan 23 10:58:53 fra-az1-comp-22 kernel: ? irqentry_exit+0x43/0x50
Jan 23 10:58:53 fra-az1-comp-22 kernel: ? srso_alias_return_thunk+0x5/0x7f
Jan 23 10:58:53 fra-az1-comp-22 kernel: ? exc_page_fault+0x92/0x1b0
Jan 23 10:58:53 fra-az1-comp-22 kernel:
entry_SYSCALL_64_after_hwframe+0x73/0xdd
Jan 23 10:58:53 fra-az1-comp-22 kernel: RIP: 0033:0x7f93d811a94f
Jan 23 10:58:53 fra-az1-comp-22 kernel: Code: 00 48 89 44 24 18 31 c0 48 8d 44
24 60 c7 04 24 10 00 00 00 48 89 44 24 08 48 8d 44 24 20 48 89 44 24 10 b8 10
00 00 00 0f 05 <41> 89 c0 3d 00 f0 ff ff 77 1f 48 8b 44 24 18 64 48 2b 04 25 28
00
Jan 23 10:58:53 fra-az1-comp-22 kernel: RSP: 002b:00007f8ec4a89120 EFLAGS:
00000246 ORIG_RAX: 0000000000000010
Jan 23 10:58:53 fra-az1-comp-22 kernel: RAX: ffffffffffffffda RBX:
00000000c018aec0 RCX: 00007f93d811a94f
Jan 23 10:58:53 fra-az1-comp-22 kernel: RDX: 00007f8ec4a89300 RSI:
ffffffffc018aec0 RDI: 0000000000000058
Jan 23 10:58:53 fra-az1-comp-22 kernel: RBP: 000055bbcc127370 R08:
0000000000040000 R09: 0000000000000000
Jan 23 10:58:53 fra-az1-comp-22 kernel: R10: 00000000000c0000 R11:
0000000000000246 R12: 00007f8ec4a89300
Jan 23 10:58:53 fra-az1-comp-22 kernel: R13: 000055bbcc128430 R14:
0000000000000001 R15: 00007f93d4410058
Jan 23 10:58:53 fra-az1-comp-22 kernel: </TASK>
Jan 23 10:58:53 fra-az1-comp-22 kernel: ---[ end trace 0000000000000000 ]---
[...]
```
The machine runs OpenStack Nova using libvirt (KVM) to run virtual machines.
There was only this one trace and no obvious issues with the workload or the
machine (before and after migration).
** Affects: linux-hwe-6.2 (Ubuntu)
Importance: Undecided
Status: New
** Summary changed:
- Kernel trace in arch/x86/kvm/mmu/mmu.c:6362 using KVM live migration
+ Kernel trace in arch/x86/kvm/mmu/mmu.c:6362 during KVM live migration
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux-hwe-6.2 in Ubuntu.
https://bugs.launchpad.net/bugs/2051114
Title:
Kernel trace in arch/x86/kvm/mmu/mmu.c:6362 during KVM live migration
Status in linux-hwe-6.2 package in Ubuntu:
New
Bug description:
We observed a kernel trace on a KVM hypervisor servers during live
migrating an instance:
```
[...]
Jan 23 10:58:53 fra-az1-comp-22 kernel: ------------[ cut here ]------------
Jan 23 10:58:53 fra-az1-comp-22 kernel: WARNING: CPU: 75 PID: 1082578 at
arch/x86/kvm/mmu/mmu.c:6362 kvm_shadow_mmu_try_split_huge_pages+0x2e3/0x6d0
[kvm]
Jan 23 10:58:53 fra-az1-comp-22 kernel: Modules linked in: authenc echainiv
esp4 cpuid xt_policy nf_conntrack_netlink xt_multiport xt_REDIRECT xt_nat
xt_connmark xt_mark ebt_arp nft_meta_bridge ip6_tables xt_CT xt_mac xt_set
xt_state ip_set_hash_net ip_set vhost_net vhost vhost_iotlb tap vxlan
ip6_udp_tunnel udp_tunnel xt_comment xt_physdev veth xt_CHECKSUM xt_MASQUERADE
xt_con>
Jan 23 10:58:53 fra-az1-comp-22 kernel: linear hid_generic cdc_ether usbhid
usbnet hid mii ast i2c_algo_bit drm_shmem_helper drm_kms_helper syscopyarea
crct10dif_pclmul sysfillrect crc32_pclmul polyval_clmulni polyval_generic
sysimgblt raid1 ghash_clmulni_intel sha512_ssse3 aesni_intel crypto_simd cryptd
ice ahci drm nvme i40e libahci xhci_pci i2c_piix4 xhci_pci_renesas nvme_c>
Jan 23 10:58:53 fra-az1-comp-22 kernel: CPU: 75 PID: 1082578 Comm:
live_migration Not tainted 6.2.0-37-generic #38~22.04.1-Ubuntu
Jan 23 10:58:53 fra-az1-comp-22 kernel: Hardware name: ASUSTeK COMPUTER INC.
RS720A-E11-RS24U/KMPP-D32 Series, BIOS 1501 08/23/2023
Jan 23 10:58:53 fra-az1-comp-22 kernel: RIP:
0010:kvm_shadow_mmu_try_split_huge_pages+0x2e3/0x6d0 [kvm]
Jan 23 10:58:53 fra-az1-comp-22 kernel: Code: 0a 04 00 00 48 81 c4 e0 00 00
00 5b 41 5c 41 5d 41 5e 41 5f 5d 31 c0 31 d2 31 c9 31 f6 31 ff 45 31 c0 45 31
c9 e9 3d 56 5e e5 <0f> 0b 48 8d bd 68 ff ff ff e8 8f 16 ff ff 49 89 c5 48 85 c0
0f 85
Jan 23 10:58:53 fra-az1-comp-22 kernel: RSP: 0018:ffffbe80b395bba0 EFLAGS:
00010246
Jan 23 10:58:53 fra-az1-comp-22 kernel: RAX: fffff4f620773340 RBX:
0000000000000001 RCX: 0000000000000000
Jan 23 10:58:53 fra-az1-comp-22 kernel: RDX: ffff9a2e5dccdfa8 RSI:
0000000000000000 RDI: 0000000000000000
Jan 23 10:58:53 fra-az1-comp-22 kernel: RBP: ffffbe80b395bca8 R08:
0000000000000001 R09: 00000000001d483f
Jan 23 10:58:53 fra-az1-comp-22 kernel: R10: 0000000000000000 R11:
0000000000000000 R12: ffffbe80af5c1000
Jan 23 10:58:53 fra-az1-comp-22 kernel: R13: ffff9a2dddccdfa8 R14:
ffff99c51046d2b0 R15: 0000000000000000
Jan 23 10:58:53 fra-az1-comp-22 kernel: FS: 00007f8ec4a8a640(0000)
GS:ffff9ab61eac0000(0000) knlGS:0000000000000000
Jan 23 10:58:53 fra-az1-comp-22 kernel: CS: 0010 DS: 0000 ES: 0000 CR0:
0000000080050033
Jan 23 10:58:53 fra-az1-comp-22 kernel: CR2: 00007f9053dff000 CR3:
000000a83eaa8001 CR4: 0000000000770ee0
Jan 23 10:58:53 fra-az1-comp-22 kernel: PKRU: 55555554
Jan 23 10:58:53 fra-az1-comp-22 kernel: Call Trace:
Jan 23 10:58:53 fra-az1-comp-22 kernel: <TASK>
Jan 23 10:58:53 fra-az1-comp-22 kernel: ? show_regs+0x72/0x90
Jan 23 10:58:53 fra-az1-comp-22 kernel: ?
kvm_shadow_mmu_try_split_huge_pages+0x2e3/0x6d0 [kvm]
Jan 23 10:58:53 fra-az1-comp-22 kernel: ? __warn+0x8d/0x160
Jan 23 10:58:53 fra-az1-comp-22 kernel: ?
kvm_shadow_mmu_try_split_huge_pages+0x2e3/0x6d0 [kvm]
Jan 23 10:58:53 fra-az1-comp-22 kernel: ? report_bug+0x1bb/0x1d0
Jan 23 10:58:53 fra-az1-comp-22 kernel: ? handle_bug+0x46/0x90
Jan 23 10:58:53 fra-az1-comp-22 kernel: ? exc_invalid_op+0x19/0x80
Jan 23 10:58:53 fra-az1-comp-22 kernel: ? asm_exc_invalid_op+0x1b/0x20
Jan 23 10:58:53 fra-az1-comp-22 kernel: ?
kvm_shadow_mmu_try_split_huge_pages+0x2e3/0x6d0 [kvm]
Jan 23 10:58:53 fra-az1-comp-22 kernel: ? srso_alias_return_thunk+0x5/0x7f
Jan 23 10:58:53 fra-az1-comp-22 kernel: ? tdp_iter_start+0x5c/0xa0 [kvm]
Jan 23 10:58:53 fra-az1-comp-22 kernel: ? srso_alias_return_thunk+0x5/0x7f
Jan 23 10:58:53 fra-az1-comp-22 kernel: ? clear_dirty_pt_masked+0x155/0x1c0
[kvm]
Jan 23 10:58:53 fra-az1-comp-22 kernel:
kvm_mmu_try_split_huge_pages+0x89/0xd0 [kvm]
Jan 23 10:58:53 fra-az1-comp-22 kernel:
kvm_arch_mmu_enable_log_dirty_pt_masked+0x1e2/0x270 [kvm]
Jan 23 10:58:53 fra-az1-comp-22 kernel: ?
__check_object_size.part.0+0x72/0x150
Jan 23 10:58:53 fra-az1-comp-22 kernel:
kvm_clear_dirty_log_protect+0x228/0x2f0 [kvm]
Jan 23 10:58:53 fra-az1-comp-22 kernel: kvm_vm_ioctl+0x171/0x8a0 [kvm]
Jan 23 10:58:53 fra-az1-comp-22 kernel: ? __handle_mm_fault+0x3c0/0x720
Jan 23 10:58:53 fra-az1-comp-22 kernel: ? srso_alias_return_thunk+0x5/0x7f
Jan 23 10:58:53 fra-az1-comp-22 kernel: ? __fget_light+0xb5/0x160
Jan 23 10:58:53 fra-az1-comp-22 kernel: __x64_sys_ioctl+0x9d/0xe0
Jan 23 10:58:53 fra-az1-comp-22 kernel: do_syscall_64+0x5c/0x90
Jan 23 10:58:53 fra-az1-comp-22 kernel: ? srso_alias_return_thunk+0x5/0x7f
Jan 23 10:58:53 fra-az1-comp-22 kernel: ? do_user_addr_fault+0x1d0/0x640
Jan 23 10:58:53 fra-az1-comp-22 kernel: ? srso_alias_return_thunk+0x5/0x7f
Jan 23 10:58:53 fra-az1-comp-22 kernel: ? exit_to_user_mode_prepare+0x3b/0xd0
Jan 23 10:58:53 fra-az1-comp-22 kernel: ? srso_alias_return_thunk+0x5/0x7f
Jan 23 10:58:53 fra-az1-comp-22 kernel: ?
irqentry_exit_to_user_mode+0x17/0x20
Jan 23 10:58:53 fra-az1-comp-22 kernel: ? srso_alias_return_thunk+0x5/0x7f
Jan 23 10:58:53 fra-az1-comp-22 kernel: ? irqentry_exit+0x43/0x50
Jan 23 10:58:53 fra-az1-comp-22 kernel: ? srso_alias_return_thunk+0x5/0x7f
Jan 23 10:58:53 fra-az1-comp-22 kernel: ? exc_page_fault+0x92/0x1b0
Jan 23 10:58:53 fra-az1-comp-22 kernel:
entry_SYSCALL_64_after_hwframe+0x73/0xdd
Jan 23 10:58:53 fra-az1-comp-22 kernel: RIP: 0033:0x7f93d811a94f
Jan 23 10:58:53 fra-az1-comp-22 kernel: Code: 00 48 89 44 24 18 31 c0 48 8d
44 24 60 c7 04 24 10 00 00 00 48 89 44 24 08 48 8d 44 24 20 48 89 44 24 10 b8
10 00 00 00 0f 05 <41> 89 c0 3d 00 f0 ff ff 77 1f 48 8b 44 24 18 64 48 2b 04 25
28 00
Jan 23 10:58:53 fra-az1-comp-22 kernel: RSP: 002b:00007f8ec4a89120 EFLAGS:
00000246 ORIG_RAX: 0000000000000010
Jan 23 10:58:53 fra-az1-comp-22 kernel: RAX: ffffffffffffffda RBX:
00000000c018aec0 RCX: 00007f93d811a94f
Jan 23 10:58:53 fra-az1-comp-22 kernel: RDX: 00007f8ec4a89300 RSI:
ffffffffc018aec0 RDI: 0000000000000058
Jan 23 10:58:53 fra-az1-comp-22 kernel: RBP: 000055bbcc127370 R08:
0000000000040000 R09: 0000000000000000
Jan 23 10:58:53 fra-az1-comp-22 kernel: R10: 00000000000c0000 R11:
0000000000000246 R12: 00007f8ec4a89300
Jan 23 10:58:53 fra-az1-comp-22 kernel: R13: 000055bbcc128430 R14:
0000000000000001 R15: 00007f93d4410058
Jan 23 10:58:53 fra-az1-comp-22 kernel: </TASK>
Jan 23 10:58:53 fra-az1-comp-22 kernel: ---[ end trace 0000000000000000 ]---
[...]
```
The machine runs OpenStack Nova using libvirt (KVM) to run virtual machines.
There was only this one trace and no obvious issues with the workload or the
machine (before and after migration).
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux-hwe-6.2/+bug/2051114/+subscriptions
--
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help : https://help.launchpad.net/ListHelp
