Testing latest qemu-kvm.git and kvm-kmod.git, ept enabled and backing guests with large pages trips a BUG in the mmu code. If I disable ept, but still use large pages, migration succeeds. Reproduce with:
hugetlbfs setup: % mkdir -p /hugetlbfs && mount -t hugetlbfs hugetlbfs /hugetlbfs % echo 10000 > /proc/sys/vm/nr_hugepages qemu commands: guest a: sudo x86_64-softmmu/qemu-system-x86_64 -L pc-bios -m 2048 -mempath /hugetlbfs -net nic -net tap -vnc :12 -monitor stdio -hda /scratch/images/rharper/rhel4u8-32-ide.raw guest b: sudo x86_64-softmmu/qemu-system-x86_64 -L pc-bios -m 2048 -mempath /hugetlbfs -net nic -net tap -vnc :13 -monitor stdio -hda /scratch/images/rharper/rhel4u8-32-ide.raw -incoming tcp:0:4444 Once the guest a is up, issued migrate command: (qemu) migrate -d tcp:localhost:444 rmap_remove: ffff880a08e00098 c0336e65c0336e5b 0->BUG ------------[ cut here ]------------ kernel BUG at /home/rharper/work/git/kvm-kmod/x86/mmu.c:615! invalid opcode: 0000 [1] SMP last sysfs file: /sys/devices/system/cpu/cpu15/cache/index2/shared_cpu_map CPU 6 Modules linked in: kvm_intel(N) kvm(N) tun nfs lockd nfs_acl sunrpc ipv6 bridge stp cpufreq_conservative cpufreq_userspace cpufreq_powersave acpi_cpufreq microcode fuse loop sr_mod cdrom dm_mod sg rtc_cmos thermal cdc_ether i2c_i801 rtc_core usbnet usb_storage shpchp i2c_core rtc_lib processor bnx2 pcspkr button pci_hotplug mii mptctl joydev usbhid hid ff_memless uhci_hcd ehci_hcd usbcore sd_mod crc_t10dif edd fan thermal_sys hwmon ext3 mbcache jbd mptsas mptscsih mptbase scsi_transport_sas scsi_mod [last unloaded: kvm] Supported: No Pid: 17635, comm: qemu-system-x86 Tainted: G 2.6.27.19-5-default #1 RIP: 0010:[<ffffffffa012d8dc>] [<ffffffffa012d8dc>] rmap_remove+0xc9/0x19e [kvm] RSP: 0018:ffff880c7a1cbba8 EFLAGS: 00010296 RAX: 0000000000000039 RBX: 00000036e65c0336 RCX: ffff880c7b405e60 RDX: ffffffff806e0d08 RSI: 0000000000000092 RDI: ffffffff806e0d00 RBP: ffff880a08e00098 R08: ffffffff806e0cf0 R09: 0000000100000000 R10: 0000000000000046 R11: 000000000000000a R12: ffff880c7b066a20 R13: ffff8806778e0000 R14: 0000000000000000 R15: 0000000000000007 FS: 00007f298b4ad950(0000) GS:ffff880c7cd83f40(0000) knlGS:0000000000000000 CS: 0010 DS: 002b ES: 002b CR0: 000000008005003b CR2: 0000000000879ba0 CR3: 0000000679da8000 CR4: 00000000000026e0 DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Process qemu-system-x86 (pid: 17635, threadinfo ffff880c7a1ca000, task ffff8809ebce4880) Stack: ffff88069822b888 0000000000000000 ffff8803f1000040 ffff880a08e00098 0000000000000001 ffffffffa012f5d3 ffff880c7a1cbc58 ffffffff8023661a 0000000000000000 8000000a08e000e7 ffffffff80228db7 00007f298e413fff Call Trace: [<ffffffffa012f5d3>] mmu_set_spte+0x98/0x302 [kvm] [<ffffffffa012ffa3>] __direct_map+0xee/0x1b8 [kvm] [<ffffffffa013014b>] tdp_page_fault+0xde/0x114 [kvm] [<ffffffffa0130f16>] kvm_mmu_page_fault+0x19/0x81 [kvm] [<ffffffffa012a64b>] kvm_arch_vcpu_ioctl_run+0x89b/0xaf2 [kvm] [<ffffffffa0123540>] kvm_vcpu_ioctl+0xf1/0x46b [kvm] [<ffffffff802bd249>] vfs_ioctl+0x21/0x6c [<ffffffff802bd4b6>] do_vfs_ioctl+0x222/0x231 [<ffffffff802bd516>] sys_ioctl+0x51/0x73 [<ffffffff8020bfbb>] system_call_fastpath+0x16/0x1b [<00007f298c3c3b77>] 0x7f298c3c3b77 Code: 80 00 00 00 48 8b 34 c1 e8 0c ff ff ff 49 89 c1 48 8b 00 48 85 c0 75 17 48 8b 55 00 48 89 ee 48 c7 c7 2f db 13 a0 e8 6d cc 36 e0 <0f> 0b eb fe a8 01 75 2a 48 39 c5 74 19 48 8b 55 00 48 89 ee 48 RIP [<ffffffffa012d8dc>] rmap_remove+0xc9/0x19e [kvm] RSP <ffff880c7a1cbba8> ---[ end trace 91e1d7963caa34a7 ]--- hugepage info: HugePages_Total: 10000 HugePages_Free: 7944 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB module info: filename: /lib/modules/2.6.27.19-5-default/extra/kvm-intel.ko license: GPL author: Qumranet version: kvm-devel srcversion: 9F14ECEFD8109654DFA20D2 depends: kvm vermagic: 2.6.27.19-5-default SMP mod_unload modversions parm: bypass_guest_pf:bool parm: vpid:bool parm: flexpriority:bool parm: ept:bool parm: emulate_invalid_guest_state:bool filename: /lib/modules/2.6.27.19-5-default/extra/kvm.ko license: GPL author: Qumranet version: kvm-devel srcversion: 157F8CB48FC31BC2F44847B depends: vermagic: 2.6.27.19-5-default SMP mod_unload modversions parm: oos_shadow:bool -- Ryan Harper Software Engineer; Linux Technology Center IBM Corp., Austin, Tx ry...@us.ibm.com -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
