[Bug 203994] bhyve kernel module may need to relax some checks when running nested under KVM
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203994 --- Comment #20 from Roman Bogorodskiy --- (In reply to Peter Grehan from comment #19) As for me, kernels >= 4.10 are fine for me. I use Fedora for my Linux-related stuff, Fedora 26 comes with 4.11, and Fedora 27 that will be released soon will have 4.13 I think. -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
[Bug 203994] bhyve kernel module may need to relax some checks when running nested under KVM
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203994 --- Comment #19 from Peter Grehan --- No updates. Do you have a specific kernel version that you are interested in ? If so I could restrict my testing to just that. My concern was that there would be a never-ending set of fidelity issues in KVM with kernels <= 4.10. -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
[Bug 203994] bhyve kernel module may need to relax some checks when running nested under KVM
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203994 --- Comment #18 from Roman Bogorodskiy --- Any updates? -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
[Bug 203994] bhyve kernel module may need to relax some checks when running nested under KVM
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203994 --- Comment #17 from Peter Grehan --- Just an update on this. I used Ubuntu since it was easy to get a range of Linux kernel versions to test with. - Ubuntu 17.04 (kernel 4.10) works without any changes to bhyve - Ubuntu 15.10 doesn't advertize INVVPID capability. While it would be possible to modify bhyve to work in this mode, I'm loathe to do it since it's such a niche case - Ubuntu 16.04 seems to have the capabilities that most folk have reported. After modifying bhyve to ignore the unused INVVPID capabilities and to fall-back to the all-context INVEPT if single-context wasn't available, the next issue was that Linux/KVM wasn't setting the GLA on EPT faults. This is used by bhyve to verify the GLA that is also obtained through the instruction emulation path for guest MMIO operations. This verification path hasn't thrown any errors for a few years now, so I relaxed this test when in this mode. (Of note, this bug was reported here https://lkml.org/lkml/2016/9/4/156, and the author has submitted a diff review to relax some of the INVVPID/INVEPT checks at https://reviews.freebsd.org/D10049) However, after doing all this, bhyve would still eventually panic with a kernel trap-9 fault when issuing a vmread instruction, so there may still be some additional KVM bugs that are being hit on this kernel rev :( I'll do some more experimentation and then post the patch that I have. -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
[Bug 203994] bhyve kernel module may need to relax some checks when running nested under KVM
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203994 Keith changed: What|Removed |Added CC||ke...@timeghost.com --- Comment #16 from Keith --- Hi, Add me to the list of folks with this issue. [root@freenas] ~# cpucontrol -m 0x480 /dev/cpuctl0 MSR 0x480: 0x00981000 0x11e57ed0 [root@freenas] ~# cpucontrol -m 0x482 /dev/cpuctl0 MSR 0x482: 0xfff9fffe 0x0401e172 [root@freenas] ~# cpucontrol -m 0x48b /dev/cpuctl0 MSR 0x48b: 0x00fb 0x [root@freenas] ~# cpucontrol -m 0x48c /dev/cpuctl0 MSR 0x48c: 0x0601 0x04114040 [root@freenas] ~# dmesg | grep CPU CPU: Intel(R) Xeon(R) CPU X5690 @ 3.47GHz (3458.07-MHz K8-class CPU) FreeBSD/SMP: Multiprocessor System Detected: 16 CPUs -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
[Bug 203994] bhyve kernel module may need to relax some checks when running nested under KVM
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203994 Martin Lucina changed: What|Removed |Added CC||mar...@lucina.net --- Comment #15 from Martin Lucina --- Hi, to add some more data points, this is Freebsd 11-RELEASE running in KVM (with nested=Y, Linux kernel 4.9.20, QEMU version 2.8.0(Debian 1:2.8+dfsg-3~bpo8+1): CPU: Intel(R) Core(TM) i5-2540M CPU @ 2.60GHz (2594.17-MHz K8-class CPU) MSR 0x480: 0x00d81000 0x11e57ed0 MSR 0x482: 0xfff9fffe 0x0401e172 MSR 0x48b: 0x00fb 0x MSR 0x48c: 0x0601 0x06114041 -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
[Bug 203994] bhyve kernel module may need to relax some checks when running nested under KVM
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203994 --- Comment #14 from Peter Grehan --- Thanks to those who posted MSR output. There are 3 issues: - the INST/OUTS capability isn't advertised on some KVM versions. I need to run a simple test to see if this matters or not in this environment. Hopefully it won't, and this test can be relaxed. - bhyve tests for all variants of INVVPID, but only uses the 2 that are advertised by KVM. This test will be dropped back to just those 2. - bhyve does use both forms of INVEPT, where some versions of KVM only advertise the coarse version. Since the finer-grained version is just an optimization, bhyve will be modified to only use the finer-grained version if it is available. I'll try and get a patch together shortly so these mods can be tested. -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
[Bug 203994] bhyve kernel module may need to relax some checks when running nested under KVM
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203994 --- Comment #13 from Peter Grehan --- For completeness, here's the output from FreeBSD running as a KVM guest on Ubuntu 16.04, itself running on VMWare Fusion, on an Intel i5-3230M MSR 0x480: 0x00981000 0x11e57ed0 MSR 0x482: 0xfff9fffe 0x0401e172 MSR 0x48b: 0x00f2 0x MSR 0x48c: 0x0601 0x04114040 -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
[Bug 203994] bhyve kernel module may need to relax some checks when running nested under KVM
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203994 --- Comment #12 from Tom M --- CPU: Intel(R) Xeon(R) CPU E3-1245 v3 @ 3.40GHz (3392.20-MHz K8-class CPU) cpucontrol -m 0x480 /dev/cpuctl0 MSR 0x480: 0x00981000 0x11e57ed0 cpucontrol -m 0x482 /dev/cpuctl0 MSR 0x482: 0xfff9fffe 0x0401e172 cpucontrol -m 0x48b /dev/cpuctl0 MSR 0x48b: 0x00fb 0x cpucontrol -m 0x48c /dev/cpuctl0 MSR 0x48c: 0x0601 0x04114040 CPU: Intel(R) Atom(TM) CPU C2750 @ 2.40GHz (2400.08-MHz K8-class CPU) cpucontrol -m 0x480 /dev/cpuctl0 MSR 0x480: 0x00981000 0x11e57ed0 cpucontrol -m 0x482 /dev/cpuctl0 MSR 0x482: 0xfff9fffe 0x0401e172 cpucontrol -m 0x48b /dev/cpuctl0 MSR 0x48b: 0x00eb 0x cpucontrol -m 0x48c /dev/cpuctl0 MSR 0x48c: 0x0601 0x04114040 -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
[Bug 203994] bhyve kernel module may need to relax some checks when running nested under KVM
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203994 ForkBender changed: What|Removed |Added CC||chrisanste...@gmail.com --- Comment #11 from ForkBender --- (In reply to Peter Grehan from comment #9) I have FreeNas running in Proxmox, and here's my output: # cpucontrol -m 0x480 /dev/cpuctl0 MSR 0x480: 0x00981000 0x11e57ed0 # cpucontrol -m 0x482 /dev/cpuctl0 MSR 0x482: 0xfff9fffe 0x0401e172 # cpucontrol -m 0x48b /dev/cpuctl0 MSR 0x48b: 0x00fb 0x # cpucontrol -m 0x48c /dev/cpuctl0 MSR 0x48c: 0x0601 0x04114040 plus a couple of other bits that might be useful: # uname -a FreeBSD freenas.local 11.0-STABLE FreeBSD 11.0-STABLE #6 ... # dmesg | grep CPU CPU: Intel(R) Xeon(R) CPU E31220 @ 3.10GHz (3092.89-MHz K8-class CPU) -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
[Bug 203994] bhyve kernel module may need to relax some checks when running nested under KVM
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203994 --- Comment #10 from Roman Bogorodskiy --- (In reply to Peter Grehan from comment #9) Here's what I have in FreeBSD (12-CURRENT as of Feb 22 2017) inside KVM (Fedora 25): # cpucontrol -m 0x480 /dev/cpuctl0 MSR 0x480: 0x00d81000 0x11e57ed0 # cpucontrol -m 0x482 /dev/cpuctl0 MSR 0x482: 0xfff9fffe 0x0401e172 # cpucontrol -m 0x48b /dev/cpuctl0 MSR 0x48b: 0x00fb 0x # cpucontrol -m 0x48c /dev/cpuctl0 MSR 0x48c: 0x0601 0x06114041 # in case if could be useful, cpu part of dmesg looks like this: CPU: Intel(R) Core(TM) i5-4690 CPU @ 3.50GHz (3492.07-MHz K8-class CPU) Origin="GenuineIntel" Id=0x306c3 Family=0x6 Model=0x3c Stepping=3 Features=0xf83fbff Features2=0xfffa3223 AMD Features=0x2c100800 AMD Features2=0x21 Structured Extended Features=0x72a XSAVE Features=0x1 VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID Hypervisor: Origin = "KVMKVMKVM" -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
[Bug 203994] bhyve kernel module may need to relax some checks when running nested under KVM
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203994 --- Comment #9 from Peter Grehan --- I've not gotten to it: will carve out some time this weekend. For those impacted, would you be able to post the output from the following commands from FreeBSD when running as a KVM guest ? kldload cpuctl cpucontrol -m 0x480 /dev/cpuctl0 cpucontrol -m 0x482 /dev/cpuctl0 cpucontrol -m 0x48b /dev/cpuctl0 cpucontrol -m 0x48c /dev/cpuctl0 Sample output from an Atom C2758: # cpucontrol -m 0x480 /dev/cpuctl0 MSR 0x480: 0x00da0400 0x0002 # cpucontrol -m 0x482 /dev/cpuctl0 MSR 0x482: 0xfff9fffe 0x0401e172 # cpucontrol -m 0x48b /dev/cpuctl0 MSR 0x48b: 0x28ef 0x # cpucontrol -m 0x48c /dev/cpuctl0 MSR 0x48c: 0x0f01 0x06114141 -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
[Bug 203994] bhyve kernel module may need to relax some checks when running nested under KVM
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203994 --- Comment #8 from Tom M --- Any update by chance? -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
[Bug 203994] bhyve kernel module may need to relax some checks when running nested under KVM
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203994 w...@ravok.de changed: What|Removed |Added CC||w...@ravok.de --- Comment #7 from w...@ravok.de --- Due to this bug I'm unable to run VMs and Docker-instances on FreeNAS Corral. I'm ready to test whatever patch you may throw at us. -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
[Bug 203994] bhyve kernel module may need to relax some checks when running nested under KVM
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203994 Tom M changed: What|Removed |Added CC||notfixin...@yahoo.com --- Comment #6 from Tom M --- Also willing to test patches when available. -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
[Bug 203994] bhyve kernel module may need to relax some checks when running nested under KVM
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203994 --- Comment #5 from Roman Bogorodskiy --- Will be happy to test bhyve with nested kvm when there are some patches ready. -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
[Bug 203994] bhyve kernel module may need to relax some checks when running nested under KVM
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203994 Ahmad A. changed: What|Removed |Added CC||aa...@outlook.com --- Comment #4 from Ahmad A. --- Hi. I thought I'd chime in and mention that bhyve doesn't work nested under Hyper-V either, at least not on my setup; the relevant lines from dmesg read: vmx_init: processor does not support desired basic capabilities module_register_init: MOD_LOAD (vmm, 0x826344e0, 0) error 22 VirtualBox (with VT-x and all) works fine though. -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
[Bug 203994] bhyve kernel module may need to relax some checks when running nested under KVM
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203994 --- Comment #3 from Peter Grehan --- No progress. Tbanks for the reminder - I have some test code that I can repurpose from another project that can at least determine if ins/outs exits work correctly, so will give that a try. -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
[Bug 203994] bhyve kernel module may need to relax some checks when running nested under KVM
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203994 Sönke Schau changed: What|Removed |Added CC||free...@schau.org --- Comment #2 from Sönke Schau --- Hello! Any progress here? This affects FreeeNAS 10 Installations running in KVM too. Neither Dockers nor VM Support - so no plugins. Best regards Sönke -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
[Bug 203994] bhyve kernel module may need to relax some checks when running nested under KVM
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203994 kay.d...@gmail.com changed: What|Removed |Added CC||kay.d...@gmail.com --- Comment #1 from kay.d...@gmail.com --- +1 Have the same issue under KVM: $ kldload vmm $ dmesg | egrep 'vmx|vmm' vmx_init: processor does not support desired basic capabilities module_register_init: MOD_LOAD (vmm, 0x81a14230, 0) error 22 -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
[Bug 203994] bhyve kernel module may need to relax some checks when running nested under KVM
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=203994 Mark Linimon changed: What|Removed |Added Assignee|freebsd-b...@freebsd.org|freebsd-virtualization@Free ||BSD.org -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
