Re: [Xen-devel] AMD VMMCALL and VM86 mode
On 25/12/2016 07:47, Suravee Suthikulpanit wrote: > > > On 12/12/16 01:37, Andrew Cooper wrote: >> On 11/12/16 17:33, Boris Ostrovsky wrote: >>> - andrew.coop...@citrix.com wrote: >>> On 09/12/16 19:55, Andrew Cooper wrote: > On 09/12/16 19:55, Boris Ostrovsky wrote: >> On 12/09/2016 02:01 PM, Andrew Cooper wrote: >>> Hello, >>> >>> While working on XSA-192, I found a curious thing. On AMD hardware, the >>> VMMCALL instruction appears to behave like a nop if executed in VM86 >>> mode. All other processor modes work fine. >>> >>> The documentation suggests it should be valid in any situation, but I >>> never get a #VMEXIT from it. >> And I assume GENERAL2_INTERCEPT_VMMCALL is set (which is what we have in >> Xen by default)? > Yes, because I have already used hypercalls to get text to the console > before entering vm86 mode. > >> What happens if you don't set it? > Let me do some hacking and see. Outside of vm86 mode, VMMCALL raises #UD, which is expected as it wasn't intercepted. From within vm86 mode, I now get #GP rather than #UD. There is certainly an argument to be made that VMMCALL inside vm86 mode should trap to the vm86 monitor and #GP would be the expected way of that happening, but this also doesn't match the documentation. >>> >>> Just curious: why do you think #GP could be a reasonable exception >>> here? >> >> In vm86 mode, #GP is raised for privileged instructions which should >> break for auditing in the vm86 monitor. It would be reasonable to >> include "talking to the hypervisor" as a privileged action. >> >>> It's #UD because if not intercepted it doesn't make sense to execute >>> it. >> >> I agree with this, if privilege isn't considered an issue. If a >> hypervisor doesn't actually get involved, the instruction shouldn't >> complete. >> >>> But either way, I think AMD should clarify this. Suravee, can you >>> find out what the expected behavior is? >> >> IMO, it should either consistently #GP and break to the vm86 monitor, or >> #UD/#VMEXIT depending on whether it is intercepted by the hypervisor. >> Either way the documentation should be clarified. >> >> Having said this, given that its behaviour is consistent on any AMD >> processor I choose to try, and given that vm86 mode is very legacy these >> days, I doubt a reasonable argument can be made to fixing the behaviour. >> >> ~Andrew >> > Appologies for the delays replying. To answer your questions out of order... > What processors are you using to reproduce this behavior? I am still out of the office until tomorrow, so haven't been able to do more thorough testing. I am observing expected behaviour on: (XEN) CPU Vendor: AMD, Family 16 (0x10), Model 2 (0x2), Stepping 3 (raw 00100f23) And observing incorrect behaviour on: (XEN) CPU Vendor: AMD, Family 21 (0x15), Model 96 (0x60), Stepping 1 (raw 00660f01) I note from your AVIC series that this machine should be capable, but it it is an SDP running with microcode version 0x600610b which (now I think about it) could easily be the source of this issue. > So, just to confirm your observation. When executing VMMCALL in vm86 > mode with the hypervisor is set to intercept VMMCALL and not-intercept > VMMCALL, you are seeing #GP in both cases or just in the later? Seeing #GP when not intercepted, and a something resembling a nop when intercepted. > According to the HW folks, in the vm86 mode, we should observe the > same behavior as in protected mode. (e.g. #UD in guest if not > intercept VMMCALL, and #vmexit if intercept) Having checked on the one other processor I have easy external access to, this does indeed appear to be the case for Fam 0x10 processors. > Could you please describe how you are reproducing this behavior? I have a short XTF test which demonstrates the issue, but it would probably be better to rule out SDP/microcode issues first. I can find a BKDG for Fam 15h, Model 60h-6fh processors, but not a revision guide. Where can I find and get the latest microcode? Thanks, ~Andrew ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] AMD VMMCALL and VM86 mode
On 12/12/16 01:37, Andrew Cooper wrote: On 11/12/16 17:33, Boris Ostrovsky wrote: - andrew.coop...@citrix.com wrote: On 09/12/16 19:55, Andrew Cooper wrote: On 09/12/16 19:55, Boris Ostrovsky wrote: On 12/09/2016 02:01 PM, Andrew Cooper wrote: Hello, While working on XSA-192, I found a curious thing. On AMD hardware, the VMMCALL instruction appears to behave like a nop if executed in VM86 mode. All other processor modes work fine. The documentation suggests it should be valid in any situation, but I never get a #VMEXIT from it. And I assume GENERAL2_INTERCEPT_VMMCALL is set (which is what we have in Xen by default)? Yes, because I have already used hypercalls to get text to the console before entering vm86 mode. What happens if you don't set it? Let me do some hacking and see. Outside of vm86 mode, VMMCALL raises #UD, which is expected as it wasn't intercepted. From within vm86 mode, I now get #GP rather than #UD. There is certainly an argument to be made that VMMCALL inside vm86 mode should trap to the vm86 monitor and #GP would be the expected way of that happening, but this also doesn't match the documentation. Just curious: why do you think #GP could be a reasonable exception here? In vm86 mode, #GP is raised for privileged instructions which should break for auditing in the vm86 monitor. It would be reasonable to include "talking to the hypervisor" as a privileged action. It's #UD because if not intercepted it doesn't make sense to execute it. I agree with this, if privilege isn't considered an issue. If a hypervisor doesn't actually get involved, the instruction shouldn't complete. But either way, I think AMD should clarify this. Suravee, can you find out what the expected behavior is? IMO, it should either consistently #GP and break to the vm86 monitor, or #UD/#VMEXIT depending on whether it is intercepted by the hypervisor. Either way the documentation should be clarified. Having said this, given that its behaviour is consistent on any AMD processor I choose to try, and given that vm86 mode is very legacy these days, I doubt a reasonable argument can be made to fixing the behaviour. ~Andrew So, just to confirm your observation. When executing VMMCALL in vm86 mode with the hypervisor is set to intercept VMMCALL and not-intercept VMMCALL, you are seeing #GP in both cases or just in the later? According to the HW folks, in the vm86 mode, we should observe the same behavior as in protected mode. (e.g. #UD in guest if not intercept VMMCALL, and #vmexit if intercept) Could you please describe how you are reproducing this behavior? What processors are you using to reproduce this behavior? Thanks, Suravee ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] AMD VMMCALL and VM86 mode
Hi Andrew/Boris, On 12/12/16 01:37, Andrew Cooper wrote: On 11/12/16 17:33, Boris Ostrovsky wrote: - andrew.coop...@citrix.com wrote: On 09/12/16 19:55, Andrew Cooper wrote: On 09/12/16 19:55, Boris Ostrovsky wrote: On 12/09/2016 02:01 PM, Andrew Cooper wrote: Hello, While working on XSA-192, I found a curious thing. On AMD hardware, the VMMCALL instruction appears to behave like a nop if executed in VM86 mode. All other processor modes work fine. The documentation suggests it should be valid in any situation, but I never get a #VMEXIT from it. And I assume GENERAL2_INTERCEPT_VMMCALL is set (which is what we have in Xen by default)? Yes, because I have already used hypercalls to get text to the console before entering vm86 mode. What happens if you don't set it? Let me do some hacking and see. Outside of vm86 mode, VMMCALL raises #UD, which is expected as it wasn't intercepted. From within vm86 mode, I now get #GP rather than #UD. There is certainly an argument to be made that VMMCALL inside vm86 mode should trap to the vm86 monitor and #GP would be the expected way of that happening, but this also doesn't match the documentation. Just curious: why do you think #GP could be a reasonable exception here? In vm86 mode, #GP is raised for privileged instructions which should break for auditing in the vm86 monitor. It would be reasonable to include "talking to the hypervisor" as a privileged action. It's #UD because if not intercepted it doesn't make sense to execute it. I agree with this, if privilege isn't considered an issue. If a hypervisor doesn't actually get involved, the instruction shouldn't complete. But either way, I think AMD should clarify this. Suravee, can you find out what the expected behavior is? IMO, it should either consistently #GP and break to the vm86 monitor, or #UD/#VMEXIT depending on whether it is intercepted by the hypervisor. Either way the documentation should be clarified. Having said this, given that its behaviour is consistent on any AMD processor I choose to try, and given that vm86 mode is very legacy these days, I doubt a reasonable argument can be made to fixing the behaviour. ~Andrew I'm checking with the HW folks. Let me get back to you. Suravee ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] AMD VMMCALL and VM86 mode
On 11/12/16 17:33, Boris Ostrovsky wrote: > - andrew.coop...@citrix.com wrote: > >> On 09/12/16 19:55, Andrew Cooper wrote: >>> On 09/12/16 19:55, Boris Ostrovsky wrote: On 12/09/2016 02:01 PM, Andrew Cooper wrote: > Hello, > > While working on XSA-192, I found a curious thing. On AMD >> hardware, the > VMMCALL instruction appears to behave like a nop if executed in >> VM86 > mode. All other processor modes work fine. > > The documentation suggests it should be valid in any situation, >> but I > never get a #VMEXIT from it. And I assume GENERAL2_INTERCEPT_VMMCALL is set (which is what we >> have in Xen by default)? >>> Yes, because I have already used hypercalls to get text to the >> console >>> before entering vm86 mode. >>> What happens if you don't set it? >>> Let me do some hacking and see. >> Outside of vm86 mode, VMMCALL raises #UD, which is expected as it >> wasn't >> intercepted. >> >> From within vm86 mode, I now get #GP rather than #UD. >> >> There is certainly an argument to be made that VMMCALL inside vm86 >> mode >> should trap to the vm86 monitor and #GP would be the expected way of >> that happening, but this also doesn't match the documentation. > > Just curious: why do you think #GP could be a reasonable exception here? In vm86 mode, #GP is raised for privileged instructions which should break for auditing in the vm86 monitor. It would be reasonable to include "talking to the hypervisor" as a privileged action. > It's #UD because if not intercepted it doesn't make sense to execute it. I agree with this, if privilege isn't considered an issue. If a hypervisor doesn't actually get involved, the instruction shouldn't complete. > But either way, I think AMD should clarify this. Suravee, can you find out > what the expected behavior is? IMO, it should either consistently #GP and break to the vm86 monitor, or #UD/#VMEXIT depending on whether it is intercepted by the hypervisor. Either way the documentation should be clarified. Having said this, given that its behaviour is consistent on any AMD processor I choose to try, and given that vm86 mode is very legacy these days, I doubt a reasonable argument can be made to fixing the behaviour. ~Andrew ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] AMD VMMCALL and VM86 mode
- andrew.coop...@citrix.com wrote: > On 09/12/16 19:55, Andrew Cooper wrote: > > On 09/12/16 19:55, Boris Ostrovsky wrote: > >> On 12/09/2016 02:01 PM, Andrew Cooper wrote: > >>> Hello, > >>> > >>> While working on XSA-192, I found a curious thing. On AMD > hardware, the > >>> VMMCALL instruction appears to behave like a nop if executed in > VM86 > >>> mode. All other processor modes work fine. > >>> > >>> The documentation suggests it should be valid in any situation, > but I > >>> never get a #VMEXIT from it. > >> And I assume GENERAL2_INTERCEPT_VMMCALL is set (which is what we > have in > >> Xen by default)? > > Yes, because I have already used hypercalls to get text to the > console > > before entering vm86 mode. > > > >> What happens if you don't set it? > > Let me do some hacking and see. > > Outside of vm86 mode, VMMCALL raises #UD, which is expected as it > wasn't > intercepted. > > From within vm86 mode, I now get #GP rather than #UD. > > There is certainly an argument to be made that VMMCALL inside vm86 > mode > should trap to the vm86 monitor and #GP would be the expected way of > that happening, but this also doesn't match the documentation. Just curious: why do you think #GP could be a reasonable exception here? It's #UD because if not intercepted it doesn't make sense to execute it. But either way, I think AMD should clarify this. Suravee, can you find out what the expected behavior is? -boris ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] AMD VMMCALL and VM86 mode
On 09/12/16 19:55, Andrew Cooper wrote: > On 09/12/16 19:55, Boris Ostrovsky wrote: >> On 12/09/2016 02:01 PM, Andrew Cooper wrote: >>> Hello, >>> >>> While working on XSA-192, I found a curious thing. On AMD hardware, the >>> VMMCALL instruction appears to behave like a nop if executed in VM86 >>> mode. All other processor modes work fine. >>> >>> The documentation suggests it should be valid in any situation, but I >>> never get a #VMEXIT from it. >> And I assume GENERAL2_INTERCEPT_VMMCALL is set (which is what we have in >> Xen by default)? > Yes, because I have already used hypercalls to get text to the console > before entering vm86 mode. > >> What happens if you don't set it? > Let me do some hacking and see. Outside of vm86 mode, VMMCALL raises #UD, which is expected as it wasn't intercepted. From within vm86 mode, I now get #GP rather than #UD. There is certainly an argument to be made that VMMCALL inside vm86 mode should trap to the vm86 monitor and #GP would be the expected way of that happening, but this also doesn't match the documentation. ~Andrew ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] AMD VMMCALL and VM86 mode
On 09/12/16 19:55, Boris Ostrovsky wrote: > On 12/09/2016 02:01 PM, Andrew Cooper wrote: >> Hello, >> >> While working on XSA-192, I found a curious thing. On AMD hardware, the >> VMMCALL instruction appears to behave like a nop if executed in VM86 >> mode. All other processor modes work fine. >> >> The documentation suggests it should be valid in any situation, but I >> never get a #VMEXIT from it. > And I assume GENERAL2_INTERCEPT_VMMCALL is set (which is what we have in > Xen by default)? Yes, because I have already used hypercalls to get text to the console before entering vm86 mode. > What happens if you don't set it? Let me do some hacking and see. ~Andrew ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] AMD VMMCALL and VM86 mode
On 12/09/2016 02:01 PM, Andrew Cooper wrote: > Hello, > > While working on XSA-192, I found a curious thing. On AMD hardware, the > VMMCALL instruction appears to behave like a nop if executed in VM86 > mode. All other processor modes work fine. > > The documentation suggests it should be valid in any situation, but I > never get a #VMEXIT from it. And I assume GENERAL2_INTERCEPT_VMMCALL is set (which is what we have in Xen by default)? What happens if you don't set it? -boris > Thus, I would have thought it would fall > into the un-intercepted category and raise a #UD fault, but I don't get > that either. > > Is this behaviour expected? The documentation would certainly seem to > indicate not. ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
