Re: [kvm-devel] [PATCH] enable gfxboot on VMX
Anthony Liguori wrote: > > Keep in mind, this is not big real mode. The gfxboot code here was > actually rewritten to not use big real mode so that it worked with > Xen. It works with Xen because Xen uses vm86 mode within the host > whereas KVM uses vm86 mode in the guest's VT context. I don't think that's right. vm86 mode doesn't work under long mode, so I don't see how Xen can use it in host context. -- error compiling committee.c: too many arguments to function - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] [PATCH] enable gfxboot on VMX
Alexander Graf wrote: > Real mode segments are mere offsets. They get lshifted by 4 (as far as > I remember) and just added to the offset. SS is on some value here > Yes, this is correct. > 0x00046e60: and$0x,%esp > 0x00046e66: shl$0x4,%eax > 0x00046e69: add%eax,%esp > 0x00046e6b: mov$0x8,%ax > 0x00046e6f: mov%eax,%ss > 0x00046e71: mov%eax,%es > 0x00046e73: mov%eax,%fs > 0x00046e75: mov%eax,%gs > > set the segments to the selector we just calculated > > [...] > > So what could be the problem with this code? It looks slightly hacky, > but works on most x86 computers (some VIAs choke here too). > > After the vmexit on writing cr0, we are in a somewhat undefined state. > For most of it, we are in protected mode by now. But the segments have > not been set to something meaningful in protected mode, so they still > are on their old values. > > Unfortunately VMX does a lot of sanity checks, if the virtual machine > is actually valid and can be entered into, when trying to do so. One > of these checks is 'CS.RPL == SS.RPL'. The RPL are lowest 2 bits (I am > not too sure about this, read in the manuals if you really need to > know) of the selector. We don't have a real selector in this > transitioning phase though, but instead have real mode offsets in > them. This means, if CS was on 0x0 and SS on 0x1235, there is no way > the RPL equals condition can be met. > Keep in mind, this is not big real mode. The gfxboot code here was actually rewritten to not use big real mode so that it worked with Xen. It works with Xen because Xen uses vm86 mode within the host whereas KVM uses vm86 mode in the guest's VT context. An alternative approach to fixing this problem (but in a less hackish way) would be to simply use the host's vm86 mode instead of vm86 mode within the guest's VT context. However, this does not fix big real mode. For that, you need to do proper 16-bit instruction emulation. Regards, Anthony Liguori - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] [PATCH] enable gfxboot on VMX
On Feb 18, 2008, at 12:46 PM, Andi Kleen wrote: > Alexander Graf <[EMAIL PROTECTED]> writes: >> >> As everything except gfxboot works, we can simply change gfxboot in >> runtime to use a different value. Unfortunately the mov instruction, >> used to read the SS register is only 2 bytes long, so there is no >> way to >> binary patch the mov to something that would contain an address. So >> the >> only way I could think of was an invalid instruction. The UD >> exception >> is intercepted in KVM and is already emulated for VMCALLs. This can >> be >> extended to an opcode, that is officially unused (0f 0c) and have the > > It quite is dangerous to use unused opcodes like this. Intel/AMD tend > to regularly add new instructions with new CPUs, so currently unused > opcodes can quick turn into used ones which don't trap anymore or trap > differently or do something unexpected. And yes this has broken > software in the past. > > If you want to continue to use this method (it is unclear given the > rest of the thread), I would recommend you keep track of the patched I don't think Avi wants anything as hacky in the kernel. > > locations instead and use some documented to be trapping 1 or 2 byte > opcode > and then check in the #UD handler if you hit one of the previously > patched locations. This is quite a nice idea I thought of as well (that's what the 'generic' binary patching approach would look like). Nevertheless it'd be a lot better to have something that works for all cases without patching binaries, as we might get into TPM problems with that either way. > > > Given that solution would have some corner cases (e.g. the table > would likely need to have a fixed upper limit), but at least for the > gfxboot case just using a very small limited table would be probably > enough. For the gfxboot case a simple variable would be enough. There is only one position where this UD would occur, so it's only one IP to save. Nevertheless please think of this as a hack, not a solution. I only wanted to have something for the meantime while no real fix is available. As I don't have the time right now to actually write a proper one, this was the only way to have something that 'works' at all right now. I strongly hope we will have a working solution until the point where CPUs show up, which will have this opcode used. Alex - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] [PATCH] enable gfxboot on VMX
Alexander Graf <[EMAIL PROTECTED]> writes: > > As everything except gfxboot works, we can simply change gfxboot in > runtime to use a different value. Unfortunately the mov instruction, > used to read the SS register is only 2 bytes long, so there is no way to > binary patch the mov to something that would contain an address. So the > only way I could think of was an invalid instruction. The UD exception > is intercepted in KVM and is already emulated for VMCALLs. This can be > extended to an opcode, that is officially unused (0f 0c) and have the It quite is dangerous to use unused opcodes like this. Intel/AMD tend to regularly add new instructions with new CPUs, so currently unused opcodes can quick turn into used ones which don't trap anymore or trap differently or do something unexpected. And yes this has broken software in the past. If you want to continue to use this method (it is unclear given the rest of the thread), I would recommend you keep track of the patched locations instead and use some documented to be trapping 1 or 2 byte opcode and then check in the #UD handler if you hit one of the previously patched locations. Given that solution would have some corner cases (e.g. the table would likely need to have a fixed upper limit), but at least for the gfxboot case just using a very small limited table would be probably enough. -Andi - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] [PATCH] enable gfxboot on VMX
On Feb 18, 2008, at 10:34 AM, Alexander Graf wrote:
>
> On Feb 18, 2008, at 10:17 AM, Guillaume Thouvenin wrote:
>
>> On Sat, 16 Feb 2008 14:34:09 +0100
>> Alexander Graf <[EMAIL PROTECTED]> wrote:
>>
Whenever the register state becomes consistent with VT again.
vmx_set_segment() looks like the right point for turning it off.
>>>
>>> Sounds good. As basically the only problem we have are the sanity
>>> checks done on VMENTER, this should work.
>>
>> Hello,
>>
>> I tried to detect VMentry failure in order to do real mode emulation.
>> I tested the patch (pasted at the end of this message) with the
>> installation of OpenSuse 10.3 but it failed to detect the VMentry
>> failure. I suspect that on my computer the failure is not due to a
>> VMentry failure. Can you test the patch and tell me if it detects a
>> VMentry failure?
>
> You won't receive any VMentry failure as is. I'll try to explain the
> real problem to you.
>
> This is the real-mode to protected-mode switch. I will comment the
> assembler commands if important for the issue.
>
> Real mode segments are mere offsets. They get lshifted by 4 (as far as
> I remember) and just added to the offset. SS is on some value here
> (can be 0x0, can be 0x5380, can be whatever)
>
> [...]
> 0x00046e26: cli
> 0x00046e27: movw %ss,%cs:2256
> 0x00046e2c: movw %ds,%cs:2260
> 0x00046e31: movw %es,%cs:2262
> 0x00046e36: movw %fs,%cs:2264
> 0x00046e3b: movw %gs,%cs:2266
> 0x00046e40: or $0x1,%al
>
> enable protected mode flag in eax
>
> 0x00046e42: lgdtl %cs:2250
> 0x00046e49: lidtl %cs:2238
> 0x00046e50: mov%eax,%cr0
>
> enable protected mode in cr0 by writing the pm flag
> -- we have a vmexit here --
>
> 0x00046e53: ljmp $0x18,$0x6e18
>
> ljmp to the protected mode segment 0x18 (CS), so we really are in
> protected mode
>
> 0x00046e58: mov$0x20,%ax
> 0x00046e5c: mov%eax,%ds
> 0x00046e5e: mov%ss,%eax
>
> read SS
> and calculate the segment based on the address
Actually esp gets calculated, based on the old SS address + SP
>
>
> 0x00046e60: and$0x,%esp
> 0x00046e66: shl$0x4,%eax
> 0x00046e69: add%eax,%esp
> 0x00046e6b: mov$0x8,%ax
> 0x00046e6f: mov%eax,%ss
> 0x00046e71: mov%eax,%es
> 0x00046e73: mov%eax,%fs
> 0x00046e75: mov%eax,%gs
>
> set the segments to the selector we just calculated
and the other segments get set to 0x8
>
>
> [...]
>
> So what could be the problem with this code? It looks slightly hacky,
> but works on most x86 computers (some VIAs choke here too).
>
> After the vmexit on writing cr0, we are in a somewhat undefined state.
> For most of it, we are in protected mode by now. But the segments have
> not been set to something meaningful in protected mode, so they still
> are on their old values.
>
> Unfortunately VMX does a lot of sanity checks, if the virtual machine
> is actually valid and can be entered into, when trying to do so. One
> of these checks is 'CS.RPL == SS.RPL'. The RPL are lowest 2 bits (I am
> not too sure about this, read in the manuals if you really need to
> know) of the selector. We don't have a real selector in this
> transitioning phase though, but instead have real mode offsets in
> them. This means, if CS was on 0x0 and SS on 0x1235, there is no way
> the RPL equals condition can be met.
>
> This is why in the vmx.c you can find in enter_pmode() code like this:
>
> vmcs_write16(GUEST_SS_SELECTOR, 0);
> vmcs_write32(GUEST_SS_AR_BYTES, 0x93);
>
> This sets SS to 0 whenever the above mov to cr0 occurs. As most code
> does not rely on SS being available after a PM transition, but instead
> simply writes a sane value to it, that's fine for 99.9% of all code
> out there. Unfortunately it is not for gfxboot.
>
> So if you want to see a VMentry failure, just remove the SS patching
> and you'll see one. My guess would be that you see a lot of problems
> with otherwise working code too then, though, as SS can be anything in
> that state.
>
> I hope this helped,
>
> Alex
>
>
>>
>>
>> Thanks for your help,
>> Regards,
>>
>> Guillaume
>>
>> ---
>> Index: kvm/arch/x86/kvm/vmx.c
>> ===
>> --- kvm.orig/arch/x86/kvm/vmx.c 2008-02-18 09:22:53.0 +0100
>> +++ kvm/arch/x86/kvm/vmx.c 2008-02-18 09:43:13.0 +0100
>> @@ -2255,6 +2255,15 @@
>> static const int kvm_vmx_max_exit_handlers =
>> ARRAY_SIZE(kvm_vmx_exit_handlers);
>>
>> +static int kvm_handle_vmentry_failure(struct kvm_run *kvm_run,
>> struct kvm_vcpu *vcpu)
>> +{
>> +printk(KERN_WARNING "VMENTRY failure detected \n");
>> +if (vcpu->arch.rmode.active)
>> +printk(KERN_WARNING "Big Real Mode emulation needed \n");
>> +
>> +return 0;
>> +}
>> +
>> /*
>> * The guest has exited. See if we
Re: [kvm-devel] [PATCH] enable gfxboot on VMX
On Feb 18, 2008, at 10:17 AM, Guillaume Thouvenin wrote:
> On Sat, 16 Feb 2008 14:34:09 +0100
> Alexander Graf <[EMAIL PROTECTED]> wrote:
>
>>> Whenever the register state becomes consistent with VT again.
>>> vmx_set_segment() looks like the right point for turning it off.
>>
>> Sounds good. As basically the only problem we have are the sanity
>> checks done on VMENTER, this should work.
>
> Hello,
>
> I tried to detect VMentry failure in order to do real mode emulation.
> I tested the patch (pasted at the end of this message) with the
> installation of OpenSuse 10.3 but it failed to detect the VMentry
> failure. I suspect that on my computer the failure is not due to a
> VMentry failure. Can you test the patch and tell me if it detects a
> VMentry failure?
You won't receive any VMentry failure as is. I'll try to explain the
real problem to you.
This is the real-mode to protected-mode switch. I will comment the
assembler commands if important for the issue.
Real mode segments are mere offsets. They get lshifted by 4 (as far as
I remember) and just added to the offset. SS is on some value here
(can be 0x0, can be 0x5380, can be whatever)
[...]
0x00046e26: cli
0x00046e27: movw %ss,%cs:2256
0x00046e2c: movw %ds,%cs:2260
0x00046e31: movw %es,%cs:2262
0x00046e36: movw %fs,%cs:2264
0x00046e3b: movw %gs,%cs:2266
0x00046e40: or $0x1,%al
enable protected mode flag in eax
0x00046e42: lgdtl %cs:2250
0x00046e49: lidtl %cs:2238
0x00046e50: mov%eax,%cr0
enable protected mode in cr0 by writing the pm flag
-- we have a vmexit here --
0x00046e53: ljmp $0x18,$0x6e18
ljmp to the protected mode segment 0x18 (CS), so we really are in
protected mode
0x00046e58: mov$0x20,%ax
0x00046e5c: mov%eax,%ds
0x00046e5e: mov%ss,%eax
read SS
and calculate the segment based on the address
0x00046e60: and$0x,%esp
0x00046e66: shl$0x4,%eax
0x00046e69: add%eax,%esp
0x00046e6b: mov$0x8,%ax
0x00046e6f: mov%eax,%ss
0x00046e71: mov%eax,%es
0x00046e73: mov%eax,%fs
0x00046e75: mov%eax,%gs
set the segments to the selector we just calculated
[...]
So what could be the problem with this code? It looks slightly hacky,
but works on most x86 computers (some VIAs choke here too).
After the vmexit on writing cr0, we are in a somewhat undefined state.
For most of it, we are in protected mode by now. But the segments have
not been set to something meaningful in protected mode, so they still
are on their old values.
Unfortunately VMX does a lot of sanity checks, if the virtual machine
is actually valid and can be entered into, when trying to do so. One
of these checks is 'CS.RPL == SS.RPL'. The RPL are lowest 2 bits (I am
not too sure about this, read in the manuals if you really need to
know) of the selector. We don't have a real selector in this
transitioning phase though, but instead have real mode offsets in
them. This means, if CS was on 0x0 and SS on 0x1235, there is no way
the RPL equals condition can be met.
This is why in the vmx.c you can find in enter_pmode() code like this:
vmcs_write16(GUEST_SS_SELECTOR, 0);
vmcs_write32(GUEST_SS_AR_BYTES, 0x93);
This sets SS to 0 whenever the above mov to cr0 occurs. As most code
does not rely on SS being available after a PM transition, but instead
simply writes a sane value to it, that's fine for 99.9% of all code
out there. Unfortunately it is not for gfxboot.
So if you want to see a VMentry failure, just remove the SS patching
and you'll see one. My guess would be that you see a lot of problems
with otherwise working code too then, though, as SS can be anything in
that state.
I hope this helped,
Alex
>
>
> Thanks for your help,
> Regards,
>
> Guillaume
>
> ---
> Index: kvm/arch/x86/kvm/vmx.c
> ===
> --- kvm.orig/arch/x86/kvm/vmx.c 2008-02-18 09:22:53.0 +0100
> +++ kvm/arch/x86/kvm/vmx.c2008-02-18 09:43:13.0 +0100
> @@ -2255,6 +2255,15 @@
> static const int kvm_vmx_max_exit_handlers =
> ARRAY_SIZE(kvm_vmx_exit_handlers);
>
> +static int kvm_handle_vmentry_failure(struct kvm_run *kvm_run,
> struct kvm_vcpu *vcpu)
> +{
> + printk(KERN_WARNING "VMENTRY failure detected \n");
> + if (vcpu->arch.rmode.active)
> + printk(KERN_WARNING "Big Real Mode emulation needed \n");
> +
> + return 0;
> +}
> +
> /*
> * The guest has exited. See if we can fix it or if we need userspace
> * assistance.
> @@ -2265,6 +2274,9 @@
> struct vcpu_vmx *vmx = to_vmx(vcpu);
> u32 vectoring_info = vmx->idt_vectoring_info;
>
> + if (unlikely(exit_reason & VMX_EXIT_REASONS_FAILED_VMENTRY))
> + return kvm_handle_vmentry_failure(kvm_run, vcpu);
>
Re: [kvm-devel] [PATCH] enable gfxboot on VMX
On Sat, 16 Feb 2008 14:34:09 +0100
Alexander Graf <[EMAIL PROTECTED]> wrote:
> > Whenever the register state becomes consistent with VT again.
> > vmx_set_segment() looks like the right point for turning it off.
>
> Sounds good. As basically the only problem we have are the sanity
> checks done on VMENTER, this should work.
Hello,
I tried to detect VMentry failure in order to do real mode emulation.
I tested the patch (pasted at the end of this message) with the
installation of OpenSuse 10.3 but it failed to detect the VMentry
failure. I suspect that on my computer the failure is not due to a
VMentry failure. Can you test the patch and tell me if it detects a
VMentry failure?
Thanks for your help,
Regards,
Guillaume
---
Index: kvm/arch/x86/kvm/vmx.c
===
--- kvm.orig/arch/x86/kvm/vmx.c 2008-02-18 09:22:53.0 +0100
+++ kvm/arch/x86/kvm/vmx.c 2008-02-18 09:43:13.0 +0100
@@ -2255,6 +2255,15 @@
static const int kvm_vmx_max_exit_handlers =
ARRAY_SIZE(kvm_vmx_exit_handlers);
+static int kvm_handle_vmentry_failure(struct kvm_run *kvm_run, struct kvm_vcpu
*vcpu)
+{
+ printk(KERN_WARNING "VMENTRY failure detected \n");
+ if (vcpu->arch.rmode.active)
+ printk(KERN_WARNING "Big Real Mode emulation needed \n");
+
+ return 0;
+}
+
/*
* The guest has exited. See if we can fix it or if we need userspace
* assistance.
@@ -2265,6 +2274,9 @@
struct vcpu_vmx *vmx = to_vmx(vcpu);
u32 vectoring_info = vmx->idt_vectoring_info;
+ if (unlikely(exit_reason & VMX_EXIT_REASONS_FAILED_VMENTRY))
+ return kvm_handle_vmentry_failure(kvm_run, vcpu);
+
if (unlikely(vmx->fail)) {
kvm_run->exit_reason = KVM_EXIT_FAIL_ENTRY;
kvm_run->fail_entry.hardware_entry_failure_reason
-
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/
___
kvm-devel mailing list
kvm-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] [PATCH] enable gfxboot on VMX
Alexander Graf wrote: > > Emulating all of the real mode shouldn't be too much of a problem on > the performance side. I wouldn't be surprised if the vmenter/exits > take about as much time as the emulation overhead. > For the bootstrap process emulation is good enough, since the process is short enough. Good real mode performance may be necessary for applications like ghost which are still used for deployment. In addition, if a real mode app uses x87/mmx/sse, then you are forced to add emulation for these instructions if you don't use v8086. Of course, if that app also uses big real mode, you are forced to do so anyway, >>> While this should fix more problems, the one thing I am concerned >>> about is that I have not encountered any other code that does have >>> this problem. >> >> I think some Ubuntus use big real mode, which can use the same fix. > > Do you have any file / pointer to where I could get one? I did try the > feisty server iso which worked just fine. No, sorry. This is just from memory. >> I don't think there's much work to get protected mode emulation >> working. There aren't that many instructions before we get to a >> vt-friendly state (a couple dozen?) and some of them are already >> implemented. > > The hardest one being ljmp. You need to do the whole pm transition in > the emulator then. I believe there is a reason this hasn't been done yet? > No reason other than it's extremely icky. Looks like Xen has implemented it though. >> An alternative is to work around it in userspace. If we recognise the >> exit reason, we can read the instructions around rip and attempt to >> fix things up. > > So just get the CR0 write and UD exception as event to the userspace? > I'd really love that approach. The "invalid opcode" hack, as I > implemented it, is actually quite extensible. You could simply put the > rip and an operation that is supposed to occur in a list and emulate > whatever comes when the UD occurs. This might be the easiest way to > fix things. I meant trapping the vmentry failure (this is propagated to userspace anyway), recognizing gfxboot, fixing up the state, and continuing. > > We could also have something more extensible, say a "generic binary > patching" framework, so we know that if memory page 0x1234000 contains > specific content, just patch it and apply a "what happens in case of > invalid opcodes" script. This could all be in userspace and should > enable us to circumvent most problems in a generic way. > I can't say I'm thrilled about it when a real fix is possible. > > Don't get me wrong on this - I really want to see something "right". I > just don't see anyone working on it, as there are a lot of places KVM > improves right now, which are a lot more important than real mode > fixes. Usually real mode is completely unused as soon as you're done > with bootstrapping, so why care about it that much? As I mentioned earlier, deployment apps like ghost. > > I'm also perfectly fine with this not being merged. I built this hack > for me, because I was rather unhappy with the situation as is and > wanted to see gfxboot working, as I couldn't just "plug in" a current > iso and install from that. If anyone benefits from it, I'm fine with > it. If not, that's ok with me too. I just couldn't stand the situation > that no fix was available at all (disabling gfxboot was no fix for me). Well, I too would really like to see a fix for gfxboot merged, but I want to avoid patching whenever possible. -- Any sufficiently difficult bug is indistinguishable from a feature. - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] [PATCH] enable gfxboot on VMX
On Feb 16, 2008, at 10:06 AM, Avi Kivity wrote: > Alexander Graf wrote: >>> >>> While enabling gfxboot over vmx is very desirable, I'd like to avoid >>> guest-specific hacks. IMO the correct fix is to set a >>> "non_vt_friendly" >>> flag when switching from real mode to protected mode, then continue >>> emulation, re-computing the flag after every instruction. After a >>> few >>> instruction, the condition disappears and we can enter guest mode >>> again. >>> >> >> So when would the flag disappear? > > Whenever the register state becomes consistent with VT again. > vmx_set_segment() looks like the right point for turning it off. Sounds good. As basically the only problem we have are the sanity checks done on VMENTER, this should work. >> Basically an SS read can happen any time after the switch, even >> after 100 instructions if it has to. > > That's okay. Most (all) code will reload all segments shortly after > the protected mode switch. > > I am concerned about switches to real-mode not reloading fs and gs, > so we'd stay in vt-unfriendly state and emulate even though the real > mode code doesn't use fs and gs at all. I believe we can live with > it; newer Xen for example emulates 100% of real mode code. Emulating all of the real mode shouldn't be too much of a problem on the performance side. I wouldn't be surprised if the vmenter/exits take about as much time as the emulation overhead. >> While this should fix more problems, the one thing I am concerned >> about is that I have not encountered any other code that does have >> this problem. > > I think some Ubuntus use big real mode, which can use the same fix. Do you have any file / pointer to where I could get one? I did try the feisty server iso which worked just fine. >>> The downside is that we have to implement more instructions in the >>> emulator for this, but these instructions will be generally >>> useful, not >>> just for gfxboot. >> >> I am not trying to talk you into anything - I would very much >> prefer a rather clean solution as well. Nevertheless I do not see >> full protected mode emulation code coming in the very near future >> and on a user perspective would prefer to have something that >> works, even if it's ugly. >> So while KVM is able to run most (if not all?) current major >> Operating Systems unmodified, it fails to install them (at least >> on the Linux side). > > I'd like to keep ugliness out of the kernel side. > > I don't think there's much work to get protected mode emulation > working. There aren't that many instructions before we get to a vt- > friendly state (a couple dozen?) and some of them are already > implemented. The hardest one being ljmp. You need to do the whole pm transition in the emulator then. I believe there is a reason this hasn't been done yet? > An alternative is to work around it in userspace. If we recognise > the exit reason, we can read the instructions around rip and attempt > to fix things up. So just get the CR0 write and UD exception as event to the userspace? I'd really love that approach. The "invalid opcode" hack, as I implemented it, is actually quite extensible. You could simply put the rip and an operation that is supposed to occur in a list and emulate whatever comes when the UD occurs. This might be the easiest way to fix things. We could also have something more extensible, say a "generic binary patching" framework, so we know that if memory page 0x1234000 contains specific content, just patch it and apply a "what happens in case of invalid opcodes" script. This could all be in userspace and should enable us to circumvent most problems in a generic way. >> Even though I would greatly appreciate any effort made to get >> things cleaned up, the gfxboot issue has been standing for months >> now without even a hacky workaround (except for disabling gfxboot >> in all) or any visible progress, so I believe a hack like this is >> at least worth something to distributions that want to enable >> their users to work with KVM. > > On the other hand, merging the hacks discourages the right fix from > being developed. I do agree that the current situation is > disgraceful. Don't get me wrong on this - I really want to see something "right". I just don't see anyone working on it, as there are a lot of places KVM improves right now, which are a lot more important than real mode fixes. Usually real mode is completely unused as soon as you're done with bootstrapping, so why care about it that much? I'm also perfectly fine with this not being merged. I built this hack for me, because I was rather unhappy with the situation as is and wanted to see gfxboot working, as I couldn't just "plug in" a current iso and install from that. If anyone benefits from it, I'm fine with it. If not, that's ok with me too. I just couldn't stand the situation that no fix was avail
Re: [kvm-devel] [PATCH] enable gfxboot on VMX
Alexander Graf wrote: >> >> While enabling gfxboot over vmx is very desirable, I'd like to avoid >> guest-specific hacks. IMO the correct fix is to set a >> "non_vt_friendly" >> flag when switching from real mode to protected mode, then continue >> emulation, re-computing the flag after every instruction. After a few >> instruction, the condition disappears and we can enter guest mode >> again. >> > > So when would the flag disappear? Whenever the register state becomes consistent with VT again. vmx_set_segment() looks like the right point for turning it off. > Basically an SS read can happen any > time after the switch, even after 100 instructions if it has to. That's okay. Most (all) code will reload all segments shortly after the protected mode switch. I am concerned about switches to real-mode not reloading fs and gs, so we'd stay in vt-unfriendly state and emulate even though the real mode code doesn't use fs and gs at all. I believe we can live with it; newer Xen for example emulates 100% of real mode code. > While > this should fix more problems, the one thing I am concerned about is > that I have not encountered any other code that does have this problem. > > I think some Ubuntus use big real mode, which can use the same fix. >> The downside is that we have to implement more instructions in the >> emulator for this, but these instructions will be generally useful, >> not >> just for gfxboot. >> > > I am not trying to talk you into anything - I would very much prefer a > rather clean solution as well. Nevertheless I do not see full > protected mode emulation code coming in the very near future and on a > user perspective would prefer to have something that works, even if > it's ugly. > So while KVM is able to run most (if not all?) current major Operating > Systems unmodified, it fails to install them (at least on the Linux > side). > I'd like to keep ugliness out of the kernel side. I don't think there's much work to get protected mode emulation working. There aren't that many instructions before we get to a vt-friendly state (a couple dozen?) and some of them are already implemented. An alternative is to work around it in userspace. If we recognise the exit reason, we can read the instructions around rip and attempt to fix things up. > Even though I would greatly appreciate any effort made to get things > cleaned up, the gfxboot issue has been standing for months now without > even a hacky workaround (except for disabling gfxboot in all) or any > visible progress, so I believe a hack like this is at least worth > something to distributions that want to enable their users to work > with KVM. On the other hand, merging the hacks discourages the right fix from being developed. I do agree that the current situation is disgraceful. -- Any sufficiently difficult bug is indistinguishable from a feature. - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] [PATCH] enable gfxboot on VMX
On Fri, 15 Feb 2008, Alexander Graf wrote: > On Feb 15, 2008, at 3:56 PM, Anthony Liguori wrote: > > >Have you tried SLES-9 or openSUSE variants of the same age? The ss issue in > >gfxboot is only something recently introduced. Prior to that, gfxboot used > >big real mode so your patch wouldn't be sufficient for those versions of > >gfxboot. > > SLES7 - SLES-9 and SUSE 9.1 through to openSUSE 10.1 do not need the patch. > They work 'as is'. SLES10 starts in text mode. > Starting with 10.2 the mov ss issue came along, but maybe Steffen can tell us > more about the history of this issue. The use of memory > 1MB was optional in older versions, so they might work even if the pm switch doesn't work. sles10 has a special check so it doesn't run in xen; maybe that gets in the way here, too. After sles10 big segments in real mode are no longer used. Steffen - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] [PATCH] enable gfxboot on VMX
On Feb 15, 2008, at 3:56 PM, Anthony Liguori wrote: > Alexander Graf wrote: >> Hi, >> >> this issue has already been talked about previously. Gfxboot on VMX >> is >> broken, because it reads SS after switching from real to protected >> mode, >> where SS contains an invalid value, which VMX does not allow. >> As far as I know, gfxboot is the only application that suffers from >> this >> issue. >> The current "fix" is to make gfxboot use a previously stored SS >> value, >> which works fine for new releases. Already shipped versions of the >> software can not be changed though, so there needs to be another >> way to >> make kvm work with older versions of gfxboot. >> >> As everything except gfxboot works, we can simply change gfxboot in >> runtime to use a different value. Unfortunately the mov instruction, >> used to read the SS register is only 2 bytes long, so there is no >> way to >> binary patch the mov to something that would contain an address. So >> the >> only way I could think of was an invalid instruction. The UD >> exception >> is intercepted in KVM and is already emulated for VMCALLs. This can >> be >> extended to an opcode, that is officially unused (0f 0c) and have the >> emulator do a mov realmode_ss, %eax. >> >> This patch implements exactly this idea and fixes openSUSE < 11.0 and >> Ubuntu CD booting on VMX for me. Comments are, as always, welcome. >> > > Have you tried SLES-9 or openSUSE variants of the same age? The ss > issue in gfxboot is only something recently introduced. Prior to > that, gfxboot used big real mode so your patch wouldn't be > sufficient for those versions of gfxboot. SLES7 - SLES-9 and SUSE 9.1 through to openSUSE 10.1 do not need the patch. They work 'as is'. SLES10 starts in text mode. Starting with 10.2 the mov ss issue came along, but maybe Steffen can tell us more about the history of this issue. > One thing I've thought about is converting gfxboot-disable[1] to > generate a qcow2 that backs to the actual CDROM ISO. Then in QEMU > we could take an MD5 of an ISO if trying to boot from it, compare it > to a white list of known bad CDs, and then generate a qcow2 > automatically with gfxboot disabled. When we eventually support big > real mode in the kernel, we can disable this. This sounds rather hacky too and does not enable the use of gfxboot. Regards, Alex > > > [1] http://hg.codemonkey.ws/gfxboot-disable > > Regards, > > Anthony Liguori > > >> Signed-off-by: Alexander Graf <[EMAIL PROTECTED]> >> >> >> >> >> >> - >> This SF.net email is sponsored by: Microsoft >> Defy all challenges. Microsoft(R) Visual Studio 2008. >> http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ >> >> >> ___ >> kvm-devel mailing list >> kvm-devel@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/kvm-devel > - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] [PATCH] enable gfxboot on VMX
Alexander Graf wrote: > Hi, > > this issue has already been talked about previously. Gfxboot on VMX is > broken, because it reads SS after switching from real to protected mode, > where SS contains an invalid value, which VMX does not allow. > As far as I know, gfxboot is the only application that suffers from this > issue. > The current "fix" is to make gfxboot use a previously stored SS value, > which works fine for new releases. Already shipped versions of the > software can not be changed though, so there needs to be another way to > make kvm work with older versions of gfxboot. > > As everything except gfxboot works, we can simply change gfxboot in > runtime to use a different value. Unfortunately the mov instruction, > used to read the SS register is only 2 bytes long, so there is no way to > binary patch the mov to something that would contain an address. So the > only way I could think of was an invalid instruction. The UD exception > is intercepted in KVM and is already emulated for VMCALLs. This can be > extended to an opcode, that is officially unused (0f 0c) and have the > emulator do a mov realmode_ss, %eax. > > This patch implements exactly this idea and fixes openSUSE < 11.0 and > Ubuntu CD booting on VMX for me. Comments are, as always, welcome. > Have you tried SLES-9 or openSUSE variants of the same age? The ss issue in gfxboot is only something recently introduced. Prior to that, gfxboot used big real mode so your patch wouldn't be sufficient for those versions of gfxboot. One thing I've thought about is converting gfxboot-disable[1] to generate a qcow2 that backs to the actual CDROM ISO. Then in QEMU we could take an MD5 of an ISO if trying to boot from it, compare it to a white list of known bad CDs, and then generate a qcow2 automatically with gfxboot disabled. When we eventually support big real mode in the kernel, we can disable this. [1] http://hg.codemonkey.ws/gfxboot-disable Regards, Anthony Liguori > Signed-off-by: Alexander Graf <[EMAIL PROTECTED]> > > > > > > - > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ > > > ___ > kvm-devel mailing list > kvm-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/kvm-devel - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] [PATCH] enable gfxboot on VMX
On Feb 15, 2008, at 2:47 PM, Avi Kivity wrote: > Alexander Graf wrote: >> Hi, >> >> this issue has already been talked about previously. Gfxboot on VMX >> is >> broken, because it reads SS after switching from real to protected >> mode, >> where SS contains an invalid value, which VMX does not allow. >> As far as I know, gfxboot is the only application that suffers from >> this >> issue. >> The current "fix" is to make gfxboot use a previously stored SS >> value, >> which works fine for new releases. Already shipped versions of the >> software can not be changed though, so there needs to be another >> way to >> make kvm work with older versions of gfxboot. >> >> As everything except gfxboot works, we can simply change gfxboot in >> runtime to use a different value. Unfortunately the mov instruction, >> used to read the SS register is only 2 bytes long, so there is no >> way to >> binary patch the mov to something that would contain an address. So >> the >> only way I could think of was an invalid instruction. The UD >> exception >> is intercepted in KVM and is already emulated for VMCALLs. This can >> be >> extended to an opcode, that is officially unused (0f 0c) and have the >> emulator do a mov realmode_ss, %eax. >> >> This patch implements exactly this idea and fixes openSUSE < 11.0 and >> Ubuntu CD booting on VMX for me. Comments are, as always, welcome. >> > > While enabling gfxboot over vmx is very desirable, I'd like to avoid > guest-specific hacks. IMO the correct fix is to set a > "non_vt_friendly" > flag when switching from real mode to protected mode, then continue > emulation, re-computing the flag after every instruction. After a few > instruction, the condition disappears and we can enter guest mode > again. So when would the flag disappear? Basically an SS read can happen any time after the switch, even after 100 instructions if it has to. While this should fix more problems, the one thing I am concerned about is that I have not encountered any other code that does have this problem. > The same approach works for big real mode. I agree on that. > The downside is that we have to implement more instructions in the > emulator for this, but these instructions will be generally useful, > not > just for gfxboot. I am not trying to talk you into anything - I would very much prefer a rather clean solution as well. Nevertheless I do not see full protected mode emulation code coming in the very near future and on a user perspective would prefer to have something that works, even if it's ugly. So while KVM is able to run most (if not all?) current major Operating Systems unmodified, it fails to install them (at least on the Linux side). Even though I would greatly appreciate any effort made to get things cleaned up, the gfxboot issue has been standing for months now without even a hacky workaround (except for disabling gfxboot in all) or any visible progress, so I believe a hack like this is at least worth something to distributions that want to enable their users to work with KVM. Alex - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] [PATCH] enable gfxboot on VMX
Alexander Graf wrote: > Hi, > > this issue has already been talked about previously. Gfxboot on VMX is > broken, because it reads SS after switching from real to protected mode, > where SS contains an invalid value, which VMX does not allow. > As far as I know, gfxboot is the only application that suffers from this > issue. > The current "fix" is to make gfxboot use a previously stored SS value, > which works fine for new releases. Already shipped versions of the > software can not be changed though, so there needs to be another way to > make kvm work with older versions of gfxboot. > > As everything except gfxboot works, we can simply change gfxboot in > runtime to use a different value. Unfortunately the mov instruction, > used to read the SS register is only 2 bytes long, so there is no way to > binary patch the mov to something that would contain an address. So the > only way I could think of was an invalid instruction. The UD exception > is intercepted in KVM and is already emulated for VMCALLs. This can be > extended to an opcode, that is officially unused (0f 0c) and have the > emulator do a mov realmode_ss, %eax. > > This patch implements exactly this idea and fixes openSUSE < 11.0 and > Ubuntu CD booting on VMX for me. Comments are, as always, welcome. > While enabling gfxboot over vmx is very desirable, I'd like to avoid guest-specific hacks. IMO the correct fix is to set a "non_vt_friendly" flag when switching from real mode to protected mode, then continue emulation, re-computing the flag after every instruction. After a few instruction, the condition disappears and we can enter guest mode again. The same approach works for big real mode. The downside is that we have to implement more instructions in the emulator for this, but these instructions will be generally useful, not just for gfxboot. -- Do not meddle in the internals of kernels, for they are subtle and quick to panic. - This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse012070mrt/direct/01/ ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
[kvm-devel] [PATCH] enable gfxboot on VMX
Hi,
this issue has already been talked about previously. Gfxboot on VMX is
broken, because it reads SS after switching from real to protected mode,
where SS contains an invalid value, which VMX does not allow.
As far as I know, gfxboot is the only application that suffers from this
issue.
The current "fix" is to make gfxboot use a previously stored SS value,
which works fine for new releases. Already shipped versions of the
software can not be changed though, so there needs to be another way to
make kvm work with older versions of gfxboot.
As everything except gfxboot works, we can simply change gfxboot in
runtime to use a different value. Unfortunately the mov instruction,
used to read the SS register is only 2 bytes long, so there is no way to
binary patch the mov to something that would contain an address. So the
only way I could think of was an invalid instruction. The UD exception
is intercepted in KVM and is already emulated for VMCALLs. This can be
extended to an opcode, that is officially unused (0f 0c) and have the
emulator do a mov realmode_ss, %eax.
This patch implements exactly this idea and fixes openSUSE < 11.0 and
Ubuntu CD booting on VMX for me. Comments are, as always, welcome.
Signed-off-by: Alexander Graf <[EMAIL PROTECTED]>
diff -ur kernel/include/asm-x86/kvm_host.h kernel.patched/include/asm-x86/kvm_host.h
--- kernel/include/asm-x86/kvm_host.h 2008-02-15 07:59:28.0 +0100
+++ kernel.patched/include/asm-x86/kvm_host.h 2008-02-15 07:42:41.0 +0100
@@ -195,6 +195,7 @@
unsigned long regs[NR_VCPU_REGS]; /* for rsp: vcpu_load_rsp_rip() */
unsigned long rip; /* needs vcpu_load_rsp_rip() */
+ u16 backup_ss;
unsigned long cr0;
unsigned long cr2;
unsigned long cr3;
diff -ur kernel/vmx.c kernel.patched/vmx.c
--- kernel/vmx.c 2008-02-15 07:59:28.0 +0100
+++ kernel.patched/vmx.c 2008-02-15 07:51:27.0 +0100
@@ -1112,6 +1112,8 @@
static void enter_pmode(struct kvm_vcpu *vcpu)
{
unsigned long flags;
+ unsigned long rip;
+ u8 opcodes[2];
vcpu->arch.rmode.active = 0;
@@ -1134,12 +1136,39 @@
fix_pmode_dataseg(VCPU_SREG_GS, &vcpu->arch.rmode.gs);
fix_pmode_dataseg(VCPU_SREG_FS, &vcpu->arch.rmode.fs);
+ /* Save real mode SS */
+ vcpu->arch.backup_ss = vmcs_read16(GUEST_SS_SELECTOR);
+
vmcs_write16(GUEST_SS_SELECTOR, 0);
vmcs_write32(GUEST_SS_AR_BYTES, 0x93);
vmcs_write16(GUEST_CS_SELECTOR,
vmcs_read16(GUEST_CS_SELECTOR) & ~SELECTOR_RPL_MASK);
vmcs_write32(GUEST_CS_AR_BYTES, 0x9b);
+
+ /* VMX checks for SS.CPL = CS.CPL on VM entry, if we are in
+ * protected mode. This fails on the transistion from real mode
+ * to protected mode, as just after that, SS still contains the
+ * real mode segment, which does not know anything about CPLs.
+ *
+ * As far as I know only gfxboot exploits this feature, by using
+ * the old real mode SS value to find a new SS selector in protected
+ * mode. This happens using a mov %ss, %eax instruction, which we
+ * can patch to an invalid opcode and emulate later on, giving eax
+ * the real SS value, that existed before the protected mode
+ * switch. */
+ rip = vcpu->arch.rip + vmcs_readl(GUEST_CS_BASE) + 14;
+ emulator_read_std(rip, (void *)opcodes, 2, vcpu);
+
+ if ( opcodes[0] == 0x8c && opcodes[1] == 0xd0 ) {
+ vcpu_printf(vcpu, "%s: patching mov SS\n", __FUNCTION__);
+ opcodes[0] = 0x0f;
+ opcodes[1] = 0x0c;
+ if (emulator_write_emulated(rip, opcodes,
+ 2, vcpu) != X86EMUL_CONTINUE)
+ vcpu_printf(vcpu, "%s: unable to patch mov SS\n",
+__FUNCTION__);
+ }
}
static gva_t rmode_tss_base(struct kvm *kvm)
diff -ur kernel/x86.c kernel.patched/x86.c
--- kernel/x86.c 2008-02-15 07:59:28.0 +0100
+++ kernel.patched/x86.c 2008-02-14 16:47:27.0 +0100
@@ -1886,13 +1886,14 @@
r = x86_decode_insn(&vcpu->arch.emulate_ctxt, &emulate_ops);
- /* Reject the instructions other than VMCALL/VMMCALL when
+ /* Reject the instructions other than VMCALL/VMMCALL/HACKS when
* try to emulate invalid opcode */
c = &vcpu->arch.emulate_ctxt.decode;
if ((emulation_type & EMULTYPE_TRAP_UD) &&
- (!(c->twobyte && c->b == 0x01 &&
+ ((!(c->twobyte && c->b == 0x01 &&
(c->modrm_reg == 0 || c->modrm_reg == 3) &&
- c->modrm_mod == 3 && c->modrm_rm == 1)))
+ c->modrm_mod == 3 && c->modrm_rm == 1)) &&
+ c->b != 0x0c))
return EMULATE_FAIL;
++vcpu->stat.insn_emulation;
diff -ur kernel/x86_emulate.c kernel.patched/x86_emulate.c
--- kernel/x86_emulate.c 2008-02-15 07:59:28.0 +0100
+++ kernel.patched/x86_emulate.c 2008-02-15 07:53:39.0 +0100
@@ -174,7 +175,7 @@
static u16 twobyte_table[256] = {
/* 0x00 - 0x0F */
0, SrcMem | ModRM | DstReg, 0, 0, 0, 0, ImplicitOps, 0,
- ImplicitOps, ImplicitOps, 0, 0, 0, ImplicitOps | ModRM, 0, 0,
+ ImplicitOps, ImplicitOps, 0, 0, ImplicitOps, ImplicitOps | ModRM, 0, 0,
/* 0x10 - 0x1F */
0, 0, 0, 0, 0, 0, 0, 0, ImplicitOps | ModRM, 0, 0, 0, 0, 0, 0, 0,
/* 0x20 - 0x2
