RE: [PATCH] kvm/fpu: Enable fully eager restore kvm FPU
-Original Message- From: Avi Kivity [mailto:a...@redhat.com] Sent: Monday, September 03, 2012 5:23 PM To: Hao, Xudong Cc: Roedel, Joerg; kvm@vger.kernel.org; Zhang, Xiantao Subject: Re: [PATCH] kvm/fpu: Enable fully eager restore kvm FPU On 08/23/2012 11:51 AM, Hao, Xudong wrote: -Original Message- From: kvm-ow...@vger.kernel.org [mailto:kvm-ow...@vger.kernel.org] On Behalf Of Avi Kivity Sent: Monday, August 20, 2012 6:43 PM To: Roedel, Joerg Cc: Hao, Xudong; kvm@vger.kernel.org; Zhang, Xiantao Subject: Re: [PATCH] kvm/fpu: Enable fully eager restore kvm FPU On 08/20/2012 01:14 PM, Roedel, Joerg wrote: On Mon, Aug 20, 2012 at 01:08:14PM +0300, Avi Kivity wrote: On 08/20/2012 12:24 PM, Roedel, Joerg wrote: So it was broken all along? Yikes. There is no LWP support in the kernel and thus KVM can't expose it to guests. So for now nothing should be broken, no? Oh, we mask out xcr0 bits not supported by the host. So it's broken in another way: it isn't exposed. Pity, it's such a nice feature. Hi, Avi/Joerg What's the decision for it? I don't understand LWP, so how about this patch? It's fine (Joerg can send the LWP change), but there was a truncation issue that needs fixing, no? Yes, I think you means to expand KVM_XSTATE_LAZY to 64-bits, I'll send another version patch. Thanks, -Xudong -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] kvm/fpu: Enable fully eager restore kvm FPU
On 08/23/2012 11:51 AM, Hao, Xudong wrote: -Original Message- From: kvm-ow...@vger.kernel.org [mailto:kvm-ow...@vger.kernel.org] On Behalf Of Avi Kivity Sent: Monday, August 20, 2012 6:43 PM To: Roedel, Joerg Cc: Hao, Xudong; kvm@vger.kernel.org; Zhang, Xiantao Subject: Re: [PATCH] kvm/fpu: Enable fully eager restore kvm FPU On 08/20/2012 01:14 PM, Roedel, Joerg wrote: On Mon, Aug 20, 2012 at 01:08:14PM +0300, Avi Kivity wrote: On 08/20/2012 12:24 PM, Roedel, Joerg wrote: So it was broken all along? Yikes. There is no LWP support in the kernel and thus KVM can't expose it to guests. So for now nothing should be broken, no? Oh, we mask out xcr0 bits not supported by the host. So it's broken in another way: it isn't exposed. Pity, it's such a nice feature. Hi, Avi/Joerg What's the decision for it? I don't understand LWP, so how about this patch? It's fine (Joerg can send the LWP change), but there was a truncation issue that needs fixing, no? -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH] kvm/fpu: Enable fully eager restore kvm FPU
-Original Message- From: kvm-ow...@vger.kernel.org [mailto:kvm-ow...@vger.kernel.org] On Behalf Of Avi Kivity Sent: Monday, August 20, 2012 6:43 PM To: Roedel, Joerg Cc: Hao, Xudong; kvm@vger.kernel.org; Zhang, Xiantao Subject: Re: [PATCH] kvm/fpu: Enable fully eager restore kvm FPU On 08/20/2012 01:14 PM, Roedel, Joerg wrote: On Mon, Aug 20, 2012 at 01:08:14PM +0300, Avi Kivity wrote: On 08/20/2012 12:24 PM, Roedel, Joerg wrote: So it was broken all along? Yikes. There is no LWP support in the kernel and thus KVM can't expose it to guests. So for now nothing should be broken, no? Oh, we mask out xcr0 bits not supported by the host. So it's broken in another way: it isn't exposed. Pity, it's such a nice feature. Hi, Avi/Joerg What's the decision for it? I don't understand LWP, so how about this patch? -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] kvm/fpu: Enable fully eager restore kvm FPU
(Back from vacation) On Thu, Aug 16, 2012 at 01:59:02PM +0300, Avi Kivity wrote: Ok. Please check that ~KVM_XSTATE_LAZY expands to 64-bits correctly, maybe we need to cast it to u64 before negating it. Note that we limit xcr0 to the bits allowed by the host, so the currect kernel is safe even on hardware with state that isn't tracked by cr0.ts. But it's better to be safe here. Joerg, IIRC LWP uses one of these bits? Should it be added to the mask? LWP uses bit 62 in xcr0 and is not tracked by cr0.ts either. So this bit should be used to the mask too (in other words LWP is a non-lazy state). Regards, Joerg -- AMD Operating System Research Center Advanced Micro Devices GmbH Einsteinring 24 85609 Dornach General Managers: Alberto Bozzo Registration: Dornach, Landkr. Muenchen; Registerger. Muenchen, HRB Nr. 43632 -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] kvm/fpu: Enable fully eager restore kvm FPU
On 08/20/2012 12:24 PM, Roedel, Joerg wrote: (Back from vacation) On Thu, Aug 16, 2012 at 01:59:02PM +0300, Avi Kivity wrote: Ok. Please check that ~KVM_XSTATE_LAZY expands to 64-bits correctly, maybe we need to cast it to u64 before negating it. Note that we limit xcr0 to the bits allowed by the host, so the currect kernel is safe even on hardware with state that isn't tracked by cr0.ts. But it's better to be safe here. Joerg, IIRC LWP uses one of these bits? Should it be added to the mask? LWP uses bit 62 in xcr0 and is not tracked by cr0.ts either. So this bit should be used to the mask too (in other words LWP is a non-lazy state). So it was broken all along? Yikes. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] kvm/fpu: Enable fully eager restore kvm FPU
On Mon, Aug 20, 2012 at 01:08:14PM +0300, Avi Kivity wrote: On 08/20/2012 12:24 PM, Roedel, Joerg wrote: So it was broken all along? Yikes. There is no LWP support in the kernel and thus KVM can't expose it to guests. So for now nothing should be broken, no? Joerg -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] kvm/fpu: Enable fully eager restore kvm FPU
On 08/20/2012 01:14 PM, Roedel, Joerg wrote: On Mon, Aug 20, 2012 at 01:08:14PM +0300, Avi Kivity wrote: On 08/20/2012 12:24 PM, Roedel, Joerg wrote: So it was broken all along? Yikes. There is no LWP support in the kernel and thus KVM can't expose it to guests. So for now nothing should be broken, no? Oh, we mask out xcr0 bits not supported by the host. So it's broken in another way: it isn't exposed. Pity, it's such a nice feature. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] kvm/fpu: Enable fully eager restore kvm FPU
On Mon, Aug 20, 2012 at 01:43:10PM +0300, Avi Kivity wrote: On 08/20/2012 01:14 PM, Roedel, Joerg wrote: On Mon, Aug 20, 2012 at 01:08:14PM +0300, Avi Kivity wrote: On 08/20/2012 12:24 PM, Roedel, Joerg wrote: So it was broken all along? Yikes. There is no LWP support in the kernel and thus KVM can't expose it to guests. So for now nothing should be broken, no? Oh, we mask out xcr0 bits not supported by the host. So it's broken in another way: it isn't exposed. Pity, it's such a nice feature. Right, that is very sad. We can support it in KVM even without Linux host support, but there is no OS which enables this feature yet, so it is rather useless to do it in KVM only. Joerg -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] kvm/fpu: Enable fully eager restore kvm FPU
On 08/16/2012 08:14 AM, Xudong Hao wrote:
Enable KVM FPU fully eager restore, if there is other FPU state which isn't
tracked by
CR0.TS bit.
Tested with these cases:
1) SpecCPU2000 workload( 1 VM, 2 VMs)
2) Program for floating point caculate
Is the motivation performance or correctness?
+
struct kvm_memory_alias {
__u32 slot; /* this has a different namespace than memory slots */
__u32 flags;
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index b6379e5..2e628e5 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -5966,7 +5966,18 @@ void kvm_put_guest_fpu(struct kvm_vcpu *vcpu)
vcpu-guest_fpu_loaded = 0;
fpu_save_init(vcpu-arch.guest_fpu);
++vcpu-stat.fpu_reload;
- kvm_make_request(KVM_REQ_DEACTIVATE_FPU, vcpu);
+ /*
+ * Currently KVM trigger FPU restore by #NM (via CR0.TS),
+ * till now only XCR0.bit0, XCR0.bit1, XCR0.bit2 is tracked
+ * by TS bit, there might be other FPU state is not tracked
+ * by TS bit.
Which state is that?
Here it only make FPU deactivate request and do
+ * FPU lazy restore for these cases: 1)xsave isn't enabled
+ * in guest, 2)all guest FPU states can be tracked by TS bit.
+ * For others, doing fully FPU eager restore.
+ */
+ if (!kvm_read_cr4_bits(vcpu, X86_CR4_OSXSAVE) ||
+ !(vcpu-arch.xcr0 ~KVM_XSTATE_LAZY))
+ kvm_make_request(KVM_REQ_DEACTIVATE_FPU, vcpu);
trace_kvm_fpu(0);
}
Is there no way to track accesses to this extended state?
Although I expect that on modern hardware which exits rarely, eager fpu
reload might be more performant.
--
error compiling committee.c: too many arguments to function
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH] kvm/fpu: Enable fully eager restore kvm FPU
-Original Message-
From: kvm-ow...@vger.kernel.org [mailto:kvm-ow...@vger.kernel.org] On
Behalf Of Avi Kivity
Sent: Thursday, August 16, 2012 5:08 PM
To: Hao, Xudong
Cc: kvm@vger.kernel.org; Zhang, Xiantao
Subject: Re: [PATCH] kvm/fpu: Enable fully eager restore kvm FPU
On 08/16/2012 08:14 AM, Xudong Hao wrote:
Enable KVM FPU fully eager restore, if there is other FPU state which isn't
tracked by
CR0.TS bit.
Tested with these cases:
1) SpecCPU2000 workload( 1 VM, 2 VMs)
2) Program for floating point caculate
Is the motivation performance or correctness?
It's not performance improvement, it could be treated as a correctness. I do
not say current code has issue, but just as code comment, it's for the other
FPU state.
+
struct kvm_memory_alias {
__u32 slot; /* this has a different namespace than memory slots */
__u32 flags;
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index b6379e5..2e628e5 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -5966,7 +5966,18 @@ void kvm_put_guest_fpu(struct kvm_vcpu *vcpu)
vcpu-guest_fpu_loaded = 0;
fpu_save_init(vcpu-arch.guest_fpu);
++vcpu-stat.fpu_reload;
- kvm_make_request(KVM_REQ_DEACTIVATE_FPU, vcpu);
+ /*
+* Currently KVM trigger FPU restore by #NM (via CR0.TS),
+* till now only XCR0.bit0, XCR0.bit1, XCR0.bit2 is tracked
+* by TS bit, there might be other FPU state is not tracked
+* by TS bit.
Which state is that?
Except the last 3 bits, other bit are these state.
Here it only make FPU deactivate request and do
+* FPU lazy restore for these cases: 1)xsave isn't enabled
+* in guest, 2)all guest FPU states can be tracked by TS bit.
+* For others, doing fully FPU eager restore.
+*/
+ if (!kvm_read_cr4_bits(vcpu, X86_CR4_OSXSAVE) ||
+ !(vcpu-arch.xcr0 ~KVM_XSTATE_LAZY))
+ kvm_make_request(KVM_REQ_DEACTIVATE_FPU, vcpu);
trace_kvm_fpu(0);
}
Is there no way to track accesses to this extended state?
Because I can't define the extended state now, so using this method. But just
as I say, the extended state are NO-LAZY except the last 3 bit.
Although I expect that on modern hardware which exits rarely, eager fpu
reload might be more performant.
--
error compiling committee.c: too many arguments to function
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] kvm/fpu: Enable fully eager restore kvm FPU
On 08/16/2012 12:48 PM, Hao, Xudong wrote:
-Original Message-
From: kvm-ow...@vger.kernel.org [mailto:kvm-ow...@vger.kernel.org] On
Behalf Of Avi Kivity
Sent: Thursday, August 16, 2012 5:08 PM
To: Hao, Xudong
Cc: kvm@vger.kernel.org; Zhang, Xiantao
Subject: Re: [PATCH] kvm/fpu: Enable fully eager restore kvm FPU
On 08/16/2012 08:14 AM, Xudong Hao wrote:
Enable KVM FPU fully eager restore, if there is other FPU state which isn't
tracked by
CR0.TS bit.
Tested with these cases:
1) SpecCPU2000 workload( 1 VM, 2 VMs)
2) Program for floating point caculate
Is the motivation performance or correctness?
It's not performance improvement, it could be treated as a correctness. I do
not say current code has issue, but just as code comment, it's for the other
FPU state.
+
struct kvm_memory_alias {
__u32 slot; /* this has a different namespace than memory slots */
__u32 flags;
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index b6379e5..2e628e5 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -5966,7 +5966,18 @@ void kvm_put_guest_fpu(struct kvm_vcpu *vcpu)
vcpu-guest_fpu_loaded = 0;
fpu_save_init(vcpu-arch.guest_fpu);
++vcpu-stat.fpu_reload;
- kvm_make_request(KVM_REQ_DEACTIVATE_FPU, vcpu);
+ /*
+ * Currently KVM trigger FPU restore by #NM (via CR0.TS),
+ * till now only XCR0.bit0, XCR0.bit1, XCR0.bit2 is tracked
+ * by TS bit, there might be other FPU state is not tracked
+ * by TS bit.
Which state is that?
Except the last 3 bits, other bit are these state.
Here it only make FPU deactivate request and do
+ * FPU lazy restore for these cases: 1)xsave isn't enabled
+ * in guest, 2)all guest FPU states can be tracked by TS bit.
+ * For others, doing fully FPU eager restore.
+ */
+ if (!kvm_read_cr4_bits(vcpu, X86_CR4_OSXSAVE) ||
+ !(vcpu-arch.xcr0 ~KVM_XSTATE_LAZY))
+ kvm_make_request(KVM_REQ_DEACTIVATE_FPU, vcpu);
trace_kvm_fpu(0);
}
Is there no way to track accesses to this extended state?
Because I can't define the extended state now, so using this method. But just
as I say, the extended state are NO-LAZY except the last 3 bit.
Ok. Please check that ~KVM_XSTATE_LAZY expands to 64-bits correctly,
maybe we need to cast it to u64 before negating it.
Note that we limit xcr0 to the bits allowed by the host, so the currect
kernel is safe even on hardware with state that isn't tracked by cr0.ts.
But it's better to be safe here.
Joerg, IIRC LWP uses one of these bits? Should it be added to the mask?
--
error compiling committee.c: too many arguments to function
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH] kvm/fpu: Enable fully eager restore kvm FPU
-Original Message-
From: Avi Kivity [mailto:a...@redhat.com]
Sent: Thursday, August 16, 2012 6:59 PM
To: Hao, Xudong
Cc: kvm@vger.kernel.org; Zhang, Xiantao; Roedel, Joerg
Subject: Re: [PATCH] kvm/fpu: Enable fully eager restore kvm FPU
On 08/16/2012 12:48 PM, Hao, Xudong wrote:
-Original Message-
From: kvm-ow...@vger.kernel.org [mailto:kvm-ow...@vger.kernel.org] On
Behalf Of Avi Kivity
Sent: Thursday, August 16, 2012 5:08 PM
To: Hao, Xudong
Cc: kvm@vger.kernel.org; Zhang, Xiantao
Subject: Re: [PATCH] kvm/fpu: Enable fully eager restore kvm FPU
On 08/16/2012 08:14 AM, Xudong Hao wrote:
Enable KVM FPU fully eager restore, if there is other FPU state which
isn't
tracked by
CR0.TS bit.
Tested with these cases:
1) SpecCPU2000 workload( 1 VM, 2 VMs)
2) Program for floating point caculate
Is the motivation performance or correctness?
It's not performance improvement, it could be treated as a correctness. I do
not say current code has issue, but just as code comment, it's for the other
FPU
state.
+
struct kvm_memory_alias {
__u32 slot; /* this has a different namespace than memory
slots */
__u32 flags;
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index b6379e5..2e628e5 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -5966,7 +5966,18 @@ void kvm_put_guest_fpu(struct kvm_vcpu
*vcpu)
vcpu-guest_fpu_loaded = 0;
fpu_save_init(vcpu-arch.guest_fpu);
++vcpu-stat.fpu_reload;
-kvm_make_request(KVM_REQ_DEACTIVATE_FPU, vcpu);
+/*
+ * Currently KVM trigger FPU restore by #NM (via CR0.TS),
+ * till now only XCR0.bit0, XCR0.bit1, XCR0.bit2 is tracked
+ * by TS bit, there might be other FPU state is not tracked
+ * by TS bit.
Which state is that?
Except the last 3 bits, other bit are these state.
Here it only make FPU deactivate request and do
+ * FPU lazy restore for these cases: 1)xsave isn't enabled
+ * in guest, 2)all guest FPU states can be tracked by TS bit.
+ * For others, doing fully FPU eager restore.
+ */
+if (!kvm_read_cr4_bits(vcpu, X86_CR4_OSXSAVE) ||
+!(vcpu-arch.xcr0 ~KVM_XSTATE_LAZY))
+kvm_make_request(KVM_REQ_DEACTIVATE_FPU, vcpu);
trace_kvm_fpu(0);
}
Is there no way to track accesses to this extended state?
Because I can't define the extended state now, so using this method. But
just
as I say, the extended state are NO-LAZY except the last 3 bit.
Ok. Please check that ~KVM_XSTATE_LAZY expands to 64-bits correctly,
maybe we need to cast it to u64 before negating it.
Thanks.
+ if (!kvm_read_cr4_bits(vcpu, X86_CR4_OSXSAVE) ||
+ !(vcpu-arch.xcr0 ~((u64)KVM_XSTATE_LAZY)))
+ kvm_make_request(KVM_REQ_DEACTIVATE_FPU, vcpu);
Note that we limit xcr0 to the bits allowed by the host, so the currect
kernel is safe even on hardware with state that isn't tracked by cr0.ts.
But it's better to be safe here.
Joerg, IIRC LWP uses one of these bits? Should it be added to the mask?
Bit 62? Maybe LWP should change to eager too, I'm not sure. Joerg?
--
error compiling committee.c: too many arguments to function
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
