Re: [kvm-devel] [PATCH] Refactor hypercall infrastructure (v2)
Amit Shah wrote:
* Anthony Liguori wrote:
Amit Shah wrote:
* Anthony Liguori wrote:
This patch refactors the current hypercall infrastructure to better
support live migration and SMP. It eliminates the hypercall page by
trapping the UD exception that would occur if you used the wrong
hypercall instruction for the underlying architecture and replacing it
with the right one lazily.
This doesn't work right for SVM. It keeps looping indefinitely; on a
kvm_stat run, I get about 230,000 light vm exits per second, with the
hypercall never returning to the guest.
...
What are you using to issue the hypercall?
+ r = kvm_hypercall1(KVM_PV_PCI_DEVICE, page_gfn);
Setup is done by:
+ if (!kvm_para_available()) {
+ printk(KERN_ERR KVM paravirt support not available\n);
+ r = -ENODEV;
+ goto out_dereg;
+ }
There was a bug where instructions with a modrm byte specifying a
register would try to access memory. In the memory was not mapped,
emulation would fail. vmcall is one such instruction. This was fixed by
commit f83562246921d6a8a7de8b76853a6835ace3699d
Author: Aurelien Jarno [EMAIL PROTECTED]
Date: Wed Oct 17 19:30:41 2007 +0200
KVM: x86 emulator: fix access registers for instructions with ModR/M
byte and Mod = 3
The patch belows changes the access type to register from memory for
instructions that are declared as SrcMem or DstMem, but have a
ModR/M byte with Mod = 3.
It fixes (at least) the lmsw and smsw instructions on an AMD64 CPU,
which are needed for FreeBSD.
Signed-off-by: Aurelien Jarno [EMAIL PROTECTED]
Signed-off-by: Avi Kivity [EMAIL PROTECTED]
diff --git a/drivers/kvm/x86_emulate.c b/drivers/kvm/x86_emulate.c
index 7c95ae5..8c50496 100644
--- a/drivers/kvm/x86_emulate.c
+++ b/drivers/kvm/x86_emulate.c
@@ -835,6 +835,14 @@ modrm_done:
if (c-twobyte c-b == 0x01
c-modrm_reg == 7)
break;
+ /*
+* For instructions with a ModR/M byte, switch to register
+* access if Mod = 3.
+*/
+ if ((c-d ModRM) c-modrm_mod == 3) {
+ c-src.type = OP_REG;
+ break;
+ }
srcmem_common:
c-src.type = OP_MEM;
break;
@@ -897,7 +905,14 @@ srcmem_common:
}
break;
case DstMem:
- c-dst.type = OP_MEM;
+ /*
+* For instructions with a ModR/M byte, switch to register
+* access if Mod = 3.
+*/
+ if ((c-d ModRM) c-modrm_mod == 3)
+ c-dst.type = OP_REG;
+ else
+ c-dst.type = OP_MEM;
break;
}
--
error compiling committee.c: too many arguments to function
-
SF.Net email is sponsored by: The Future of Linux Business White Paper
from Novell. From the desktop to the data center, Linux is going
mainstream. Let it simplify your IT future.
http://altfarm.mediaplex.com/ad/ck/8857-50307-18918-4
___
kvm-devel mailing list
kvm-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] [PATCH] Refactor hypercall infrastructure (v2)
* Anthony Liguori wrote:
Amit Shah wrote:
* Anthony Liguori wrote:
This patch refactors the current hypercall infrastructure to better
support live migration and SMP. It eliminates the hypercall page by
trapping the UD exception that would occur if you used the wrong
hypercall instruction for the underlying architecture and replacing it
with the right one lazily.
This doesn't work right for SVM. It keeps looping indefinitely; on a
kvm_stat run, I get about 230,000 light vm exits per second, with the
hypercall never returning to the guest.
...
What are you using to issue the hypercall?
+ r = kvm_hypercall1(KVM_PV_PCI_DEVICE, page_gfn);
Setup is done by:
+ if (!kvm_para_available()) {
+ printk(KERN_ERR KVM paravirt support not available\n);
+ r = -ENODEV;
+ goto out_dereg;
+ }
I also couldn't get the has_feature to work properly.
See:
http://lkml.org/lkml/2007/11/7/129
I had to change the hypercall address to 0f 01 d9 for it to get working.
Regards,
Anthony Liguori
-
SF.Net email is sponsored by: The Future of Linux Business White Paper
from Novell. From the desktop to the data center, Linux is going
mainstream. Let it simplify your IT future.
http://altfarm.mediaplex.com/ad/ck/8857-50307-18918-4
___
kvm-devel mailing list
kvm-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] [PATCH] Refactor hypercall infrastructure (v2)
* Avi Kivity wrote:
Amit Shah wrote:
* Anthony Liguori wrote:
Amit Shah wrote:
What are you using to issue the hypercall?
+ r = kvm_hypercall1(KVM_PV_PCI_DEVICE, page_gfn);
Setup is done by:
+ if (!kvm_para_available()) {
+ printk(KERN_ERR KVM paravirt support not available\n);
+ r = -ENODEV;
+ goto out_dereg;
+ }
There was a bug where instructions with a modrm byte specifying a
register would try to access memory. In the memory was not mapped,
emulation would fail. vmcall is one such instruction. This was fixed by
commit f83562246921d6a8a7de8b76853a6835ace3699d
Author: Aurelien Jarno [EMAIL PROTECTED]
Date: Wed Oct 17 19:30:41 2007 +0200
KVM: x86 emulator: fix access registers for instructions with ModR/M
byte and Mod = 3
Thanks, and thank you, Aurelien!
-
SF.Net email is sponsored by: The Future of Linux Business White Paper
from Novell. From the desktop to the data center, Linux is going
mainstream. Let it simplify your IT future.
http://altfarm.mediaplex.com/ad/ck/8857-50307-18918-4
___
kvm-devel mailing list
kvm-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] [PATCH] Refactor hypercall infrastructure (v2)
* Anthony Liguori wrote:
This patch refactors the current hypercall infrastructure to better support
live migration and SMP. It eliminates the hypercall page by trapping the
UD exception that would occur if you used the wrong hypercall instruction
for the underlying architecture and replacing it with the right one lazily.
This doesn't work right for SVM. It keeps looping indefinitely; on a kvm_stat
run, I get about 230,000 light vm exits per second, with the hypercall never
returning to the guest.
...
diff --git a/drivers/kvm/svm.c b/drivers/kvm/svm.c
index 729f1cd..d09a9f5 100644
--- a/drivers/kvm/svm.c
+++ b/drivers/kvm/svm.c
@@ -476,7 +476,8 @@ static void init_vmcb(struct vmcb *vmcb)
INTERCEPT_DR5_MASK |
INTERCEPT_DR7_MASK;
- control-intercept_exceptions = 1 PF_VECTOR;
+ control-intercept_exceptions = (1 PF_VECTOR) |
+ (1 UD_VECTOR);
control-intercept =(1ULL INTERCEPT_INTR) |
@@ -970,6 +971,17 @@ static int pf_interception(struct vcpu_svm *svm,
struct kvm_run *kvm_run) return 0;
}
+static int ud_interception(struct vcpu_svm *svm, struct kvm_run *kvm_run)
+{
+ int er;
+
+ er = emulate_instruction(svm-vcpu, kvm_run, 0, 0);
+ if (er != EMULATE_DONE)
+ inject_ud(svm-vcpu);
+
+ return 1;
+}
+
static int nm_interception(struct vcpu_svm *svm, struct kvm_run *kvm_run)
{
svm-vmcb-control.intercept_exceptions = ~(1 NM_VECTOR);
@@ -1036,7 +1048,8 @@ static int vmmcall_interception(struct vcpu_svm *svm,
struct kvm_run *kvm_run) {
svm-next_rip = svm-vmcb-save.rip + 3;
skip_emulated_instruction(svm-vcpu);
- return kvm_hypercall(svm-vcpu, kvm_run);
+ kvm_emulate_hypercall(svm-vcpu);
+ return 1;
}
static int invalid_op_interception(struct vcpu_svm *svm,
@@ -1232,6 +1245,7 @@ static int (*svm_exit_handlers[])(struct vcpu_svm
*svm, [SVM_EXIT_WRITE_DR3]= emulate_on_interception,
[SVM_EXIT_WRITE_DR5]= emulate_on_interception,
[SVM_EXIT_WRITE_DR7]= emulate_on_interception,
+ [SVM_EXIT_EXCP_BASE + UD_VECTOR]= ud_interception,
[SVM_EXIT_EXCP_BASE + PF_VECTOR]= pf_interception,
[SVM_EXIT_EXCP_BASE + NM_VECTOR]= nm_interception,
[SVM_EXIT_INTR] = nop_on_interception,
@@ -1664,7 +1678,6 @@ svm_patch_hypercall(struct kvm_vcpu *vcpu, unsigned
char *hypercall) hypercall[0] = 0x0f;
hypercall[1] = 0x01;
hypercall[2] = 0xd9;
- hypercall[3] = 0xc3;
}
...
+/* This instruction is vmcall. On non-VT architectures, it will generate
a + * trap that we will then rewrite to the appropriate instruction. */
-#define __NR_hypercalls 0
+#define KVM_HYPERCALL .byte 0x0f,0x01,0xc1
.. which never happens.
-
SF.Net email is sponsored by: The Future of Linux Business White Paper
from Novell. From the desktop to the data center, Linux is going
mainstream. Let it simplify your IT future.
http://altfarm.mediaplex.com/ad/ck/8857-50307-18918-4
___
kvm-devel mailing list
kvm-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] [PATCH] Refactor hypercall infrastructure (v2)
Amit Shah wrote: * Anthony Liguori wrote: This patch refactors the current hypercall infrastructure to better support live migration and SMP. It eliminates the hypercall page by trapping the UD exception that would occur if you used the wrong hypercall instruction for the underlying architecture and replacing it with the right one lazily. This doesn't work right for SVM. It keeps looping indefinitely; on a kvm_stat run, I get about 230,000 light vm exits per second, with the hypercall never returning to the guest. I just tested kvm.git with (the new) hypercall.flat testcase. Seems to work fine. -- error compiling committee.c: too many arguments to function - SF.Net email is sponsored by: The Future of Linux Business White Paper from Novell. From the desktop to the data center, Linux is going mainstream. Let it simplify your IT future. http://altfarm.mediaplex.com/ad/ck/8857-50307-18918-4 ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] [PATCH] Refactor hypercall infrastructure (v2)
Amit Shah wrote:
* Anthony Liguori wrote:
This patch refactors the current hypercall infrastructure to better support
live migration and SMP. It eliminates the hypercall page by trapping the
UD exception that would occur if you used the wrong hypercall instruction
for the underlying architecture and replacing it with the right one lazily.
This doesn't work right for SVM. It keeps looping indefinitely; on a kvm_stat
run, I get about 230,000 light vm exits per second, with the hypercall never
returning to the guest.
...
What are you using to issue the hypercall?
Regards,
Anthony Liguori
diff --git a/drivers/kvm/svm.c b/drivers/kvm/svm.c
index 729f1cd..d09a9f5 100644
--- a/drivers/kvm/svm.c
+++ b/drivers/kvm/svm.c
@@ -476,7 +476,8 @@ static void init_vmcb(struct vmcb *vmcb)
INTERCEPT_DR5_MASK |
INTERCEPT_DR7_MASK;
-control-intercept_exceptions = 1 PF_VECTOR;
+control-intercept_exceptions = (1 PF_VECTOR) |
+(1 UD_VECTOR);
control-intercept =(1ULL INTERCEPT_INTR) |
@@ -970,6 +971,17 @@ static int pf_interception(struct vcpu_svm *svm,
struct kvm_run *kvm_run) return 0;
}
+static int ud_interception(struct vcpu_svm *svm, struct kvm_run *kvm_run)
+{
+int er;
+
+er = emulate_instruction(svm-vcpu, kvm_run, 0, 0);
+if (er != EMULATE_DONE)
+inject_ud(svm-vcpu);
+
+return 1;
+}
+
static int nm_interception(struct vcpu_svm *svm, struct kvm_run *kvm_run)
{
svm-vmcb-control.intercept_exceptions = ~(1 NM_VECTOR);
@@ -1036,7 +1048,8 @@ static int vmmcall_interception(struct vcpu_svm *svm,
struct kvm_run *kvm_run) {
svm-next_rip = svm-vmcb-save.rip + 3;
skip_emulated_instruction(svm-vcpu);
-return kvm_hypercall(svm-vcpu, kvm_run);
+kvm_emulate_hypercall(svm-vcpu);
+return 1;
}
static int invalid_op_interception(struct vcpu_svm *svm,
@@ -1232,6 +1245,7 @@ static int (*svm_exit_handlers[])(struct vcpu_svm
*svm, [SVM_EXIT_WRITE_DR3] = emulate_on_interception,
[SVM_EXIT_WRITE_DR5]= emulate_on_interception,
[SVM_EXIT_WRITE_DR7]= emulate_on_interception,
+[SVM_EXIT_EXCP_BASE + UD_VECTOR]= ud_interception,
[SVM_EXIT_EXCP_BASE + PF_VECTOR]= pf_interception,
[SVM_EXIT_EXCP_BASE + NM_VECTOR]= nm_interception,
[SVM_EXIT_INTR] = nop_on_interception,
@@ -1664,7 +1678,6 @@ svm_patch_hypercall(struct kvm_vcpu *vcpu, unsigned
char *hypercall) hypercall[0] = 0x0f;
hypercall[1] = 0x01;
hypercall[2] = 0xd9;
-hypercall[3] = 0xc3;
}
...
+/* This instruction is vmcall. On non-VT architectures, it will generate
a + * trap that we will then rewrite to the appropriate instruction. */
-#define __NR_hypercalls 0
+#define KVM_HYPERCALL .byte 0x0f,0x01,0xc1
.. which never happens.
-
SF.Net email is sponsored by: The Future of Linux Business White Paper
from Novell. From the desktop to the data center, Linux is going
mainstream. Let it simplify your IT future.
http://altfarm.mediaplex.com/ad/ck/8857-50307-18918-4
___
kvm-devel mailing list
kvm-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/kvm-devel
[kvm-devel] [PATCH] Refactor hypercall infrastructure (v2)
This patch refactors the current hypercall infrastructure to better support live
migration and SMP. It eliminates the hypercall page by trapping the UD
exception that would occur if you used the wrong hypercall instruction for the
underlying architecture and replacing it with the right one lazily.
It also introduces the infrastructure to probe for hypercall available via
CPUID leaves 0x40001000. CPUID leaf 0x40001001 should be filled out by
userspace.
A fall-out of this patch is that the unhandled hypercalls no longer trap to
userspace. There is very little reason though to use a hypercall to communicate
with userspace as PIO or MMIO can be used. There is no code in tree that uses
userspace hypercalls.
Since the last patchset, I've changed the CPUID leaves to better avoid Xen's
CPUID range and fixed a bug spotted by Muli in masking off hypercall arguments.
Signed-off-by: Anthony Liguori [EMAIL PROTECTED]
diff --git a/drivers/kvm/kvm.h b/drivers/kvm/kvm.h
index ad08138..1cde572 100644
--- a/drivers/kvm/kvm.h
+++ b/drivers/kvm/kvm.h
@@ -46,6 +46,7 @@
#define KVM_MAX_CPUID_ENTRIES 40
#define DE_VECTOR 0
+#define UD_VECTOR 6
#define NM_VECTOR 7
#define DF_VECTOR 8
#define TS_VECTOR 10
@@ -317,9 +318,6 @@ struct kvm_vcpu {
unsigned long cr0;
unsigned long cr2;
unsigned long cr3;
- gpa_t para_state_gpa;
- struct page *para_state_page;
- gpa_t hypercall_gpa;
unsigned long cr4;
unsigned long cr8;
u64 pdptrs[4]; /* pae */
@@ -622,7 +620,9 @@ void __kvm_mmu_free_some_pages(struct kvm_vcpu *vcpu);
int kvm_mmu_load(struct kvm_vcpu *vcpu);
void kvm_mmu_unload(struct kvm_vcpu *vcpu);
-int kvm_hypercall(struct kvm_vcpu *vcpu, struct kvm_run *run);
+int kvm_emulate_hypercall(struct kvm_vcpu *vcpu);
+
+int kvm_fix_hypercall(struct kvm_vcpu *vcpu);
static inline int kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gva_t gva,
u32 error_code)
diff --git a/drivers/kvm/kvm_main.c b/drivers/kvm/kvm_main.c
index 99e4917..d720290 100644
--- a/drivers/kvm/kvm_main.c
+++ b/drivers/kvm/kvm_main.c
@@ -39,6 +39,7 @@
#include linux/smp.h
#include linux/anon_inodes.h
#include linux/profile.h
+#include linux/kvm_para.h
#include asm/processor.h
#include asm/msr.h
@@ -1383,51 +1384,61 @@ int kvm_emulate_halt(struct kvm_vcpu *vcpu)
}
EXPORT_SYMBOL_GPL(kvm_emulate_halt);
-int kvm_hypercall(struct kvm_vcpu *vcpu, struct kvm_run *run)
+int kvm_emulate_hypercall(struct kvm_vcpu *vcpu)
{
- unsigned long nr, a0, a1, a2, a3, a4, a5, ret;
+ unsigned long nr, a0, a1, a2, a3, ret;
kvm_x86_ops-cache_regs(vcpu);
- ret = -KVM_EINVAL;
-#ifdef CONFIG_X86_64
- if (is_long_mode(vcpu)) {
- nr = vcpu-regs[VCPU_REGS_RAX];
- a0 = vcpu-regs[VCPU_REGS_RDI];
- a1 = vcpu-regs[VCPU_REGS_RSI];
- a2 = vcpu-regs[VCPU_REGS_RDX];
- a3 = vcpu-regs[VCPU_REGS_RCX];
- a4 = vcpu-regs[VCPU_REGS_R8];
- a5 = vcpu-regs[VCPU_REGS_R9];
- } else
-#endif
- {
- nr = vcpu-regs[VCPU_REGS_RBX] -1u;
- a0 = vcpu-regs[VCPU_REGS_RAX] -1u;
- a1 = vcpu-regs[VCPU_REGS_RCX] -1u;
- a2 = vcpu-regs[VCPU_REGS_RDX] -1u;
- a3 = vcpu-regs[VCPU_REGS_RSI] -1u;
- a4 = vcpu-regs[VCPU_REGS_RDI] -1u;
- a5 = vcpu-regs[VCPU_REGS_RBP] -1u;
- }
+
+ nr = vcpu-regs[VCPU_REGS_RAX];
+ a0 = vcpu-regs[VCPU_REGS_RBX];
+ a1 = vcpu-regs[VCPU_REGS_RCX];
+ a2 = vcpu-regs[VCPU_REGS_RDX];
+ a3 = vcpu-regs[VCPU_REGS_RSI];
+
+ if (!is_long_mode(vcpu)) {
+ nr = 0x;
+ a0 = 0x;
+ a1 = 0x;
+ a2 = 0x;
+ a3 = 0x;
+ }
+
switch (nr) {
default:
- run-hypercall.nr = nr;
- run-hypercall.args[0] = a0;
- run-hypercall.args[1] = a1;
- run-hypercall.args[2] = a2;
- run-hypercall.args[3] = a3;
- run-hypercall.args[4] = a4;
- run-hypercall.args[5] = a5;
- run-hypercall.ret = ret;
- run-hypercall.longmode = is_long_mode(vcpu);
- kvm_x86_ops-decache_regs(vcpu);
- return 0;
+ ret = -KVM_ENOSYS;
+ break;
}
vcpu-regs[VCPU_REGS_RAX] = ret;
kvm_x86_ops-decache_regs(vcpu);
- return 1;
+ return 0;
+}
+EXPORT_SYMBOL_GPL(kvm_emulate_hypercall);
+
+int kvm_fix_hypercall(struct kvm_vcpu *vcpu)
+{
+ char instruction[3];
+ int ret = 0;
+
+ mutex_lock(vcpu-kvm-lock);
+
+ /*
+* Blow out the MMU to ensure that no other VCPU has an active mapping
+* to ensure that the updated hypercall appears atomically across all
+* VCPUs.
+*/
+
