Re: [PATCH 22/24] KVM: x86 emulator: restart string instruction without going back to a guest.
Gleb Natapov wrote: On Wed, Mar 10, 2010 at 07:08:31PM +0900, Takuya Yoshikawa wrote: Gleb Natapov wrote: Entering guest from time to time will not change semantics of the processor (if code is not modified under processor's feet at least). Currently we reenter guest mode after each iteration of string instruction for all instruction but ins/outs. E.g., is there no chance that during the repetitions, in the middle of the repetitions, page faults occur? If it can, without entering the guest, can we handle it? -- I lack some basic assumptions? If page fault occurs we inject it to the guest. Oh, I maight fail to tell what I worried about. Opposite, I mean, I worried about NOT reentering the guest case. Are you thinking about something specific here? If we inject exceptions Yes. when they occur and we inject interrupt when they arrive what problem do you see? I guess this is how real CPU actually works. I doubt it re-reads string instruction on each iteration. No problem if we detect and inject page faults like that. I just didn't so certain that when we encounter a page fault in the middle of the repetitions(about rep specific case), if we can inject it, suspend the repetition and enter the guest immediately like SDM Vol.2B says: A repeating string operation can be suspended by an exception or interrupt. When this happens, the state of the registers is preserved to allow the string operation to be resumed upon a return from the exception or interrupt handler. ... This mechanism allows long string operations to proceed without affecting the interrupt response time of the system. Ah, I might misunderstand that if we reenter the guest every time for rep, page fault detection, not injection, can be done by the other ways easily, by EXIT reason or something. Both ways may need the same thing, sorry. Another concern I wrote was just about the dependencies between your time to time criteria and SDM's without affecting the interrupt response time. This is just the problem of how we can determine the criteria appropriately. I know that current implementation with reentrance is OK. Current implementation does not reenter guest on each iteration for pio string, so currently we have both variants. I'm sorry, I was confused as if the current implementation already included some of your patches. To inject a page fault without reentering the guest, we need to add some more hacks to the emulator IIUC. No, we just need to enter guest if exception happens. I see that this in handled incorrectly in my current patch series. I was just not certain if the following condition(from SDM Vol.2B) is satisfied The source and destination registers point to the next string elements to be operated on, the EIP register points to the string instruction, and the ECX register has the value it held following the last successful iteration of the instruction. in the emulator's fault handling. I should have read your patch more closely. Thanks, Takuya -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 22/24] KVM: x86 emulator: restart string instruction without going back to a guest.
On Thu, Mar 11, 2010 at 06:58:14PM +0900, Takuya Yoshikawa wrote: Gleb Natapov wrote: On Wed, Mar 10, 2010 at 07:08:31PM +0900, Takuya Yoshikawa wrote: Gleb Natapov wrote: Entering guest from time to time will not change semantics of the processor (if code is not modified under processor's feet at least). Currently we reenter guest mode after each iteration of string instruction for all instruction but ins/outs. E.g., is there no chance that during the repetitions, in the middle of the repetitions, page faults occur? If it can, without entering the guest, can we handle it? -- I lack some basic assumptions? If page fault occurs we inject it to the guest. Oh, I maight fail to tell what I worried about. Opposite, I mean, I worried about NOT reentering the guest case. Are you thinking about something specific here? If we inject exceptions Yes. when they occur and we inject interrupt when they arrive what problem do you see? I guess this is how real CPU actually works. I doubt it re-reads string instruction on each iteration. No problem if we detect and inject page faults like that. Yes, that part is missing from my patch. I just didn't so certain that when we encounter a page fault in the middle of the repetitions(about rep specific case), if we can inject it, suspend the repetition and enter the guest immediately like SDM Vol.2B says: A repeating string operation can be suspended by an exception or interrupt. When this happens, the state of the registers is preserved to allow the string operation to be resumed upon a return from the exception or interrupt handler. ... This mechanism allows long string operations to proceed without affecting the interrupt response time of the system. Ah, I might misunderstand that if we reenter the guest every time for rep, page fault detection, not injection, can be done by the other ways easily, by EXIT reason or something. Both ways may need the same thing, sorry. When instruction is emulated page fault detection is done by the emulator itself. During guest entry the exception is injected. So all we need to do in the emulator is to enter guest immediately when exception condition is detected. Another concern I wrote was just about the dependencies between your time to time criteria and SDM's without affecting the interrupt response time. This is just the problem of how we can determine the criteria appropriately. We can reenter guest immediately if there is pending interrupt (we can't do that with ins read ahead, but this optimization is non architectural anyway). I know that current implementation with reentrance is OK. Current implementation does not reenter guest on each iteration for pio string, so currently we have both variants. I'm sorry, I was confused as if the current implementation already included some of your patches. It's independent from my patches. This is how string pio always worked. Otherwise certain workloads are too slow. To inject a page fault without reentering the guest, we need to add some more hacks to the emulator IIUC. No, we just need to enter guest if exception happens. I see that this in handled incorrectly in my current patch series. I was just not certain if the following condition(from SDM Vol.2B) is satisfied The source and destination registers point to the next string elements to be operated on, the EIP register points to the string instruction, and the ECX register has the value it held following the last successful iteration of the instruction. It is satisfied. Writeback is done on each iteration. in the emulator's fault handling. I should have read your patch more closely. Thanks, Takuya -- Gleb. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 22/24] KVM: x86 emulator: restart string instruction without going back to a guest.
On Wed, Mar 10, 2010 at 11:30:20AM +0900, Takuya Yoshikawa wrote: Gleb Natapov wrote: On Tue, Mar 09, 2010 at 04:50:29PM +0200, Avi Kivity wrote: On 03/09/2010 04:09 PM, Gleb Natapov wrote: Currently when string instruction is only partially complete we go back to a guest mode, guest tries to reexecute instruction and exits again and at this point emulation continues. Avoid all of this by restarting instruction without going back to a guest mode. What happens if rcx is really big? Going back into the guest gave us a preemption point. Two solutions. We can check if reschedule is required and yield cpu if needed. Or we can enter guest from time to time. One generic question: from the viewpoint of KVM's policy, is it OK to make the semantics different from real CPUs? Semantics, may be better to use other words, but I'm little bit worried that the second solution may change something, not mentioning about bugs but some behavior patterns depending on the time to time. Entering guest from time to time will not change semantics of the processor (if code is not modified under processor's feet at least). Currently we reenter guest mode after each iteration of string instruction for all instruction but ins/outs. -- Gleb. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 22/24] KVM: x86 emulator: restart string instruction without going back to a guest.
Gleb Natapov wrote: On Wed, Mar 10, 2010 at 11:30:20AM +0900, Takuya Yoshikawa wrote: Gleb Natapov wrote: On Tue, Mar 09, 2010 at 04:50:29PM +0200, Avi Kivity wrote: On 03/09/2010 04:09 PM, Gleb Natapov wrote: Currently when string instruction is only partially complete we go back to a guest mode, guest tries to reexecute instruction and exits again and at this point emulation continues. Avoid all of this by restarting instruction without going back to a guest mode. What happens if rcx is really big? Going back into the guest gave us a preemption point. Two solutions. We can check if reschedule is required and yield cpu if needed. Or we can enter guest from time to time. One generic question: from the viewpoint of KVM's policy, is it OK to make the semantics different from real CPUs? Semantics, may be better to use other words, but I'm little bit worried that the second solution may change something, not mentioning about bugs but some behavior patterns depending on the time to time. Entering guest from time to time will not change semantics of the processor (if code is not modified under processor's feet at least). Currently we reenter guest mode after each iteration of string instruction for all instruction but ins/outs. E.g., is there no chance that during the repetitions, in the middle of the repetitions, page faults occur? If it can, without entering the guest, can we handle it? -- I lack some basic assumptions? -- Gleb. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 22/24] KVM: x86 emulator: restart string instruction without going back to a guest.
On 03/09/2010 08:11 PM, Gleb Natapov wrote: On Tue, Mar 09, 2010 at 04:50:29PM +0200, Avi Kivity wrote: On 03/09/2010 04:09 PM, Gleb Natapov wrote: Currently when string instruction is only partially complete we go back to a guest mode, guest tries to reexecute instruction and exits again and at this point emulation continues. Avoid all of this by restarting instruction without going back to a guest mode. What happens if rcx is really big? Going back into the guest gave us a preemption point. Two solutions. We can check if reschedule is required and yield cpu if needed. Or we can enter guest from time to time. I'd stick with the current solution, reentering the guest every page or so. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 22/24] KVM: x86 emulator: restart string instruction without going back to a guest.
On 03/10/2010 11:12 AM, Takuya Yoshikawa wrote: Entering guest from time to time will not change semantics of the processor (if code is not modified under processor's feet at least). Currently we reenter guest mode after each iteration of string instruction for all instruction but ins/outs. E.g., is there no chance that during the repetitions, in the middle of the repetitions, page faults occur? If it can, without entering the guest, can we handle it? Page faults can occur, and we need to handle them. Another reason for reentering the guest is so that we can inject guest interrupts. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 22/24] KVM: x86 emulator: restart string instruction without going back to a guest.
On Wed, Mar 10, 2010 at 06:12:34PM +0900, Takuya Yoshikawa wrote: Gleb Natapov wrote: On Wed, Mar 10, 2010 at 11:30:20AM +0900, Takuya Yoshikawa wrote: Gleb Natapov wrote: On Tue, Mar 09, 2010 at 04:50:29PM +0200, Avi Kivity wrote: On 03/09/2010 04:09 PM, Gleb Natapov wrote: Currently when string instruction is only partially complete we go back to a guest mode, guest tries to reexecute instruction and exits again and at this point emulation continues. Avoid all of this by restarting instruction without going back to a guest mode. What happens if rcx is really big? Going back into the guest gave us a preemption point. Two solutions. We can check if reschedule is required and yield cpu if needed. Or we can enter guest from time to time. One generic question: from the viewpoint of KVM's policy, is it OK to make the semantics different from real CPUs? Semantics, may be better to use other words, but I'm little bit worried that the second solution may change something, not mentioning about bugs but some behavior patterns depending on the time to time. Entering guest from time to time will not change semantics of the processor (if code is not modified under processor's feet at least). Currently we reenter guest mode after each iteration of string instruction for all instruction but ins/outs. E.g., is there no chance that during the repetitions, in the middle of the repetitions, page faults occur? If it can, without entering the guest, can we handle it? -- I lack some basic assumptions? If page fault occurs we inject it to the guest. -- Gleb. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 22/24] KVM: x86 emulator: restart string instruction without going back to a guest.
Gleb Natapov wrote: Entering guest from time to time will not change semantics of the processor (if code is not modified under processor's feet at least). Currently we reenter guest mode after each iteration of string instruction for all instruction but ins/outs. E.g., is there no chance that during the repetitions, in the middle of the repetitions, page faults occur? If it can, without entering the guest, can we handle it? -- I lack some basic assumptions? If page fault occurs we inject it to the guest. Oh, I maight fail to tell what I worried about. Opposite, I mean, I worried about NOT reentering the guest case. I know that current implementation with reentrance is OK. To inject a page fault without reentering the guest, we need to add some more hacks to the emulator IIUC. Thanks, Takuya -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 22/24] KVM: x86 emulator: restart string instruction without going back to a guest.
On Wed, Mar 10, 2010 at 07:08:31PM +0900, Takuya Yoshikawa wrote: Gleb Natapov wrote: Entering guest from time to time will not change semantics of the processor (if code is not modified under processor's feet at least). Currently we reenter guest mode after each iteration of string instruction for all instruction but ins/outs. E.g., is there no chance that during the repetitions, in the middle of the repetitions, page faults occur? If it can, without entering the guest, can we handle it? -- I lack some basic assumptions? If page fault occurs we inject it to the guest. Oh, I maight fail to tell what I worried about. Opposite, I mean, I worried about NOT reentering the guest case. Are you thinking about something specific here? If we inject exceptions when they occur and we inject interrupt when they arrive what problem do you see? I guess this is how real CPU actually works. I doubt it re-reads string instruction on each iteration. I know that current implementation with reentrance is OK. Current implementation does not reenter guest on each iteration for pio string, so currently we have both variants. To inject a page fault without reentering the guest, we need to add some more hacks to the emulator IIUC. No, we just need to enter guest if exception happens. I see that this in handled incorrectly in my current patch series. -- Gleb. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 22/24] KVM: x86 emulator: restart string instruction without going back to a guest.
Currently when string instruction is only partially complete we go back
to a guest mode, guest tries to reexecute instruction and exits again
and at this point emulation continues. Avoid all of this by restarting
instruction without going back to a guest mode.
Signed-off-by: Gleb Natapov g...@redhat.com
---
arch/x86/include/asm/kvm_emulate.h |1 +
arch/x86/kvm/emulate.c | 22 --
arch/x86/kvm/x86.c | 16 +++-
3 files changed, 28 insertions(+), 11 deletions(-)
diff --git a/arch/x86/include/asm/kvm_emulate.h
b/arch/x86/include/asm/kvm_emulate.h
index 7d323d5..f74b4ad 100644
--- a/arch/x86/include/asm/kvm_emulate.h
+++ b/arch/x86/include/asm/kvm_emulate.h
@@ -193,6 +193,7 @@ struct x86_emulate_ctxt {
/* interruptibility state, as a result of execution of STI or MOV SS */
int interruptibility;
+ bool restart; /* restart string instruction after writeback */
/* decode cache */
struct decode_cache decode;
};
diff --git a/arch/x86/kvm/emulate.c b/arch/x86/kvm/emulate.c
index ba1ce61..76ed77d 100644
--- a/arch/x86/kvm/emulate.c
+++ b/arch/x86/kvm/emulate.c
@@ -925,8 +925,11 @@ x86_decode_insn(struct x86_emulate_ctxt *ctxt, struct
x86_emulate_ops *ops)
int mode = ctxt-mode;
int def_op_bytes, def_ad_bytes, group;
- /* Shadow copy of register state. Committed on successful emulation. */
+ /* we cannot decode insn before we complete previous rep insn */
+ WARN_ON(ctxt-restart);
+
+ /* Shadow copy of register state. Committed on successful emulation. */
memset(c, 0, sizeof(struct decode_cache));
c-eip = ctxt-eip;
ctxt-cs_base = seg_base(ctxt, VCPU_SREG_CS);
@@ -2412,8 +2415,11 @@ x86_emulate_insn(struct x86_emulate_ctxt *ctxt, struct
x86_emulate_ops *ops)
memop = c-modrm_ea;
if (c-rep_prefix (c-d String)) {
+ ctxt-restart = true;
/* All REP prefixes have the same first termination condition */
if (c-regs[VCPU_REGS_RCX] == 0) {
+ string_done:
+ ctxt-restart = false;
kvm_rip_write(ctxt-vcpu, c-eip);
goto done;
}
@@ -2425,17 +2431,13 @@ x86_emulate_insn(struct x86_emulate_ctxt *ctxt, struct
x86_emulate_ops *ops)
* - if REPNE/REPNZ and ZF = 1 then done
*/
if ((c-b == 0xa6) || (c-b == 0xa7) ||
- (c-b == 0xae) || (c-b == 0xaf)) {
+ (c-b == 0xae) || (c-b == 0xaf)) {
if ((c-rep_prefix == REPE_PREFIX)
- ((ctxt-eflags EFLG_ZF) == 0)) {
- kvm_rip_write(ctxt-vcpu, c-eip);
- goto done;
- }
+ ((ctxt-eflags EFLG_ZF) == 0))
+ goto string_done;
if ((c-rep_prefix == REPNE_PREFIX)
- ((ctxt-eflags EFLG_ZF) == EFLG_ZF)) {
- kvm_rip_write(ctxt-vcpu, c-eip);
- goto done;
- }
+ ((ctxt-eflags EFLG_ZF) == EFLG_ZF))
+ goto string_done;
}
if (c-src.type == OP_MEM)
memop = register_address(c, seg_override_base(ctxt, c),
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index b25ef4b..82379e1 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -3724,6 +3724,7 @@ int emulate_instruction(struct kvm_vcpu *vcpu,
return EMULATE_DONE;
}
+restart:
r = x86_emulate_insn(vcpu-arch.emulate_ctxt, emulate_ops);
shadow_mask = vcpu-arch.emulate_ctxt.interruptibility;
@@ -3746,7 +3747,7 @@ int emulate_instruction(struct kvm_vcpu *vcpu,
if (r) {
if (kvm_mmu_unprotect_page_virt(vcpu, cr2))
- return EMULATE_DONE;
+ goto done;
if (!vcpu-mmio_needed) {
kvm_report_emulation_failure(vcpu, mmio);
return EMULATE_FAIL;
@@ -3761,6 +3762,10 @@ int emulate_instruction(struct kvm_vcpu *vcpu,
return EMULATE_DO_MMIO;
}
+done:
+ if (vcpu-arch.emulate_ctxt.restart)
+ goto restart;
+
return EMULATE_DONE;
}
EXPORT_SYMBOL_GPL(emulate_instruction);
@@ -4516,6 +4521,15 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu,
struct kvm_run *kvm_run)
goto out;
}
}
+ if (vcpu-arch.emulate_ctxt.restart) {
+ vcpu-srcu_idx = srcu_read_lock(vcpu-kvm-srcu);
+ r = emulate_instruction(vcpu, 0, 0, EMULTYPE_NO_DECODE);
+ srcu_read_unlock(vcpu-kvm-srcu,
Re: [PATCH 22/24] KVM: x86 emulator: restart string instruction without going back to a guest.
On 03/09/2010 04:09 PM, Gleb Natapov wrote: Currently when string instruction is only partially complete we go back to a guest mode, guest tries to reexecute instruction and exits again and at this point emulation continues. Avoid all of this by restarting instruction without going back to a guest mode. What happens if rcx is really big? Going back into the guest gave us a preemption point. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 22/24] KVM: x86 emulator: restart string instruction without going back to a guest.
On Tue, Mar 09, 2010 at 04:50:29PM +0200, Avi Kivity wrote: On 03/09/2010 04:09 PM, Gleb Natapov wrote: Currently when string instruction is only partially complete we go back to a guest mode, guest tries to reexecute instruction and exits again and at this point emulation continues. Avoid all of this by restarting instruction without going back to a guest mode. What happens if rcx is really big? Going back into the guest gave us a preemption point. Two solutions. We can check if reschedule is required and yield cpu if needed. Or we can enter guest from time to time. -- Gleb. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 22/24] KVM: x86 emulator: restart string instruction without going back to a guest.
Gleb Natapov wrote: On Tue, Mar 09, 2010 at 04:50:29PM +0200, Avi Kivity wrote: On 03/09/2010 04:09 PM, Gleb Natapov wrote: Currently when string instruction is only partially complete we go back to a guest mode, guest tries to reexecute instruction and exits again and at this point emulation continues. Avoid all of this by restarting instruction without going back to a guest mode. What happens if rcx is really big? Going back into the guest gave us a preemption point. Two solutions. We can check if reschedule is required and yield cpu if needed. Or we can enter guest from time to time. One generic question: from the viewpoint of KVM's policy, is it OK to make the semantics different from real CPUs? Semantics, may be better to use other words, but I'm little bit worried that the second solution may change something, not mentioning about bugs but some behavior patterns depending on the time to time. -- Gleb. -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe kvm in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
