Re: Added VM Exit on RDTSC, trouble handling in userspace
On Tue, Oct 13, 2009 at 10:51:48PM -0700, Kurt Kiefer wrote:
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Hi all,
In short, I have a need for trapping RDTSC with a VM Exit and this
works, but I'm having trouble handling it in userspace. I have added the
hooks I need (I only care about VMX right now), but a piece of the
puzzle is missing and I don't know which. When I go back to userspace,
it's triggering a different (faulty) execution vs. handling only in the
kernel. Here's what I've done:
1. Added the CPU_BASED_RDTSC_EXITING flag to MSR_IA32_VMX_PROCBASED_CTLS
in vmx.c:setup_vmcs_config()
2. Defined KVM_EXIT_RDTSC, and hooked into EXIT_REASON_RDTSC my handler
for the exit:
static int (*kvm_vmx_exit_handlers[])(struct kvm_vcpu *vcpu,
struct kvm_run *kvm_run) = {
// ...
[EXIT_REASON_RDTSC] = handle_rdtsc,
// ...
}
static int handle_rdtsc(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run)
{
u64 data;
if (vmx_get_msr(vcpu, MSR_IA32_TIME_STAMP_COUNTER, data)) {
kvm_inject_gp(vcpu, 0);
return 1;
}
vcpu-run-exit_reason = KVM_EXIT_RDTSC;
vcpu-arch.regs[VCPU_REGS_RAX] = data -1u;
vcpu-arch.regs[VCPU_REGS_RDX] = (data 32) -1u;
skip_emulated_instruction(vcpu);
// flag a need for userspace invervention
// note: this works when we return 1 and we don't involve userspace
return 0;
}
3. Handle KVM_EXIT_RDTSC in libkvm.c:kvm_run() :
case KVM_EXIT_RDTSC:
r = handle_rdtsc_usp(kvm, vcpu, env);
break;
via a handler where I do _nothing_ :
static int handle_rdtsc_usp(kvm_context_t kvm, int vcpu, void *data)
{
return 0;
}
All well and good, right? I can add print statements to my userspace
handle_rtsc_usp() and see I get in there just fine. However, when I try
to boot Linux, the following code is called over and over and over, and
Linux will never load:
Breakpoint 4, 0xc01103d3 in ?? ()
(gdb) x/10i $rip-10
0xc01103c9: lea0x0(%rdi,%riz,1),%edi
0xc01103d0: push %rbp
0xc01103d1: mov%esp,%ebp
0xc01103d3: rdtsc
0xc01103d5: pop%rbp
0xc01103d6: retq
If I only handle the exit in the kernel (by returning 1 from
handle_rdtsc()), everything works and Linux will load! I counted the
number of RDTSC exits before linux fully loads to be somewhere around
20. If I exit all the way to userspace (return 0 in my
handle_rdtsc()) that count is infinitely surpassed in number of exits,
wall time, and the value of RDTSC.
So is anything glaringly wrong with my modifications? Maybe there is
there some extra state that needs to be restored on VM entry? Is there
an interrupt flag that needs to be cleared? Maybe I need to do something
with kvm_run.if_flag or kvm_run.ready_for_interrupt_injection? Please, I
need help, I'm losing sleep over this!
Can't see anything wrong. Perhaps the userspace exit breaks a latency
assumption of that algorithm (can you translate it to function names?).
Try the tsc.flat test (from qemu-kvm.git):
x86_64-softmmu/qemu-system-x86_64 -chardev file,path=/tmp/log.txt,id=testlog \
-device testdev,chardev=testlog -kernel kvm/user/test/x86/tsc.flat
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Added VM Exit on RDTSC, trouble handling in userspace
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Hi all,
In short, I have a need for trapping RDTSC with a VM Exit and this
works, but I'm having trouble handling it in userspace. I have added
the hooks I need (I only care about VMX right now), but a piece of the
puzzle is missing and I don't know which. When I go back to userspace,
it's triggering a different (faulty) execution vs. handling only in
the kernel. Here's what I've done:
1. Added the CPU_BASED_RDTSC_EXITING flag to
MSR_IA32_VMX_PROCBASED_CTLS in vmx.c:setup_vmcs_config()
2. Defined KVM_EXIT_RDTSC, and hooked into EXIT_REASON_RDTSC my
handler for the exit:
static int (*kvm_vmx_exit_handlers[])(struct kvm_vcpu *vcpu,
struct kvm_run *kvm_run) = {
// ...
[EXIT_REASON_RDTSC] = handle_rdtsc,
// ...
}
static int handle_rdtsc(struct kvm_vcpu *vcpu, struct kvm_run *kvm_run)
{
u64 data;
if (vmx_get_msr(vcpu, MSR_IA32_TIME_STAMP_COUNTER, data)) {
kvm_inject_gp(vcpu, 0);
return 1;
}
vcpu-run-exit_reason = KVM_EXIT_RDTSC;
vcpu-arch.regs[VCPU_REGS_RAX] = data -1u;
vcpu-arch.regs[VCPU_REGS_RDX] = (data 32) -1u;
skip_emulated_instruction(vcpu);
// flag a need for userspace invervention
// note: this works when we return 1 and we don't involve userspace
return 0;
}
3. Handle KVM_EXIT_RDTSC in libkvm.c:kvm_run() :
case KVM_EXIT_RDTSC:
r = handle_rdtsc_usp(kvm, vcpu, env);
break;
via a handler where I do _nothing_ :
static int handle_rdtsc_usp(kvm_context_t kvm, int vcpu, void *data)
{
return 0;
}
All well and good, right? I can add print statements to my userspace
handle_rtsc_usp() and see I get in there just fine. However, when I
try to boot Linux, the following code is called over and over and
over, and Linux will never load:
Breakpoint 4, 0xc01103d3 in ?? ()
(gdb) x/10i $rip-10
0xc01103c9: lea0x0(%rdi,%riz,1),%edi
0xc01103d0: push %rbp
0xc01103d1: mov%esp,%ebp
0xc01103d3: rdtsc
0xc01103d5: pop%rbp
0xc01103d6: retq
If I only handle the exit in the kernel (by returning 1 from
handle_rdtsc()), everything works and Linux will load! I counted the
number of RDTSC exits before linux fully loads to be somewhere around
20. If I exit all the way to userspace (return 0 in my
handle_rdtsc()) that count is infinitely surpassed in number of exits,
wall time, and the value of RDTSC.
So is anything glaringly wrong with my modifications? Maybe there is
there some extra state that needs to be restored on VM entry? Is there
an interrupt flag that needs to be cleared? Maybe I need to do
something with kvm_run.if_flag or
kvm_run.ready_for_interrupt_injection? Please, I need help, I'm losing
sleep over this!
Thanks,
Kurt
-BEGIN PGP SIGNATURE-
Version: GnuPG/MacGPG2 v2.0.12 (Darwin)
iEYEARECAAYFAkrVZvQACgkQYFGmU9mnI1FqvgCcC/+PswoXHQ5kVgv5tC6UadiA
KKgAoKrLgsYSJN0+1d0pox9vzsLHoQIc
=cQzR
-END PGP SIGNATURE-
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
