Re: [PATCH v1] kvm/x86: Hyper-V tsc page setup

[Peter Hornyack]

On Thu, Dec 24, 2015 at 1:33 AM, Andrey Smetanin
<asmeta...@virtuozzo.com> wrote:
> Lately tsc page was implemented but filled with empty
> values. This patch setup tsc page scale and offset based
> on vcpu tsc, tsc_khz and  HV_X64_MSR_TIME_REF_COUNT value.
>
> The valid tsc page drops HV_X64_MSR_TIME_REF_COUNT msr
> reads count to zero which potentially improves performance.
>
> The patch applies on top of
> 'kvm: Make vcpu->requests as 64 bit bitmap'
> previously sent.
>
> Signed-off-by: Andrey Smetanin <asmeta...@virtuozzo.com>
> CC: Paolo Bonzini <pbonz...@redhat.com>
> CC: Gleb Natapov <g...@kernel.org>
> CC: Roman Kagan <rka...@virtuozzo.com>
> CC: Denis V. Lunev <d...@openvz.org>
> CC: qemu-de...@nongnu.org
Reviewed-by: Peter Hornyack <peterhorny...@google.com>

>
> ---
>  arch/x86/kvm/hyperv.c| 117 
> +--
>  arch/x86/kvm/hyperv.h|   2 +
>  arch/x86/kvm/x86.c   |  12 +
>  include/linux/kvm_host.h |   1 +
>  4 files changed, 117 insertions(+), 15 deletions(-)
>
> diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c
> index d50675a..504fdc7 100644
> --- a/arch/x86/kvm/hyperv.c
> +++ b/arch/x86/kvm/hyperv.c
> @@ -753,6 +753,105 @@ static int kvm_hv_msr_set_crash_data(struct kvm_vcpu 
> *vcpu,
> return 0;
>  }
>
> +static u64 calc_tsc_page_scale(u32 tsc_khz)
> +{
> +   /*
> +* reftime (in 100ns) = tsc * tsc_scale / 2^64 + tsc_offset
> +* so reftime_delta = (tsc_delta * tsc_scale) / 2^64
> +* so tsc_scale = (2^64 * reftime_delta)/tsc_delta
> +* so tsc_scale = (2^64 * 10 * 10^6) / tsc_hz = (2^64 * 1) / 
> tsc_khz
> +* so tsc_scale = (2^63 * 2 * 1) / tsc_khz
> +*/
> +   return mul_u64_u32_div(1ULL << 63, 2 * 1, tsc_khz);
> +}
> +
> +static int write_tsc_page(struct kvm *kvm, u64 gfn,
> + PHV_REFERENCE_TSC_PAGE tsc_ref)
> +{
> +   if (kvm_write_guest(kvm, gfn_to_gpa(gfn),
> +   tsc_ref, sizeof(*tsc_ref)))
> +   return 1;
> +   mark_page_dirty(kvm, gfn);
> +   return 0;
> +}
> +
> +static int read_tsc_page(struct kvm *kvm, u64 gfn,
> +PHV_REFERENCE_TSC_PAGE tsc_ref)
> +{
> +   if (kvm_read_guest(kvm, gfn_to_gpa(gfn),
> +  tsc_ref, sizeof(*tsc_ref)))
> +   return 1;
> +   return 0;
> +}
> +
> +static u64 calc_tsc_page_time(struct kvm_vcpu *vcpu,
> + PHV_REFERENCE_TSC_PAGE tsc_ref)
> +{
> +
> +   u64 tsc = kvm_read_l1_tsc(vcpu, rdtsc());
> +
> +   return mul_u64_u64_shr(tsc, tsc_ref->tsc_scale, 64)
> +   + tsc_ref->tsc_offset;
> +}
> +
> +static int setup_blank_tsc_page(struct kvm_vcpu *vcpu, u64 gfn)
> +{
> +   HV_REFERENCE_TSC_PAGE tsc_ref;
> +
> +   memset(_ref, 0, sizeof(tsc_ref));
> +   return write_tsc_page(vcpu->kvm, gfn, _ref);
> +}
> +
> +int kvm_hv_setup_tsc_page(struct kvm_vcpu *vcpu)
> +{
> +   struct kvm *kvm = vcpu->kvm;
> +   struct kvm_hv *hv = >arch.hyperv;
> +   HV_REFERENCE_TSC_PAGE tsc_ref;
> +   u32 tsc_khz;
> +   int r;
> +   u64 gfn, ref_time, tsc_scale, tsc_offset, tsc;
> +
> +   if (WARN_ON_ONCE(!(hv->hv_tsc_page & 
> HV_X64_MSR_TSC_REFERENCE_ENABLE)))
> +   return -EINVAL;
> +
> +   gfn = hv->hv_tsc_page >> HV_X64_MSR_TSC_REFERENCE_ADDRESS_SHIFT;
> +   vcpu_debug(vcpu, "tsc page gfn 0x%llx\n", gfn);
> +
> +   tsc_khz = vcpu->arch.virtual_tsc_khz;
> +   if (!tsc_khz) {
> +   vcpu_unimpl(vcpu, "no tsc khz\n");
> +   return setup_blank_tsc_page(vcpu, gfn);
> +   }
> +
> +   r = read_tsc_page(kvm, gfn, _ref);
> +   if (r) {
> +   vcpu_err(vcpu, "can't access tsc page gfn 0x%llx\n", gfn);
> +   return r;
> +   }
> +
> +   tsc_scale = calc_tsc_page_scale(tsc_khz);
> +   ref_time = get_time_ref_counter(kvm);
> +   tsc = kvm_read_l1_tsc(vcpu, rdtsc());
> +
> +   /* tsc_offset = reftime - tsc * tsc_scale / 2^64 */
> +   tsc_offset = ref_time - mul_u64_u64_shr(tsc, tsc_scale, 64);
> +   vcpu_debug(vcpu, "tsc khz %u tsc %llu scale %llu offset %llu\n",
> +  tsc_khz, tsc, tsc_scale, tsc_offset);
> +
> +   tsc_ref.tsc_sequence++;
> +   if (tsc_ref.tsc_sequence == 0)

Also avoid tsc_sequence == 0x here. In the Hyper-V TLFS 4.0
(Win2012 R2) 0x is the special sequenc

Re: [RFC PATCH 2/5] KVM: add KVM_EXIT_MSR exit reason and capability.

[Peter Hornyack]

On Fri, Dec 18, 2015 at 1:25 PM, Paolo Bonzini <pbonz...@redhat.com> wrote:
>
>
> On 18/08/2015 20:46, Peter Hornyack wrote:
>> Define KVM_EXIT_MSR, a new exit reason for accesses to MSRs that kvm
>> does not handle. Define KVM_CAP_UNHANDLED_MSR_EXITS, a vm-wide
>> capability that guards the new exit reason and which can be enabled via
>> the KVM_ENABLE_CAP ioctl.
>>
>> Signed-off-by: Peter Hornyack <peterhorny...@google.com>
>> ---
>>  Documentation/virtual/kvm/api.txt | 48 
>> +++
>>  include/uapi/linux/kvm.h  | 14 
>>  2 files changed, 62 insertions(+)
>
> Let's instead add:
>
> - KVM_CAP_MSR_EXITS
>
> - KVM_CAP_ENABLE_MSR_EXIT
>
> - KVM_CAP_DISABLE_MSR_EXIT
>
> and 3 kinds of exits: KVM_EXIT_MSR_READ, KVM_EXIT_MSR_WRITE,
> KVM_EXIT_MSR_AFTER_WRITE.  The first two use four fields (type, msr,
> data, handled) and #GP if handled=0 is zero on the next entry.  The last
> one uses the first three fields and can be used for HyperV.
>
> Paolo

Paolo, I've included an updated version of this patch below. Does this
match the API that you had in mind? If so, I'll continue adjusting the
rest of the code and will send the entire new patchset.

This updated version of the API seems more cumbersome to me (three new
capabilities, three exit reasons when just one or two might suffice)
than the original interface I used, but maybe you have a good reason
for that. Also, will it be ok to have just one capability to enable
all three kinds of exits, or will KVM_EXIT_MSR_AFTER_WRITE want a
separate capability?


commit a684d520ed62cf0db4495e5197d5bf722e4f8109
Author: Peter Hornyack <peterhorny...@google.com>
Date:   Fri Dec 18 14:44:04 2015 -0800

KVM: add capabilities and exit reasons for MSRs.

Define KVM_EXIT_MSR_READ, KVM_EXIT_MSR_WRITE, and
KVM_EXIT_MSR_AFTER_WRITE, new exit reasons for accesses to MSRs that kvm
does not handle or that user space needs to be notified about. Define the
KVM_CAP_MSR_EXITS, KVM_CAP_ENABLE_MSR_EXITS, and KVM_CAP_DISABLE_MSR_EXITS
capabilities to control these new exits for a VM.

diff --git a/Documentation/virtual/kvm/api.txt
b/Documentation/virtual/kvm/api.txt
index 053f613fc9a9..3bba3248df3d 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -3359,6 +3359,43 @@ Hyper-V SynIC state change. Notification is
used to remap SynIC
 event/message pages and to enable/disable SynIC messages/events processing
 in userspace.

+ /*
+ * KVM_EXIT_MSR_READ, KVM_EXIT_MSR_WRITE,
+ * KVM_EXIT_MSR_AFTER_WRITE
+ */
+ struct {
+ __u32 index;/* i.e. ecx; out */
+ __u64 data; /* out (wrmsr) / in (rdmsr) */
+#define KVM_EXIT_MSR_COMPLETION_FAILED 1
+ __u64 type; /* out */
+#define KVM_EXIT_MSR_UNHANDLED 0
+#define KVM_EXIT_MSR_HANDLED   1
+ __u8 handled;   /* in */
+ } msr;
+
+If exit_reason is KVM_EXIT_MSR_READ or KVM_EXIT_MSR_WRITE, then the vcpu has
+executed a rdmsr or wrmsr instruction which could not be satisfied by kvm. The
+msr struct is used for both output to and input from user space. index is the
+target MSR number held in ecx; user space must not modify this field. data
+holds the payload from a wrmsr or must be filled in with a payload on a rdmsr.
+For a normal exit, type will be 0.
+
+On the return path into kvm, user space should set handled to
+KVM_EXIT_MSR_HANDLED if it successfully handled the MSR access. Otherwise,
+handled should be set to KVM_EXIT_MSR_UNHANDLED, which will cause a general
+protection fault to be injected into the vcpu. If an error occurs during the
+return into kvm, the vcpu will not be run and another exit will be generated
+with type set to KVM_EXIT_MSR_COMPLETION_FAILED.
+
+If exit_reason is KVM_EXIT_MSR_AFTER_WRITE, then the vcpu has executed a wrmsr
+instruction which is handled by kvm but which user space may need to be
+notified about. index and data are set as described above; the value of type
+depends on the MSR that was written. handled is ignored on reentry into kvm.
+
+KVM_EXIT_MSR_READ, KVM_EXIT_MSR_WRITE, and KVM_EXIT_MSR_AFTER_WRITE can only
+occur when KVM_CAP_MSR_EXITS is present and KVM_CAP_ENABLE_MSR_EXITS has been
+set. A detailed description of these capabilities is below.
+
  /* Fix the size of the union. */
  char padding[256];
  };
@@ -3697,6 +3734,26 @@ a KVM_EXIT_IOAPIC_EOI vmexit will be reported
to userspace.
 Fails if VCPU has already been created, or if the irqchip is already in the
 kernel (i.e. KVM_CREATE_IRQCHIP has already been called).

+7.6 KVM_CAP_ENABLE_MSR_EXITS, KVM_CAP_DISABLE_MSR_EXITS
+
+Architectures: x86 (vmx-only)
+Parameters: none
+Returns: 0 on success, -1 on error
+
+These capabilities enable and disable exits to user space for certain guest MSR
+accesses. These capabilities are only available if KVM_CHECK_EXTENSION
+indicates that KVM_CAP_MSR_EXITS is pres

Re: [PATCH v4 5/5] kvm/x86: Hyper-V kvm exit

[Peter Hornyack]

On Fri, Dec 18, 2015 at 8:01 AM, Paolo Bonzini  wrote:
>
>
> On 18/12/2015 16:19, Pavel Fedin wrote:
>> As far as i understand this code, KVM_EXIT_HYPERV is called when one
>> of three MSRs are accessed. But, shouldn't we have implemented
>> instead something more generic, like KVM_EXIT_REG_IO, which would
>> work similar to KVM_EXIT_PIO or KVM_EXIT_MMIO, but carry register
>> code and value?
>
> Yes, we considered that.  There were actually patches for this as well.
>  However, in this case the register is still emulated in the kernel, and
> userspace just gets informed of the new value.

On brief inspection of Andrey's patch (I have not been following
closely) it looks like the kvm_hyperv_exit struct that's returned to
userspace contains more data (control, evt_page, and msg_page fields)
than simply the value of the MSR, so would the desired SynIC exit fit
into a general-purpose exit for MSR emulation?

>> This would allow us to solve the same task which we have done here,
>> but this solution would be reusable for other devices and other
>> archirectures. What if in future we have more system registers to
>> emulate in userspace?
>
> If we do get that, we will just rename KVM_EXIT_HYPERV to
> KVM_EXIT_MSR_ACCESS, and KVM_EXIT_HYPERV_SYNIC to
> KVM_EXIT_MSR_HYPERV_SYNIC, and struct kvm_hyperv_exit to kvm_msr_exit.
>
> Actually, we can do it now.  What do you guys think?  Peter?  I might
> even be convinced to document the capability (in Documentation/ and
> uapi/) even if the upstream kernel doesn't provide it.  We still have a
> lot of time until 4.5 is out, it can be done after the merge window even.

I will update and re-send that patchset for discussion.

>
> Paolo
>
>> I write this because at one point i suggested similar thing for ARM64
>> (but i never actually wrote it), to emulate physical CP15 timer. And
>> it would require exactly the same capability - process some trapped
>> system register accesses in userspace.

Peter
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC PATCH 2/5] KVM: add KVM_EXIT_MSR exit reason and capability.

[Peter Hornyack]

On Fri, Dec 18, 2015 at 1:25 PM, Paolo Bonzini <pbonz...@redhat.com> wrote:
>
>
> On 18/08/2015 20:46, Peter Hornyack wrote:
>> Define KVM_EXIT_MSR, a new exit reason for accesses to MSRs that kvm
>> does not handle. Define KVM_CAP_UNHANDLED_MSR_EXITS, a vm-wide
>> capability that guards the new exit reason and which can be enabled via
>> the KVM_ENABLE_CAP ioctl.
>>
>> Signed-off-by: Peter Hornyack <peterhorny...@google.com>
>> ---
>>  Documentation/virtual/kvm/api.txt | 48 
>> +++
>>  include/uapi/linux/kvm.h  | 14 
>>  2 files changed, 62 insertions(+)
>
> Let's instead add:
>
> - KVM_CAP_MSR_EXITS
>
> - KVM_CAP_ENABLE_MSR_EXIT
>
> - KVM_CAP_DISABLE_MSR_EXIT
>
> and 3 kinds of exits: KVM_EXIT_MSR_READ, KVM_EXIT_MSR_WRITE,
> KVM_EXIT_MSR_AFTER_WRITE.  The first two use four fields (type, msr,
> data, handled) and #GP if handled=0 is zero on the next entry.  The last
> one uses the first three fields and can be used for HyperV.
>
> Paolo

Ok. I'm working on these adjustments now, will send the updated
patchset on Monday.

Peter
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] KVM: x86: expose MSR_TSC_AUX to userspace

[Peter Hornyack]

On Fri, Nov 13, 2015 at 1:42 AM, Paolo Bonzini  wrote:
>
>> Paolo, under what circumstances (which versions of Windows? Anything
>> special running in the guest?) has this failure happened? I'd like to repro
>> this, I'm not sure if we've observed it before.
>
> We saw it with migration under Windows 10, nothing special running in the
> guest.  It's very hard to reproduce, we've only seen it once but the BSOD
> parameters provided surprisingly good evidence:

Great, thanks for that information and for the patch. I'll let you
know if I successfully reproduce the issue here.

> --
> CRITICAL_STRUCTURE_CORRUPTION (109)
> This bugcheck is generated when the kernel detects that critical kernel code 
> or
> data have been corrupted. There are generally three causes for a corruption:
> 1) A driver has inadvertently or deliberately modified critical kernel code
>  or data. See http://www.microsoft.com/whdc/driver/kernel/64bitPatching.mspx
> 2) A developer attempted to set a normal kernel breakpoint using a kernel
>  debugger that was not attached when the system was booted. Normal 
> breakpoints,
>  "bp", can only be set if the debugger is attached at boot time. Hardware
>  breakpoints, "ba", can be set at any time.
> 3) A hardware corruption occurred, e.g. failing RAM holding kernel code or 
> data.
> Arguments:
> Arg1: a3a01f58a88e3638, Reserved
> Arg2: b3b72bdefb0f076f, Reserved
> Arg3: 0001c103, Failure type dependent information
> Arg4: 0007, Type of corrupted region, can be
> ...
> 7   : Critical MSR modification
> --
>
> Argument 1 and 2 might be related to the old and new value (perhaps some
> kind of hash).
>
> Argument 3 is not documented either, but the low 32 bits look a lot like
> the MSR_TSC_AUX index. :)
>
> Paolo

Peter
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC PATCH 0/5] KVM: x86: exit to user space on unhandled MSR accesses

[Peter Hornyack]

On Wed, Aug 19, 2015 at 2:43 PM, Bandan Das b...@redhat.com wrote:
 Peter Hornyack peterhorny...@google.com writes:

 There are numerous MSRs that kvm does not currently handle. On Intel
 platforms we have observed guest VMs accessing some of these MSRs (for
 example, MSR_PLATFORM_INFO) and behaving poorly (to the point of guest OS
 crashes) when they receive a GP fault because the MSR is not emulated. This
 patchset adds a new kvm exit path for unhandled MSR accesses that allows
 user space to emulate additional MSRs without having to implement them in
 kvm.

 ^^ So, I am trying to understand this motivation. A while back when
 a patch was posted to emulate MSR_PLATFORM_INFO, it was rejected.
 Why ? Because it seemed impossible to emulate it correctly (most concerns were
 related to migration iirc). Although I haven't reviewed all patches in this 
 series
 yet, what I understand from the above message is-It's ok to emulate
 MSR_PLATFORM_INFO *incorrectly* as long as we are doing it in the user space.

 I understand the part where it makes sense to move stuff to userspace.
 But if kvm isn't emulating certain msrs yet, either we should add support,
 or they haven't been added because it's not possible to emulate them
 correctly. The logic that it's probably ok to let userspace do the (incorrect)
 emulation is something I don't understand.

I wasn't aware of the past discussion of MSR_PLATFORM_INFO, I'll
search for it - thanks for pointing it out.

MSR_PLATFORM_INFO is just one example though. In addition to the
benefits I already mentioned for user space MSR emulation (agility,
reduced attack surface), this patchset offers a benefit even if user
space declines to emulate the MSR by returning into kvm with
KVM_EXIT_MSR_UNHANDLED: it makes it easier to monitor in the first
place, via logging mechanisms in user space instead of in the kernel,
for when guests are accessing MSRs that kvm doesn't implement.

This patchset has already revealed a few other MSRs (IA32_MPERF,
IA32_APERF) that guests in our test environment are accessing but
which kvm doesn't implement yet. I haven't yet investigated deeply
what these MSRs are used for and how they might be emulated (or if
they can't actually be emulated correctly), but if we do discover an
unimplemented non-performance-sensitive MSR that we could emulate
correctly, with this patchset we would quickly just implement it in
user space.

 It seems like the next in line
 is to let userspace emulate thier own version of unimplemented x86 
 instructions.


 The core of the patchset modifies the vmx handle_rdmsr and handle_wrmsr
 functions to exit to user space on MSR reads/writes that kvm can't handle
 itself. Then, on the return path into kvm we check for outstanding user
 space MSR completions and either complete the MSR access successfully or
 inject a GP fault as kvm would do by default. This new exit path must be
 enabled for the vm via the KVM_CAP_UNHANDLED_MSR_EXITS capability.

 In the future we plan to extend this functionality to allow user space to
 register the MSRs that it would like to handle itself, even if kvm already
 provides an implementation. In the long-term we will move the

 I seriously hope we don't do this!

 Bandan
 implementation of all non-performance-sensitive MSRs to user space,
 reducing the potential attack surface of kvm and allowing us to respond to
 bugs more quickly.

 This patchset has been tested with our non-qemu user space hypervisor on
 vmx platforms; svm support is not implemented.

 Peter Hornyack (5):
   KVM: x86: refactor vmx rdmsr/wrmsr completion into new functions
   KVM: add KVM_EXIT_MSR exit reason and capability.
   KVM: x86: add msr_exits_supported to kvm_x86_ops
   KVM: x86: enable unhandled MSR exits for vmx
   KVM: x86: add trace events for unhandled MSR exits

  Documentation/virtual/kvm/api.txt |  48 +++
  arch/x86/include/asm/kvm_host.h   |   2 +
  arch/x86/kvm/svm.c|   6 ++
  arch/x86/kvm/trace.h  |  28 +
  arch/x86/kvm/vmx.c| 126 
 ++
  arch/x86/kvm/x86.c|  13 
  include/trace/events/kvm.h|   2 +-
  include/uapi/linux/kvm.h  |  14 +
  8 files changed, 227 insertions(+), 12 deletions(-)
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[RFC PATCH 2/5] KVM: add KVM_EXIT_MSR exit reason and capability.

[Peter Hornyack]

Define KVM_EXIT_MSR, a new exit reason for accesses to MSRs that kvm
does not handle. Define KVM_CAP_UNHANDLED_MSR_EXITS, a vm-wide
capability that guards the new exit reason and which can be enabled via
the KVM_ENABLE_CAP ioctl.

Signed-off-by: Peter Hornyack peterhorny...@google.com
---
 Documentation/virtual/kvm/api.txt | 48 +++
 include/uapi/linux/kvm.h  | 14 
 2 files changed, 62 insertions(+)

diff --git a/Documentation/virtual/kvm/api.txt 
b/Documentation/virtual/kvm/api.txt
index a4ebcb712375..bda540b3dd03 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -3302,6 +3302,36 @@ Valid values for 'type' are:
to ignore the request, or to gather VM memory core dump and/or
reset/shutdown of the VM.
 
+   /* KVM_EXIT_MSR */
+   struct {
+#define KVM_EXIT_MSR_RDMSR 1
+#define KVM_EXIT_MSR_WRMSR 2
+#define KVM_EXIT_MSR_COMPLETION_FAILED 3
+   __u8 direction; /* out */
+#define KVM_EXIT_MSR_UNHANDLED 1
+#define KVM_EXIT_MSR_HANDLED   2
+   __u8 handled;   /* in */
+   __u32 index;/* i.e. ecx; out */
+   __u64 data; /* out (wrmsr) / in (rdmsr) */
+   } msr;
+
+If exit_reason is KVM_EXIT_MSR, then the vcpu has executed a rdmsr or wrmsr
+instruction which could not be satisfied by kvm. The msr struct is used for
+both output to and input from user space. direction indicates whether the
+instruction was rdmsr or wrmsr, and index is the target MSR number held in
+ecx. User space must not modify index. data holds the payload from a wrmsr or
+must be filled in with a payload on a rdmsr.
+
+On the return path into kvm, user space should set handled to
+KVM_EXIT_MSR_HANDLED if it successfully handled the MSR access; otherwise,
+handled should be set to KVM_EXIT_MSR_UNHANDLED, which will cause a general
+protection fault to be injected into the vcpu. If an error occurs during the
+return into kvm, the vcpu will not be run and another KVM_EXIT_MSR will be
+generated with direction KVM_EXIT_MSR_COMPLETION_FAILED.
+
+KVM_EXIT_MSR can only occur when KVM_CAP_UNHANDLED_MSR_EXITS has been enabled;
+a detailed description of this capability is below.
+
/* Fix the size of the union. */
char padding[256];
};
@@ -3620,6 +3650,24 @@ struct {
 
 KVM handlers should exit to userspace with rc = -EREMOTE.
 
+7.5 KVM_CAP_UNHANDLED_MSR_EXITS
+
+Architectures: x86 (vmx-only)
+Parameters: args[0] enables or disables unhandled MSR exits
+Returns: 0 on success; -1 on error
+
+This capability enables exits to user space on unhandled MSR accesses.
+
+When enabled (args[0] != 0), when the guest accesses an MSR that kvm does not
+handle kvm will exit to user space with the reason KVM_EXIT_MSR. When disabled
+(by default, or with args[0] == 0), when the guest accesses an MSR that kvm
+does not handle a GP fault is immediately injected into the guest.
+
+Currently only implemented for vmx; attempts to enable this capability on svm
+systems will return an error. Also, note that this capability is overridden if
+the kvm module's ignore_msrs flag is set, in which case unhandled MSR accesses
+are simply ignored and the guest is re-entered immediately.
+
 
 8. Other capabilities.
 --
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index 0d831f94f8a8..43d2d1e15ac4 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -183,6 +183,7 @@ struct kvm_s390_skeys {
 #define KVM_EXIT_EPR  23
 #define KVM_EXIT_SYSTEM_EVENT 24
 #define KVM_EXIT_S390_STSI25
+#define KVM_EXIT_MSR  26
 
 /* For KVM_EXIT_INTERNAL_ERROR */
 /* Emulate instruction failed. */
@@ -330,6 +331,18 @@ struct kvm_run {
__u8 sel1;
__u16 sel2;
} s390_stsi;
+   /* KVM_EXIT_MSR */
+   struct {
+#define KVM_EXIT_MSR_RDMSR 1
+#define KVM_EXIT_MSR_WRMSR 2
+#define KVM_EXIT_MSR_COMPLETION_FAILED 3
+   __u8 direction; /* out */
+#define KVM_EXIT_MSR_UNHANDLED 1
+#define KVM_EXIT_MSR_HANDLED   2
+   __u8 handled;   /* in */
+   __u32 index;/* i.e. ecx; out */
+   __u64 data; /* out (wrmsr) / in (rdmsr) */
+   } msr;
/* Fix the size of the union. */
char padding[256];
};
@@ -819,6 +832,7 @@ struct kvm_ppc_smmu_info {
 #define KVM_CAP_DISABLE_QUIRKS 116
 #define KVM_CAP_X86_SMM 117
 #define KVM_CAP_MULTI_ADDRESS_SPACE 118
+#define KVM_CAP_UNHANDLED_MSR_EXITS 119
 
 #ifdef KVM_CAP_IRQ_ROUTING
 
-- 
2.5.0.276.gf5e568e

--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org

[RFC PATCH 3/5] KVM: x86: add msr_exits_supported to kvm_x86_ops

[Peter Hornyack]

msr_exits_supported will be checked when user space attempts to enable
the KVM_CAP_UNHANDLED_MSR_EXITS capability for the vm. This is needed
because MSR exit support will be implemented for vmx but not svm later
in this patchset.

Signed-off-by: Peter Hornyack peterhorny...@google.com
---
 arch/x86/include/asm/kvm_host.h | 1 +
 arch/x86/kvm/svm.c  | 6 ++
 arch/x86/kvm/vmx.c  | 6 ++
 3 files changed, 13 insertions(+)

diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index c12e845f59e6..a6e145b1e271 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -854,6 +854,7 @@ struct kvm_x86_ops {
void (*handle_external_intr)(struct kvm_vcpu *vcpu);
bool (*mpx_supported)(void);
bool (*xsaves_supported)(void);
+   bool (*msr_exits_supported)(void);
 
int (*check_nested_events)(struct kvm_vcpu *vcpu, bool external_intr);
 
diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c
index 74d825716f4f..bcbb56f49b9f 100644
--- a/arch/x86/kvm/svm.c
+++ b/arch/x86/kvm/svm.c
@@ -4249,6 +4249,11 @@ static bool svm_xsaves_supported(void)
return false;
 }
 
+static bool svm_msr_exits_supported(void)
+{
+   return false;
+}
+
 static bool svm_has_wbinvd_exit(void)
 {
return true;
@@ -4540,6 +4545,7 @@ static struct kvm_x86_ops svm_x86_ops = {
.invpcid_supported = svm_invpcid_supported,
.mpx_supported = svm_mpx_supported,
.xsaves_supported = svm_xsaves_supported,
+   .msr_exits_supported = svm_msr_exits_supported,
 
.set_supported_cpuid = svm_set_supported_cpuid,
 
diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index acc38e27d221..27fec385d79d 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -8161,6 +8161,11 @@ static bool vmx_xsaves_supported(void)
SECONDARY_EXEC_XSAVES;
 }
 
+static bool vmx_msr_exits_supported(void)
+{
+   return false;
+}
+
 static void vmx_recover_nmi_blocking(struct vcpu_vmx *vmx)
 {
u32 exit_intr_info;
@@ -10413,6 +10418,7 @@ static struct kvm_x86_ops vmx_x86_ops = {
.handle_external_intr = vmx_handle_external_intr,
.mpx_supported = vmx_mpx_supported,
.xsaves_supported = vmx_xsaves_supported,
+   .msr_exits_supported = vmx_msr_exits_supported,
 
.check_nested_events = vmx_check_nested_events,
 
-- 
2.5.0.276.gf5e568e

--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[RFC PATCH 5/5] KVM: x86: add trace events for unhandled MSR exits

[Peter Hornyack]

Add trace_kvm_userspace_msr and call it when user space reenters kvm
after KVM_EXIT_MSR.

Add KVM_EXIT_MSR to kvm_trace_exit_reason list.

Signed-off-by: Peter Hornyack peterhorny...@google.com
---
 arch/x86/kvm/trace.h   | 28 
 arch/x86/kvm/vmx.c |  4 
 arch/x86/kvm/x86.c |  1 +
 include/trace/events/kvm.h |  2 +-
 4 files changed, 34 insertions(+), 1 deletion(-)

diff --git a/arch/x86/kvm/trace.h b/arch/x86/kvm/trace.h
index 4eae7c35ddf5..6d144d424896 100644
--- a/arch/x86/kvm/trace.h
+++ b/arch/x86/kvm/trace.h
@@ -330,6 +330,34 @@ TRACE_EVENT(kvm_msr,
 #define trace_kvm_msr_read_ex(ecx) trace_kvm_msr(0, ecx, 0, true)
 #define trace_kvm_msr_write_ex(ecx, data)  trace_kvm_msr(1, ecx, data, true)
 
+TRACE_EVENT(kvm_userspace_msr,
+   TP_PROTO(u8 direction, u8 handled, u32 index, u64 data),
+   TP_ARGS(direction, handled, index, data),
+
+   TP_STRUCT__entry(
+   __field(u8, direction)
+   __field(u8, handled)
+   __field(u32, index)
+   __field(u64, data)
+   ),
+
+   TP_fast_assign(
+   __entry-direction  = direction;
+   __entry-handled= handled;
+   __entry-index  = index;
+   __entry-data   = data;
+   ),
+
+   TP_printk(userspace %s %x = 0x%llx, %s,
+ __entry-direction == KVM_EXIT_MSR_RDMSR ? rdmsr :
+ __entry-direction == KVM_EXIT_MSR_WRMSR ? wrmsr :
+unknown!,
+ __entry-index, __entry-data,
+ __entry-handled == KVM_EXIT_MSR_UNHANDLED ? unhandled :
+ __entry-handled == KVM_EXIT_MSR_HANDLED   ? handled :
+  unknown!)
+);
+
 /*
  * Tracepoint for guest CR access.
  */
diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index ba26d382d785..46d276235f78 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -5500,6 +5500,10 @@ static int vmx_complete_userspace_msr(struct kvm_vcpu 
*vcpu)
 {
struct msr_data msr;
 
+   trace_kvm_userspace_msr(vcpu-run-msr.direction,
+   vcpu-run-msr.handled, vcpu-run-msr.index,
+   vcpu-run-msr.data);
+
if (vcpu-run-msr.index != vcpu-arch.regs[VCPU_REGS_RCX]) {
pr_debug(msr.index 0x%x changed, does not match ecx 0x%lx\n,
 vcpu-run-msr.index,
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 5c22f4655741..cc74ba1d01e6 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -8040,6 +8040,7 @@ EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_exit);
 EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_inj_virq);
 EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_page_fault);
 EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_msr);
+EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_userspace_msr);
 EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_cr);
 EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_nested_vmrun);
 EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_nested_vmexit);
diff --git a/include/trace/events/kvm.h b/include/trace/events/kvm.h
index a44062da684b..aa6ce656d658 100644
--- a/include/trace/events/kvm.h
+++ b/include/trace/events/kvm.h
@@ -14,7 +14,7 @@
ERSN(SHUTDOWN), ERSN(FAIL_ENTRY), ERSN(INTR), ERSN(SET_TPR),\
ERSN(TPR_ACCESS), ERSN(S390_SIEIC), ERSN(S390_RESET), ERSN(DCR),\
ERSN(NMI), ERSN(INTERNAL_ERROR), ERSN(OSI), ERSN(PAPR_HCALL),   \
-   ERSN(S390_UCONTROL), ERSN(WATCHDOG), ERSN(S390_TSCH)
+   ERSN(S390_UCONTROL), ERSN(WATCHDOG), ERSN(S390_TSCH), ERSN(MSR)
 
 TRACE_EVENT(kvm_userspace_exit,
TP_PROTO(__u32 reason, int errno),
-- 
2.5.0.276.gf5e568e

--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[RFC PATCH 1/5] KVM: x86: refactor vmx rdmsr/wrmsr completion into new functions

[Peter Hornyack]

After handling a rdmsr or wrmsr, refactor the success and failure code
paths into separate functions. This will allow us to also complete or
fail MSR accesses on the entry path from userspace into kvm.

Signed-off-by: Peter Hornyack peterhorny...@google.com
---
 arch/x86/kvm/vmx.c | 44 +---
 1 file changed, 33 insertions(+), 11 deletions(-)

diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index 37eae551857c..acc38e27d221 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -5463,6 +5463,34 @@ static int handle_cpuid(struct kvm_vcpu *vcpu)
return 1;
 }
 
+static void complete_rdmsr(struct kvm_vcpu *vcpu, const struct msr_data *msr)
+{
+   trace_kvm_msr_read(msr-index, msr-data);
+
+   /* FIXME: handling of bits 32:63 of rax, rdx */
+   vcpu-arch.regs[VCPU_REGS_RAX] = msr-data  -1u;
+   vcpu-arch.regs[VCPU_REGS_RDX] = (msr-data  32)  -1u;
+   skip_emulated_instruction(vcpu);
+}
+
+static void fail_rdmsr(struct kvm_vcpu *vcpu, const struct msr_data *msr)
+{
+   trace_kvm_msr_read_ex(msr-index);
+   kvm_inject_gp(vcpu, 0);
+}
+
+static void complete_wrmsr(struct kvm_vcpu *vcpu, const struct msr_data *msr)
+{
+   trace_kvm_msr_write(msr-index, msr-data);
+   skip_emulated_instruction(vcpu);
+}
+
+static void fail_wrmsr(struct kvm_vcpu *vcpu, const struct msr_data *msr)
+{
+   trace_kvm_msr_write_ex(msr-index, msr-data);
+   kvm_inject_gp(vcpu, 0);
+}
+
 static int handle_rdmsr(struct kvm_vcpu *vcpu)
 {
u32 ecx = vcpu-arch.regs[VCPU_REGS_RCX];
@@ -5471,17 +5499,12 @@ static int handle_rdmsr(struct kvm_vcpu *vcpu)
msr_info.index = ecx;
msr_info.host_initiated = false;
if (vmx_get_msr(vcpu, msr_info)) {
-   trace_kvm_msr_read_ex(ecx);
-   kvm_inject_gp(vcpu, 0);
+   fail_rdmsr(vcpu, msr_info);
return 1;
}
 
-   trace_kvm_msr_read(ecx, msr_info.data);
+   complete_rdmsr(vcpu, msr_info);
 
-   /* FIXME: handling of bits 32:63 of rax, rdx */
-   vcpu-arch.regs[VCPU_REGS_RAX] = msr_info.data  -1u;
-   vcpu-arch.regs[VCPU_REGS_RDX] = (msr_info.data  32)  -1u;
-   skip_emulated_instruction(vcpu);
return 1;
 }
 
@@ -5496,13 +5519,12 @@ static int handle_wrmsr(struct kvm_vcpu *vcpu)
msr.index = ecx;
msr.host_initiated = false;
if (kvm_set_msr(vcpu, msr) != 0) {
-   trace_kvm_msr_write_ex(ecx, data);
-   kvm_inject_gp(vcpu, 0);
+   fail_wrmsr(vcpu, msr);
return 1;
}
 
-   trace_kvm_msr_write(ecx, data);
-   skip_emulated_instruction(vcpu);
+   complete_wrmsr(vcpu, msr);
+
return 1;
 }
 
-- 
2.5.0.276.gf5e568e

--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[RFC PATCH 0/5] KVM: x86: exit to user space on unhandled MSR accesses

[Peter Hornyack]

There are numerous MSRs that kvm does not currently handle. On Intel
platforms we have observed guest VMs accessing some of these MSRs (for
example, MSR_PLATFORM_INFO) and behaving poorly (to the point of guest OS
crashes) when they receive a GP fault because the MSR is not emulated. This
patchset adds a new kvm exit path for unhandled MSR accesses that allows
user space to emulate additional MSRs without having to implement them in
kvm.

The core of the patchset modifies the vmx handle_rdmsr and handle_wrmsr
functions to exit to user space on MSR reads/writes that kvm can't handle
itself. Then, on the return path into kvm we check for outstanding user
space MSR completions and either complete the MSR access successfully or
inject a GP fault as kvm would do by default. This new exit path must be
enabled for the vm via the KVM_CAP_UNHANDLED_MSR_EXITS capability.

In the future we plan to extend this functionality to allow user space to
register the MSRs that it would like to handle itself, even if kvm already
provides an implementation. In the long-term we will move the
implementation of all non-performance-sensitive MSRs to user space,
reducing the potential attack surface of kvm and allowing us to respond to
bugs more quickly.

This patchset has been tested with our non-qemu user space hypervisor on
vmx platforms; svm support is not implemented.

Peter Hornyack (5):
  KVM: x86: refactor vmx rdmsr/wrmsr completion into new functions
  KVM: add KVM_EXIT_MSR exit reason and capability.
  KVM: x86: add msr_exits_supported to kvm_x86_ops
  KVM: x86: enable unhandled MSR exits for vmx
  KVM: x86: add trace events for unhandled MSR exits

 Documentation/virtual/kvm/api.txt |  48 +++
 arch/x86/include/asm/kvm_host.h   |   2 +
 arch/x86/kvm/svm.c|   6 ++
 arch/x86/kvm/trace.h  |  28 +
 arch/x86/kvm/vmx.c| 126 ++
 arch/x86/kvm/x86.c|  13 
 include/trace/events/kvm.h|   2 +-
 include/uapi/linux/kvm.h  |  14 +
 8 files changed, 227 insertions(+), 12 deletions(-)

-- 
2.5.0.276.gf5e568e

--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[RFC PATCH 4/5] KVM: x86: enable unhandled MSR exits for vmx

[Peter Hornyack]

Set the vm's unhandled_msr_exits flag when user space calls the
KVM_ENABLE_CAP ioctl with KVM_CAP_UNHANDLED_MSR_EXITS. After kvm fails
to handle a guest rdmsr or wrmsr, check this flag and exit to user space
with KVM_EXIT_MSR rather than immediately injecting a GP fault.

On reentry into kvm, use the complete_userspace_io callback path to call
vmx_complete_userspace_msr. Complete the MSR access if user space was
able to handle it successfully, or fail the MSR access and inject a GP
fault if user space could not handle the access.

Signed-off-by: Peter Hornyack peterhorny...@google.com
---
 arch/x86/include/asm/kvm_host.h |  1 +
 arch/x86/kvm/vmx.c  | 74 -
 arch/x86/kvm/x86.c  | 12 +++
 3 files changed, 86 insertions(+), 1 deletion(-)

diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index a6e145b1e271..1b06cea06a8e 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -682,6 +682,7 @@ struct kvm_arch {
u32 bsp_vcpu_id;
 
u64 disabled_quirks;
+   bool unhandled_msr_exits;
 };
 
 struct kvm_vm_stat {
diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index 27fec385d79d..ba26d382d785 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -32,6 +32,7 @@
 #include linux/slab.h
 #include linux/tboot.h
 #include linux/hrtimer.h
+#include uapi/linux/kvm.h
 #include kvm_cache_regs.h
 #include x86.h
 
@@ -5491,6 +5492,53 @@ static void fail_wrmsr(struct kvm_vcpu *vcpu, const 
struct msr_data *msr)
kvm_inject_gp(vcpu, 0);
 }
 
+/*
+ * On success, returns 1 so that __vcpu_run() will happen next. On error,
+ * returns 0.
+ */
+static int vmx_complete_userspace_msr(struct kvm_vcpu *vcpu)
+{
+   struct msr_data msr;
+
+   if (vcpu-run-msr.index != vcpu-arch.regs[VCPU_REGS_RCX]) {
+   pr_debug(msr.index 0x%x changed, does not match ecx 0x%lx\n,
+vcpu-run-msr.index,
+vcpu-arch.regs[VCPU_REGS_RCX]);
+   goto err_out;
+   }
+   msr.index = vcpu-run-msr.index;
+   msr.data = vcpu-run-msr.data;
+   msr.host_initiated = false;
+
+   switch (vcpu-run-msr.direction) {
+   case KVM_EXIT_MSR_RDMSR:
+   if (vcpu-run-msr.handled == KVM_EXIT_MSR_HANDLED)
+   complete_rdmsr(vcpu, msr);
+   else
+   fail_rdmsr(vcpu, msr);
+   break;
+   case KVM_EXIT_MSR_WRMSR:
+   if (vcpu-run-msr.handled == KVM_EXIT_MSR_HANDLED)
+   complete_wrmsr(vcpu, msr);
+   else
+   fail_wrmsr(vcpu, msr);
+   break;
+   default:
+   pr_debug(bad msr.direction %u\n, vcpu-run-msr.direction);
+   goto err_out;
+   }
+
+   return 1;
+err_out:
+   vcpu-run-exit_reason = KVM_EXIT_MSR;
+   vcpu-run-msr.direction = KVM_EXIT_MSR_COMPLETION_FAILED;
+   return 0;
+}
+
+/*
+ * Returns 1 if the rdmsr handling is complete; returns 0 if kvm should exit to
+ * user space to handle this rdmsr.
+ */
 static int handle_rdmsr(struct kvm_vcpu *vcpu)
 {
u32 ecx = vcpu-arch.regs[VCPU_REGS_RCX];
@@ -5499,6 +5547,16 @@ static int handle_rdmsr(struct kvm_vcpu *vcpu)
msr_info.index = ecx;
msr_info.host_initiated = false;
if (vmx_get_msr(vcpu, msr_info)) {
+   if (vcpu-kvm-arch.unhandled_msr_exits) {
+   vcpu-run-exit_reason = KVM_EXIT_MSR;
+   vcpu-run-msr.direction = KVM_EXIT_MSR_RDMSR;
+   vcpu-run-msr.index = msr_info.index;
+   vcpu-run-msr.data = 0;
+   vcpu-run-msr.handled = 0;
+   vcpu-arch.complete_userspace_io =
+   vmx_complete_userspace_msr;
+   return 0;
+   }
fail_rdmsr(vcpu, msr_info);
return 1;
}
@@ -5508,6 +5566,10 @@ static int handle_rdmsr(struct kvm_vcpu *vcpu)
return 1;
 }
 
+/*
+ * Returns 1 if the wrmsr handling is complete; returns 0 if kvm should exit to
+ * user space to handle this wrmsr.
+ */
 static int handle_wrmsr(struct kvm_vcpu *vcpu)
 {
struct msr_data msr;
@@ -5519,6 +5581,16 @@ static int handle_wrmsr(struct kvm_vcpu *vcpu)
msr.index = ecx;
msr.host_initiated = false;
if (kvm_set_msr(vcpu, msr) != 0) {
+   if (vcpu-kvm-arch.unhandled_msr_exits) {
+   vcpu-run-exit_reason = KVM_EXIT_MSR;
+   vcpu-run-msr.direction = KVM_EXIT_MSR_WRMSR;
+   vcpu-run-msr.index = msr.index;
+   vcpu-run-msr.data = msr.data;
+   vcpu-run-msr.handled = 0;
+   vcpu-arch.complete_userspace_io =
+   vmx_complete_userspace_msr

Re: [PATCH 3/9] kvm: add hyper-v crash msrs values

[Peter Hornyack]

If userspace is controlling the crash capabilities then
HV_X64_MSR_CRASH_CTL_CONTENTS is not needed.

On Wed, Jul 1, 2015 at 8:53 AM, Denis V. Lunev d...@openvz.org wrote:
 On 01/07/15 18:00, Paolo Bonzini wrote:


 On 30/06/2015 13:33, Denis V. Lunev wrote:

 +#define HV_X64_MSR_CRASH_CTL_NOTIFY(1ULL  63)
 +#define HV_X64_MSR_CRASH_CTL_CONTENTS  \
 +   (HV_X64_MSR_CRASH_CTL_NOTIFY)

 Why is HV_X64_MSR_CRASH_CTL_CONTENTS needed?  Can I just remove it?

 Paolo

 this was a direct request from Peter Hornyack peterhorny...@google.com

 I suggest here:

 #define HV_X64_MSR_CRASH_CTL_CONTENTS  \
 (HV_CRASH_CTL_CRASH_NOTIFY)

 To allow for more crash actions to be added in the future.

 Den
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 4/9] kvm/x86: added hyper-v crash msrs into kvm hyperv context

[Peter Hornyack]

On Tue, Jun 30, 2015 at 4:33 AM, Denis V. Lunev d...@openvz.org wrote:
 From: Andrey Smetanin asmeta...@virtuozzo.com

 Added kvm Hyper-V context hv crash variables as storage
 of Hyper-V crash msrs.

 Signed-off-by: Andrey Smetanin asmeta...@virtuozzo.com
 Signed-off-by: Denis V. Lunev d...@openvz.org
 CC: Paolo Bonzini pbonz...@redhat.com
 CC: Gleb Natapov g...@kernel.org
Reviewed-by: Peter Hornyack peterhorny...@google.com

 ---
  arch/x86/include/asm/kvm_host.h | 4 
  1 file changed, 4 insertions(+)

 diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
 index 78616aa..697c1f3 100644
 --- a/arch/x86/include/asm/kvm_host.h
 +++ b/arch/x86/include/asm/kvm_host.h
 @@ -595,6 +595,10 @@ struct kvm_hv {
 u64 hv_guest_os_id;
 u64 hv_hypercall;
 u64 hv_tsc_page;
 +
 +   /* Hyper-v based guest crash (NT kernel bugcheck) parameters */
 +   u64 hv_crash_param[HV_X64_MSR_CRASH_PARAMS];
 +   u64 hv_crash_ctl;
  };

  struct kvm_arch {
 --
 2.1.4

 --
 To unsubscribe from this list: send the line unsubscribe kvm in
 the body of a message to majord...@vger.kernel.org
 More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 3/9] kvm: add hyper-v crash msrs values

[Peter Hornyack]

On Tue, Jun 30, 2015 at 4:33 AM, Denis V. Lunev d...@openvz.org wrote:
 From: Andrey Smetanin asmeta...@virtuozzo.com

 Added Hyper-V crash msrs values - HV_X64_MSR_CRASH*.

 Signed-off-by: Andrey Smetanin asmeta...@virtuozzo.com
 Signed-off-by: Denis V. Lunev d...@openvz.org
 CC: Paolo Bonzini pbonz...@redhat.com
 CC: Gleb Natapov g...@kernel.org
Reviewed-by: Peter Hornyack peterhorny...@google.com

 ---
  arch/x86/include/uapi/asm/hyperv.h | 15 +++
  1 file changed, 15 insertions(+)

 diff --git a/arch/x86/include/uapi/asm/hyperv.h 
 b/arch/x86/include/uapi/asm/hyperv.h
 index ce6068d..3ed7a5c 100644
 --- a/arch/x86/include/uapi/asm/hyperv.h
 +++ b/arch/x86/include/uapi/asm/hyperv.h
 @@ -199,6 +199,21 @@
  #define HV_X64_MSR_STIMER3_CONFIG  0x40B6
  #define HV_X64_MSR_STIMER3_COUNT   0x40B7

 +/* Hyper-V guest crash notification MSR's */
 +#define HV_X64_MSR_CRASH_P00x4100
 +#define HV_X64_MSR_CRASH_P10x4101
 +#define HV_X64_MSR_CRASH_P20x4102
 +#define HV_X64_MSR_CRASH_P30x4103
 +#define HV_X64_MSR_CRASH_P40x4104
 +#define HV_X64_MSR_CRASH_CTL   0x4105
 +#define HV_X64_MSR_CRASH_CTL_NOTIFY(1ULL  63)
 +#define HV_X64_MSR_CRASH_CTL_CONTENTS  \

This is unused and can be deleted, right?

 +   (HV_X64_MSR_CRASH_CTL_NOTIFY)
 +
 +#define HV_X64_MSR_CRASH_PARAMS\
 +   (1 + (HV_X64_MSR_CRASH_P4 - HV_X64_MSR_CRASH_P0))
 +
 +
  #define HV_X64_MSR_HYPERCALL_ENABLE0x0001
  #define HV_X64_MSR_HYPERCALL_PAGE_ADDRESS_SHIFT12
  #define HV_X64_MSR_HYPERCALL_PAGE_ADDRESS_MASK \
 --
 2.1.4

 --
 To unsubscribe from this list: send the line unsubscribe kvm in
 the body of a message to majord...@vger.kernel.org
 More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 8/9] kvm/x86: add sending hyper-v crash notification to user space

[Peter Hornyack]

On Tue, Jun 30, 2015 at 4:33 AM, Denis V. Lunev d...@openvz.org wrote:
 From: Andrey Smetanin asmeta...@virtuozzo.com

 Sending of notification is done by exiting vcpu to user space
 if KVM_REQ_HV_CRASH is enabled for vcpu. kvm_run structure
 will contains system_event with type KVM_SYSTEM_EVENT_CRASH
 and flag KVM_SYSTEM_EVENT_FL_HV_CRASH to clarify that
 crash occures inside Hyper-V based guest.

 Signed-off-by: Andrey Smetanin asmeta...@virtuozzo.com
 Signed-off-by: Denis V. Lunev d...@openvz.org
 CC: Paolo Bonzini pbonz...@redhat.com
 CC: Gleb Natapov g...@kernel.org
Reviewed-by: Peter Hornyack peterhorny...@google.com

 ---
  arch/x86/kvm/x86.c   | 8 
  include/uapi/linux/kvm.h | 2 ++
  2 files changed, 10 insertions(+)

 diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
 index 47b7507..55a4b92 100644
 --- a/arch/x86/kvm/x86.c
 +++ b/arch/x86/kvm/x86.c
 @@ -6264,6 +6264,14 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)
 vcpu_scan_ioapic(vcpu);
 if (kvm_check_request(KVM_REQ_APIC_PAGE_RELOAD, vcpu))
 kvm_vcpu_reload_apic_access_page(vcpu);
 +   if (kvm_check_request(KVM_REQ_HV_CRASH, vcpu)) {
 +   vcpu-run-exit_reason = KVM_EXIT_SYSTEM_EVENT;
 +   vcpu-run-system_event.type = KVM_SYSTEM_EVENT_CRASH;
 +   vcpu-run-system_event.flags =
 +   KVM_SYSTEM_EVENT_FL_HV_CRASH;
 +   r = 0;
 +   goto out;
 +   }
 }

 if (kvm_check_request(KVM_REQ_EVENT, vcpu) || req_int_win) {
 diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
 index 716ad4a..eefb8b9 100644
 --- a/include/uapi/linux/kvm.h
 +++ b/include/uapi/linux/kvm.h
 @@ -317,6 +317,8 @@ struct kvm_run {
 struct {
  #define KVM_SYSTEM_EVENT_SHUTDOWN   1
  #define KVM_SYSTEM_EVENT_RESET  2
 +#define KVM_SYSTEM_EVENT_CRASH  3
 +#define KVM_SYSTEM_EVENT_FL_HV_CRASH(1ULL  0)
 __u32 type;
 __u64 flags;
 } system_event;
 --
 2.1.4

 --
 To unsubscribe from this list: send the line unsubscribe kvm in
 the body of a message to majord...@vger.kernel.org
 More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 7/9] kvm/x86: added hyper-v crash data and ctl msr's get/set'ers

[Peter Hornyack]

On Tue, Jun 30, 2015 at 4:33 AM, Denis V. Lunev d...@openvz.org wrote:
 From: Andrey Smetanin asmeta...@virtuozzo.com

 Added hyper-v crash msr's(HV_X64_MSR_CRASH*) data and control
 geters and setters.

 User space allowed to setup Hyper-V crash ctl msr.
 This msr should be setup to HV_X64_MSR_CRASH_CTL_NOTIFY
 value so Hyper-V guest knows it can send crash data to host.
 But Hyper-V guest notifies about crash event by writing
 the same HV_X64_MSR_CRASH_CTL_NOTIFY value into crash ctl msr.
 So both user space and guest writes inside ctl msr the same value
 and this patch distingiush the moment of actual guest crash
 by checking host initiated value from msr info. Also patch
 prevents modification of crash ctl msr by guest.

 Signed-off-by: Andrey Smetanin asmeta...@virtuozzo.com
 Signed-off-by: Denis V. Lunev d...@openvz.org
 CC: Paolo Bonzini pbonz...@redhat.com
 CC: Gleb Natapov g...@kernel.org
Reviewed-by: Peter Hornyack peterhorny...@google.com

 ---
  arch/x86/kvm/hyperv.c | 74 
 ---
  arch/x86/kvm/hyperv.h |  2 +-
  arch/x86/kvm/x86.c|  7 -
  3 files changed, 78 insertions(+), 5 deletions(-)

 diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c
 index af83c96..a860165 100644
 --- a/arch/x86/kvm/hyperv.c
 +++ b/arch/x86/kvm/hyperv.c
 @@ -48,7 +48,63 @@ static bool kvm_hv_msr_partition_wide(u32 msr)
 return r;
  }

 -static int kvm_hv_set_msr_pw(struct kvm_vcpu *vcpu, u32 msr, u64 data)
 +static int kvm_hv_msr_get_crash_data(struct kvm_vcpu *vcpu,
 +u32 index, u64 *pdata)
 +{
 +   struct kvm_hv *hv = vcpu-kvm-arch.hyperv;
 +
 +   if (WARN_ON_ONCE(index = ARRAY_SIZE(hv-hv_crash_param)))
 +   return -EINVAL;
 +
 +   *pdata = hv-hv_crash_param[index];
 +   return 0;
 +}
 +
 +static int kvm_hv_msr_get_crash_ctl(struct kvm_vcpu *vcpu, u64 *pdata)
 +{
 +   struct kvm_hv *hv = vcpu-kvm-arch.hyperv;
 +
 +   *pdata = hv-hv_crash_ctl;
 +   return 0;
 +}
 +
 +static int kvm_hv_msr_set_crash_ctl(struct kvm_vcpu *vcpu, u64 data, bool 
 host)
 +{
 +   struct kvm_hv *hv = vcpu-kvm-arch.hyperv;
 +
 +   if (host)
 +   hv-hv_crash_ctl = data;
 +
 +   if ((data  HV_X64_MSR_CRASH_CTL_NOTIFY)  !host) {
 +
 +   vcpu_debug(vcpu, hv crash (0x%llx 0x%llx 0x%llx 0x%llx 
 0x%llx)\n,
 + hv-hv_crash_param[0],
 + hv-hv_crash_param[1],
 + hv-hv_crash_param[2],
 + hv-hv_crash_param[3],
 + hv-hv_crash_param[4]);
 +
 +   /* Send notification about crash to user space */
 +   kvm_make_request(KVM_REQ_HV_CRASH, vcpu);
 +   }
 +
 +   return 0;
 +}
 +
 +static int kvm_hv_msr_set_crash_data(struct kvm_vcpu *vcpu,
 +u32 index, u64 data)
 +{
 +   struct kvm_hv *hv = vcpu-kvm-arch.hyperv;
 +
 +   if (WARN_ON_ONCE(index = ARRAY_SIZE(hv-hv_crash_param)))
 +   return -EINVAL;
 +
 +   hv-hv_crash_param[index] = data;
 +   return 0;
 +}
 +
 +static int kvm_hv_set_msr_pw(struct kvm_vcpu *vcpu, u32 msr, u64 data,
 +bool host)
  {
 struct kvm *kvm = vcpu-kvm;
 struct kvm_hv *hv = kvm-arch.hyperv;
 @@ -101,6 +157,12 @@ static int kvm_hv_set_msr_pw(struct kvm_vcpu *vcpu, u32 
 msr, u64 data)
 mark_page_dirty(kvm, gfn);
 break;
 }
 +   case HV_X64_MSR_CRASH_P0 ... HV_X64_MSR_CRASH_P4:
 +   return kvm_hv_msr_set_crash_data(vcpu,
 +msr - HV_X64_MSR_CRASH_P0,
 +data);
 +   case HV_X64_MSR_CRASH_CTL:
 +   return kvm_hv_msr_set_crash_ctl(vcpu, data, host);
 default:
 vcpu_unimpl(vcpu, Hyper-V uhandled wrmsr: 0x%x data 
 0x%llx\n,
 msr, data);
 @@ -173,6 +235,12 @@ static int kvm_hv_get_msr_pw(struct kvm_vcpu *vcpu, u32 
 msr, u64 *pdata)
 case HV_X64_MSR_REFERENCE_TSC:
 data = hv-hv_tsc_page;
 break;
 +   case HV_X64_MSR_CRASH_P0 ... HV_X64_MSR_CRASH_P4:
 +   return kvm_hv_msr_get_crash_data(vcpu,
 +msr - HV_X64_MSR_CRASH_P0,
 +pdata);
 +   case HV_X64_MSR_CRASH_CTL:
 +   return kvm_hv_msr_get_crash_ctl(vcpu, pdata);
 default:
 vcpu_unimpl(vcpu, Hyper-V unhandled rdmsr: 0x%x\n, msr);
 return 1;
 @@ -217,13 +285,13 @@ static int kvm_hv_get_msr(struct kvm_vcpu *vcpu, u32 
 msr, u64 *pdata)
 return 0;
  }

 -int kvm_hv_set_msr_common(struct kvm_vcpu *vcpu, u32 msr, u64 data)
 +int kvm_hv_set_msr_common(struct kvm_vcpu *vcpu, u32 msr, u64 data, bool 
 host)
  {
 if (kvm_hv_msr_partition_wide

Re: [PATCH 5/9] kvm: added KVM_REQ_HV_CRASH value to notify qemu about hyper-v crash

[Peter Hornyack]

On Tue, Jun 30, 2015 at 4:33 AM, Denis V. Lunev d...@openvz.org wrote:
 From: Andrey Smetanin asmeta...@virtuozzo.com

 Added KVM_REQ_HV_CRASH - vcpu request used for notify user space(QEMU)
 about Hyper-V crash.

 Signed-off-by: Andrey Smetanin asmeta...@virtuozzo.com
 Signed-off-by: Denis V. Lunev d...@openvz.org
 CC: Paolo Bonzini pbonz...@redhat.com
 CC: Gleb Natapov g...@kernel.org
Reviewed-by: Peter Hornyack peterhorny...@google.com

 ---
  include/linux/kvm_host.h | 1 +
  1 file changed, 1 insertion(+)

 diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
 index 2b2edf1..a377e00 100644
 --- a/include/linux/kvm_host.h
 +++ b/include/linux/kvm_host.h
 @@ -139,6 +139,7 @@ static inline bool is_error_page(struct page *page)
  #define KVM_REQ_DISABLE_IBS   24
  #define KVM_REQ_APIC_PAGE_RELOAD  25
  #define KVM_REQ_SMI   26
 +#define KVM_REQ_HV_CRASH  27

  #define KVM_USERSPACE_IRQ_SOURCE_ID0
  #define KVM_IRQFD_RESAMPLE_IRQ_SOURCE_ID   1
 --
 2.1.4

 --
 To unsubscribe from this list: send the line unsubscribe kvm in
 the body of a message to majord...@vger.kernel.org
 More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 6/9] kvm/x86: mark hyper-v crash msrs as partition wide

[Peter Hornyack]

On Tue, Jun 30, 2015 at 4:33 AM, Denis V. Lunev d...@openvz.org wrote:
 From: Andrey Smetanin asmeta...@virtuozzo.com

 Hyper-V crash msr's are per vm, aren't per vcpu, so mark them
 as partition wide.

 Signed-off-by: Andrey Smetanin asmeta...@virtuozzo.com
 Signed-off-by: Denis V. Lunev d...@openvz.org
 CC: Paolo Bonzini pbonz...@redhat.com
 CC: Gleb Natapov g...@kernel.org
Reviewed-by: Peter Hornyack peterhorny...@google.com

 ---
  arch/x86/kvm/hyperv.c | 2 ++
  1 file changed, 2 insertions(+)

 diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c
 index 2b49f10..af83c96 100644
 --- a/arch/x86/kvm/hyperv.c
 +++ b/arch/x86/kvm/hyperv.c
 @@ -39,6 +39,8 @@ static bool kvm_hv_msr_partition_wide(u32 msr)
 case HV_X64_MSR_HYPERCALL:
 case HV_X64_MSR_REFERENCE_TSC:
 case HV_X64_MSR_TIME_REF_COUNT:
 +   case HV_X64_MSR_CRASH_CTL:
 +   case HV_X64_MSR_CRASH_P0 ... HV_X64_MSR_CRASH_P4:
 r = true;
 break;
 }
 --
 2.1.4

 --
 To unsubscribe from this list: send the line unsubscribe kvm in
 the body of a message to majord...@vger.kernel.org
 More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 1/9] kvm/x86: move Hyper-V MSR's/hypercall code into hyperv.c file

[Peter Hornyack]

On Tue, Jun 30, 2015 at 4:33 AM, Denis V. Lunev d...@openvz.org wrote:
 From: Andrey Smetanin asmeta...@virtuozzo.com

 This patch introduce Hyper-V related source code file - hyperv.c and
 per vm and per vcpu hyperv context structures.
 All Hyper-V MSR's and hypercall code moved into hyperv.c.
 All Hyper-V kvm/vcpu fields moved into appropriate hyperv context
 structures. Copyrights and authors information copied from x86.c
 to hyperv.c.

 Signed-off-by: Andrey Smetanin asmeta...@virtuozzo.com
 Signed-off-by: Denis V. Lunev d...@openvz.org
 CC: Paolo Bonzini pbonz...@redhat.com
 CC: Gleb Natapov g...@kernel.org
Reviewed-by: Peter Hornyack peterhorny...@google.com

 ---
  arch/x86/include/asm/kvm_host.h |  20 ++-
  arch/x86/kvm/Makefile   |   4 +-
  arch/x86/kvm/hyperv.c   | 307 
 
  arch/x86/kvm/hyperv.h   |  32 +
  arch/x86/kvm/lapic.h|   2 +-
  arch/x86/kvm/x86.c  | 265 +-
  arch/x86/kvm/x86.h  |   5 +
  7 files changed, 366 insertions(+), 269 deletions(-)
  create mode 100644 arch/x86/kvm/hyperv.c
  create mode 100644 arch/x86/kvm/hyperv.h

 diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
 index c7fa57b..78616aa 100644
 --- a/arch/x86/include/asm/kvm_host.h
 +++ b/arch/x86/include/asm/kvm_host.h
 @@ -358,6 +358,11 @@ struct kvm_mtrr {
 struct list_head head;
  };

 +/* Hyper-V per vcpu emulation context */
 +struct kvm_vcpu_hv {
 +   u64 hv_vapic;
 +};
 +
  struct kvm_vcpu_arch {
 /*
  * rip and regs accesses must go through
 @@ -514,8 +519,7 @@ struct kvm_vcpu_arch {
 /* used for guest single stepping over the given code position */
 unsigned long singlestep_rip;

 -   /* fields used by HYPER-V emulation */
 -   u64 hv_vapic;
 +   struct kvm_vcpu_hv hyperv;

 cpumask_var_t wbinvd_dirty_mask;

 @@ -586,6 +590,13 @@ struct kvm_apic_map {
 struct kvm_lapic *logical_map[16][16];
  };

 +/* Hyper-V emulation context */
 +struct kvm_hv {
 +   u64 hv_guest_os_id;
 +   u64 hv_hypercall;
 +   u64 hv_tsc_page;
 +};
 +
  struct kvm_arch {
 unsigned int n_used_mmu_pages;
 unsigned int n_requested_mmu_pages;
 @@ -643,10 +654,7 @@ struct kvm_arch {
 /* reads protected by irq_srcu, writes by irq_lock */
 struct hlist_head mask_notifier_list;

 -   /* fields used by HYPER-V emulation */
 -   u64 hv_guest_os_id;
 -   u64 hv_hypercall;
 -   u64 hv_tsc_page;
 +   struct kvm_hv hyperv;

 #ifdef CONFIG_KVM_MMU_AUDIT
 int audit_point;
 diff --git a/arch/x86/kvm/Makefile b/arch/x86/kvm/Makefile
 index 67d215c..a1ff508 100644
 --- a/arch/x86/kvm/Makefile
 +++ b/arch/x86/kvm/Makefile
 @@ -12,7 +12,9 @@ kvm-y += $(KVM)/kvm_main.o 
 $(KVM)/coalesced_mmio.o \
  kvm-$(CONFIG_KVM_ASYNC_PF) += $(KVM)/async_pf.o

  kvm-y  += x86.o mmu.o emulate.o i8259.o irq.o lapic.o \
 -  i8254.o ioapic.o irq_comm.o cpuid.o pmu.o mtrr.o
 +  i8254.o ioapic.o irq_comm.o cpuid.o pmu.o mtrr.o \
 +  hyperv.o
 +
  kvm-$(CONFIG_KVM_DEVICE_ASSIGNMENT)+= assigned-dev.o iommu.o
  kvm-intel-y+= vmx.o pmu_intel.o
  kvm-amd-y  += svm.o pmu_amd.o
 diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c
 new file mode 100644
 index 000..2b49f10
 --- /dev/null
 +++ b/arch/x86/kvm/hyperv.c
 @@ -0,0 +1,307 @@
 +/*
 + * KVM Microsoft Hyper-V emulation
 + *
 + * derived from arch/x86/kvm/x86.c
 + *
 + * Copyright (C) 2006 Qumranet, Inc.
 + * Copyright (C) 2008 Qumranet, Inc.
 + * Copyright IBM Corporation, 2008
 + * Copyright 2010 Red Hat, Inc. and/or its affiliates.
 + * Copyright (C) 2015 Andrey Smetanin asmeta...@virtuozzo.com
 + *
 + * Authors:
 + *   Avi Kivity   a...@qumranet.com
 + *   Yaniv Kamay  ya...@qumranet.com
 + *   Amit Shahamit.s...@qumranet.com
 + *   Ben-Ami Yassour ben...@il.ibm.com
 + *   Andrey Smetanin asmeta...@virtuozzo.com
 + *
 + * This work is licensed under the terms of the GNU GPL, version 2.  See
 + * the COPYING file in the top-level directory.
 + *
 + */
 +
 +#include x86.h
 +#include lapic.h
 +#include hyperv.h
 +
 +#include linux/kvm_host.h
 +#include trace/events/kvm.h
 +
 +#include trace.h
 +
 +static bool kvm_hv_msr_partition_wide(u32 msr)
 +{
 +   bool r = false;
 +
 +   switch (msr) {
 +   case HV_X64_MSR_GUEST_OS_ID:
 +   case HV_X64_MSR_HYPERCALL:
 +   case HV_X64_MSR_REFERENCE_TSC:
 +   case HV_X64_MSR_TIME_REF_COUNT:
 +   r = true;
 +   break;
 +   }
 +
 +   return r;
 +}
 +
 +static int kvm_hv_set_msr_pw(struct kvm_vcpu *vcpu, u32 msr, u64 data)
 +{
 +   struct kvm *kvm = vcpu-kvm;
 +   struct kvm_hv *hv = kvm-arch.hyperv;
 +
 +   switch (msr) {
 +   case HV_X64_MSR_GUEST_OS_ID

Re: [PATCH 2/9] kvm: introduce vcpu_debug = kvm_debug + vcpu context

[Peter Hornyack]

On Tue, Jun 30, 2015 at 4:33 AM, Denis V. Lunev d...@openvz.org wrote:
 From: Andrey Smetanin asmeta...@virtuozzo.com

 vcpu_debug is useful macro like kvm_debug but additionally
 includes vcpu context inside output.

 Signed-off-by: Andrey Smetanin asmeta...@virtuozzo.com
 Signed-off-by: Denis V. Lunev d...@openvz.org
 CC: Paolo Bonzini pbonz...@redhat.com
 CC: Gleb Natapov g...@kernel.org
Reviewed-by: Peter Hornyack peterhorny...@google.com

 ---
  include/linux/kvm_host.h | 3 +++
  1 file changed, 3 insertions(+)

 diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
 index 9564fd7..2b2edf1 100644
 --- a/include/linux/kvm_host.h
 +++ b/include/linux/kvm_host.h
 @@ -424,6 +424,9 @@ struct kvm {
  #define vcpu_unimpl(vcpu, fmt, ...)\
 kvm_pr_unimpl(vcpu%i  fmt, (vcpu)-vcpu_id, ## __VA_ARGS__)

 +#define vcpu_debug(vcpu, fmt, ...) \
 +   kvm_debug(vcpu%i  fmt, (vcpu)-vcpu_id, ## __VA_ARGS__)
 +
  static inline struct kvm_vcpu *kvm_get_vcpu(struct kvm *kvm, int i)
  {
 smp_rmb();
 --
 2.1.4

 --
 To unsubscribe from this list: send the line unsubscribe kvm in
 the body of a message to majord...@vger.kernel.org
 More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line unsubscribe kvm in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 09/11] kvm/x86: distingiush hyper-v guest crash notification

[Peter Hornyack]

On Mon, Jun 22, 2015 at 9:05 AM, Denis V. Lunev d...@openvz.org wrote:
 From: Andrey Smetanin asmeta...@virtuozzo.com

 Previous patches allowes userspace to setup Hyper-V crash ctl msr.
 This msr should expose HV_X64_MSR_CRASH_CTL_NOTIFY value to Hyper-V
 guest to allow to send crash data. Unfortunately Hyper-V guest notifies
 hardware about crash by writing the same HV_X64_MSR_CRASH_CTL_NOTIFY value
 into crash ctl msr. Thus both user space and guest writes inside ctl msr the
 same value and this patch distingiush the moment of actual guest crash
 by checking host initiated value from msr info.

 Signed-off-by: Andrey Smetanin asmeta...@virtuozzo.com
 Signed-off-by: Denis V. Lunev d...@openvz.org
 CC: Paolo Bonzini pbonz...@redhat.com
 CC: Gleb Natapov g...@kernel.org
 ---
  arch/x86/kvm/hyperv.c | 17 +
  arch/x86/kvm/hyperv.h |  2 +-
  arch/x86/kvm/x86.c|  3 ++-
  3 files changed, 12 insertions(+), 10 deletions(-)

 diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c
 index 6b18015..f49502a 100644
 --- a/arch/x86/kvm/hyperv.c
 +++ b/arch/x86/kvm/hyperv.c
 @@ -54,12 +54,12 @@ static int kvm_hv_msr_get_crash_ctl(struct kvm_vcpu 
 *vcpu, u64 *pdata)
 return 0;
  }

 -static int kvm_hv_msr_set_crash_ctl(struct kvm_vcpu *vcpu, u64 data)
 +static int kvm_hv_msr_set_crash_ctl(struct kvm_vcpu *vcpu, u64 data, bool 
 host)
  {
 struct kvm_arch_hyperv *hv = vcpu-kvm-arch.hyperv;

 hv-hv_crash_ctl = data;

Should we allow hv_crash_ctl to be set if !host? It's a small detail,
but it doesn't seem like the guest should be able to disable crash
actions that userspace has enabled in hv_crash_ctl.

 -   if ((data  HV_X64_MSR_CRASH_CTL_NOTIFY)) {
 +   if ((data  HV_X64_MSR_CRASH_CTL_NOTIFY)  !host) {
 vcpu_debug(vcpu, hv crash (0x%llx 0x%llx 0x%llx 0x%llx 
   0x%llx)\n, hv-hv_crash_param[0],
   hv-hv_crash_param[1],
 @@ -99,7 +99,8 @@ static int kvm_hv_msr_get_crash_data(struct kvm_vcpu *vcpu,
 return 0;
  }

 -static int kvm_hv_set_msr_pw(struct kvm_vcpu *vcpu, u32 msr, u64 data)
 +static int kvm_hv_set_msr_pw(struct kvm_vcpu *vcpu,
 +u32 msr, u64 data, bool host)
  {
 struct kvm *kvm = vcpu-kvm;
 struct kvm_arch_hyperv *hv = kvm-arch.hyperv;
 @@ -156,7 +157,7 @@ static int kvm_hv_set_msr_pw(struct kvm_vcpu *vcpu, u32 
 msr, u64 data)
  msr - HV_X64_MSR_CRASH_P0,
  data);
 case HV_X64_MSR_CRASH_CTL:
 -   return kvm_hv_msr_set_crash_ctl(vcpu, data);
 +   return kvm_hv_msr_set_crash_ctl(vcpu, data, host);
 default:
 vcpu_unimpl(vcpu, Hyper-V unimpl wrmsr: 0x%x data 0x%llx\n,
   msr, data);
 @@ -165,7 +166,7 @@ static int kvm_hv_set_msr_pw(struct kvm_vcpu *vcpu, u32 
 msr, u64 data)
 return 0;
  }

 -static int kvm_hv_set_msr(struct kvm_vcpu *vcpu, u32 msr, u64 data)
 +static int kvm_hv_set_msr(struct kvm_vcpu *vcpu, u32 msr, u64 data, bool 
 host)
  {
 struct kvm_vcpu_arch_hyperv *hv = vcpu-arch.hyperv;

 @@ -278,17 +279,17 @@ static int kvm_hv_get_msr(struct kvm_vcpu *vcpu, u32 
 msr, u64 *pdata)
 return 0;
  }

 -int kvm_hv_set_msr_common(struct kvm_vcpu *vcpu, u32 msr, u64 data)
 +int kvm_hv_set_msr_common(struct kvm_vcpu *vcpu, u32 msr, u64 data, bool 
 host)
  {
 if (kvm_hv_msr_partition_wide(msr)) {
 int r;

 mutex_lock(vcpu-kvm-lock);
 -   r = kvm_hv_set_msr_pw(vcpu, msr, data);
 +   r = kvm_hv_set_msr_pw(vcpu, msr, data, host);
 mutex_unlock(vcpu-kvm-lock);
 return r;
 } else
 -   return kvm_hv_set_msr(vcpu, msr, data);
 +   return kvm_hv_set_msr(vcpu, msr, data, host);
  }

  int kvm_hv_get_msr_common(struct kvm_vcpu *vcpu, u32 msr, u64 *pdata)
 diff --git a/arch/x86/kvm/hyperv.h b/arch/x86/kvm/hyperv.h
 index 39aee93..dc49527 100644
 --- a/arch/x86/kvm/hyperv.h
 +++ b/arch/x86/kvm/hyperv.h
 @@ -22,7 +22,7 @@
  #ifndef __ARCH_X86_KVM_HYPERV_H__
  #define __ARCH_X86_KVM_HYPERV_H__

 -int kvm_hv_set_msr_common(struct kvm_vcpu *vcpu, u32 msr, u64 data);
 +int kvm_hv_set_msr_common(struct kvm_vcpu *vcpu, u32 msr, u64 data, bool 
 host);
  int kvm_hv_get_msr_common(struct kvm_vcpu *vcpu, u32 msr, u64 *pdata);

  bool kvm_hv_hypercall_enabled(struct kvm *kvm);
 diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
 index 111fa83..db4eecb 100644
 --- a/arch/x86/kvm/x86.c
 +++ b/arch/x86/kvm/x86.c
 @@ -2210,7 +2210,8 @@ int kvm_set_msr_common(struct kvm_vcpu *vcpu, struct 
 msr_data *msr_info)
 case HV_X64_MSR_GUEST_OS_ID ... HV_X64_MSR_SINT15:
 case HV_X64_MSR_CRASH_P0 ... HV_X64_MSR_CRASH_P4:
 case HV_X64_MSR_CRASH_CTL:
 -   return kvm_hv_set_msr_common(vcpu, msr, data);
 +  

Re: [PATCH 07/11] kvm/x86: added hyper-v crash data and ctl msr's get/set'ers

[Peter Hornyack]

On Mon, Jun 22, 2015 at 9:05 AM, Denis V. Lunev d...@openvz.org wrote:
 From: Andrey Smetanin asmeta...@virtuozzo.com

 Added hyper-v crash msr's(HV_X64_MSR_CRASH*) data and control
 geters and setters.

 Signed-off-by: Andrey Smetanin asmeta...@virtuozzo.com
 Signed-off-by: Denis V. Lunev d...@openvz.org
 CC: Paolo Bonzini pbonz...@redhat.com
 CC: Gleb Natapov g...@kernel.org
 ---
  arch/x86/kvm/hyperv.c | 66 
 +++
  arch/x86/kvm/x86.c|  4 
  2 files changed, 70 insertions(+)

 diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c
 index f65fb622..0a7d373 100644
 --- a/arch/x86/kvm/hyperv.c
 +++ b/arch/x86/kvm/hyperv.c
 @@ -46,6 +46,59 @@ static bool kvm_hv_msr_partition_wide(u32 msr)
 return r;
  }

 +static int kvm_hv_msr_get_crash_ctl(struct kvm_vcpu *vcpu, u64 *pdata)
 +{
 +   struct kvm_arch_hyperv *hv = vcpu-kvm-arch.hyperv;
 +
 +   *pdata = hv-hv_crash_ctl;

I see that this is implemented so that userspace controls the Hyper-V
crash capabilities that are enabled - userspace must set
HV_X64_MSR_CRASH_CTL_NOTIFY before the guest reads
HV_X64_MSR_CRASH_CTL. I just want to check that you considered an
alternative: a simpler implementation would have the crash
capabilities statically defined by kvm, and
HV_X64_MSR_CRASH_CTL_CONTENTS could just be returned here (and
hv_crash_ctl could be removed from struct kvm_arch_hyperv).

The current implementation is potentially more flexible but makes the
MSR handling a little more awkward since the host_initiated bool needs
to be passed around (patch 09). I guess either approach seems ok to
me.

Also, if this patchset is used then it looks like
HV_X64_MSR_CRASH_CTL_CONTENTS can be removed.

 +   return 0;
 +}
 +
 +static int kvm_hv_msr_set_crash_ctl(struct kvm_vcpu *vcpu, u64 data)
 +{
 +   struct kvm_arch_hyperv *hv = vcpu-kvm-arch.hyperv;
 +
 +   hv-hv_crash_ctl = data;
 +   if ((data  HV_X64_MSR_CRASH_CTL_NOTIFY)) {
 +   vcpu_debug(vcpu, hv crash (0x%llx 0x%llx 0x%llx 0x%llx 
 + 0x%llx)\n, hv-hv_crash_param[0],
 + hv-hv_crash_param[1],
 + hv-hv_crash_param[2],
 + hv-hv_crash_param[3],
 + hv-hv_crash_param[4]);
 +
 +   /* Send notification about crash to user space */
 +   kvm_make_request(KVM_REQ_HV_CRASH, vcpu);
 +   return 0;

Returning from here seems unnecessary - if more crash capabilities are
added in the future, the guest might want to invoke multiple
capabilities at once, so we'd want to check for those here too.

 +   }
 +
 +   return 0;
 +}
 +
 +static int kvm_hv_msr_set_crash_data(struct kvm_vcpu *vcpu,
 +u32 index, u64 data)
 +{
 +   struct kvm_arch_hyperv *hv = vcpu-kvm-arch.hyperv;
 +
 +   if (WARN_ON_ONCE(index = ARRAY_SIZE(hv-hv_crash_param)))
 +   return -EINVAL;
 +
 +   hv-hv_crash_param[index] = data;
 +   return 0;
 +}
 +
 +static int kvm_hv_msr_get_crash_data(struct kvm_vcpu *vcpu,
 +u32 index, u64 *pdata)
 +{
 +   struct kvm_arch_hyperv *hv = vcpu-kvm-arch.hyperv;
 +
 +   if (WARN_ON_ONCE(index = ARRAY_SIZE(hv-hv_crash_param)))
 +   return -EINVAL;
 +
 +   *pdata = hv-hv_crash_param[index];
 +   return 0;
 +}
 +
  static int kvm_hv_set_msr_pw(struct kvm_vcpu *vcpu, u32 msr, u64 data)
  {
 struct kvm *kvm = vcpu-kvm;
 @@ -98,6 +152,12 @@ static int kvm_hv_set_msr_pw(struct kvm_vcpu *vcpu, u32 
 msr, u64 data)
 mark_page_dirty(kvm, gfn);
 break;
 }
 +   case HV_X64_MSR_CRASH_P0 ... HV_X64_MSR_CRASH_P4:
 +   return kvm_hv_msr_set_crash_data(vcpu,
 +msr - HV_X64_MSR_CRASH_P0,
 +data);
 +   case HV_X64_MSR_CRASH_CTL:
 +   return kvm_hv_msr_set_crash_ctl(vcpu, data);
 default:
 vcpu_unimpl(vcpu, Hyper-V unimpl wrmsr: 0x%x data 0x%llx\n,
   msr, data);
 @@ -170,6 +230,12 @@ static int kvm_hv_get_msr_pw(struct kvm_vcpu *vcpu, u32 
 msr, u64 *pdata)
 case HV_X64_MSR_REFERENCE_TSC:
 data = hv-hv_tsc_page;
 break;
 +   case HV_X64_MSR_CRASH_P0 ... HV_X64_MSR_CRASH_P4:
 +   return kvm_hv_msr_get_crash_data(vcpu,
 +msr - HV_X64_MSR_CRASH_P0,
 +pdata);
 +   case HV_X64_MSR_CRASH_CTL:
 +   return kvm_hv_msr_get_crash_ctl(vcpu, pdata);
 default:
 vcpu_unimpl(vcpu, Hyper-V unhandled rdmsr: 0x%x\n, msr);
 return 1;
 diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
 index 2755c37..2046b78 100644
 --- a/arch/x86/kvm/x86.c
 +++ 

Re: [PATCH 1/2] kvm/x86: Hyper-V based guest crash data handling

[Peter Hornyack]

Hi Denis, Andrey, I have a few comments and questions. (re-sending in
plain-text mode, apologies for sending twice.)

On Thu, Jun 11, 2015 at 6:18 AM, Denis V. Lunev d...@openvz.org wrote:
 From: Andrey Smetanin asmeta...@virtuozzo.com

 Windows 2012 guests can notify hypervisor about occurred guest crash
 (Windows bugcheck(BSOD)) by writing specific Hyper-V msrs. This patch does
 handling of this MSR's by KVM and sending notification to user space that
 allows to gather Windows guest crash dump by QEMU/LIBVIRT.

 The idea is to provide functionality equal to pvpanic device without
 QEMU guest agent for Windows.

 The idea is borrowed from Linux HyperV bus driver and validated against
 Windows 2k12.

 Signed-off-by: Andrey Smetanin asmeta...@virtuozzo.com
 Signed-off-by: Denis V. Lunev d...@openvz.org
 CC: Gleb Natapov g...@kernel.org
 CC: Paolo Bonzini pbonz...@redhat.com
 ---
  arch/x86/include/uapi/asm/hyperv.h | 10 +
  arch/x86/kvm/Makefile  |  2 +-
  arch/x86/kvm/mshv.c| 84 
 ++
  arch/x86/kvm/mshv.h| 32 +++
  arch/x86/kvm/x86.c | 25 
  include/linux/kvm_host.h   | 17 
  include/uapi/linux/kvm.h   | 11 +
  7 files changed, 180 insertions(+), 1 deletion(-)
  create mode 100644 arch/x86/kvm/mshv.c
  create mode 100644 arch/x86/kvm/mshv.h

 diff --git a/arch/x86/include/uapi/asm/hyperv.h 
 b/arch/x86/include/uapi/asm/hyperv.h
 index ce6068d..25f3064 100644
 --- a/arch/x86/include/uapi/asm/hyperv.h
 +++ b/arch/x86/include/uapi/asm/hyperv.h
 @@ -199,6 +199,16 @@
  #define HV_X64_MSR_STIMER3_CONFIG  0x40B6
  #define HV_X64_MSR_STIMER3_COUNT   0x40B7

 +
 +/* Hypev-V guest crash notification MSR's */
 +#define HV_X64_MSR_CRASH_P00x4100
 +#define HV_X64_MSR_CRASH_P10x4101
 +#define HV_X64_MSR_CRASH_P20x4102
 +#define HV_X64_MSR_CRASH_P30x4103
 +#define HV_X64_MSR_CRASH_P40x4104
 +#define HV_X64_MSR_CRASH_CTL   0x4105
 +#define HV_CRASH_CTL_CRASH_NOTIFY  (1ULL  63)
 +
  #define HV_X64_MSR_HYPERCALL_ENABLE0x0001
  #define HV_X64_MSR_HYPERCALL_PAGE_ADDRESS_SHIFT12
  #define HV_X64_MSR_HYPERCALL_PAGE_ADDRESS_MASK \
 diff --git a/arch/x86/kvm/Makefile b/arch/x86/kvm/Makefile
 index 16e8f96..b1ec24d 100644
 --- a/arch/x86/kvm/Makefile
 +++ b/arch/x86/kvm/Makefile
 @@ -12,7 +12,7 @@ kvm-y += $(KVM)/kvm_main.o 
 $(KVM)/coalesced_mmio.o \
  kvm-$(CONFIG_KVM_ASYNC_PF) += $(KVM)/async_pf.o

  kvm-y  += x86.o mmu.o emulate.o i8259.o irq.o lapic.o \
 -  i8254.o ioapic.o irq_comm.o cpuid.o pmu.o
 +  i8254.o ioapic.o irq_comm.o cpuid.o pmu.o mshv.o
  kvm-$(CONFIG_KVM_DEVICE_ASSIGNMENT)+= assigned-dev.o iommu.o
  kvm-intel-y+= vmx.o
  kvm-amd-y  += svm.o
 diff --git a/arch/x86/kvm/mshv.c b/arch/x86/kvm/mshv.c
 new file mode 100644
 index 000..ad367c44
 --- /dev/null
 +++ b/arch/x86/kvm/mshv.c
 @@ -0,0 +1,84 @@
 +/*
 + * KVM Microsoft Hyper-V extended paravirtualization
 + *
 + * This work is licensed under the terms of the GNU GPL, version 2.  See
 + * the COPYING file in the top-level directory.
 + *
 + * Copyright (C) 2015 Andrey Smetanin asmeta...@virtuozzo.com
 + *
 + * Authors: Andrey Smetanin asmeta...@virtuozzo.com
 + */
 +
 +#include linux/kvm_host.h
 +#include mshv.h
 +
 +int kvm_mshv_ctx_create(struct kvm *kvm)
 +{
 +   struct kvm_mshv_ctx *ctx;
 +
 +   ctx = kzalloc(sizeof(struct kvm_mshv_ctx), GFP_KERNEL);
 +   if (!ctx)
 +   return -ENOMEM;
 +
 +   ctx-kvm = kvm;
 +   atomic_set(ctx-crash_pending, 0);
 +   kvm-mshv_ctx = ctx;
 +   return 0;
 +}
 +
 +void kvm_mshv_ctx_destroy(struct kvm *kvm)
 +{
 +   kfree(kvm-mshv_ctx);
 +}
 +
 +int kvm_mshv_msr_get_crash_ctl(struct kvm_vcpu *vcpu, u32 msr, u64 *pdata)
 +{
 +   struct kvm_mshv_ctx *ctx = kvm_vcpu_get_mshv_ctx(vcpu);
 +
 +   atomic_set(ctx-crash_pending, 1);

Can you explain why crash_pending is needed?

Do you know what the Windows guest behavior is here? From my reading
of the Hyper-V TLFS 4.0, the guest will read HV_X64_MSR_CRASH_CTL at
some point (not necessarily at the time of the crash, potentially at
boot time?) to determine the crash actions that the hypervisor
supports. I'm not sure why kvm needs to remember any state on reads
from HV_X64_MSR_CRASH_CTL.

  +
 +   /* Response that KVM ready to receive crash data */
 +   *pdata = HV_CRASH_CTL_CRASH_NOTIFY;

The TLFS says that CrashNotify is the only current action that is
supported on a guest crash, but other actions may be supported in the
future. I suggest defining something like
HV_X64_MSR_CRASH_CTL_CONTENTS (see below), which will only have
HV_CRASH_CTL_CRASH_NOTIFY set for