Re: [PATCH v2 00/12] KVM: arm/arm64: move VGIC MMIO to kvm_io_bus
On Mon, Mar 23, 2015 at 5:58 PM, Andre Przywara wrote: > This series converts the VGIC MMIO handling routines to the generic > kvm_io_bus framework. The framework is needed for the ioeventfd > functionality, some people on the list wanted to see the VGIC > converted over to use it, too. > Beside from now moving to a generic framework instead of relying on > an ARM specific one we also clean up quite some code and get rid of > some unnecessary copying. > On that way the MMIO abort handling for ARM has changed quite a bit, > so please have a closer look and test it on your setup if possible. > > Based on the v1 review I addressed Christoffer's minor comments, but > also heavily changed [11/12]: "KVM: ARM: on IO mem abort - route the > call to KVM MMIO bus" to get rid of the now unnecessary copying and > the usage of kvm_exit_mmio in that early stage. See the respective > commit message for more details. > > The series is loosely based on Nikolay's work[1], thanks especially > for the tedious first patch. > I totally reworked Nikolay's 3/5 to avoid adding another MMIO handling > layer on top of the already quite convoluted VGIC MMIO handling. > Also Nikolay's 2/5 get extended and changed significantly, that's why > I dropped his Signed-off-by. > > Unfortunately kvm_io_bus lacks an opaque pointer to pass in some data, > so I worked around this by using container_of. > Now for every struct kvm_mmio_range array a KVM I/O device is > registered (one for VGICv2, 2*nr_vcpus + 1 for VGICv3), using the > struct kvm_io_device variable as an anchor into the new > struct vgic_io_device. This one holds the base address, the > vgic_io_range pointer and (in case of the GICv3 redistributor) the > associated vCPU, so that we can access all instance-specific data > easily. > > Patch 2 moves the iodev.h header file around, that solves a problem > when embedding a struct in arm_vgic.h later. That looks like a nice > cleanup anyway, so I added two patches to remove the compiler switch > to add virt/kvm as a include directory. This has been tested for > arm/arm64 and x86. As soon as I get around to compile-test the other > architectures, I can send out the respective patches for those, too. > > Patches 5-7 tweak the existing code a bit to make it fit for the > conversion. > Patch 8 contains the framework for the new handling, while > patch 9 and 10 enable the GICv2 and GICv3 emulation, respectively. > Patch 11 finally switches over to the new kvm_io_bus handling, > reworking the early ARM KVM MMIO handling quite a bit. Patch 12 > removes the now unneeded code. I split this up to ease reviewing, I > could merge patches as well if needed. Shall we add here also the last 2 patches from my series that actually enable the eventfd compilation and KVM_CAP_IOEVENTFD? Or should I send them separately? regards, Nikolay Nikolaev > > The series goes on top of the kvmarm.git/next branch and was briefly > tested on an arm64 model with a GICv2 and a GICv3 guest and on Midway > (GICv2 guest). > > Cheers, > Andre. > > [1] https://lists.cs.columbia.edu/pipermail/kvmarm/2015-January/013379.html > > Andre Przywara (11): > KVM: move iodev.h from virt/kvm/ to include/kvm > KVM: arm/arm64: remove now unneeded include directory from Makefile > KVM: x86: remove now unneeded include directory from Makefile > KVM: arm/arm64: rename struct kvm_mmio_range to vgic_io_range > KVM: mark kvm->buses as empty once they were destroyed > KVM: arm/arm64: simplify vgic_find_range() and callers > KVM: arm/arm64: implement kvm_io_bus MMIO handling for the VGIC > KVM: arm/arm64: prepare GICv2 emulation to be handled by kvm_io_bus > KVM: arm/arm64: prepare GICv3 emulation to use kvm_io_bus MMIO > handling > KVM: arm/arm64: rework MMIO abort handling to use KVM MMIO bus > KVM: arm/arm64: remove now obsolete VGIC specific MMIO handling code > > Nikolay Nikolaev (1): > KVM: Redesign kvm_io_bus_ API to pass VCPU structure to the > callbacks. > > arch/arm/include/asm/kvm_mmio.h | 22 > arch/arm/kvm/Makefile |2 +- > arch/arm/kvm/mmio.c | 60 --- > arch/arm64/include/asm/kvm_mmio.h | 22 > arch/arm64/kvm/Makefile |2 +- > arch/powerpc/kvm/mpic.c | 12 ++- > arch/powerpc/kvm/powerpc.c|4 +- > arch/s390/kvm/diag.c |2 +- > arch/x86/kvm/Makefile |2 +- > arch/x86/kvm/i8254.c | 14 ++- > arch/x86/kvm/i8254.h |2 +- > arch/x86/kvm/i8259.c | 12 +-- > arch/x86/kvm/ioapic.c |8 +- > arch/x86/kvm/ioapic.h |2 +- > arch/x86/kvm/
Re: [PATCH v2 11/12] KVM: arm/arm64: rework MMIO abort handling to use KVM MMIO bus
On Mon, Mar 23, 2015 at 5:58 PM, Andre Przywara wrote:
>
> Currently we have struct kvm_exit_mmio for encapsulating MMIO abort
> data to be passed on from syndrome decoding all the way down to the
> VGIC register handlers. Now as we switch the MMIO handling to be
> routed through the KVM MMIO bus, it does not make sense anymore to
> use that structure already from the beginning. So we put the data into
> kvm_run very early and use that encapsulation till the MMIO bus call.
> Then we fill kvm_exit_mmio in the VGIC only, making it a VGIC private
> structure. On that way we replace the data buffer in that structure
> with a pointer pointing to a single location in kvm_run, so we get
> rid of some copying on the way.
> I didn't bother to rename kvm_exit_mmio (to vgic_mmio or something),
I would vote for the renaming.
Otherwise the patch looks much cleaner and straightforward than what
it was before.
Nikolay Nikolaev
> because that touches a lot of code lines without any good reason.
>
> This is based on an original patch by Nikolay.
>
> Signed-off-by: Andre Przywara
> Cc: Nikolay Nikolaev
> ---
> arch/arm/include/asm/kvm_mmio.h | 22 --
> arch/arm/kvm/mmio.c | 60
> ++---
> arch/arm64/include/asm/kvm_mmio.h | 22 --
> include/kvm/arm_vgic.h|3 --
> virt/kvm/arm/vgic.c | 18 +++
> virt/kvm/arm/vgic.h |8 +
> 6 files changed, 55 insertions(+), 78 deletions(-)
>
> diff --git a/arch/arm/include/asm/kvm_mmio.h b/arch/arm/include/asm/kvm_mmio.h
> index 3f83db2..d8e90c8 100644
> --- a/arch/arm/include/asm/kvm_mmio.h
> +++ b/arch/arm/include/asm/kvm_mmio.h
> @@ -28,28 +28,6 @@ struct kvm_decode {
> bool sign_extend;
> };
>
> -/*
> - * The in-kernel MMIO emulation code wants to use a copy of run->mmio,
> - * which is an anonymous type. Use our own type instead.
> - */
> -struct kvm_exit_mmio {
> - phys_addr_t phys_addr;
> - u8 data[8];
> - u32 len;
> - boolis_write;
> - void*private;
> -};
> -
> -static inline void kvm_prepare_mmio(struct kvm_run *run,
> - struct kvm_exit_mmio *mmio)
> -{
> - run->mmio.phys_addr = mmio->phys_addr;
> - run->mmio.len = mmio->len;
> - run->mmio.is_write = mmio->is_write;
> - memcpy(run->mmio.data, mmio->data, mmio->len);
> - run->exit_reason= KVM_EXIT_MMIO;
> -}
> -
> int kvm_handle_mmio_return(struct kvm_vcpu *vcpu, struct kvm_run *run);
> int io_mem_abort(struct kvm_vcpu *vcpu, struct kvm_run *run,
> phys_addr_t fault_ipa);
> diff --git a/arch/arm/kvm/mmio.c b/arch/arm/kvm/mmio.c
> index 5d3bfc0..bb2ab44 100644
> --- a/arch/arm/kvm/mmio.c
> +++ b/arch/arm/kvm/mmio.c
> @@ -122,7 +122,7 @@ int kvm_handle_mmio_return(struct kvm_vcpu *vcpu, struct
> kvm_run *run)
> }
>
> static int decode_hsr(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
> - struct kvm_exit_mmio *mmio)
> + struct kvm_run *run)
> {
> unsigned long rt;
> int len;
> @@ -148,9 +148,9 @@ static int decode_hsr(struct kvm_vcpu *vcpu, phys_addr_t
> fault_ipa,
> sign_extend = kvm_vcpu_dabt_issext(vcpu);
> rt = kvm_vcpu_dabt_get_rd(vcpu);
>
> - mmio->is_write = is_write;
> - mmio->phys_addr = fault_ipa;
> - mmio->len = len;
> + run->mmio.is_write = is_write;
> + run->mmio.phys_addr = fault_ipa;
> + run->mmio.len = len;
> vcpu->arch.mmio_decode.sign_extend = sign_extend;
> vcpu->arch.mmio_decode.rt = rt;
>
> @@ -162,23 +162,49 @@ static int decode_hsr(struct kvm_vcpu *vcpu,
> phys_addr_t fault_ipa,
> return 0;
> }
>
> +/**
> + * handle_kernel_mmio - handle an in-kernel MMIO access
> + * @vcpu: pointer to the vcpu performing the access
> + * @run: pointer to the kvm_run structure
> + *
> + * returns true if the MMIO access has been performed in kernel space,
> + * and false if it needs to be emulated in user space.
> + */
> +static bool handle_kernel_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run)
> +{
> + int ret;
> +
> + if (run->mmio.is_write) {
> + ret = kvm_io_bus_write(vcpu, KVM_MMIO_BUS,
> run->mmio.phys_addr,
> + run->mmio.len, run->mmio.data);
> +
> + } else {
> + ret = kvm_io_bus_read(vcpu, KVM_MMIO_BUS, run->mmio.phys_addr,
Re: [PATCH v3 3/5] KVM: ARM VGIC add kvm_io_bus_ frontend
On Tue, Jan 27, 2015 at 7:44 PM, Andre Przywara wrote:
> Hi,
>
> On 27/01/15 17:26, Eric Auger wrote:
>> On 01/27/2015 05:51 PM, Nikolay Nikolaev wrote:
>>> Hi Andre,
>>>
>>> On Tue, Jan 27, 2015 at 3:31 PM, Andre Przywara
>>> wrote:
>>>>
>>>> Hi Nikolay,
>>>>
>>>> On 24/01/15 11:59, Nikolay Nikolaev wrote:
>>>>> In io_mem_abort remove the call to vgic_handle_mmio. The target is to have
>>>>> a single MMIO handling path - that is through the kvm_io_bus_ API.
>>>>>
>>>>> Register a kvm_io_device in kvm_vgic_init on the whole vGIC MMIO region.
>>>>> Both read and write calls are redirected to vgic_io_dev_access where
>>>>> kvm_exit_mmio is composed to pass it to vm_ops.handle_mmio.
>>>>>
>>>>>
>>>>> Signed-off-by: Nikolay Nikolaev
>>>>> ---
>>>>> arch/arm/kvm/mmio.c|3 -
>>>>> include/kvm/arm_vgic.h |3 -
>>>>> virt/kvm/arm/vgic.c| 123
>>>>>
>>>>> 3 files changed, 114 insertions(+), 15 deletions(-)
>>>>>
>>>>> diff --git a/arch/arm/kvm/mmio.c b/arch/arm/kvm/mmio.c
>>>>> index d852137..8dc2fde 100644
>>>>> --- a/arch/arm/kvm/mmio.c
>>>>> +++ b/arch/arm/kvm/mmio.c
>>>>> @@ -230,9 +230,6 @@ int io_mem_abort(struct kvm_vcpu *vcpu, struct
>>>>> kvm_run *run,
>>>>> fault_ipa, 0);
>>>>> }
>>>>>
>>>>> - if (vgic_handle_mmio(vcpu, run, &mmio))
>>>>> - return 1;
>>>>> -
>>>>
>>>> Why is this (whole patch) actually needed? Is that just to make it nicer
>>>> by pulling everything under one umbrella?
>>>
>>>
>>> It started from this mail form Christofer:
>>> https://lkml.org/lkml/2014/3/28/403
>> Hi Nikolay, Andre,
>>
>> I also understood that the target was to handle all kernel mmio through
>> the same API, hence the first patch. This patch shows that at least for
>> GICv2 it was doable without upheavals in vgic code and it also serves
>> ioeventd which is good. Andre do you think the price to pay to integrate
>> missing redistributors and forthcoming components is too high?
>
> Hopefully not, actually I reckon that moving the "upper level" MMIO
> dispatching out of vgic.c and letting the specific VGIC models register
> what they need themselves (in their -emul.c files) sounds quite promising.
> But this particular patch does not serve this purpose:
> a) we replace two lines with a bunch of more layered code
> b) we copy the MMIOed data to convert between the interfaces
> c) we miss GICv3 emulation
>
> So this needs to be addressed in a more general way (which maybe I will
> give a try). That being sad I don't see why we would need to do this
Andre,
we've already overspent our resource budget on this so I would be more
than glad if you can take over the VGIC part here.
Of course I'll help with whatever I can (review, testing, patches
logistics etc)
Here is the v3 of the patches in case you need to clone and base your
work on top of it:
https://git.virtualopensystems.com/common/linux/commits/ioeventfd_v3
regards,
Nikolay Nikolaev
> right now and hold back ioeventfd by this rather orthogonal issue.
>
> Christoffer, what's your take on this?
>
> Cheers,
> Andre.
>
>> Best Regards
>>
>> Eric
>>
>>
>>>
>>>>
>>>> For enabling ioeventfd you actually don't need this patch, right?
>>> Yes, we don't need it.
>>>> (I am asking because this breaks GICv3 emulation, see below)
>>>>
>>>>> if (handle_kernel_mmio(vcpu, run, &mmio))
>>>>> return 1;
>>>>>
>>>>> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
>>>>> index 7c55dd5..60639b1 100644
>>>>> --- a/include/kvm/arm_vgic.h
>>>>> +++ b/include/kvm/arm_vgic.h
>>>>> @@ -237,6 +237,7 @@ struct vgic_dist {
>>>>> unsigned long *irq_pending_on_cpu;
>>>>>
>>>>> struct vgic_vm_ops vm_ops;
>>>>> + struct kvm_io_device*io_dev;
>>>>> #endif
>>>>> };
>>>>>
>>>>> @@ -311,8 +312,6 @@ int kvm_vgic_inject_irq(struct kvm *kvm, int cpuid,
>>>&
Re: [PATCH v3 3/5] KVM: ARM VGIC add kvm_io_bus_ frontend
Hi Andre,
On Tue, Jan 27, 2015 at 3:31 PM, Andre Przywara wrote:
>
> Hi Nikolay,
>
> On 24/01/15 11:59, Nikolay Nikolaev wrote:
> > In io_mem_abort remove the call to vgic_handle_mmio. The target is to have
> > a single MMIO handling path - that is through the kvm_io_bus_ API.
> >
> > Register a kvm_io_device in kvm_vgic_init on the whole vGIC MMIO region.
> > Both read and write calls are redirected to vgic_io_dev_access where
> > kvm_exit_mmio is composed to pass it to vm_ops.handle_mmio.
> >
> >
> > Signed-off-by: Nikolay Nikolaev
> > ---
> > arch/arm/kvm/mmio.c|3 -
> > include/kvm/arm_vgic.h |3 -
> > virt/kvm/arm/vgic.c| 123
> >
> > 3 files changed, 114 insertions(+), 15 deletions(-)
> >
> > diff --git a/arch/arm/kvm/mmio.c b/arch/arm/kvm/mmio.c
> > index d852137..8dc2fde 100644
> > --- a/arch/arm/kvm/mmio.c
> > +++ b/arch/arm/kvm/mmio.c
> > @@ -230,9 +230,6 @@ int io_mem_abort(struct kvm_vcpu *vcpu, struct kvm_run
> > *run,
> > fault_ipa, 0);
> > }
> >
> > - if (vgic_handle_mmio(vcpu, run, &mmio))
> > - return 1;
> > -
>
> Why is this (whole patch) actually needed? Is that just to make it nicer
> by pulling everything under one umbrella?
It started from this mail form Christofer:
https://lkml.org/lkml/2014/3/28/403
>
> For enabling ioeventfd you actually don't need this patch, right?
Yes, we don't need it.
> (I am asking because this breaks GICv3 emulation, see below)
>
> > if (handle_kernel_mmio(vcpu, run, &mmio))
> > return 1;
> >
> > diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> > index 7c55dd5..60639b1 100644
> > --- a/include/kvm/arm_vgic.h
> > +++ b/include/kvm/arm_vgic.h
> > @@ -237,6 +237,7 @@ struct vgic_dist {
> > unsigned long *irq_pending_on_cpu;
> >
> > struct vgic_vm_ops vm_ops;
> > + struct kvm_io_device*io_dev;
> > #endif
> > };
> >
> > @@ -311,8 +312,6 @@ int kvm_vgic_inject_irq(struct kvm *kvm, int cpuid,
> > unsigned int irq_num,
> > bool level);
> > void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg);
> > int kvm_vgic_vcpu_pending_irq(struct kvm_vcpu *vcpu);
> > -bool vgic_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
> > - struct kvm_exit_mmio *mmio);
> >
> > #define irqchip_in_kernel(k) (!!((k)->arch.vgic.in_kernel))
> > #define vgic_initialized(k) (!!((k)->arch.vgic.nr_cpus))
> > diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
> > index 0cc6ab6..195d2ba 100644
> > --- a/virt/kvm/arm/vgic.c
> > +++ b/virt/kvm/arm/vgic.c
> > @@ -31,6 +31,9 @@
> > #include
> > #include
> > #include
> > +#include
> > +
> > +#include "iodev.h"
> >
> > /*
> > * How the whole thing works (courtesy of Christoffer Dall):
> > @@ -77,6 +80,7 @@
> >
> > #include "vgic.h"
> >
> > +static int vgic_register_kvm_io_dev(struct kvm *kvm);
> > static void vgic_retire_disabled_irqs(struct kvm_vcpu *vcpu);
> > static void vgic_retire_lr(int lr_nr, int irq, struct kvm_vcpu *vcpu);
> > static struct vgic_lr vgic_get_lr(const struct kvm_vcpu *vcpu, int lr);
> > @@ -97,6 +101,7 @@ static bool queue_sgi(struct kvm_vcpu *vcpu, int irq)
> >
> > int kvm_vgic_map_resources(struct kvm *kvm)
> > {
> > + vgic_register_kvm_io_dev(kvm);
> > return kvm->arch.vgic.vm_ops.map_resources(kvm, vgic);
> > }
> >
> > @@ -776,27 +781,123 @@ bool vgic_handle_mmio_range(struct kvm_vcpu *vcpu,
> > struct kvm_run *run,
> > }
> >
> > /**
> > - * vgic_handle_mmio - handle an in-kernel MMIO access for the GIC emulation
> > + * vgic_io_dev_access - handle an in-kernel MMIO access for the GIC
> > emulation
> > * @vcpu: pointer to the vcpu performing the access
> > - * @run: pointer to the kvm_run structure
> > - * @mmio: pointer to the data describing the access
> > + * @this: pointer to the kvm_io_device structure
> > + * @addr: the MMIO address being accessed
> > + * @len: the length of the accessed data
> > + * @val: pointer to the value being written,
> > + * or where the read operation will store its result
> > + * @is_write: flag to show whether a write access is performed
> > *
> > - * returns true if th
[PATCH v3 3/5] KVM: ARM VGIC add kvm_io_bus_ frontend
In io_mem_abort remove the call to vgic_handle_mmio. The target is to have
a single MMIO handling path - that is through the kvm_io_bus_ API.
Register a kvm_io_device in kvm_vgic_init on the whole vGIC MMIO region.
Both read and write calls are redirected to vgic_io_dev_access where
kvm_exit_mmio is composed to pass it to vm_ops.handle_mmio.
Signed-off-by: Nikolay Nikolaev
---
arch/arm/kvm/mmio.c|3 -
include/kvm/arm_vgic.h |3 -
virt/kvm/arm/vgic.c| 123
3 files changed, 114 insertions(+), 15 deletions(-)
diff --git a/arch/arm/kvm/mmio.c b/arch/arm/kvm/mmio.c
index d852137..8dc2fde 100644
--- a/arch/arm/kvm/mmio.c
+++ b/arch/arm/kvm/mmio.c
@@ -230,9 +230,6 @@ int io_mem_abort(struct kvm_vcpu *vcpu, struct kvm_run *run,
fault_ipa, 0);
}
- if (vgic_handle_mmio(vcpu, run, &mmio))
- return 1;
-
if (handle_kernel_mmio(vcpu, run, &mmio))
return 1;
diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index 7c55dd5..60639b1 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -237,6 +237,7 @@ struct vgic_dist {
unsigned long *irq_pending_on_cpu;
struct vgic_vm_ops vm_ops;
+ struct kvm_io_device*io_dev;
#endif
};
@@ -311,8 +312,6 @@ int kvm_vgic_inject_irq(struct kvm *kvm, int cpuid,
unsigned int irq_num,
bool level);
void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg);
int kvm_vgic_vcpu_pending_irq(struct kvm_vcpu *vcpu);
-bool vgic_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
- struct kvm_exit_mmio *mmio);
#define irqchip_in_kernel(k) (!!((k)->arch.vgic.in_kernel))
#define vgic_initialized(k)(!!((k)->arch.vgic.nr_cpus))
diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
index 0cc6ab6..195d2ba 100644
--- a/virt/kvm/arm/vgic.c
+++ b/virt/kvm/arm/vgic.c
@@ -31,6 +31,9 @@
#include
#include
#include
+#include
+
+#include "iodev.h"
/*
* How the whole thing works (courtesy of Christoffer Dall):
@@ -77,6 +80,7 @@
#include "vgic.h"
+static int vgic_register_kvm_io_dev(struct kvm *kvm);
static void vgic_retire_disabled_irqs(struct kvm_vcpu *vcpu);
static void vgic_retire_lr(int lr_nr, int irq, struct kvm_vcpu *vcpu);
static struct vgic_lr vgic_get_lr(const struct kvm_vcpu *vcpu, int lr);
@@ -97,6 +101,7 @@ static bool queue_sgi(struct kvm_vcpu *vcpu, int irq)
int kvm_vgic_map_resources(struct kvm *kvm)
{
+ vgic_register_kvm_io_dev(kvm);
return kvm->arch.vgic.vm_ops.map_resources(kvm, vgic);
}
@@ -776,27 +781,123 @@ bool vgic_handle_mmio_range(struct kvm_vcpu *vcpu,
struct kvm_run *run,
}
/**
- * vgic_handle_mmio - handle an in-kernel MMIO access for the GIC emulation
+ * vgic_io_dev_access - handle an in-kernel MMIO access for the GIC emulation
* @vcpu: pointer to the vcpu performing the access
- * @run: pointer to the kvm_run structure
- * @mmio: pointer to the data describing the access
+ * @this: pointer to the kvm_io_device structure
+ * @addr: the MMIO address being accessed
+ * @len: the length of the accessed data
+ * @val: pointer to the value being written,
+ * or where the read operation will store its result
+ * @is_write: flag to show whether a write access is performed
*
- * returns true if the MMIO access has been performed in kernel space,
- * and false if it needs to be emulated in user space.
+ * returns 0 if the MMIO access has been performed in kernel space,
+ * and 1 if it needs to be emulated in user space.
* Calls the actual handling routine for the selected VGIC model.
*/
-bool vgic_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
- struct kvm_exit_mmio *mmio)
+static int vgic_io_dev_access(struct kvm_vcpu *vcpu, struct kvm_io_device
*this,
+ gpa_t addr, int len, void *val, bool is_write)
{
- if (!irqchip_in_kernel(vcpu->kvm))
- return false;
+ struct kvm_exit_mmio mmio;
+ bool ret;
+
+ mmio = (struct kvm_exit_mmio) {
+ .phys_addr = addr,
+ .len = len,
+ .is_write = is_write,
+ };
+
+ if (is_write)
+ memcpy(mmio.data, val, len);
/*
* This will currently call either vgic_v2_handle_mmio() or
* vgic_v3_handle_mmio(), which in turn will call
* vgic_handle_mmio_range() defined above.
*/
- return vcpu->kvm->arch.vgic.vm_ops.handle_mmio(vcpu, run, mmio);
+ ret = vcpu->kvm->arch.vgic.vm_ops.handle_mmio(vcpu, vcpu->run, &mmio);
+
+ if (!is_write)
+ memcpy(val, mmio.data, len);
+
+ return ret ? 0 : 1;
+}
+
+static int vgic_io_dev_read(struct kvm_vcpu *vcpu, struct kvm_io_device *this,
+
[PATCH v3 1/5] KVM: Redesign kvm_io_bus_ API to pass VCPU structure to the callbacks.
This is needed in e.g. ARM vGIC emulation, where the MMIO handling
depends on the VCPU that does the access.
Signed-off-by: Nikolay Nikolaev
---
arch/powerpc/kvm/mpic.c| 10 ++
arch/powerpc/kvm/powerpc.c |4 ++--
arch/s390/kvm/diag.c |2 +-
arch/x86/kvm/i8254.c | 14 +-
arch/x86/kvm/i8259.c | 12 ++--
arch/x86/kvm/ioapic.c |8
arch/x86/kvm/lapic.c |4 ++--
arch/x86/kvm/vmx.c |2 +-
arch/x86/kvm/x86.c | 13 +++--
include/linux/kvm_host.h | 10 +-
virt/kvm/coalesced_mmio.c |5 +++--
virt/kvm/eventfd.c |4 ++--
virt/kvm/iodev.h | 23 +++
virt/kvm/kvm_main.c| 32
14 files changed, 79 insertions(+), 64 deletions(-)
diff --git a/arch/powerpc/kvm/mpic.c b/arch/powerpc/kvm/mpic.c
index 39b3a8f..8542f07 100644
--- a/arch/powerpc/kvm/mpic.c
+++ b/arch/powerpc/kvm/mpic.c
@@ -1374,8 +1374,9 @@ static int kvm_mpic_write_internal(struct openpic *opp,
gpa_t addr, u32 val)
return -ENXIO;
}
-static int kvm_mpic_read(struct kvm_io_device *this, gpa_t addr,
-int len, void *ptr)
+static int kvm_mpic_read(struct kvm_vcpu *vcpu,
+struct kvm_io_device *this,
+gpa_t addr, int len, void *ptr)
{
struct openpic *opp = container_of(this, struct openpic, mmio);
int ret;
@@ -1415,8 +1416,9 @@ static int kvm_mpic_read(struct kvm_io_device *this,
gpa_t addr,
return ret;
}
-static int kvm_mpic_write(struct kvm_io_device *this, gpa_t addr,
- int len, const void *ptr)
+static int kvm_mpic_write(struct kvm_vcpu *vcpu,
+ struct kvm_io_device *this,
+ gpa_t addr, int len, const void *ptr)
{
struct openpic *opp = container_of(this, struct openpic, mmio);
int ret;
diff --git a/arch/powerpc/kvm/powerpc.c b/arch/powerpc/kvm/powerpc.c
index c45eaab..0aac251 100644
--- a/arch/powerpc/kvm/powerpc.c
+++ b/arch/powerpc/kvm/powerpc.c
@@ -808,7 +808,7 @@ int kvmppc_handle_load(struct kvm_run *run, struct kvm_vcpu
*vcpu,
idx = srcu_read_lock(&vcpu->kvm->srcu);
- ret = kvm_io_bus_read(vcpu->kvm, KVM_MMIO_BUS, run->mmio.phys_addr,
+ ret = kvm_io_bus_read(vcpu, KVM_MMIO_BUS, run->mmio.phys_addr,
bytes, &run->mmio.data);
srcu_read_unlock(&vcpu->kvm->srcu, idx);
@@ -881,7 +881,7 @@ int kvmppc_handle_store(struct kvm_run *run, struct
kvm_vcpu *vcpu,
idx = srcu_read_lock(&vcpu->kvm->srcu);
- ret = kvm_io_bus_write(vcpu->kvm, KVM_MMIO_BUS, run->mmio.phys_addr,
+ ret = kvm_io_bus_write(vcpu, KVM_MMIO_BUS, run->mmio.phys_addr,
bytes, &run->mmio.data);
srcu_read_unlock(&vcpu->kvm->srcu, idx);
diff --git a/arch/s390/kvm/diag.c b/arch/s390/kvm/diag.c
index 9254aff..329ec75 100644
--- a/arch/s390/kvm/diag.c
+++ b/arch/s390/kvm/diag.c
@@ -213,7 +213,7 @@ static int __diag_virtio_hypercall(struct kvm_vcpu *vcpu)
* - gpr 3 contains the virtqueue index (passed as datamatch)
* - gpr 4 contains the index on the bus (optionally)
*/
- ret = kvm_io_bus_write_cookie(vcpu->kvm, KVM_VIRTIO_CCW_NOTIFY_BUS,
+ ret = kvm_io_bus_write_cookie(vcpu, KVM_VIRTIO_CCW_NOTIFY_BUS,
vcpu->run->s.regs.gprs[2] & 0x,
8, &vcpu->run->s.regs.gprs[3],
vcpu->run->s.regs.gprs[4]);
diff --git a/arch/x86/kvm/i8254.c b/arch/x86/kvm/i8254.c
index 298781d..4dce6f8 100644
--- a/arch/x86/kvm/i8254.c
+++ b/arch/x86/kvm/i8254.c
@@ -443,7 +443,8 @@ static inline int pit_in_range(gpa_t addr)
(addr < KVM_PIT_BASE_ADDRESS + KVM_PIT_MEM_LENGTH));
}
-static int pit_ioport_write(struct kvm_io_device *this,
+static int pit_ioport_write(struct kvm_vcpu *vcpu,
+ struct kvm_io_device *this,
gpa_t addr, int len, const void *data)
{
struct kvm_pit *pit = dev_to_pit(this);
@@ -519,7 +520,8 @@ static int pit_ioport_write(struct kvm_io_device *this,
return 0;
}
-static int pit_ioport_read(struct kvm_io_device *this,
+static int pit_ioport_read(struct kvm_vcpu *vcpu,
+ struct kvm_io_device *this,
gpa_t addr, int len, void *data)
{
struct kvm_pit *pit = dev_to_pit(this);
@@ -589,7 +591,8 @@ static int pit_ioport_read(struct kvm_io_device *this,
return 0;
}
-static int speaker_ioport_write(struct kvm_io_device *this,
+static int speaker_ioport_write(struct kvm_vcpu *vcpu,
+ struct kvm_io_device *this,
[PATCH v3 5/5] ARM: enable KVM_CAP_IOEVENTFD
KVM on arm will support the eventfd extension. Signed-off-by: Nikolay Nikolaev --- arch/arm/kvm/arm.c |3 +++ 1 file changed, 3 insertions(+) diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c index 6fbfa5f..ec5ebef 100644 --- a/arch/arm/kvm/arm.c +++ b/arch/arm/kvm/arm.c @@ -197,6 +197,9 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) case KVM_CAP_MAX_VCPUS: r = KVM_MAX_VCPUS; break; + case KVM_CAP_IOEVENTFD: + r = 1; + break; default: r = kvm_arch_dev_ioctl_check_extension(ext); break; -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 4/5] ARM/ARM64: enable linking against eventfd
This enables compilation of the eventfd feature on ARM/ARM64. Signed-off-by: Nikolay Nikolaev --- arch/arm/kvm/Kconfig|1 + arch/arm/kvm/Makefile |2 +- arch/arm64/kvm/Kconfig |1 + arch/arm64/kvm/Makefile |2 +- 4 files changed, 4 insertions(+), 2 deletions(-) diff --git a/arch/arm/kvm/Kconfig b/arch/arm/kvm/Kconfig index a8d1ace..ca23ded 100644 --- a/arch/arm/kvm/Kconfig +++ b/arch/arm/kvm/Kconfig @@ -20,6 +20,7 @@ config KVM bool "Kernel-based Virtual Machine (KVM) support" select PREEMPT_NOTIFIERS select ANON_INODES + select HAVE_KVM_EVENTFD select HAVE_KVM_CPU_RELAX_INTERCEPT select HAVE_KVM_ARCH_TLB_FLUSH_ALL select KVM_MMIO diff --git a/arch/arm/kvm/Makefile b/arch/arm/kvm/Makefile index 443b8be..539c1a5 100644 --- a/arch/arm/kvm/Makefile +++ b/arch/arm/kvm/Makefile @@ -15,7 +15,7 @@ AFLAGS_init.o := -Wa,-march=armv7-a$(plus_virt) AFLAGS_interrupts.o := -Wa,-march=armv7-a$(plus_virt) KVM := ../../../virt/kvm -kvm-arm-y = $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o +kvm-arm-y = $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o $(KVM)/eventfd.o obj-y += kvm-arm.o init.o interrupts.o obj-y += arm.o handle_exit.o guest.o mmu.o emulate.o reset.o diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig index 3ce389b..2cc985c 100644 --- a/arch/arm64/kvm/Kconfig +++ b/arch/arm64/kvm/Kconfig @@ -21,6 +21,7 @@ config KVM select MMU_NOTIFIER select PREEMPT_NOTIFIERS select ANON_INODES + select HAVE_KVM_EVENTFD select HAVE_KVM_CPU_RELAX_INTERCEPT select HAVE_KVM_ARCH_TLB_FLUSH_ALL select KVM_MMIO diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile index 4e6e09e..0dffb5f 100644 --- a/arch/arm64/kvm/Makefile +++ b/arch/arm64/kvm/Makefile @@ -11,7 +11,7 @@ ARM=../../../arch/arm/kvm obj-$(CONFIG_KVM_ARM_HOST) += kvm.o -kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o +kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o $(KVM)/eventfd.o kvm-$(CONFIG_KVM_ARM_HOST) += $(ARM)/arm.o $(ARM)/mmu.o $(ARM)/mmio.o kvm-$(CONFIG_KVM_ARM_HOST) += $(ARM)/psci.o $(ARM)/perf.o -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 0/5] ARM: KVM: Enable the ioeventfd capability of KVM on ARM
The IOEVENTFD KVM capability is a prerequisite for vhost support. This series enables the ioeventfd KVM capability on ARM. The implementation routes MMIO access in the IO abort handler to the KVM IO bus. If there is already a registered ioeventfd handler for this address, the file descriptor will be triggered. We extended the KVM IO bus API to expose the VCPU struct pointer. Now the VGIC MMIO access is done through this API. For this to operate the VGIC registers a kvm_io_device which represents the whole dist MMIO region. The code was tested on Dual Cortex-A15 Exynos5250 (ARM Chromebook). ARM64 build was verified, but not run on actual HW. Changes since v2: - rebased on top of kvmarm/next - moved vgic_register_kvm_io_dev to kvm_vgic_map_resources Changes since v1: - fixed x86 compilation - GICv2/GICv3 dist base selection - added vgic_unregister_kvm_io_dev to free the iodev resources - enable eventfd on ARM64 --- Nikolay Nikolaev (5): KVM: Redesign kvm_io_bus_ API to pass VCPU structure to the callbacks. KVM: ARM: on IO mem abort - route the call to KVM MMIO bus KVM: ARM VGIC add kvm_io_bus_ frontend ARM/ARM64: enable linking against eventfd ARM: enable KVM_CAP_IOEVENTFD arch/arm/kvm/Kconfig |1 arch/arm/kvm/Makefile |2 - arch/arm/kvm/arm.c |3 + arch/arm/kvm/mmio.c| 32 +++ arch/arm64/kvm/Kconfig |1 arch/arm64/kvm/Makefile|2 - arch/powerpc/kvm/mpic.c| 10 ++-- arch/powerpc/kvm/powerpc.c |4 + arch/s390/kvm/diag.c |2 - arch/x86/kvm/i8254.c | 14 +++-- arch/x86/kvm/i8259.c | 12 ++-- arch/x86/kvm/ioapic.c |8 +-- arch/x86/kvm/lapic.c |4 + arch/x86/kvm/vmx.c |2 - arch/x86/kvm/x86.c | 13 +++-- include/kvm/arm_vgic.h |3 - include/linux/kvm_host.h | 10 ++-- virt/kvm/arm/vgic.c| 123 virt/kvm/coalesced_mmio.c |5 +- virt/kvm/eventfd.c |4 + virt/kvm/iodev.h | 23 +--- virt/kvm/kvm_main.c| 32 ++- 22 files changed, 231 insertions(+), 79 deletions(-) -- Signature -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3 2/5] KVM: ARM: on IO mem abort - route the call to KVM MMIO bus
On IO memory abort, try to handle the MMIO access thorugh the KVM
registered read/write callbacks. This is done by invoking the relevant
kvm_io_bus_* API.
Signed-off-by: Nikolay Nikolaev
---
arch/arm/kvm/mmio.c | 33 +
1 file changed, 33 insertions(+)
diff --git a/arch/arm/kvm/mmio.c b/arch/arm/kvm/mmio.c
index 5d3bfc0..d852137 100644
--- a/arch/arm/kvm/mmio.c
+++ b/arch/arm/kvm/mmio.c
@@ -162,6 +162,36 @@ static int decode_hsr(struct kvm_vcpu *vcpu, phys_addr_t
fault_ipa,
return 0;
}
+/**
+ * handle_kernel_mmio - handle an in-kernel MMIO access
+ * @vcpu: pointer to the vcpu performing the access
+ * @run: pointer to the kvm_run structure
+ * @mmio: pointer to the data describing the access
+ *
+ * returns true if the MMIO access has been performed in kernel space,
+ * and false if it needs to be emulated in user space.
+ */
+static bool handle_kernel_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
+ struct kvm_exit_mmio *mmio)
+{
+ int ret;
+
+ if (mmio->is_write) {
+ ret = kvm_io_bus_write(vcpu, KVM_MMIO_BUS, mmio->phys_addr,
+ mmio->len, &mmio->data);
+
+ } else {
+ ret = kvm_io_bus_read(vcpu, KVM_MMIO_BUS, mmio->phys_addr,
+ mmio->len, &mmio->data);
+ }
+ if (!ret) {
+ kvm_prepare_mmio(run, mmio);
+ kvm_handle_mmio_return(vcpu, run);
+ }
+
+ return !ret;
+}
+
int io_mem_abort(struct kvm_vcpu *vcpu, struct kvm_run *run,
phys_addr_t fault_ipa)
{
@@ -203,6 +233,9 @@ int io_mem_abort(struct kvm_vcpu *vcpu, struct kvm_run *run,
if (vgic_handle_mmio(vcpu, run, &mmio))
return 1;
+ if (handle_kernel_mmio(vcpu, run, &mmio))
+ return 1;
+
kvm_prepare_mmio(run, &mmio);
return 0;
}
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2 2/5] KVM: ARM: on IO mem abort - route the call to KVM MMIO bus
On Mon, Jan 12, 2015 at 7:09 PM, Eric Auger wrote:
> Hi Nikolay,
> On 12/07/2014 10:37 AM, Nikolay Nikolaev wrote:
>> On IO memory abort, try to handle the MMIO access thorugh the KVM
>> registered read/write callbacks. This is done by invoking the relevant
>> kvm_io_bus_* API.
>>
>> Signed-off-by: Nikolay Nikolaev
>> ---
>> arch/arm/kvm/mmio.c | 33 +
>> 1 file changed, 33 insertions(+)
>>
>> diff --git a/arch/arm/kvm/mmio.c b/arch/arm/kvm/mmio.c
>> index 4cb5a93..e42469f 100644
>> --- a/arch/arm/kvm/mmio.c
>> +++ b/arch/arm/kvm/mmio.c
>> @@ -162,6 +162,36 @@ static int decode_hsr(struct kvm_vcpu *vcpu,
>> phys_addr_t fault_ipa,
>> return 0;
>> }
>>
>> +/**
>> + * handle_kernel_mmio - handle an in-kernel MMIO access
>> + * @vcpu:pointer to the vcpu performing the access
>> + * @run: pointer to the kvm_run structure
>> + * @mmio:pointer to the data describing the access
>> + *
>> + * returns true if the MMIO access has been performed in kernel space,
>> + * and false if it needs to be emulated in user space.
>> + */
>> +static bool handle_kernel_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
>> + struct kvm_exit_mmio *mmio)
>> +{
>> + int ret;
>> +
>> + if (mmio->is_write) {
>> + ret = kvm_io_bus_write(vcpu, KVM_MMIO_BUS, mmio->phys_addr,
>> + mmio->len, &mmio->data);
>> +
>> + } else {
>> + ret = kvm_io_bus_read(vcpu, KVM_MMIO_BUS, mmio->phys_addr,
>> + mmio->len, &mmio->data);
>> + }
>> + if (!ret) {
>> + kvm_prepare_mmio(run, mmio);
>> + kvm_handle_mmio_return(vcpu, run);
>> + }
>> +
>> + return !ret;
> in case ret < 0 (-EOPNOTSUPP = -95) aren't we returning true too? return
> (ret==0)?
>
>> +}
>> +
>> int io_mem_abort(struct kvm_vcpu *vcpu, struct kvm_run *run,
>>phys_addr_t fault_ipa)
>> {
>> @@ -200,6 +230,9 @@ int io_mem_abort(struct kvm_vcpu *vcpu, struct kvm_run
>> *run,
>> if (vgic_handle_mmio(vcpu, run, &mmio))
>> return 1;
>>
>> + if (handle_kernel_mmio(vcpu, run, &mmio))
>> + return 1;
>> +
>> kvm_prepare_mmio(run, &mmio);
>> return 0;
> currently the io_mem_abort returned value is not used by mmu.c code. I
> think this should be handed in kvm_handle_guest_abort. What do you think?
You're right that the returned value is not handled further after we
exit io_mem_abort, it's just passed up the call stack.
However I'm not sure how to handle it better. If you have ideas, please share.
regards,
Nikolay Nikolaev
>
> Best Regards
>
> Eric
>> }
>>
>
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2 3/5] KVM: ARM VGIC add kvm_io_bus_ frontend
On Mon, Jan 12, 2015 at 11:41 PM, Eric Auger wrote:
> On 12/07/2014 10:37 AM, Nikolay Nikolaev wrote:
>> In io_mem_abort remove the call to vgic_handle_mmio. The target is to have
>> a single MMIO handling path - that is through the kvm_io_bus_ API.
>>
>> Register a kvm_io_device in kvm_vgic_init on the whole vGIC MMIO region.
>> Both read and write calls are redirected to vgic_io_dev_access where
>> kvm_exit_mmio is composed to pass it to vm_ops.handle_mmio.
>>
>>
>> Signed-off-by: Nikolay Nikolaev
>> ---
>> arch/arm/kvm/mmio.c|3 -
>> include/kvm/arm_vgic.h |3 -
>> virt/kvm/arm/vgic.c| 127
>>
>> 3 files changed, 118 insertions(+), 15 deletions(-)
>>
>> diff --git a/arch/arm/kvm/mmio.c b/arch/arm/kvm/mmio.c
>> index e42469f..bf466c8 100644
>> --- a/arch/arm/kvm/mmio.c
>> +++ b/arch/arm/kvm/mmio.c
>> @@ -227,9 +227,6 @@ int io_mem_abort(struct kvm_vcpu *vcpu, struct kvm_run
>> *run,
>> if (mmio.is_write)
>> mmio_write_buf(mmio.data, mmio.len, data);
>>
>> - if (vgic_handle_mmio(vcpu, run, &mmio))
>> - return 1;
>> -
>> if (handle_kernel_mmio(vcpu, run, &mmio))
>> return 1;
>>
>> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
>> index e452ef7..d9b7d2a 100644
>> --- a/include/kvm/arm_vgic.h
>> +++ b/include/kvm/arm_vgic.h
>> @@ -233,6 +233,7 @@ struct vgic_dist {
>> unsigned long *irq_pending_on_cpu;
>>
>> struct vgic_vm_ops vm_ops;
>> + struct kvm_io_device*io_dev;
>> #endif
>> };
>>
>> @@ -307,8 +308,6 @@ int kvm_vgic_inject_irq(struct kvm *kvm, int cpuid,
>> unsigned int irq_num,
>> bool level);
>> void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg);
>> int kvm_vgic_vcpu_pending_irq(struct kvm_vcpu *vcpu);
>> -bool vgic_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
>> - struct kvm_exit_mmio *mmio);
>>
>> #define irqchip_in_kernel(k) (!!((k)->arch.vgic.in_kernel))
>> #define vgic_initialized(k) ((k)->arch.vgic.ready)
>> diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
>> index bd74207..1c7cbec 100644
>> --- a/virt/kvm/arm/vgic.c
>> +++ b/virt/kvm/arm/vgic.c
>> @@ -31,6 +31,9 @@
>> #include
>> #include
>> #include
>> +#include
>> +
>> +#include "iodev.h"
>>
>> /*
>> * How the whole thing works (courtesy of Christoffer Dall):
>> @@ -776,27 +779,127 @@ bool vgic_handle_mmio_range(struct kvm_vcpu *vcpu,
>> struct kvm_run *run,
>> }
>>
>> /**
>> - * vgic_handle_mmio - handle an in-kernel MMIO access for the GIC emulation
>> + * vgic_io_dev_access - handle an in-kernel MMIO access for the GIC
>> emulation
>> * @vcpu: pointer to the vcpu performing the access
>> - * @run: pointer to the kvm_run structure
>> - * @mmio: pointer to the data describing the access
>> + * @this: pointer to the kvm_io_device structure
>> + * @addr: the MMIO address being accessed
>> + * @len: the length of the accessed data
>> + * @val: pointer to the value being written,
>> + * or where the read operation will store its result
>> + * @is_write: flag to show whether a write access is performed
>> *
>> - * returns true if the MMIO access has been performed in kernel space,
>> - * and false if it needs to be emulated in user space.
>> + * returns 0 if the MMIO access has been performed in kernel space,
>> + * and 1 if it needs to be emulated in user space.
>> * Calls the actual handling routine for the selected VGIC model.
>> */
>> -bool vgic_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
>> - struct kvm_exit_mmio *mmio)
>> +static int vgic_io_dev_access(struct kvm_vcpu *vcpu, struct kvm_io_device
>> *this,
>> + gpa_t addr, int len, void *val, bool is_write)
>> {
>> - if (!irqchip_in_kernel(vcpu->kvm))
>> - return false;
>> + struct kvm_exit_mmio mmio;
>> + bool ret;
>> +
>> + mmio = (struct kvm_exit_mmio) {
>> + .phys_addr = addr,
>> + .len = len,
>> + .is_write = is_write,
>> + };
>> +
>> + if (is_write)
>> + memcpy(mmio.data, val, len);
>>
>> /*
>>* This will currently
Re: [RFC PATCH 2/5] ARM: on IO mem abort - route the call to KVM MMIO bus
On Mon, Jan 12, 2015 at 6:21 PM, Eric Auger wrote:
>
> On 12/05/2014 01:06 PM, Nikolay Nikolaev wrote:
> > On Sat, Nov 29, 2014 at 1:28 PM, Christoffer Dall
> > wrote:
> >> On Mon, Nov 24, 2014 at 11:26:51PM +0200, Nikolay Nikolaev wrote:
> >>> On IO memory abort, try to handle the MMIO access thorugh the KVM
> >>> registered read/write callbacks. This is done by invoking the relevant
> >>> kvm_io_bus_* API.
> >>>
> >>> Signed-off-by: Nikolay Nikolaev
> >>> ---
> >>> arch/arm/kvm/mmio.c | 33 +
> >>> 1 file changed, 33 insertions(+)
> >>>
> >>> diff --git a/arch/arm/kvm/mmio.c b/arch/arm/kvm/mmio.c
> >>> index 4cb5a93..81230da 100644
> >>> --- a/arch/arm/kvm/mmio.c
> >>> +++ b/arch/arm/kvm/mmio.c
> >>> @@ -162,6 +162,36 @@ static int decode_hsr(struct kvm_vcpu *vcpu,
> >>> phys_addr_t fault_ipa,
> >>> return 0;
> >>> }
> >>>
> >>> +/**
> >>> + * kvm_handle_mmio - handle an in-kernel MMIO access
> >>> + * @vcpu:pointer to the vcpu performing the access
> >>> + * @run: pointer to the kvm_run structure
> >>> + * @mmio:pointer to the data describing the access
> >>> + *
> >>> + * returns true if the MMIO access has been performed in kernel space,
> >>> + * and false if it needs to be emulated in user space.
> >>> + */
> >>> +static bool handle_kernel_mmio(struct kvm_vcpu *vcpu, struct kvm_run
> >>> *run,
> >>> + struct kvm_exit_mmio *mmio)
> >>> +{
> >>> + int ret;
> >>> +
> >>> + if (mmio->is_write) {
> >>> + ret = kvm_io_bus_write(vcpu, KVM_MMIO_BUS, mmio->phys_addr,
> >>> + mmio->len, &mmio->data);
> >>> +
> >>> + } else {
> >>> + ret = kvm_io_bus_read(vcpu, KVM_MMIO_BUS, mmio->phys_addr,
> >>> + mmio->len, &mmio->data);
> >>> + }
> >>> + if (!ret) {
> >>> + kvm_prepare_mmio(run, mmio);
> >>> + kvm_handle_mmio_return(vcpu, run);
> >>> + }
> >>> +
> >>> + return !ret;
> >>> +}
> >>> +
> >>> int io_mem_abort(struct kvm_vcpu *vcpu, struct kvm_run *run,
> >>>phys_addr_t fault_ipa)
> >>> {
> >>> @@ -200,6 +230,9 @@ int io_mem_abort(struct kvm_vcpu *vcpu, struct
> >>> kvm_run *run,
> >>> if (vgic_handle_mmio(vcpu, run, &mmio))
> >>> return 1;
> >>>
> >>> + if (handle_kernel_mmio(vcpu, run, &mmio))
> >>> + return 1;
> >>> +
> >>
> >> Is this stuff always synchronously handled so that the mmio is properly
> >> populated upon handle_kernel_mmio on reads?
> >
> > If I get it right the kvm_io_bus_ API is intended to work
> > synchronously. Of course it probably depends
> > on how the registered device handles the read/write call.
> > Or maybe I misunderstand your question? Please clarify in that case.
>
> in case of ioeventfd implementation it is not the case since the write
> is deferred in the kernel thread which handles the eventfd.
Christofer was asking about reads. I guess about writes it does not
make any diiference as long as the needed values are copied into the
new thread.
regards,
Nikolay Nikolaev
>
> Best Regards
>
> Eric
> >
> > regards,
> > Nikolay Nikolaev
> >
> >>
> >> -Christoffer
>
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 1/5] KVM: Redesign kvm_io_bus_ API to pass VCPU structure to the callbacks.
This is needed in e.g. ARM vGIC emulation, where the MMIO handling
depends on the VCPU that does the access.
Signed-off-by: Nikolay Nikolaev
---
arch/ia64/kvm/kvm-ia64.c |4 ++--
arch/powerpc/kvm/mpic.c| 10 ++
arch/powerpc/kvm/powerpc.c |4 ++--
arch/s390/kvm/diag.c |2 +-
arch/x86/kvm/i8254.c | 14 +-
arch/x86/kvm/i8259.c | 12 ++--
arch/x86/kvm/lapic.c |4 ++--
arch/x86/kvm/vmx.c |2 +-
arch/x86/kvm/x86.c | 13 +++--
include/linux/kvm_host.h | 10 +-
virt/kvm/coalesced_mmio.c |5 +++--
virt/kvm/eventfd.c |4 ++--
virt/kvm/ioapic.c |8
virt/kvm/iodev.h | 23 +++
virt/kvm/kvm_main.c| 32
15 files changed, 81 insertions(+), 66 deletions(-)
diff --git a/arch/ia64/kvm/kvm-ia64.c b/arch/ia64/kvm/kvm-ia64.c
index dbe46f4..287df8d 100644
--- a/arch/ia64/kvm/kvm-ia64.c
+++ b/arch/ia64/kvm/kvm-ia64.c
@@ -246,10 +246,10 @@ static int handle_mmio(struct kvm_vcpu *vcpu, struct
kvm_run *kvm_run)
return 0;
mmio:
if (p->dir)
- r = kvm_io_bus_read(vcpu->kvm, KVM_MMIO_BUS, p->addr,
+ r = kvm_io_bus_read(vcpu, KVM_MMIO_BUS, p->addr,
p->size, &p->data);
else
- r = kvm_io_bus_write(vcpu->kvm, KVM_MMIO_BUS, p->addr,
+ r = kvm_io_bus_write(vcpu, KVM_MMIO_BUS, p->addr,
p->size, &p->data);
if (r)
printk(KERN_ERR"kvm: No iodevice found! addr:%lx\n", p->addr);
diff --git a/arch/powerpc/kvm/mpic.c b/arch/powerpc/kvm/mpic.c
index 39b3a8f..8542f07 100644
--- a/arch/powerpc/kvm/mpic.c
+++ b/arch/powerpc/kvm/mpic.c
@@ -1374,8 +1374,9 @@ static int kvm_mpic_write_internal(struct openpic *opp,
gpa_t addr, u32 val)
return -ENXIO;
}
-static int kvm_mpic_read(struct kvm_io_device *this, gpa_t addr,
-int len, void *ptr)
+static int kvm_mpic_read(struct kvm_vcpu *vcpu,
+struct kvm_io_device *this,
+gpa_t addr, int len, void *ptr)
{
struct openpic *opp = container_of(this, struct openpic, mmio);
int ret;
@@ -1415,8 +1416,9 @@ static int kvm_mpic_read(struct kvm_io_device *this,
gpa_t addr,
return ret;
}
-static int kvm_mpic_write(struct kvm_io_device *this, gpa_t addr,
- int len, const void *ptr)
+static int kvm_mpic_write(struct kvm_vcpu *vcpu,
+ struct kvm_io_device *this,
+ gpa_t addr, int len, const void *ptr)
{
struct openpic *opp = container_of(this, struct openpic, mmio);
int ret;
diff --git a/arch/powerpc/kvm/powerpc.c b/arch/powerpc/kvm/powerpc.c
index c1f8f53..5ac065b 100644
--- a/arch/powerpc/kvm/powerpc.c
+++ b/arch/powerpc/kvm/powerpc.c
@@ -814,7 +814,7 @@ int kvmppc_handle_load(struct kvm_run *run, struct kvm_vcpu
*vcpu,
idx = srcu_read_lock(&vcpu->kvm->srcu);
- ret = kvm_io_bus_read(vcpu->kvm, KVM_MMIO_BUS, run->mmio.phys_addr,
+ ret = kvm_io_bus_read(vcpu, KVM_MMIO_BUS, run->mmio.phys_addr,
bytes, &run->mmio.data);
srcu_read_unlock(&vcpu->kvm->srcu, idx);
@@ -887,7 +887,7 @@ int kvmppc_handle_store(struct kvm_run *run, struct
kvm_vcpu *vcpu,
idx = srcu_read_lock(&vcpu->kvm->srcu);
- ret = kvm_io_bus_write(vcpu->kvm, KVM_MMIO_BUS, run->mmio.phys_addr,
+ ret = kvm_io_bus_write(vcpu, KVM_MMIO_BUS, run->mmio.phys_addr,
bytes, &run->mmio.data);
srcu_read_unlock(&vcpu->kvm->srcu, idx);
diff --git a/arch/s390/kvm/diag.c b/arch/s390/kvm/diag.c
index 9254aff..329ec75 100644
--- a/arch/s390/kvm/diag.c
+++ b/arch/s390/kvm/diag.c
@@ -213,7 +213,7 @@ static int __diag_virtio_hypercall(struct kvm_vcpu *vcpu)
* - gpr 3 contains the virtqueue index (passed as datamatch)
* - gpr 4 contains the index on the bus (optionally)
*/
- ret = kvm_io_bus_write_cookie(vcpu->kvm, KVM_VIRTIO_CCW_NOTIFY_BUS,
+ ret = kvm_io_bus_write_cookie(vcpu, KVM_VIRTIO_CCW_NOTIFY_BUS,
vcpu->run->s.regs.gprs[2] & 0x,
8, &vcpu->run->s.regs.gprs[3],
vcpu->run->s.regs.gprs[4]);
diff --git a/arch/x86/kvm/i8254.c b/arch/x86/kvm/i8254.c
index 298781d..4dce6f8 100644
--- a/arch/x86/kvm/i8254.c
+++ b/arch/x86/kvm/i8254.c
@@ -443,7 +443,8 @@ static inline int pit_in_range(gpa_t addr)
(addr < KVM_PIT_BASE_ADDRESS + KVM_PIT_MEM_LENGTH));
}
-static int pit_ioport_write(struct kvm_io_device *this,
+static int p
[PATCH v2 5/5] ARM: enable KVM_CAP_IOEVENTFD
KVM on arm will support the eventfd extension. Signed-off-by: Nikolay Nikolaev --- arch/arm/kvm/arm.c |3 +++ 1 file changed, 3 insertions(+) diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c index c3d0fbd..266b618 100644 --- a/arch/arm/kvm/arm.c +++ b/arch/arm/kvm/arm.c @@ -197,6 +197,9 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) case KVM_CAP_MAX_VCPUS: r = KVM_MAX_VCPUS; break; + case KVM_CAP_IOEVENTFD: + r = 1; + break; default: r = kvm_arch_dev_ioctl_check_extension(ext); break; -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 3/5] KVM: ARM VGIC add kvm_io_bus_ frontend
In io_mem_abort remove the call to vgic_handle_mmio. The target is to have
a single MMIO handling path - that is through the kvm_io_bus_ API.
Register a kvm_io_device in kvm_vgic_init on the whole vGIC MMIO region.
Both read and write calls are redirected to vgic_io_dev_access where
kvm_exit_mmio is composed to pass it to vm_ops.handle_mmio.
Signed-off-by: Nikolay Nikolaev
---
arch/arm/kvm/mmio.c|3 -
include/kvm/arm_vgic.h |3 -
virt/kvm/arm/vgic.c| 127
3 files changed, 118 insertions(+), 15 deletions(-)
diff --git a/arch/arm/kvm/mmio.c b/arch/arm/kvm/mmio.c
index e42469f..bf466c8 100644
--- a/arch/arm/kvm/mmio.c
+++ b/arch/arm/kvm/mmio.c
@@ -227,9 +227,6 @@ int io_mem_abort(struct kvm_vcpu *vcpu, struct kvm_run *run,
if (mmio.is_write)
mmio_write_buf(mmio.data, mmio.len, data);
- if (vgic_handle_mmio(vcpu, run, &mmio))
- return 1;
-
if (handle_kernel_mmio(vcpu, run, &mmio))
return 1;
diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index e452ef7..d9b7d2a 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -233,6 +233,7 @@ struct vgic_dist {
unsigned long *irq_pending_on_cpu;
struct vgic_vm_ops vm_ops;
+ struct kvm_io_device*io_dev;
#endif
};
@@ -307,8 +308,6 @@ int kvm_vgic_inject_irq(struct kvm *kvm, int cpuid,
unsigned int irq_num,
bool level);
void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg);
int kvm_vgic_vcpu_pending_irq(struct kvm_vcpu *vcpu);
-bool vgic_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
- struct kvm_exit_mmio *mmio);
#define irqchip_in_kernel(k) (!!((k)->arch.vgic.in_kernel))
#define vgic_initialized(k)((k)->arch.vgic.ready)
diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
index bd74207..1c7cbec 100644
--- a/virt/kvm/arm/vgic.c
+++ b/virt/kvm/arm/vgic.c
@@ -31,6 +31,9 @@
#include
#include
#include
+#include
+
+#include "iodev.h"
/*
* How the whole thing works (courtesy of Christoffer Dall):
@@ -776,27 +779,127 @@ bool vgic_handle_mmio_range(struct kvm_vcpu *vcpu,
struct kvm_run *run,
}
/**
- * vgic_handle_mmio - handle an in-kernel MMIO access for the GIC emulation
+ * vgic_io_dev_access - handle an in-kernel MMIO access for the GIC emulation
* @vcpu: pointer to the vcpu performing the access
- * @run: pointer to the kvm_run structure
- * @mmio: pointer to the data describing the access
+ * @this: pointer to the kvm_io_device structure
+ * @addr: the MMIO address being accessed
+ * @len: the length of the accessed data
+ * @val: pointer to the value being written,
+ * or where the read operation will store its result
+ * @is_write: flag to show whether a write access is performed
*
- * returns true if the MMIO access has been performed in kernel space,
- * and false if it needs to be emulated in user space.
+ * returns 0 if the MMIO access has been performed in kernel space,
+ * and 1 if it needs to be emulated in user space.
* Calls the actual handling routine for the selected VGIC model.
*/
-bool vgic_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
- struct kvm_exit_mmio *mmio)
+static int vgic_io_dev_access(struct kvm_vcpu *vcpu, struct kvm_io_device
*this,
+ gpa_t addr, int len, void *val, bool is_write)
{
- if (!irqchip_in_kernel(vcpu->kvm))
- return false;
+ struct kvm_exit_mmio mmio;
+ bool ret;
+
+ mmio = (struct kvm_exit_mmio) {
+ .phys_addr = addr,
+ .len = len,
+ .is_write = is_write,
+ };
+
+ if (is_write)
+ memcpy(mmio.data, val, len);
/*
* This will currently call either vgic_v2_handle_mmio() or
* vgic_v3_handle_mmio(), which in turn will call
* vgic_handle_mmio_range() defined above.
*/
- return vcpu->kvm->arch.vgic.vm_ops.handle_mmio(vcpu, run, mmio);
+ ret = vcpu->kvm->arch.vgic.vm_ops.handle_mmio(vcpu, vcpu->run, &mmio);
+
+ if (!is_write)
+ memcpy(val, mmio.data, len);
+
+ return ret ? 0 : 1;
+}
+
+static int vgic_io_dev_read(struct kvm_vcpu *vcpu, struct kvm_io_device *this,
+ gpa_t addr, int len, void *val)
+{
+ return vgic_io_dev_access(vcpu, this, addr, len, val, false);
+}
+
+static int vgic_io_dev_write(struct kvm_vcpu *vcpu, struct kvm_io_device *this,
+ gpa_t addr, int len, const void *val)
+{
+ return vgic_io_dev_access(vcpu, this, addr, len, (void *)val, true);
+}
+
+static const struct kvm_io_device_ops vgic_io_dev_ops = {
+ .read = vgic_io_dev_read,
+ .write = vgic_io_dev_write,
+};
+
+static int vgic_register_kvm_i
[PATCH v2 0/5] ARM: KVM: Enable the ioeventfd capability of KVM on ARM
The IOEVENTFD KVM capability is a prerequisite for vhost support. This series enables the ioeventfd KVM capability on ARM. The implementation routes MMIO access in the IO abort handler to the KVM IO bus. If there is already a registered ioeventfd handler for this address, the file descriptor will be triggered. We extended the KVM IO bus API to expose the VCPU struct pointer. Now the VGIC MMIO access is done through this API. For this to operate the VGIC registers a kvm_io_device which represents the whole dist MMIO region. The patches are implemented on top of the latest Andre's vGICv3 work from here: http://www.linux-arm.org/git?p=linux-ap.git;a=shortlog;h=refs/heads/kvm-gicv3/v4 The code was tested on Dual Cortex-A15 Exynos5250 (ARM Chromebook). ARM64 build was verified, but not run on actual HW. Changes since v1: - fixed x86 compilation - GICv3/GICv3 dist base selection - added vgic_unregister_kvm_io_dev to free the iodev resources - enable eventfd on ARM64 --- Nikolay Nikolaev (5): KVM: Redesign kvm_io_bus_ API to pass VCPU structure to the callbacks. KVM: ARM: on IO mem abort - route the call to KVM MMIO bus KVM: ARM VGIC add kvm_io_bus_ frontend ARM/ARM64: enable linking against eventfd ARM: enable KVM_CAP_IOEVENTFD arch/arm/kvm/Kconfig |1 arch/arm/kvm/Makefile |2 - arch/arm/kvm/arm.c |3 + arch/arm/kvm/mmio.c| 32 +++ arch/arm64/kvm/Kconfig |1 arch/arm64/kvm/Makefile|2 - arch/ia64/kvm/kvm-ia64.c |4 + arch/powerpc/kvm/mpic.c| 10 ++- arch/powerpc/kvm/powerpc.c |4 + arch/s390/kvm/diag.c |2 - arch/x86/kvm/i8254.c | 14 +++-- arch/x86/kvm/i8259.c | 12 ++-- arch/x86/kvm/lapic.c |4 + arch/x86/kvm/vmx.c |2 - arch/x86/kvm/x86.c | 13 ++--- include/kvm/arm_vgic.h |3 - include/linux/kvm_host.h | 10 ++- virt/kvm/arm/vgic.c| 127 +--- virt/kvm/coalesced_mmio.c |5 +- virt/kvm/eventfd.c |4 + virt/kvm/ioapic.c |8 +-- virt/kvm/iodev.h | 23 +--- virt/kvm/kvm_main.c| 32 ++- 23 files changed, 237 insertions(+), 81 deletions(-) -- Signature -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 4/5] ARM/ARM64: enable linking against eventfd
This enables compilation of the eventfd feature on ARM/ARM64. Signed-off-by: Nikolay Nikolaev --- arch/arm/kvm/Kconfig|1 + arch/arm/kvm/Makefile |2 +- arch/arm64/kvm/Kconfig |1 + arch/arm64/kvm/Makefile |2 +- 4 files changed, 4 insertions(+), 2 deletions(-) diff --git a/arch/arm/kvm/Kconfig b/arch/arm/kvm/Kconfig index 466bd29..a4b0312 100644 --- a/arch/arm/kvm/Kconfig +++ b/arch/arm/kvm/Kconfig @@ -20,6 +20,7 @@ config KVM bool "Kernel-based Virtual Machine (KVM) support" select PREEMPT_NOTIFIERS select ANON_INODES + select HAVE_KVM_EVENTFD select HAVE_KVM_CPU_RELAX_INTERCEPT select KVM_MMIO select KVM_ARM_HOST diff --git a/arch/arm/kvm/Makefile b/arch/arm/kvm/Makefile index 443b8be..539c1a5 100644 --- a/arch/arm/kvm/Makefile +++ b/arch/arm/kvm/Makefile @@ -15,7 +15,7 @@ AFLAGS_init.o := -Wa,-march=armv7-a$(plus_virt) AFLAGS_interrupts.o := -Wa,-march=armv7-a$(plus_virt) KVM := ../../../virt/kvm -kvm-arm-y = $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o +kvm-arm-y = $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o $(KVM)/eventfd.o obj-y += kvm-arm.o init.o interrupts.o obj-y += arm.o handle_exit.o guest.o mmu.o emulate.o reset.o diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig index 8ba85e9..cb839d0 100644 --- a/arch/arm64/kvm/Kconfig +++ b/arch/arm64/kvm/Kconfig @@ -21,6 +21,7 @@ config KVM select MMU_NOTIFIER select PREEMPT_NOTIFIERS select ANON_INODES + select HAVE_KVM_EVENTFD select HAVE_KVM_CPU_RELAX_INTERCEPT select KVM_MMIO select KVM_ARM_HOST diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile index 4e6e09e..0dffb5f 100644 --- a/arch/arm64/kvm/Makefile +++ b/arch/arm64/kvm/Makefile @@ -11,7 +11,7 @@ ARM=../../../arch/arm/kvm obj-$(CONFIG_KVM_ARM_HOST) += kvm.o -kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o +kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o $(KVM)/eventfd.o kvm-$(CONFIG_KVM_ARM_HOST) += $(ARM)/arm.o $(ARM)/mmu.o $(ARM)/mmio.o kvm-$(CONFIG_KVM_ARM_HOST) += $(ARM)/psci.o $(ARM)/perf.o -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v2 2/5] KVM: ARM: on IO mem abort - route the call to KVM MMIO bus
On IO memory abort, try to handle the MMIO access thorugh the KVM
registered read/write callbacks. This is done by invoking the relevant
kvm_io_bus_* API.
Signed-off-by: Nikolay Nikolaev
---
arch/arm/kvm/mmio.c | 33 +
1 file changed, 33 insertions(+)
diff --git a/arch/arm/kvm/mmio.c b/arch/arm/kvm/mmio.c
index 4cb5a93..e42469f 100644
--- a/arch/arm/kvm/mmio.c
+++ b/arch/arm/kvm/mmio.c
@@ -162,6 +162,36 @@ static int decode_hsr(struct kvm_vcpu *vcpu, phys_addr_t
fault_ipa,
return 0;
}
+/**
+ * handle_kernel_mmio - handle an in-kernel MMIO access
+ * @vcpu: pointer to the vcpu performing the access
+ * @run: pointer to the kvm_run structure
+ * @mmio: pointer to the data describing the access
+ *
+ * returns true if the MMIO access has been performed in kernel space,
+ * and false if it needs to be emulated in user space.
+ */
+static bool handle_kernel_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
+ struct kvm_exit_mmio *mmio)
+{
+ int ret;
+
+ if (mmio->is_write) {
+ ret = kvm_io_bus_write(vcpu, KVM_MMIO_BUS, mmio->phys_addr,
+ mmio->len, &mmio->data);
+
+ } else {
+ ret = kvm_io_bus_read(vcpu, KVM_MMIO_BUS, mmio->phys_addr,
+ mmio->len, &mmio->data);
+ }
+ if (!ret) {
+ kvm_prepare_mmio(run, mmio);
+ kvm_handle_mmio_return(vcpu, run);
+ }
+
+ return !ret;
+}
+
int io_mem_abort(struct kvm_vcpu *vcpu, struct kvm_run *run,
phys_addr_t fault_ipa)
{
@@ -200,6 +230,9 @@ int io_mem_abort(struct kvm_vcpu *vcpu, struct kvm_run *run,
if (vgic_handle_mmio(vcpu, run, &mmio))
return 1;
+ if (handle_kernel_mmio(vcpu, run, &mmio))
+ return 1;
+
kvm_prepare_mmio(run, &mmio);
return 0;
}
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC PATCH 4/5] ARM: enable linking against eventfd and irqchip
On Sat, Nov 29, 2014 at 1:29 PM, Christoffer Dall wrote: > Inidicate KVM in the subject please. > > On Mon, Nov 24, 2014 at 11:27:06PM +0200, Nikolay Nikolaev wrote: >> This enables compilation of the eventfd feature on ARM. > > does this enable anything else than the ioeventfd stuff so that > everything is in place to do this safely now? Inspecting the eventfd.c I see that there is the IRQFD code conditionally compiled when CONFIG_HAVE_KVM_IRQFD is defined. I think this is enabled for arm/arm64 in Eric's IRQFD patcheries, so we should be safe now. regards, Nikolay Nikolaev > > -Christoffer -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC PATCH 3/5] KVM: ARM VGIC add kvm_io_bus_ frontend
On Sat, Nov 29, 2014 at 3:54 PM, Nikolay Nikolaev
wrote:
> On Sat, Nov 29, 2014 at 1:29 PM, Christoffer Dall
> wrote:
>> On Mon, Nov 24, 2014 at 11:26:58PM +0200, Nikolay Nikolaev wrote:
>>> In io_mem_abort remove the call to vgic_handle_mmio. The target is to have
>>> a single MMIO handling path - that is through the kvm_io_bus_ API.
>>>
>>> Register a kvm_io_device in kvm_vgic_init on the whole vGIC MMIO region.
>>> Both read and write calls are redirected to vgic_io_dev_access where
>>> kvm_exit_mmio is composed to pass it to vm_ops.handle_mmio.
>>>
>>>
>>> Signed-off-by: Nikolay Nikolaev
>>> ---
>>> arch/arm/kvm/mmio.c|3 --
>>> include/kvm/arm_vgic.h |3 +-
>>> virt/kvm/arm/vgic.c| 88
>>>
>>> 3 files changed, 74 insertions(+), 20 deletions(-)
>>>
>>> diff --git a/arch/arm/kvm/mmio.c b/arch/arm/kvm/mmio.c
>>> index 81230da..1c44a2b 100644
>>> --- a/arch/arm/kvm/mmio.c
>>> +++ b/arch/arm/kvm/mmio.c
>>> @@ -227,9 +227,6 @@ int io_mem_abort(struct kvm_vcpu *vcpu, struct kvm_run
>>> *run,
>>> if (mmio.is_write)
>>> mmio_write_buf(mmio.data, mmio.len, data);
>>>
>>> - if (vgic_handle_mmio(vcpu, run, &mmio))
>>> - return 1;
>>> -
>>> if (handle_kernel_mmio(vcpu, run, &mmio))
>>> return 1;
>>>
>>> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
>>> index e452ef7..d9b7d2a 100644
>>> --- a/include/kvm/arm_vgic.h
>>> +++ b/include/kvm/arm_vgic.h
>>> @@ -233,6 +233,7 @@ struct vgic_dist {
>>> unsigned long *irq_pending_on_cpu;
>>>
>>> struct vgic_vm_ops vm_ops;
>>> + struct kvm_io_device*io_dev;
>>> #endif
>>> };
>>>
>>> @@ -307,8 +308,6 @@ int kvm_vgic_inject_irq(struct kvm *kvm, int cpuid,
>>> unsigned int irq_num,
>>> bool level);
>>> void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg);
>>> int kvm_vgic_vcpu_pending_irq(struct kvm_vcpu *vcpu);
>>> -bool vgic_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
>>> - struct kvm_exit_mmio *mmio);
>>>
>>> #define irqchip_in_kernel(k) (!!((k)->arch.vgic.in_kernel))
>>> #define vgic_initialized(k) ((k)->arch.vgic.ready)
>>> diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
>>> index 1213da5..3da1115 100644
>>> --- a/virt/kvm/arm/vgic.c
>>> +++ b/virt/kvm/arm/vgic.c
>>> @@ -31,6 +31,9 @@
>>> #include
>>> #include
>>> #include
>>> +#include
>>> +
>>> +#include "iodev.h"
>>>
>>> /*
>>> * How the whole thing works (courtesy of Christoffer Dall):
>>> @@ -775,28 +778,81 @@ bool vgic_handle_mmio_range(struct kvm_vcpu *vcpu,
>>> struct kvm_run *run,
>>> return true;
>>> }
>>>
>>> -/**
>>> - * vgic_handle_mmio - handle an in-kernel MMIO access for the GIC emulation
>>> - * @vcpu: pointer to the vcpu performing the access
>>> - * @run: pointer to the kvm_run structure
>>> - * @mmio: pointer to the data describing the access
>>> - *
>>> - * returns true if the MMIO access has been performed in kernel space,
>>> - * and false if it needs to be emulated in user space.
>>> - * Calls the actual handling routine for the selected VGIC model.
>>> - */
>>> -bool vgic_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
>>> - struct kvm_exit_mmio *mmio)
>>> +static int vgic_io_dev_access(struct kvm_vcpu *vcpu, struct kvm_io_device
>>> *this,
>>> + gpa_t addr, int len, void *val, bool is_write)
>>> {
>>> - if (!irqchip_in_kernel(vcpu->kvm))
>>> - return false;
>>> + struct kvm_exit_mmio mmio;
>>> + bool ret;
>>> +
>>> + mmio = (struct kvm_exit_mmio) {
>>> + .phys_addr = addr,
>>> + .len = len,
>>> + .is_write = is_write,
>>> + };
>>> +
>>> + if (is_write)
>>> + memcpy(mmio.data, val, len);
>>>
>>> /*
>>>* This will currently call either vgic_v2_handle_mmio() or
>>>* vgic_v3_handle_mmio(), which in
Re: [RFC PATCH 2/5] ARM: on IO mem abort - route the call to KVM MMIO bus
On Sat, Nov 29, 2014 at 1:28 PM, Christoffer Dall
wrote:
> On Mon, Nov 24, 2014 at 11:26:51PM +0200, Nikolay Nikolaev wrote:
>> On IO memory abort, try to handle the MMIO access thorugh the KVM
>> registered read/write callbacks. This is done by invoking the relevant
>> kvm_io_bus_* API.
>>
>> Signed-off-by: Nikolay Nikolaev
>> ---
>> arch/arm/kvm/mmio.c | 33 +
>> 1 file changed, 33 insertions(+)
>>
>> diff --git a/arch/arm/kvm/mmio.c b/arch/arm/kvm/mmio.c
>> index 4cb5a93..81230da 100644
>> --- a/arch/arm/kvm/mmio.c
>> +++ b/arch/arm/kvm/mmio.c
>> @@ -162,6 +162,36 @@ static int decode_hsr(struct kvm_vcpu *vcpu,
>> phys_addr_t fault_ipa,
>> return 0;
>> }
>>
>> +/**
>> + * kvm_handle_mmio - handle an in-kernel MMIO access
>> + * @vcpu:pointer to the vcpu performing the access
>> + * @run: pointer to the kvm_run structure
>> + * @mmio:pointer to the data describing the access
>> + *
>> + * returns true if the MMIO access has been performed in kernel space,
>> + * and false if it needs to be emulated in user space.
>> + */
>> +static bool handle_kernel_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
>> + struct kvm_exit_mmio *mmio)
>> +{
>> + int ret;
>> +
>> + if (mmio->is_write) {
>> + ret = kvm_io_bus_write(vcpu, KVM_MMIO_BUS, mmio->phys_addr,
>> + mmio->len, &mmio->data);
>> +
>> + } else {
>> + ret = kvm_io_bus_read(vcpu, KVM_MMIO_BUS, mmio->phys_addr,
>> + mmio->len, &mmio->data);
>> + }
>> + if (!ret) {
>> + kvm_prepare_mmio(run, mmio);
>> + kvm_handle_mmio_return(vcpu, run);
>> + }
>> +
>> + return !ret;
>> +}
>> +
>> int io_mem_abort(struct kvm_vcpu *vcpu, struct kvm_run *run,
>>phys_addr_t fault_ipa)
>> {
>> @@ -200,6 +230,9 @@ int io_mem_abort(struct kvm_vcpu *vcpu, struct kvm_run
>> *run,
>> if (vgic_handle_mmio(vcpu, run, &mmio))
>> return 1;
>>
>> + if (handle_kernel_mmio(vcpu, run, &mmio))
>> + return 1;
>> +
>
> Is this stuff always synchronously handled so that the mmio is properly
> populated upon handle_kernel_mmio on reads?
If I get it right the kvm_io_bus_ API is intended to work
synchronously. Of course it probably depends
on how the registered device handles the read/write call.
Or maybe I misunderstand your question? Please clarify in that case.
regards,
Nikolay Nikolaev
>
> -Christoffer
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC PATCH 3/5] KVM: ARM VGIC add kvm_io_bus_ frontend
On Sat, Nov 29, 2014 at 9:14 AM, Shannon Zhao wrote:
> On 2014/11/25 5:26, Nikolay Nikolaev wrote:
>> In io_mem_abort remove the call to vgic_handle_mmio. The target is to have
>> a single MMIO handling path - that is through the kvm_io_bus_ API.
>>
>> Register a kvm_io_device in kvm_vgic_init on the whole vGIC MMIO region.
>> Both read and write calls are redirected to vgic_io_dev_access where
>> kvm_exit_mmio is composed to pass it to vm_ops.handle_mmio.
>>
>>
>> Signed-off-by: Nikolay Nikolaev
>> ---
>> arch/arm/kvm/mmio.c|3 --
>> include/kvm/arm_vgic.h |3 +-
>> virt/kvm/arm/vgic.c| 88
>>
>> 3 files changed, 74 insertions(+), 20 deletions(-)
>>
>> diff --git a/arch/arm/kvm/mmio.c b/arch/arm/kvm/mmio.c
>> index 81230da..1c44a2b 100644
>> --- a/arch/arm/kvm/mmio.c
>> +++ b/arch/arm/kvm/mmio.c
>> @@ -227,9 +227,6 @@ int io_mem_abort(struct kvm_vcpu *vcpu, struct kvm_run
>> *run,
>> if (mmio.is_write)
>> mmio_write_buf(mmio.data, mmio.len, data);
>>
>> - if (vgic_handle_mmio(vcpu, run, &mmio))
>> - return 1;
>> -
>> if (handle_kernel_mmio(vcpu, run, &mmio))
>> return 1;
>>
>> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
>> index e452ef7..d9b7d2a 100644
>> --- a/include/kvm/arm_vgic.h
>> +++ b/include/kvm/arm_vgic.h
>> @@ -233,6 +233,7 @@ struct vgic_dist {
>> unsigned long *irq_pending_on_cpu;
>>
>> struct vgic_vm_ops vm_ops;
>> + struct kvm_io_device*io_dev;
>> #endif
>> };
>>
>> @@ -307,8 +308,6 @@ int kvm_vgic_inject_irq(struct kvm *kvm, int cpuid,
>> unsigned int irq_num,
>> bool level);
>> void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg);
>> int kvm_vgic_vcpu_pending_irq(struct kvm_vcpu *vcpu);
>> -bool vgic_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
>> - struct kvm_exit_mmio *mmio);
>>
>> #define irqchip_in_kernel(k) (!!((k)->arch.vgic.in_kernel))
>> #define vgic_initialized(k) ((k)->arch.vgic.ready)
>> diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
>> index 1213da5..3da1115 100644
>> --- a/virt/kvm/arm/vgic.c
>> +++ b/virt/kvm/arm/vgic.c
>> @@ -31,6 +31,9 @@
>> #include
>> #include
>> #include
>> +#include
>> +
>> +#include "iodev.h"
>>
>> /*
>> * How the whole thing works (courtesy of Christoffer Dall):
>> @@ -775,28 +778,81 @@ bool vgic_handle_mmio_range(struct kvm_vcpu *vcpu,
>> struct kvm_run *run,
>> return true;
>> }
>>
>> -/**
>> - * vgic_handle_mmio - handle an in-kernel MMIO access for the GIC emulation
>> - * @vcpu: pointer to the vcpu performing the access
>> - * @run: pointer to the kvm_run structure
>> - * @mmio: pointer to the data describing the access
>> - *
>> - * returns true if the MMIO access has been performed in kernel space,
>> - * and false if it needs to be emulated in user space.
>> - * Calls the actual handling routine for the selected VGIC model.
>> - */
>> -bool vgic_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
>> - struct kvm_exit_mmio *mmio)
>> +static int vgic_io_dev_access(struct kvm_vcpu *vcpu, struct kvm_io_device
>> *this,
>> + gpa_t addr, int len, void *val, bool is_write)
>> {
>> - if (!irqchip_in_kernel(vcpu->kvm))
>> - return false;
>> + struct kvm_exit_mmio mmio;
>> + bool ret;
>> +
>> + mmio = (struct kvm_exit_mmio) {
>> + .phys_addr = addr,
>> + .len = len,
>> + .is_write = is_write,
>> + };
>> +
>> + if (is_write)
>> + memcpy(mmio.data, val, len);
>>
>> /*
>>* This will currently call either vgic_v2_handle_mmio() or
>>* vgic_v3_handle_mmio(), which in turn will call
>>* vgic_handle_mmio_range() defined above.
>>*/
>> - return vcpu->kvm->arch.vgic.vm_ops.handle_mmio(vcpu, run, mmio);
>> + ret = vcpu->kvm->arch.vgic.vm_ops.handle_mmio(vcpu, vcpu->run, &mmio);
>> +
>> + if (!is_write)
>> + memcpy(val, mmio.data, len);
>> +
>> + return ret ? 0 : 1;
>> +}
>> +
>> +static int vgic_io_dev_read(struct kvm_vcpu *vcpu, struc
Re: [RFC PATCH 3/5] KVM: ARM VGIC add kvm_io_bus_ frontend
On Sat, Nov 29, 2014 at 1:29 PM, Christoffer Dall
wrote:
> On Mon, Nov 24, 2014 at 11:26:58PM +0200, Nikolay Nikolaev wrote:
>> In io_mem_abort remove the call to vgic_handle_mmio. The target is to have
>> a single MMIO handling path - that is through the kvm_io_bus_ API.
>>
>> Register a kvm_io_device in kvm_vgic_init on the whole vGIC MMIO region.
>> Both read and write calls are redirected to vgic_io_dev_access where
>> kvm_exit_mmio is composed to pass it to vm_ops.handle_mmio.
>>
>>
>> Signed-off-by: Nikolay Nikolaev
>> ---
>> arch/arm/kvm/mmio.c|3 --
>> include/kvm/arm_vgic.h |3 +-
>> virt/kvm/arm/vgic.c| 88
>>
>> 3 files changed, 74 insertions(+), 20 deletions(-)
>>
>> diff --git a/arch/arm/kvm/mmio.c b/arch/arm/kvm/mmio.c
>> index 81230da..1c44a2b 100644
>> --- a/arch/arm/kvm/mmio.c
>> +++ b/arch/arm/kvm/mmio.c
>> @@ -227,9 +227,6 @@ int io_mem_abort(struct kvm_vcpu *vcpu, struct kvm_run
>> *run,
>> if (mmio.is_write)
>> mmio_write_buf(mmio.data, mmio.len, data);
>>
>> - if (vgic_handle_mmio(vcpu, run, &mmio))
>> - return 1;
>> -
>> if (handle_kernel_mmio(vcpu, run, &mmio))
>> return 1;
>>
>> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
>> index e452ef7..d9b7d2a 100644
>> --- a/include/kvm/arm_vgic.h
>> +++ b/include/kvm/arm_vgic.h
>> @@ -233,6 +233,7 @@ struct vgic_dist {
>> unsigned long *irq_pending_on_cpu;
>>
>> struct vgic_vm_ops vm_ops;
>> + struct kvm_io_device*io_dev;
>> #endif
>> };
>>
>> @@ -307,8 +308,6 @@ int kvm_vgic_inject_irq(struct kvm *kvm, int cpuid,
>> unsigned int irq_num,
>> bool level);
>> void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg);
>> int kvm_vgic_vcpu_pending_irq(struct kvm_vcpu *vcpu);
>> -bool vgic_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
>> - struct kvm_exit_mmio *mmio);
>>
>> #define irqchip_in_kernel(k) (!!((k)->arch.vgic.in_kernel))
>> #define vgic_initialized(k) ((k)->arch.vgic.ready)
>> diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
>> index 1213da5..3da1115 100644
>> --- a/virt/kvm/arm/vgic.c
>> +++ b/virt/kvm/arm/vgic.c
>> @@ -31,6 +31,9 @@
>> #include
>> #include
>> #include
>> +#include
>> +
>> +#include "iodev.h"
>>
>> /*
>> * How the whole thing works (courtesy of Christoffer Dall):
>> @@ -775,28 +778,81 @@ bool vgic_handle_mmio_range(struct kvm_vcpu *vcpu,
>> struct kvm_run *run,
>> return true;
>> }
>>
>> -/**
>> - * vgic_handle_mmio - handle an in-kernel MMIO access for the GIC emulation
>> - * @vcpu: pointer to the vcpu performing the access
>> - * @run: pointer to the kvm_run structure
>> - * @mmio: pointer to the data describing the access
>> - *
>> - * returns true if the MMIO access has been performed in kernel space,
>> - * and false if it needs to be emulated in user space.
>> - * Calls the actual handling routine for the selected VGIC model.
>> - */
>> -bool vgic_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
>> - struct kvm_exit_mmio *mmio)
>> +static int vgic_io_dev_access(struct kvm_vcpu *vcpu, struct kvm_io_device
>> *this,
>> + gpa_t addr, int len, void *val, bool is_write)
>> {
>> - if (!irqchip_in_kernel(vcpu->kvm))
>> - return false;
>> + struct kvm_exit_mmio mmio;
>> + bool ret;
>> +
>> + mmio = (struct kvm_exit_mmio) {
>> + .phys_addr = addr,
>> + .len = len,
>> + .is_write = is_write,
>> + };
>> +
>> + if (is_write)
>> + memcpy(mmio.data, val, len);
>>
>> /*
>>* This will currently call either vgic_v2_handle_mmio() or
>>* vgic_v3_handle_mmio(), which in turn will call
>>* vgic_handle_mmio_range() defined above.
>>*/
>> - return vcpu->kvm->arch.vgic.vm_ops.handle_mmio(vcpu, run, mmio);
>> + ret = vcpu->kvm->arch.vgic.vm_ops.handle_mmio(vcpu, vcpu->run, &mmio);
>> +
>> + if (!is_write)
>> + memcpy(val, mmio.data, len);
>> +
>> + return ret ? 0 : 1;
>> +}
>> +
>> +static int vgic_io_dev_read(struct kvm_vcpu *
Re: [RFC PATCH 4/5] ARM: enable linking against eventfd and irqchip
On Sat, Nov 29, 2014 at 9:18 AM, Shannon Zhao wrote: > > On 2014/11/25 5:27, Nikolay Nikolaev wrote: > > This enables compilation of the eventfd feature on ARM. > > > > Only enable on ARM? I think we should enable it on ARM64 as well because the > eventfd featrue is common for ARM32 and ARM64. If course - thanks for pointing this out. regards, Nikolay Nikolaev > > Thanks, > Shannon > > > Signed-off-by: Nikolay Nikolaev > > --- > > arch/arm/kvm/Kconfig |1 + > > arch/arm/kvm/Makefile |2 +- > > 2 files changed, 2 insertions(+), 1 deletion(-) > > > > diff --git a/arch/arm/kvm/Kconfig b/arch/arm/kvm/Kconfig > > index 466bd29..a4b0312 100644 > > --- a/arch/arm/kvm/Kconfig > > +++ b/arch/arm/kvm/Kconfig > > @@ -20,6 +20,7 @@ config KVM > > bool "Kernel-based Virtual Machine (KVM) support" > > select PREEMPT_NOTIFIERS > > select ANON_INODES > > + select HAVE_KVM_EVENTFD > > select HAVE_KVM_CPU_RELAX_INTERCEPT > > select KVM_MMIO > > select KVM_ARM_HOST > > diff --git a/arch/arm/kvm/Makefile b/arch/arm/kvm/Makefile > > index 443b8be..539c1a5 100644 > > --- a/arch/arm/kvm/Makefile > > +++ b/arch/arm/kvm/Makefile > > @@ -15,7 +15,7 @@ AFLAGS_init.o := -Wa,-march=armv7-a$(plus_virt) > > AFLAGS_interrupts.o := -Wa,-march=armv7-a$(plus_virt) > > > > KVM := ../../../virt/kvm > > -kvm-arm-y = $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o > > +kvm-arm-y = $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o $(KVM)/eventfd.o > > > > obj-y += kvm-arm.o init.o interrupts.o > > obj-y += arm.o handle_exit.o guest.o mmu.o emulate.o reset.o > > > > ___ > > kvmarm mailing list > > kvm...@lists.cs.columbia.edu > > https://lists.cs.columbia.edu/mailman/listinfo/kvmarm > > > > > -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[RFC PATCH 2/5] ARM: on IO mem abort - route the call to KVM MMIO bus
On IO memory abort, try to handle the MMIO access thorugh the KVM
registered read/write callbacks. This is done by invoking the relevant
kvm_io_bus_* API.
Signed-off-by: Nikolay Nikolaev
---
arch/arm/kvm/mmio.c | 33 +
1 file changed, 33 insertions(+)
diff --git a/arch/arm/kvm/mmio.c b/arch/arm/kvm/mmio.c
index 4cb5a93..81230da 100644
--- a/arch/arm/kvm/mmio.c
+++ b/arch/arm/kvm/mmio.c
@@ -162,6 +162,36 @@ static int decode_hsr(struct kvm_vcpu *vcpu, phys_addr_t
fault_ipa,
return 0;
}
+/**
+ * kvm_handle_mmio - handle an in-kernel MMIO access
+ * @vcpu: pointer to the vcpu performing the access
+ * @run: pointer to the kvm_run structure
+ * @mmio: pointer to the data describing the access
+ *
+ * returns true if the MMIO access has been performed in kernel space,
+ * and false if it needs to be emulated in user space.
+ */
+static bool handle_kernel_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
+ struct kvm_exit_mmio *mmio)
+{
+ int ret;
+
+ if (mmio->is_write) {
+ ret = kvm_io_bus_write(vcpu, KVM_MMIO_BUS, mmio->phys_addr,
+ mmio->len, &mmio->data);
+
+ } else {
+ ret = kvm_io_bus_read(vcpu, KVM_MMIO_BUS, mmio->phys_addr,
+ mmio->len, &mmio->data);
+ }
+ if (!ret) {
+ kvm_prepare_mmio(run, mmio);
+ kvm_handle_mmio_return(vcpu, run);
+ }
+
+ return !ret;
+}
+
int io_mem_abort(struct kvm_vcpu *vcpu, struct kvm_run *run,
phys_addr_t fault_ipa)
{
@@ -200,6 +230,9 @@ int io_mem_abort(struct kvm_vcpu *vcpu, struct kvm_run *run,
if (vgic_handle_mmio(vcpu, run, &mmio))
return 1;
+ if (handle_kernel_mmio(vcpu, run, &mmio))
+ return 1;
+
kvm_prepare_mmio(run, &mmio);
return 0;
}
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[RFC PATCH 1/5] KVM: redesing kvm_io_bus_ API to pass VCPU structure to the callbacks.
This is needed in e.g. ARM vGIC emulation, where the MMIO handling
depends on the VCPU that does the access.
Signed-off-by: Nikolay Nikolaev
---
arch/ia64/kvm/kvm-ia64.c |4 ++--
arch/powerpc/kvm/powerpc.c |4 ++--
arch/s390/kvm/diag.c |2 +-
arch/x86/kvm/vmx.c |2 +-
arch/x86/kvm/x86.c | 11 ++-
include/linux/kvm_host.h | 10 +-
virt/kvm/coalesced_mmio.c |5 +++--
virt/kvm/eventfd.c |4 ++--
virt/kvm/iodev.h | 23 +++
virt/kvm/kvm_main.c| 32
10 files changed, 53 insertions(+), 44 deletions(-)
diff --git a/arch/ia64/kvm/kvm-ia64.c b/arch/ia64/kvm/kvm-ia64.c
index ec6b9ac..16f1d26 100644
--- a/arch/ia64/kvm/kvm-ia64.c
+++ b/arch/ia64/kvm/kvm-ia64.c
@@ -246,10 +246,10 @@ static int handle_mmio(struct kvm_vcpu *vcpu, struct
kvm_run *kvm_run)
return 0;
mmio:
if (p->dir)
- r = kvm_io_bus_read(vcpu->kvm, KVM_MMIO_BUS, p->addr,
+ r = kvm_io_bus_read(vcpu, KVM_MMIO_BUS, p->addr,
p->size, &p->data);
else
- r = kvm_io_bus_write(vcpu->kvm, KVM_MMIO_BUS, p->addr,
+ r = kvm_io_bus_write(vcpu, KVM_MMIO_BUS, p->addr,
p->size, &p->data);
if (r)
printk(KERN_ERR"kvm: No iodevice found! addr:%lx\n", p->addr);
diff --git a/arch/powerpc/kvm/powerpc.c b/arch/powerpc/kvm/powerpc.c
index c1f8f53..5ac065b 100644
--- a/arch/powerpc/kvm/powerpc.c
+++ b/arch/powerpc/kvm/powerpc.c
@@ -814,7 +814,7 @@ int kvmppc_handle_load(struct kvm_run *run, struct kvm_vcpu
*vcpu,
idx = srcu_read_lock(&vcpu->kvm->srcu);
- ret = kvm_io_bus_read(vcpu->kvm, KVM_MMIO_BUS, run->mmio.phys_addr,
+ ret = kvm_io_bus_read(vcpu, KVM_MMIO_BUS, run->mmio.phys_addr,
bytes, &run->mmio.data);
srcu_read_unlock(&vcpu->kvm->srcu, idx);
@@ -887,7 +887,7 @@ int kvmppc_handle_store(struct kvm_run *run, struct
kvm_vcpu *vcpu,
idx = srcu_read_lock(&vcpu->kvm->srcu);
- ret = kvm_io_bus_write(vcpu->kvm, KVM_MMIO_BUS, run->mmio.phys_addr,
+ ret = kvm_io_bus_write(vcpu, KVM_MMIO_BUS, run->mmio.phys_addr,
bytes, &run->mmio.data);
srcu_read_unlock(&vcpu->kvm->srcu, idx);
diff --git a/arch/s390/kvm/diag.c b/arch/s390/kvm/diag.c
index 9254aff..329ec75 100644
--- a/arch/s390/kvm/diag.c
+++ b/arch/s390/kvm/diag.c
@@ -213,7 +213,7 @@ static int __diag_virtio_hypercall(struct kvm_vcpu *vcpu)
* - gpr 3 contains the virtqueue index (passed as datamatch)
* - gpr 4 contains the index on the bus (optionally)
*/
- ret = kvm_io_bus_write_cookie(vcpu->kvm, KVM_VIRTIO_CCW_NOTIFY_BUS,
+ ret = kvm_io_bus_write_cookie(vcpu, KVM_VIRTIO_CCW_NOTIFY_BUS,
vcpu->run->s.regs.gprs[2] & 0x,
8, &vcpu->run->s.regs.gprs[3],
vcpu->run->s.regs.gprs[4]);
diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index 3e556c6..e6d9f01 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -5620,7 +5620,7 @@ static int handle_ept_misconfig(struct kvm_vcpu *vcpu)
gpa_t gpa;
gpa = vmcs_read64(GUEST_PHYSICAL_ADDRESS);
- if (!kvm_io_bus_write(vcpu->kvm, KVM_FAST_MMIO_BUS, gpa, 0, NULL)) {
+ if (!kvm_io_bus_write(vcpu, KVM_FAST_MMIO_BUS, gpa, 0, NULL)) {
skip_emulated_instruction(vcpu);
return 1;
}
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 0033df3..bbf9375 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -4053,7 +4053,7 @@ static int vcpu_mmio_write(struct kvm_vcpu *vcpu, gpa_t
addr, int len,
n = min(len, 8);
if (!(vcpu->arch.apic &&
!kvm_iodevice_write(&vcpu->arch.apic->dev, addr, n, v))
- && kvm_io_bus_write(vcpu->kvm, KVM_MMIO_BUS, addr, n, v))
+ && kvm_io_bus_write(vcpu, KVM_MMIO_BUS, addr, n, v))
break;
handled += n;
addr += n;
@@ -4072,8 +4072,9 @@ static int vcpu_mmio_read(struct kvm_vcpu *vcpu, gpa_t
addr, int len, void *v)
do {
n = min(len, 8);
if (!(vcpu->arch.apic &&
- !kvm_iodevice_read(&vcpu->arch.apic->dev, addr, n, v))
- && kvm_io_bus_read(vcpu->kvm, KVM_MMIO_BUS, addr, n, v))
+ !kvm_iodevice_read(vcpu, &vcpu->arch.apic->dev,
+addr, n, v))
+ && kv
[RFC PATCH 4/5] ARM: enable linking against eventfd and irqchip
This enables compilation of the eventfd feature on ARM. Signed-off-by: Nikolay Nikolaev --- arch/arm/kvm/Kconfig |1 + arch/arm/kvm/Makefile |2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/arm/kvm/Kconfig b/arch/arm/kvm/Kconfig index 466bd29..a4b0312 100644 --- a/arch/arm/kvm/Kconfig +++ b/arch/arm/kvm/Kconfig @@ -20,6 +20,7 @@ config KVM bool "Kernel-based Virtual Machine (KVM) support" select PREEMPT_NOTIFIERS select ANON_INODES + select HAVE_KVM_EVENTFD select HAVE_KVM_CPU_RELAX_INTERCEPT select KVM_MMIO select KVM_ARM_HOST diff --git a/arch/arm/kvm/Makefile b/arch/arm/kvm/Makefile index 443b8be..539c1a5 100644 --- a/arch/arm/kvm/Makefile +++ b/arch/arm/kvm/Makefile @@ -15,7 +15,7 @@ AFLAGS_init.o := -Wa,-march=armv7-a$(plus_virt) AFLAGS_interrupts.o := -Wa,-march=armv7-a$(plus_virt) KVM := ../../../virt/kvm -kvm-arm-y = $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o +kvm-arm-y = $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o $(KVM)/eventfd.o obj-y += kvm-arm.o init.o interrupts.o obj-y += arm.o handle_exit.o guest.o mmu.o emulate.o reset.o -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[RFC PATCH 5/5] ARM: enable KVM_CAP_IOEVENTFD
KVM on arm will support the eventfd extension. Signed-off-by: Nikolay Nikolaev --- arch/arm/kvm/arm.c |3 +++ 1 file changed, 3 insertions(+) diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c index c3d0fbd..266b618 100644 --- a/arch/arm/kvm/arm.c +++ b/arch/arm/kvm/arm.c @@ -197,6 +197,9 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) case KVM_CAP_MAX_VCPUS: r = KVM_MAX_VCPUS; break; + case KVM_CAP_IOEVENTFD: + r = 1; + break; default: r = kvm_arch_dev_ioctl_check_extension(ext); break; -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[RFC PATCH 3/5] KVM: ARM VGIC add kvm_io_bus_ frontend
In io_mem_abort remove the call to vgic_handle_mmio. The target is to have
a single MMIO handling path - that is through the kvm_io_bus_ API.
Register a kvm_io_device in kvm_vgic_init on the whole vGIC MMIO region.
Both read and write calls are redirected to vgic_io_dev_access where
kvm_exit_mmio is composed to pass it to vm_ops.handle_mmio.
Signed-off-by: Nikolay Nikolaev
---
arch/arm/kvm/mmio.c|3 --
include/kvm/arm_vgic.h |3 +-
virt/kvm/arm/vgic.c| 88
3 files changed, 74 insertions(+), 20 deletions(-)
diff --git a/arch/arm/kvm/mmio.c b/arch/arm/kvm/mmio.c
index 81230da..1c44a2b 100644
--- a/arch/arm/kvm/mmio.c
+++ b/arch/arm/kvm/mmio.c
@@ -227,9 +227,6 @@ int io_mem_abort(struct kvm_vcpu *vcpu, struct kvm_run *run,
if (mmio.is_write)
mmio_write_buf(mmio.data, mmio.len, data);
- if (vgic_handle_mmio(vcpu, run, &mmio))
- return 1;
-
if (handle_kernel_mmio(vcpu, run, &mmio))
return 1;
diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index e452ef7..d9b7d2a 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -233,6 +233,7 @@ struct vgic_dist {
unsigned long *irq_pending_on_cpu;
struct vgic_vm_ops vm_ops;
+ struct kvm_io_device*io_dev;
#endif
};
@@ -307,8 +308,6 @@ int kvm_vgic_inject_irq(struct kvm *kvm, int cpuid,
unsigned int irq_num,
bool level);
void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg);
int kvm_vgic_vcpu_pending_irq(struct kvm_vcpu *vcpu);
-bool vgic_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
- struct kvm_exit_mmio *mmio);
#define irqchip_in_kernel(k) (!!((k)->arch.vgic.in_kernel))
#define vgic_initialized(k)((k)->arch.vgic.ready)
diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
index 1213da5..3da1115 100644
--- a/virt/kvm/arm/vgic.c
+++ b/virt/kvm/arm/vgic.c
@@ -31,6 +31,9 @@
#include
#include
#include
+#include
+
+#include "iodev.h"
/*
* How the whole thing works (courtesy of Christoffer Dall):
@@ -775,28 +778,81 @@ bool vgic_handle_mmio_range(struct kvm_vcpu *vcpu, struct
kvm_run *run,
return true;
}
-/**
- * vgic_handle_mmio - handle an in-kernel MMIO access for the GIC emulation
- * @vcpu: pointer to the vcpu performing the access
- * @run: pointer to the kvm_run structure
- * @mmio: pointer to the data describing the access
- *
- * returns true if the MMIO access has been performed in kernel space,
- * and false if it needs to be emulated in user space.
- * Calls the actual handling routine for the selected VGIC model.
- */
-bool vgic_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
- struct kvm_exit_mmio *mmio)
+static int vgic_io_dev_access(struct kvm_vcpu *vcpu, struct kvm_io_device
*this,
+ gpa_t addr, int len, void *val, bool is_write)
{
- if (!irqchip_in_kernel(vcpu->kvm))
- return false;
+ struct kvm_exit_mmio mmio;
+ bool ret;
+
+ mmio = (struct kvm_exit_mmio) {
+ .phys_addr = addr,
+ .len = len,
+ .is_write = is_write,
+ };
+
+ if (is_write)
+ memcpy(mmio.data, val, len);
/*
* This will currently call either vgic_v2_handle_mmio() or
* vgic_v3_handle_mmio(), which in turn will call
* vgic_handle_mmio_range() defined above.
*/
- return vcpu->kvm->arch.vgic.vm_ops.handle_mmio(vcpu, run, mmio);
+ ret = vcpu->kvm->arch.vgic.vm_ops.handle_mmio(vcpu, vcpu->run, &mmio);
+
+ if (!is_write)
+ memcpy(val, mmio.data, len);
+
+ return ret ? 0 : 1;
+}
+
+static int vgic_io_dev_read(struct kvm_vcpu *vcpu, struct kvm_io_device *this,
+ gpa_t addr, int len, void *val)
+{
+ return vgic_io_dev_access(vcpu, this, addr, len, val, false);
+}
+
+static int vgic_io_dev_write(struct kvm_vcpu *vcpu, struct kvm_io_device *this,
+ gpa_t addr, int len, const void *val)
+{
+ return vgic_io_dev_access(vcpu, this, addr, len, (void *)val, true);
+}
+
+static const struct kvm_io_device_ops vgic_io_dev_ops = {
+ .read = vgic_io_dev_read,
+ .write = vgic_io_dev_write,
+};
+
+static int vgic_register_kvm_io_dev(struct kvm *kvm)
+{
+ struct kvm_io_device *dev;
+ int ret;
+
+ struct vgic_dist *dist = &kvm->arch.vgic;
+ unsigned long base = dist->vgic_dist_base;
+
+ dev = kzalloc(sizeof(struct kvm_io_device), GFP_KERNEL);
+ if (!dev)
+ return -ENOMEM;
+
+ kvm_iodevice_init(dev, &vgic_io_dev_ops);
+
+ mutex_lock(&kvm->slots_lock);
+
+ ret = kvm_io_bus_register_dev(kvm, KVM_MMIO_BUS,
+ base, KVM
[RFC PATCH 0/5] ARM: KVM: Enable the ioeventfd capability of KVM on ARM
The IOEVENTFD KVM capability is a prerequisite for vhost support, and is also used to implement improved interrupt handling in VFIO drivers. This series enables the ioeventfd KVM capability on ARM. The implementation routes MMIO access in the IO abort handler to the KVM IO bus. If there is already a registered ioeventfd handler for this address, the file descriptor will be triggered. We extended the KVM IO bus API to expose the VCPU struct pointer. Now the VGIC MMIO access is done through this API. For this to operate the VGIC registers a kvm_io_device which reprresents the whole MMIO region. The patches are implemented on top of the latest Andre's vGICv3 work from here: http://www.linux-arm.org/git?p=linux-ap.git;a=shortlog;h=refs/heads/kvm-gicv3/v4 The code was tested on Dual Cortex-A15 Exynos5250 (ARM Chromebook). --- Nikolay Nikolaev (5): KVM: redesing kvm_io_bus_ API to pass VCPU structure to the callbacks. ARM: on IO mem abort - route the call to KVM MMIO bus KVM: ARM VGIC add kvm_io_bus_ frontend ARM: enable linking against eventfd and irqchip ARM: enable KVM_CAP_IOEVENTFD arch/arm/kvm/Kconfig |1 + arch/arm/kvm/Makefile |2 + arch/arm/kvm/arm.c |3 ++ arch/arm/kvm/mmio.c| 32 arch/ia64/kvm/kvm-ia64.c |4 +- arch/powerpc/kvm/powerpc.c |4 +- arch/s390/kvm/diag.c |2 + arch/x86/kvm/vmx.c |2 + arch/x86/kvm/x86.c | 11 +++--- include/kvm/arm_vgic.h |3 +- include/linux/kvm_host.h | 10 +++-- virt/kvm/arm/vgic.c| 88 +--- virt/kvm/coalesced_mmio.c |5 ++- virt/kvm/eventfd.c |4 +- virt/kvm/iodev.h | 23 virt/kvm/kvm_main.c| 32 16 files changed, 163 insertions(+), 63 deletions(-) -- Signature -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [RFC PATCH 1/4] ARM: KVM: on unhandled IO mem abort, route the call to the KVM MMIO bus
On Thu, Nov 13, 2014 at 4:23 PM, Eric Auger wrote:
> On 11/13/2014 03:16 PM, Eric Auger wrote:
>> On 11/13/2014 11:45 AM, Nikolay Nikolaev wrote:
>>> On Mon, Nov 10, 2014 at 6:27 PM, Christoffer Dall
>>> wrote:
>>>> On Mon, Nov 10, 2014 at 05:09:07PM +0200, Nikolay Nikolaev wrote:
>>>>> Hello,
>>>>>
>>>>> On Fri, Mar 28, 2014 at 9:09 PM, Christoffer Dall
>>>>> wrote:
>>>>>>
>>>>>> On Thu, Mar 13, 2014 at 04:57:26PM +0100, Antonios Motakis wrote:
>>>>>>> On an unhandled IO memory abort, use the kvm_io_bus_* API in order to
>>>>>>> handle the MMIO access through any registered read/write callbacks. This
>>>>>>> is a dependency for eventfd support (ioeventfd and irqfd).
>>>>>>>
>>>>>>> However, accesses to the VGIC are still left implemented independently,
>>>>>>> since the kvm_io_bus_* API doesn't pass the VCPU pointer doing the
>>>>>>> access.
>>>>>>>
>>>>>>> Signed-off-by: Antonios Motakis
>>>>>>> Signed-off-by: Nikolay Nikolaev
>>>>>>> ---
>>>>>>> arch/arm/kvm/mmio.c | 32
>>>>>>> virt/kvm/arm/vgic.c | 5 -
>>>>>>> 2 files changed, 36 insertions(+), 1 deletion(-)
>>>>>>>
>>>>>>> diff --git a/arch/arm/kvm/mmio.c b/arch/arm/kvm/mmio.c
>>>>>>> index 4cb5a93..1d17831 100644
>>>>>>> --- a/arch/arm/kvm/mmio.c
>>>>>>> +++ b/arch/arm/kvm/mmio.c
>>>>>>> @@ -162,6 +162,35 @@ static int decode_hsr(struct kvm_vcpu *vcpu,
>>>>>>> phys_addr_t fault_ipa,
>>>>>>> return 0;
>>>>>>> }
>>>>>>>
>>>>>>> +/**
>>>>>>> + * kvm_handle_mmio - handle an in-kernel MMIO access
>>>>>>> + * @vcpu:pointer to the vcpu performing the access
>>>>>>> + * @run: pointer to the kvm_run structure
>>>>>>> + * @mmio:pointer to the data describing the access
>>>>>>> + *
>>>>>>> + * returns true if the MMIO access has been performed in kernel space,
>>>>>>> + * and false if it needs to be emulated in user space.
>>>>>>> + */
>>>>>>> +static bool handle_kernel_mmio(struct kvm_vcpu *vcpu, struct kvm_run
>>>>>>> *run,
>>>>>>> + struct kvm_exit_mmio *mmio)
>>>>>>> +{
>>>>>>> + int ret;
>>>>>>> + if (mmio->is_write) {
>>>>>>> + ret = kvm_io_bus_write(vcpu->kvm, KVM_MMIO_BUS,
>>>>>>> mmio->phys_addr,
>>>>>>> + mmio->len, &mmio->data);
>>>>>>> +
>>>>>>> + } else {
>>>>>>> + ret = kvm_io_bus_read(vcpu->kvm, KVM_MMIO_BUS,
>>>>>>> mmio->phys_addr,
>>>>>>> + mmio->len, &mmio->data);
>>>>>>> + }
>>>>>>> + if (!ret) {
>>>>>>> + kvm_prepare_mmio(run, mmio);
>>>>>>> + kvm_handle_mmio_return(vcpu, run);
>>>>>>> + }
>>>>>>> +
>>>>>>> + return !ret;
>>>>>>> +}
>>>>>>> +
>>>>>>> int io_mem_abort(struct kvm_vcpu *vcpu, struct kvm_run *run,
>>>>>>>phys_addr_t fault_ipa)
>>>>>>> {
>>>>>>> @@ -200,6 +229,9 @@ int io_mem_abort(struct kvm_vcpu *vcpu, struct
>>>>>>> kvm_run *run,
>>>>>>> if (vgic_handle_mmio(vcpu, run, &mmio))
>>>>>>> return 1;
>>>>>>>
>>>>>>> + if (handle_kernel_mmio(vcpu, run, &mmio))
>>>>>>> + return 1;
>>>>>>> +
>>>>>
>>>>>
>>>>> We're reconsidering ioeventfds patchseries and we tried to evaluate
>>>>> what you suggested here.
>>>>>
>>>>>>
>>>>>> this special-casing of the vgic is now really terrible. Is there
>
Re: [RFC PATCH 1/4] ARM: KVM: on unhandled IO mem abort, route the call to the KVM MMIO bus
On Thu, Nov 13, 2014 at 1:52 PM, Andre Przywara wrote: > Hi Nikolay, > > On 13/11/14 11:37, Marc Zyngier wrote: >> [fixing Andre's email address] >> >> On 13/11/14 11:20, Christoffer Dall wrote: >>> On Thu, Nov 13, 2014 at 12:45:42PM +0200, Nikolay Nikolaev wrote: >>> >>> [...] >>> >>>>>> >>>>>> Going through the vgic_handle_mmio we see that it will require large >>>>>> refactoring: >>>>>> - there are 15 MMIO ranges for the vgic now - each should be >>>>>> registered as a separate device >>>>>> - the handler of each range should be split into read and write >>>>>> - all handlers take 'struct kvm_exit_mmio', and pass it to >>>>>> 'vgic_reg_access', 'mmio_data_read' and 'mmio_data_read' >>>>>> >>>>>> To sum up - if we do this refactoring of vgic's MMIO handling + >>>>>> kvm_io_bus_ API getting 'vcpu" argument we'll get a 'much' cleaner >>>>>> vgic code and as a bonus we'll get 'ioeventfd' capabilities. >>>>>> >>>>>> We have 3 questions: >>>>>> - is the kvm_io_bus_ getting 'vcpu' argument acceptable for the other >>>>>> architectures too? >>>>>> - is this huge vgic MMIO handling redesign acceptable/desired (it >>>>>> touches a lot of code)? >>>>>> - is there a way that ioeventfd is accepted leaving vgic.c in it's >>>>>> current state? >>>>>> >>>>> Not sure how the latter question is relevant to this, but check with >>>>> Andre who recently looked at this as well and decided that for GICv3 the >>>>> only sane thing was to remove that comment for the gic. >>>> @Andre - what's your experience with the GICv3 and MMIO handling, >>>> anything specific? >>>>> >>>>> I don't recall the details of what you were trying to accomplish here >>>>> (it's been 8 months or so) but the surely the vgic handling code should >>>>> *somehow* be integrated into the handle_kernel_mmio (like Paolo >>>>> suggested), unless you come back and tell me that that would involve a >>>>> complete rewrite of the vgic code. >>>> I'm experimenting now - it's not exactly rewrite of whole vgic code, >>>> but it will touch a lot of it - all MMIO access handlers and the >>>> supporting functions. >>>> We're ready to spend the effort. My question is - is this acceptable? >>>> >>> I certainly appreciate the offer to do this work, but it's hard to say >>> at this point if it is worth it. >>> >>> What I was trying to say above is that Andre looked at this, and came to >>> the conclusion that it is not worth it. >>> >>> Marc, what are your thoughts? >> >> Same here, I rely on Andre's view that it was not very useful. Now, it >> would be good to see a mock-up of the patches and find out: > > Seconded, can you send a pointer to the VGIC rework patches mentioned? They are still in WiP state - not exactly working. I'm still exploring what the status is. Our major target is having ioeventfd suport in ARM. For this we need to support kvm_io_bus_ mechanisms for MMIO access (cause ioevent fd device is registered this way). Then this subject of integrating vgic with the kvm_io_bus_ APIs came up. My personal opinion - they should be able to coexist in peace. > >> - if it is a major improvement for the general quality of the code >> - if that allow us to *delete* a lot of code (if it is just churn, I'm >> not really interested) >> - if it helps or hinders further developments that are currently in the >> pipeline >> >> Andre, can you please share your findings? I don't remember the >> specifics of the discussion we had a few months ago... > > 1) Given the date in the reply I sense that your patches are from March > this year or earlier. So this is based on VGIC code from March, which > predates Marc's vgic_dyn changes that just went in 3.18-rc1? His patches > introduced another member to struct mmio_range to check validity of > accesses with a reduced number of SPIs supported (.bits_per_irq). > So is this covered in your rework? Still no (rebased to 3.17) - didn't see it, but should not be an issue. > > 2) >>>> - there are 15 MMIO ranges for the vgic now - each sho
Re: [RFC PATCH 1/4] ARM: KVM: on unhandled IO mem abort, route the call to the KVM MMIO bus
On Mon, Nov 10, 2014 at 6:27 PM, Christoffer Dall
wrote:
> On Mon, Nov 10, 2014 at 05:09:07PM +0200, Nikolay Nikolaev wrote:
>> Hello,
>>
>> On Fri, Mar 28, 2014 at 9:09 PM, Christoffer Dall
>> wrote:
>> >
>> > On Thu, Mar 13, 2014 at 04:57:26PM +0100, Antonios Motakis wrote:
>> > > On an unhandled IO memory abort, use the kvm_io_bus_* API in order to
>> > > handle the MMIO access through any registered read/write callbacks. This
>> > > is a dependency for eventfd support (ioeventfd and irqfd).
>> > >
>> > > However, accesses to the VGIC are still left implemented independently,
>> > > since the kvm_io_bus_* API doesn't pass the VCPU pointer doing the
>> > > access.
>> > >
>> > > Signed-off-by: Antonios Motakis
>> > > Signed-off-by: Nikolay Nikolaev
>> > > ---
>> > > arch/arm/kvm/mmio.c | 32
>> > > virt/kvm/arm/vgic.c | 5 -
>> > > 2 files changed, 36 insertions(+), 1 deletion(-)
>> > >
>> > > diff --git a/arch/arm/kvm/mmio.c b/arch/arm/kvm/mmio.c
>> > > index 4cb5a93..1d17831 100644
>> > > --- a/arch/arm/kvm/mmio.c
>> > > +++ b/arch/arm/kvm/mmio.c
>> > > @@ -162,6 +162,35 @@ static int decode_hsr(struct kvm_vcpu *vcpu,
>> > > phys_addr_t fault_ipa,
>> > > return 0;
>> > > }
>> > >
>> > > +/**
>> > > + * kvm_handle_mmio - handle an in-kernel MMIO access
>> > > + * @vcpu:pointer to the vcpu performing the access
>> > > + * @run: pointer to the kvm_run structure
>> > > + * @mmio:pointer to the data describing the access
>> > > + *
>> > > + * returns true if the MMIO access has been performed in kernel space,
>> > > + * and false if it needs to be emulated in user space.
>> > > + */
>> > > +static bool handle_kernel_mmio(struct kvm_vcpu *vcpu, struct kvm_run
>> > > *run,
>> > > + struct kvm_exit_mmio *mmio)
>> > > +{
>> > > + int ret;
>> > > + if (mmio->is_write) {
>> > > + ret = kvm_io_bus_write(vcpu->kvm, KVM_MMIO_BUS,
>> > > mmio->phys_addr,
>> > > + mmio->len, &mmio->data);
>> > > +
>> > > + } else {
>> > > + ret = kvm_io_bus_read(vcpu->kvm, KVM_MMIO_BUS,
>> > > mmio->phys_addr,
>> > > + mmio->len, &mmio->data);
>> > > + }
>> > > + if (!ret) {
>> > > + kvm_prepare_mmio(run, mmio);
>> > > + kvm_handle_mmio_return(vcpu, run);
>> > > + }
>> > > +
>> > > + return !ret;
>> > > +}
>> > > +
>> > > int io_mem_abort(struct kvm_vcpu *vcpu, struct kvm_run *run,
>> > >phys_addr_t fault_ipa)
>> > > {
>> > > @@ -200,6 +229,9 @@ int io_mem_abort(struct kvm_vcpu *vcpu, struct
>> > > kvm_run *run,
>> > > if (vgic_handle_mmio(vcpu, run, &mmio))
>> > > return 1;
>> > >
>> > > + if (handle_kernel_mmio(vcpu, run, &mmio))
>> > > + return 1;
>> > > +
>>
>>
>> We're reconsidering ioeventfds patchseries and we tried to evaluate
>> what you suggested here.
>>
>> >
>> > this special-casing of the vgic is now really terrible. Is there
>> > anything holding you back from doing the necessary restructure of the
>> > kvm_bus_io_*() API instead?
>>
>> Restructuring the kvm_io_bus_ API is not a big thing (we actually did
>> it), but is not directly related to the these patches.
>> Of course it can be justified if we do it in the context of removing
>> vgic_handle_mmio and leaving only handle_kernel_mmio.
>>
>> >
>> > That would allow us to get rid of the ugly
>> > Fix it! in the vgic driver as well.
>>
>> Going through the vgic_handle_mmio we see that it will require large
>> refactoring:
>> - there are 15 MMIO ranges for the vgic now - each should be
>> registered as a separate device
>> - the handler of each range should be split into read and write
>> - all handlers take 'struct kvm_exit_mmio', and pass it to
>> 'vgic_reg_access', 'mmio_data_read' and 'mmio_data_read'
>
Re: [RFC PATCH 1/4] ARM: KVM: on unhandled IO mem abort, route the call to the KVM MMIO bus
Hello,
On Fri, Mar 28, 2014 at 9:09 PM, Christoffer Dall
wrote:
>
> On Thu, Mar 13, 2014 at 04:57:26PM +0100, Antonios Motakis wrote:
> > On an unhandled IO memory abort, use the kvm_io_bus_* API in order to
> > handle the MMIO access through any registered read/write callbacks. This
> > is a dependency for eventfd support (ioeventfd and irqfd).
> >
> > However, accesses to the VGIC are still left implemented independently,
> > since the kvm_io_bus_* API doesn't pass the VCPU pointer doing the access.
> >
> > Signed-off-by: Antonios Motakis
> > Signed-off-by: Nikolay Nikolaev
> > ---
> > arch/arm/kvm/mmio.c | 32
> > virt/kvm/arm/vgic.c | 5 -
> > 2 files changed, 36 insertions(+), 1 deletion(-)
> >
> > diff --git a/arch/arm/kvm/mmio.c b/arch/arm/kvm/mmio.c
> > index 4cb5a93..1d17831 100644
> > --- a/arch/arm/kvm/mmio.c
> > +++ b/arch/arm/kvm/mmio.c
> > @@ -162,6 +162,35 @@ static int decode_hsr(struct kvm_vcpu *vcpu,
> > phys_addr_t fault_ipa,
> > return 0;
> > }
> >
> > +/**
> > + * kvm_handle_mmio - handle an in-kernel MMIO access
> > + * @vcpu:pointer to the vcpu performing the access
> > + * @run: pointer to the kvm_run structure
> > + * @mmio:pointer to the data describing the access
> > + *
> > + * returns true if the MMIO access has been performed in kernel space,
> > + * and false if it needs to be emulated in user space.
> > + */
> > +static bool handle_kernel_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
> > + struct kvm_exit_mmio *mmio)
> > +{
> > + int ret;
> > + if (mmio->is_write) {
> > + ret = kvm_io_bus_write(vcpu->kvm, KVM_MMIO_BUS,
> > mmio->phys_addr,
> > + mmio->len, &mmio->data);
> > +
> > + } else {
> > + ret = kvm_io_bus_read(vcpu->kvm, KVM_MMIO_BUS,
> > mmio->phys_addr,
> > + mmio->len, &mmio->data);
> > + }
> > + if (!ret) {
> > + kvm_prepare_mmio(run, mmio);
> > + kvm_handle_mmio_return(vcpu, run);
> > + }
> > +
> > + return !ret;
> > +}
> > +
> > int io_mem_abort(struct kvm_vcpu *vcpu, struct kvm_run *run,
> >phys_addr_t fault_ipa)
> > {
> > @@ -200,6 +229,9 @@ int io_mem_abort(struct kvm_vcpu *vcpu, struct kvm_run
> > *run,
> > if (vgic_handle_mmio(vcpu, run, &mmio))
> > return 1;
> >
> > + if (handle_kernel_mmio(vcpu, run, &mmio))
> > + return 1;
> > +
We're reconsidering ioeventfds patchseries and we tried to evaluate
what you suggested here.
>
> this special-casing of the vgic is now really terrible. Is there
> anything holding you back from doing the necessary restructure of the
> kvm_bus_io_*() API instead?
Restructuring the kvm_io_bus_ API is not a big thing (we actually did
it), but is not directly related to the these patches.
Of course it can be justified if we do it in the context of removing
vgic_handle_mmio and leaving only handle_kernel_mmio.
>
> That would allow us to get rid of the ugly
> Fix it! in the vgic driver as well.
Going through the vgic_handle_mmio we see that it will require large
refactoring:
- there are 15 MMIO ranges for the vgic now - each should be
registered as a separate device
- the handler of each range should be split into read and write
- all handlers take 'struct kvm_exit_mmio', and pass it to
'vgic_reg_access', 'mmio_data_read' and 'mmio_data_read'
To sum up - if we do this refactoring of vgic's MMIO handling +
kvm_io_bus_ API getting 'vcpu" argument we'll get a 'much' cleaner
vgic code and as a bonus we'll get 'ioeventfd' capabilities.
We have 3 questions:
- is the kvm_io_bus_ getting 'vcpu' argument acceptable for the other
architectures too?
- is this huge vgic MMIO handling redesign acceptable/desired (it
touches a lot of code)?
- is there a way that ioeventfd is accepted leaving vgic.c in it's
current state?
regards,
Nikolay NIkolaev
Virtual Open Systems
>
> -Christoffer
>
> > kvm_prepare_mmio(run, &mmio);
> > return 0;
> > }
> > diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
> > index 8ca405c..afdecc3 100644
> > --- a/virt/kvm/arm/vgic.c
> > +++ b/virt/kvm/arm/vgic.c
> > @@ -849,13 +849,16 @@ struct mmio_range *find_matching_range(const struct
> > mmio_range *ranges,
> > }
> >
> > /*
Re: The status about vhost-net on kvm-arm?
On Wed, Aug 13, 2014 at 12:10 PM, Nikolay Nikolaev wrote: > On Tue, Aug 12, 2014 at 6:47 PM, Nikolay Nikolaev > wrote: >> >> Hello, >> >> >> On Tue, Aug 12, 2014 at 5:41 AM, Li Liu wrote: >> > >> > Hi all, >> > >> > Is anyone there can tell the current status of vhost-net on kvm-arm? >> > >> > Half a year has passed from Isa Ansharullah asked this question: >> > http://www.spinics.net/lists/kvm-arm/msg08152.html >> > >> > I have found two patches which have provided the kvm-arm support of >> > eventfd and irqfd: >> > >> > 1) [RFC PATCH 0/4] ARM: KVM: Enable the ioeventfd capability of KVM on ARM >> > http://lists.gnu.org/archive/html/qemu-devel/2014-01/msg01770.html >> > >> > 2) [RFC,v3] ARM: KVM: add irqfd and irq routing support >> > https://patches.linaro.org/32261/ >> > >> > And there's a rough patch for qemu to support eventfd from Ying-Shiuan Pan: >> > >> > [Qemu-devel] [PATCH 0/4] ioeventfd support for virtio-mmio >> > https://lists.gnu.org/archive/html/qemu-devel/2014-02/msg00715.html >> > >> > But there no any comments of this patch. And I can found nothing about qemu >> > to support irqfd. Do I lost the track? >> > >> > If nobody try to fix it. We have a plan to complete it about virtio-mmio >> > supporing irqfd and multiqueue. >> > >> > >> >> we at Virtual Open Systems did some work and tested vhost-net on ARM >> back in March. >> The setup was based on: >> - host kernel with our ioeventfd patches: >> http://www.spinics.net/lists/kvm-arm/msg08413.html >> >> - qemu with the aforementioned patches from Ying-Shiuan Pan >> https://lists.gnu.org/archive/html/qemu-devel/2014-02/msg00715.html >> >> The testbed was ARM Chromebook with Exynos 5250, using a 1Gbps USB3 >> Ethernet adapter connected to a 1Gbps switch. I can't find the actual >> numbers but I remember that with multiple streams the gain was clearly >> seen. Note that it used the minimum required ioventfd implementation >> and not irqfd. >> >> I guess it is feasible to think that it all can be put together and >> rebased + the recent irqfd work. One can achiev even better >> performance (because of the irqfd). >> > > Managed to replicate the setup with the old versions e used in March: > > Single stream from another machine to chromebook with 1Gbps USB3 > Ethernet adapter. > iperf -c -P 1 -i 1 -p 5001 -f k -t 10 > to HOST: 858316 Kbits/sec > to GUEST: 761563 Kbits/sec to GUEST vhost=off: 508150 Kbits/sec > > 10 parallel streams > iperf -c -P 10 -i 1 -p 5001 -f k -t 10 > to HOST: 842420 Kbits/sec > to GUEST: 625144 Kbits/sec to GUEST vhost=off: 425276 Kbits/sec > >> >> > >> > >> > >> > >> > >> > ___ >> > kvmarm mailing list >> > kvm...@lists.cs.columbia.edu >> > https://lists.cs.columbia.edu/mailman/listinfo/kvmarm >> >> >> regards, >> Nikolay Nikolaev >> Virtual Open Systems -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: The status about vhost-net on kvm-arm?
On Tue, Aug 12, 2014 at 6:47 PM, Nikolay Nikolaev wrote: > > Hello, > > > On Tue, Aug 12, 2014 at 5:41 AM, Li Liu wrote: > > > > Hi all, > > > > Is anyone there can tell the current status of vhost-net on kvm-arm? > > > > Half a year has passed from Isa Ansharullah asked this question: > > http://www.spinics.net/lists/kvm-arm/msg08152.html > > > > I have found two patches which have provided the kvm-arm support of > > eventfd and irqfd: > > > > 1) [RFC PATCH 0/4] ARM: KVM: Enable the ioeventfd capability of KVM on ARM > > http://lists.gnu.org/archive/html/qemu-devel/2014-01/msg01770.html > > > > 2) [RFC,v3] ARM: KVM: add irqfd and irq routing support > > https://patches.linaro.org/32261/ > > > > And there's a rough patch for qemu to support eventfd from Ying-Shiuan Pan: > > > > [Qemu-devel] [PATCH 0/4] ioeventfd support for virtio-mmio > > https://lists.gnu.org/archive/html/qemu-devel/2014-02/msg00715.html > > > > But there no any comments of this patch. And I can found nothing about qemu > > to support irqfd. Do I lost the track? > > > > If nobody try to fix it. We have a plan to complete it about virtio-mmio > > supporing irqfd and multiqueue. > > > > > > we at Virtual Open Systems did some work and tested vhost-net on ARM > back in March. > The setup was based on: > - host kernel with our ioeventfd patches: > http://www.spinics.net/lists/kvm-arm/msg08413.html > > - qemu with the aforementioned patches from Ying-Shiuan Pan > https://lists.gnu.org/archive/html/qemu-devel/2014-02/msg00715.html > > The testbed was ARM Chromebook with Exynos 5250, using a 1Gbps USB3 > Ethernet adapter connected to a 1Gbps switch. I can't find the actual > numbers but I remember that with multiple streams the gain was clearly > seen. Note that it used the minimum required ioventfd implementation > and not irqfd. > > I guess it is feasible to think that it all can be put together and > rebased + the recent irqfd work. One can achiev even better > performance (because of the irqfd). > Managed to replicate the setup with the old versions e used in March: Single stream from another machine to chromebook with 1Gbps USB3 Ethernet adapter. iperf -c -P 1 -i 1 -p 5001 -f k -t 10 to HOST: 858316 Kbits/sec to GUEST: 761563 Kbits/sec 10 parallel streams iperf -c -P 10 -i 1 -p 5001 -f k -t 10 to HOST: 842420 Kbits/sec to GUEST: 625144 Kbits/sec > > > > > > > > > > > > > ___ > > kvmarm mailing list > > kvm...@lists.cs.columbia.edu > > https://lists.cs.columbia.edu/mailman/listinfo/kvmarm > > > regards, > Nikolay Nikolaev > Virtual Open Systems -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: The status about vhost-net on kvm-arm?
Hello, On Tue, Aug 12, 2014 at 5:41 AM, Li Liu wrote: > > Hi all, > > Is anyone there can tell the current status of vhost-net on kvm-arm? > > Half a year has passed from Isa Ansharullah asked this question: > http://www.spinics.net/lists/kvm-arm/msg08152.html > > I have found two patches which have provided the kvm-arm support of > eventfd and irqfd: > > 1) [RFC PATCH 0/4] ARM: KVM: Enable the ioeventfd capability of KVM on ARM > http://lists.gnu.org/archive/html/qemu-devel/2014-01/msg01770.html > > 2) [RFC,v3] ARM: KVM: add irqfd and irq routing support > https://patches.linaro.org/32261/ > > And there's a rough patch for qemu to support eventfd from Ying-Shiuan Pan: > > [Qemu-devel] [PATCH 0/4] ioeventfd support for virtio-mmio > https://lists.gnu.org/archive/html/qemu-devel/2014-02/msg00715.html > > But there no any comments of this patch. And I can found nothing about qemu > to support irqfd. Do I lost the track? > > If nobody try to fix it. We have a plan to complete it about virtio-mmio > supporing irqfd and multiqueue. > > we at Virtual Open Systems did some work and tested vhost-net on ARM back in March. The setup was based on: - host kernel with our ioeventfd patches: http://www.spinics.net/lists/kvm-arm/msg08413.html - qemu with the aforementioned patches from Ying-Shiuan Pan https://lists.gnu.org/archive/html/qemu-devel/2014-02/msg00715.html The testbed was ARM Chromebook with Exynos 5250, using a 1Gbps USB3 Ethernet adapter connected to a 1Gbps switch. I can't find the actual numbers but I remember that with multiple streams the gain was clearly seen. Note that it used the minimum required ioventfd implementation and not irqfd. I guess it is feasible to think that it all can be put together and rebased + the recent irqfd work. One can achiev even better performance (because of the irqfd). > > > > > > ___ > kvmarm mailing list > kvm...@lists.cs.columbia.edu > https://lists.cs.columbia.edu/mailman/listinfo/kvmarm regards, Nikolay Nikolaev Virtual Open Systems -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
