[PATCH v11 00/21] KVM: ARM64: Add guest PMU support

From: Shannon Zhao This patchset adds guest PMU support for KVM on ARM64. It takes trap-and-emulate approach. When guest wants to monitor one event, it will be trapped by KVM and KVM will call perf_event API to create a perf event and call relevant perf_event APIs to get the count value of event.

[PATCH v11 18/21] KVM: ARM64: Free perf event of PMU when destroying vcpu

From: Shannon Zhao When KVM frees VCPU, it needs to free the perf_event of PMU. Signed-off-by: Shannon Zhao Reviewed-by: Marc Zyngier Reviewed-by: Andrew Jones --- arch/arm/kvm/arm.c| 1 + include/kvm/arm_pmu.h | 2 ++ virt/kvm/arm/pmu.c| 21 + 3 files changed,

[PATCH v11 15/21] KVM: ARM64: Add access handler for PMUSERENR register

From: Shannon Zhao This register resets as unknown in 64bit mode while it resets as zero in 32bit mode. Here we choose to reset it as zero for consistency. PMUSERENR_EL0 holds some bits which decide whether PMU registers can be accessed from EL0. Add some check helpers to handle the access from

[PATCH v11 13/21] KVM: ARM64: Add access handler for PMSWINC register

From: Shannon Zhao Add access handler which emulates writing and reading PMSWINC register and add support for creating software increment event. Signed-off-by: Shannon Zhao Reviewed-by: Andrew Jones --- arch/arm64/include/asm/pmu.h | 2 ++ arch/arm64/kvm/sys_regs.c| 20 ++

[PATCH v11 16/21] KVM: ARM64: Add PMU overflow interrupt routing

From: Shannon Zhao When calling perf_event_create_kernel_counter to create perf_event, assign a overflow handler. Then when the perf event overflows, set the corresponding bit of guest PMOVSSET register. If this counter is enabled and its interrupt is enabled as well, kick the vcpu to sync the in

[PATCH v11 07/21] KVM: ARM64: PMU: Add perf event map and introduce perf event creating function

From: Shannon Zhao When we use tools like perf on host, perf passes the event type and the id of this event type category to kernel, then kernel will map them to hardware event number and write this number to PMU PMEVTYPER_EL0 register. When getting the event number in KVM, directly use raw event

[PATCH v11 04/21] KVM: ARM64: Add access handler for PMCR register

From: Shannon Zhao Add reset handler which gets host value of PMCR_EL0 and make writable bits architecturally UNKNOWN except PMCR.E which is zero. Add an access handler for PMCR. Signed-off-by: Shannon Zhao Reviewed-by: Andrew Jones --- arch/arm64/kvm/sys_regs.c | 42 +

[PATCH v11 06/21] KVM: ARM64: Add access handler for PMCEID0 and PMCEID1 register

From: Shannon Zhao Add access handler which gets host value of PMCEID0 or PMCEID1 when guest access these registers. Writing action to PMCEID0 or PMCEID1 is UNDEFINED. Signed-off-by: Shannon Zhao --- arch/arm64/kvm/sys_regs.c | 29 + 1 file changed, 25 insertions(+)

[PATCH v11 05/21] KVM: ARM64: Add access handler for PMSELR register

From: Shannon Zhao Since the reset value of PMSELR_EL0 is UNKNOWN, use reset_unknown for its reset handler. When reading PMSELR, return the PMSELR.SEL field to guest. Signed-off-by: Shannon Zhao Reviewed-by: Andrew Jones --- arch/arm64/kvm/sys_regs.c | 19 +-- 1 file changed,

[PATCH v11 14/21] KVM: ARM64: Add helper to handle PMCR register bits

From: Shannon Zhao According to ARMv8 spec, when writing 1 to PMCR.E, all counters are enabled by PMCNTENSET, while writing 0 to PMCR.E, all counters are disabled. When writing 1 to PMCR.P, reset all event counters, not including PMCCNTR, to zero. When writing 1 to PMCR.C, reset PMCCNTR to zero.

[PATCH v11 09/21] KVM: ARM64: Add access handler for event counter register

From: Shannon Zhao These kind of registers include PMEVCNTRn, PMCCNTR and PMXEVCNTR which is mapped to PMEVCNTRn. The access handler translates all aarch32 register offsets to aarch64 ones and uses vcpu_sys_reg() to access their values to avoid taking care of big endian. When reading these regi

[PATCH v11 12/21] KVM: ARM64: Add access handler for PMOVSSET and PMOVSCLR register

From: Shannon Zhao Since the reset value of PMOVSSET and PMOVSCLR is UNKNOWN, use reset_unknown for its reset handler. Add a handler to emulate writing PMOVSSET or PMOVSCLR register. When writing non-zero value to PMOVSSET, the counter and its interrupt is enabled, kick this vcpu to sync PMU int

[PATCH v11 01/21] ARM64: Move PMU register related defines to asm/pmu.h

From: Shannon Zhao To use the ARMv8 PMU related register defines from the KVM code, we move the relevant definitions to asm/pmu.h header file. Signed-off-by: Anup Patel Signed-off-by: Shannon Zhao Acked-by: Marc Zyngier Reviewed-by: Andrew Jones --- arch/arm64/include/asm/pmu.h | 67 +

[PATCH v11 21/21] KVM: ARM64: Add a new vcpu device control group for PMUv3

From: Shannon Zhao To configure the virtual PMUv3 overflow interrupt number, we use the vcpu kvm_device ioctl, encapsulating the KVM_ARM_VCPU_PMU_V3_IRQ attribute within the KVM_ARM_VCPU_PMU_V3_CTRL group. After configuring the PMUv3, call the vcpu ioctl with attribute KVM_ARM_VCPU_PMU_V3_INIT t

[PATCH v11 11/21] KVM: ARM64: Add access handler for PMINTENSET and PMINTENCLR register

From: Shannon Zhao Since the reset value of PMINTENSET and PMINTENCLR is UNKNOWN, use reset_unknown for its reset handler. Add a handler to emulate writing PMINTENSET or PMINTENCLR register. Signed-off-by: Shannon Zhao Reviewed-by: Andrew Jones --- arch/arm64/kvm/sys_regs.c | 32 +

[PATCH v11 19/21] KVM: ARM64: Add a new feature bit for PMUv3

From: Shannon Zhao To support guest PMUv3, use one bit of the VCPU INIT feature array. Initialize the PMU when initialzing the vcpu with that bit and PMU overflow interrupt set. Signed-off-by: Shannon Zhao Acked-by: Peter Maydell Reviewed-by: Andrew Jones --- CC: Peter Maydell --- Documenta

[PATCH v11 20/21] KVM: ARM: Introduce per-vcpu kvm device controls

From: Shannon Zhao In some cases it needs to get/set attributes specific to a vcpu and so needs something else than ONE_REG. Let's copy the KVM_DEVICE approach, and define the respective ioctls for the vcpu file descriptor. Signed-off-by: Shannon Zhao Reviewed-by: Andrew Jones Acked-by: Peter

[PATCH v11 17/21] KVM: ARM64: Reset PMU state when resetting vcpu

From: Shannon Zhao When resetting vcpu, it needs to reset the PMU state to initial status. Signed-off-by: Shannon Zhao Reviewed-by: Marc Zyngier Reviewed-by: Andrew Jones --- arch/arm64/kvm/reset.c | 3 +++ include/kvm/arm_pmu.h | 2 ++ virt/kvm/arm/pmu.c | 17 + 3 fil

[PATCH v11 02/21] KVM: ARM64: Define PMU data structure for each vcpu

From: Shannon Zhao Here we plan to support virtual PMU for guest by full software emulation, so define some basic structs and functions preparing for futher steps. Define struct kvm_pmc for performance monitor counter and struct kvm_pmu for performance monitor unit for each vcpu. According to ARM

[PATCH v11 03/21] KVM: ARM64: Add offset defines for PMU registers

From: Shannon Zhao We are about to trap and emulate accesses to each PMU register individually. This adds the context offsets for the AArch64 PMU registers. Signed-off-by: Shannon Zhao Reviewed-by: Marc Zyngier Reviewed-by: Andrew Jones --- arch/arm64/include/asm/kvm_host.h | 15

[PATCH v11 08/21] KVM: ARM64: Add access handler for event type register

From: Shannon Zhao These kind of registers include PMEVTYPERn, PMCCFILTR and PMXEVTYPER which is mapped to PMEVTYPERn or PMCCFILTR. The access handler translates all aarch32 register offsets to aarch64 ones and uses vcpu_sys_reg() to access their values to avoid taking care of big endian. When

[PATCH v11 10/21] KVM: ARM64: Add access handler for PMCNTENSET and PMCNTENCLR register

From: Shannon Zhao Since the reset value of PMCNTENSET and PMCNTENCLR is UNKNOWN, use reset_unknown for its reset handler. Add a handler to emulate writing PMCNTENSET or PMCNTENCLR register. When writing to PMCNTENSET, call perf_event_enable to enable the perf event. When writing to PMCNTENCLR,

Re: [PATCH v3 00/23] arm64: Virtualization Host Extension support

On Wed, Feb 03, 2016 at 05:59:53PM +, Marc Zyngier wrote: > ARMv8.1 comes with the "Virtualization Host Extension" (VHE for > short), which enables simpler support of Type-2 hypervisors. > > This extension allows the kernel to directly run at EL2, and > significantly reduces the number of syst

Re: [PATCH v3 23/23] arm64: Panic when VHE and non VHE CPUs coexist

On Wed, Feb 03, 2016 at 06:00:16PM +, Marc Zyngier wrote: > Having both VHE and non-VHE capable CPUs in the same system > is likely to be a recipe for disaster. > > If the boot CPU has VHE, but a secondary is not, we won't be > able to downgrade and run the kernel at EL1. Add CPU hotplug > to

Re: [PATCH v3 22/23] arm64: VHE: Add support for running Linux in EL2 mode

On Wed, Feb 03, 2016 at 06:00:15PM +, Marc Zyngier wrote: > With ARMv8.1 VHE, the architecture is able to (almost) transparently > run the kernel at EL2, despite being written for EL1. > > This patch takes care of the "almost" part, mostly preventing the kernel > from dropping from EL2 to EL1,

Re: [PATCH v3 20/23] arm64: perf: Count EL2 events if the kernel is running in HYP

On Wed, Feb 03, 2016 at 06:00:13PM +, Marc Zyngier wrote: > When the kernel is running in HYP (with VHE), it is necessary to > include EL2 events if the user requests counting kernel or > hypervisor events. > > Signed-off-by: Marc Zyngier > --- > arch/arm64/kernel/perf_event.c | 14 +

Re: [PATCH v3 19/23] arm64: KVM: Move most of the fault decoding to C

On Wed, Feb 03, 2016 at 06:00:12PM +, Marc Zyngier wrote: > The fault decoding process (including computing the IPA in the case > of a permission fault) would be much better done in C code, as we > have a reasonable infrastructure to deal with the VHE/non-VHE > differences. > > Let's move the

Re: [PATCH v3 11/23] arm64: KVM: VHE: Split save/restore of registers shared between guest and host

On Wed, Feb 03, 2016 at 06:00:04PM +, Marc Zyngier wrote: > A handful of system registers are still shared between host and guest, > even while using VHE (tpidr*_el[01] and actlr_el1). > > Also, some of the vcpu state (sp_el0, PC and PSTATE) must be > save/restored on entry/exit, as they are u

Re: MPIDR Aff0 question

Hi Drew, On 04/02/16 18:38, Andrew Jones wrote: > > Hi Marc and Andre, > > I completely understand why reset_mpidr() limits Aff0 to 16, thanks > to Andre's nice comment about ICC_SGIxR. Now, here's my question; > it seems that the Cortex-A{53,57,72} manuals want to further limit > Aff0 to 4, goi

MPIDR Aff0 question

Hi Marc and Andre, I completely understand why reset_mpidr() limits Aff0 to 16, thanks to Andre's nice comment about ICC_SGIxR. Now, here's my question; it seems that the Cortex-A{53,57,72} manuals want to further limit Aff0 to 4, going so far as to say bits 7:2 are RES0. I'm looking at userspace

Re: PCIe passthrough support on ARM

Hi Edward, On 02/04/2016 05:53 PM, Edward Cragg wrote: > Hi, > > I'm involved in planning a project for which there is a requirement for PCIe > passthrough in KVM on ARMv8. We have no hardware to test on at the moment. > > I understand that virtualisation support for ARM is quite young, and it se

PCIe passthrough support on ARM

Hi, I'm involved in planning a project for which there is a requirement for PCIe passthrough in KVM on ARMv8. We have no hardware to test on at the moment. I understand that virtualisation support for ARM is quite young, and it seems like support is trickling in at the moment for this sort of thi

Re: [kvm-unit-tests PATCH 07/11] arm/pci: PCI device operation test

On Thu, Feb 04, 2016 at 01:18:38PM +0100, Alexander Gordeev wrote: > On Fri, Jan 15, 2016 at 04:32:11PM +0100, Andrew Jones wrote: > > > +struct pci_testdev_ops { > > > + u8 (*read8)(const volatile void *addr); > > > + u16 (*read16)(const volatile void *addr); > > > + u32 (*read32)(const volatile v

Re: [kvm-unit-tests PATCH 07/11] arm/pci: PCI device operation test

On Fri, Jan 15, 2016 at 04:32:11PM +0100, Andrew Jones wrote: > > +struct pci_testdev_ops { > > + u8 (*read8)(const volatile void *addr); > > + u16 (*read16)(const volatile void *addr); > > + u32 (*read32)(const volatile void *addr); > > + void (*write8)(u8 value, volatile void *addr); > >

Re: [kvm-unit-tests PATCH 08/11] arm/pci: PCI device read/write test

On Fri, Jan 15, 2016 at 04:33:50PM +0100, Andrew Jones wrote: > On Sat, Jan 09, 2016 at 01:22:55PM +0100, Alexander Gordeev wrote: > > Cc: Andrew Jones > > Signed-off-by: Alexander Gordeev > > --- > > lib/pci-testdev.c | 16 > > 1 file changed, 16 insertions(+) > > > > diff --g

[PATCH v2 18/28] ARM: KVM: Add HYP mode entry code

This part is almost entierely borrowed from the existing code, just slightly simplifying the HYP function call (as we now save SPSR_hyp in the world switch). Signed-off-by: Marc Zyngier --- arch/arm/kvm/hyp/Makefile| 1 + arch/arm/kvm/hyp/hyp-entry.S | 157 +

[PATCH v2 23/28] ARM: KVM: Remove __weak attributes

Now that the old code is long gone, we can remove all the weak attributes, as there is only one version of the code. Signed-off-by: Marc Zyngier --- arch/arm/kvm/hyp/hyp-entry.S | 4 +--- arch/arm/kvm/hyp/switch.c| 2 +- arch/arm/kvm/hyp/tlb.c | 6 +++--- 3 files changed, 5 insertions(

[PATCH v2 19/28] ARM: KVM: Add panic handling code

Instead of spinning forever, let's "properly" handle any unexpected exception ("properly" meaning "print a spat on the console and die"). This has proved useful quite a few times... Signed-off-by: Marc Zyngier --- arch/arm/kvm/hyp/hyp-entry.S | 28 +--- arch/arm/kvm/hyp/

[PATCH v2 28/28] ARM: KVM: Remove __kvm_hyp_exit/__kvm_hyp_exit_end

I have no idea what these were for - probably a leftover from an early implementation. Good bye! Signed-off-by: Marc Zyngier --- arch/arm/include/asm/kvm_asm.h | 3 --- 1 file changed, 3 deletions(-) diff --git a/arch/arm/include/asm/kvm_asm.h b/arch/arm/include/asm/kvm_asm.h index 083825f..15d

[PATCH v2 22/28] ARM: KVM: Switch to C-based stage2 init

Signed-off-by: Marc Zyngier --- arch/arm/include/asm/kvm_asm.h | 2 ++ arch/arm/include/asm/kvm_host.h | 1 + arch/arm/kvm/hyp/Makefile | 1 + arch/arm/kvm/hyp/hyp.h | 2 ++ arch/arm/kvm/hyp/s2-setup.c | 34 ++ arch/arm/kvm/init.S

[PATCH v2 26/28] ARM: KVM: Remove unused hyp_pc field

This field was never populated, and the panic code already does something similar. Delete the related code. Signed-off-by: Marc Zyngier --- arch/arm/include/asm/kvm_emulate.h | 5 - arch/arm/include/asm/kvm_host.h| 1 - arch/arm/kernel/asm-offsets.c | 1 - arch/arm/kvm/handle_exit.c

[PATCH v2 24/28] ARM: KVM: Turn CP15 defines to an enum

Just like on arm64, having the CP15 registers expressed as a set of #defines has been very conflict-prone. Let's turn it into an enum, which should make it more manageable. Signed-off-by: Marc Zyngier --- arch/arm/include/asm/kvm_asm.h | 33 - arch/arm/include/as

[PATCH v2 27/28] ARM: KVM: Remove handling of ARM_EXCEPTION_DATA/PREF_ABORT

These are now handled as a panic, so there is little point in keeping them around. Signed-off-by: Marc Zyngier --- arch/arm/kvm/handle_exit.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/arch/arm/kvm/handle_exit.c b/arch/arm/kvm/handle_exit.c index 5377d753..3f1ef0d 100644 --- a/arch/arm

[PATCH v2 25/28] ARM: KVM: Cleanup asm-offsets.c

Since we don't have much assembler left, most of the KVM stuff in asm-offsets.c is now superfluous. Let's get rid of it. Signed-off-by: Marc Zyngier --- arch/arm/kernel/asm-offsets.c | 30 -- 1 file changed, 30 deletions(-) diff --git a/arch/arm/kernel/asm-offsets.c

[PATCH v2 15/28] ARM: KVM: Add VFP lazy save/restore handler

Similar to the arm64 version, add the code that deals with VFP traps, re-enabling VFP, save/restoring the registers and resuming the guest. Signed-off-by: Marc Zyngier --- arch/arm/kvm/hyp/entry.S | 31 +++ 1 file changed, 31 insertions(+) diff --git a/arch/arm/kvm/h

[PATCH v2 11/28] ARM: KVM: Add vgic v2 save/restore

This patch shouldn't exist, as we should be able to reuse the arm64 version for free. I'll get there eventually, but in the meantime I need an interrupt controller. Signed-off-by: Marc Zyngier --- arch/arm/kvm/hyp/Makefile | 1 + arch/arm/kvm/hyp/hyp.h| 3 ++ arch/arm/kvm/hyp/vgic-

[PATCH v2 02/28] ARM: KVM: Remove __kvm_hyp_code_start/__kvm_hyp_code_end

Now that we've unified the way we refer to the HYP text between arm and arm64, drop __kvm_hyp_code_start/end, and just use the __hyp_text_start/end symbols. Signed-off-by: Marc Zyngier --- arch/arm/include/asm/kvm_asm.h | 3 --- arch/arm/kvm/arm.c | 2 +- arch/arm64/include/asm/k

[PATCH v2 04/28] ARM: KVM: Move CP15 array into the CPU context structure

Continuing our rework of the CPU context, we now move the CP15 array into the CPU context structure. As this causes quite a bit of churn, we introduce the vcpu_cp15() macro that abstract the location of the actual array. This will probably help next time we have to revisit that code. Signed-off-by

[PATCH v2 20/28] ARM: KVM: Change kvm_call_hyp return type to unsigned long

Having u64 as the kvm_call_hyp return type is problematic, as it forces all kind of tricks for the return values from HYP to be promoted to 64bit (LE has the LSB in r0, and BE has them in r1). Since the only user of the return value is perfectly happy with a 32bit value, let's make kvm_call_hyp re

[PATCH v2 12/28] ARM: KVM: Add VFP save/restore

This is almost a copy/paste of the existing version, with a couple of subtle differences: - Only write to FPEXC once on the save path - Add an isb when enabling VFP access The patch also defines a few sysreg accessors and a __vfp_enabled predicate that test the VFP trapping state. Signed-off-by:

[PATCH v2 00/28] ARM: KVM: Rewrite the world switch in C (mostly)

Now that the arm64 rewrite is in mainline, I've taken a stab at fixing the 32bit code the same way. This is fairly straightforward (once you've been through it once...), with a few patches that adapt the code to be similar to the 64bit version. Note that the timer and GIC code should be made commo

[PATCH v2 10/28] ARM: KVM: Add timer save/restore

This patch shouldn't exist, as we should be able to reuse the arm64 version for free. I'll get there eventually, but in the meantime I need a timer ticking. Signed-off-by: Marc Zyngier --- arch/arm/kvm/hyp/Makefile | 1 + arch/arm/kvm/hyp/hyp.h | 8 + arch/arm/kvm/hyp/timer-sr.c | 7

[PATCH v2 13/28] ARM: KVM: Add banked registers save/restore

Banked registers are one of the many perks of the 32bit architecture, and the world switch needs to cope with it. This requires some "special" accessors, as these are not accessed using a standard coprocessor instruction. Signed-off-by: Marc Zyngier --- arch/arm/kvm/hyp/Makefile| 1 + arch

[PATCH v2 14/28] ARM: KVM: Add guest entry code

Add the very minimal piece of code that is now required to jump into the guest (and return from it). This code is only concerned with save/restoring the USR registers (r0-r12+lr for the guest, r4-r12+lr for the host), as everything else is dealt with in C (VFP is another matter though). Signed-off

[PATCH v2 01/28] ARM: KVM: Move the HYP code to its own section

In order to be able to spread the HYP code into multiple compilation units, adopt a layout similar to that of arm64: - the HYP text is emited in its own section (.hyp.text) - two linker generated symbols are use to identify the boundaries of that section No functionnal change. Signed-off-by: Ma

[PATCH v2 05/28] ARM: KVM: Move GP registers into the CPU context structure

Continuing our rework of the CPU context, we now move the GP registers into the CPU context structure. Signed-off-by: Marc Zyngier --- arch/arm/include/asm/kvm_emulate.h | 8 arch/arm/include/asm/kvm_host.h| 3 +-- arch/arm/kernel/asm-offsets.c | 18 +- arch/a

[PATCH v2 03/28] ARM: KVM: Move VFP registers to a CPU context structure

In order to turn the WS code into something that looks a bit more like the arm64 version, move the VFP registers into a CPU context container for both the host and the guest. Signed-off-by: Marc Zyngier --- arch/arm/include/asm/kvm_host.h | 11 +++ arch/arm/kernel/asm-offsets.c | 5 ++

[PATCH v2 06/28] ARM: KVM: Add a HYP-specific header file

In order to expose the various HYP services that are private to the hypervisor, add a new hyp.h file. So far, it only contains mundane things such as section annotation and VA manipulation. Signed-off-by: Marc Zyngier --- arch/arm/kvm/hyp/hyp.h | 30 ++ 1 file change

[PATCH v2 07/28] ARM: KVM: Add system register accessor macros

In order to move system register (CP15, mostly) access to C code, add a few macros to facilitate this, and minimize the difference between 32 and 64bit CP15 registers. This will get heavily used in the following patches. Signed-off-by: Marc Zyngier --- arch/arm/kvm/hyp/hyp.h | 15 ++