Re: [PATCH 1/4] iommu: Add virtio-iommu driver

2018-02-27 Thread Michael S. Tsirkin
On Thu, Feb 22, 2018 at 11:04:30AM +, Jean-Philippe Brucker wrote: > On 21/02/18 20:12, kbuild test robot wrote: > [...] > > config: arm64-allmodconfig (attached as .config) > [...] > >aarch64-linux-gnu-ld: arch/arm64/kernel/head.o: relocation > > R_AARCH64_ABS32 against `_kernel_offset_le

Re: [PATCH v5 13/40] KVM: arm64: Introduce VHE-specific kvm_vcpu_run

2018-02-27 Thread Andrew Jones
On Tue, Feb 27, 2018 at 12:34:02PM +0100, Christoffer Dall wrote: > From: Christoffer Dall > > So far this is mostly (see below) a copy of the legacy non-VHE switch > function, but we will start reworking these functions in separate > directions to work on VHE and non-VHE in the most optimal way

Re: VCPU hotplug on KVM/ARM

2018-02-27 Thread Andrew Jones
On Tue, Feb 27, 2018 at 01:46:04PM +0100, Christoffer Dall wrote: > On Tue, Feb 27, 2018 at 05:34:28PM +0530, btha...@codeaurora.org wrote: > > Hi Christoffer, > > > > Thanks for your reply. > > > > On 2018-02-27 16:17, Christoffer Dall wrote: > > >Hi Bhupinder, > > > > > >On Tue, Feb 27, 2018 at

Re: VCPU hotplug on KVM/ARM

2018-02-27 Thread Christoffer Dall
On Tue, Feb 27, 2018 at 05:34:28PM +0530, btha...@codeaurora.org wrote: > Hi Christoffer, > > Thanks for your reply. > > On 2018-02-27 16:17, Christoffer Dall wrote: > >Hi Bhupinder, > > > >On Tue, Feb 27, 2018 at 03:01:17PM +0530, btha...@codeaurora.org wrote: > >>I hope it is the right forum to

Re: VCPU hotplug on KVM/ARM

2018-02-27 Thread bthakur
Hi Christoffer, Thanks for your reply. On 2018-02-27 16:17, Christoffer Dall wrote: Hi Bhupinder, On Tue, Feb 27, 2018 at 03:01:17PM +0530, btha...@codeaurora.org wrote: I hope it is the right forum to post my query. I am currently looking at the possibility of adding a new VCPU to a runn

[PATCH v5 40/40] KVM: arm/arm64: Avoid VGICv3 save/restore on VHE with no IRQs

2018-02-27 Thread Christoffer Dall
From: Christoffer Dall We can finally get completely rid of any calls to the VGICv3 save/restore functions when the AP lists are empty on VHE systems. This requires carefully factoring out trap configuration from saving and restoring state, and carefully choosing what to do on the VHE and non-VH

[PATCH v5 39/40] KVM: arm/arm64: Move VGIC APR save/restore to vgic put/load

2018-02-27 Thread Christoffer Dall
From: Christoffer Dall The APRs can only have bits set when the guest acknowledges an interrupt in the LR and can only have a bit cleared when the guest EOIs an interrupt in the LR. Therefore, if we have no LRs with any pending/active interrupts, the APR cannot change value and there is no need

[PATCH v5 36/40] KVM: arm/arm64: Handle VGICv2 save/restore from the main VGIC code

2018-02-27 Thread Christoffer Dall
From: Christoffer Dall We can program the GICv2 hypervisor control interface logic directly from the core vgic code and can instead do the save/restore directly from the flush/sync functions, which can lead to a number of future optimizations. Signed-off-by: Christoffer Dall --- Notes: Cha

[PATCH v5 37/40] KVM: arm/arm64: Move arm64-only vgic-v2-sr.c file to arm64

2018-02-27 Thread Christoffer Dall
From: Christoffer Dall The vgic-v2-sr.c file now only contains the logic to replay unaligned accesses to the virtual CPU interface on 16K and 64K page systems, which is only relevant on 64-bit platforms. Therefore move this file to the arm64 KVM tree, remove the compile directive from the 32-bit

[PATCH v5 38/40] KVM: arm/arm64: Handle VGICv3 save/restore from the main VGIC code on VHE

2018-02-27 Thread Christoffer Dall
From: Christoffer Dall Just like we can program the GICv2 hypervisor control interface directly from the core vgic code, we can do the same for the GICv3 hypervisor control interface on VHE systems. We do this by simply calling the save/restore functions when we have VHE and we can then get rid

[PATCH v5 33/40] KVM: arm64: Configure c15, PMU, and debug register traps on cpu load/put for VHE

2018-02-27 Thread Christoffer Dall
From: Christoffer Dall We do not have to change the c15 trap setting on each switch to/from the guest on VHE systems, because this setting only affects guest EL1/EL0 (and therefore not the VHE host). The PMU and debug trap configuration can also be done on vcpu load/put instead, because they don

[PATCH v5 35/40] KVM: arm/arm64: Get rid of vgic_elrsr

2018-02-27 Thread Christoffer Dall
From: Christoffer Dall There is really no need to store the vgic_elrsr on the VGIC data structures as the only need we have for the elrsr is to figure out if an LR is inactive when we save the VGIC state upon returning from the guest. We can might as well store this in a temporary local variable

[PATCH v5 34/40] KVM: arm64: Cleanup __activate_traps and __deactive_traps for VHE and non-VHE

2018-02-27 Thread Christoffer Dall
From: Christoffer Dall To make the code more readable and to avoid the overhead of a function call, let's get rid of a pair of the alternative function selectors and explicitly call the VHE and non-VHE functions using the has_vhe() static key based selector instead, telling the compiler to try to

[PATCH v5 31/40] KVM: arm64: Move common VHE/non-VHE trap config in separate functions

2018-02-27 Thread Christoffer Dall
From: Christoffer Dall As we are about to be more lazy with some of the trap configuration register read/writes for VHE systems, move the logic that is currently shared between VHE and non-VHE into a separate function which can be called from either the world-switch path or from vcpu_load/vcpu_pu

[PATCH v5 32/40] KVM: arm64: Directly call VHE and non-VHE FPSIMD enabled functions

2018-02-27 Thread Christoffer Dall
From: Christoffer Dall There is no longer a need for an alternative to choose the right function to tell us whether or not FPSIMD was enabled for the VM, because we can simply can the appropriate functions directly from within the _vhe and _nvhe run functions. Reviewed-by: Marc Zyngier Reviewed

[PATCH v5 30/40] KVM: arm64: Defer saving/restoring 32-bit sysregs to vcpu load/put

2018-02-27 Thread Christoffer Dall
From: Christoffer Dall When running a 32-bit VM (EL1 in AArch32), the AArch32 system registers can be deferred to vcpu load/put on VHE systems because neither the host kernel nor host userspace uses these registers. Note that we can't save DBGVCR32_EL2 conditionally based on the state of the deb

[PATCH v5 28/40] KVM: arm64: Defer saving/restoring 64-bit sysregs to vcpu load/put on VHE

2018-02-27 Thread Christoffer Dall
From: Christoffer Dall Some system registers do not affect the host kernel's execution and can therefore be loaded when we are about to run a VCPU and we don't have to restore the host state to the hardware before the time when we are actually about to return to userspace or schedule out the VCPU

[PATCH v5 29/40] KVM: arm64: Prepare to handle deferred save/restore of 32-bit registers

2018-02-27 Thread Christoffer Dall
From: Christoffer Dall 32-bit registers are not used by a 64-bit host kernel and can be deferred, but we need to rework the accesses to these register to access the latest values depending on whether or not guest system registers are loaded on the CPU or only reside in memory. Reviewed-by: Marc

[PATCH v5 24/40] KVM: arm64: Rewrite system register accessors to read/write functions

2018-02-27 Thread Christoffer Dall
From: Christoffer Dall Currently we access the system registers array via the vcpu_sys_reg() macro. However, we are about to change the behavior to some times modify the register file directly, so let's change this to two primitives: * Accessor macros vcpu_write_sys_reg() and vcpu_read_sys_reg

[PATCH v5 26/40] KVM: arm/arm64: Prepare to handle deferred save/restore of SPSR_EL1

2018-02-27 Thread Christoffer Dall
From: Christoffer Dall SPSR_EL1 is not used by a VHE host kernel and can be deferred, but we need to rework the accesses to this register to access the latest value depending on whether or not guest system registers are loaded on the CPU or only reside in memory. The handling of accessing the va

[PATCH v5 27/40] KVM: arm64: Prepare to handle deferred save/restore of ELR_EL1

2018-02-27 Thread Christoffer Dall
From: Christoffer Dall ELR_EL1 is not used by a VHE host kernel and can be deferred, but we need to rework the accesses to this register to access the latest value depending on whether or not guest system registers are loaded on the CPU or only reside in memory. Reviewed-by: Marc Zyngier Review

[PATCH v5 25/40] KVM: arm64: Introduce framework for accessing deferred sysregs

2018-02-27 Thread Christoffer Dall
From: Christoffer Dall We are about to defer saving and restoring some groups of system registers to vcpu_put and vcpu_load on supported systems. This means that we need some infrastructure to access system registes which supports either accessing the memory backing of the register or directly a

[PATCH v5 23/40] KVM: arm64: Change 32-bit handling of VM system registers

2018-02-27 Thread Christoffer Dall
From: Christoffer Dall We currently handle 32-bit accesses to trapped VM system registers using the 32-bit index into the coproc array on the vcpu structure, which is a union of the coproc array and the sysreg array. Since all the 32-bit coproc indices are created to correspond to the architectu

[PATCH v5 22/40] KVM: arm64: Don't save the host ELR_EL2 and SPSR_EL2 on VHE systems

2018-02-27 Thread Christoffer Dall
From: Christoffer Dall On non-VHE systems we need to save the ELR_EL2 and SPSR_EL2 so that we can return to the host in EL1 in the same state and location where we issued a hypercall to EL2, but on VHE ELR_EL2 and SPSR_EL2 are not useful because we never enter a guest as a result of an exception

[PATCH v5 21/40] KVM: arm64: Unify non-VHE host/guest sysreg save and restore functions

2018-02-27 Thread Christoffer Dall
From: Christoffer Dall There is no need to have multiple identical functions with different names for saving host and guest state. When saving and restoring state for the host and guest, the state is the same for both contexts, and that's why we have the kvm_cpu_context structure. Delete one ve

[PATCH v5 20/40] KVM: arm/arm64: Remove leftover comment from kvm_vcpu_run_vhe

2018-02-27 Thread Christoffer Dall
From: Christoffer Dall The comment only applied to SPE on non-VHE systems, so we simply remove it. Suggested-by: Andrew Jones Acked-by: Marc Zyngier Reviewed-by: Andrew Jones Signed-off-by: Christoffer Dall --- arch/arm64/kvm/hyp/switch.c | 4 1 file changed, 4 deletions(-) diff --git

[PATCH v5 19/40] KVM: arm64: Introduce separate VHE/non-VHE sysreg save/restore functions

2018-02-27 Thread Christoffer Dall
From: Christoffer Dall As we are about to handle system registers quite differently between VHE and non-VHE systems. In preparation for that, we need to split some of the handling functions between VHE and non-VHE functionality. For now, we simply copy the non-VHE functions, but we do change th

[PATCH v5 18/40] KVM: arm64: Rewrite sysreg alternatives to static keys

2018-02-27 Thread Christoffer Dall
From: Christoffer Dall As we are about to move calls around in the sysreg save/restore logic, let's first rewrite the alternative function callers, because it is going to make the next patches much easier to read. Acked-by: Marc Zyngier Reviewed-by: Andrew Jones Signed-off-by: Christoffer Dall

[PATCH v5 17/40] KVM: arm64: Move userspace system registers into separate function

2018-02-27 Thread Christoffer Dall
From: Christoffer Dall There's a semantic difference between the EL1 registers that control operation of a kernel running in EL1 and EL1 registers that only control userspace execution in EL0. Since we can defer saving/restoring the latter, move them into their own function. The ARMv8 ARM (ARM

[PATCH v5 16/40] KVM: arm64: Remove noop calls to timer save/restore from VHE switch

2018-02-27 Thread Christoffer Dall
From: Christoffer Dall The VHE switch function calls __timer_enable_traps and __timer_disable_traps which don't do anything on VHE systems. Therefore, simply remove these calls from the VHE switch function and make the functions non-conditional as they are now only called from the non-VHE switch

[PATCH v5 15/40] KVM: arm64: Don't deactivate VM on VHE systems

2018-02-27 Thread Christoffer Dall
From: Christoffer Dall There is no need to reset the VTTBR to zero when exiting the guest on VHE systems. VHE systems don't use stage 2 translations for the EL2&0 translation regime used by the host. Reviewed-by: Andrew Jones Acked-by: Marc Zyngier Signed-off-by: Christoffer Dall --- Notes:

[PATCH v5 13/40] KVM: arm64: Introduce VHE-specific kvm_vcpu_run

2018-02-27 Thread Christoffer Dall
From: Christoffer Dall So far this is mostly (see below) a copy of the legacy non-VHE switch function, but we will start reworking these functions in separate directions to work on VHE and non-VHE in the most optimal way in later patches. The only difference after this patch between the VHE and

[PATCH v5 14/40] KVM: arm64: Remove kern_hyp_va() use in VHE switch function

2018-02-27 Thread Christoffer Dall
From: Christoffer Dall VHE kernels run completely in EL2 and therefore don't have a notion of kernel and hyp addresses, they are all just kernel addresses. Therefore don't call kern_hyp_va() in the VHE switch function. Reviewed-by: Andrew Jones Reviewed-by: Marc Zyngier Signed-off-by: Christo

[PATCH v5 11/40] KVM: arm64: Improve debug register save/restore flow

2018-02-27 Thread Christoffer Dall
From: Christoffer Dall Instead of having multiple calls from the world switch path to the debug logic, each figuring out if the dirty bit is set and if we should save/restore the debug registers, let's just provide two hooks to the debug save/restore functionality, one for switching to the guest

[PATCH v5 12/40] KVM: arm64: Factor out fault info population and gic workarounds

2018-02-27 Thread Christoffer Dall
From: Christoffer Dall The current world-switch function has functionality to detect a number of cases where we need to fixup some part of the exit condition and possibly run the guest again, before having restored the host state. This includes populating missing fault info, emulating GICv2 CPU

[PATCH v5 10/40] KVM: arm64: Slightly improve debug save/restore functions

2018-02-27 Thread Christoffer Dall
From: Christoffer Dall The debug save/restore functions can be improved by using the has_vhe() static key instead of the instruction alternative. Using the static key uses the same paradigm as we're going to use elsewhere, it makes the code more readable, and it generates slightly better code (n

[PATCH v5 09/40] KVM: arm64: Move debug dirty flag calculation out of world switch

2018-02-27 Thread Christoffer Dall
From: Christoffer Dall There is no need to figure out inside the world-switch if we should save/restore the debug registers or not, we might as well do that in the higher level debug setup code, making it easier to optimize down the line. Reviewed-by: Julien Thierry Reviewed-by: Marc Zyngier R

[PATCH v5 05/40] KVM: arm64: Move HCR_INT_OVERRIDE to default HCR_EL2 guest flag

2018-02-27 Thread Christoffer Dall
From: Shih-Wei Li We always set the IMO and FMO bits in the HCR_EL2 when running the guest, regardless if we use the vgic or not. By moving these flags to HCR_GUEST_FLAGS we can avoid one of the extra save/restore operations of HCR_EL2 in the world switch code, and we can also soon get rid of th

[PATCH v5 08/40] KVM: arm/arm64: Introduce vcpu_el1_is_32bit

2018-02-27 Thread Christoffer Dall
From: Christoffer Dall We have numerous checks around that checks if the HCR_EL2 has the RW bit set to figure out if we're running an AArch64 or AArch32 VM. In some cases, directly checking the RW bit (given its unintuitive name), is a bit confusing, and that's not going to improve as we move lo

[PATCH v5 07/40] KVM: arm/arm64: Add kvm_vcpu_load_sysregs and kvm_vcpu_put_sysregs

2018-02-27 Thread Christoffer Dall
From: Christoffer Dall As we are about to move a bunch of save/restore logic for VHE kernels to the load and put functions, we need some infrastructure to do this. Reviewed-by: Andrew Jones Acked-by: Marc Zyngier Signed-off-by: Christoffer Dall --- Notes: Changes since v1: - Reworde

[PATCH v5 06/40] KVM: arm/arm64: Get rid of vcpu->arch.irq_lines

2018-02-27 Thread Christoffer Dall
From: Christoffer Dall We currently have a separate read-modify-write of the HCR_EL2 on entry to the guest for the sole purpose of setting the VF and VI bits, if set. Since this is most rarely the case (only when using userspace IRQ chip and interrupts are in flight), let's get rid of this operat

[PATCH v5 04/40] KVM: arm64: Rework hyp_panic for VHE and non-VHE

2018-02-27 Thread Christoffer Dall
From: Christoffer Dall VHE actually doesn't rely on clearing the VTTBR when returning to the host kernel, and that is the current key mechanism of hyp_panic to figure out how to attempt to return to a state good enough to print a panic statement. Therefore, we split the hyp_panic function into t

[PATCH v5 02/40] KVM: arm/arm64: Move vcpu_load call after kvm_vcpu_first_run_init

2018-02-27 Thread Christoffer Dall
From: Christoffer Dall Moving the call to vcpu_load() in kvm_arch_vcpu_ioctl_run() to after we've called kvm_vcpu_first_run_init() simplifies some of the vgic and there is also no need to do vcpu_load() for things such as handling the immediate_exit flag. Reviewed-by: Julien Grall Reviewed-by:

[PATCH v5 03/40] KVM: arm64: Avoid storing the vcpu pointer on the stack

2018-02-27 Thread Christoffer Dall
From: Christoffer Dall We already have the percpu area for the host cpu state, which points to the VCPU, so there's no need to store the VCPU pointer on the stack on every context switch. We can be a little more clever and just use tpidr_el2 for the percpu offset and load the VCPU pointer from t

[PATCH v5 01/40] KVM: arm/arm64: Avoid vcpu_load for other vcpu ioctls than KVM_RUN

2018-02-27 Thread Christoffer Dall
From: Christoffer Dall Calling vcpu_load() registers preempt notifiers for this vcpu and calls kvm_arch_vcpu_load(). The latter will soon be doing a lot of heavy lifting on arm/arm64 and will try to do things such as enabling the virtual timer and setting us up to handle interrupts from the time

[PATCH v5 00/40] Optimize KVM/ARM for VHE systems

2018-02-27 Thread Christoffer Dall
This series redesigns parts of KVM/ARM to optimize the performance on VHE systems. The general approach is to try to do as little work as possible when transitioning between the VM and the hypervisor. This has the benefit of lower latency when waiting for interrupts and delivering virtual interru

Re: VCPU hotplug on KVM/ARM

2018-02-27 Thread Christoffer Dall
Hi Bhupinder, On Tue, Feb 27, 2018 at 03:01:17PM +0530, btha...@codeaurora.org wrote: > I hope it is the right forum to post my query. > > > > I am currently looking at the possibility of adding a new VCPU to a running > guest VM in KVM/ARM. I see that currently, it is not allowed to add a new

VCPU hotplug on KVM/ARM

2018-02-27 Thread bthakur
Hi, I hope it is the right forum to post my query. I am currently looking at the possibility of adding a new VCPU to a running guest VM in KVM/ARM. I see that currently, it is not allowed to add a new VCPU to a guest VM, if it is already initialized. The first check in kvm_arch_vcpu_create(