This patchset implements emulation of GICv4 in our TCG GIC and ITS models, and makes the virt board use it where appropriate.
The GICv4 provides a single new feature: direct injection of virtual interrupts from the ITS to a VM. In QEMU terms this means that if you have an outer QEMU which is emulating a CPU with EL2, and the outer guest passes through a PCI device (probably one emulated by the outer QEMU) to an inner guest, interrupts from that device can go directly to the inner guest, rather than having to go to the outer guest and the outer guest then synthesizing virtual interrupts to the inner guest. (If you aren't configuring the inner guest with a passthrough PCI device then this new feature is of no interest.) The basic structure of the patchset is as follows: (1) There are a handful of preliminary patches fixing some minor existing nits. (2) The v4 ITS has some new in-guest-memory data structures and new ITS commands that let the guest set them up. The next sequence of patches implement all those commands. Where the command needs to actually do something (eg "deliver a vLPI"), these patches call functions in the redistributor which are left as unimplemented stubs to be filled in in subsequent patches. This first chunk of patches sticks to the data-structure handling and all the command argument unpacking and error checking. (3) The redistributor has a new redistributor frame (ie the amount of guest memory used by redistributor registers is larger) with a two new registers in it. We implement these initially as reads-as-written. (4) The CPU interface needs relatively minor changes: as well as looking at the list registers to determine the highest priority pending virtual interrupt, we must also look at the highest priority pending vLPI. We implement these changes, again leaving the interfaces from this code into the redistributor as stubs for the moment. (5) Now we can fill in all the stub code in the redistributor. This is almost all working with the pending and config tables for virtual LPIs. (Side note: in real hardware some of this work is done by the ITS rather than the redistributor, but in our implementation we split between the two source files slightly differently. I've made the vLPI handling follow the pattern of the existing LPI handling.) (6) Finally, we can update the ID registers which tell the guest about the presence of v4 features, allow the GIC device to accept 4 as a value for its QOM revision property, and make the virt board set that when appropriate. General notes: Since the only useful thing in GICv4 is direct virtual interrupt injection, it isn't expected that you would have a system with a GICv4 and a CPU without EL2. So I've made this an error, and the virt board will only use GICv4 if the user also enables emulation of virtualization. Because the redistributor frame is twice the size in GICv4, the number of redistributors we can fit into a given area of memory is reduced. This means that when using GICv4 the maximum number of CPUs supported on the virt board drops from 512 to 317. (No, I'm not sure why this is 317 and not 256 :-)) I have not particularly considered performance in this initial implementation. In particular, we will do a complete re-scan of a virtual LPI pending table every time the outer guest reschedules a vCPU (and writes GICR_VPENDBASER). The spec provides scope for optimisation here, by allowing part of the LPI table to have IMPDEF contents, which we could in principle use to cache information like the current highest priority pending vLPI. Given that emulating nested guests with PCI passthrough is a fairly niche activity, I propose that we not do this unless the three people doing that complain about this all being too slow :-) Tested with a Linux kernel passing through a virtio-blk device to an inner Linux VM with KVM/QEMU. (NB that to get the outer Linux kernel to actually use the new GICv4 functionality you need to pass it "kvm-arm.vgic_v4_enable=1", as the kernel will not use it by default.) thanks -- PMM Peter Maydell (41): hw/intc/arm_gicv3_its: Add missing blank line hw/intc/arm_gicv3: Sanity-check num-cpu property hw/intc/arm_gicv3: Insist that redist region capacity matches CPU count hw/intc/arm_gicv3: Report correct PIDR0 values for ID registers target/arm/cpu.c: ignore VIRQ and VFIQ if no EL2 hw/intc/arm_gicv3_its: Factor out "is intid a valid LPI ID?" hw/intc/arm_gicv3_its: Implement GITS_BASER2 for GICv4 hw/intc/arm_gicv3_its: Implement VMAPI and VMAPTI hw/intc/arm_gicv3_its: Implement VMAPP hw/intc/arm_gicv3_its: Distinguish success and error cases of CMD_CONTINUE hw/intc/arm_gicv3_its: Factor out "find ITE given devid, eventid" hw/intc/arm_gicv3_its: Factor out CTE lookup sequence hw/intc/arm_gicv3_its: Split out process_its_cmd() physical interrupt code hw/intc/arm_gicv3_its: Handle virtual interrupts in process_its_cmd() hw/intc/arm_gicv3: Keep pointers to every connected ITS hw/intc/arm_gicv3_its: Implement VMOVP hw/intc/arm_gicv3_its: Implement VSYNC hw/intc/arm_gicv3_its: Implement INV command properly hw/intc/arm_gicv3_its: Implement INV for virtual interrupts hw/intc/arm_gicv3_its: Implement VMOVI hw/intc/arm_gicv3_its: Implement VINVALL hw/intc/arm_gicv3: Implement GICv4's new redistributor frame hw/intc/arm_gicv3: Implement new GICv4 redistributor registers hw/intc/arm_gicv3_cpuif: Split "update vIRQ/vFIQ" from gicv3_cpuif_virt_update() hw/intc/arm_gicv3_cpuif: Support vLPIs hw/intc/arm_gicv3_cpuif: Don't recalculate maintenance irq unnecessarily hw/intc/arm_gicv3_redist: Factor out "update hpplpi for one LPI" logic hw/intc/arm_gicv3_redist: Factor out "update hpplpi for all LPIs" logic hw/intc/arm_gicv3_redist: Recalculate hppvlpi on VPENDBASER writes hw/intc/arm_gicv3_redist: Factor out "update bit in pending table" code hw/intc/arm_gicv3_redist: Implement gicv3_redist_process_vlpi() hw/intc/arm_gicv3_redist: Implement gicv3_redist_vlpi_pending() hw/intc/arm_gicv3_redist: Use set_pending_table_bit() in mov handling hw/intc/arm_gicv3_redist: Implement gicv3_redist_mov_vlpi() hw/intc/arm_gicv3_redist: Implement gicv3_redist_vinvall() hw/intc/arm_gicv3_redist: Implement gicv3_redist_inv_vlpi() hw/intc/arm_gicv3: Update ID and feature registers for GICv4 hw/intc/arm_gicv3: Allow 'revision' property to be set to 4 hw/arm/virt: Use VIRT_GIC_VERSION_* enum values in create_gic() hw/arm/virt: Abstract out calculation of redistributor region capacity hw/arm/virt: Support TCG GICv4 docs/system/arm/virt.rst | 5 +- hw/intc/gicv3_internal.h | 231 ++++++- include/hw/arm/virt.h | 19 +- include/hw/intc/arm_gicv3_common.h | 13 + include/hw/intc/arm_gicv3_its_common.h | 1 + hw/arm/virt.c | 102 ++- hw/intc/arm_gicv3_common.c | 54 +- hw/intc/arm_gicv3_cpuif.c | 195 +++++- hw/intc/arm_gicv3_dist.c | 7 +- hw/intc/arm_gicv3_its.c | 876 ++++++++++++++++++++----- hw/intc/arm_gicv3_its_kvm.c | 2 + hw/intc/arm_gicv3_kvm.c | 5 + hw/intc/arm_gicv3_redist.c | 480 +++++++++++--- target/arm/cpu.c | 12 +- hw/intc/trace-events | 18 +- 15 files changed, 1695 insertions(+), 325 deletions(-) -- 2.25.1