From: Jinjie Ruan <[email protected]>
MPAM (Memory System Resource Partitioning and Monitoring) is Arm's
architecture for partitioning and monitoring system memory resources
- Cache portions
- Capacity limits
- Memory bandwidth and priority
And MPAM is the Arm counterpart of x86 RDT, which Linux exposes to user
space via resctrl.
QEMU so far has no MPAM support at all: the CPU-side system registers
are missing (a guest kernel that probes FEAT_MPAM takes an UNDEFINED
exception), and there are no Memory System Components (MSCs), so a virt
guest cannot exercise resctrl on Arm.
This series adds MPAM 1.1 emulation for the virt machine,
and the structure is as below:
* target/arm (patches 1-3): the CPU registers:
- MPAMIDR_EL1 and MPAM0_EL1/MPAM1_EL1
- MPAM2_EL2/MPAMHCR_EL2
- MPAMSM_EL1 for FEAT_MPAM_SMS
* hw/acpi (patch 4): PPTT revision 3 with Cache IDs, which the kernel
needs to resolve the MPAM table's cache references to PPTT cache
nodes.
* hw/arm/mpam device model (patches 5-15): built up layer by layer
- The register map as pure registerfields declarations
- The base MSC device with MMIO access
- The mpam-msc-mem and mpam-msc-cache subtypes
- MPAMCFG_PART_SEL selection
- Per-RIS partition config storage
- The partitioning features one at a time (CPOR cache portions, MBW
bandwidth, PRI priority, CCAP capacity)
- The ECR/ESR register surface
* hw/arm/mpam (patch 16): PARTID narrowing via MPAMCFG_INTPARTID with
the identity-wrap default map
- The linux mainline driver exercises it by default and resctrl
reports num_closids=32.
* hw/arm/virt + hw/acpi (patches 17-23): the board wiring, one concern
per patch
- Machine properties
- The MMIO window reservation
- Device locator/irq properties
- Memory-side MSC creation
- Cache MSC creation derived from the smp cache topology
- The ACPI MPAM table, split into the DEN0065 MSC node builders
and the table assembly
* Monitoring (patches 24-29):
- The MSMON selection/window infrastructure
- The MBWU bandwidth monitor whose counters track the configured
bandwidth control (MBW_MAX or MBW_PBM, whichever the guest drives)
- CMIN/CASSOC storage
- PRI advertised on the memory side (backing the driver's MBPRI
pseudo-resource)
- The CSU occupancy monitor (a snapshot of the capacity scaled by
the CPBM fraction)
- The board sizing of that capacity per cache level
*Error reporting (patches 30-33):
- ECR/ESR as MSC-global registers (IHI 0099 4.3)
- Device reset with power-on state initialisation
- The two level-held interrupt outputs (8.2/8.3)
- Table 7-1 error recording injected on the invalid-configuration
write paths.
* patch 34: the board wiring of the MSC interrupts to free SPIs on the
GIC, with the assigned GSIVs reported in the ACPI MPAM table.
* patch 35: defaulting to a single node when none is specified.
* patches 36-37: the query-mpam-cache QMP introspection, split into
schema+stub and implementation.
With this, a Linux guest booted with -M virt,mpam=on (plus NUMA nodes)
mounts resctrl with the full schemata set (L2/L3 CAT, capacity, MBA,
priority), and MBM bandwidth and L2/L3 occupancy counters track the
configured controls.
Verified with a Linux 7.3-rc6 guest on gic-v3: the arm_mpam driver probes
cleanly, the resctrl regression suite passes and no MPAM error interrupt
fires during normal boot.
Patches 1, 5-8 and 36-37 carry Jonathan Cameron's authorship, being
derived from his RFC series and its QMP face. Compared to that RFC [1],
this series completes most of its TODO list and tracks what the
mainline arm_mpam driver now requires:
- Monitoring
- The MSC interrupts
- Error reporting on invalid parameters
- PARTID narrowing
- The missing CPU registers
- PPTT rev 3 with Cache IDs
- CMIN/CASSOC
- Memory-side
- CSU monitoring
Still open from the RFC's TODO list: ACPI table unit tests. Like the
RFC, this is ACPI-only; the DT binding remains unimplemented.
[1] https://patchew.org/QEMU/[email protected]/
Jinjie Ruan (30):
target/arm: Add MPAM2_EL2 and MPAMHCR_EL2 registers
target/arm: Add MPAMSM_EL1 register
hw/acpi: Build PPTT rev 3 with cache IDs for MPAM cross-references
hw/arm/mpam: Handle MPAMCFG_PART_SEL selection
hw/arm/mpam: Add per-RIS partition config storage
hw/arm/mpam: Implement CPOR cache portion partitioning
hw/arm/mpam: Implement MBW bandwidth partitioning
hw/arm/mpam: Implement PRI priority partitioning
hw/arm/mpam: Implement CCAP capacity partitioning
hw/arm/mpam: Add ECR/ESR and error MSI register storage
hw/arm/mpam: Implement PARTID narrowing
hw/arm/virt: Add mpam and mpam-min-msc machine properties
hw/arm/virt: Reserve the MPAM MSC MMIO window
hw/arm/mpam: Add node and interrupt GSIV properties
hw/arm/virt: Create memory-side MPAM MSCs
hw/arm/virt: Create L2/L3 cache MPAM MSCs
hw/acpi: Add MPAM table MSC node builders
hw/acpi: Build the MPAM ACPI table
hw/arm/mpam: Add MSMON monitor selection and window configuration
hw/arm/mpam: Emulate MBWU bandwidth monitoring
hw/arm/mpam: Implement CMIN/CASSOC storage
hw/arm/mpam: Advertise PRI on memory-side MSCs
hw/arm/mpam: Emulate CSU cache-usage monitoring
hw/arm/virt: Size the cache MSC occupancy model by level
hw/arm/mpam: Handle ECR/ESR as MSC-global registers
hw/arm/mpam: Add device reset and power-on state initialisation
hw/arm/mpam: Add error and overflow interrupt outputs
hw/arm/mpam: Record errors and reject invalid configuration writes
hw/arm/virt: Wire MSC error and overflow interrupts to the GIC
hw/arm/virt: Default to implicit node 0 for MPAM without -numa
Jonathan Cameron (7):
target/arm: Add support for MPAM CPU registers
hw/arm/mpam: Add MPAM MSC register map
hw/arm/mpam: Add MPAM MSC base device
hw/arm/mpam: Add mpam-msc-mem memory-side MSC subtype
hw/arm/mpam: Add mpam-msc-cache MSC subtype
qapi: Add MPAM cache introspection schema and stub
hw/arm/mpam: Implement query-mpam-cache
hw/acpi/aml-build.c | 31 +-
hw/arm/Kconfig | 4 +
hw/arm/meson.build | 3 +
hw/arm/mpam-qapi-stubs.c | 14 +
hw/arm/mpam-qapi.c | 62 ++
hw/arm/mpam.c | 1620 +++++++++++++++++++++++++++++++++++
hw/arm/virt-acpi-build.c | 231 +++++
hw/arm/virt.c | 239 ++++++
include/hw/acpi/aml-build.h | 8 +
include/hw/arm/mpam.h | 23 +
include/hw/arm/virt.h | 3 +
qapi/meson.build | 1 +
qapi/mpam.json | 120 +++
qapi/qapi-schema.json | 1 +
target/arm/cpu-features.h | 11 +
target/arm/cpu.c | 10 +-
target/arm/cpu.h | 6 +
target/arm/helper.c | 33 +
18 files changed, 2410 insertions(+), 10 deletions(-)
create mode 100644 hw/arm/mpam-qapi-stubs.c
create mode 100644 hw/arm/mpam-qapi.c
create mode 100644 hw/arm/mpam.c
create mode 100644 include/hw/arm/mpam.h
create mode 100644 qapi/mpam.json
--
2.43.0
