On 11/9/23 21:32, Joao Martins wrote:
On 09/11/2023 14:10, Bui Quang Minh wrote:
On 11/9/23 17:11, Santosh Shukla wrote:
On 10/24/2023 8:51 PM, Bui Quang Minh wrote:
Hi everyone,

This series implements x2APIC mode in userspace local APIC and the
RDMSR/WRMSR helper to access x2APIC registers in x2APIC mode. Intel iommu
and AMD iommu are adjusted to support x2APIC interrupt remapping. With this
series, we can now boot Linux kernel into x2APIC mode with TCG accelerator
using either Intel or AMD iommu.

Testing to boot my own built Linux 6.3.0-rc2, the kernel successfully boot
with enabled x2APIC and can enumerate CPU with APIC ID 257

Using Intel IOMMU

qemu/build/qemu-system-x86_64 \
    -smp 2,maxcpus=260 \
    -cpu qemu64,x2apic=on \
    -machine q35 \
    -device intel-iommu,intremap=on,eim=on \
    -device qemu64-x86_64-cpu,x2apic=on,core-id=257,socket-id=0,thread-id=0 \
    -m 2G \
    -kernel $KERNEL_DIR \
    -append "nokaslr console=ttyS0 root=/dev/sda earlyprintk=serial
net.ifnames=0" \
    -drive file=$IMAGE_DIR,format=raw \
    -nographic \
    -s

Using AMD IOMMU

qemu/build/qemu-system-x86_64 \
    -smp 2,maxcpus=260 \
    -cpu qemu64,x2apic=on \
    -machine q35 \
    -device amd-iommu,intremap=on,xtsup=on \
    -device qemu64-x86_64-cpu,x2apic=on,core-id=257,socket-id=0,thread-id=0 \
    -m 2G \
    -kernel $KERNEL_DIR \
    -append "nokaslr console=ttyS0 root=/dev/sda earlyprintk=serial
net.ifnames=0" \
    -drive file=$IMAGE_DIR,format=raw \
    -nographic \
    -s

Testing the emulated userspace APIC with kvm-unit-tests, disable test
device with this patch

diff --git a/lib/x86/fwcfg.c b/lib/x86/fwcfg.c
index 1734afb..f56fe1c 100644
--- a/lib/x86/fwcfg.c
+++ b/lib/x86/fwcfg.c
@@ -27,6 +27,7 @@ static void read_cfg_override(void)

          if ((str = getenv("TEST_DEVICE")))
                  no_test_device = !atol(str);
+       no_test_device = true;

          if ((str = getenv("MEMLIMIT")))
                  fw_override[FW_CFG_MAX_RAM] = atol(str) * 1024 * 1024;

~ env QEMU=/home/minh/Desktop/oss/qemu/build/qemu-system-x86_64 ACCEL=tcg \
./run_tests.sh -v -g apic

TESTNAME=apic-split TIMEOUT=90s ACCEL=tcg ./x86/run x86/apic.flat -smp 2
-cpu qemu64,+x2apic,+tsc-deadline -machine kernel_irqchip=split FAIL
apic-split (54 tests, 8 unexpected failures, 1 skipped)
TESTNAME=ioapic-split TIMEOUT=90s ACCEL=tcg ./x86/run x86/ioapic.flat -smp
1 -cpu qemu64 -machine kernel_irqchip=split PASS ioapic-split (19 tests)
TESTNAME=x2apic TIMEOUT=30 ACCEL=tcg ./x86/run x86/apic.flat -smp 2 -cpu
qemu64,+x2apic,+tsc-deadline FAIL x2apic (54 tests, 8 unexpected failures,
1 skipped) TESTNAME=xapic TIMEOUT=60 ACCEL=tcg ./x86/run x86/apic.flat -smp
2 -cpu qemu64,-x2apic,+tsc-deadline -machine pit=off FAIL xapic (43 tests,
6 unexpected failures, 2 skipped)

    FAIL: apic_disable: *0xfee00030: 50014
    FAIL: apic_disable: *0xfee00080: f0
    FAIL: apic_disable: *0xfee00030: 50014
    FAIL: apic_disable: *0xfee00080: f0
    FAIL: apicbase: relocate apic

These errors are because we don't disable MMIO region when switching to
x2APIC and don't support relocate MMIO region yet. This is a problem
because, MMIO region is the same for all CPUs, in order to support these we
need to figure out how to allocate and manage different MMIO regions for
each CPUs. This can be an improvement in the future.

    FAIL: nmi-after-sti
    FAIL: multiple nmi

These errors are in the way we handle CPU_INTERRUPT_NMI in core TCG.

    FAIL: TMCCT should stay at zero

This error is related to APIC timer which should be addressed in separate
patch.

Version 9 changes,

Hi Bui,

I have tested v9 on EPYC-Genoa system with kvm acceleration mode on, I could
see > 255 vCPU for Linux and Windows Guest.

Tested-by: Santosh Shukla <santosh.shu...@amd.com>

Hi Santosh,

With KVM enabled, you may be using the in kernel APIC from KVM not the emulated
APIC in userspace as in this series.


Your XTSup code isn't necessarily userspace APIC specific. You can have
accel=kvm with split irqchip and things will still work. I suspect that's how
Santosh tested it.

Ah, I got it. Thanks Santosh, Joao.
Quang Minh.

Reply via email to