On 11/9/2023 8:12 PM, Bui Quang Minh wrote:
> On 11/9/23 21:32, Joao Martins wrote:
>> On 09/11/2023 14:10, Bui Quang Minh wrote:
>>> On 11/9/23 17:11, Santosh Shukla wrote:
>>>> On 10/24/2023 8:51 PM, Bui Quang Minh wrote:
>>>>> Hi everyone,
>>>>>
>>>>> This series implements x2APIC mode in userspace local APIC and the
>>>>> RDMSR/WRMSR helper to access x2APIC registers in x2APIC mode. Intel iommu
>>>>> and AMD iommu are adjusted to support x2APIC interrupt remapping. With
>>>>> this
>>>>> series, we can now boot Linux kernel into x2APIC mode with TCG accelerator
>>>>> using either Intel or AMD iommu.
>>>>>
>>>>> Testing to boot my own built Linux 6.3.0-rc2, the kernel successfully boot
>>>>> with enabled x2APIC and can enumerate CPU with APIC ID 257
>>>>>
>>>>> Using Intel IOMMU
>>>>>
>>>>> qemu/build/qemu-system-x86_64 \
>>>>> -smp 2,maxcpus=260 \
>>>>> -cpu qemu64,x2apic=on \
>>>>> -machine q35 \
>>>>> -device intel-iommu,intremap=on,eim=on \
>>>>> -device
>>>>> qemu64-x86_64-cpu,x2apic=on,core-id=257,socket-id=0,thread-id=0 \
>>>>> -m 2G \
>>>>> -kernel $KERNEL_DIR \
>>>>> -append "nokaslr console=ttyS0 root=/dev/sda earlyprintk=serial
>>>>> net.ifnames=0" \
>>>>> -drive file=$IMAGE_DIR,format=raw \
>>>>> -nographic \
>>>>> -s
>>>>>
>>>>> Using AMD IOMMU
>>>>>
>>>>> qemu/build/qemu-system-x86_64 \
>>>>> -smp 2,maxcpus=260 \
>>>>> -cpu qemu64,x2apic=on \
>>>>> -machine q35 \
>>>>> -device amd-iommu,intremap=on,xtsup=on \
>>>>> -device
>>>>> qemu64-x86_64-cpu,x2apic=on,core-id=257,socket-id=0,thread-id=0 \
>>>>> -m 2G \
>>>>> -kernel $KERNEL_DIR \
>>>>> -append "nokaslr console=ttyS0 root=/dev/sda earlyprintk=serial
>>>>> net.ifnames=0" \
>>>>> -drive file=$IMAGE_DIR,format=raw \
>>>>> -nographic \
>>>>> -s
>>>>>
>>>>> Testing the emulated userspace APIC with kvm-unit-tests, disable test
>>>>> device with this patch
>>>>>
>>>>> diff --git a/lib/x86/fwcfg.c b/lib/x86/fwcfg.c
>>>>> index 1734afb..f56fe1c 100644
>>>>> --- a/lib/x86/fwcfg.c
>>>>> +++ b/lib/x86/fwcfg.c
>>>>> @@ -27,6 +27,7 @@ static void read_cfg_override(void)
>>>>>
>>>>> if ((str = getenv("TEST_DEVICE")))
>>>>> no_test_device = !atol(str);
>>>>> + no_test_device = true;
>>>>>
>>>>> if ((str = getenv("MEMLIMIT")))
>>>>> fw_override[FW_CFG_MAX_RAM] = atol(str) * 1024 * 1024;
>>>>>
>>>>> ~ env QEMU=/home/minh/Desktop/oss/qemu/build/qemu-system-x86_64 ACCEL=tcg
>>>>> \
>>>>> ./run_tests.sh -v -g apic
>>>>>
>>>>> TESTNAME=apic-split TIMEOUT=90s ACCEL=tcg ./x86/run x86/apic.flat -smp 2
>>>>> -cpu qemu64,+x2apic,+tsc-deadline -machine kernel_irqchip=split FAIL
>>>>> apic-split (54 tests, 8 unexpected failures, 1 skipped)
>>>>> TESTNAME=ioapic-split TIMEOUT=90s ACCEL=tcg ./x86/run x86/ioapic.flat -smp
>>>>> 1 -cpu qemu64 -machine kernel_irqchip=split PASS ioapic-split (19 tests)
>>>>> TESTNAME=x2apic TIMEOUT=30 ACCEL=tcg ./x86/run x86/apic.flat -smp 2 -cpu
>>>>> qemu64,+x2apic,+tsc-deadline FAIL x2apic (54 tests, 8 unexpected failures,
>>>>> 1 skipped) TESTNAME=xapic TIMEOUT=60 ACCEL=tcg ./x86/run x86/apic.flat
>>>>> -smp
>>>>> 2 -cpu qemu64,-x2apic,+tsc-deadline -machine pit=off FAIL xapic (43 tests,
>>>>> 6 unexpected failures, 2 skipped)
>>>>>
>>>>> FAIL: apic_disable: *0xfee00030: 50014
>>>>> FAIL: apic_disable: *0xfee00080: f0
>>>>> FAIL: apic_disable: *0xfee00030: 50014
>>>>> FAIL: apic_disable: *0xfee00080: f0
>>>>> FAIL: apicbase: relocate apic
>>>>>
>>>>> These errors are because we don't disable MMIO region when switching to
>>>>> x2APIC and don't support relocate MMIO region yet. This is a problem
>>>>> because, MMIO region is the same for all CPUs, in order to support these
>>>>> we
>>>>> need to figure out how to allocate and manage different MMIO regions for
>>>>> each CPUs. This can be an improvement in the future.
>>>>>
>>>>> FAIL: nmi-after-sti
>>>>> FAIL: multiple nmi
>>>>>
>>>>> These errors are in the way we handle CPU_INTERRUPT_NMI in core TCG.
>>>>>
>>>>> FAIL: TMCCT should stay at zero
>>>>>
>>>>> This error is related to APIC timer which should be addressed in separate
>>>>> patch.
>>>>>
>>>>> Version 9 changes,
>>>>
>>>> Hi Bui,
>>>>
>>>> I have tested v9 on EPYC-Genoa system with kvm acceleration mode on, I
>>>> could
>>>> see > 255 vCPU for Linux and Windows Guest.
>>>>
>>>> Tested-by: Santosh Shukla <santosh.shu...@amd.com>
>>>
>>> Hi Santosh,
>>>
>>> With KVM enabled, you may be using the in kernel APIC from KVM not the
>>> emulated
>>> APIC in userspace as in this series.
>>>
>>
>> Your XTSup code isn't necessarily userspace APIC specific. You can have
>> accel=kvm with split irqchip and things will still work. I suspect that's how
>> Santosh tested it.
>
That's correct.
> Ah, I got it. Thanks Santosh, Joao.
> Quang Minh.
>
Thanks,
Santosh
