On 8/3/2023 11:07 PM, Andrew Jones wrote:
> On Mon, Jul 31, 2023 at 09:53:17AM +0800, Fei Wu wrote:
>> riscv virt platform's memory started at 0x80000000 and
>> straddled the 4GiB boundary. Curiously enough, this choice
>> of a memory layout will prevent from launching a VM with
>> a bit more than 2000MiB and PCIe pass-thru on an x86 host, due
>> to identity mapping requirements for the MSI doorbell on x86,
>> and these (APIC/IOAPIC) live right below 4GiB.
>>
>> So just split the RAM range into two portions:
>> - 1 GiB range from 0x80000000 to 0xc0000000.
>> - The remainder at 0x100000000
>>
>> ...leaving a hole between the ranges.
>
> Can you elaborate on the use case? Maybe provide details of the host
> system and the QEMU command line? I'm wondering why we didn't have
> any problems with the arm virt machine type. Has nobody tried this
> use case with that? Is the use case something valid for riscv, but
> not arm?
>
Firstly we have to enable pcie passthru on host, find the device groups,
e.g. the vga card, and add their pci ids to host kernel cmdline:
vfio-pci.ids=10de:0f02,10de:0e08
then start vm through qemu as follows:
$Q -machine virt -m 4G -smp 4 -nographic \
-bios /usr/lib/riscv64-linux-gnu/opensbi/generic/fw_jump.elf \
-kernel ./vmlinuz -initrd initrd.img -append "root=/dev/vda1 rw" \
-drive
file=ubuntu-22.04.1-preinstalled-server-riscv64+unmatched.img,if=virtio,format=raw
\
-device vfio-pci,host=01:00.0 -device vfio-pci,host=01:00.1 \
-netdev user,id=vnet,hostfwd=:127.0.0.1:2223-:22 -device
virtio-net-pci,netdev=vnet
Without this patch, qemu exits immediately instead of boots up.
Just tried pcie passthru on arm, it cannot handle 4G memory either.
$Q -m 4G -smp 4 -cpu max -M virt -nographic \
-pflash /usr/share/AAVMF/AAVMF_CODE.fd -pflash flash1.img \
-drive if=none,file=ubuntu-22.04-server-cloudimg-arm64.img,id=hd0 \
-device virtio-blk-device,drive=hd0 \
-device vfio-pci,host=01:00.0 -device vfio-pci,host=01:00.1
qemu-system-aarch64: -device vfio-pci,host=01:00.0: VFIO_MAP_DMA failed:
Invalid argument
qemu-system-aarch64: -device vfio-pci,host=01:00.0: vfio 0000:01:00.0:
failed to setup container for group 11: memory listener initialization
failed: Region mach-virt.ram: vfio_dma_map(0x55de3c2a97f0, 0x40000000,
0x100000000, 0x7f8fcbe00000) = -22 (Invalid argument)
Thanks,
Fei.
> Thanks,
> drew
