Hi On Wed, Mar 11, 2026 at 11:18 AM Chenyi Qiang <[email protected]> wrote: > > > > On 3/11/2026 12:09 AM, Marc-André Lureau wrote: > > Hi > > > > On Tue, Mar 10, 2026 at 6:35 AM Chenyi Qiang <[email protected]> wrote: > >> > >> > >> > >> On 2/26/2026 9:59 PM, [email protected] wrote: > >>> From: Marc-André Lureau <[email protected]> > >>> > >>> Hi, > >>> > >>> This is an attempt to fix the incompatibility of virtio-mem with > >>> confidential > >>> VMs. The solution implements what was discussed earlier with D. > >>> Hildenbrand: > >>> https://patchwork.ozlabs.org/project/qemu-devel/patch/[email protected]/#3502238 > >>> > >>> The first patches are misc cleanups. Then some code refactoring to have > >>> split a > >>> manager/source. And finally, the manager learns to deal with multiple > >>> sources. > >>> > >>> I haven't done thorough testing. I only launched a SEV guest with a > >>> virtio-mem > >>> device. It would be nice to have more tests for those scenarios with > >>> VFIO/virtio-mem/confvm.. In any case, review & testing needed! > >> > >> Is this series aimed to enable virtio-mem (memory hotplug) in a > >> confidential VM? > >> I tested it within a TD guest using the qemu command: > >> > >> qemu-system-x86_64 > >> ... > >> -m 2G,maxmem=10G \ > >> -smp sockets=1,cores=2 \ > >> -object memory-backend-ram,id=mem0,size=2G \ > >> -numa node,nodeid=0,cpus=0-1,memdev=mem0 \ > >> -object memory-backend-ram,id=vmem0,size=8G \ > >> -device > >> virtio-mem-pci,id=vm0,memdev=vmem0,node=0,requested-size=300M \ > >> -object tdx-guest,id=tdx \ > >> -machine q35,kernel_irqchip=split,hpet=off,memory-encryption=tdx \ > >> ... > >> > >> The TD VM will exit with the error when the guest kernel loads the > >> virtio-mem driver: > >> > >> kvm_intel: Guest access before accepting 0x108008000 on vCPU 0 > >> > >> I think it still lacks some support to accept the TD guest memory before > >> using it. > >> > > > > It becomes increasingly hard to provide easy to setup reproducibe > > environments.. but here is the libvirt qemu command I just tested > > succesfully against: > > > > qemu-system-x86_64 -name guest=rhel10,debug-threads=on -S -object > > {"qom-type":"secret","id":"masterKey0","format":"raw","file":"/home/elmarco/.config/libvirt/qemu/lib/domain-3-rhel10/master-key.aes"} > > -machine > > pc-q35-10.1,usb=off,smm=off,dump-guest-core=off,confidential-guest-support=lsec0,hpet=off,acpi=on > > -accel kvm -cpu host,migratable=on -bios > > /usr/share/edk2/ovmf/OVMF.inteltdx.fd -m > > size=4096000k,slots=1,maxmem=8192000k -overcommit mem-lock=off -smp > > 2,sockets=2,cores=1,threads=1 -object > > {"qom-type":"memory-backend-ram","id":"ram-node0","size":4194304000} > > -numa node,nodeid=0,cpus=0-1,memdev=ram-node0 -uuid > > 0f3e6b74-80e3-45e7-a0e6-ae7dfc897606 -display none -no-user-config > > -nodefaults -chardev socket,id=charmonitor,fd=23,server=on,wait=off > > -mon chardev=charmonitor,id=monitor,mode=control -rtc > > base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=delay > > -no-shutdown -global ICH9-LPC.disable_s3=1 -global > > ICH9-LPC.disable_s4=1 -boot strict=on -device > > {"driver":"pcie-root-port","port":8,"chassis":1,"id":"pci.1","bus":"pcie.0","multifunction":true,"addr":"0x1"} > > -device > > {"driver":"pcie-root-port","port":9,"chassis":2,"id":"pci.2","bus":"pcie.0","addr":"0x1.0x1"} > > -device > > {"driver":"pcie-root-port","port":10,"chassis":3,"id":"pci.3","bus":"pcie.0","addr":"0x1.0x2"} > > -device > > {"driver":"pcie-root-port","port":11,"chassis":4,"id":"pci.4","bus":"pcie.0","addr":"0x1.0x3"} > > -device > > {"driver":"pcie-root-port","port":12,"chassis":5,"id":"pci.5","bus":"pcie.0","addr":"0x1.0x4"} > > -device > > {"driver":"pcie-root-port","port":13,"chassis":6,"id":"pci.6","bus":"pcie.0","addr":"0x1.0x5"} > > -device > > {"driver":"pcie-root-port","port":14,"chassis":7,"id":"pci.7","bus":"pcie.0","addr":"0x1.0x6"} > > -device > > {"driver":"pcie-root-port","port":15,"chassis":8,"id":"pci.8","bus":"pcie.0","addr":"0x1.0x7"} > > -device > > {"driver":"pcie-root-port","port":16,"chassis":9,"id":"pci.9","bus":"pcie.0","multifunction":true,"addr":"0x2"} > > -device > > {"driver":"pcie-root-port","port":17,"chassis":10,"id":"pci.10","bus":"pcie.0","addr":"0x2.0x1"} > > -device > > {"driver":"pcie-root-port","port":18,"chassis":11,"id":"pci.11","bus":"pcie.0","addr":"0x2.0x2"} > > -device > > {"driver":"pcie-root-port","port":19,"chassis":12,"id":"pci.12","bus":"pcie.0","addr":"0x2.0x3"} > > -device > > {"driver":"pcie-root-port","port":20,"chassis":13,"id":"pci.13","bus":"pcie.0","addr":"0x2.0x4"} > > -device > > {"driver":"pcie-root-port","port":21,"chassis":14,"id":"pci.14","bus":"pcie.0","addr":"0x2.0x5"} > > -device > > {"driver":"qemu-xhci","p2":15,"p3":15,"id":"usb","bus":"pci.2","addr":"0x0"} > > -device > > {"driver":"virtio-serial-pci","id":"virtio-serial0","bus":"pci.3","addr":"0x0"} > > -object > > {"qom-type":"memory-backend-ram","id":"memvirtiomem0","reserve":false,"size":4194304000} > > -device > > {"driver":"virtio-mem-pci","node":0,"block-size":2097152,"memdev":"memvirtiomem0","id":"virtiomem0","bus":"pci.7","addr":"0x0"} > > -blockdev > > {"driver":"file","filename":"/home/elmarco/rhel-10.qcow2","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"} > > -blockdev > > {"node-name":"libvirt-1-format","read-only":false,"driver":"qcow2","file":"libvirt-1-storage","backing":null} > > -device > > {"driver":"virtio-blk-pci","bus":"pci.4","addr":"0x0","drive":"libvirt-1-format","id":"virtio-disk0","bootindex":1} > > -netdev {"type":"user","id":"hostnet0"} -device > > {"driver":"virtio-net-pci","netdev":"hostnet0","id":"net0","mac":"52:54:00:82:08:40","bus":"pci.1","addr":"0x0"} > > -chardev pty,id=charserial0 -device > > {"driver":"isa-serial","chardev":"charserial0","id":"serial0","index":0} > > -chardev socket,id=charchannel0,fd=22,server=on,wait=off -device > > {"driver":"virtserialport","bus":"virtio-serial0.0","nr":1,"chardev":"charchannel0","id":"channel0","name":"org.qemu.guest_agent.0"} > > -chardev > > socket,id=chrtpm,path=/run/user/1006/libvirt/qemu/run/swtpm/3-rhel10-swtpm.sock > > -tpmdev emulator,id=tpm-tpm0,chardev=chrtpm -device > > {"driver":"tpm-crb","tpmdev":"tpm-tpm0","id":"tpm0"} -audiodev > > {"id":"audio1","driver":"none"} -global ICH9-LPC.noreboot=off > > -watchdog-action reset -device > > {"driver":"virtio-balloon-pci","id":"balloon0","bus":"pci.5","addr":"0x0"} > > -object {"qom-type":"rng-random","id":"objrng0","filename":"/dev/urandom"} > > -device > > {"driver":"virtio-rng-pci","rng":"objrng0","id":"rng0","bus":"pci.6","addr":"0x0"} > > -object > > {"qom-type":"tdx-guest","id":"lsec0","quote-generation-socket":{"type":"unix","path":"/var/run/tdx-qgs/qgs.socket"},"attributes":268435456} > > -sandbox > > on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny > > -msg timestamp=on > > > > Then I hotplugged some mem: > > virsh --connect qemu:///session update-memory-device rhel10 > > --requested-size 2GiB > > > > [ 166.228943] virtio_mem virtio5: plugged size: 0x0 > > [ 166.231784] virtio_mem virtio5: requested size: 0x80000000 > > > > > > [root@rhel10-server ~]# lsmem > > RANGE SIZE STATE REMOVABLE BLOCK > > 0x0000000000000000-0x000000007fffffff 2G online yes 0-15 > > 0x0000000100000000-0x00000001ffffffff 4G online yes 32-63 > > > > Memory block size: 128M > > Total online memory: 6G > > Total offline memory: 0B > > > > The host kernel used is experimental; it's a patched version of > > 6.19.0-rc8 that I don't have the details) > > It seems related to the memory size. Maybe you can have a try with "-m > 2G,maxmem=4G". > I hit the problem when the size <= 2G and no range above 4g at the beginning.
Right, it crashes when I use -m 2G: error: kvm run failed Input/output error EAX=00000000 EBX=00000000 ECX=00000000 EDX=000c06f2 ESI=00000000 EDI=00000000 EBP=00000000 ESP=00000000 EIP=0000fff0 EFL=00000002 [-------] CPL=0 II=0 A20=1 SMM=0 HLT=0 ES =0000 00000000 0000ffff 00009300 CS =f000 ffff0000 0000ffff 00009b00 SS =0000 00000000 0000ffff 00009300 DS =0000 00000000 0000ffff 00009300 FS =0000 00000000 0000ffff 00009300 GS =0000 00000000 0000ffff 00009300 LDT=0000 00000000 0000ffff 00008200 TR =0000 00000000 0000ffff 00008b00 GDT= 00000000 0000ffff IDT= 00000000 0000ffff CR0=60000010 CR2=00000000 CR3=00000000 CR4=00000000 DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 DR6=00000000ffff0ff0 DR7=0000000000000400 EFER=0000000000000000 Code=00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 <00> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 kvm_mem_ioeventfd_del: error deleting ioeventfd: Input/output error (5) This is a bit outside of my comfort zone, maybe Paolo can provide some hints?
