On 5/27/24 02:46, Dmitry Osipenko wrote: > On 5/22/24 12:00, Alex Bennée wrote: >> Dmitry Osipenko <dmitry.osipe...@collabora.com> writes: >> >>> On 5/21/24 17:57, Alex Bennée wrote: >>>> Alex Bennée <alex.ben...@linaro.org> writes: >>>> >>>>> Dmitry Osipenko <dmitry.osipe...@collabora.com> writes: >>>>> >>>>>> Hello, >>>>>> >>>>>> This series enables Vulkan Venus context support on virtio-gpu. >>>>>> >>>>>> All virglrender and almost all Linux kernel prerequisite changes >>>>>> needed by Venus are already in upstream. For kernel there is a pending >>>>>> KVM patchset that fixes mapping of compound pages needed for DRM drivers >>>>>> using TTM [1], othewrwise hostmem blob mapping will fail with a KVM error >>>>>> from Qemu. >>>>>> >>>>>> [1] >>>>>> https://lore.kernel.org/kvm/20240229025759.1187910-1-steve...@google.com/ >>>>>> >>>>>> You'll need to use recent Mesa version containing patch that removes >>>>>> dependency on cross-device feature from Venus that isn't supported by >>>>>> Qemu [2]. >>>>>> >>>>>> [2] >>>>>> https://gitlab.freedesktop.org/mesa/mesa/-/commit/087e9a96d13155e26987befae78b6ccbb7ae242b >>>>>> >>>>>> Example Qemu cmdline that enables Venus: >>>>>> >>>>>> qemu-system-x86_64 -device >>>>>> virtio-vga-gl,hostmem=4G,blob=true,venus=true \ >>>>>> -machine q35,accel=kvm,memory-backend=mem1 \ >>>>>> -object memory-backend-memfd,id=mem1,size=8G -m 8G >>>>> >>>>> What is the correct device for non-x86 guests? We have virtio-gpu-gl-pci >>>>> but when doing that I get: >>>>> >>>>> -device virtio-gpu-gl-pci,hostmem=4G,blob=true,venus=true >>>>> qemu-system-aarch64: -device >>>>> virtio-gpu-gl-pci,hostmem=4G,blob=true,venus=true: opengl is not available >>>>> >>>>> According to 37f86af087 (virtio-gpu: move virgl realize + properties): >>>>> >>>>> Drop the virgl property, the virtio-gpu-gl-device has virgl enabled no >>>>> matter what. Just use virtio-gpu-device instead if you don't want >>>>> enable virgl and opengl. This simplifies the logic and reduces the test >>>>> matrix. >>>>> >>>>> but that's not a good solution because that needs virtio-mmio and there >>>>> are reasons to have a PCI device (for one thing no ambiguity about >>>>> discovery). >>>> >>>> Oops my mistake forgetting: >>>> >>>> --display gtk,gl=on >>>> >>>> Although I do see a lot of eglMakeContext failures. >>> >>> Please post the full Qemu cmdline you're using >> >> With: >> >> ./qemu-system-aarch64 \ >> -machine type=virt,virtualization=on,pflash0=rom,pflash1=efivars \ >> -cpu neoverse-n1 \ >> -smp 4 \ >> -accel tcg \ >> -device virtio-net-pci,netdev=unet \ >> -device virtio-scsi-pci \ >> -device scsi-hd,drive=hd \ >> -netdev user,id=unet,hostfwd=tcp::2222-:22 \ >> -blockdev >> driver=raw,node-name=hd,file.driver=host_device,file.filename=/dev/zen-ssd2/trixie-arm64,discard=unmap >> \ >> -serial mon:stdio \ >> -blockdev >> node-name=rom,driver=file,filename=(pwd)/pc-bios/edk2-aarch64-code.fd,read-only=true >> \ >> -blockdev >> node-name=efivars,driver=file,filename=$HOME/images/qemu-arm64-efivars \ >> -m 8192 \ >> -object memory-backend-memfd,id=mem,size=8G,share=on \ >> -device virtio-gpu-gl-pci,hostmem=4G,blob=true,venus=true \ >> -display gtk,gl=on,show-cursor=on -vga none \ >> -device qemu-xhci -device usb-kbd -device usb-tablet >> >> I get a boot up with a lot of: >> >> >> >> >> (qemu:1545322): Gdk-WARNING **: 09:26:09.470: eglMakeCurrent failed >> >> >> >> (qemu:1545322): Gdk-WARNING **: 09:26:09.470: eglMakeCurrent failed >> >> >> >> (qemu:1545322): Gdk-WARNING **: 09:26:09.470: eglMakeCurrent failed >> >> >> >> (qemu:1545322): Gdk-WARNING **: 09:26:09.470: eglMakeCurrent failed >> >> >> In the guest I run: >> >> meson devenv -C /root/lsrc/graphics/mesa.git/build fish >> >> to bring in the latest Mesa (with virtio enabled). Running vulkaninfo >> reports two cards: >> >> ========== >> >> VULKANINFO >> ========== >> >> >> Vulkan Instance Version: 1.3.280 >> >> >> >> Instance Extensions: count = 14 >> ------------------------------- >> VK_EXT_debug_report : extension revision 10 >> VK_EXT_debug_utils : extension revision 2 >> VK_EXT_headless_surface : extension revision 1 >> VK_KHR_device_group_creation : extension revision 1 >> VK_KHR_external_fence_capabilities : extension revision 1 >> VK_KHR_external_memory_capabilities : extension revision 1 >> VK_KHR_external_semaphore_capabilities : extension revision 1 >> VK_KHR_get_physical_device_properties2 : extension revision 2 >> VK_KHR_get_surface_capabilities2 : extension revision 1 >> VK_KHR_portability_enumeration : extension revision 1 >> VK_KHR_surface : extension revision 25 >> VK_KHR_surface_protected_capabilities : extension revision 1 >> VK_KHR_wayland_surface : extension revision 6 >> VK_LUNARG_direct_driver_loading : extension revision 1 >> >> Instance Layers: count = 2 >> -------------------------- >> VK_LAYER_MESA_device_select Linux device selection layer 1.3.211 version 1 >> VK_LAYER_MESA_overlay Mesa Overlay layer 1.3.211 >> version 1 >> >> Devices: >> ======== >> GPU0: >> apiVersion = 1.3.230 >> driverVersion = 24.1.99 >> vendorID = 0x8086 >> deviceID = 0xa780 >> deviceType = PHYSICAL_DEVICE_TYPE_INTEGRATED_GPU >> deviceName = Virtio-GPU Venus (Intel(R) Graphics (RPL-S)) >> driverID = DRIVER_ID_MESA_VENUS >> driverName = venus >> driverInfo = Mesa 24.2.0-devel (git-0b582449f0) >> conformanceVersion = 1.3.0.0 >> deviceUUID = 29d2e940-a1a0-3054-0f9a-9f7dec52a084 >> driverUUID = 3694c390-f245-612a-12ce-7d3a99127622 >> GPU1: >> apiVersion = 1.2.0 >> driverVersion = 24.1.99 >> vendorID = 0x10005 >> deviceID = 0x0000 >> deviceType = PHYSICAL_DEVICE_TYPE_CPU >> deviceName = Virtio-GPU Venus (llvmpipe (LLVM 15.0.6, 256 >> bits)) >> driverID = DRIVER_ID_MESA_VENUS >> driverName = venus >> driverInfo = Mesa 24.2.0-devel (git-0b582449f0) >> conformanceVersion = 1.3.0.0 >> deviceUUID = 5fb5c03f-c537-f0fe-a7e6-9cd5866acb8d >> driverUUID = 3694c390-f245-612a-12ce-7d3a99127622 >> >> Running weston and then vkcube-wayland reports its selecting "GPU 0: >> Virtio-GPU Venus (Intel(R) Graphics (RPL-S))" but otherwise produces no >> output. >> >> If I run with "-display sdl,gl=on,show-cursor=on" and the same other >> command line options the results for vulkaninfo are the same. However >> vkcube-wayland gets a little further and draws the initial cube on the >> screen before locking up with: >> >> MESA-VIRTIO: debug: stuck in fence wait with iter at xxxx >> >> where xxxx grows each time it prints. On shutting down I see some virgl >> errors interspersed with the systemd logs: >> >> [drm:virtio_gpu_dequeue_ctrl_func [virtio_gpu]] *ERROR* response 0x1200 >> (command 0x209) >> [ OK ] Stopped systemd-logind.service - User Login Management. >> virtio_gpu_virgl_process_cmd: ctrl 0x209, error 0x1200 >> [ 475.257111] [drm:virtio_gpu_dequeue_ctrl_func [virtio_gpu]] *ERROR* >> response 0x1200 (command 0x209) >> [drm:virtio_gpu_dequeue_ctrl_func [virtio_gpu]] *ERROR* response 0x1200 >> (command 0x209) >> [ OK ] Stopped target network-online.target - Network is Online. >> [ OK ] Stopped target remote-fs.target - Remote File Systems. >> [ OK ] Stopped NetworkManager-wait-online…vice - Network Manager Wait >> Online. >> Stopping avahi-daemon.service - Avahi mDNS/DNS-SD Stack... >> Stopping cups.service - CUPS Scheduler... >> Stopping user-runtime-dir@0.servic…er Runtime Directory >> /run/user/0... >> [ OK ] Stopped avahi-daemon.service - Avahi mDNS/DNS-SD Stack. >> [ OK ] Stopped cups.service - CUPS Scheduler. >> virtio_gpu_virgl_process_cmd: ctrl 0x209, error 0x1200 >> [ 475.357543] [drm:virtio_gpu_dequeue_ctrl_func [virtio_gpu]] *ERROR* >> response 0x1200 (command 0x209) >> [drm:virtio_gpu_dequeue_ctrl_func [virtio_gpu]] *ERROR* response 0x1200 >> (command 0x209) >> [ OK ] Stopped target network.target - Network. >> [ OK ] Stopped target nss-user-lookup.target - User and Group Name >> Lookups. >> Stopping NetworkManager.service - Network Manager... >> Stopping networking.service - Raise network interfaces... >> Stopping wpa_supplicant.service - WPA supplicant... >> [ OK ] Stopped wpa_supplicant.service - WPA supplicant. >> virtio_gpu_virgl_process_cmd: ctrl 0x209, error 0x1200 >> [ 493.585261] [drm:virtio_gpu_dequeue_ctrl_func [virtio_gpu]] *ERROR* >> response 0x1200 (command 0x209) >> [drm:virtio_gpu_dequeue_ctrl_func [virtio_gpu]] *ERROR* response 0x1200 >> (command 0x209) >> > > I've reproduced this with qemu-system-aarch64. Vkcube works for a second > and then stops, Qemu compeltely gets frozen after closing and re-running > vkcube. Doesn't feel like this is a problem with venus, but with arm64. > For now don't know where is the bug, will take a closer look.
Interestingly, on another try vkcube now works with no issues. -- Best regards, Dmitry
