I installed qemu and virt-manager from bullseye-backports and the issue went away. I did not try to replicate this issue on bookworm or sid but I think it's okay to assume that this issue doesn't affect them because their qemu version is newer.
$ qemu-system-x86_64 --version QEMU emulator version 7.1.0 (Debian 1:7.1+dfsg-2~bpo11+3)
