Re: scsi passthrough differs between guests
Greetings Peter, > Sent: Thursday, October 15, 2020 at 2:23 PM > From: "Peter Krempa" > To: "daggs" > Cc: "libvirt-usersredhat.com" > Subject: Re: scsi passthrough differs between guests > > The new syntax was added in libvirt-6.6.0. Your VM probably was started > with libvirt-6.6.0 and you then upgraded your system to newer version. > All running VMs keep their configuration. Restarting it will use the new > syntax and thus break. I've checked the log, first vm boot was with libvirt version: 5.6.0, qemu version: 4.0.0, kernel: 5.3.5 > Cool. I'll CC you on the patches, I need to do some changes to the test > suite first though as we didn't even have unit tests for this case. np. send them when you can. Dagg.
Re: scsi passthrough differs between guests
On Thu, Oct 15, 2020 at 13:14:56 +0200, daggs wrote:
> Greetings Peter,
>
> > Sent: Thursday, October 15, 2020 at 2:01 PM
> > From: "Peter Krempa"
> > To: "daggs"
> > Cc: "libvirt-usersredhat.com"
> > Subject: Re: scsi passthrough differs between guests
> >
> > [...]
> >
> > > -drive file=/dev/sg5,if=none,format=raw,id=drive-hostdev0,readonly=on \
> > > -device
> > > scsi-generic,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-hostdev0,id=hostdev0
> > > \
> >
> > [...]
> >
> > > the bad one:
> > > /usr/bin/qemu-system-x86_64 \
> >
> > [...]
> >
> > > -blockdev
> > > '{"driver":"host_device","filename":"/dev/sg0","node-name":"libvirt-hostdev0-backend","read-only":true}'
> > > \
> > > -device
> > > scsi-generic,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=libvirt-hostdev0-backend,id=hostdev0
> > > \
> >
> > This doesn't corelate with the version numbers you've mentioned, because
> > the "new" syntax which uses -blockdev was present both in 6.7.0 and
> > 6.8.0.
> the "good" syntax was created with libvirt of version which is earlier than
> 6.7, libvirt was upgraded several times since the vm creation.
> I don't know if the syntax is changed every version or not.
The new syntax was added in libvirt-6.6.0. Your VM probably was started
with libvirt-6.6.0 and you then upgraded your system to newer version.
All running VMs keep their configuration. Restarting it will use the new
syntax and thus break.
> > Anyways the problem is almost certainly that the hostdev code doesn't
> > detect that it's a cdrom. We have such a hack in the disk code which
> > turns a 'host_device' into a 'host_cdrom'. I'll try fixing it but I
> > don't have a machine with a cdrom handy, so it would be nice if you
> > could test it afterwards.
> >
> > Thanks for the report.
> >
> >
>
> my system is gentoo, hence, every pkg is compiled, if you provide a patch, I
> can test it easily.
Cool. I'll CC you on the patches, I need to do some changes to the test
suite first though as we didn't even have unit tests for this case.
Re: scsi passthrough differs between guests
Greetings Peter,
> Sent: Thursday, October 15, 2020 at 2:01 PM
> From: "Peter Krempa"
> To: "daggs"
> Cc: "libvirt-usersredhat.com"
> Subject: Re: scsi passthrough differs between guests
>
> [...]
>
> > -drive file=/dev/sg5,if=none,format=raw,id=drive-hostdev0,readonly=on \
> > -device
> > scsi-generic,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-hostdev0,id=hostdev0
> > \
>
> [...]
>
> > the bad one:
> > /usr/bin/qemu-system-x86_64 \
>
> [...]
>
> > -blockdev
> > '{"driver":"host_device","filename":"/dev/sg0","node-name":"libvirt-hostdev0-backend","read-only":true}'
> > \
> > -device
> > scsi-generic,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=libvirt-hostdev0-backend,id=hostdev0
> > \
>
> This doesn't corelate with the version numbers you've mentioned, because
> the "new" syntax which uses -blockdev was present both in 6.7.0 and
> 6.8.0.
the "good" syntax was created with libvirt of version which is earlier than
6.7, libvirt was upgraded several times since the vm creation.
I don't know if the syntax is changed every version or not.
>
> Anyways the problem is almost certainly that the hostdev code doesn't
> detect that it's a cdrom. We have such a hack in the disk code which
> turns a 'host_device' into a 'host_cdrom'. I'll try fixing it but I
> don't have a machine with a cdrom handy, so it would be nice if you
> could test it afterwards.
>
> Thanks for the report.
>
>
my system is gentoo, hence, every pkg is compiled, if you provide a patch, I
can test it easily.
Thanks for the effort,
Dagg.
Re: scsi passthrough differs between guests
On Thu, Oct 15, 2020 at 12:36:08 +0200, daggs wrote:
> Greetings Peter,
>
> > Sent: Thursday, October 15, 2020 at 9:52 AM
> > From: "Peter Krempa"
> > To: "daggs"
> > Cc: "libvirt-usersredhat.com"
> > Subject: Re: scsi passthrough differs between guests
> >
> > I don't see anything wrong with you configs. There were some changes
> > related to SCSI hostdevs, between 6.7.0 an 6.8.0, but none of them
> > should actually impact that use case.
> >
> > Said that, could you please post the actual qemu command lines that
> > libvirt formatted for the two VMs you mention above.
> >
> > The command line can be found in /var/log/libvirt/qemu/$VMNAME.log .
> > Please make sure you post the latest/actual one. It'll help showing
> > whether anything changed between the two or there is a different
> > problem.
> >
>
> here is the good cmd:
> /usr/bin/qemu-system-x86_64 \
[...]
> -drive file=/dev/sg5,if=none,format=raw,id=drive-hostdev0,readonly=on \
> -device
> scsi-generic,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-hostdev0,id=hostdev0
> \
[...]
> the bad one:
> /usr/bin/qemu-system-x86_64 \
[...]
> -blockdev
> '{"driver":"host_device","filename":"/dev/sg0","node-name":"libvirt-hostdev0-backend","read-only":true}'
> \
> -device
> scsi-generic,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=libvirt-hostdev0-backend,id=hostdev0
> \
This doesn't corelate with the version numbers you've mentioned, because
the "new" syntax which uses -blockdev was present both in 6.7.0 and
6.8.0.
Anyways the problem is almost certainly that the hostdev code doesn't
detect that it's a cdrom. We have such a hack in the disk code which
turns a 'host_device' into a 'host_cdrom'. I'll try fixing it but I
don't have a machine with a cdrom handy, so it would be nice if you
could test it afterwards.
Thanks for the report.
Re: scsi passthrough differs between guests
Greetings Peter,
> Sent: Thursday, October 15, 2020 at 9:52 AM
> From: "Peter Krempa"
> To: "daggs"
> Cc: "libvirt-usersredhat.com"
> Subject: Re: scsi passthrough differs between guests
>
> I don't see anything wrong with you configs. There were some changes
> related to SCSI hostdevs, between 6.7.0 an 6.8.0, but none of them
> should actually impact that use case.
>
> Said that, could you please post the actual qemu command lines that
> libvirt formatted for the two VMs you mention above.
>
> The command line can be found in /var/log/libvirt/qemu/$VMNAME.log .
> Please make sure you post the latest/actual one. It'll help showing
> whether anything changed between the two or there is a different
> problem.
>
here is the good cmd:
/usr/bin/qemu-system-x86_64 \
-name guest=debian10,debug-threads=on \
-S \
-object
secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-1-debian10/master-key.aes
\
-blockdev
'{"driver":"file","filename":"/usr/share/edk2-ovmf/OVMF_CODE.fd","node-name":"libvirt-pflash0-storage","auto-read-only":true,"discard":"unmap"}'
\
-blockdev
'{"node-name":"libvirt-pflash0-format","read-only":true,"driver":"raw","file":"libvirt-pflash0-storage"}'
\
-blockdev
'{"driver":"file","filename":"/var/lib/libvirt/qemu/nvram/debian9_VARS.fd","node-name":"libvirt-pflash1-storage","auto-read-only":true,"discard":"unmap"}'
\
-blockdev
'{"node-name":"libvirt-pflash1-format","read-only":false,"driver":"raw","file":"libvirt-pflash1-storage"}'
\
-machine
pc-q35-4.0,accel=kvm,usb=off,vmport=off,dump-guest-core=off,pflash0=libvirt-pflash0-format,pflash1=libvirt-pflash1-format
\
-cpu host,migratable=on \
-m 6144 \
-overcommit mem-lock=off \
-smp 4,sockets=4,cores=1,threads=1 \
-uuid 84af935f-0afd-4021-a431-b6408a53efea \
-no-user-config \
-nodefaults \
-chardev socket,id=charmonitor,fd=26,server,nowait \
-mon chardev=charmonitor,id=monitor,mode=control \
-rtc base=utc,driftfix=slew \
-global kvm-pit.lost_tick_policy=delay \
-no-hpet \
-no-shutdown \
-global ICH9-LPC.disable_s3=1 \
-global ICH9-LPC.disable_s4=1 \
-boot strict=on \
-device
pcie-root-port,port=0x10,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,addr=0x2
\
-device pcie-root-port,port=0x11,chassis=2,id=pci.2,bus=pcie.0,addr=0x2.0x1 \
-device pcie-root-port,port=0x12,chassis=3,id=pci.3,bus=pcie.0,addr=0x2.0x2 \
-device pcie-root-port,port=0x13,chassis=4,id=pci.4,bus=pcie.0,addr=0x2.0x3 \
-device pcie-root-port,port=0x14,chassis=5,id=pci.5,bus=pcie.0,addr=0x2.0x4 \
-device pcie-root-port,port=0x15,chassis=6,id=pci.6,bus=pcie.0,addr=0x2.0x5 \
-device pcie-root-port,port=0x16,chassis=7,id=pci.7,bus=pcie.0,addr=0x2.0x6 \
-device qemu-xhci,p2=15,p3=15,id=usb,bus=pci.2,addr=0x0 \
-device virtio-scsi-pci,id=scsi0,bus=pci.7,addr=0x0 \
-device virtio-serial-pci,id=virtio-serial0,bus=pci.3,addr=0x0 \
-blockdev
'{"driver":"file","filename":"/home/virt_admin/Machines/kvm/debian10.qcow2","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}'
\
-blockdev
'{"node-name":"libvirt-1-format","read-only":false,"driver":"qcow2","file":"libvirt-1-storage","backing":null}'
\
-device
virtio-blk-pci,bus=pci.4,addr=0x0,drive=libvirt-1-format,id=virtio-disk0,bootindex=2
\
-netdev tap,fd=28,id=hostnet0 \
-device
virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:11:92:dd,bus=pci.1,addr=0x0
\
-chardev pty,id=charserial0 \
-device isa-serial,chardev=charserial0,id=serial0 \
-chardev socket,id=charchannel0,fd=29,server,nowait \
-device
virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0
\
-vnc 127.0.0.1:0 \
-k en-us \
-device
qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vram64_size_mb=0,vgamem_mb=16,max_outputs=1,bus=pcie.0,addr=0x1
\
-device ich9-intel-hda,id=sound0,bus=pcie.0,addr=0x1b \
-device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 \
-drive file=/dev/sg5,if=none,format=raw,id=drive-hostdev0,readonly=on \
-device
scsi-generic,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-hostdev0,id=hostdev0
\
-device virtio-balloon-pci,id=balloon0,bus=pci.5,addr=0x0 \
-object rng-random,id=objrng0,filename=/dev/urandom \
-device virtio-rng-pci,rng=objrng0,id=rng0,bus=pci.6,addr=0x0 \
-sandbox
on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
-msg timestamp=on
the bad one:
/usr/bin/qemu-system-x86_64 \
-name guest=streamer-vm-q35,debug-threads=on \
-S \
-object
secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-12-streamer-vm-q35/master-key.aes
\
-blockdev
'{"driver":"file","filename":"/usr/share/qemu/edk2-x86_64-secure-code.fd","node-name":"libvirt-pflash0-storage","auto-read-only":true,"discard":"unmap"}'
\
-blockdev
'{"node-name":"libvirt-pflash0-format","read-only":true,"driver":"raw","file":"libvirt-pflash0-storage"}'
\
-blockdev
'{"driver":"file","filename":"/var/lib/libvirt/qemu/nvram/streamer-vm-q35_VARS.fd","node-name":"libvirt-pflash1-storage","auto-read-only":true,"discard":"unmap"}'
\
-blockdev
'{"no
