Re: Virtio fix for testing

2023-10-21 Thread Andrew Cagney
On Mon, 21 Aug 2023 at 15:31, Andrew Cagney  wrote:
>
> On Sun, 20 Aug 2023 at 06:23, Stefan Fritsch  wrote:
> >
> > Am 13.08.23 um 17:38 schrieb Tobias Heider:

> > You could try something like
> >
> > -device virtio-scsi-pci,id=scsi
> > -drive file=install73.iso,format=raw,id=cdinst,if=none
> > -device scsi-cd,drive=cdinst
>
> good idea (just need to translate it to virt-install speak)
>
> > That depends on your seabios having support for virtio cdroms. Not sure
> > if that is the default by now.
> >
> > Or maybe try a SATA cdrom, but you would need to figure out the qemu
> > options for that yourself.

This is what I did.  Using virt-install and replacing --cdrom with
something like:

+KVM_OPENBSD_VIRT_INSTALL_FLAGS = \
+ --disk path=$(KVM_OPENBSD_BASE_ISO),readonly=on,device=cdrom,target.bus=sata \
+ --install bootdev=cdrom

I can boot/install 7.4's ISO.

thanks,
Andrew



Re: Virtio fix for testing

2023-08-21 Thread Tobias Heider
On Sun, Aug 20, 2023 at 12:23:49PM +0200, Stefan Fritsch wrote:
> Am 13.08.23 um 17:38 schrieb Tobias Heider:
> > On Sun, Aug 13, 2023 at 08:33:54AM -0400, Andrew Cagney wrote:
> > > > Hi Andrew,
> > > > 
> > > > can you share the qemu cmd you are using in your tests?
> > > > I'd like to see if I can reproduce this.
> > > 
> > > Here's pretty much everything.  Thanks for looking at it.
> > 
> > Thank you, I managed to reproduce your crash.
> > I am not yet sure what the exact problem is but you could try using
> > install73.img > instead of install73.iso. It looks like only the iso
> > triggers the bug
> for me.
> 
> -cdrom makes qemu add an ATA cdrom drive. This issue has nothing to do with
> the virtio scsi issue / fix from May.
> 
> The "wdc_atapi_start" here
> 
> >> --:-- ETAwdc_atapi_start: not ready, st = 50
> 
> also points to the problem being related to ATA.
> 

That matches what I'm seeing. I can reliably reproduce the crash here.
My debug prints show that xfer->chp seems to be garbage:

  wdcstart: xfer: 0xfd8016abeea8 xfer->chp: 0x8007f710
  wdc_free_xfer: TAILQ_REMOVE(0xfd8016abeea8)
  wdcstart: xfer: 0xfd8016abeea8 xfer->chp: 0x8007f710
  wdc_free_xfer: TAILQ_REMOVE(0xfd8016abeea8)
  wdcstart: xfer: 0xfd8016abeea8 xfer->chp: 0x75d4af0594eaf807

in:

  887 /* adjust chp, in case we have a shared queue */
  888 chp = xfer->chp;
  889
  890 if ((chp->ch_flags & WDCF_ACTIVE) != 0 ) {

I haven't had time yet to bisect if and find out when and why that happens.

trace:

wdcstart(8007f710) at wdcstart+0x38 [/usr/src/sys/dev/ic/wdc .c:890]
wdc_atapi_the_machine(8007f710,fd8016abeea8,2) at wdc_atapi_the_mac 
hine+0x14a [/usr/src/sys/dev/atapiscsi/atapiscsi.c:640]
wdc_atapi_intr(8007f710,fd8016abeea8,1) at wdc_atapi_intr+0x47 
[/usr/src/sys/dev/atapiscsi/atapiscsi.c:550]
wdcintr(8007f710) at wdcintr+0xae [/usr/g/src/sys/dev/ic/wdc.c :969]
intr_handler(8aface68,8006a100) at intr_handler+0x26 
[/usr/src/sys/arch/amd64/amd64/intr.c:537]
Xintr_ioapic_edge15_untramp() at Xintr_ioapic_edge15_untramp+0x18f Xspllower() 
at Xspllower+0x10
uvm_fault_upper(8afad0d8,8afad110,8afacfd0,0) at uvm_fa 
ult_upper+0x1b6 [/usr/src/sys/uvm/uvm_fault.c:1102]
uvm_fault(fd801785ee60,29fd78000,0,2) at uvm_fault+0xb4 
[/usr/src/sys/uvm/uvm_fault.c:0]
upageflttrap(8afad230,29fd78da8) at upageflttrap+0x4d 
[/usr/src/sys/arch/amd64/amd64/trap.c:189]
usertrap(8afad230) at usertrap+0xbd 
[/usr/src/sys/arch/amd64/amd64/trap.c:436]



Re: Virtio fix for testing

2023-08-21 Thread Andrew Cagney
On Sun, 20 Aug 2023 at 06:23, Stefan Fritsch  wrote:
>
> Am 13.08.23 um 17:38 schrieb Tobias Heider:
> > On Sun, Aug 13, 2023 at 08:33:54AM -0400, Andrew Cagney wrote:
> >>> Hi Andrew,
> >>>
> >>> can you share the qemu cmd you are using in your tests?
> >>> I'd like to see if I can reproduce this.
> >>
> >> Here's pretty much everything.  Thanks for looking at it.
> >
> > Thank you, I managed to reproduce your crash.
> > I am not yet sure what the exact problem is but you could try using 
> > install73.img > instead of install73.iso. It looks like only the iso 
> > triggers the bug
> for me.
>
> -cdrom makes qemu add an ATA cdrom drive. This issue has nothing to do
> with the virtio scsi issue / fix from May.
>
> The "wdc_atapi_start" here
>
>  >> --:-- ETAwdc_atapi_start: not ready, st = 50
>
> also points to the problem being related to ATA.
>
> You could try something like
>
> -device virtio-scsi-pci,id=scsi
> -drive file=install73.iso,format=raw,id=cdinst,if=none
> -device scsi-cd,drive=cdinst

good idea (just need to translate it to virt-install speak)

> That depends on your seabios having support for virtio cdroms. Not sure
> if that is the default by now.
>
> Or maybe try a SATA cdrom, but you would need to figure out the qemu
> options for that yourself.
>
> Oh, and the exact qemu version you used may be interesting for people
> trying to debug this.

virt-install-4.1.0-2.fc38.noarch
qemu-7.2.4-2.fc38.x86_64



Re: Virtio fix for testing

2023-08-20 Thread Stefan Fritsch

Am 13.08.23 um 17:38 schrieb Tobias Heider:

On Sun, Aug 13, 2023 at 08:33:54AM -0400, Andrew Cagney wrote:

Hi Andrew,

can you share the qemu cmd you are using in your tests?
I'd like to see if I can reproduce this.


Here's pretty much everything.  Thanks for looking at it.


Thank you, I managed to reproduce your crash.
I am not yet sure what the exact problem is but you could try using install73.img > instead of install73.iso. It looks like only the iso triggers the bug 

for me.

-cdrom makes qemu add an ATA cdrom drive. This issue has nothing to do 
with the virtio scsi issue / fix from May.


The "wdc_atapi_start" here

>> --:-- ETAwdc_atapi_start: not ready, st = 50

also points to the problem being related to ATA.

You could try something like

-device virtio-scsi-pci,id=scsi
-drive file=install73.iso,format=raw,id=cdinst,if=none
-device scsi-cd,drive=cdinst

That depends on your seabios having support for virtio cdroms. Not sure 
if that is the default by now.


Or maybe try a SATA cdrom, but you would need to figure out the qemu 
options for that yourself.


Oh, and the exact qemu version you used may be interesting for people 
trying to debug this.


Cheers,
Stefan






virt-install \
--connect=qemu:///system \
  --check=path_in_use=off \
  --graphics=none \
  --virt-type=kvm \
  --noreboot \
  --console=pty,target_type=serial \
  --cpu=host-passthrough \
  --network=network:swandefault,model=virtio \
  --rng=type=random,device=/dev/random \
  --security=type=static,model=dac,label='1000:107',relabel=yes \
--vcpus=1 \
--memory=2048 \
--name=w.openbsd-base \
--os-variant=openbsd7.3 \
--disk=path=/home/libreswan/pool/w.openbsd-base.qcow2,size=10,bus=virtio,format=qcow2
\
--filesystem=target=pool,type=mount,accessmode=squash,source=/home/libreswan/pool
\
--cdrom=/home/libreswan/pool/w.openbsd-base.iso

base.conf which gets added to the iso looks like:

#install.conf file for OpenBSD
Terminal type? = com0
System hostname = openbsd
Which network interface do you wish to configure? = vio0
IPv4 address for = dhcp
DNS Domain name = testing.libreswan.org
Password for root account? =
$2a$12$YZ8bMn19IHPQpBoD6Xf/re/4pp2kbJtVkIl/Mc4G3WA96qyG7/6qW
Start sshd(8) by default = yes
Start ntpd(8) by default? = no
NTP server? (hostname or 'default') = default
Do you expect to run the X Window System? = no
Do you want the X Window System to be started by xdm(1)? = no
Which speed should com0 use? (or 'done') = 19200
What timezone are you in? = EST
Change the default console to com0? = yes
Setup a user? = no
Allow root ssh login = yes
Use (W)hole disk or (E)dit the MBR? = W
URL to autopartitioning template for disklabel? = file:/base.disk
Which disk is the root disk? = sd0
Use DUIDs rather than device names in fstab? = yes
Which disk do you wish to initialize? = done
Set name(s)? = all
Location of sets? = cd0
oPathname to the sets = 7.1/amd64
Directory does not contain SHA256.sig. Continue without verification? = yes

this is the console log, part way through pexpect feeds the VM
commands to start the installer:

cannot open cd0a:/etc/random.seed: No such file or directory
booting cd0a:/7.3/amd64/bsd.rd: 3973828+1655808+3882568+0+708608
[109+444720+297256]=0xa76648
entry point at 0x81001000
Copyright (c) 1982, 1986, 1989, 1991, 1993
The Regents of the University of California.  All rights reserved.
Copyright (c) 1995-2023 OpenBSD. All rights reserved.  https://www.OpenBSD.org

OpenBSD 7.3-current (RAMDISK_CD) #1262: Sat Aug 12 11:54:24 MDT 2023
 dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/RAMDISK_CD
real mem = 2130542592 (2031MB)
avail mem = 2062016512 (1966MB)
random: good seed from bootblocks
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.8 @ 0xf59a0 (9 entries)
bios0: vendor SeaBIOS version "1.16.2-1.fc38" date 04/01/2014
bios0: QEMU Standard PC (i440FX + PIIX, 1996)
acpi0 at bios0: ACPI 1.0
acpi0: tables DSDT FACP APIC WAET
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: AMD Ryzen 9 3950X 16-Core Processor, 3500.43 MHz, 17-71-00
cpu0: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,SSE3,PCLMUL,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,HV,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,CPCTR,FSGSBASE,TSC_ADJUST,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,SHA,UMIP,STIBP,SSBD,IBPB,STIBP,SSBD,VIRTSSBD,XSAVEOPT,XSAVEC,XGETBV1
cpu0: 64KB 64b/line 2-way D-cache, 64KB 64b/line 2-way I-cache
cpu0: 512KB 64b/line 16-way L2 cache
cpu0: apic clock running at 1000MHz
ioapic0 at mainbus0: apid 0 pa 0xfec0, version 11, 24 pins
acpiprt0 at acpi0: bus 0 (PCI0)
"ACPI0006" at acpi0 not configured
acpipci0 at acpi0 PCI0
"PNP0A06" at acpi0 not configured
"PNP0A06" at acpi0 not configured
"PNP0A06" at acpi0 not configured
"QEMU0002" at acpi0 not configured
com0 at acpi0 COM1 addr 0x3f8/0x8 irq 4: ns16550a, 16 byte 

Re: Virtio fix for testing

2023-08-13 Thread Stuart Henderson
On 2023/08/13 16:10, Andrew Cagney wrote:
> On Sun, 13 Aug 2023 at 11:38, Tobias Heider  wrote:
> >
> > On Sun, Aug 13, 2023 at 08:33:54AM -0400, Andrew Cagney wrote:
> > > > Hi Andrew,
> > > >
> > > > can you share the qemu cmd you are using in your tests?
> > > > I'd like to see if I can reproduce this.
> > >
> > > Here's pretty much everything.  Thanks for looking at it.
> >
> > Thank you, I managed to reproduce your crash.
> 
> Some good news (of sorts).
> 
> > I am not yet sure what the exact problem is but you could try using 
> > install73.img
> > instead of install73.iso. It looks like only the iso triggers the bug for 
> > me.
> 
> I'll need to think about how to do this.  The script is adding files
> to the iso so they are available after the installer has run.  I do
> see there's an EFI partition.

If the installer is run in an environment with network access,
fetching sets from http instead of cd/disk might do the trick.



Re: Virtio fix for testing

2023-08-13 Thread Andrew Cagney
On Sun, 13 Aug 2023 at 11:38, Tobias Heider  wrote:
>
> On Sun, Aug 13, 2023 at 08:33:54AM -0400, Andrew Cagney wrote:
> > > Hi Andrew,
> > >
> > > can you share the qemu cmd you are using in your tests?
> > > I'd like to see if I can reproduce this.
> >
> > Here's pretty much everything.  Thanks for looking at it.
>
> Thank you, I managed to reproduce your crash.

Some good news (of sorts).

> I am not yet sure what the exact problem is but you could try using 
> install73.img
> instead of install73.iso. It looks like only the iso triggers the bug for me.

I'll need to think about how to do this.  The script is adding files
to the iso so they are available after the installer has run.  I do
see there's an EFI partition.

thanks



Re: Virtio fix for testing

2023-08-13 Thread Tobias Heider
On Sun, Aug 13, 2023 at 08:33:54AM -0400, Andrew Cagney wrote:
> > Hi Andrew,
> >
> > can you share the qemu cmd you are using in your tests?
> > I'd like to see if I can reproduce this.
> 
> Here's pretty much everything.  Thanks for looking at it.

Thank you, I managed to reproduce your crash.
I am not yet sure what the exact problem is but you could try using 
install73.img
instead of install73.iso. It looks like only the iso triggers the bug for me.

> 
> virt-install \
> --connect=qemu:///system \
>  --check=path_in_use=off \
>  --graphics=none \
>  --virt-type=kvm \
>  --noreboot \
>  --console=pty,target_type=serial \
>  --cpu=host-passthrough \
>  --network=network:swandefault,model=virtio \
>  --rng=type=random,device=/dev/random \
>  --security=type=static,model=dac,label='1000:107',relabel=yes \
> --vcpus=1 \
> --memory=2048 \
> --name=w.openbsd-base \
> --os-variant=openbsd7.3 \
> --disk=path=/home/libreswan/pool/w.openbsd-base.qcow2,size=10,bus=virtio,format=qcow2
> \
> --filesystem=target=pool,type=mount,accessmode=squash,source=/home/libreswan/pool
> \
> --cdrom=/home/libreswan/pool/w.openbsd-base.iso
> 
> base.conf which gets added to the iso looks like:
> 
> #install.conf file for OpenBSD
> Terminal type? = com0
> System hostname = openbsd
> Which network interface do you wish to configure? = vio0
> IPv4 address for = dhcp
> DNS Domain name = testing.libreswan.org
> Password for root account? =
> $2a$12$YZ8bMn19IHPQpBoD6Xf/re/4pp2kbJtVkIl/Mc4G3WA96qyG7/6qW
> Start sshd(8) by default = yes
> Start ntpd(8) by default? = no
> NTP server? (hostname or 'default') = default
> Do you expect to run the X Window System? = no
> Do you want the X Window System to be started by xdm(1)? = no
> Which speed should com0 use? (or 'done') = 19200
> What timezone are you in? = EST
> Change the default console to com0? = yes
> Setup a user? = no
> Allow root ssh login = yes
> Use (W)hole disk or (E)dit the MBR? = W
> URL to autopartitioning template for disklabel? = file:/base.disk
> Which disk is the root disk? = sd0
> Use DUIDs rather than device names in fstab? = yes
> Which disk do you wish to initialize? = done
> Set name(s)? = all
> Location of sets? = cd0
> oPathname to the sets = 7.1/amd64
> Directory does not contain SHA256.sig. Continue without verification? = yes
> 
> this is the console log, part way through pexpect feeds the VM
> commands to start the installer:
> 
> cannot open cd0a:/etc/random.seed: No such file or directory
> booting cd0a:/7.3/amd64/bsd.rd: 3973828+1655808+3882568+0+708608
> [109+444720+297256]=0xa76648
> entry point at 0x81001000
> Copyright (c) 1982, 1986, 1989, 1991, 1993
> The Regents of the University of California.  All rights reserved.
> Copyright (c) 1995-2023 OpenBSD. All rights reserved.  https://www.OpenBSD.org
> 
> OpenBSD 7.3-current (RAMDISK_CD) #1262: Sat Aug 12 11:54:24 MDT 2023
> dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/RAMDISK_CD
> real mem = 2130542592 (2031MB)
> avail mem = 2062016512 (1966MB)
> random: good seed from bootblocks
> mainbus0 at root
> bios0 at mainbus0: SMBIOS rev. 2.8 @ 0xf59a0 (9 entries)
> bios0: vendor SeaBIOS version "1.16.2-1.fc38" date 04/01/2014
> bios0: QEMU Standard PC (i440FX + PIIX, 1996)
> acpi0 at bios0: ACPI 1.0
> acpi0: tables DSDT FACP APIC WAET
> acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
> cpu0 at mainbus0: apid 0 (boot processor)
> cpu0: AMD Ryzen 9 3950X 16-Core Processor, 3500.43 MHz, 17-71-00
> cpu0: 
> FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,SSE3,PCLMUL,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,HV,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,CPCTR,FSGSBASE,TSC_ADJUST,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,SHA,UMIP,STIBP,SSBD,IBPB,STIBP,SSBD,VIRTSSBD,XSAVEOPT,XSAVEC,XGETBV1
> cpu0: 64KB 64b/line 2-way D-cache, 64KB 64b/line 2-way I-cache
> cpu0: 512KB 64b/line 16-way L2 cache
> cpu0: apic clock running at 1000MHz
> ioapic0 at mainbus0: apid 0 pa 0xfec0, version 11, 24 pins
> acpiprt0 at acpi0: bus 0 (PCI0)
> "ACPI0006" at acpi0 not configured
> acpipci0 at acpi0 PCI0
> "PNP0A06" at acpi0 not configured
> "PNP0A06" at acpi0 not configured
> "PNP0A06" at acpi0 not configured
> "QEMU0002" at acpi0 not configured
> com0 at acpi0 COM1 addr 0x3f8/0x8 irq 4: ns16550a, 16 byte fifo
> com0: console
> acpicmos0 at acpi0
> "ACPI0010" at acpi0 not configured
> acpicpu at acpi0 not configured
> pvbus0 at mainbus0: KVM
> pci0 at mainbus0 bus 0
> pchb0 at pci0 dev 0 function 0 "Intel 82441FX" rev 0x02
> "Intel 82371SB ISA" rev 0x00 at pci0 dev 1 function 0 not configured
> pciide0 at pci0 dev 1 function 1 "Intel 82371SB IDE" rev 0x00: DMA,
> channel 0 wired to compatibility, channel 1 wired to compatibility
> atapiscsi0 at pciide0 channel 0 drive 0
> scsibus0 at atapiscsi0: 2 targets
> cd0 at scsibus0 targ 0 lun 0:  removable
> cd0(pciide0:0:0): 

Re: Virtio fix for testing

2023-08-13 Thread Andrew Cagney
> Hi Andrew,
>
> can you share the qemu cmd you are using in your tests?
> I'd like to see if I can reproduce this.

Here's pretty much everything.  Thanks for looking at it.

virt-install \
--connect=qemu:///system \
 --check=path_in_use=off \
 --graphics=none \
 --virt-type=kvm \
 --noreboot \
 --console=pty,target_type=serial \
 --cpu=host-passthrough \
 --network=network:swandefault,model=virtio \
 --rng=type=random,device=/dev/random \
 --security=type=static,model=dac,label='1000:107',relabel=yes \
--vcpus=1 \
--memory=2048 \
--name=w.openbsd-base \
--os-variant=openbsd7.3 \
--disk=path=/home/libreswan/pool/w.openbsd-base.qcow2,size=10,bus=virtio,format=qcow2
\
--filesystem=target=pool,type=mount,accessmode=squash,source=/home/libreswan/pool
\
--cdrom=/home/libreswan/pool/w.openbsd-base.iso

base.conf which gets added to the iso looks like:

#install.conf file for OpenBSD
Terminal type? = com0
System hostname = openbsd
Which network interface do you wish to configure? = vio0
IPv4 address for = dhcp
DNS Domain name = testing.libreswan.org
Password for root account? =
$2a$12$YZ8bMn19IHPQpBoD6Xf/re/4pp2kbJtVkIl/Mc4G3WA96qyG7/6qW
Start sshd(8) by default = yes
Start ntpd(8) by default? = no
NTP server? (hostname or 'default') = default
Do you expect to run the X Window System? = no
Do you want the X Window System to be started by xdm(1)? = no
Which speed should com0 use? (or 'done') = 19200
What timezone are you in? = EST
Change the default console to com0? = yes
Setup a user? = no
Allow root ssh login = yes
Use (W)hole disk or (E)dit the MBR? = W
URL to autopartitioning template for disklabel? = file:/base.disk
Which disk is the root disk? = sd0
Use DUIDs rather than device names in fstab? = yes
Which disk do you wish to initialize? = done
Set name(s)? = all
Location of sets? = cd0
oPathname to the sets = 7.1/amd64
Directory does not contain SHA256.sig. Continue without verification? = yes

this is the console log, part way through pexpect feeds the VM
commands to start the installer:

cannot open cd0a:/etc/random.seed: No such file or directory
booting cd0a:/7.3/amd64/bsd.rd: 3973828+1655808+3882568+0+708608
[109+444720+297256]=0xa76648
entry point at 0x81001000
Copyright (c) 1982, 1986, 1989, 1991, 1993
The Regents of the University of California.  All rights reserved.
Copyright (c) 1995-2023 OpenBSD. All rights reserved.  https://www.OpenBSD.org

OpenBSD 7.3-current (RAMDISK_CD) #1262: Sat Aug 12 11:54:24 MDT 2023
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/RAMDISK_CD
real mem = 2130542592 (2031MB)
avail mem = 2062016512 (1966MB)
random: good seed from bootblocks
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.8 @ 0xf59a0 (9 entries)
bios0: vendor SeaBIOS version "1.16.2-1.fc38" date 04/01/2014
bios0: QEMU Standard PC (i440FX + PIIX, 1996)
acpi0 at bios0: ACPI 1.0
acpi0: tables DSDT FACP APIC WAET
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: AMD Ryzen 9 3950X 16-Core Processor, 3500.43 MHz, 17-71-00
cpu0: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,SSE3,PCLMUL,SSSE3,FMA3,CX16,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,HV,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,CPCTR,FSGSBASE,TSC_ADJUST,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,SHA,UMIP,STIBP,SSBD,IBPB,STIBP,SSBD,VIRTSSBD,XSAVEOPT,XSAVEC,XGETBV1
cpu0: 64KB 64b/line 2-way D-cache, 64KB 64b/line 2-way I-cache
cpu0: 512KB 64b/line 16-way L2 cache
cpu0: apic clock running at 1000MHz
ioapic0 at mainbus0: apid 0 pa 0xfec0, version 11, 24 pins
acpiprt0 at acpi0: bus 0 (PCI0)
"ACPI0006" at acpi0 not configured
acpipci0 at acpi0 PCI0
"PNP0A06" at acpi0 not configured
"PNP0A06" at acpi0 not configured
"PNP0A06" at acpi0 not configured
"QEMU0002" at acpi0 not configured
com0 at acpi0 COM1 addr 0x3f8/0x8 irq 4: ns16550a, 16 byte fifo
com0: console
acpicmos0 at acpi0
"ACPI0010" at acpi0 not configured
acpicpu at acpi0 not configured
pvbus0 at mainbus0: KVM
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel 82441FX" rev 0x02
"Intel 82371SB ISA" rev 0x00 at pci0 dev 1 function 0 not configured
pciide0 at pci0 dev 1 function 1 "Intel 82371SB IDE" rev 0x00: DMA,
channel 0 wired to compatibility, channel 1 wired to compatibility
atapiscsi0 at pciide0 channel 0 drive 0
scsibus0 at atapiscsi0: 2 targets
cd0 at scsibus0 targ 0 lun 0:  removable
cd0(pciide0:0:0): using PIO mode 4, DMA mode 2
pciide0: channel 1 disabled (no drives)
"Intel 82371AB Power" rev 0x03 at pci0 dev 1 function 3 not configured
virtio0 at pci0 dev 2 function 0 vendor "Qumranet", unknown product
0x1009 rev 0x00
virtio0: no matching child driver; not configured
virtio1 at pci0 dev 3 function 0 "Qumranet Virtio Network" rev 0x00
vio0 at virtio1: address 52:54:00:38:8c:5d
virtio1: msix per-VQ
uhci0 at pci0 dev 4 function 0 "Intel 82801I USB" rev 0x03: apic 0 int 11
uhci1 at pci0 

Re: Virtio fix for testing

2023-08-13 Thread Tobias Heider
On Sat, Aug 12, 2023 at 06:41:17PM -0400, Andrew Cagney wrote:
> On Sat, 12 Aug 2023 at 16:18, Stuart Henderson  wrote:
> 
> > > Is there a way to get an updated ISO or kernel with the fix?
> > > (we're already adding an installer config file to the ISO, so why not a 
> > > kernel)
> > >
> > > Andrew
> > >
> >
> > It was committed to -current, so if you're able to use a snapshot
> > build (https://cdn.openbsd.org/pub/OpenBSD/snapshots) that would
> > likely be the simplest fix.
> 
> Greg, Stuart, thanks for the pointers.  Unfortunately I had no luck:
> 
> OpenBSD 7.3-current (RAMDISK_CD) #1262: Sat Aug 12 11:54:24 MDT 2023
> dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/RAMDISK_CD
> ...
> Installing base73.tgz19% |  | 72320 KB
> 00:25 ETAwdc_atapi_start: not ready, st = 50
> Installing base73.tgz22% |* | 83840 KB
> 00:27 ETAfatal protection fault in supervisor mode
> trap type 4 code 0 rip 81008c29 cs 8 rflags 10286 cr2
> 251d44000 cpl 6 rsp 800021749050
> gsbase 0x81908ff0  kgsbase 0x0
> panic: trap type 4, code=0, pc=81008c29
> syncing disks...
> 
> I guess I've a different problem.  Now I need a cheat sheet on how to
> pull a backtrace :-/
> 

Hi Andrew,

can you share the qemu cmd you are using in your tests?
I'd like to see if I can reproduce this.

Tobias



Re: Virtio fix for testing

2023-08-12 Thread Andrew Cagney
On Sat, 12 Aug 2023 at 16:18, Stuart Henderson  wrote:

> > Is there a way to get an updated ISO or kernel with the fix?
> > (we're already adding an installer config file to the ISO, so why not a 
> > kernel)
> >
> > Andrew
> >
>
> It was committed to -current, so if you're able to use a snapshot
> build (https://cdn.openbsd.org/pub/OpenBSD/snapshots) that would
> likely be the simplest fix.

Greg, Stuart, thanks for the pointers.  Unfortunately I had no luck:

OpenBSD 7.3-current (RAMDISK_CD) #1262: Sat Aug 12 11:54:24 MDT 2023
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/RAMDISK_CD
...
Installing base73.tgz19% |  | 72320 KB
00:25 ETAwdc_atapi_start: not ready, st = 50
Installing base73.tgz22% |* | 83840 KB
00:27 ETAfatal protection fault in supervisor mode
trap type 4 code 0 rip 81008c29 cs 8 rflags 10286 cr2
251d44000 cpl 6 rsp 800021749050
gsbase 0x81908ff0  kgsbase 0x0
panic: trap type 4, code=0, pc=81008c29
syncing disks...

I guess I've a different problem.  Now I need a cheat sheet on how to
pull a backtrace :-/



Re: Virtio fix for testing

2023-08-12 Thread Stuart Henderson
On 2023/08/12 13:16, Andrew Cagney wrote:
> Ref: https://marc.info/?l=openbsd-tech=168458764424059=2
> https://marc.info/?l=openbsd-misc=168071258109433=2
> 
> I'm trying to update libreswan's automated test framework so that it
> creates an OpenBSD 7.3 test VM using install7.3.iso.  Unfortunately
> I've hit what I'm assuming is this bug vis:
> 
> Installing bsd  100% |**| 24400 KB00:01
> Installing bsd.rd   100% |**|  4547 KB00:00
> Installing base73.tgz 3% |  | 12800 KB
> 00:28 ETAwdc_atapi_start: not ready, st = 50
> Installing base73.tgz 4% |* | 15104 KB
> 00:52 ETAfatal protection fault in supervisor mode
> trap type 4 code 0 rip 810089d9 cs 8 rflags 10286 cr2
> 2446c8000 cpl 6 rsp 80002174d4e0
> 
> Is there a way to get an updated ISO or kernel with the fix?
> (we're already adding an installer config file to the ISO, so why not a 
> kernel)
> 
> Andrew
> 

It was committed to -current, so if you're able to use a snapshot
build (https://cdn.openbsd.org/pub/OpenBSD/snapshots) that would
likely be the simplest fix.



Re: Virtio fix for testing

2023-08-12 Thread Andrew Cagney
Ref: https://marc.info/?l=openbsd-tech=168458764424059=2
https://marc.info/?l=openbsd-misc=168071258109433=2

I'm trying to update libreswan's automated test framework so that it
creates an OpenBSD 7.3 test VM using install7.3.iso.  Unfortunately
I've hit what I'm assuming is this bug vis:

Installing bsd  100% |**| 24400 KB00:01
Installing bsd.rd   100% |**|  4547 KB00:00
Installing base73.tgz 3% |  | 12800 KB
00:28 ETAwdc_atapi_start: not ready, st = 50
Installing base73.tgz 4% |* | 15104 KB
00:52 ETAfatal protection fault in supervisor mode
trap type 4 code 0 rip 810089d9 cs 8 rflags 10286 cr2
2446c8000 cpl 6 rsp 80002174d4e0

Is there a way to get an updated ISO or kernel with the fix?
(we're already adding an installer config file to the ISO, so why not a kernel)

Andrew



Re: Virtio fix for testing

2023-05-26 Thread Jan Klemkow
On Wed, May 24, 2023 at 08:50:26PM +0200, Stefan Fritsch wrote:
> I forgot to mention that no stress test is necessary. If it boots and the
> virtio devices work at all, that should be enough.

Works for me on Linux/KVM with the following devices:

vga1 at pci0 dev 2 function 0 "Qumranet Virtio 1.x GPU" rev 0x01
virtio0 at pci0 dev 4 function 0 "Qumranet Virtio Storage" rev 0x00
virtio1 at pci0 dev 6 function 0 "Qumranet Virtio Console" rev 0x00
virtio2 at pci0 dev 7 function 0 "Qumranet Virtio Memory Balloon" rev 0x00
virtio3 at pci0 dev 8 function 0 "Qumranet Virtio Network" rev 0x00

and on OpenBSD/VMM with:

virtio0 at pci0 dev 1 function 0 "Qumranet Virtio RNG" rev 0x00
virtio1 at pci0 dev 2 function 0 "Qumranet Virtio Network" rev 0x00
virtio2 at pci0 dev 3 function 0 "Qumranet Virtio Storage" rev 0x00
virtio3 at pci0 dev 4 function 0 "Qumranet Virtio SCSI" rev 0x00

Thanks,
Jan



Re: Virtio fix for testing

2023-05-25 Thread Klemens Nanni
On Wed, May 24, 2023 at 08:50:26PM +0200, Stefan Fritsch wrote:
> I forgot to mention that no stress test is necessary. If it boots and the
> virtio devices work at all, that should be enough.

amd64 bsd.mp keeps working with disks and network interfaces behind
VirtIO, no dmesg or behaviour change inside OpenBSD VMM and Linux KVM.

Linux KVM dmesg below:

OpenBSD 7.3-current (GENERIC.MP) #3: Thu May 25 15:05:23 +04 2023
k...@atar.my.domain:/sys/arch/amd64/compile/GENERIC.MP
real mem = 1056788480 (1007MB)
avail mem = 1005174784 (958MB)
random: good seed from bootblocks
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.8 @ 0xf5ab0 (10 entries)
bios0: vendor SeaBIOS version "1.14.0-2" date 04/01/2014
bios0: QEMU Standard PC (i440FX + PIIX, 1996)
acpi0 at bios0: ACPI 1.0
acpi0: sleep states S5
acpi0: tables DSDT FACP APIC WAET
acpi0: wakeup devices
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee0: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: AMD EPYC 73F3 16-Core Processor, 3500.37 MHz, 19-01-01
cpu0: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,SSE3,PCLMUL,SSSE3,FMA3,CX16,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,HV,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,CPCTR,FSGSBASE,TSC_ADJUST,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,SHA,UMIP,PKU,WAITPKG,IBRS,IBPB,STIBP,SSBD,IBPB,STIBP,SSBD,VIRTSSBD,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
cpu0: 64KB 64b/line 2-way D-cache, 64KB 64b/line 2-way I-cache
cpu0: 512KB 64b/line 16-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 8 var ranges, 88 fixed ranges
cpu0: apic clock running at 1000MHz
cpu1 at mainbus0: apid 1 (application processor)
cpu1: AMD EPYC 73F3 16-Core Processor, 3500.93 MHz, 19-01-01
cpu1: 
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,SSE3,PCLMUL,SSSE3,FMA3,CX16,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,AVX,F16C,RDRAND,HV,NXE,MMXX,FFXSR,PAGE1GB,RDTSCP,LONG,LAHF,CMPLEG,SVM,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,CPCTR,FSGSBASE,TSC_ADJUST,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,RDSEED,ADX,SMAP,CLFLUSHOPT,CLWB,SHA,UMIP,PKU,IBRS,IBPB,STIBP,SSBD,IBPB,STIBP,SSBD,VIRTSSBD,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
cpu1: 64KB 64b/line 2-way D-cache, 64KB 64b/line 2-way I-cache
cpu1: 512KB 64b/line 16-way L2 cache
cpu1: smt 0, core 0, package 1
ioapic0 at mainbus0: apid 0 pa 0xfec0, version 11, 24 pins
acpiprt0 at acpi0: bus 0 (PCI0)
"ACPI0006" at acpi0 not configured
acpipci0 at acpi0 PCI0
com0 at acpi0 COM1 addr 0x3f8/0x8 irq 4: ns16550a, 16 byte fifo
com0: console
acpicmos0 at acpi0
"PNP0A06" at acpi0 not configured
"PNP0A06" at acpi0 not configured
"PNP0A06" at acpi0 not configured
"QEMU0002" at acpi0 not configured
"ACPI0010" at acpi0 not configured
acpicpu0 at acpi0: C1(@1 halt!)
acpicpu1 at acpi0: C1(@1 halt!)
pvbus0 at mainbus0: KVM
pvclock0 at pvbus0
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel 82441FX" rev 0x02
pcib0 at pci0 dev 1 function 0 "Intel 82371SB ISA" rev 0x00
pciide0 at pci0 dev 1 function 1 "Intel 82371SB IDE" rev 0x00: DMA, channel 0 
wired to compatibility, channel 1 wired to compatibility
pciide0: channel 0 disabled (no drives)
pciide0: channel 1 disabled (no drives)
piixpm0 at pci0 dev 1 function 3 "Intel 82371AB Power" rev 0x03: apic 0 int 9
iic0 at piixpm0
vga1 at pci0 dev 2 function 0 "Red Hat QXL Video" rev 0x05
wsdisplay at vga1 not configured
virtio0 at pci0 dev 3 function 0 "Qumranet Virtio Network" rev 0x00
vio0 at virtio0: address 52:54:00:56:32:db
virtio0: msix shared
uhci0 at pci0 dev 4 function 0 "Intel 82801I USB" rev 0x03: apic 0 int 11
uhci1 at pci0 dev 4 function 1 "Intel 82801I USB" rev 0x03: apic 0 int 10
uhci2 at pci0 dev 4 function 2 "Intel 82801I USB" rev 0x03: apic 0 int 10
ehci0 at pci0 dev 4 function 7 "Intel 82801I USB" rev 0x03: apic 0 int 11
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 configuration 1 interface 0 "Intel EHCI root hub" rev 2.00/1.00 
addr 1
virtio1 at pci0 dev 5 function 0 "Qumranet Virtio Console" rev 0x00
virtio1: no matching child driver; not configured
virtio2 at pci0 dev 6 function 0 "Qumranet Virtio Storage" rev 0x00
vioblk0 at virtio2
scsibus1 at vioblk0: 1 targets
sd0 at scsibus1 targ 0 lun 0: 
sd0: 5120MB, 512 bytes/sector, 10485760 sectors
virtio2: msix per-VQ
virtio3 at pci0 dev 7 function 0 "Qumranet Virtio Storage" rev 0x00
vioblk1 at virtio3
scsibus2 at vioblk1: 1 targets
sd1 at scsibus2 targ 0 lun 0: 
sd1: 5MB, 512 bytes/sector, 11392 sectors, readonly
virtio3: msix per-VQ
virtio4 at pci0 dev 8 function 0 "Qumranet Virtio Memory Balloon" rev 0x00
viomb0 at virtio4
virtio4: apic 0 int 11
virtio5 at pci0 dev 9 function 0 "Qumranet Virtio Network" rev 0x00
vio1 at virtio5: address 52:54:00:93:3a:c1
virtio5: msix shared
isa0 at pcib0
isadma0 at isa0
fdc0 at isa0 port 

Re: Virtio fix for testing

2023-05-24 Thread Stefan Fritsch
I forgot to mention that no stress test is necessary. If it boots and 
the virtio devices work at all, that should be enough.


Cheers,
Stefan

Am 20.05.23 um 15:00 schrieb Stefan Fritsch:

Hi,

with help from Aaron Mason, I have found the problem responsible for the
vioscsi panic on oracle cloud and windows qemu. Basically before using any
virt-queues, the guest must set the DRIVER_OK status bit, or the host may
not process the queues. This affects vioscsi(4) and viogpu(4) in openbsd,
but I have changed the code in all virtio drivers to work the same.

This seems to be a common bug in guests and qemu has workarounds in some
code paths, but not in others. I think that is the reason why I could not
reproduce the issue on my own qemu/linux system.

I have only tested this on amd64 / qemu. I would be interested in test
results on other systems, especially on top of vmd and on arm64 with
viogpu.

Aaron, can you please also test with the updated diff?

Thanks in advance.

Cheers,
Stefan

commit d3209e34d77ab7353973efb2a57303f636fff4f4
Author: Stefan Fritsch 
Date:   Sat May 20 14:18:57 2023 +0200

 virtio: Set DRIVER_OK earlier
 
 The DRIVER_OK must be set before using any virt-queues. To allow virtio

 device drivers to use the virt-queues in their attach functions, set the
 bit there and not in the virtio transport attach function.  Only vioscsi
 and viogpu really need this, but let's only have one standard way to do
 this .
 
 Noticed because of hangs with vioscsi on qemu/windows and in the Oracle

 cloud.  With much debugging help by Aaron Mason .
 
 Also revert vioscsi.c 1.31 "Temporarily workaround double calls into

 vioscsi_req_done()"

diff --git a/sys/dev/fdt/virtio_mmio.c b/sys/dev/fdt/virtio_mmio.c
index 12f5a6cdeb6..682cbd304a1 100644
--- a/sys/dev/fdt/virtio_mmio.c
+++ b/sys/dev/fdt/virtio_mmio.c
@@ -293,7 +293,6 @@ virtio_mmio_attach(struct device *parent, struct device 
*self, void *aux)
goto fail_2;
}
  
-	virtio_set_status(vsc, VIRTIO_CONFIG_DEVICE_STATUS_DRIVER_OK);

return;
  
  fail_2:

diff --git a/sys/dev/pci/virtio_pci.c b/sys/dev/pci/virtio_pci.c
index 236120cf5a6..747101b8382 100644
--- a/sys/dev/pci/virtio_pci.c
+++ b/sys/dev/pci/virtio_pci.c
@@ -651,7 +651,6 @@ virtio_pci_attach(struct device *parent, struct device 
*self, void *aux)
}
printf("%s: %s\n", vsc->sc_dev.dv_xname, intrstr);
  
-	virtio_set_status(vsc, VIRTIO_CONFIG_DEVICE_STATUS_DRIVER_OK);

return;
  
  fail_2:

diff --git a/sys/dev/pv/if_vio.c b/sys/dev/pv/if_vio.c
index be53ccaf7c3..5d1ca8d0030 100644
--- a/sys/dev/pv/if_vio.c
+++ b/sys/dev/pv/if_vio.c
@@ -600,6 +600,7 @@ vio_attach(struct device *parent, struct device *self, void 
*aux)
timeout_set(>sc_txtick, vio_txtick, >sc_vq[VQTX]);
timeout_set(>sc_rxtick, vio_rxtick, >sc_vq[VQRX]);
  
+	virtio_set_status(vsc, VIRTIO_CONFIG_DEVICE_STATUS_DRIVER_OK);

if_attach(ifp);
ether_ifattach(ifp);
  
diff --git a/sys/dev/pv/vioblk.c b/sys/dev/pv/vioblk.c

index 093ce9b5a58..9f8a10e0fa2 100644
--- a/sys/dev/pv/vioblk.c
+++ b/sys/dev/pv/vioblk.c
@@ -251,6 +251,7 @@ vioblk_attach(struct device *parent, struct device *self, 
void *aux)
saa.saa_quirks = 0;
saa.saa_wwpn = saa.saa_wwnn = 0;
  
+	virtio_set_status(vsc, VIRTIO_CONFIG_DEVICE_STATUS_DRIVER_OK);

config_found(self, , scsiprint);
  
  	return;

diff --git a/sys/dev/pv/viocon.c b/sys/dev/pv/viocon.c
index bd96467c810..882f0961ada 100644
--- a/sys/dev/pv/viocon.c
+++ b/sys/dev/pv/viocon.c
@@ -203,6 +203,7 @@ viocon_attach(struct device *parent, struct device *self, 
void *aux)
goto err;
}
viocon_rx_fill(sc->sc_ports[0]);
+   virtio_set_status(vsc, VIRTIO_CONFIG_DEVICE_STATUS_DRIVER_OK);
  
  	return;

  err:
diff --git a/sys/dev/pv/viogpu.c b/sys/dev/pv/viogpu.c
index e8a91c34310..06f4388a7b0 100644
--- a/sys/dev/pv/viogpu.c
+++ b/sys/dev/pv/viogpu.c
@@ -235,6 +235,8 @@ viogpu_attach(struct device *parent, struct device *self, 
void *aux)
sc->sc_fb_dma_kva, sc->sc_fb_dma_size, NULL, BUS_DMA_NOWAIT) != 0)
goto fb_unmap;
  
+	virtio_set_status(vsc, VIRTIO_CONFIG_DEVICE_STATUS_DRIVER_OK);

+
if (viogpu_create_2d(sc, 1, sc->sc_fb_width, sc->sc_fb_height) != 0)
goto fb_unmap;
  
diff --git a/sys/dev/pv/viomb.c b/sys/dev/pv/viomb.c

index 8169770553e..a8d408e7392 100644
--- a/sys/dev/pv/viomb.c
+++ b/sys/dev/pv/viomb.c
@@ -220,6 +220,7 @@ viomb_attach(struct device *parent, struct device *self, 
void *aux)
sensordev_install(>sc_sensdev);
  
  	printf("\n");

+   virtio_set_status(vsc, VIRTIO_CONFIG_DEVICE_STATUS_DRIVER_OK);
return;
  err_dmamap:
bus_dmamap_destroy(vsc->sc_dmat, sc->sc_req.bl_dmamap);
diff --git a/sys/dev/pv/viornd.c b/sys/dev/pv/viornd.c
index 39442bc8391..490e823173b 100644
--- a/sys/dev/pv/viornd.c
+++ b/sys/dev/pv/viornd.c
@@ -138,6 

Virtio fix for testing

2023-05-20 Thread Stefan Fritsch
Hi,

with help from Aaron Mason, I have found the problem responsible for the 
vioscsi panic on oracle cloud and windows qemu. Basically before using any 
virt-queues, the guest must set the DRIVER_OK status bit, or the host may 
not process the queues. This affects vioscsi(4) and viogpu(4) in openbsd, 
but I have changed the code in all virtio drivers to work the same.

This seems to be a common bug in guests and qemu has workarounds in some 
code paths, but not in others. I think that is the reason why I could not 
reproduce the issue on my own qemu/linux system.

I have only tested this on amd64 / qemu. I would be interested in test 
results on other systems, especially on top of vmd and on arm64 with 
viogpu.

Aaron, can you please also test with the updated diff?

Thanks in advance.

Cheers,
Stefan

commit d3209e34d77ab7353973efb2a57303f636fff4f4
Author: Stefan Fritsch 
Date:   Sat May 20 14:18:57 2023 +0200

virtio: Set DRIVER_OK earlier

The DRIVER_OK must be set before using any virt-queues. To allow virtio
device drivers to use the virt-queues in their attach functions, set the
bit there and not in the virtio transport attach function.  Only vioscsi
and viogpu really need this, but let's only have one standard way to do
this .

Noticed because of hangs with vioscsi on qemu/windows and in the Oracle
cloud.  With much debugging help by Aaron Mason .

Also revert vioscsi.c 1.31 "Temporarily workaround double calls into
vioscsi_req_done()"

diff --git a/sys/dev/fdt/virtio_mmio.c b/sys/dev/fdt/virtio_mmio.c
index 12f5a6cdeb6..682cbd304a1 100644
--- a/sys/dev/fdt/virtio_mmio.c
+++ b/sys/dev/fdt/virtio_mmio.c
@@ -293,7 +293,6 @@ virtio_mmio_attach(struct device *parent, struct device 
*self, void *aux)
goto fail_2;
}
 
-   virtio_set_status(vsc, VIRTIO_CONFIG_DEVICE_STATUS_DRIVER_OK);
return;
 
 fail_2:
diff --git a/sys/dev/pci/virtio_pci.c b/sys/dev/pci/virtio_pci.c
index 236120cf5a6..747101b8382 100644
--- a/sys/dev/pci/virtio_pci.c
+++ b/sys/dev/pci/virtio_pci.c
@@ -651,7 +651,6 @@ virtio_pci_attach(struct device *parent, struct device 
*self, void *aux)
}
printf("%s: %s\n", vsc->sc_dev.dv_xname, intrstr);
 
-   virtio_set_status(vsc, VIRTIO_CONFIG_DEVICE_STATUS_DRIVER_OK);
return;
 
 fail_2:
diff --git a/sys/dev/pv/if_vio.c b/sys/dev/pv/if_vio.c
index be53ccaf7c3..5d1ca8d0030 100644
--- a/sys/dev/pv/if_vio.c
+++ b/sys/dev/pv/if_vio.c
@@ -600,6 +600,7 @@ vio_attach(struct device *parent, struct device *self, void 
*aux)
timeout_set(>sc_txtick, vio_txtick, >sc_vq[VQTX]);
timeout_set(>sc_rxtick, vio_rxtick, >sc_vq[VQRX]);
 
+   virtio_set_status(vsc, VIRTIO_CONFIG_DEVICE_STATUS_DRIVER_OK);
if_attach(ifp);
ether_ifattach(ifp);
 
diff --git a/sys/dev/pv/vioblk.c b/sys/dev/pv/vioblk.c
index 093ce9b5a58..9f8a10e0fa2 100644
--- a/sys/dev/pv/vioblk.c
+++ b/sys/dev/pv/vioblk.c
@@ -251,6 +251,7 @@ vioblk_attach(struct device *parent, struct device *self, 
void *aux)
saa.saa_quirks = 0;
saa.saa_wwpn = saa.saa_wwnn = 0;
 
+   virtio_set_status(vsc, VIRTIO_CONFIG_DEVICE_STATUS_DRIVER_OK);
config_found(self, , scsiprint);
 
return;
diff --git a/sys/dev/pv/viocon.c b/sys/dev/pv/viocon.c
index bd96467c810..882f0961ada 100644
--- a/sys/dev/pv/viocon.c
+++ b/sys/dev/pv/viocon.c
@@ -203,6 +203,7 @@ viocon_attach(struct device *parent, struct device *self, 
void *aux)
goto err;
}
viocon_rx_fill(sc->sc_ports[0]);
+   virtio_set_status(vsc, VIRTIO_CONFIG_DEVICE_STATUS_DRIVER_OK);
 
return;
 err:
diff --git a/sys/dev/pv/viogpu.c b/sys/dev/pv/viogpu.c
index e8a91c34310..06f4388a7b0 100644
--- a/sys/dev/pv/viogpu.c
+++ b/sys/dev/pv/viogpu.c
@@ -235,6 +235,8 @@ viogpu_attach(struct device *parent, struct device *self, 
void *aux)
sc->sc_fb_dma_kva, sc->sc_fb_dma_size, NULL, BUS_DMA_NOWAIT) != 0)
goto fb_unmap;
 
+   virtio_set_status(vsc, VIRTIO_CONFIG_DEVICE_STATUS_DRIVER_OK);
+
if (viogpu_create_2d(sc, 1, sc->sc_fb_width, sc->sc_fb_height) != 0)
goto fb_unmap;
 
diff --git a/sys/dev/pv/viomb.c b/sys/dev/pv/viomb.c
index 8169770553e..a8d408e7392 100644
--- a/sys/dev/pv/viomb.c
+++ b/sys/dev/pv/viomb.c
@@ -220,6 +220,7 @@ viomb_attach(struct device *parent, struct device *self, 
void *aux)
sensordev_install(>sc_sensdev);
 
printf("\n");
+   virtio_set_status(vsc, VIRTIO_CONFIG_DEVICE_STATUS_DRIVER_OK);
return;
 err_dmamap:
bus_dmamap_destroy(vsc->sc_dmat, sc->sc_req.bl_dmamap);
diff --git a/sys/dev/pv/viornd.c b/sys/dev/pv/viornd.c
index 39442bc8391..490e823173b 100644
--- a/sys/dev/pv/viornd.c
+++ b/sys/dev/pv/viornd.c
@@ -138,6 +138,7 @@ viornd_attach(struct device *parent, struct device *self, 
void *aux)
timeout_add(>sc_tick, 1);
 
printf("\n");
+   virtio_set_status(vsc,