On Tue, Apr 25, 2023, at 9:47 AM, Dave Voutila wrote:
> tech@:
>
> The below diff splits out virtio device emulation for virtio block and
> network devices into separate fork+exec'd & pledge(2)'d subprocesses.
>
> In order of priority, this diff:
>
> 1. Isolates common exploit targets (e.g. emulated network devices) from
> the rest of the vm process, tightening pledge to "stdio" per device.
>
> 2. Increases responsiveness of guest i/o since we no longer have a
> single thread servicing virtio pci and device emulation.
>
> I'd like to land this diff this week, so if you use atypical vmd
> configurations like:
>
> 1. multiple vioblk disks per vm
> 2. multiple nics per vm
> 3. send/receive
> 4. qcow2 base images
>
> This diff has lots of info logging enabled by default to help me
> identify what's breaking, so please reply with log message output if
> something goes sideways.
>
> -dv
Apologies for being late to the party, I'm testing this now via snapshots. The
host in question was last updated a few weeks ago, i.e. before the deluge of
recent work. It's an amd64 VMM host running (among other things) an i386 7.3
release VM. After updating the host to the latest snapshot, things appear to
run fine, but I did notice the following new messages consistently when
powering off (halt -p) the i386 VM:
Apr 29 18:41:09 coofun /bsd: uvn_flush: obj=0xfffffd815a44fee0,
offset=0x43287000. error during pageout.
Apr 29 18:41:09 coofun /bsd: uvn_flush: WARNING: changes to page may be lost!
Apr 30 17:28:26 coofun /bsd: uvn_flush: obj=0xfffffd812ff90c08,
offset=0x4378c000. error during pageout.
Apr 30 17:28:26 coofun /bsd: uvn_flush: WARNING: changes to page may be lost!
The guest *appears* to run fine and reboots with a clean filesystem, but I also
haven't gone through the filesystem with a comb. I have not had a chance to
spin up an amd64 guest this weekend yet.
Thanks!
Brian Conway
amd64 host dmesg:
OpenBSD 7.3-current (GENERIC.MP) #1168: Fri Apr 28 14:47:06 MDT 2023
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 4072996864 (3884MB)
avail mem = 3929907200 (3747MB)
random: good seed from bootblocks
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 3.2 @ 0x79851000 (17 entries)
bios0: vendor American Megatrends Inc. version "GB1B 0.04" date 10/20/2021
bios0: BESSTAR TECH LIMITED N40
efi0 at bios0: UEFI 2.7
efi0: American Megatrends rev 0x5000d
acpi0 at bios0: ACPI 6.2
acpi0: sleep states S0 S3 S4 S5
acpi0: tables DSDT FACP FPDT FIDT MSDM MCFG SSDT DBG2 DBGP HPET LPIT APIC NPKT
SSDT SSDT SSDT SSDT SSDT SSDT BGRT TPM2 DMAR WDAT NHLT WSMT
acpi0: wakeup devices HDAS(S3) XHC_(S4) XDCI(S4) RP01(S4) PXSX(S4) RP02(S4)
PXSX(S4) RP03(S4) PXSX(S4) RP04(S4) PXSX(S4) RP05(S4) PXSX(S4) RP06(S4) PXSX(S4)
acpitimer0 at acpi0: 3579545 Hz, 32 bits
acpimcfg0 at acpi0
acpimcfg0: addr 0xe0000000, bus 0-255
acpihpet0 at acpi0: 19200000 Hz
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Celeron(R) N4020 CPU @ 1.10GHz, 1095.50 MHz, 06-7a-08
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,CX16,xTPR,PDCM,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,SGX,SMEP,ERMS,MPX,RDSEED,SMAP,CLFLUSHOPT,PT,SHA,UMIP,MD_CLEAR,IBRS,IBPB,STIBP,SSBD,SENSOR,ARAT,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
cpu0: 24KB 64b/line 6-way D-cache, 32KB 64b/line 8-way I-cache, 4MB 64b/line
16-way L2 cache
cpu0: smt 0, core 0, package 0
mtrr: Pentium Pro MTRR support, 10 var ranges, 88 fixed ranges
cpu0: apic clock running at 19MHz
cpu0: mwait min=64, max=64, C-substates=0.2.0.2.4.2.1.1, IBE
cpu1 at mainbus0: apid 2 (application processor)
cpu1: Intel(R) Celeron(R) N4020 CPU @ 1.10GHz, 1096.97 MHz, 06-7a-08
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE,SSE3,PCLMUL,DTES64,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,SDBG,CX16,xTPR,PDCM,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,DEADLINE,AES,XSAVE,RDRAND,NXE,PAGE1GB,RDTSCP,LONG,LAHF,3DNOWP,PERF,ITSC,FSGSBASE,TSC_ADJUST,SGX,SMEP,ERMS,MPX,RDSEED,SMAP,CLFLUSHOPT,PT,SHA,UMIP,MD_CLEAR,IBRS,IBPB,STIBP,SSBD,SENSOR,ARAT,XSAVEOPT,XSAVEC,XGETBV1,XSAVES
cpu1: 24KB 64b/line 6-way D-cache, 32KB 64b/line 8-way I-cache, 4MB 64b/line
16-way L2 cache
cpu1: smt 0, core 1, package 0
ioapic0 at mainbus0: apid 1 pa 0xfec00000, version 20, 120 pins
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 4 (RP01)
acpiprt2 at acpi0: bus 5 (RP02)
acpiprt3 at acpi0: bus 1 (RP03)
acpiprt4 at acpi0: bus 2 (RP04)
acpiprt5 at acpi0: bus 3 (RP05)
acpiprt6 at acpi0: bus -1 (RP06)
acpipci0 at acpi0 PCI0: 0x00000000 0x00000011 0x00000001
"ESSX8336" at acpi0 not configured
acpicmos0 at acpi0
glkgpio0 at acpi0 GPO1 uid 1 addr 0xd0c40000/0xcef irq 14, 80 pins
glkgpio1 at acpi0 GPO0 uid 2 addr 0xd0c50000/0xaff irq 14, 80 pins
glkgpio2 at acpi0 GPO2 uid 3 addr 0xd0c90000/0x7bf irq 15, 20 pins
glkgpio3 at acpi0 GPO3 uid 4 addr 0xd0c80000/0x82f irq 14, 35 pins
acpibtn0 at acpi0: PWRB
"INT33A1" at acpi0 not configured
tpm0 at acpi0 TPM_ 2.0 (CRB) addr 0xfed40000/0x5000, device 0x00000000 rev 0x0
"PNP0C0B" at acpi0 not configured
acpipwrres0 at acpi0: DRST
acpipwrres1 at acpi0: DRST
acpipwrres2 at acpi0: DRST
acpipwrres3 at acpi0: DRST
acpipwrres4 at acpi0: DRST
acpipwrres5 at acpi0: DRST
acpipwrres6 at acpi0: WRST
acpicpu0 at acpi0: C3(10@150 mwait.1@0x60), C2(10@50 mwait.1@0x21), C1(1000@1
mwait.1@0x1), PSS
acpicpu1 at acpi0: C3(10@150 mwait.1@0x60), C2(10@50 mwait.1@0x21), C1(1000@1
mwait.1@0x1), PSS
acpipwrres7 at acpi0: FN00, resource for FAN0
acpitz0 at acpi0: critical temperature is 95 degC
acpivideo0 at acpi0: GFX0
acpivout0 at acpivideo0: DD1F
cpu0: Enhanced SpeedStep 1095 MHz: speeds: 1101, 1100, 1000, 900, 800 MHz
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel Gemini Lake Host" rev 0x06
"Intel Gemini Lake DPTF" rev 0x06 at pci0 dev 0 function 1 not configured
inteldrm0 at pci0 dev 2 function 0 "Intel UHD Graphics 600" rev 0x06
drm0 at inteldrm0
inteldrm0: msi, GEMINILAKE, gen 9
azalia0 at pci0 dev 14 function 0 "Intel Gemini Lake HD Audio" rev 0x06: msi
azalia0: no supported codecs
"Intel Gemini Lake MEI" rev 0x06 at pci0 dev 15 function 0 not configured
ahci0 at pci0 dev 18 function 0 "Intel Gemini Lake AHCI" rev 0x06: msi, AHCI
1.3.1
ahci0: PHY offline on port 0
ahci0: port 1: 6.0Gb/s
scsibus1 at ahci0: 32 targets
sd0 at scsibus1 targ 1 lun 0: <ATA, NT-256 2242, SN11> naa.50000000000027b6
sd0: 244198MB, 512 bytes/sector, 500118192 sectors, thin
ppb0 at pci0 dev 19 function 0 "Intel Gemini Lake PCIE" rev 0xf6: msi
pci1 at ppb0 bus 1
iwm0 at pci1 dev 0 function 0 "Intel AC 3165" rev 0x79, msi
ppb1 at pci0 dev 19 function 1 "Intel Gemini Lake PCIE" rev 0xf6: msi
pci2 at ppb1 bus 2
re0 at pci2 dev 0 function 0 "Realtek 8168" rev 0x15: RTL8168H/8111H (0x5400),
msi, address 84:47:09:09:b7:8e
rgephy0 at re0 phy 7: RTL8251 PHY, rev. 0
ppb2 at pci0 dev 19 function 2 "Intel Gemini Lake PCIE" rev 0xf6: msi
pci3 at ppb2 bus 3
ppb3 at pci0 dev 20 function 0 "Intel Gemini Lake PCIE" rev 0xf6
pci4 at ppb3 bus 4
ppb4 at pci0 dev 20 function 1 "Intel Gemini Lake PCIE" rev 0xf6
pci5 at ppb4 bus 5
xhci0 at pci0 dev 21 function 0 "Intel Gemini Lake xHCI" rev 0x06: msi, xHCI 1.0
usb0 at xhci0: USB revision 3.0
uhub0 at usb0 configuration 1 interface 0 "Intel xHCI root hub" rev 3.00/1.00
addr 1
dwiic0 at pci0 dev 22 function 0 "Intel Gemini Lake I2C" rev 0x06: apic 1 int 27
iic0 at dwiic0
"ESSX8336" at iic0 addr 0x11 not configured
dwiic1 at pci0 dev 22 function 3 "Intel Gemini Lake I2C" rev 0x06: apic 1 int 30
iic1 at dwiic1
sdhc0 at pci0 dev 28 function 0 "Intel Gemini Lake eMMC" rev 0x06: apic 1 int 39
sdhc0: SDHC 3.0, 200 MHz base clock
sdmmc0 at sdhc0: 8-bit, sd high-speed, mmc high-speed, ddr52, dma
pcib0 at pci0 dev 31 function 0 "Intel Gemini Lake LPC" rev 0x06
ichiic0 at pci0 dev 31 function 1 "Intel Gemini Lake SMBus" rev 0x06: apic 1
int 20
iic2 at ichiic0
isa0 at pcib0
isadma0 at isa0
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
vmm0 at mainbus0: VMX/EPT
efifb at mainbus0 not configured
scsibus2 at sdmmc0: 2 targets, initiator 0
sd1 at scsibus2 targ 1 lun 0: <SD/MMC, SCA64G, 0000> removable
sd1: 59648MB, 512 bytes/sector, 122159104 sectors
uhidev0 at uhub0 port 2 configuration 1 interface 0 "Logitech USB Keyboard" rev
1.10/64.00 addr 2
uhidev0: iclass 3/1
ukbd0 at uhidev0: 8 variable keys, 6 key codes
wskbd0 at ukbd0: console keyboard
uhidev1 at uhub0 port 2 configuration 1 interface 1 "Logitech USB Keyboard" rev
1.10/64.00 addr 2
uhidev1: iclass 3/0, 3 report ids
ucc0 at uhidev1 reportid 1: 2 usages, 3 keys, enum
wskbd1 at ucc0 mux 1
uhid0 at uhidev1 reportid 2: input=1, output=0, feature=0
ucc1 at uhidev1 reportid 3: 21 usages, 14 keys, enum
wskbd2 at ucc1 mux 1
ugen0 at uhub0 port 6 "Generic USB2.0-CRW" rev 2.00/39.60 addr 3
ugen1 at uhub0 port 7 "Intel Bluetooth" rev 2.00/0.01 addr 4
vscsi0 at root
scsibus3 at vscsi0: 256 targets
softraid0 at root
scsibus4 at softraid0: 256 targets
root on sd0a (b6b9cb10be699de6.a) swap on sd0b dump on sd0b
drm:pid0:intel_step_init *WARNING* [drm] Unknown revid 0x06
drm:pid0:intel_dp_aux_wait_done *ERROR* [drm] *ERROR* AUX C/DDI C/PHY C: did
not complete or timeout within 10ms (status 0xad4002ff)
inteldrm0: 1920x1080, 32bpp
wsdisplay0 at inteldrm0 mux 1: console (std, vt100 emulation), using wskbd0
wskbd1: connecting to wsdisplay0
wskbd2: connecting to wsdisplay0
wsdisplay0: screen 1-5 added (std, vt100 emulation)
iwm0: hw rev 0x210, fw ver 29.4063824552.0, address a4:42:3b:34:d2:80
uid 0 on /tmp: file system full
uvn_flush: obj=0xfffffd812ff90c08, offset=0x4378c000. error during pageout.
uvn_flush: WARNING: changes to page may be lost!
i386 guest dmesg:
OpenBSD 7.3 (GENERIC) #1: Thu Mar 30 11:37:43 UTC 2023
bcon...@b73-i386.int.rcesoftware.com:/usr/src/sys/arch/i386/compile/GENERIC
real mem = 1341657088 (1279MB)
avail mem = 1300058112 (1239MB)
random: good seed from bootblocks
mpath0 at root
scsibus0 at mpath0: 256 targets
mainbus0 at root
bios0 at mainbus0: date 06/23/99, BIOS32 rev. 0 @ 0xfefc2, SMBIOS rev. 2.4 @
0xf3740 (10 entries)
bios0: vendor SeaBIOS version "1.14.0p3-OpenBSD-vmm" date 01/01/2011
bios0: OpenBSD VMM
acpi at bios0 function 0x0 not configured
pcibios at bios0 function 0x1a not configured
bios0: ROM list: 0xef000/0x1000!
cpu0 at mainbus0: (uniprocessor)
cpu0: Intel(R) Celeron(R) N4020 CPU @ 1.10GHz ("GenuineIntel" 686-class) 24
MHz, 06-7a-08
cpu0:
FPU,V86,DE,PSE,TSC,MSR,PAE,CX8,SEP,PGE,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,SSE3,PCLMUL,SSSE3,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,RDRAND,HV,NXE,PAGE1GB,LONG,LAHF,3DNOWP,ITSC,FSGSBASE,SMEP,ERMS,RDSEED,SMAP,CLFLUSHOPT,SHA,UMIP,MD_CLEAR,MELTDOWN
pvbus0 at mainbus0: OpenBSD
pvclock0 at pvbus0
pci0 at mainbus0 bus 0: configuration mode 1 (no bios)
pchb0 at pci0 dev 0 function 0 "OpenBSD VMM Host" rev 0x00
virtio0 at pci0 dev 1 function 0 "Qumranet Virtio RNG" rev 0x00
viornd0 at virtio0
virtio0: irq 3
virtio1 at pci0 dev 2 function 0 "Qumranet Virtio Network" rev 0x00
vio0 at virtio1: address fe:e1:bb:d1:21:46
virtio1: irq 5
virtio2 at pci0 dev 3 function 0 "Qumranet Virtio Storage" rev 0x00
vioblk0 at virtio2
scsibus1 at vioblk0: 1 targets
sd0 at scsibus1 targ 0 lun 0: <VirtIO, Block Device, >
sd0: 20480MB, 512 bytes/sector, 41943040 sectors
virtio2: irq 6
virtio3 at pci0 dev 4 function 0 "OpenBSD VMM Control" rev 0x00
vmmci0 at virtio3
virtio3: irq 7
isa0 at mainbus0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns8250, no fifo
com0: console
pcdisplay0 at isa0 port 0x3d0/16 iomem 0xb8000/32768
wsdisplay0 at pcdisplay0 mux 1: console (80x25, vt100 emulation)
npx0 at isa0 port 0xf0/16: reported by CPUID; using exception 16
vscsi0 at root
scsibus2 at vscsi0: 256 targets
softraid0 at root
scsibus3 at softraid0: 256 targets
root on sd0a (7b376d3efe69a188.a) swap on sd0b dump on sd0b
