I get a rather curious error when shutting down a machine with a RAID 1
setup that contains a crypto partition and a "normal" partition:
syncing disks... done
sd3 detached
softraid0: I/O error 5 on dev 0x433 at block 16
softraid0: could not write metadata to sd3d
sd4 detached
rebooting...
When the machine starts up again, I see:
softraid0: sd4 was not shutdown properly (twice)
The funny thing is that everything works just fine. No problems whatsoever.
That said, the error message is more than a little unsettling and I'm
worried that one day I'm going to access files on the crypto volume only
to find the data isn't available.
Any hints on what I did wrong when setting this up?
Details:
The RAID volume, sd3, is made up from two 3TB as sd0 and sd1:
# disklabel -pg sd0
# /dev/rsd0c:
type: SCSI
disk: SCSI disk
label: ST3000DM001-1CH1
duid: ec7586498efe98b8
flags:
bytes/sector: 512
sectors/track: 63
tracks/cylinder: 255
sectors/cylinder: 16065
cylinders: 364801
total sectors: 5860533168 # total bytes: 2794.5G
boundstart: 64
boundend: 4294961685
drivedata: 0
16 partitions:
# size offset fstype [fsize bsize cpg]
a: 2794.5G 64 RAID
c: 2794.5G 0 unused
# disklabel -pg sd1
# /dev/rsd1c:
type: SCSI
disk: SCSI disk
label: ST3000DM001-1CH1
duid: 0372b77b79b2e168
flags:
bytes/sector: 512
sectors/track: 63
tracks/cylinder: 255
sectors/cylinder: 16065
cylinders: 364801
total sectors: 5860533168 # total bytes: 2794.5G
boundstart: 64
boundend: 4294961685
drivedata: 0
16 partitions:
# size offset fstype [fsize bsize cpg]
a: 2794.5G 64 RAID
c: 2794.5G 0 unused
Those two drives then become sd3: (sd2 is an SSD boot drive)
# disklabel -pg sd3
# /dev/rsd3c:
type: SCSI
disk: SCSI disk
label: SR RAID 1
duid: 6ca0888211d57886
flags:
bytes/sector: 512
sectors/track: 255
tracks/cylinder: 511
sectors/cylinder: 130305
cylinders: 44975
total sectors: 5860532576 # total bytes: 2794.5G
boundstart: 0
boundend: 5860532576
drivedata: 0
16 partitions:
# size offset fstype [fsize bsize cpg]
a: 1397.3G 0 4.2BSD 8192 65536 1 # /st2
c: 2794.5G 0 unused
d: 1397.3G 2930266240 RAID
from which a crypto volume was created, becoming sd4:
# disklabel -pg sd4
# /dev/rsd4c:
type: SCSI
disk: SCSI disk
label: SR CRYPTO
duid: 6c6e53ab843ef6c8
flags:
bytes/sector: 512
sectors/track: 63
tracks/cylinder: 255
sectors/cylinder: 16065
cylinders: 182400
total sectors: 2930265808 # total bytes: 1397.3G
boundstart: 0
boundend: 2930265808
drivedata: 0
16 partitions:
# size offset fstype [fsize bsize cpg]
a: 1397.3G 0 4.2BSD 8192 65536 1
c: 1397.3G 0 unused
I'm running a snapshot from Jan 21. Full dmesg:
OpenBSD 5.2-current (GENERIC.MP) #20: Mon Jan 21 17:23:23 MST 2013
t...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
real mem = 12613910528 (12029MB)
avail mem = 12255641600 (11687MB)
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.5 @ 0x9f400 (68 entries)
bios0: vendor American Megatrends Inc. version "2105" date 07/23/2010
bios0: ASUSTeK Computer INC. M4A785TD-V EVO
acpi0 at bios0: rev 2
acpi0: sleep states S0 S1 S3 S4 S5
acpi0: tables DSDT FACP APIC MCFG OEMB SRAT HPET SSDT
acpi0: wakeup devices PCE2(S4) PCE3(S4) PCE4(S4) PCE5(S4) PCE6(S4)
PCE7(S4) PCE9(S4) PCEA(S4) PCEB(S4) PCEC(S4) SBAZ(S4) PS2M(S4) PS2K(S4)
UAR1(S4) P0PC(S4) UHC1(S4) UHC2(S4) UHC3(S4) USB4(S4) UHC5(S4) UHC6(S4)
UHC7(S4)
acpitimer0 at acpi0: 3579545 Hz, 32 bits
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: AMD Phenom(tm) II X6 1100T Processor, 3300.74 MHz
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,MWAIT,CX16,POPCNT,NXE,MMXX,FFXSR,LONG,3DNOW2,3DNOW,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,IBS,SKINIT,ITSC
cpu0: 64KB 64b/line 2-way I-cache, 64KB 64b/line 2-way D-cache, 512KB
64b/line 16-way L2 cache, 6MB 64b/line 48-way L3 cache
cpu0: ITLB 32 4KB entries fully associative, 16 4MB entries fully
associative
cpu0: DTLB 48 4KB entries fully associative, 48 4MB entries fully
associative
cpu0: AMD erratum 721 detected and fixed
cpu0: apic clock running at 200MHz
cpu1 at mainbus0: apid 1 (application processor)
cpu1: AMD Phenom(tm) II X6 1100T Processor, 3300.25 MHz
cpu1:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,MWAIT,CX16,POPCNT,NXE,MMXX,FFXSR,LONG,3DNOW2,3DNOW,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,IBS,SKINIT,ITSC
cpu1: 64KB 64b/line 2-way I-cache, 64KB 64b/line 2-way D-cache, 512KB
64b/line 16-way L2 cache, 6MB 64b/line 48-way L3 cache
cpu1: ITLB 32 4KB entries fully associative, 16 4MB entries fully
associative
cpu1: DTLB 48 4KB entries fully associative, 48 4MB entries fully
associative
cpu1: AMD erratum 721 detected and fixed
cpu2 at mainbus0: apid 2 (application processor)
cpu2: AMD Phenom(tm) II X6 1100T Processor, 3300.25 MHz
cpu2:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,MWAIT,CX16,POPCNT,NXE,MMXX,FFXSR,LONG,3DNOW2,3DNOW,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,IBS,SKINIT,ITSC
cpu2: 64KB 64b/line 2-way I-cache, 64KB 64b/line 2-way D-cache, 512KB
64b/line 16-way L2 cache, 6MB 64b/line 48-way L3 cache
cpu2: ITLB 32 4KB entries fully associative, 16 4MB entries fully
associative
cpu2: DTLB 48 4KB entries fully associative, 48 4MB entries fully
associative
cpu2: AMD erratum 721 detected and fixed
cpu3 at mainbus0: apid 3 (application processor)
cpu3: AMD Phenom(tm) II X6 1100T Processor, 3300.25 MHz
cpu3:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,MWAIT,CX16,POPCNT,NXE,MMXX,FFXSR,LONG,3DNOW2,3DNOW,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,IBS,SKINIT,ITSC
cpu3: 64KB 64b/line 2-way I-cache, 64KB 64b/line 2-way D-cache, 512KB
64b/line 16-way L2 cache, 6MB 64b/line 48-way L3 cache
cpu3: ITLB 32 4KB entries fully associative, 16 4MB entries fully
associative
cpu3: DTLB 48 4KB entries fully associative, 48 4MB entries fully
associative
cpu3: AMD erratum 721 detected and fixed
cpu4 at mainbus0: apid 4 (application processor)
cpu4: AMD Phenom(tm) II X6 1100T Processor, 3300.25 MHz
cpu4:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,MWAIT,CX16,POPCNT,NXE,MMXX,FFXSR,LONG,3DNOW2,3DNOW,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,IBS,SKINIT,ITSC
cpu4: 64KB 64b/line 2-way I-cache, 64KB 64b/line 2-way D-cache, 512KB
64b/line 16-way L2 cache, 6MB 64b/line 48-way L3 cache
cpu4: ITLB 32 4KB entries fully associative, 16 4MB entries fully
associative
cpu4: DTLB 48 4KB entries fully associative, 48 4MB entries fully
associative
cpu4: AMD erratum 721 detected and fixed
cpu5 at mainbus0: apid 5 (application processor)
cpu5: AMD Phenom(tm) II X6 1100T Processor, 3300.25 MHz
cpu5:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,HTT,SSE3,MWAIT,CX16,POPCNT,NXE,MMXX,FFXSR,LONG,3DNOW2,3DNOW,LAHF,CMPLEG,SVM,EAPICSP,AMCR8,ABM,SSE4A,MASSE,3DNOWP,OSVW,IBS,SKINIT,ITSC
cpu5: 64KB 64b/line 2-way I-cache, 64KB 64b/line 2-way D-cache, 512KB
64b/line 16-way L2 cache, 6MB 64b/line 48-way L3 cache
cpu5: ITLB 32 4KB entries fully associative, 16 4MB entries fully
associative
cpu5: DTLB 48 4KB entries fully associative, 48 4MB entries fully
associative
cpu5: AMD erratum 721 detected and fixed
ioapic0 at mainbus0: apid 6 pa 0xfec00000, version 21, 24 pins
acpimcfg0 at acpi0 addr 0xe0000000, bus 0-255
acpihpet0 at acpi0: 14318180 Hz
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 1 (P0P1)
acpiprt2 at acpi0: bus -1 (PCE2)
acpiprt3 at acpi0: bus -1 (PCE3)
acpiprt4 at acpi0: bus -1 (PCE4)
acpiprt5 at acpi0: bus -1 (PCE5)
acpiprt6 at acpi0: bus -1 (PCE6)
acpiprt7 at acpi0: bus -1 (PCE7)
acpiprt8 at acpi0: bus -1 (PCE9)
acpiprt9 at acpi0: bus 2 (PCEA)
acpiprt10 at acpi0: bus 3 (P0PC)
acpicpu0 at acpi0: C2, PSS
acpicpu1 at acpi0: C2, PSS
acpicpu2 at acpi0: C2, PSS
acpicpu3 at acpi0: C2, PSS
acpicpu4 at acpi0: C2, PSS
acpicpu5 at acpi0: C2, PSS
aibs0 at acpi0: RTMP RVLT RFAN GGRP GITM SITM
acpibtn0 at acpi0: PWRB
cpu0: 3300 MHz: speeds: 3300 2500 1700 800 MHz
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "AMD RS880 Host" rev 0x00
ppb0 at pci0 dev 1 function 0 vendor "Asustek", unknown product 0x9602
rev 0x00
pci1 at ppb0 bus 1
vga1 at pci1 dev 5 function 0 "ATI Radeon HD 4200" rev 0x00
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
radeondrm0 at vga1: apic 6 int 18
drm0 at radeondrm0
azalia0 at pci1 dev 5 function 1 "ATI Radeon HD 4200 HD Audio" rev 0x00: msi
azalia0: no supported codecs
ppb1 at pci0 dev 10 function 0 "AMD RS780 PCIE" rev 0x00: msi
pci2 at ppb1 bus 2
re0 at pci2 dev 0 function 0 "Realtek 8168" rev 0x03: RTL8168D/8111D
(0x2800), apic 6 int 18, address 20:cf:30:b0:06:02
rgephy0 at re0 phy 7: RTL8169S/8110S PHY, rev. 2
ahci0 at pci0 dev 17 function 0 "ATI SBx00 SATA" rev 0x00: apic 6 int
22, AHCI 1.1
scsibus0 at ahci0: 32 targets
sd0 at scsibus0 targ 0 lun 0: <ATA, ST3000DM001-1CH1, CC43> SCSI3
0/direct fixed naa.5000c5004e455146
sd0: 2861588MB, 512 bytes/sector, 5860533168 sectors
sd1 at scsibus0 targ 1 lun 0: <ATA, ST3000DM001-1CH1, CC43> SCSI3
0/direct fixed naa.5000c5004e4a8141
sd1: 2861588MB, 512 bytes/sector, 5860533168 sectors
sd2 at scsibus0 targ 2 lun 0: <ATA, OCZ-VERTEX4, 1.5> SCSI3 0/direct
fixed naa.5e83a97810e6145a
sd2: 122104MB, 512 bytes/sector, 250069680 sectors, thin
ohci0 at pci0 dev 18 function 0 "ATI SB700 USB" rev 0x00: apic 6 int 16,
version 1.0, legacy support
ohci1 at pci0 dev 18 function 1 "ATI SB700 USB" rev 0x00: apic 6 int 16,
version 1.0, legacy support
ehci0 at pci0 dev 18 function 2 "ATI SB700 USB2" rev 0x00: apic 6 int 17
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 "ATI EHCI root hub" rev 2.00/1.00 addr 1
ohci2 at pci0 dev 19 function 0 "ATI SB700 USB" rev 0x00: apic 6 int 18,
version 1.0, legacy support
ohci3 at pci0 dev 19 function 1 "ATI SB700 USB" rev 0x00: apic 6 int 18,
version 1.0, legacy support
ehci1 at pci0 dev 19 function 2 "ATI SB700 USB2" rev 0x00: apic 6 int 19
usb1 at ehci1: USB revision 2.0
uhub1 at usb1 "ATI EHCI root hub" rev 2.00/1.00 addr 1
piixpm0 at pci0 dev 20 function 0 "ATI SBx00 SMBus" rev 0x3c: SMI
iic0 at piixpm0
iic0: addr 0x20 01=19 02=24 03=2e 04=00 05=00 06=00 07=00 08=00 09=00
0a=10 0b=10 0c=10 0d=10 0e=22 0f=92 10=3d 11=00 12=00 13=00 14=0a 15=0a
16=2c 17=a0 18=e0 1a=ae 1b=a4 1c=b3 1d=00 1e=0c 1f=01 20=09 21=09 22=09
23=09 24=bb 3e=03 words 00=ff19 01=1924 02=242e 03=2e00 04=0000 05=0000
06=0000 07=0000
spdmem0 at iic0 addr 0x50: 4GB DDR3 SDRAM PC3-10600
spdmem1 at iic0 addr 0x51: 4GB DDR3 SDRAM PC3-10600
spdmem2 at iic0 addr 0x52: 2GB DDR3 SDRAM PC3-10600
spdmem3 at iic0 addr 0x53: 2GB DDR3 SDRAM PC3-10600
pciide0 at pci0 dev 20 function 1 "ATI SB700 IDE" rev 0x00: DMA, channel
0 configured to compatibility, channel 1 configured to compatibility
atapiscsi0 at pciide0 channel 0 drive 0
scsibus1 at atapiscsi0: 2 targets
cd0 at scsibus1 targ 0 lun 0: <HL-DT-ST, DVDRAM GSA-4163B, AX13> ATAPI
5/cdrom removable
cd0(pciide0:0:0): using PIO mode 4, DMA mode 2, Ultra-DMA mode 4
azalia1 at pci0 dev 20 function 2 "ATI SBx00 HD Audio" rev 0x00: apic 6
int 16
azalia1: codecs: VIA/0x0397
audio0 at azalia1
pcib0 at pci0 dev 20 function 3 "ATI SB700 ISA" rev 0x00
ppb2 at pci0 dev 20 function 4 "ATI SB600 PCI" rev 0x00
pci3 at ppb2 bus 3
re1 at pci3 dev 5 function 0 "D-Link DGE-530T C1" rev 0x10:
RTL8169/8110SB (0x1000), apic 6 int 20, address 5c:d9:98:ae:3c:7b
rgephy1 at re1 phy 7: RTL8169S/8110S PHY, rev. 3
"VIA VT6306 FireWire" rev 0xc0 at pci3 dev 8 function 0 not configured
ohci4 at pci0 dev 20 function 5 "ATI SB700 USB" rev 0x00: apic 6 int 18,
version 1.0, legacy support
pchb1 at pci0 dev 24 function 0 "AMD AMD64 10h HyperTransport" rev 0x00
pchb2 at pci0 dev 24 function 1 "AMD AMD64 10h Address Map" rev 0x00
pchb3 at pci0 dev 24 function 2 "AMD AMD64 10h DRAM Cfg" rev 0x00
km0 at pci0 dev 24 function 3 "AMD AMD64 10h Misc Cfg" rev 0x00
pchb4 at pci0 dev 24 function 4 "AMD AMD64 10h Link Cfg" rev 0x00
usb2 at ohci0: USB revision 1.0
uhub2 at usb2 "ATI OHCI root hub" rev 1.00/1.00 addr 1
usb3 at ohci1: USB revision 1.0
uhub3 at usb3 "ATI OHCI root hub" rev 1.00/1.00 addr 1
usb4 at ohci2: USB revision 1.0
uhub4 at usb4 "ATI OHCI root hub" rev 1.00/1.00 addr 1
usb5 at ohci3: USB revision 1.0
uhub5 at usb5 "ATI OHCI root hub" rev 1.00/1.00 addr 1
isa0 at pcib0
isadma0 at isa0
com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
pckbc0 at isa0 port 0x60/5
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
lpt0 at isa0 port 0x378/4 irq 7
it0 at isa0 port 0x2e/2: IT8712F rev 8, EC port 0x290
usb6 at ohci4: USB revision 1.0
uhub6 at usb6 "ATI OHCI root hub" rev 1.00/1.00 addr 1
mtrr: Pentium Pro MTRR support
uhub7 at uhub0 port 4 "HP\M^? f2105 2PORT USB 2.0 HUB" rev 2.00/7.02 addr 2
ugen0 at uhub2 port 3 "APC Back-UPS ES 550G FW:843.K4 .D USB FW:K4" rev
1.10/1.06 addr 2
uhidev0 at uhub4 port 3 configuration 1 interface 0 "Logitech USB
Optical Mouse" rev 2.00/43.01 addr 2
uhidev0: iclass 3/1
ums0 at uhidev0: 3 buttons, Z dir
wsmouse0 at ums0 mux 0
vscsi0 at root
scsibus2 at vscsi0: 256 targets
softraid0 at root
scsibus3 at softraid0: 256 targets
sd3 at scsibus3 targ 1 lun 0: <OPENBSD, SR RAID 1, 005> SCSI2 0/direct fixed
sd3: 2861588MB, 512 bytes/sector, 5860532576 sectors
root on sd2a (27a551cc8502d62c.a) swap on sd2b dump on sd2b
softraid0: sd4 was not shutdown properly
softraid0: sd4 was not shutdown properly
sd4 at scsibus3 targ 2 lun 0: <OPENBSD, SR CRYPTO, 005> SCSI2 0/direct fixed
sd4: 1430793MB, 512 bytes/sector, 2930265808 sectors
--
Scott McEachern
https://www.blackstaff.ca
