On Tue, Dec 26, 2017 at 12:56:53PM -0800, Jordan wrote: > Hi everyone, long time lurker, first time poster. I've been around since the > 5.* days, so I would consider myself fairly seasoned in the ways of OpenBSD. > I've obviously done the RTFM dance, done it once, done it twice, been doing > it all week long now-- this problem really has me banging my head against a > wall. > > I'm cross posting this to both the misc and sparc64 mailing lists in the > hopes that I can entice both the softraid and sparc connoisseurs. > > I've recently gotten my hands on a couple shiny new SPARC T4-1 and T3-1 > servers and I was looking to install OpenBSD with a softraid mirror on them > for production use. The problem is, is that I end up with this upon > following the install instructions and rebooting: > > Boot device: /pci@400/pci@1/pci@0/pci@4/scsi@0/disk@p0 File and > args: sr0a:/bsd OpenBSD IEEE 1275 Bootblock 1.4 > > OpenBSD BOOT 1.9 Error: /iscsi-hba: no > iscsi-network-bootpath property Unknown device: sr0 Cannot > boot from softraid: Unknown error: code 19 Program terminated > > The only other information I can find online involving a softraid error code > 19 was the following: > http://openbsd-archive.7691.n7.nabble.com/5-8-sparc64-boot-from-softraid-4-fails-td284700.html > > > Considering he was running 5.8 and I am on 6.2 and the fix for his issue has > already been committed, I???m left scratching my head as to what the problem > here could be. > > The install procedure I followed on the T4 was: > > 1) Boot install kernel and drop to shell and provision RAID partitions on > both disks using the letter ???a??? via disklabel(8) > > 2) Assemble RAID volume with # bioctl -c 1 -l /dev/sd0a,/dev/sd1a softraid0 > > 3) I zeroed the first 10MB of the RAID volume with # dd if=/dev/urandom > of=/dev/rsd2c bs=1m
shouldn't that have been # dd if=/dev/zero of=/dev/rsd2c bs=1m count=1 > > 4) I finished off the install as usual by typing ???install??? into command > line and proceeded normally > > 5) I then rebooted and set the boot parameters at the ok prompt as per > boot_sparc(8) > > Aside from the sparc64 specific stuff, these same steps (plus x86 specific > fdisk stuff) get me a working, bootable softraid mirror on my i386/amd64 > test boxes. > > > It appears the issue I linked to in the OpenBSD archive, was that the > bootloader couldn't probe beyond 4 levels deep in the device tree. According > to Stefan Sperling he committed a patch to allow it to probe to (an > arbitrarily set) 10 levels deep in the device tree. According to my devalias > output, my disk0 appears to be 6 levels deep in the tree if I am reading it > correctly. > > I did manage to get the T4 booting from softraid using this guide: > http://brycv.com/blog/2012/openbsd-sparc64-and-root-on-softraid/ > > That guide however does seem to contravene the guidelines set forth in > softraid(4): ???On sparc64, bootable chunks must be RAID partitions using > the letter ???a??? .??? > > With the above guide from brycv.com we end up with 3 separate partitions on > our disks. This doesn't seem right as on amd64/i386 you would only have a > single raid partition per disk, not a separate root and swap partition in > addition to the RAID partitions. It would also seem that a rebuild would > involve significantly more manual intervention to maintain drive bootability > in the event of one of the drives in the mirror going down ie: One wouldn't > just be able to run bioctl -R /dev/rsd1a sd2 and have the array be golden > again, they would instead also need to mount and copy over the contents of > the other non RAID partitions on the disk. > > In short, on sparc64, is softraid boot like that of i386/amd64 where > everything including root is stored within the RAID volume, or does sparc64 > require me to have a small partition at the beginning of my physical disks > containing the kernel and ofwboot? One fellow on reddit said he had a T1000 > booting softraid crypto no problem so I am wondering if this is a T4 > specific issue. > > Sorry for the wall of text, I really appreciate any help you guys can offer. > Let me know if you need any other information and I???ll be happy to provide > it. Does anyone else here have any experience with sparc64 softraid booting > or with OpenBSD on a T3 or T4? Please let me know, and Merry Christmas to > all! > > Obligatory dmesg: > > T4 -1 Output > # dmesg > console is /virtual-devices@100/console@1 > Copyright (c) 1982, 1986, 1989, 1991, 1993 > The Regents of the University of California. All rights reserved. > Copyright (c) 1995-2017 OpenBSD. All rights reserved. > https://www.OpenBSD.org > OpenBSD 6.2 (RAMDISK) #282: Tue Oct 3 23:21:19 MDT 2017 > dera...@sparc64.openbsd.org:/usr/src/sys/arch/sparc64/compile/RAMDISK > real mem = 136902082560 (130560MB) > avail mem = 134512222208 (128280MB) > mainbus0 at root: SPARC T4-1 > cpu0 at mainbus0: SPARC-T4 (rev 0.0) @ 2847.862 MHz > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "SPARC-T4" at mainbus0 not configured > "reboot-memory" at mainbus0 not configured > vbus0 at mainbus0 > "flashprom" at vbus0 not configured > "tpm" at vbus0 not configured > cbus0 at vbus0 > "virtual-channel" at cbus0 not configured > "virtual-channel-client" at cbus0 not configured > "virtual-channel-client" at cbus0 not configured > "virtual-channel" at cbus0 not configured > "virtual-domain-service" at cbus0 not configured > vrng0 at vbus0 > vcons0 at vbus0: ivec 0x111, console > vrtc0 at vbus0 > vpci0 at mainbus0: bus 0 to 18, dvma map 80000000-ffffffff > pci0 at vpci0 > ppb0 at pci0 dev 1 function 0 "Sun SPARC PCIE" rev 0x01 > pci1 at ppb0 bus 1 > ppb1 at pci1 dev 0 function 0 vendor "IDT", unknown product 0x807a rev 0x02 > pci2 at ppb1 bus 2 > ppb2 at pci2 dev 0 function 0 vendor "IDT", unknown product 0x807a rev 0x02: > msi > pci3 at ppb2 bus 3 > ppb3 at pci2 dev 4 function 0 vendor "IDT", unknown product 0x807a rev 0x02 > pci4 at ppb3 bus 4 > mpii0 at pci4 dev 0 function 0 "Symbios Logic SAS2008" rev 0x03: msi > mpii0: RF On-Board, firmware 9.5.0.0 IR, MPI 2.0 > scsibus0 at mpii0: 834 targets > ppb4 at pci2 dev 6 function 0 vendor "IDT", unknown product 0x807a rev 0x02: > msi > pci5 at ppb4 bus 5 > ppb5 at pci2 dev 8 function 0 vendor "IDT", unknown product 0x807a rev 0x02: > msi > pci6 at ppb5 bus 6 > ppb6 at pci0 dev 2 function 0 "Sun SPARC PCIE" rev 0x01 > pci7 at ppb6 bus 7 > ppb7 at pci7 dev 0 function 0 vendor "IDT", unknown product 0x8077 rev 0x02 > pci8 at ppb7 bus 8 > ppb8 at pci8 dev 0 function 0 vendor "IDT", unknown product 0x8077 rev 0x02 > pci9 at ppb8 bus 9 > ppb9 at pci9 dev 0 function 0 "ASPEED Technology AST1150 PCI" rev 0x02 > pci10 at ppb9 bus 10 > "ASPEED Technology AST2000" rev 0x10 at pci10 dev 0 function 0 not > configured > ppb10 at pci8 dev 4 function 0 vendor "IDT", unknown product 0x8077 rev 0x02 > pci11 at ppb10 bus 11 > mpii1 at pci11 dev 0 function 0 "Symbios Logic SAS2008" rev 0x03: msi > mpii1: RF On-Board, firmware 9.5.0.0 IR, MPI 2.0 > scsibus1 at mpii1: 834 targets > ppb11 at pci8 dev 6 function 0 vendor "IDT", unknown product 0x8077 rev 0x02 > pci12 at ppb11 bus 12 > em0 at pci12 dev 0 function 0 "Intel 82576" rev 0x01: msi, address > 00:10:e0:3e:a1:de > em1 at pci12 dev 0 function 1 "Intel 82576" rev 0x01: msi, address > 00:10:e0:3e:a1:df > ppb12 at pci8 dev 7 function 0 vendor "IDT", unknown product 0x8077 rev 0x02 > pci13 at ppb12 bus 13 > em2 at pci13 dev 0 function 0 "Intel 82576" rev 0x01: msi, address > 00:10:e0:3e:a1:e0 > em3 at pci13 dev 0 function 1 "Intel 82576" rev 0x01: msi, address > 00:10:e0:3e:a1:e1 > ppb13 at pci8 dev 8 function 0 vendor "IDT", unknown product 0x8077 rev > 0x02: msi > pci14 at ppb13 bus 14 > ppb14 at pci8 dev 10 function 0 vendor "IDT", unknown product 0x8077 rev > 0x02: msi > pci15 at ppb14 bus 15 > ppb15 at pci8 dev 12 function 0 vendor "IDT", unknown product 0x8077 rev > 0x02: msi > pci16 at ppb15 bus 16 > ppb16 at pci8 dev 15 function 0 vendor "IDT", unknown product 0x8077 rev > 0x02 > pci17 at ppb16 bus 17 > ppb17 at pci17 dev 0 function 0 "PLX PEX 8112" rev 0xaa > pci18 at ppb17 bus 18 > ohci0 at pci18 dev 0 function 0 "NEC USB" rev 0x43: ivec 0x402, version 1.0 > ohci1 at pci18 dev 0 function 1 "NEC USB" rev 0x43: ivec 0x403, version 1.0 > ehci0 at pci18 dev 0 function 2 "NEC USB" rev 0x04: ivec 0x404 > usb0 at ehci0: USB revision 2.0 > uhub0 at usb0 configuration 1 interface 0 "NEC EHCI root hub" rev 2.00/1.00 > addr 1 > usb1 at ohci0: USB revision 1.0 > uhub1 at usb1 configuration 1 interface 0 "NEC OHCI root hub" rev 1.00/1.00 > addr 1 > usb2 at ohci1: USB revision 1.0 > uhub2 at usb2 configuration 1 interface 0 "NEC OHCI root hub" rev 1.00/1.00 > addr 1 > "niu" at mainbus0 not configured > "pci-performance-counters" at mainbus0 not configured > cd0 at scsibus1 targ 8 lun 0: <TEAC, DV-W28SS-W, 1.0A> ATAPI 5/cdrom > removable > sd0 at scsibus0 targ 2 lun 0: <HP, EG0300FAWHV, HPDD> SCSI3 0/direct fixed > naa.5000c5001b1e4cd3 > sd0: 286102MB, 512 bytes/sector, 585937500 sectors > sd1 at scsibus0 targ 5 lun 0: <ATA, SanDisk SDSSDA24, 30RL> SCSI3 0/direct > fixed naa.5001b448b4351630 > sd1: 228936MB, 512 bytes/sector, 468862128 sectors > sd2 at scsibus0 targ 3 lun 0: <HP, EG0300FAWHV, HPDD> SCSI3 0/direct fixed > naa.5000c50029674d0f > sd2: 286102MB, 512 bytes/sector, 585937500 sectors > uhub3 at uhub0 port 2 configuration 1 interface 0 "vendor 0x04b4 product > 0x6560" rev 2.00/0.0b addr 2 > uhub4 at uhub3 port 3 configuration 1 interface 0 "American Megatrends Inc. > Generic Hub" rev 2.00/1.00 addr 3 > umass0 at uhub4 port 2 configuration 1 interface 0 "American Megatrends Inc. > Virtual Cdrom Device" rev 2.00/1.00 addr 4 > umass0: using SCSI over Bulk-Only > scsibus2 at umass0: 2 targets, initiator 0 > cd1 at scsibus2 targ 1 lun 0: <AMI, Virtual CDROM, 1.00> SCSI0 5/cdrom > removable > cdce0 at uhub4 port 3 configuration 2 interface 0 "SunMicro Virtual Eth > Device" rev 2.00/1.00 addr 5 > cdce0: address 02:21:28:57:47:17 > uhidev0 at uhub3 port 4 configuration 1 interface 0 "American Megatrends > Inc. Virtual Keyboard and Mouse" rev 1.10/1.00 addr 6 > uhidev0: iclass 3/1 > ukbd0 at uhidev0 > wskbd0 at ukbd0 > uhidev1 at uhub3 port 4 configuration 1 interface 1 "American Megatrends > Inc. Virtual Keyboard and Mouse" rev 1.10/1.00 addr 6 > uhidev1: iclass 3/1 > uhid at uhidev1 not configured > uhub5 at uhub0 port 4 configuration 1 interface 0 "vendor 0x04b4 product > 0x6560" rev 2.00/0.0b addr 7 > softraid0 at root > scsibus3 at softraid0: 256 targets > sd3 at scsibus3 targ 1 lun 0: <OPENBSD, SR RAID 1, 006> SCSI2 0/direct fixed > sd3: 286101MB, 512 bytes/sector, 585934896 sectors > bootpath: > /pci@400,0/pci@1,0/pci@0,0/pci@4,0/scsi@0,0/disk@5001b448b4351630,0:c > root on rd0a swap on rd0b dump on rd0b > umass1 at uhub3 port 1 configuration 1 interface 0 "LEXR PLUG DRIVE LEXR > PLUG DRIVE" rev 1.10/0.01 addr 8 > umass1: using SCSI over Bulk-Only > scsibus4 at umass1: 2 targets, initiator 0 > sd4 at scsibus4 targ 1 lun 0: <LEXAR, JUMPDRIVE, 1.02> SCSI1 0/direct > removable serial.05dc0080000000000000 > sd4: 30MB, 512 bytes/sector, 62464 sectors > sd4 detached > scsibus4 detached > umass1 detached > # >
