On Sun, Sep 04, 2022 at 10:21:24PM +0000, Klemens Nanni wrote:
...
> > Booting from Hard Disk...
> > Using drive 0, partition 3.
> > Loading......
> > ERR M
> >
> >
> > To keep it short, it is because of installboot(8) installs boot blocks
> > on both wd0a and wd0d:
> >
> > ramdisk# bioctl sd0
> > Volume Status Size Device
> > softraid0 0 Online 268426960384 sd0 CRYPTO
> > 0 Online 268426960384 0:0.0 noencl <wd0d>
> > 1 Online key disk 0:1.0 noencl <wd0a>
>
> So CRYPTO on wd0d and keydisk on wd0a...
Correct.
> >
> > ramdisk# installboot -r /mnt -nv sd0
> > Using /mnt as root
> > would install bootstrap on /dev/rsd0c
> > using first-stage /mnt/usr/mdec/biosboot, second-stage
> > /mnt/usr/mdec/boot
> > sd0: softraid volume with 2 disk(s)
> > sd0: would install boot loader on softraid volume
> > /mnt/usr/mdec/boot is 6 blocks x 16384 bytes
> > wd0d: would install boot blocks on /dev/rwd0c, part offset 16145
> > master boot record (MBR) at sector 0
> > partition 3: type 0xA6 offset 64 size 524287936
> > /mnt/usr/mdec/biosboot will be written at sector 64
> > wd0a: would install boot blocks on /dev/rwd0c, part offset 144
> > master boot record (MBR) at sector 0
> > partition 3: type 0xA6 offset 64 size 524287936
> > /mnt/usr/mdec/biosboot will be written at sector 64
> >
> > We see above that sd0 is softraid volume with 2 disks and then boot
> > blocks is installed on wd0d and wd0a. With my change boot blocks are
> > only installed on wd0d and wd0a (keydisk) is skipped.
>
> ... and installboot writes to the keydisk, making it bootable.
>
> Is this failing to boot for you because wd0a is tried first but fails
> because it is a keydisk? Put differently, would your questionable
> setup boot without a diff if CRYPTO was on wd0a and the keydisk on wd0d?
I don't know. I'm puzzled. I just tested with the other way around:
# bioctl sd0
Volume Status Size Device
softraid0 0 Online 268426960384 sd0 CRYPTO
0 Online 268426960384 0:0.0 noencl <wd0a>
1 Online key disk 0:1.0 noencl <wd0d>
and installboot(8) did each device in different order:
# installboot -r /mnt -nv sd0
Using /mnt as root
would install bootstrap on /dev/rsd0c
using first-stage /mnt/usr/mdec/biosboot, second-stage
/mnt/usr/mdec/boot
sd0: softraid volume with 2 disk(s)
sd0: would install boot loader on softraid volume
/mnt/usr/mdec/boot is 6 blocks x 16384 bytes
wd0a: would install boot blocks on /dev/rwd0c, part offset 144
master boot record (MBR) at sector 0
partition 3: type 0xA6 offset 64 size 524287936
/mnt/usr/mdec/biosboot will be written at sector 64
wd0d: would install boot blocks on /dev/rwd0c, part offset 524272079
master boot record (MBR) at sector 0
partition 3: type 0xA6 offset 64 size 524287936
/mnt/usr/mdec/biosboot will be written at sector 64
so far, this is what I expected, that installboot would iterate each
disk as they are shown in bioctl(8) output, wd0a first, wd0d second.
However, surprisingly to me, machine still didn't boot.
Press ESC for boot menu.
Booting from Hard Disk...
Using drive 0, partition 3.
Loading......
ERR M
Full log at the end of this email (without OpenBSD install itself, which is
typical).
> Seing installboot write to keydisks seems unexpected, I'd say it should
> always ignore them.
>
> Are keydisks expected to be present after unlocking the volume?
> I'm picturing users having a USB stick they put in temporarily to unlock
> and then boot/run the system without the stick plugged in.
>
> But installboot seems to expect keydisks to be present, which is really
> odd. It means you can't install bootstraps on your currently unlocked
> crypto volume without having the keydisk available...
I don't think installboot expects keydisks to be present. I don't think
it cares. It just iterates over softraid(4) chunks which are online.
> That does not sound right, but I'm not a keydisk user myself, so I have
> to test/dig around a little.
>
I'm not sure did I made any mistake below, from what Klemens wanted to know.
erase ^?, werase ^W, kill ^U, intr ^C, status ^T
Welcome to the OpenBSD/amd64 7.2 installation program.
(I)nstall, (U)pgrade, (A)utoinstall or (S)hell? s
# sysctl -n hw.disknames
wd0:ccb72943e9945c9c,rd0:73efb2b48b898ab3
# cd /dev
# sh MAKEDEV wd0 wd1 wd2 wd3
# sh MAKEDEV sd0 sd1 sd2 sd3
# dd if=/dev/zero bs=1024 count=10240 of=/dev/rwd0c
10240+0 records in
10240+0 records out
10485760 bytes transferred in 4.955 secs (2115830 bytes/sec)
# fdisk -iy wd0
Writing MBR at offset 0.
# disklabel -E wd0
Label editor (enter '?' for help at any prompt)
wd0> p g
OpenBSD area: 64-524288000; size: 250.0G; free: 250.0G
# size offset fstype [fsize bsize cpg]
c: 250.0G 0 unused
wd0> a a
offset: [64]
size: [524287936] 524271935
FS type: [4.2BSD] RAID
wd0*> a d
offset: [524271999]
size: [16001]
FS type: [4.2BSD] RAID
wd0*> p g
OpenBSD area: 64-524288000; size: 250.0G; free: 0.0G
# size offset fstype [fsize bsize cpg]
a: 250.0G 64 RAID
c: 250.0G 0 unused
d: 0.0G 524271999 RAID
wd0*> a m
No space left, you need to shrink a partition
wd0*> p m
OpenBSD area: 64-524288000; size: 256000.0M; free: 0.0M
# size offset fstype [fsize bsize cpg]
a: 255992.2M 64 RAID
c: 256000.0M 0 unused
d: 7.8M 524271999 RAID
wd0*> w
wd0> q
No label changes.
# dd if=/dev/zero bs=1024 count=10240 of=/dev/rwd0d
dd: /dev/rwd0d: short write on character device
dd: /dev/rwd0d: end of device
8001+0 records in
8000+1 records out
8192512 bytes transferred in 3.514 secs (2330784 bytes/sec)
# dd if=/dev/zero bs=1024 count=10240 of=/dev/rwd0a
10240+0 records in
10240+0 records out
10485760 bytes transferred in 4.806 secs (2181493 bytes/sec)
# sync
# bioctl -c C -l wd0a -k wd0d softraid0
sd0 at scsibus0 targ 1 lun 0: <OPENBSD, SR CRYPTO, 006>
sd0: 255991MB, 512 bytes/sector, 524271407 sectors
softraid0: CRYPTO volume attached as sd0
# dd if=/dev/zero bs=1024 count=10240 of=/dev/rsd0c
10240+0 records in
10240+0 records out
10485760 bytes transferred in 6.472 secs (1620141 bytes/sec)
# fdisk -iy sd0
Writing MBR at offset 0.
# bioctl sd0
Volume Status Size Device
softraid0 0 Online 268426960384 sd0 CRYPTO
0 Online 268426960384 0:0.0 noencl <wd0a>
1 Online key disk 0:1.0 noencl <wd0d>
# exit
...
Exit to (S)hell, (H)alt or (R)eboot? [reboot] s
To boot the new system, enter 'reboot' at the command prompt.
keydisk# installboot -r /mnt -nv sd0
Using /mnt as root
would install bootstrap on /dev/rsd0c
using first-stage /mnt/usr/mdec/biosboot, second-stage /mnt/usr/mdec/boot
sd0: softraid volume with 2 disk(s)
sd0: would install boot loader on softraid volume
/mnt/usr/mdec/boot is 6 blocks x 16384 bytes
wd0a: would install boot blocks on /dev/rwd0c, part offset 144
master boot record (MBR) at sector 0
partition 3: type 0xA6 offset 64 size 524287936
/mnt/usr/mdec/biosboot will be written at sector 64
wd0d: would install boot blocks on /dev/rwd0c, part offset 524272079
master boot record (MBR) at sector 0
partition 3: type 0xA6 offset 64 size 524287936
/mnt/usr/mdec/biosboot will be written at sector 64
keydisk# reboot
Press ESC for boot menu.
Booting from Hard Disk...
Using drive 0, partition 3.
Loading......
ERR M
--
Regards,
Mikolaj
