Softraid crypto with keydisk and installboot, skip on the same disk
Hi,
I have strange setup on some of my machines, when I want to encrypt disk
where OpenBSD is installed, but still be able to boot them up without
any user interaction, like passphrase entry for CRYPTO softraid(4). I
have this so I can with little time spent lock out access to the disk,
by wiping beginning of the disk, instead of entire disk. I do recognise
magnitute of limitations of this. I still try to wipe entire disk, when
it's time for a machine decommission, but first I break CRYPTO softraid
by wiping beginning and then switch to proper full disk wipe.
All in all that brings me to the below diff. I was only able to test on
amd64, as this is the only type of machine which I have.
Index: i386_softraid.c
===
RCS file: /cvs/src/usr.sbin/installboot/i386_softraid.c,v
retrieving revision 1.19
diff -u -p -u -r1.19 i386_softraid.c
--- i386_softraid.c 29 Aug 2022 18:54:43 - 1.19
+++ i386_softraid.c 3 Sep 2022 11:28:55 -
@@ -65,6 +65,13 @@ sr_install_bootblk(int devfd, int vol, i
return;
}
+ /* Key disk has size of zero */
+ if (bd.bd_size == 0) {
+ fprintf(stderr, "softraid chunk %u looks like key disk - "
+ "skipping...\n", disk);
+ return;
+ }
+
if (strlen(bd.bd_vendor) < 1)
errx(1, "invalid disk name");
part = bd.bd_vendor[strlen(bd.bd_vendor) - 1];
Below follows my test and comments what happens without the diff
and with the diff.
First without the diff machine doesn't boot when I use keydisk on the
same disk which has the OpenBSD operaring system, wd0a and wd0d:
Booting from Hard Disk...
Using drive 0, partition 3.
Loading..
ERR M
To keep it short, it is because of installboot(8) installs boot blocks
on both wd0a and wd0d:
ramdisk# bioctl sd0
Volume Status Size Device
softraid0 0 Online 268426960384 sd0 CRYPTO
0 Online 268426960384 0:0.0 noencl
1 Online key disk 0:1.0 noencl
ramdisk# installboot -r /mnt -nv sd0
Using /mnt as root
would install bootstrap on /dev/rsd0c
using first-stage /mnt/usr/mdec/biosboot, second-stage
/mnt/usr/mdec/boot
sd0: softraid volume with 2 disk(s)
sd0: would install boot loader on softraid volume
/mnt/usr/mdec/boot is 6 blocks x 16384 bytes
wd0d: would install boot blocks on /dev/rwd0c, part offset 16145
master boot record (MBR) at sector 0
partition 3: type 0xA6 offset 64 size 524287936
/mnt/usr/mdec/biosboot will be written at sector 64
wd0a: would install boot blocks on /dev/rwd0c, part offset 144
master boot record (MBR) at sector 0
partition 3: type 0xA6 offset 64 size 524287936
/mnt/usr/mdec/biosboot will be written at sector 64
We see above that sd0 is softraid volume with 2 disks and then boot
blocks is installed on wd0d and wd0a. With my change boot blocks are
only installed on wd0d and wd0a (keydisk) is skipped.
ramdisk# cd /mnt
ramdisk# ftp https://example.com/installboot.bin
Trying 10.123.123.123...
Requesting https://example.com/installboot.bin
149040 bytes received in 0.21 seconds (671.19 KB/s)
ramdisk# chmod 0755 /mnt/installboot.bin
ramdisk# /mnt/installboot.bin -r /mnt -nv sd0
Using /mnt as root
would install bootstrap on /dev/rsd0c
using first-stage /mnt/usr/mdec/biosboot, second-stage /mnt/usr/mdec/boot
sd0: softraid volume with 2 disk(s)
sd0: would install boot loader on softraid volume
/mnt/usr/mdec/boot is 6 blocks x 16384 bytes
wd0d: would install boot blocks on /dev/rwd0c, part offset 16145
master boot record (MBR) at sector 0
partition 3: type 0xA6 offset 64 size 524287936
/mnt/usr/mdec/biosboot will be written at sector 64
softraid chunk 1 looks like key disk - skipping...
Here is full installation log on OpenBSD -current when installaton is
not bootable with above ERR M error.
root on rd0a swap on rd0b dump on rd0b
erase ^?, werase ^W, kill ^U, intr ^C, status ^T
Welcome to the OpenBSD/amd64 7.2 installation program.
(I)nstall, (U)pgrade, (A)utoinstall or (S)hell? s
# cd /dev
# sysctl -n hw.disknames
wd0:ccb72943e9945c9c,rd0:73efb2b48b898ab3
# sh MAKEDEV wd0 wd1 wd2 wd3
# sh MAKEDEV sd0 sd1 sd2 sd3
# dd if=/dev/zero bs=1024 count=10240 of=/dev/rwd0c
10240+0 records in
10240+0 records out
10485760 bytes transferred in 5.606 secs (1870132 bytes/sec)
# fdisk -iy wd0
Writing MBR at offset 0.
# disklabel -E wd0
Label editor (enter '?' for help at any prompt)
wd0> p g
OpenBSD area: 64-524288000; size: 250.0G; free: 250.0G
#size offset fstype [fsize bsize cpg]
c: 250.0G0 unused
wd0> a a
offset: [64]
size: [524287936] 4m
FS type: [4.2BSD] RAID
wd0*> a d
offset: [16065]
size: [524271935] *
FS type: [4.2BSD] RAID
wd0*> w
wd0> q
No label changes.
# dd if=/dev/zero bs=1024 count=10240 of=/dev/rwd0d
10240+0 records in
10240+0 records out
10485760 bytes transferred in 4.258 secs
Re: Softraid crypto with keydisk and installboot, skip on the same disk
On Sun, Sep 04, 2022 at 07:08:51PM +, Mikolaj Kucharski wrote: > I have strange setup on some of my machines, when I want to encrypt disk > where OpenBSD is installed, but still be able to boot them up without > any user interaction, like passphrase entry for CRYPTO softraid(4). I > have this so I can with little time spent lock out access to the disk, > by wiping beginning of the disk, instead of entire disk. An alternative method is just hard-code the passphrase in sr_crypto_passphrase_decrypt, which we mentioned here some time ago: https://research.exoticsilicon.com/series/reckless_guide_to_openbsd/kernel_softraid
macppc: boot.8, boot_macppc.8: fix file specification and boot disk names
On Sun, Sep 04, 2022 at 06:53:00AM +0100, Jason McIntyre wrote: > On Sat, Sep 03, 2022 at 06:12:16PM +, Klemens Nanni wrote: > > The current text is badly formatted and incomplete as to which optional > > parts can be omitted in which way > > > > The file specification used is of the form: > > promdev:partition/filename options > > where: ???promdev??? is an optional Open Firmware device name (such as > > ???hd??? > > or ???ide???) and ???partition??? is an optional disk partition number. > > Normal > > line editing characters can be used when typing the file specification. > > > > 1. 'partition' seems like an artifact that is not at all parsed as such, > >see sys/arch/macppc/stand/ ofwdev.c and main.c where the double colon is > >parsed; the comments confirm this with > > * this function is passed [device specifier]:[kernel] > >and > > * if the kernel image specified contains a ':' it is > > * [device]:[kernel], so separate the two fields. > > > >Manual tests confirm this. > > > > 2. if promdev is given, the double-colon MUST exist. > > double colon? Colon means "Doppelpunkt" in german, I mistakenly carried over the Doppel/double part. > > >'boot hd' tries to boot the kernel "hd" > >'boot hd:' tries to boot the kernel "hd:" > >'boot hd:/bsd' does what it says > >'boot hd:bsd' is equivalent to the previous line since ofwboot prepends > > trailing slashes; don't document to keep it simple > > > > 3. no vertical spacing makes the manual harder to read. > > > > We could zap the entire specification and rely on wording, similar to what > > boot_sparc64(8) does, but I prefer fixing the specification by removing > > partition from it and adding proper markup, yielding: > > > > The file specification used is of the form: > > > >[promdev:]filename [-acds] > > > > where promdev is an optional Open Firmware device name (such as hd or > > ide). Normal line editing characters can be used when typing the file > > specification. > > > > That should be much clearer. > > > > gkoehler greatly helped in confirming/testing various lines on his hardware. > > > > Feedback? Objection? OK? > > > > ok by me, but you probably want a technical ok as well. Thanks. I slightly changed the spec to [[promdev:]filename] [-acds] to match boot(8) which I have now also fixed, see below. > > jmc > > > NB: boot(8/macppc) also mentions this, but some parts are plain wrong... > > I suggest fixing that in a separate diff. The problem there is that it was copied over from amd64 boot.8 but then only partly adjusted. 1. amd64 biosboot(8) can boot off any disklabel slices, macppc ofwboot only uses 'a' and and it is not possible to specify a label in the first place. 2. both amd64 and macppc boot(8) say When selecting the device to boot from, boot makes no distinction between SCSI and IDE type drives; they are detected as ‘hd’ devices. which means they attach as hd(4) in the device. For amd64 biosboot(8) does take boot devices as 'hd', but macppc only takes 'cd', 'ide' and 'hd'... that has nothing to do with the kernel driver's name but is entirely an OpenFirmware thing. Therefore remove the mention of slices alltogether in macppc's boot(8) except for the /etc/boot.conf part which obviously covers the OS disk/filesystem; there explicitly add the slice 'a' since, again, macppc always forcec 'a' rather than dynamically picking the slice ofwboot itself was loaded from (read amd64 boot(8) for that). It's still debatable whether we should document things twice in both manuals, but I don't want to spend too much time on this. So let's just fix the obvious mistakes and help users navigate through macppc's boot process. Feedback? OK? Index: sys/arch/macppc/stand/boot.8 === RCS file: /cvs/src/sys/arch/macppc/stand/boot.8,v retrieving revision 1.4 diff -u -p -r1.4 boot.8 --- sys/arch/macppc/stand/boot.823 Dec 2019 12:44:34 - 1.4 +++ sys/arch/macppc/stand/boot.84 Sep 2022 15:02:44 - @@ -77,7 +77,7 @@ If the file exists on the filesystem in slice .Sq a on the first disk drive -.Pq wd0 , +.Pq wd0a , open and parse it. Lines beginning with the .Sq # @@ -125,29 +125,26 @@ The following commands are accepted at t .Nm prompt: .Bl -tag -width shorten -.It Ic boot Op Ar image Op Fl acds -Boots the kernel image specified by +.It Ic boot Oo Oo Ar device : Oc Ns Ar image Oc Op Fl acds +Boots the specified kernel image with any options given. +If +.Ar device +or .Ar image -with any options given. -Image specification consists of a pair -.Ar device : Ns Ar filename ; -either or both can be omitted (`:' is not needed if both are omitted), -in which case values from +are omitted, values from the .Nm variables will be used. .Pp The only bootable
Re: /etc/rc.d/iked regression after r1.9
On Sat, Sep 03, 2022 at 11:51:41PM +, Lucas wrote:
> Hello tech@,
>
> commit r1.9 removed the rc_exec call in iked's rc_pre. Because of that,
> the exit code of rc_pre is that of the && list. In the case of
> sasyncd_flags=NO, this means that rc_pre fails and triggers the break
> in the while true loop in rc_cmd start case. I solved this using the
> same approach as in isakmpd rc file: hardcoding a return 0 afterwards.
> The other alternative is using an if instead of && list, idiom which I
> prefer, but haven't seen much in these files.
Good finding.
Committed, thanks.
> -Lucas
>
>
> diff /usr/src
> commit - a300f670c8e17f72646e4eaedfbfeb3ce01a295f
> path + /usr/src
> blob - 8cfb786e6a883c83d4d27de65fb55ce894632cab
> file + etc/rc.d/iked
> --- etc/rc.d/iked
> +++ etc/rc.d/iked
> @@ -15,6 +15,7 @@ rc_configtest() {
> rc_pre() {
> [[ ${sasyncd_flags} != NO ]] &&
> daemon_flags="-S ${daemon_flags}"
> + return 0
> }
>
> rc_cmd $1
>
--
Antoine
Re: Softraid crypto with keydisk and installboot, skip on the same disk
On Sun, Sep 04, 2022 at 07:08:51PM +, Mikolaj Kucharski wrote:
> Hi,
>
> I have strange setup on some of my machines, when I want to encrypt disk
> where OpenBSD is installed, but still be able to boot them up without
> any user interaction, like passphrase entry for CRYPTO softraid(4). I
> have this so I can with little time spent lock out access to the disk,
> by wiping beginning of the disk, instead of entire disk. I do recognise
> magnitute of limitations of this. I still try to wipe entire disk, when
> it's time for a machine decommission, but first I break CRYPTO softraid
> by wiping beginning and then switch to proper full disk wipe.
I don't see that as a supported use-case at all; this reads like
hand-waving to me and I fail to see any benefit in such a setup.
Keydisks ought to be separate devices (not just slices) by design.
Actively supporting such a setup feels like a step backwards.
>
> All in all that brings me to the below diff. I was only able to test on
> amd64, as this is the only type of machine which I have.
>
>
> Index: i386_softraid.c
> ===
> RCS file: /cvs/src/usr.sbin/installboot/i386_softraid.c,v
> retrieving revision 1.19
> diff -u -p -u -r1.19 i386_softraid.c
> --- i386_softraid.c 29 Aug 2022 18:54:43 - 1.19
> +++ i386_softraid.c 3 Sep 2022 11:28:55 -
> @@ -65,6 +65,13 @@ sr_install_bootblk(int devfd, int vol, i
> return;
> }
>
> + /* Key disk has size of zero */
> + if (bd.bd_size == 0) {
> + fprintf(stderr, "softraid chunk %u looks like key disk - "
> + "skipping...\n", disk);
> + return;
> + }
> +
> if (strlen(bd.bd_vendor) < 1)
> errx(1, "invalid disk name");
> part = bd.bd_vendor[strlen(bd.bd_vendor) - 1];
>
>
> Below follows my test and comments what happens without the diff
> and with the diff.
>
> First without the diff machine doesn't boot when I use keydisk on the
> same disk which has the OpenBSD operaring system, wd0a and wd0d:
You discovered it works half-way, I wouldn't be surprised if other
platforms/boot loaders behaved differently.
Pushing this keydisk-on-same-device design would only encourage users,
which is bad idea, imho.
It seems like stating the obvious, but maybe softraid(4) should contain
clear words like 'keeping the key next to the crypto volume is insecure'
or 'put the keydisk on separate device'.
>
> Booting from Hard Disk...
> Using drive 0, partition 3.
> Loading..
> ERR M
>
>
> To keep it short, it is because of installboot(8) installs boot blocks
> on both wd0a and wd0d:
>
> ramdisk# bioctl sd0
> Volume Status Size Device
> softraid0 0 Online 268426960384 sd0 CRYPTO
> 0 Online 268426960384 0:0.0 noencl
> 1 Online key disk 0:1.0 noencl
So CRYPTO on wd0d and keydisk on wd0a...
>
>
> ramdisk# installboot -r /mnt -nv sd0
> Using /mnt as root
> would install bootstrap on /dev/rsd0c
> using first-stage /mnt/usr/mdec/biosboot, second-stage
> /mnt/usr/mdec/boot
> sd0: softraid volume with 2 disk(s)
> sd0: would install boot loader on softraid volume
> /mnt/usr/mdec/boot is 6 blocks x 16384 bytes
> wd0d: would install boot blocks on /dev/rwd0c, part offset 16145
> master boot record (MBR) at sector 0
> partition 3: type 0xA6 offset 64 size 524287936
> /mnt/usr/mdec/biosboot will be written at sector 64
> wd0a: would install boot blocks on /dev/rwd0c, part offset 144
> master boot record (MBR) at sector 0
> partition 3: type 0xA6 offset 64 size 524287936
> /mnt/usr/mdec/biosboot will be written at sector 64
>
> We see above that sd0 is softraid volume with 2 disks and then boot
> blocks is installed on wd0d and wd0a. With my change boot blocks are
> only installed on wd0d and wd0a (keydisk) is skipped.
... and installboot writes to the keydisk, making it bootable.
Is this failing to boot for you because wd0a is tried first but fails
because it is a keydisk? Put differently, would your questionable
setup boot without a diff if CRYPTO was on wd0a and the keydisk on wd0d?
Seing installboot write to keydisks seems unexpected, I'd say it should
always ignore them.
Are keydisks expected to be present after unlocking the volume?
I'm picturing users having a USB stick they put in temporarily to unlock
and then boot/run the system without the stick plugged in.
But installboot seems to expect keydisks to be present, which is really
odd. It means you can't install bootstraps on your currently unlocked
crypto volume without having the keydisk available...
That does not sound right, but I'm not a keydisk user myself, so I have
to test/dig around a little.
>
>
> ramdisk# cd /mnt
> ramdisk# ftp https://example.com/installboot.bin
> Trying 10.123.123.123...
> Requesting https://example.com/installboot.bin
> 149040 bytes received in 0.21 seconds (671.19 KB/s)
>
Re: Softraid crypto with keydisk and installboot, skip on the same disk
On Sun, Sep 04, 2022 at 10:21:24PM +, Klemens Nanni wrote: ... > > Booting from Hard Disk... > > Using drive 0, partition 3. > > Loading.. > > ERR M > > > > > > To keep it short, it is because of installboot(8) installs boot blocks > > on both wd0a and wd0d: > > > > ramdisk# bioctl sd0 > > Volume Status Size Device > > softraid0 0 Online 268426960384 sd0 CRYPTO > > 0 Online 268426960384 0:0.0 noencl > > 1 Online key disk 0:1.0 noencl > > So CRYPTO on wd0d and keydisk on wd0a... Correct. > > > > ramdisk# installboot -r /mnt -nv sd0 > > Using /mnt as root > > would install bootstrap on /dev/rsd0c > > using first-stage /mnt/usr/mdec/biosboot, second-stage > > /mnt/usr/mdec/boot > > sd0: softraid volume with 2 disk(s) > > sd0: would install boot loader on softraid volume > > /mnt/usr/mdec/boot is 6 blocks x 16384 bytes > > wd0d: would install boot blocks on /dev/rwd0c, part offset 16145 > > master boot record (MBR) at sector 0 > > partition 3: type 0xA6 offset 64 size 524287936 > > /mnt/usr/mdec/biosboot will be written at sector 64 > > wd0a: would install boot blocks on /dev/rwd0c, part offset 144 > > master boot record (MBR) at sector 0 > > partition 3: type 0xA6 offset 64 size 524287936 > > /mnt/usr/mdec/biosboot will be written at sector 64 > > > > We see above that sd0 is softraid volume with 2 disks and then boot > > blocks is installed on wd0d and wd0a. With my change boot blocks are > > only installed on wd0d and wd0a (keydisk) is skipped. > > ... and installboot writes to the keydisk, making it bootable. > > Is this failing to boot for you because wd0a is tried first but fails > because it is a keydisk? Put differently, would your questionable > setup boot without a diff if CRYPTO was on wd0a and the keydisk on wd0d? I don't know. I'm puzzled. I just tested with the other way around: # bioctl sd0 Volume Status Size Device softraid0 0 Online 268426960384 sd0 CRYPTO 0 Online 268426960384 0:0.0 noencl 1 Online key disk 0:1.0 noencl and installboot(8) did each device in different order: # installboot -r /mnt -nv sd0 Using /mnt as root would install bootstrap on /dev/rsd0c using first-stage /mnt/usr/mdec/biosboot, second-stage /mnt/usr/mdec/boot sd0: softraid volume with 2 disk(s) sd0: would install boot loader on softraid volume /mnt/usr/mdec/boot is 6 blocks x 16384 bytes wd0a: would install boot blocks on /dev/rwd0c, part offset 144 master boot record (MBR) at sector 0 partition 3: type 0xA6 offset 64 size 524287936 /mnt/usr/mdec/biosboot will be written at sector 64 wd0d: would install boot blocks on /dev/rwd0c, part offset 524272079 master boot record (MBR) at sector 0 partition 3: type 0xA6 offset 64 size 524287936 /mnt/usr/mdec/biosboot will be written at sector 64 so far, this is what I expected, that installboot would iterate each disk as they are shown in bioctl(8) output, wd0a first, wd0d second. However, surprisingly to me, machine still didn't boot. Press ESC for boot menu. Booting from Hard Disk... Using drive 0, partition 3. Loading.. ERR M Full log at the end of this email (without OpenBSD install itself, which is typical). > Seing installboot write to keydisks seems unexpected, I'd say it should > always ignore them. > > Are keydisks expected to be present after unlocking the volume? > I'm picturing users having a USB stick they put in temporarily to unlock > and then boot/run the system without the stick plugged in. > > But installboot seems to expect keydisks to be present, which is really > odd. It means you can't install bootstraps on your currently unlocked > crypto volume without having the keydisk available... I don't think installboot expects keydisks to be present. I don't think it cares. It just iterates over softraid(4) chunks which are online. > That does not sound right, but I'm not a keydisk user myself, so I have > to test/dig around a little. > I'm not sure did I made any mistake below, from what Klemens wanted to know. erase ^?, werase ^W, kill ^U, intr ^C, status ^T Welcome to the OpenBSD/amd64 7.2 installation program. (I)nstall, (U)pgrade, (A)utoinstall or (S)hell? s # sysctl -n hw.disknames wd0:ccb72943e9945c9c,rd0:73efb2b48b898ab3 # cd /dev # sh MAKEDEV wd0 wd1 wd2 wd3 # sh MAKEDEV sd0 sd1 sd2 sd3 # dd if=/dev/zero bs=1024 count=10240 of=/dev/rwd0c 10240+0 records in 10240+0 records out 10485760 bytes transferred in 4.955 secs (2115830 bytes/sec) # fdisk -iy wd0 Writing MBR at offset 0. # disklabel -E wd0 Label editor (enter '?' for help at any prompt) wd0> p g OpenBSD area: 64-524288000; size: 250.0G; free: 250.0G #size offset fstype [fsize bsize cpg] c: 250.0G0 unused wd0> a a offset: [64] size: [524287936] 524271935 FS type: [4.2BSD] RAID wd0*> a d offset: [524271999]
Re: Softraid crypto with keydisk and installboot, skip on the same disk
On Sun, Sep 04, 2022 at 11:25:08PM +, Mikolaj Kucharski wrote: > # bioctl sd0 > Volume Status Size Device > softraid0 0 Online 268426960384 sd0 CRYPTO > 0 Online 268426960384 0:0.0 noencl > 1 Online key disk 0:1.0 noencl ... > keydisk# installboot -r /mnt -nv sd0 > Using /mnt as root > would install bootstrap on /dev/rsd0c > using first-stage /mnt/usr/mdec/biosboot, second-stage /mnt/usr/mdec/boot > sd0: softraid volume with 2 disk(s) > sd0: would install boot loader on softraid volume > /mnt/usr/mdec/boot is 6 blocks x 16384 bytes > wd0a: would install boot blocks on /dev/rwd0c, part offset 144 > master boot record (MBR) at sector 0 > partition 3: type 0xA6 offset 64 size 524287936 > /mnt/usr/mdec/biosboot will be written at sector 64 > wd0d: would install boot blocks on /dev/rwd0c, part offset 524272079 > master boot record (MBR) at sector 0 > partition 3: type 0xA6 offset 64 size 524287936 > /mnt/usr/mdec/biosboot will be written at sector 64 > keydisk# reboot > > > Press ESC for boot menu. > > Booting from Hard Disk... > Using drive 0, partition 3. > Loading.. > ERR M Ah no. It make sense why it doesn't boot. Because key disk is always last on the list of chunks and that means no matter what is the order on wd0, installboot will install boot blocks on key disk always last, so last chunk (key disk) always wins, hence boot failure. -- Regards, Mikolaj
