Re: Installer support to fetch/verify bsd.rd for release upgrade
On Sun, Oct 08, 2017 at 09:56:15AM +, Robert Peichaer wrote: > Up to now, the upgrade procedure from one to the next release meant > that you had to manually download and verify the new ramdisk kernel. > > What about if you just needed to boot into the existing bsd.rd and > it would support downloading and verifying the bsd.rd of the next > release? > > This diff changes the installer script to support such a scenario. > > 1) Boot the existing bsd.rd and choose (U)pgrade > 2) Enter the "Server directory" of the new release >The installer then offers just the bsd.rd >The on-disk signify key of the new release is used for verify it > 3) Reboot into the new bsd.rd and do the upgrade > > > An important assumption for this to work properly is: > >Upgrades are only supported from one release to the release >immediately following it. [1] > > > It would look like this for the 6.2 to 6.3 upgrade situation. > (The version numbers are obviously faked) > > Let's upgrade the sets! > Location of sets? (cd0 disk http or 'done') [http] > HTTP proxy URL? (e.g. 'http://proxy:8080', or 'none') [none] > HTTP Server? (hostname, list#, 'done' or '?') [ftp.hostserver.de] > Server directory? [pub/OpenBSD/6.2/amd64] pub/OpenBSD/6.3/amd64 > Unable to get a verified list of distribution sets. > > Select sets by entering a set name, a file name pattern or 'all'. De-select > sets by prepending a '-', e.g.: '-game*'. Selected sets are labelled '[X]'. > [X] bsd.rd > Set name(s)? (or 'abort' or 'done') [done] > Get/Verify SHA256.sig 100% |**| 2152 00:00 > Signature Verified > Get/Verify bsd.rd 100% |**| 9565 KB00:14 > Installing bsd.rd 100% |**| 9565 KB00:00 > Location of sets? (cd0 disk http or 'done') [done] > Making all device nodes...done. > > CONGRATULATIONS! Your OpenBSD upgrade has been successfully completed! > To boot the new system, enter 'reboot' at the command prompt. In October 2017 I've added this "feature" to the installer that allows to boot with an existing bsd.rd and to download/install the bsd.rd of the next release which can then be used to do the actual system upgrade. It depends on differing release versions of the booted bsd.rd and sets found in the specified HTTP "Server directory". To be more precise it assumes the sets release version is one version ahead of the bsd.rd. The logic is meant to be simple and to support this scenario reliably. This is a best-effort, convenience feature intended for users that do upgrades from release to release. Users will be able to use it when they upgrade from 6.3 to 6.4 and so forth. I'm aware of reports that this does not work as expected, but as far as I was able to test and verify, the problem most probably was that in these cases the 6.2-current bsd.rd was too old to have the feature or was already on some version of 6.3 which means no release version difference and so this feature was not triggerd. Cheers Robert
Re: Installer support to fetch/verify bsd.rd for release upgrade
On Sun, Oct 08, 2017 at 09:56:15AM +, Robert Peichaer wrote: > Up to now, the upgrade procedure from one to the next release meant > that you had to manually download and verify the new ramdisk kernel. > > What about if you just needed to boot into the existing bsd.rd and > it would support downloading and verifying the bsd.rd of the next > release? > > This diff changes the installer script to support such a scenario. > > 1) Boot the existing bsd.rd and choose (U)pgrade > 2) Enter the "Server directory" of the new release >The installer then offers just the bsd.rd >The on-disk signify key of the new release is used for verify it > 3) Reboot into the new bsd.rd and do the upgrade > > > An important assumption for this to work properly is: > >Upgrades are only supported from one release to the release >immediately following it. [1] > > > It would look like this for the 6.2 to 6.3 upgrade situation. > (The version numbers are obviously faked) > > Let's upgrade the sets! > Location of sets? (cd0 disk http or 'done') [http] > HTTP proxy URL? (e.g. 'http://proxy:8080', or 'none') [none] > HTTP Server? (hostname, list#, 'done' or '?') [ftp.hostserver.de] > Server directory? [pub/OpenBSD/6.2/amd64] pub/OpenBSD/6.3/amd64 > Unable to get a verified list of distribution sets. > > Select sets by entering a set name, a file name pattern or 'all'. De-select > sets by prepending a '-', e.g.: '-game*'. Selected sets are labelled '[X]'. > [X] bsd.rd > Set name(s)? (or 'abort' or 'done') [done] > Get/Verify SHA256.sig 100% |**| 2152 00:00 > Signature Verified > Get/Verify bsd.rd 100% |**| 9565 KB00:14 > Installing bsd.rd 100% |**| 9565 KB00:00 > Location of sets? (cd0 disk http or 'done') [done] > Making all device nodes...done. > > CONGRATULATIONS! Your OpenBSD upgrade has been successfully completed! > To boot the new system, enter 'reboot' at the command prompt. > > > Here's the diff and below is a more detailed description. Well, here's a diff, that actually works. Index: install.sub === RCS file: /cvs/src/distrib/miniroot/install.sub,v retrieving revision 1.1036 diff -u -p -p -u -r1.1036 install.sub --- install.sub 4 Oct 2017 18:19:33 - 1.1036 +++ install.sub 9 Oct 2017 17:41:59 - @@ -1330,6 +1330,15 @@ sane_install() { # select_sets() { local _avail=$1 _selected=$2 _f _action _col=$COLUMNS + local _bsd_rd _no_sets=true + + if [[ $MODE == upgrade ]]; then + for _f in $_avail; do + [[ $_f != bsd* ]] && _no_sets=false + [[ $_f == bsd.rd* ]] && _bsd_rd=$_f + done + $_no_sets && UPGRADE_BSDRD=true _avail=$_bsd_rd _selected=$_bsd_rd + fi # account for 4 spaces added to the sets list let COLUMNS=_col-8 @@ -1516,6 +1525,9 @@ install_files() { ! $_unpriv ftp -D "$_t" -Vmo - "$_src/SHA256.sig" >"$_cfile.sig" && _issue="Cannot fetch SHA256.sig" && break + $UPGRADE_BSDRD && + PUB_KEY=/mnt/etc/signify/openbsd-$((VERSION + 1))-base.pub + # Verify signature file with public keys. ! unpriv -f "$_cfile" \ signify -Vep $PUB_KEY -x "$_cfile.sig" -m "$_cfile" && @@ -1576,7 +1588,9 @@ install_files() { tar -zxphf - -C /mnt fi ;; - *) $_unpriv ftp -D Installing -Vmo - "$_fsrc" >"/mnt/$_f" + *) $UPGRADE_BSDRD && [[ $_f == bsd.rd* ]] && + cp /mnt/$_f /mnt/$_f.old.$VERSION + $_unpriv ftp -D Installing -Vmo - "$_fsrc" >"/mnt/$_f" ;; esac if (($?)); then @@ -1587,6 +1601,7 @@ install_files() { fi else DEFAULTSETS=$(rmel $_f $DEFAULTSETS) + $UPGRADE_BSDRD && DEFAULTSETS= fi [[ -d $_tmpsrc ]] && rm -f "$_tmpsrc/$_f" done @@ -3139,6 +3154,7 @@ PUB_KEY=/etc/signify/openbsd-${VERSION}- ROOTDEV= ROOTDISK= SETDIR="$VNAME/$ARCH" +UPGRADE_BSDRD=false V4_DHCPCONF=false V6_AUTOCONF=false WLANLIST=/tmp/i/wlanlist === Stats: --- 1 lines 60 chars Stats: +++ 17 lines 525 chars Stats: 16 lines Stats: 465 chars -- -=[rpe]=-
Re: Installer support to fetch/verify bsd.rd for release upgrade
On Sun, Oct 08, 2017 at 05:27:09PM +0200, Sebastian Benoit wrote: > Robert Peichaer(rob...@peichaer.org) on 2017.10.08 09:56:15 +: > > Up to now, the upgrade procedure from one to the next release meant > > that you had to manually download and verify the new ramdisk kernel. > > > > What about if you just needed to boot into the existing bsd.rd and > > it would support downloading and verifying the bsd.rd of the next > > release? > > Why put this into the installer and not into a script on the system? > With this, it takes an extra reboot, just to get bsd.rd. > > (In my case the number one mistake i make when upgrading is that i sleep > through the boot> prompt timeout, so this gives me 2 chances ;)). > > Also, you would have to type > > Server directory? [pub/OpenBSD/6.2/amd64] pub/OpenBSD/6.3/amd64 > > by hand. Could this automatically be the next version? > > /B. You're assuming release-to-release... the more common scenario for most of us is snapshot suddenly bumping keys and version. As far as I'm concerned, I've already rebooted, and suddenly I figure out the bsd.rd on THAT machine does not know the correct key. Being able to grab/verify the new bsd.rd will save me from one reboot to full userland and whatever verification script I have.
Re: Installer support to fetch/verify bsd.rd for release upgrade
> Robert Peichaer(rob...@peichaer.org) on 2017.10.08 09:56:15 +: > > Up to now, the upgrade procedure from one to the next release meant > > that you had to manually download and verify the new ramdisk kernel. > > > > What about if you just needed to boot into the existing bsd.rd and > > it would support downloading and verifying the bsd.rd of the next > > release? > > Why put this into the installer and not into a script on the system? > With this, it takes an extra reboot, just to get bsd.rd. oh you want a script #!/bin/sh ftp -o /bsd.rd https://ftp.openbsd.org/pub/OpenBSD/6.3/amd64/bsd.rd but oh, signature check against what signature +1 is easy. more than +1 is difficult. The proposed diff is trying to solve 3 different problems. 1. you have booted into an old bsd.rd, and discover it is useless for doing the upgrade. an old bsd.rd does not know exactly how to install sets from a future release. 2. might as well use that bsd.rd to install newer bsd.rd, right? 3. but by default, it installs /bsd also, which could be a hazard in case the upgrade sequence gets aborted. that new /bsd may be ABI incompatible. so rpe's diff skips /bsd 4. if only going +1 release, it is possible to perform a signature check, more than that isn't easy. What is happening here doesn't block the writing of a script. I've seen talk of such a script, but I haven't seen one. Perhaps because >+1 isn't so easy.
Re: Installer support to fetch/verify bsd.rd for release upgrade
Robert Peichaer(rob...@peichaer.org) on 2017.10.08 09:56:15 +: > Up to now, the upgrade procedure from one to the next release meant > that you had to manually download and verify the new ramdisk kernel. > > What about if you just needed to boot into the existing bsd.rd and > it would support downloading and verifying the bsd.rd of the next > release? Why put this into the installer and not into a script on the system? With this, it takes an extra reboot, just to get bsd.rd. (In my case the number one mistake i make when upgrading is that i sleep through the boot> prompt timeout, so this gives me 2 chances ;)). Also, you would have to type Server directory? [pub/OpenBSD/6.2/amd64] pub/OpenBSD/6.3/amd64 by hand. Could this automatically be the next version? /B. > This diff changes the installer script to support such a scenario. > > 1) Boot the existing bsd.rd and choose (U)pgrade > 2) Enter the "Server directory" of the new release >The installer then offers just the bsd.rd >The on-disk signify key of the new release is used for verify it > 3) Reboot into the new bsd.rd and do the upgrade > > An important assumption for this to work properly is: > >Upgrades are only supported from one release to the release >immediately following it. [1] > > > It would look like this for the 6.2 to 6.3 upgrade situation. > (The version numbers are obviously faked) > > Let's upgrade the sets! > Location of sets? (cd0 disk http or 'done') [http] > HTTP proxy URL? (e.g. 'http://proxy:8080', or 'none') [none] > HTTP Server? (hostname, list#, 'done' or '?') [ftp.hostserver.de] > Server directory? [pub/OpenBSD/6.2/amd64] pub/OpenBSD/6.3/amd64 > Unable to get a verified list of distribution sets. > > Select sets by entering a set name, a file name pattern or 'all'. De-select > sets by prepending a '-', e.g.: '-game*'. Selected sets are labelled '[X]'. > [X] bsd.rd > Set name(s)? (or 'abort' or 'done') [done] > Get/Verify SHA256.sig 100% |**| 2152 00:00 > Signature Verified > Get/Verify bsd.rd 100% |**| 9565 KB00:14 > Installing bsd.rd 100% |**| 9565 KB00:00 > Location of sets? (cd0 disk http or 'done') [done] > Making all device nodes...done. > > CONGRATULATIONS! Your OpenBSD upgrade has been successfully completed! > To boot the new system, enter 'reboot' at the command prompt. > > > Here's the diff and below is a more detailed description. > > > Index: install.sub > === > RCS file: /cvs/src/distrib/miniroot/install.sub,v > retrieving revision 1.1036 > diff -u -p -p -u -r1.1036 install.sub > --- install.sub 4 Oct 2017 18:19:33 - 1.1036 > +++ install.sub 7 Oct 2017 14:02:19 - > @@ -1330,6 +1330,13 @@ sane_install() { > # > select_sets() { > local _avail=$1 _selected=$2 _f _action _col=$COLUMNS > + local _bsd_rd _no_sets=true > + > + [[ $MODE == upgrade ]] && for _f in $_avail; do > + [[ $_f != bsd* ]] && _no_sets=false > + [[ $_f == bsd.rd* ]] && _bsd_rd=$_f > + done > + $_no_sets && UPGRADE_BSDRD=true _avail=$_bsd_rd _selected=$_bsd_rd > > # account for 4 spaces added to the sets list > let COLUMNS=_col-8 > @@ -1517,6 +1524,8 @@ install_files() { > _issue="Cannot fetch SHA256.sig" && break > > # Verify signature file with public keys. > + $UPGRADE_BSDRD && > + PUB_KEY=/mnt/etc/signify/openbsd-$((VERSION + > 1))-base.pub > ! unpriv -f "$_cfile" \ > signify -Vep $PUB_KEY -x "$_cfile.sig" -m "$_cfile" && > _issue="Signature check of SHA256.sig failed" && break > @@ -1576,7 +1585,9 @@ install_files() { > tar -zxphf - -C /mnt > fi > ;; > - *) $_unpriv ftp -D Installing -Vmo - "$_fsrc" >"/mnt/$_f" > + *) $UPGRADE_BSDRD && [[ $_f == bsd.rd* ]] && > + cp /mnt/$_f /mnt/$_f.old.$VERSION > + $_unpriv ftp -D Installing -Vmo - "$_fsrc" >"/mnt/$_f" > ;; > esac > if (($?)); then > @@ -1587,6 +1598,7 @@ install_files() { > fi > else > DEFAULTSETS=$(rmel $_f $DEFAULTSETS) > + $UPGRADE_BSDRD && DEFAULTSETS= > fi > [[ -d $_tmpsrc ]] && rm -f "$_tmpsrc/$_f" > done > @@ -3139,6 +3151,7 @@ PUB_KEY=/etc/signify/openbsd-${VERSION}- > ROOTDEV= > ROOTDISK= > SETDIR="$VNAME/$ARCH" > +UPGRADE_BSDRD=false > V4_DHCPCONF=false > V6_AUTOCONF=false > WLANLIST=/tmp/i/wlanlist > === > Stats: --- 1 lines 60 chars > Stats:
Re: Installer support to fetch/verify bsd.rd for release upgrade
> Up to now, the upgrade procedure from one to the next release meant > that you had to manually download and verify the new ramdisk kernel. Well you already could follow that procedure. That's what I've been doing. The risk is that it also updates your bsd kernels, not just bsd.rd in case you run into a problem and change your mind. Your diff makes it explicit that it should only upgrade bsd.rd, so that the subsequent upgrade procedure can occur. > What about if you just needed to boot into the existing bsd.rd and > it would support downloading and verifying the bsd.rd of the next > release? > > This diff changes the installer script to support such a scenario. > > 1) Boot the existing bsd.rd and choose (U)pgrade > 2) Enter the "Server directory" of the new release >The installer then offers just the bsd.rd >The on-disk signify key of the new release is used for verify it > 3) Reboot into the new bsd.rd and do the upgrade > > > An important assumption for this to work properly is: > >Upgrades are only supported from one release to the release >immediately following it. [1] > > > It would look like this for the 6.2 to 6.3 upgrade situation. > (The version numbers are obviously faked) > > Let's upgrade the sets! > Location of sets? (cd0 disk http or 'done') [http] > HTTP proxy URL? (e.g. 'http://proxy:8080', or 'none') [none] > HTTP Server? (hostname, list#, 'done' or '?') [ftp.hostserver.de] > Server directory? [pub/OpenBSD/6.2/amd64] pub/OpenBSD/6.3/amd64 > Unable to get a verified list of distribution sets. > > Select sets by entering a set name, a file name pattern or 'all'. De-select > sets by prepending a '-', e.g.: '-game*'. Selected sets are labelled '[X]'. > [X] bsd.rd > Set name(s)? (or 'abort' or 'done') [done] > Get/Verify SHA256.sig 100% |**| 2152 00:00 > Signature Verified > Get/Verify bsd.rd 100% |**| 9565 KB00:14 > Installing bsd.rd 100% |**| 9565 KB00:00 > Location of sets? (cd0 disk http or 'done') [done] > Making all device nodes...done. > > CONGRATULATIONS! Your OpenBSD upgrade has been successfully completed! > To boot the new system, enter 'reboot' at the command prompt. > > > Here's the diff and below is a more detailed description. > > > Index: install.sub > === > RCS file: /cvs/src/distrib/miniroot/install.sub,v > retrieving revision 1.1036 > diff -u -p -p -u -r1.1036 install.sub > --- install.sub 4 Oct 2017 18:19:33 - 1.1036 > +++ install.sub 7 Oct 2017 14:02:19 - > @@ -1330,6 +1330,13 @@ sane_install() { > # > select_sets() { > local _avail=$1 _selected=$2 _f _action _col=$COLUMNS > + local _bsd_rd _no_sets=true > + > + [[ $MODE == upgrade ]] && for _f in $_avail; do > + [[ $_f != bsd* ]] && _no_sets=false > + [[ $_f == bsd.rd* ]] && _bsd_rd=$_f > + done > + $_no_sets && UPGRADE_BSDRD=true _avail=$_bsd_rd _selected=$_bsd_rd > > # account for 4 spaces added to the sets list > let COLUMNS=_col-8 > @@ -1517,6 +1524,8 @@ install_files() { > _issue="Cannot fetch SHA256.sig" && break > > # Verify signature file with public keys. > + $UPGRADE_BSDRD && > + PUB_KEY=/mnt/etc/signify/openbsd-$((VERSION + > 1))-base.pub > ! unpriv -f "$_cfile" \ > signify -Vep $PUB_KEY -x "$_cfile.sig" -m "$_cfile" && > _issue="Signature check of SHA256.sig failed" && break > @@ -1576,7 +1585,9 @@ install_files() { > tar -zxphf - -C /mnt > fi > ;; > - *) $_unpriv ftp -D Installing -Vmo - "$_fsrc" >"/mnt/$_f" > + *) $UPGRADE_BSDRD && [[ $_f == bsd.rd* ]] && > + cp /mnt/$_f /mnt/$_f.old.$VERSION > + $_unpriv ftp -D Installing -Vmo - "$_fsrc" >"/mnt/$_f" > ;; > esac > if (($?)); then > @@ -1587,6 +1598,7 @@ install_files() { > fi > else > DEFAULTSETS=$(rmel $_f $DEFAULTSETS) > + $UPGRADE_BSDRD && DEFAULTSETS= > fi > [[ -d $_tmpsrc ]] && rm -f "$_tmpsrc/$_f" > done > @@ -3139,6 +3151,7 @@ PUB_KEY=/etc/signify/openbsd-${VERSION}- > ROOTDEV= > ROOTDISK= > SETDIR="$VNAME/$ARCH" > +UPGRADE_BSDRD=false > V4_DHCPCONF=false > V6_AUTOCONF=false > WLANLIST=/tmp/i/wlanlist > === > Stats: --- 1 lines 60 chars > Stats: +++ 14 lines 508 chars > Stats: 13 lines > Stats: 448 chars > > > The installer downloads the new SHA256.sig from the location of the > new release and extracts the list
Installer support to fetch/verify bsd.rd for release upgrade
Up to now, the upgrade procedure from one to the next release meant that you had to manually download and verify the new ramdisk kernel. What about if you just needed to boot into the existing bsd.rd and it would support downloading and verifying the bsd.rd of the next release? This diff changes the installer script to support such a scenario. 1) Boot the existing bsd.rd and choose (U)pgrade 2) Enter the "Server directory" of the new release The installer then offers just the bsd.rd The on-disk signify key of the new release is used for verify it 3) Reboot into the new bsd.rd and do the upgrade An important assumption for this to work properly is: Upgrades are only supported from one release to the release immediately following it. [1] It would look like this for the 6.2 to 6.3 upgrade situation. (The version numbers are obviously faked) Let's upgrade the sets! Location of sets? (cd0 disk http or 'done') [http] HTTP proxy URL? (e.g. 'http://proxy:8080', or 'none') [none] HTTP Server? (hostname, list#, 'done' or '?') [ftp.hostserver.de] Server directory? [pub/OpenBSD/6.2/amd64] pub/OpenBSD/6.3/amd64 Unable to get a verified list of distribution sets. Select sets by entering a set name, a file name pattern or 'all'. De-select sets by prepending a '-', e.g.: '-game*'. Selected sets are labelled '[X]'. [X] bsd.rd Set name(s)? (or 'abort' or 'done') [done] Get/Verify SHA256.sig 100% |**| 2152 00:00 Signature Verified Get/Verify bsd.rd 100% |**| 9565 KB00:14 Installing bsd.rd 100% |**| 9565 KB00:00 Location of sets? (cd0 disk http or 'done') [done] Making all device nodes...done. CONGRATULATIONS! Your OpenBSD upgrade has been successfully completed! To boot the new system, enter 'reboot' at the command prompt. Here's the diff and below is a more detailed description. Index: install.sub === RCS file: /cvs/src/distrib/miniroot/install.sub,v retrieving revision 1.1036 diff -u -p -p -u -r1.1036 install.sub --- install.sub 4 Oct 2017 18:19:33 - 1.1036 +++ install.sub 7 Oct 2017 14:02:19 - @@ -1330,6 +1330,13 @@ sane_install() { # select_sets() { local _avail=$1 _selected=$2 _f _action _col=$COLUMNS + local _bsd_rd _no_sets=true + + [[ $MODE == upgrade ]] && for _f in $_avail; do + [[ $_f != bsd* ]] && _no_sets=false + [[ $_f == bsd.rd* ]] && _bsd_rd=$_f + done + $_no_sets && UPGRADE_BSDRD=true _avail=$_bsd_rd _selected=$_bsd_rd # account for 4 spaces added to the sets list let COLUMNS=_col-8 @@ -1517,6 +1524,8 @@ install_files() { _issue="Cannot fetch SHA256.sig" && break # Verify signature file with public keys. + $UPGRADE_BSDRD && + PUB_KEY=/mnt/etc/signify/openbsd-$((VERSION + 1))-base.pub ! unpriv -f "$_cfile" \ signify -Vep $PUB_KEY -x "$_cfile.sig" -m "$_cfile" && _issue="Signature check of SHA256.sig failed" && break @@ -1576,7 +1585,9 @@ install_files() { tar -zxphf - -C /mnt fi ;; - *) $_unpriv ftp -D Installing -Vmo - "$_fsrc" >"/mnt/$_f" + *) $UPGRADE_BSDRD && [[ $_f == bsd.rd* ]] && + cp /mnt/$_f /mnt/$_f.old.$VERSION + $_unpriv ftp -D Installing -Vmo - "$_fsrc" >"/mnt/$_f" ;; esac if (($?)); then @@ -1587,6 +1598,7 @@ install_files() { fi else DEFAULTSETS=$(rmel $_f $DEFAULTSETS) + $UPGRADE_BSDRD && DEFAULTSETS= fi [[ -d $_tmpsrc ]] && rm -f "$_tmpsrc/$_f" done @@ -3139,6 +3151,7 @@ PUB_KEY=/etc/signify/openbsd-${VERSION}- ROOTDEV= ROOTDISK= SETDIR="$VNAME/$ARCH" +UPGRADE_BSDRD=false V4_DHCPCONF=false V6_AUTOCONF=false WLANLIST=/tmp/i/wlanlist === Stats: --- 1 lines 60 chars Stats: +++ 14 lines 508 chars Stats: 13 lines Stats: 448 chars The installer downloads the new SHA256.sig from the location of the new release and extracts the list of files. It then prepares the list for the selection step. At this point all the set files containing the new release number are skipped, because they don't match the version of the current (old) bsd.rd leaving only the kernels. Right before the set selection step, the installer looks at the list of files and if there are only kernels, it assumes to be in this "upgrade only the bsd.rd" scenario. It then sets the list to the bsd.rd kernel and sets the global UPGRADE_BSDRD variable to