Re: Vague proposal: ship prebuilt initramfs images
Hi, On 22-01-2020 04:20, Matthew Garrett wrote: On Tue, Jan 21, 2020 at 12:43:47PM +, Matthew Garrett wrote: configinitrd file1 file2 file3 initrd initramfs1.img initramfs2.img CONFIG Huh - it seems like grub may already support this? It looks like: initrd initramfs.img newc:/etc/crypttab:/boot/crypttab will add /boot/crypttab to the initramfs as /etc/crypttab. Haven't tested this yet, though, and it's entirely undocumented. Oh, that is quite interesting, then just copying /etc/vconsole.conf to /boot and using the above might be enough for the keymap issue. Regards, Hans ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Vague proposal: ship prebuilt initramfs images
On Tue, Jan 21, 2020 at 12:43:47PM +, Matthew Garrett wrote: > configinitrd file1 file2 file3 > initrd initramfs1.img initramfs2.img CONFIG Huh - it seems like grub may already support this? It looks like: initrd initramfs.img newc:/etc/crypttab:/boot/crypttab will add /boot/crypttab to the initramfs as /etc/crypttab. Haven't tested this yet, though, and it's entirely undocumented. -- Matthew Garrett | mj...@srcf.ucam.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Vague proposal: ship prebuilt initramfs images
On Tue, Jan 21, 2020 at 12:57:50AM +, Matthew Garrett wrote: > a) Stick the config in UEFI variables. It's small enough that we > wouldn't run out. > b) Extend grub to read some config files and synthesise an initramfs > image for them. If we measure the paths that those images use then > we don't need to worry about the contents as long as the tools that > read the config can't be subverted via that configuration. I think a problem with (a) is that it won't work for disk images if part of it is tied to the hardware, everything needs to be on the image. So (b) has my vote for now. -- Brian C. Lane (PST8PDT) - weldr.io - lorax - parted - pykickstart ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Vague proposal: ship prebuilt initramfs images
On Tue, Jan 21, 2020 at 11:25:29AM +0100, Hans de Goede wrote: > For the size problem one approach which I think is worthwhile at least > for the x86/workstation case is making one initrd with the most common > stuff in there (say i915,nouveau and amd GPU drivers + NVME and AHCI > disk support, maybe some eMMC support) and another which contains > "everything" and then rework the dracut host-only code so that we can > leverage it to figure out which variant we want. Yeah, the important thing is that we have known values - there's no real problem with there being a large number of known values, so shipping multiple images (potentially to be combined if necessary) is entirely reasonable from that perspective. > So b it is, I actually suggested b. on a discussion about how to set > the keyboard map for the initrd (for full disk encryption) which currently > is a problem for rpm-ostree / SilverBlue which uses pre-generated > initrds with no custom config in there. Ok. I'll look into adding support to grub for reading some files and assembling them into an image. My rough idea here would be something like: configinitrd file1 file2 file3 initrd initramfs1.img initramfs2.img CONFIG where the first command generates the image and the second command causes it to be placed at the end of the final cpio blob? -- Matthew Garrett | mj...@srcf.ucam.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Vague proposal: ship prebuilt initramfs images
On Tue, Jan 21, 2020 at 09:09:16AM +0100, Petr Pisar wrote: > On Tue, Jan 21, 2020 at 12:57:50AM +, Matthew Garrett wrote: > > Any thoughts on this? > > > Properly measured system must measure all inputs. If you move the varying > bits from initramfs to another file, a boot loader will have to measure that > another file. At the end that's exactly what GRUB2 does. It measures any > loaded file. Yes, I wrote that code. The point of measurements is to be able to make a policy determination. If the contents of a file aren't security relevant then you don't care about its contents, but you do want to ensure that it ends up in a position where it can't interfere with any other security relevant codepath. In that scenario you want to measure the path, not the contents (or, rather, you can measure both and the policy agent can ignore the contents) > In my opinon, your proposal does not solve the problem. It actually makes > things worse because the booted code would become bigger and probably slower. I'm not clear how that follows. -- Matthew Garrett | mj...@srcf.ucam.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Vague proposal: ship prebuilt initramfs images
On Tue, Jan 21, 2020 at 12:29:13AM -0700, Chris Murphy wrote: > What about the on-going cost: downloading ~80M initramfs for each kernel > update; systemd-analyze on NVMe says initrd time is 2.5s for a host-only > ~25M initramfs. No-host-only initramfs is about 3x bigger. If the size to > time relationship is linear, that's a chunk of extra time. Maybe there's a > way to improve the read performance in the bootloader to compensate? I don't see this as an obligatory choice - users should still be free to generate images locally instead. I'm certainly willing to do the work on performance calculations if there's no absolute objection to the idea. > Any expected hardware with TPM2 but without UEFI? Not on x86 - the PC client spec for TPM2 only covers UEFI. > If the first initramfs contains systemd, could systemd start things in > parallel while unpacking a second initramfs? The straightforward implementation involves the kernel unpacking all the initramfs archives before it starts init. In theory we could add functionality to the kernel to expose additional archives to userland rather than have the kernel unpack them, but that's not currently achievable - and it's one of those situations where we'd need to be very careful about ensuring there's no potential for races. > I take it you've found some liability with measuring a locally produced > initramfs? You'd need a trusted mechanism for passing the new initramfs measurements to whatever's verifying the measurements. That's not easy. -- Matthew Garrett | mj...@srcf.ucam.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Vague proposal: ship prebuilt initramfs images
Hi, On 21-01-2020 01:57, Matthew Garrett wrote: Measured boot involves generating cryptographic measurements of boot components and configuration and using that to either control access to a local secret (in the case of sealing secrets to a TPM) or proving to another device (eg, a remote server or a local phone) what was booted. We're shipping most of the infrastructure to do this, but we're still left with a pretty fundamental problem - we need to know what the expected values are in order to know whether something's been tampered with or not. For many components this isn't a huge problem (we build and distribute the files - users can extract them and calculate the appropriate measurements, and maybe long term we'll be able to ship the measurements in a digestable way), but our initramfs images are generated on the user system and include system-specific data. This makes it impractical to know the expected measurements in advance. I've been thinking about ways to solve this for a while, and I'm coming to the conclusion that the best plan is probably to just ship pre-built initramfs images. I can think of three main reasons to want to use system-specific images: 1) They're smaller. By default we're already generating a generic image for rescue purposes, so disk space isn't the concern here - we're largely looking at losing boot speed. As machines have got faster this is probably not a huge deal. Using pre-generated initrds is something which comes up from time to time and I do believe it is something which we want to move towards. For the size problem one approach which I think is worthwhile at least for the x86/workstation case is making one initrd with the most common stuff in there (say i915,nouveau and amd GPU drivers + NVME and AHCI disk support, maybe some eMMC support) and another which contains "everything" and then rework the dracut host-only code so that we can leverage it to figure out which variant we want. 2) They contain machine-specific configuration. Some of this can be passed on the kernel command line instead (eg, the machine ID), but we'd need answers for the rest. I can think of a couple of solutions: a) Stick the config in UEFI variables. It's small enough that we wouldn't run out. b) Extend grub to read some config files and synthesise an initramfs image for them. If we measure the paths that those images use then we don't need to worry about the contents as long as the tools that read the config can't be subverted via that configuration. We still have quite a lot of users using classic BIOS boot (and early x86_64 hw does not do UEFI at all) so a) is not really an option as I would like to see a common approach here for both UEFI + classic BIOS, moreover we also have other architectures and if we change this we should really change it everywhere rather then have multiple code paths to support. So b it is, I actually suggested b. on a discussion about how to set the keyboard map for the initrd (for full disk encryption) which currently is a problem for rpm-ostree / SilverBlue which uses pre-generated initrds with no custom config in there. 3) User customisation, such as including extra tooling. grub supports loading multiple initramfs images. Packages that right now install stuff in the initramfs could instead ship a prebuilt image that grub could append to the main initramfs. This would allow for things like overriding Plymouth themes, and we could ship the measurements for these pre-built images in order to allow them to be validated. Ack for using appended initrds for this, this also mixes well with my preferred solution for 2. Regards, Hans ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Vague proposal: ship prebuilt initramfs images
On Tue, Jan 21, 2020 at 12:57:50AM +, Matthew Garrett wrote: > 2) They contain machine-specific configuration. Some of this can be > passed on the kernel command line instead (eg, the machine ID), but we'd > need answers for the rest. I can think of a couple of solutions: > > a) Stick the config in UEFI variables. It's small enough that we > wouldn't run out. > b) Extend grub to read some config files and synthesise an initramfs > image for them. If we measure the paths that those images use then > we don't need to worry about the contents as long as the tools that > read the config can't be subverted via that configuration. > > 3) User customisation, such as including extra tooling. grub supports > loading multiple initramfs images. Packages that right now install stuff > in the initramfs could instead ship a prebuilt image that grub could > append to the main initramfs. This would allow for things like > overriding Plymouth themes, and we could ship the measurements for these > pre-built images in order to allow them to be validated. > > Any thoughts on this? > Properly measured system must measure all inputs. If you move the varying bits from initramfs to another file, a boot loader will have to measure that another file. At the end that's exactly what GRUB2 does. It measures any loaded file. In my opinon, your proposal does not solve the problem. It actually makes things worse because the booted code would become bigger and probably slower. -- Petr signature.asc Description: PGP signature ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Vague proposal: ship prebuilt initramfs images
On Mon, Jan 20, 2020 at 5:58 PM Matthew Garrett wrote: > > I've been thinking about ways to solve this for a while, and I'm coming > to the conclusion that the best plan is probably to just ship pre-built > initramfs images. I can think of three main reasons to want to use > system-specific images: > > 1) They're smaller. By default we're already generating a generic image > for rescue purposes, so disk space isn't the concern here - we're > largely looking at losing boot speed. As machines have got faster this > is probably not a huge deal. What about the on-going cost: downloading ~80M initramfs for each kernel update; systemd-analyze on NVMe says initrd time is 2.5s for a host-only ~25M initramfs. No-host-only initramfs is about 3x bigger. If the size to time relationship is linear, that's a chunk of extra time. Maybe there's a way to improve the read performance in the bootloader to compensate? Some of the kernel and initramfs time hit comes from xz decompression. I wonder if zstd can also compensate? Getting zstd module built into the kernel to support zstd compressed initramfs is straightforward, but I vaguely recall something extra is needed to support zstd compressed kernels. > > 2) They contain machine-specific configuration. Some of this can be > passed on the kernel command line instead (eg, the machine ID), but we'd > need answers for the rest. I can think of a couple of solutions: > > a) Stick the config in UEFI variables. It's small enough that we > wouldn't run out. > b) Extend grub to read some config files and synthesise an initramfs > image for them. If we measure the paths that those images use then > we don't need to worry about the contents as long as the tools that > read the config can't be subverted via that configuration. Any expected hardware with TPM2 but without UEFI? There's a systemd facility for extra boot params to contend with the arch specific COMMAND_LINE_SIZE limit. I forget what it is, maybe it uses efivars. > > 3) User customisation, such as including extra tooling. grub supports > loading multiple initramfs images. Packages that right now install stuff > in the initramfs could instead ship a prebuilt image that grub could > append to the main initramfs. This would allow for things like > overriding Plymouth themes, and we could ship the measurements for these > pre-built images in order to allow them to be validated. If the first initramfs contains systemd, could systemd start things in parallel while unpacking a second initramfs? I take it you've found some liability with measuring a locally produced initramfs? -- Chris Murphy ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Vague proposal: ship prebuilt initramfs images
On 1/20/20 7:57 PM, Matthew Garrett wrote: > > I've been thinking about ways to solve this for a while, and I'm coming > to the conclusion that the best plan is probably to just ship pre-built > initramfs images. I can think of three main reasons to want to use > system-specific images: For what it's worth, this is what is done for OSTree based systems (Fedora Atomic Host, Fedora CoreOS, Fedora Silverblue). There is an option for the user to generate an initramfs client side, but I don't think it's used very much at all. I know in the case of OSTree based systems you are shipping more of an image than a case like Fedora Server where you are assembling a new system from a set of defined RPMs, but it's at least a data point. Dusty ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Re: Vague proposal: ship prebuilt initramfs images
On Mon, Jan 20, 2020, at 7:57 PM, Matthew Garrett wrote: > I've been thinking about ways to solve this for a while, and I'm coming > to the conclusion that the best plan is probably to just ship pre-built > initramfs images. rpm-ostree[1] (used for Fedora CoreOS and Silverblue and IoT among others) already works this way by default. (You can also enable client-side initramfs regeneration, which is used e.g. when overlaying a custom kernel - part of being a hybrid image/package system) For FCOS, signing the initramfs is already part of the plans; see: https://github.com/projectatomic/rpm-ostree/issues/1883 and https://github.com/ostreedev/ostree/pull/1959 [1] https://github.com/coreos/rpm-ostree/ ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Vague proposal: ship prebuilt initramfs images
Measured boot involves generating cryptographic measurements of boot components and configuration and using that to either control access to a local secret (in the case of sealing secrets to a TPM) or proving to another device (eg, a remote server or a local phone) what was booted. We're shipping most of the infrastructure to do this, but we're still left with a pretty fundamental problem - we need to know what the expected values are in order to know whether something's been tampered with or not. For many components this isn't a huge problem (we build and distribute the files - users can extract them and calculate the appropriate measurements, and maybe long term we'll be able to ship the measurements in a digestable way), but our initramfs images are generated on the user system and include system-specific data. This makes it impractical to know the expected measurements in advance. I've been thinking about ways to solve this for a while, and I'm coming to the conclusion that the best plan is probably to just ship pre-built initramfs images. I can think of three main reasons to want to use system-specific images: 1) They're smaller. By default we're already generating a generic image for rescue purposes, so disk space isn't the concern here - we're largely looking at losing boot speed. As machines have got faster this is probably not a huge deal. 2) They contain machine-specific configuration. Some of this can be passed on the kernel command line instead (eg, the machine ID), but we'd need answers for the rest. I can think of a couple of solutions: a) Stick the config in UEFI variables. It's small enough that we wouldn't run out. b) Extend grub to read some config files and synthesise an initramfs image for them. If we measure the paths that those images use then we don't need to worry about the contents as long as the tools that read the config can't be subverted via that configuration. 3) User customisation, such as including extra tooling. grub supports loading multiple initramfs images. Packages that right now install stuff in the initramfs could instead ship a prebuilt image that grub could append to the main initramfs. This would allow for things like overriding Plymouth themes, and we could ship the measurements for these pre-built images in order to allow them to be validated. Any thoughts on this? -- Matthew Garrett | mj...@srcf.ucam.org ___ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
