[systemd-devel] Service Management and systemd Microconference @ Linux Plumbers
I'll be hosting this microconference at Linux Plumbers Conference 2022! You can submit talks to this microconference through https://lpc.events/event/16/abstracts/ and select the corresponding track. LPC is happening Sept 12-14. It will be hybrid this year and presenters are encouraged to come in-person to Dublin, Ireland. Abstract copied here: *The focus of this microconference will be on topics related to the current state of host-level service management and ideas for the future.As systemd is one of the most widely adopted service managers in Linux distributions today, we expect that most of the contributions will be based around the systemd ecosystem. We're also welcome to proposals that are not specific to systemd so we can discover and share new ideas on how to improve service management in general.We expect submissions to be either open discussions or presentations that discuss new proposals/ideas, ongoing work, or problems we are/should be solving in this space. Submissions are recommended to be 15 - 45 minutes long. Please specify the format, the desired length of your submission, and how much, if any, additional time to allocate for discussion in your abstracts.* Feel free to reach out if you have any questions. - Anita Zhang
Re: [systemd-devel] Splitting sd-boot from systemd/bootctl for enabling sd-boot in Fedora
On Wed, 2022-04-27 at 13:31 -0400, Neal Gompa wrote: > On Wed, Apr 27, 2022 at 1:07 PM Neal Gompa wrote: > > > > On Wed, Apr 27, 2022 at 12:19 PM Zbigniew Jędrzejewski-Szmek > > wrote: > > > > > > On Wed, Apr 27, 2022 at 11:40:36AM -0400, Neal Gompa wrote: > > > > On Wed, Apr 27, 2022 at 11:26 AM Dan Nicholson > > > > wrote: > > > > > > > > > > On Wed, Apr 27, 2022 at 9:10 AM Neal Gompa wrote: > > > > > > > > > > > > Note that it means Fedora CI, pull requests from contributors, and > > > > > > releng auto-rebuilds will no longer work. Maintainers basically > > > > > > sign-on to do all of those things manually and have to be responsive > > > > > > for doing it. You will get FTBFS tickets every cycle because of it, > > > > > > for example. > > > > > > > > > > Asking systemd folks to change their development process because of > > > > > limitations in Fedora/Koji seems like a big ask, don't you think? > > > > > > > > Honestly, that never factors in for me, because then I'd never ask > > > > anything of anyone. :) > > > > > > > > But no, it doesn't sound unreasonable to me, because I reasonably > > > > expect the parts that make up the EFI boot manager code to be somewhat > > > > separate from the rest of the codebase in the first place. It targets > > > > a different build environment and has to be built differently from the > > > > rest of systemd anyway. So to me, it's not a "big ask" as you put it. > > > > > > Once more (this has already been written *independently* by Lennart > > > and me, but let's say it once again): THE USERSPACE AND EFI-SPACE > > > PARTS SHARE CODE, the configuration system and the build system. So if > > > we split them, we'd probably want to update them in tandem anyway, and > > > it's just easier to do it, then to try to figure out if the sd-boot parts > > > didn't change and we can skip the update. > > > > > > > Yes, well he was asking me why I bothered to ask, and I answered. That > > doesn't have a bearing on what reality turned out to be. > > > > > Since the signing is automatic once configured, I can't grok what > > > savings people foresee by doing separate builds. > > > > > > > That's the point: correct signing is *not* automatic. It's *manual* > > and done by *you* doing the build. Automatic signing is independent of > > who does it, this is not that. > > > > Another option that preserves the single source tree thing and > probably will make Lennart happy is splitting this process in two: > 1. systemd keeps building systemd-boot, but it gets split out as a > subpackage (systemd-boot-unsigned-%efi_arch as noarch) > 2. A new systemd-boot-signed source package BRs all > systemd-boot-unsigned-$EFIARCH packages and signs them. It then > produces "systemd-boot-$EFIARCH" subpackages that are signed that > people can use. > > That second package gets specifically marked to not get autobuilt, > doesn't have a disttag, and basically goes through the entire > exception path that shim uses today. > > I think this matches what Michael Biebl was talking about for Debian > that died on the vine. Yes, this is how the EFI signing process was implemented for all relevant Debian packages (not just for the sd-boot PoC), in order to work with the, er, clunky infrastructure we have. More details can be found here: https://wiki.debian.org/SecureBoot/Discussion -- Kind regards, Luca Boccassi signature.asc Description: This is a digitally signed message part
[systemd-devel] Waiting for all jobs to finish
I have two target files that I use. prod.target is used to start up all the production services. When I use systemctl start prod.target it blocks until all the services are running. I also have prod-upgrade.target that is used to stop service via Conflicts=. When I use systemctl start prod-upgrade.target it returns immediately but there are stop jobs running. I believe that this is expected as all the services that need to be started have been. Is there a systemctl command that will wait for all the stop jobs without the need to poll for the systemctl list-jobs to be empty? The pattern I'm implementing is to do this on a server upgrade: systemctl start prod-upgrade.target # need to wait for stop jobs to finish dnf update --refresh -y systemctl start prod.target Barry
Re: [systemd-devel] Splitting sd-boot from systemd/bootctl for enabling sd-boot in Fedora
On Wed, Apr 27, 2022 at 1:07 PM Neal Gompa wrote: > > On Wed, Apr 27, 2022 at 12:19 PM Zbigniew Jędrzejewski-Szmek > wrote: > > > > On Wed, Apr 27, 2022 at 11:40:36AM -0400, Neal Gompa wrote: > > > On Wed, Apr 27, 2022 at 11:26 AM Dan Nicholson wrote: > > > > > > > > On Wed, Apr 27, 2022 at 9:10 AM Neal Gompa wrote: > > > > > > > > > > Note that it means Fedora CI, pull requests from contributors, and > > > > > releng auto-rebuilds will no longer work. Maintainers basically > > > > > sign-on to do all of those things manually and have to be responsive > > > > > for doing it. You will get FTBFS tickets every cycle because of it, > > > > > for example. > > > > > > > > Asking systemd folks to change their development process because of > > > > limitations in Fedora/Koji seems like a big ask, don't you think? > > > > > > Honestly, that never factors in for me, because then I'd never ask > > > anything of anyone. :) > > > > > > But no, it doesn't sound unreasonable to me, because I reasonably > > > expect the parts that make up the EFI boot manager code to be somewhat > > > separate from the rest of the codebase in the first place. It targets > > > a different build environment and has to be built differently from the > > > rest of systemd anyway. So to me, it's not a "big ask" as you put it. > > > > Once more (this has already been written *independently* by Lennart > > and me, but let's say it once again): THE USERSPACE AND EFI-SPACE > > PARTS SHARE CODE, the configuration system and the build system. So if > > we split them, we'd probably want to update them in tandem anyway, and > > it's just easier to do it, then to try to figure out if the sd-boot parts > > didn't change and we can skip the update. > > > > Yes, well he was asking me why I bothered to ask, and I answered. That > doesn't have a bearing on what reality turned out to be. > > > Since the signing is automatic once configured, I can't grok what > > savings people foresee by doing separate builds. > > > > That's the point: correct signing is *not* automatic. It's *manual* > and done by *you* doing the build. Automatic signing is independent of > who does it, this is not that. > Another option that preserves the single source tree thing and probably will make Lennart happy is splitting this process in two: 1. systemd keeps building systemd-boot, but it gets split out as a subpackage (systemd-boot-unsigned-%efi_arch as noarch) 2. A new systemd-boot-signed source package BRs all systemd-boot-unsigned-$EFIARCH packages and signs them. It then produces "systemd-boot-$EFIARCH" subpackages that are signed that people can use. That second package gets specifically marked to not get autobuilt, doesn't have a disttag, and basically goes through the entire exception path that shim uses today. I think this matches what Michael Biebl was talking about for Debian that died on the vine. -- 真実はいつも一つ！/ Always, there's only one truth!
Re: [systemd-devel] Splitting sd-boot from systemd/bootctl for enabling sd-boot in Fedora
On Wed, Apr 27, 2022 at 12:19 PM Zbigniew Jędrzejewski-Szmek wrote: > > On Wed, Apr 27, 2022 at 11:40:36AM -0400, Neal Gompa wrote: > > On Wed, Apr 27, 2022 at 11:26 AM Dan Nicholson wrote: > > > > > > On Wed, Apr 27, 2022 at 9:10 AM Neal Gompa wrote: > > > > > > > > Note that it means Fedora CI, pull requests from contributors, and > > > > releng auto-rebuilds will no longer work. Maintainers basically > > > > sign-on to do all of those things manually and have to be responsive > > > > for doing it. You will get FTBFS tickets every cycle because of it, > > > > for example. > > > > > > Asking systemd folks to change their development process because of > > > limitations in Fedora/Koji seems like a big ask, don't you think? > > > > Honestly, that never factors in for me, because then I'd never ask > > anything of anyone. :) > > > > But no, it doesn't sound unreasonable to me, because I reasonably > > expect the parts that make up the EFI boot manager code to be somewhat > > separate from the rest of the codebase in the first place. It targets > > a different build environment and has to be built differently from the > > rest of systemd anyway. So to me, it's not a "big ask" as you put it. > > Once more (this has already been written *independently* by Lennart > and me, but let's say it once again): THE USERSPACE AND EFI-SPACE > PARTS SHARE CODE, the configuration system and the build system. So if > we split them, we'd probably want to update them in tandem anyway, and > it's just easier to do it, then to try to figure out if the sd-boot parts > didn't change and we can skip the update. > Yes, well he was asking me why I bothered to ask, and I answered. That doesn't have a bearing on what reality turned out to be. > Since the signing is automatic once configured, I can't grok what > savings people foresee by doing separate builds. > That's the point: correct signing is *not* automatic. It's *manual* and done by *you* doing the build. Automatic signing is independent of who does it, this is not that. -- 真実はいつも一つ！/ Always, there's only one truth!
Re: [systemd-devel] Splitting sd-boot from systemd/bootctl for enabling sd-boot in Fedora
Am Mi., 27. Apr. 2022 um 18:02 Uhr schrieb Michael Biebl : > > Am Mi., 27. Apr. 2022 um 17:16 Uhr schrieb Dan Nicholson : > > > > On Wed, Apr 27, 2022 at 9:01 AM Michael Biebl wrote: > > > > > > Slightly related > > > https://salsa.debian.org/systemd-team/systemd/-/merge_requests/138 > > > [sd-boot split] > > > https://salsa.debian.org/systemd-team/systemd/-/merge_requests/132 > > > [Draft: Prepare for EFI signing] > > > > Oh, nice. We've been signing sd-boot in Endless for a couple years now > > with our systemd package based on Debian's. Currently the entire > > systemd package is sent through the signing flow just for sd-boot. > > When sd-boot is a separate package that can be much simpler with the > > normal non-sd-boot targets unaffected. > > > This discussion might be relevant to you then > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=996202 > > Automatically signing sd-boot in Debian was rejected by Julian Andres Klode >From https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=996202#60 """ we have recently discussed the matter of systemd-boot in an upstream shim review gathering. We reject a signing of systemd-boot as * systemd-boot does not use current ways of communicating with shim * There was some concern over general quality * systemd-boot is an additional bootloader, rather than replacing an existing one, thus increasing the attack surface. If people want to experiment with other bootloaders than the default one, they can disable secure boot, or load their own keys into the machine. We do not consider it valid to have a choice of bootloaders. I want to note that the current shim has been signed with the understanding that it will trust grub, kernels, and fwupd. A signing of systemd-boot might be considered reasons for revoking the existing shim, and will certainly result in new shims not getting signed. """
Re: [systemd-devel] Splitting sd-boot from systemd/bootctl for enabling sd-boot in Fedora
On Mi, 27.04.22 11:40, Neal Gompa (ngomp...@gmail.com) wrote: > But I also strongly believe that it was a mistake to merge gummiboot > into the systemd source tree because it creates all kinds of problems > for distro maintenance, as I've already said earlier. That's very vague. I still don#t get what those problems are specifically supposed to be. Why can't you just have two srpm for the same uptsream tarball? You keep suggesting things were all so obviously untenable, but I never see an explanation why? > Ita also discourages making the sd-boot code tighter and simpler > since your primary focus is reuse across the tree rather than > strictly separating the functional domains. Dunno, as one of the developers of sd-boot I think this is a pretty comprehensively bogus statement. Sharing code means less code, duplicating code in multiple projects means more code. That should be obvious. Moreover, sd-boot is kinda simplistic. I did some sloccount analysis for you: $ sloccount src/fundamental/ src/boot/efi/ SLOCDirectory SLOC-by-Language (Sorted) 6117efi ansic=6117 797 fundamental ansic=797 i.e. that's the code linked into the uefi stuff. In fact, given it also includes sd-stub it's a lot more than actually sd-boot. And now compare with the supposedly simple "shim": SLOCDirectory SLOC-by-Language (Sorted) 127120 Cryptlibansic=126650,sh=470 15633 top_dir ansic=15019,sh=564,asm=50 3756include ansic=3756 2024lib ansic=2024 Even if we ignore that it imports "Cryptlib", that's still like 3x as much code... btw, grub is 283895 lines of code according to sloccount, just for comparison... So thank you very much, but I think we are good regarding simplicity... I'd rather share more code with userspace, and thus have less stuff to think about, get better testing and so on... Lennart -- Lennart Poettering, Berlin
Re: [systemd-devel] Splitting sd-boot from systemd/bootctl for enabling sd-boot in Fedora
On Wed, Apr 27, 2022 at 11:40:36AM -0400, Neal Gompa wrote: > On Wed, Apr 27, 2022 at 11:26 AM Dan Nicholson wrote: > > > > On Wed, Apr 27, 2022 at 9:10 AM Neal Gompa wrote: > > > > > > Note that it means Fedora CI, pull requests from contributors, and > > > releng auto-rebuilds will no longer work. Maintainers basically > > > sign-on to do all of those things manually and have to be responsive > > > for doing it. You will get FTBFS tickets every cycle because of it, > > > for example. > > > > Asking systemd folks to change their development process because of > > limitations in Fedora/Koji seems like a big ask, don't you think? > > Honestly, that never factors in for me, because then I'd never ask > anything of anyone. :) > > But no, it doesn't sound unreasonable to me, because I reasonably > expect the parts that make up the EFI boot manager code to be somewhat > separate from the rest of the codebase in the first place. It targets > a different build environment and has to be built differently from the > rest of systemd anyway. So to me, it's not a "big ask" as you put it. Once more (this has already been written *independently* by Lennart and me, but let's say it once again): THE USERSPACE AND EFI-SPACE PARTS SHARE CODE, the configuration system and the build system. So if we split them, we'd probably want to update them in tandem anyway, and it's just easier to do it, then to try to figure out if the sd-boot parts didn't change and we can skip the update. Since the signing is automatic once configured, I can't grok what savings people foresee by doing separate builds. > But I also strongly believe that it was a mistake to merge gummiboot > into the systemd source tree because it creates all kinds of problems > for distro maintenance, as I've already said earlier. It also > discourages making the sd-boot code tighter and simpler since your > primary focus is reuse across the tree rather than strictly separating > the functional domains. There are downsides, but there are also benefits: 1. code sharing (it's not a lot, but even some is better than nothing, and the amount is growing). 2. when implementing features that have a "producer side" and a "consumer side", e.g. as with the random-seed integration in the bootloader and pid1 and bootctl, it's *much* easier if all parts of the code can be implemented and viewed and tested together. Splitting things into separate projects adds overhead. This overhead is not too great, so it's always a question of balance, but so far the integration of sd-boot with systemds has generally made the the development of both sides easier. Also, with any boot menu stuff, we want to have functional equivalency for sd-boot and bootctl, and that also is much easier if it's just one repo, even if the implementations are separate. 3. one project is less work than two projects. Zbyszek
Re: [systemd-devel] Splitting sd-boot from systemd/bootctl for enabling sd-boot in Fedora
On Wed, Apr 27, 2022 at 11:59 AM Luca Boccassi wrote: > > On Wed, 2022-04-27 at 11:48 -0400, Neal Gompa wrote: > > On Wed, Apr 27, 2022 at 11:46 AM Luca Boccassi wrote: > > > > > > On Wed, 2022-04-27 at 11:26 -0400, Neal Gompa wrote: > > > > On Wed, Apr 27, 2022 at 11:13 AM Zbigniew Jędrzejewski-Szmek > > > > wrote: > > > > > > > > > > On Wed, Apr 27, 2022 at 09:31:10AM -0400, Neal Gompa wrote: > > > > > > On Wed, Apr 27, 2022 at 7:41 AM Lennart Poettering > > > > > > > > Realistically, I think if we want to make movement on making > > > > > > > > systemd-boot fully supported in Fedora, the systemd-boot boot > > > > > > > > manager > > > > > > > > code itself should be split out into its own repository with > > > > > > > > its own > > > > > > > > release cadence, while bootctl(1) and related infrastructure > > > > > > > > remains > > > > > > > > in the systemd source tree and evolves to be able to manage > > > > > > > > arbitrary > > > > > > > > BLS-conformant boot managers. > > > > > > > > > > > > > > Why though? I don't follow? as long as we provide you with a > > > > > > > tarball > > > > > > > you can build your stuff from it shouldn't really matter if > > > > > > > there's > > > > > > > more stuff in it you are not immediately interested in. > > > > > > > > > > > > > > I mean, if you like I could do a "cp systemd-251.tar.gz > > > > > > > systemd-boot-251.tar.gz" for you if you want two tarballs instead > > > > > > > of > > > > > > > one, but I don't see the point? > > > > > > > > > > > > > > > > > > > As I illustrated in another email[5], decoupling the lifecycle of > > > > > > the > > > > > > EFI boot manager code from the rest of systemd would be ideal to not > > > > > > make the constraints around building sd-boot with secure boot > > > > > > support > > > > > > painful. > > > > > > > > > > > > [5]: > > > > > > https://lists.freedesktop.org/archives/systemd-devel/2022-April/047801.html > > > > > > > > > > Apart from the constraint who can build official packages, is there > > > > > anything else? If it's just that, that doesn't seem onerous. > > > > > > > > It also means Fedora CI, pull requests from contributors, and > > > > releng auto-rebuilds will no longer work. Maintainers basically > > > > sign-on to do all of those things manually and have to be responsive > > > > for doing it. You will get FTBFS tickets every cycle because of it, > > > > for example. > > > > > > > > Koji doesn't conceptually know the difference because there is no > > > > namespacing for builders, only who is approved to build. > > > > > > > > (In contrast, in the Open Build Service like what Luca Boccassi was > > > > talking about, packagers don't control the builds at all, so OBS only > > > > has to trust itself to sign it, so everything works properly.) > > > > > > You could simply have a separate source RPM, no? That should be pretty > > > simple, and limit the impact on team maintenance of the main source > > > package? > > > > > > > Yep. I was hoping we could have the upstream sources split too, but if > > we can't, then that's definitely the preferred way to go. > > With this PR https://github.com/systemd/systemd/pull/23204 you'll be > able to do this, which takes a fraction of a second to complete, as > opposed to the full build: > > $ ninja src/boot/efi/linuxx64.efi.stub > [21/21] Generating src/boot/efi/linuxx64.efi.stub with a custom command > $ ninja src/boot/efi/systemd-bootx64.efi > [10/10] Generating src/boot/efi/systemd-bootx64.efi with a custom > command > $ DESTDIR=/tmp/foo meson install --tags sd-boot,sd-stub --no-rebuild > Installing src/boot/efi/systemd-bootx64.efi to > /tmp/foo/usr/lib/systemd/boot/efi > Installing src/boot/efi/linuxx64.elf.stub to > /tmp/foo/usr/lib/systemd/boot/efi > Installing src/boot/efi/linuxx64.efi.stub to > /tmp/foo/usr/lib/systemd/boot/efi > > It should make it easier and more manageable I hope? > Yup, for sure! Being able to avoid compiling all of systemd when just building sd-boot would be really useful! > > > (OBS is awesome :-) ) > > > > > > > Indeed. I run an OBS instance for my workplace, and I've contributed > > to OBS over the years. It has its warts, but I think it got more > > right than wrong in the philosophy of a build system. > > Same at $previous_job, and miss it dearly at $current_job. The way > they've integrated signing EFI binaries for packages is quite nice too > - I contributed some bits and pieces to enable EFI binary signing for > Debian builds too, as it was supported only for RPM before, PoC with > shim + grub + kernel = secure bootable Debian image: > https://build.opensuse.org/project/show/home:bluca:debian_secure_boot > Interesting, I'll have to look at it at some point... -- 真実はいつも一つ！/ Always, there's only one truth!
Re: [systemd-devel] Splitting sd-boot from systemd/bootctl for enabling sd-boot in Fedora
On Wed, Apr 27, 2022 at 11:48:04AM -0400, Neal Gompa wrote: > On Wed, Apr 27, 2022 at 11:46 AM Luca Boccassi wrote: > > > > On Wed, 2022-04-27 at 11:26 -0400, Neal Gompa wrote: > > > On Wed, Apr 27, 2022 at 11:13 AM Zbigniew Jędrzejewski-Szmek > > > wrote: > > > > > > > > On Wed, Apr 27, 2022 at 09:31:10AM -0400, Neal Gompa wrote: > > > > > On Wed, Apr 27, 2022 at 7:41 AM Lennart Poettering > > > > > > > Realistically, I think if we want to make movement on making > > > > > > > systemd-boot fully supported in Fedora, the systemd-boot boot > > > > > > > manager > > > > > > > code itself should be split out into its own repository with its > > > > > > > own > > > > > > > release cadence, while bootctl(1) and related infrastructure > > > > > > > remains > > > > > > > in the systemd source tree and evolves to be able to manage > > > > > > > arbitrary > > > > > > > BLS-conformant boot managers. > > > > > > > > > > > > Why though? I don't follow? as long as we provide you with a tarball > > > > > > you can build your stuff from it shouldn't really matter if there's > > > > > > more stuff in it you are not immediately interested in. > > > > > > > > > > > > I mean, if you like I could do a "cp systemd-251.tar.gz > > > > > > systemd-boot-251.tar.gz" for you if you want two tarballs instead of > > > > > > one, but I don't see the point? > > > > > > > > > > > > > > > > As I illustrated in another email[5], decoupling the lifecycle of the > > > > > EFI boot manager code from the rest of systemd would be ideal to not > > > > > make the constraints around building sd-boot with secure boot support > > > > > painful. > > > > > > > > > > [5]: > > > > > https://lists.freedesktop.org/archives/systemd-devel/2022-April/047801.html > > > > > > > > Apart from the constraint who can build official packages, is there > > > > anything else? If it's just that, that doesn't seem onerous. > > > > > > It also means Fedora CI, pull requests from contributors, and > > > releng auto-rebuilds will no longer work. Maintainers basically > > > sign-on to do all of those things manually and have to be responsive > > > for doing it. You will get FTBFS tickets every cycle because of it, > > > for example. > > > > > > Koji doesn't conceptually know the difference because there is no > > > namespacing for builders, only who is approved to build. > > > > > > (In contrast, in the Open Build Service like what Luca Boccassi was > > > talking about, packagers don't control the builds at all, so OBS only > > > has to trust itself to sign it, so everything works properly.) > > > > You could simply have a separate source RPM, no? That should be pretty > > simple, and limit the impact on team maintenance of the main source > > package? >From what I can see, it'd be strictly more work for the maintainers (*). I keep asking, but I'm not getting any answers! There's some supposed benefits, but apart from vague handwaving, I don't see anything. (*) Two packages two build instead of one. It's not a huge difference, but from experience, it's confusing to contributors and annoying for maintainers to remember to duplicate the work. > Yep. I was hoping we could have the upstream sources split too, but if > we can't, then that's definitely the preferred way to go. Again, [citation needed] for why this would be of benefit. Zbsyzek
Re: [systemd-devel] Splitting sd-boot from systemd/bootctl for enabling sd-boot in Fedora
On Mi, 27.04.22 11:10, Neal Gompa (ngomp...@gmail.com) wrote: > > Rebooting from the DE has advantages: nice UI without much work, l10n, > > accessibility, help, integration with normal auth mechanisms (e.g. polkit > > auth for non-default boot entries or firmware setup), no need to > > fiddle with pressing keys at the exactly right time. > > It also has a major downside that in the event the OS doesn't boot, > you don't have a friendly way to do recovery. What does "recovery" precisely mean for you? I mean, on Linux this usually means you'll be dumped at a login prompt/shell in one way or another. How does it matter whether you first showed a graphical icon in that case? > Nowadays both Windows and macOS provide graphical boot managers and > graphical tools/environments for recovery. These are both things I > want in Fedora as well. Well, it sounds backwards to focus on the boot loader UI side of "recovery" so much if you don't even have any reasonably thing you could do in case of recovery better than a login prompt/shell... Quite frankly, I think we should actually focus on real improvements to recovery stuff, i.e. boot counting/automatic fallback on failed boots. which sd-boot all implements btw, in conjunction with systemd userspace. That kind of stuff makes whole sets of problems go away entirely, and is *actually* helpful. Whether we first show a graphical icon or just a text before we dump you in a shell prompt once all is lost anyway is kinda a pointless discussion if you ask me. For me recovery means something very different than graphical icons I must say. Lennart -- Lennart Poettering, Berlin
Re: [systemd-devel] Splitting sd-boot from systemd/bootctl for enabling sd-boot in Fedora
Am Mi., 27. Apr. 2022 um 17:16 Uhr schrieb Dan Nicholson : > > On Wed, Apr 27, 2022 at 9:01 AM Michael Biebl wrote: > > > > Slightly related > > https://salsa.debian.org/systemd-team/systemd/-/merge_requests/138 > > [sd-boot split] > > https://salsa.debian.org/systemd-team/systemd/-/merge_requests/132 > > [Draft: Prepare for EFI signing] > > Oh, nice. We've been signing sd-boot in Endless for a couple years now > with our systemd package based on Debian's. Currently the entire > systemd package is sent through the signing flow just for sd-boot. > When sd-boot is a separate package that can be much simpler with the > normal non-sd-boot targets unaffected. This discussion might be relevant to you then https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=996202 Automatically signing sd-boot in Debian was rejected by Julian Andres Klode
Re: [systemd-devel] Splitting sd-boot from systemd/bootctl for enabling sd-boot in Fedora
On Wed, 2022-04-27 at 11:48 -0400, Neal Gompa wrote: > On Wed, Apr 27, 2022 at 11:46 AM Luca Boccassi wrote: > > > > On Wed, 2022-04-27 at 11:26 -0400, Neal Gompa wrote: > > > On Wed, Apr 27, 2022 at 11:13 AM Zbigniew Jędrzejewski-Szmek > > > wrote: > > > > > > > > On Wed, Apr 27, 2022 at 09:31:10AM -0400, Neal Gompa wrote: > > > > > On Wed, Apr 27, 2022 at 7:41 AM Lennart Poettering > > > > > > > Realistically, I think if we want to make movement on making > > > > > > > systemd-boot fully supported in Fedora, the systemd-boot boot > > > > > > > manager > > > > > > > code itself should be split out into its own repository with its > > > > > > > own > > > > > > > release cadence, while bootctl(1) and related infrastructure > > > > > > > remains > > > > > > > in the systemd source tree and evolves to be able to manage > > > > > > > arbitrary > > > > > > > BLS-conformant boot managers. > > > > > > > > > > > > Why though? I don't follow? as long as we provide you with a tarball > > > > > > you can build your stuff from it shouldn't really matter if there's > > > > > > more stuff in it you are not immediately interested in. > > > > > > > > > > > > I mean, if you like I could do a "cp systemd-251.tar.gz > > > > > > systemd-boot-251.tar.gz" for you if you want two tarballs instead of > > > > > > one, but I don't see the point? > > > > > > > > > > > > > > > > As I illustrated in another email[5], decoupling the lifecycle of the > > > > > EFI boot manager code from the rest of systemd would be ideal to not > > > > > make the constraints around building sd-boot with secure boot support > > > > > painful. > > > > > > > > > > [5]: > > > > > https://lists.freedesktop.org/archives/systemd-devel/2022-April/047801.html > > > > > > > > Apart from the constraint who can build official packages, is there > > > > anything else? If it's just that, that doesn't seem onerous. > > > > > > It also means Fedora CI, pull requests from contributors, and > > > releng auto-rebuilds will no longer work. Maintainers basically > > > sign-on to do all of those things manually and have to be responsive > > > for doing it. You will get FTBFS tickets every cycle because of it, > > > for example. > > > > > > Koji doesn't conceptually know the difference because there is no > > > namespacing for builders, only who is approved to build. > > > > > > (In contrast, in the Open Build Service like what Luca Boccassi was > > > talking about, packagers don't control the builds at all, so OBS only > > > has to trust itself to sign it, so everything works properly.) > > > > You could simply have a separate source RPM, no? That should be pretty > > simple, and limit the impact on team maintenance of the main source > > package? > > > > Yep. I was hoping we could have the upstream sources split too, but if > we can't, then that's definitely the preferred way to go. With this PR https://github.com/systemd/systemd/pull/23204 you'll be able to do this, which takes a fraction of a second to complete, as opposed to the full build: $ ninja src/boot/efi/linuxx64.efi.stub [21/21] Generating src/boot/efi/linuxx64.efi.stub with a custom command $ ninja src/boot/efi/systemd-bootx64.efi [10/10] Generating src/boot/efi/systemd-bootx64.efi with a custom command $ DESTDIR=/tmp/foo meson install --tags sd-boot,sd-stub --no-rebuild Installing src/boot/efi/systemd-bootx64.efi to /tmp/foo/usr/lib/systemd/boot/efi Installing src/boot/efi/linuxx64.elf.stub to /tmp/foo/usr/lib/systemd/boot/efi Installing src/boot/efi/linuxx64.efi.stub to /tmp/foo/usr/lib/systemd/boot/efi It should make it easier and more manageable I hope? > > (OBS is awesome :-) ) > > > > Indeed. I run an OBS instance for my workplace, and I've contributed > to OBS over the years. It has its warts, but I think it got more > right than wrong in the philosophy of a build system. Same at $previous_job, and miss it dearly at $current_job. The way they've integrated signing EFI binaries for packages is quite nice too - I contributed some bits and pieces to enable EFI binary signing for Debian builds too, as it was supported only for RPM before, PoC with shim + grub + kernel = secure bootable Debian image: https://build.opensuse.org/project/show/home:bluca:debian_secure_boot -- Kind regards, Luca Boccassi signature.asc Description: This is a digitally signed message part
Re: [systemd-devel] Splitting sd-boot from systemd/bootctl for enabling sd-boot in Fedora
On Wed, Apr 27, 2022 at 11:49 AM Zbigniew Jędrzejewski-Szmek wrote: > > On Wed, Apr 27, 2022 at 11:26:14AM -0400, Neal Gompa wrote: > > On Wed, Apr 27, 2022 at 11:13 AM Zbigniew Jędrzejewski-Szmek > > wrote: > > > > > > On Wed, Apr 27, 2022 at 09:31:10AM -0400, Neal Gompa wrote: > > > > On Wed, Apr 27, 2022 at 7:41 AM Lennart Poettering > > > > wrote: > > > > It is not very friendly when you're in a failure scenario or have to > > > > deal with boot stuff. > > > > > > Well, if you have a boot failure, then a text-based interface is better > > > than a graphical one. I.e. while we might want to make it easier to > > > discover > > > disks and state from sd-boot, I doubt that adding support for icons and > > > themes would be of any help in failure scenarios. > > > > > > > I know it more or less looks like Fedora's GRUB > > > > is configured today, but Fedora is an outlier among Linux > > > > distributions in that it doesn't theme GRUB or provide more > > > > user-friendly boot configure out of the box. I don't like it and I'd > > > > like to change this someday. > > > > > > E.g. the biggest development in how the boot looks in recent years > > > in Fedora has been hiding on the boot menus and boot messages by > > > default. I.e. the _design_ is that you start with the logo of the > > > manufacturer which is seamlessly replaced by the gdm login screen. > > > How the boot menu looks never factors into any of this. > > > > > > > Hiding them by default doesn't mean making them scary and semi-useless > > when you access them. Most people don't access UEFI menus very often, > > if at all, and yet a huge amount of investment went into making that > > UX better than it was in the past. Is it spectacular? No. Is it less > > scary? Absolutely. > > > > Even with rEFInd, I'd probably want to do it that way because the > > point of rEFInd is to make the emergency cases and multiboot cases > > more approachable, not to present it all the time by default. > > > > > > Well, if you're installing and managing EFI binaries (as bootctl > > > > does), you can design a scheme to allow bootctl to know what to do in > > > > those circumstances. > > > > > > > > As a back of the napkin example, say you offer the user three EFI boot > > > > manager options: rEFInd, sd-boot, or GRUB. The distribution may prefer > > > > GRUB, sd-boot, or rEFInd, and define that as a config file for bootctl > > > > to read. But if the user wants to override this choice, they could > > > > pass --bootloader= to the install subcommand. That would write > > > > out a config file in /etc/systemd/boot declaring which bootloader the > > > > user chose as the "default" for future bootctl invocations and allow > > > > the commands to work. > > > > > > --bootloader= sounds like something we can do. OTOH, I agree > > > with what Lennart wrote elsewhere: we don't want to get into the > > > business of fiddling with special files that'd be specific so some > > > bootloader. > > > > > > Currently the update command only does the update if it find sd-boot. > > > We could enhance it to update other installations (if it find matching > > > files under /usr/lib somewhere. (I'd rather make it /usr/lib/boot/, > > > but I think we can talk about directory names later.) > > > > > > > That's more or less what I want. I don't really want bootctl being > > *too* smart. Even for sd-boot, it doesn't do that much, and I'd rather > > keep it that limited. If you need to do fancier bootloader specific > > things, use the appropriate tools. > > > > > > The bootloaders themselves could be stored in > > > > /usr/lib/systemd/boot/efi//, where is the bootloader name > > > > passed to bootctl. It would then know to copy all the files from that > > > > directory into the ESP. > > > > > > > > > > The second problem is because having sd-boot in the systemd source > > > > > > tree means that in order for Fedora to sign the boot manager EFI > > > > > > binaries, we have to lock down the systemd source package to the > > > > > > secure boot Koji build channel. This is unequivocally unacceptable > > > > > > from my point of view, as the restrictions around the secure boot > > > > > > channel make it realistically impossible for community contribution > > > > > > and maintenance of the package. > > > > > > > > > > I don't follow. Where's the problem using the same source package for > > > > > two independently built RPM packages? > > > > > > > > > > If Fedora wants to build systemd userspace packages one way, and > > > > > systemd-boot another way, that's entirely fine actually. they can just > > > > > do that… > > > > > > > > > > > Realistically, I think if we want to make movement on making > > > > > > systemd-boot fully supported in Fedora, the systemd-boot boot > > > > > > manager > > > > > > code itself should be split out into its own repository with its own > > > > > > release cadence, while bootctl(1) and related infrastructure remains > > > > > > in the systemd source tree and ev
Re: [systemd-devel] Splitting sd-boot from systemd/bootctl for enabling sd-boot in Fedora
On Mi, 27.04.22 11:26, Neal Gompa (ngomp...@gmail.com) wrote: > > E.g. the biggest development in how the boot looks in recent years > > in Fedora has been hiding on the boot menus and boot messages by > > default. I.e. the _design_ is that you start with the logo of the > > manufacturer which is seamlessly replaced by the gdm login screen. > > How the boot menu looks never factors into any of this. > > Hiding them by default doesn't mean making them scary and semi-useless > when you access them. sd-boot's UI is "semi-useless"? How so? This is very vague. >From your comments I mostly get that you don't like the aesthetics of sd-boot. Fine. But how does it make it "semi-useless"? Aesthetics is one thing, usefulness another. And in which way is the refind UI better? less "useless"? What specifically can it do that sd-boot cannot do also UI wise? I mean, we dont do graphical stuff, no themes, no icons. But let's ignore that for a minute. What else remains of your criticism? > Even with rEFInd, I'd probably want to do it that way because the > point of rEFInd is to make the emergency cases and multiboot cases > more approachable, not to present it all the time by default. "emergency cases more approachable"? what does that means? what specific emergency features does it have? it shows graphical icons, OK, but how that that help you in case of "emergency"? puzzled... Lennart -- Lennart Poettering, Berlin
Re: [systemd-devel] Splitting sd-boot from systemd/bootctl for enabling sd-boot in Fedora
On Wed, Apr 27, 2022 at 11:26:14AM -0400, Neal Gompa wrote: > On Wed, Apr 27, 2022 at 11:13 AM Zbigniew Jędrzejewski-Szmek > wrote: > > > > On Wed, Apr 27, 2022 at 09:31:10AM -0400, Neal Gompa wrote: > > > On Wed, Apr 27, 2022 at 7:41 AM Lennart Poettering > > > wrote: > > > It is not very friendly when you're in a failure scenario or have to > > > deal with boot stuff. > > > > Well, if you have a boot failure, then a text-based interface is better > > than a graphical one. I.e. while we might want to make it easier to discover > > disks and state from sd-boot, I doubt that adding support for icons and > > themes would be of any help in failure scenarios. > > > > > I know it more or less looks like Fedora's GRUB > > > is configured today, but Fedora is an outlier among Linux > > > distributions in that it doesn't theme GRUB or provide more > > > user-friendly boot configure out of the box. I don't like it and I'd > > > like to change this someday. > > > > E.g. the biggest development in how the boot looks in recent years > > in Fedora has been hiding on the boot menus and boot messages by > > default. I.e. the _design_ is that you start with the logo of the > > manufacturer which is seamlessly replaced by the gdm login screen. > > How the boot menu looks never factors into any of this. > > > > Hiding them by default doesn't mean making them scary and semi-useless > when you access them. Most people don't access UEFI menus very often, > if at all, and yet a huge amount of investment went into making that > UX better than it was in the past. Is it spectacular? No. Is it less > scary? Absolutely. > > Even with rEFInd, I'd probably want to do it that way because the > point of rEFInd is to make the emergency cases and multiboot cases > more approachable, not to present it all the time by default. > > > > Well, if you're installing and managing EFI binaries (as bootctl > > > does), you can design a scheme to allow bootctl to know what to do in > > > those circumstances. > > > > > > As a back of the napkin example, say you offer the user three EFI boot > > > manager options: rEFInd, sd-boot, or GRUB. The distribution may prefer > > > GRUB, sd-boot, or rEFInd, and define that as a config file for bootctl > > > to read. But if the user wants to override this choice, they could > > > pass --bootloader= to the install subcommand. That would write > > > out a config file in /etc/systemd/boot declaring which bootloader the > > > user chose as the "default" for future bootctl invocations and allow > > > the commands to work. > > > > --bootloader= sounds like something we can do. OTOH, I agree > > with what Lennart wrote elsewhere: we don't want to get into the > > business of fiddling with special files that'd be specific so some > > bootloader. > > > > Currently the update command only does the update if it find sd-boot. > > We could enhance it to update other installations (if it find matching > > files under /usr/lib somewhere. (I'd rather make it /usr/lib/boot/, > > but I think we can talk about directory names later.) > > > > That's more or less what I want. I don't really want bootctl being > *too* smart. Even for sd-boot, it doesn't do that much, and I'd rather > keep it that limited. If you need to do fancier bootloader specific > things, use the appropriate tools. > > > > The bootloaders themselves could be stored in > > > /usr/lib/systemd/boot/efi//, where is the bootloader name > > > passed to bootctl. It would then know to copy all the files from that > > > directory into the ESP. > > > > > > > > The second problem is because having sd-boot in the systemd source > > > > > tree means that in order for Fedora to sign the boot manager EFI > > > > > binaries, we have to lock down the systemd source package to the > > > > > secure boot Koji build channel. This is unequivocally unacceptable > > > > > from my point of view, as the restrictions around the secure boot > > > > > channel make it realistically impossible for community contribution > > > > > and maintenance of the package. > > > > > > > > I don't follow. Where's the problem using the same source package for > > > > two independently built RPM packages? > > > > > > > > If Fedora wants to build systemd userspace packages one way, and > > > > systemd-boot another way, that's entirely fine actually. they can just > > > > do that… > > > > > > > > > Realistically, I think if we want to make movement on making > > > > > systemd-boot fully supported in Fedora, the systemd-boot boot manager > > > > > code itself should be split out into its own repository with its own > > > > > release cadence, while bootctl(1) and related infrastructure remains > > > > > in the systemd source tree and evolves to be able to manage arbitrary > > > > > BLS-conformant boot managers. > > > > > > > > Why though? I don't follow? as long as we provide you with a tarball > > > > you can build your stuff from it shouldn't really matter if there's > > > > more stuff in it you are
Re: [systemd-devel] Starting transient services securely from other service without root
Hello Vašek. On Mon, Apr 25, 2022 at 10:45:34AM +0200, Vašek Šraier wrote: > TL;DR: I want to start transient system services from another system > service via DBus. All services should have as little privileges as > possible, definitely not root. How can I do that securely? PolicyKit popped to my mind with this short description, basically what you extend later. (Also I understand the "starter" and "started" are both the same user.) > Because the workers are slightly different (e.g. command line args) and > because it could be confusing to admins, we decided to use transient > services so that the workers can't be started without the master > process. Note this may be also capture with scopes (if you decide to track lifecycle of workers yourself instead of by systemd). But also scopes within PID1 require privileges, so that just redresses your problem. > - User sessions > The master process and worker processes can also run in a user > session. This directly solves problems with privileges. However, I am > not sure if running a user session with the semantics of a system > service is possible or a good idea. I also don't know if there is any > documentation related to user sessions without physical users. Do you mean having all your stuff as services of user instance of systemd? Or putting them in proper sessions (as PAMName=foo does)? I assume the former. It sounds also a bit strange (unusual use of user instance of unusual(?) requirements). One consequence that I see directly is that any resource assigned via cgroups would be restricted by the single user instance for the whole assemply of workers together. (That can be intended or not.) (For the latter, I wrote it just for completeness, I don't think it'd be useful in this case.) > - Use other service managers, not systemd Or minimize functions of your main process to just process the config and figure out jobs so that it can run as root with anything "sensitive" (open to external world) moved to unprivileged workers/helpers. In the end, I think it goes along axes like: - Is there any benefit of having the workers in individual systemd units? (That suggests just controlling everything by the main process (or 3rd party supervisor.) - Is there any privilege that is actually needed from PID1 or could a given user self-serve themselves? (That suggest the user instance services below.) HTH, Michal
Re: [systemd-devel] Splitting sd-boot from systemd/bootctl for enabling sd-boot in Fedora
On Wed, Apr 27, 2022 at 11:46 AM Luca Boccassi wrote: > > On Wed, 2022-04-27 at 11:26 -0400, Neal Gompa wrote: > > On Wed, Apr 27, 2022 at 11:13 AM Zbigniew Jędrzejewski-Szmek > > wrote: > > > > > > On Wed, Apr 27, 2022 at 09:31:10AM -0400, Neal Gompa wrote: > > > > On Wed, Apr 27, 2022 at 7:41 AM Lennart Poettering > > > > > > Realistically, I think if we want to make movement on making > > > > > > systemd-boot fully supported in Fedora, the systemd-boot boot > > > > > > manager > > > > > > code itself should be split out into its own repository with its own > > > > > > release cadence, while bootctl(1) and related infrastructure remains > > > > > > in the systemd source tree and evolves to be able to manage > > > > > > arbitrary > > > > > > BLS-conformant boot managers. > > > > > > > > > > Why though? I don't follow? as long as we provide you with a tarball > > > > > you can build your stuff from it shouldn't really matter if there's > > > > > more stuff in it you are not immediately interested in. > > > > > > > > > > I mean, if you like I could do a "cp systemd-251.tar.gz > > > > > systemd-boot-251.tar.gz" for you if you want two tarballs instead of > > > > > one, but I don't see the point? > > > > > > > > > > > > > As I illustrated in another email[5], decoupling the lifecycle of the > > > > EFI boot manager code from the rest of systemd would be ideal to not > > > > make the constraints around building sd-boot with secure boot support > > > > painful. > > > > > > > > [5]: > > > > https://lists.freedesktop.org/archives/systemd-devel/2022-April/047801.html > > > > > > Apart from the constraint who can build official packages, is there > > > anything else? If it's just that, that doesn't seem onerous. > > > > It also means Fedora CI, pull requests from contributors, and > > releng auto-rebuilds will no longer work. Maintainers basically > > sign-on to do all of those things manually and have to be responsive > > for doing it. You will get FTBFS tickets every cycle because of it, > > for example. > > > > Koji doesn't conceptually know the difference because there is no > > namespacing for builders, only who is approved to build. > > > > (In contrast, in the Open Build Service like what Luca Boccassi was > > talking about, packagers don't control the builds at all, so OBS only > > has to trust itself to sign it, so everything works properly.) > > You could simply have a separate source RPM, no? That should be pretty > simple, and limit the impact on team maintenance of the main source > package? > Yep. I was hoping we could have the upstream sources split too, but if we can't, then that's definitely the preferred way to go. > (OBS is awesome :-) ) > Indeed. I run an OBS instance for my workplace, and I've contributed to OBS over the years. It has its warts, but I think it got more right than wrong in the philosophy of a build system. -- 真実はいつも一つ！/ Always, there's only one truth!
Re: [systemd-devel] Splitting sd-boot from systemd/bootctl for enabling sd-boot in Fedora
On Wed, 2022-04-27 at 11:26 -0400, Neal Gompa wrote: > On Wed, Apr 27, 2022 at 11:13 AM Zbigniew Jędrzejewski-Szmek > wrote: > > > > On Wed, Apr 27, 2022 at 09:31:10AM -0400, Neal Gompa wrote: > > > On Wed, Apr 27, 2022 at 7:41 AM Lennart Poettering > > > > > Realistically, I think if we want to make movement on making > > > > > systemd-boot fully supported in Fedora, the systemd-boot boot manager > > > > > code itself should be split out into its own repository with its own > > > > > release cadence, while bootctl(1) and related infrastructure remains > > > > > in the systemd source tree and evolves to be able to manage arbitrary > > > > > BLS-conformant boot managers. > > > > > > > > Why though? I don't follow? as long as we provide you with a tarball > > > > you can build your stuff from it shouldn't really matter if there's > > > > more stuff in it you are not immediately interested in. > > > > > > > > I mean, if you like I could do a "cp systemd-251.tar.gz > > > > systemd-boot-251.tar.gz" for you if you want two tarballs instead of > > > > one, but I don't see the point? > > > > > > > > > > As I illustrated in another email[5], decoupling the lifecycle of the > > > EFI boot manager code from the rest of systemd would be ideal to not > > > make the constraints around building sd-boot with secure boot support > > > painful. > > > > > > [5]: > > > https://lists.freedesktop.org/archives/systemd-devel/2022-April/047801.html > > > > Apart from the constraint who can build official packages, is there > > anything else? If it's just that, that doesn't seem onerous. > > It also means Fedora CI, pull requests from contributors, and > releng auto-rebuilds will no longer work. Maintainers basically > sign-on to do all of those things manually and have to be responsive > for doing it. You will get FTBFS tickets every cycle because of it, > for example. > > Koji doesn't conceptually know the difference because there is no > namespacing for builders, only who is approved to build. > > (In contrast, in the Open Build Service like what Luca Boccassi was > talking about, packagers don't control the builds at all, so OBS only > has to trust itself to sign it, so everything works properly.) You could simply have a separate source RPM, no? That should be pretty simple, and limit the impact on team maintenance of the main source package? (OBS is awesome :-) ) -- Kind regards, Luca Boccassi signature.asc Description: This is a digitally signed message part
Re: [systemd-devel] Splitting sd-boot from systemd/bootctl for enabling sd-boot in Fedora
On Wed, Apr 27, 2022 at 11:26 AM Dan Nicholson wrote: > > On Wed, Apr 27, 2022 at 9:10 AM Neal Gompa wrote: > > > > Note that it means Fedora CI, pull requests from contributors, and > > releng auto-rebuilds will no longer work. Maintainers basically > > sign-on to do all of those things manually and have to be responsive > > for doing it. You will get FTBFS tickets every cycle because of it, > > for example. > > Asking systemd folks to change their development process because of > limitations in Fedora/Koji seems like a big ask, don't you think? Honestly, that never factors in for me, because then I'd never ask anything of anyone. :) But no, it doesn't sound unreasonable to me, because I reasonably expect the parts that make up the EFI boot manager code to be somewhat separate from the rest of the codebase in the first place. It targets a different build environment and has to be built differently from the rest of systemd anyway. So to me, it's not a "big ask" as you put it. But I'm also not demanding anything. I'm *asking*. If you don't want to do it, then that's fine. I only very occasionally help out with systemd in Fedora, and mostly deal with systemd in other distributions (CentOS Stream, etc.). I used to help with dracut and systemd maintenance in Mageia, so I'm familiar with the source structures. And I've contributed to systemd before, as well. I don't help more with systemd in Fedora because the packaging is pretty difficult for me to understand. If I did, I'd probably help out more. > Having implemented UEFI secure boot signing for Endless, I can concur > it is a PITA. However, there are certainly ways to make it work that > have no effect on upstream. Our Endless system is pretty hacky, but > Debian's is pretty well thought out. What both have in common is that > the signer generates a separate package so that the normal build flow > isn't affected. In the case of systemd, there would be both an > unsigned and signed version of the sd-boot EFI program in separate > packages. > > I'm sure it would require work to fix, but this seems like more of a > Koji problem than a systemd problem. I also feel like Lennart's > suggestion that sd-boot get split out as a separate source package but > using the same tarball is completely reasonable if your signing system > is too onerous to use. > It's definitely problems with both. I agree that a chunk of this is Koji's problem. I spent two years doing research on this topic because I was working on kernel, kmod, and bootloader builds and discovered how terrible it is and how nobody wants to improve it. My saltiness about the whole affair is pretty evident in the Fedora discussion about discontinuing BIOS support (and was even in an LWN article, apparently!). The Open Build Service is the only build system I know of that avoids this problem by totally removing the packager's ability to control builds *entirely*. They don't get to choose how the Release field works, they don't get to choose when builds are made, and they don't get to choose when builds are released. That's all entirely under the control of the build system, which does its own decision-making by constantly tracking the repoclosure state of the repository. Since no humans are involved, everyone is equally happy and unhappy. :) But I also strongly believe that it was a mistake to merge gummiboot into the systemd source tree because it creates all kinds of problems for distro maintenance, as I've already said earlier. It also discourages making the sd-boot code tighter and simpler since your primary focus is reuse across the tree rather than strictly separating the functional domains. -- 真実はいつも一つ！/ Always, there's only one truth!
Re: [systemd-devel] Splitting sd-boot from systemd/bootctl for enabling sd-boot in Fedora
On Wed, Apr 27, 2022 at 11:13 AM Zbigniew Jędrzejewski-Szmek wrote: > > On Wed, Apr 27, 2022 at 09:31:10AM -0400, Neal Gompa wrote: > > On Wed, Apr 27, 2022 at 7:41 AM Lennart Poettering > > wrote: > > It is not very friendly when you're in a failure scenario or have to > > deal with boot stuff. > > Well, if you have a boot failure, then a text-based interface is better > than a graphical one. I.e. while we might want to make it easier to discover > disks and state from sd-boot, I doubt that adding support for icons and > themes would be of any help in failure scenarios. > > > I know it more or less looks like Fedora's GRUB > > is configured today, but Fedora is an outlier among Linux > > distributions in that it doesn't theme GRUB or provide more > > user-friendly boot configure out of the box. I don't like it and I'd > > like to change this someday. > > E.g. the biggest development in how the boot looks in recent years > in Fedora has been hiding on the boot menus and boot messages by > default. I.e. the _design_ is that you start with the logo of the > manufacturer which is seamlessly replaced by the gdm login screen. > How the boot menu looks never factors into any of this. > Hiding them by default doesn't mean making them scary and semi-useless when you access them. Most people don't access UEFI menus very often, if at all, and yet a huge amount of investment went into making that UX better than it was in the past. Is it spectacular? No. Is it less scary? Absolutely. Even with rEFInd, I'd probably want to do it that way because the point of rEFInd is to make the emergency cases and multiboot cases more approachable, not to present it all the time by default. > > Well, if you're installing and managing EFI binaries (as bootctl > > does), you can design a scheme to allow bootctl to know what to do in > > those circumstances. > > > > As a back of the napkin example, say you offer the user three EFI boot > > manager options: rEFInd, sd-boot, or GRUB. The distribution may prefer > > GRUB, sd-boot, or rEFInd, and define that as a config file for bootctl > > to read. But if the user wants to override this choice, they could > > pass --bootloader= to the install subcommand. That would write > > out a config file in /etc/systemd/boot declaring which bootloader the > > user chose as the "default" for future bootctl invocations and allow > > the commands to work. > > --bootloader= sounds like something we can do. OTOH, I agree > with what Lennart wrote elsewhere: we don't want to get into the > business of fiddling with special files that'd be specific so some > bootloader. > > Currently the update command only does the update if it find sd-boot. > We could enhance it to update other installations (if it find matching > files under /usr/lib somewhere. (I'd rather make it /usr/lib/boot/, > but I think we can talk about directory names later.) > That's more or less what I want. I don't really want bootctl being *too* smart. Even for sd-boot, it doesn't do that much, and I'd rather keep it that limited. If you need to do fancier bootloader specific things, use the appropriate tools. > > The bootloaders themselves could be stored in > > /usr/lib/systemd/boot/efi//, where is the bootloader name > > passed to bootctl. It would then know to copy all the files from that > > directory into the ESP. > > > > > > The second problem is because having sd-boot in the systemd source > > > > tree means that in order for Fedora to sign the boot manager EFI > > > > binaries, we have to lock down the systemd source package to the > > > > secure boot Koji build channel. This is unequivocally unacceptable > > > > from my point of view, as the restrictions around the secure boot > > > > channel make it realistically impossible for community contribution > > > > and maintenance of the package. > > > > > > I don't follow. Where's the problem using the same source package for > > > two independently built RPM packages? > > > > > > If Fedora wants to build systemd userspace packages one way, and > > > systemd-boot another way, that's entirely fine actually. they can just > > > do that… > > > > > > > Realistically, I think if we want to make movement on making > > > > systemd-boot fully supported in Fedora, the systemd-boot boot manager > > > > code itself should be split out into its own repository with its own > > > > release cadence, while bootctl(1) and related infrastructure remains > > > > in the systemd source tree and evolves to be able to manage arbitrary > > > > BLS-conformant boot managers. > > > > > > Why though? I don't follow? as long as we provide you with a tarball > > > you can build your stuff from it shouldn't really matter if there's > > > more stuff in it you are not immediately interested in. > > > > > > I mean, if you like I could do a "cp systemd-251.tar.gz > > > systemd-boot-251.tar.gz" for you if you want two tarballs instead of > > > one, but I don't see the point? > > > > > > > As I illustrated in anoth
Re: [systemd-devel] Splitting sd-boot from systemd/bootctl for enabling sd-boot in Fedora
On Wed, Apr 27, 2022 at 9:10 AM Neal Gompa wrote: > > Note that it means Fedora CI, pull requests from contributors, and > releng auto-rebuilds will no longer work. Maintainers basically > sign-on to do all of those things manually and have to be responsive > for doing it. You will get FTBFS tickets every cycle because of it, > for example. Asking systemd folks to change their development process because of limitations in Fedora/Koji seems like a big ask, don't you think? Having implemented UEFI secure boot signing for Endless, I can concur it is a PITA. However, there are certainly ways to make it work that have no effect on upstream. Our Endless system is pretty hacky, but Debian's is pretty well thought out. What both have in common is that the signer generates a separate package so that the normal build flow isn't affected. In the case of systemd, there would be both an unsigned and signed version of the sd-boot EFI program in separate packages. I'm sure it would require work to fix, but this seems like more of a Koji problem than a systemd problem. I also feel like Lennart's suggestion that sd-boot get split out as a separate source package but using the same tarball is completely reasonable if your signing system is too onerous to use. -- Dan
Re: [systemd-devel] Splitting sd-boot from systemd/bootctl for enabling sd-boot in Fedora
On Wed, Apr 27, 2022 at 9:01 AM Michael Biebl wrote: > > Slightly related > https://salsa.debian.org/systemd-team/systemd/-/merge_requests/138 > [sd-boot split] > https://salsa.debian.org/systemd-team/systemd/-/merge_requests/132 > [Draft: Prepare for EFI signing] Oh, nice. We've been signing sd-boot in Endless for a couple years now with our systemd package based on Debian's. Currently the entire systemd package is sent through the signing flow just for sd-boot. When sd-boot is a separate package that can be much simpler with the normal non-sd-boot targets unaffected. -- Dan
Re: [systemd-devel] Splitting sd-boot from systemd/bootctl for enabling sd-boot in Fedora
On Wed, Apr 27, 2022 at 09:31:10AM -0400, Neal Gompa wrote: > On Wed, Apr 27, 2022 at 7:41 AM Lennart Poettering > wrote: > It is not very friendly when you're in a failure scenario or have to > deal with boot stuff. Well, if you have a boot failure, then a text-based interface is better than a graphical one. I.e. while we might want to make it easier to discover disks and state from sd-boot, I doubt that adding support for icons and themes would be of any help in failure scenarios. > I know it more or less looks like Fedora's GRUB > is configured today, but Fedora is an outlier among Linux > distributions in that it doesn't theme GRUB or provide more > user-friendly boot configure out of the box. I don't like it and I'd > like to change this someday. E.g. the biggest development in how the boot looks in recent years in Fedora has been hiding on the boot menus and boot messages by default. I.e. the _design_ is that you start with the logo of the manufacturer which is seamlessly replaced by the gdm login screen. How the boot menu looks never factors into any of this. > Well, if you're installing and managing EFI binaries (as bootctl > does), you can design a scheme to allow bootctl to know what to do in > those circumstances. > > As a back of the napkin example, say you offer the user three EFI boot > manager options: rEFInd, sd-boot, or GRUB. The distribution may prefer > GRUB, sd-boot, or rEFInd, and define that as a config file for bootctl > to read. But if the user wants to override this choice, they could > pass --bootloader= to the install subcommand. That would write > out a config file in /etc/systemd/boot declaring which bootloader the > user chose as the "default" for future bootctl invocations and allow > the commands to work. --bootloader= sounds like something we can do. OTOH, I agree with what Lennart wrote elsewhere: we don't want to get into the business of fiddling with special files that'd be specific so some bootloader. Currently the update command only does the update if it find sd-boot. We could enhance it to update other installations (if it find matching files under /usr/lib somewhere. (I'd rather make it /usr/lib/boot/, but I think we can talk about directory names later.) > The bootloaders themselves could be stored in > /usr/lib/systemd/boot/efi//, where is the bootloader name > passed to bootctl. It would then know to copy all the files from that > directory into the ESP. > > > > The second problem is because having sd-boot in the systemd source > > > tree means that in order for Fedora to sign the boot manager EFI > > > binaries, we have to lock down the systemd source package to the > > > secure boot Koji build channel. This is unequivocally unacceptable > > > from my point of view, as the restrictions around the secure boot > > > channel make it realistically impossible for community contribution > > > and maintenance of the package. > > > > I don't follow. Where's the problem using the same source package for > > two independently built RPM packages? > > > > If Fedora wants to build systemd userspace packages one way, and > > systemd-boot another way, that's entirely fine actually. they can just > > do that… > > > > > Realistically, I think if we want to make movement on making > > > systemd-boot fully supported in Fedora, the systemd-boot boot manager > > > code itself should be split out into its own repository with its own > > > release cadence, while bootctl(1) and related infrastructure remains > > > in the systemd source tree and evolves to be able to manage arbitrary > > > BLS-conformant boot managers. > > > > Why though? I don't follow? as long as we provide you with a tarball > > you can build your stuff from it shouldn't really matter if there's > > more stuff in it you are not immediately interested in. > > > > I mean, if you like I could do a "cp systemd-251.tar.gz > > systemd-boot-251.tar.gz" for you if you want two tarballs instead of > > one, but I don't see the point? > > > > As I illustrated in another email[5], decoupling the lifecycle of the > EFI boot manager code from the rest of systemd would be ideal to not > make the constraints around building sd-boot with secure boot support > painful. > > [5]: > https://lists.freedesktop.org/archives/systemd-devel/2022-April/047801.html Apart from the constraint who can build official packages, is there anything else? If it's just that, that doesn't seem onerous. > > > This would also (hopefully) encourage other boot managers to support > > > the Bootloader Spec configuration model, making it succeed as a > > > semi-universal abstraction for boot manager configuration. > > > > I don't think grub developers really care about bootctl. They probably > > never heard of it I figure ;-). > > > > I am not sure what the reason of the general disinterest from their > > camp towards integration with userspace/systemd is. But I doubt that > > a missing reorganization our tarballs is what is stopping them... >
Re: [systemd-devel] Splitting sd-boot from systemd/bootctl for enabling sd-boot in Fedora
On Wed, Apr 27, 2022 at 10:13 AM Zbigniew Jędrzejewski-Szmek
wrote:
>
> On Wed, Apr 27, 2022 at 08:55:55AM -0400, Neal Gompa wrote:
> > On Wed, Apr 27, 2022 at 6:47 AM Zbigniew Jędrzejewski-Szmek
> > wrote:
> > >
> > > On Tue, Apr 26, 2022 at 07:12:02PM -0400, Neal Gompa wrote:
> > > > Hey all,
> > >
> > > Hi Neal,
> > >
> > > thank you for starting the discussion. I think it'd be good to figure
> > > out what are the high-level options we have as a community…
> > >
> > > > Some of you might know about the recent discussion in Fedora about
> > > > dropping BIOS support[1][2]. While the end result for now is that
> > > > we're not dropping it[3], several side discussions involved enabling
> > > > systemd-boot as an option in Fedora in the future.
> > > >
> > > > While I *personally* am not a huge fan of systemd-boot itself
> > >
> > > You mentioned that a few times, and we (at least Lennnart and I) have
> > > asked for details. If there's something important missing from sd-boot,
> > > we would like to know.
> > >
> >
> > I think this is probably a distraction from this discussion, but sure
> > I guess I can answer: I fundamentally dislike systemd-boot because I
> > feel it's not a very user-friendly boot manager because of its
> > absolutely spartan interface. I'm much more of a fan of rEFInd[1],
> > which provides a feature-rich and user-friendly EFI boot manager[2].
>
> Ah, icons and graphics ;) Yeah, we don't provide that, and I don't think
> there are any plans to work on this. Instead of trying to make the menu
> better, we can follow the example of windows: integrate the boot menu
> directly in the graphical environment. We already have command-line access
> to this: bootctl reboot-to-firmware, bootctl set-oneshot,
> systemctl reboot --boot-loader-entry=. With a little bit of integration
> users should be able to select the system / kernel to boot directly
> from the reboot dialogue.
>
Windows also provides a functioning graphical boot manager since at
least Windows 10.
> Rebooting from the DE has advantages: nice UI without much work, l10n,
> accessibility, help, integration with normal auth mechanisms (e.g. polkit
> auth for non-default boot entries or firmware setup), no need to
> fiddle with pressing keys at the exactly right time.
>
It also has a major downside that in the event the OS doesn't boot,
you don't have a friendly way to do recovery. Nowadays both Windows
and macOS provide graphical boot managers and graphical
tools/environments for recovery. These are both things I want in
Fedora as well.
> > Essentially, the Koji build channel for secure boot is made up of three
> > things:
>
> [snip]
>
> Thanks for the description. If the limitation that only RH folks can
> build the official package is true, it'd be annoying, but not such a big
> issue. In the recent times, I made the most builds, with Adam Williamson
> and Yu Watanabe also doing an occasional build, i.e. all RH employees. It
> is important that people can do pull requests for dist-git, but limiting the
> offical builds to a few people wouldn't be a big issue. (Don't get me wrong:
> I would prefer to keep the current state where a bunch of maintainer *and*
> any proven packager or relengee can build systemd, but in practice it doesn't
> happen much.)
>
> > For sd-boot, it'd be making sure the package is "ExclusiveArch:
> > %{efi}", then in %install, you'd do:
> >
> > pushd %{buildroot}%{_prefix}/lib/systemd/boot/efi
> > %pesign -s -i systemd-boot%{efi_arch}.efi -o
> > systemd-boot%{efi_arch}.efi.signed
> > popd
>
> https://src.fedoraproject.org/rpms/systemd/pull-request/77
> does the deed. PTAL. (Though it just conditionalized the build on
> %ifarch %efi, instead of limiting where the package is built. In mock
> this produces an additional .signed thingy that 'bootctl update'
> should pick up automatically.)
>
Note that it means Fedora CI, pull requests from contributors, and
releng auto-rebuilds will no longer work. Maintainers basically
sign-on to do all of those things manually and have to be responsive
for doing it. You will get FTBFS tickets every cycle because of it,
for example.
--
真実はいつも一つ！/ Always, there's only one truth!
Re: [systemd-devel] Splitting sd-boot from systemd/bootctl for enabling sd-boot in Fedora
Slightly related
https://salsa.debian.org/systemd-team/systemd/-/merge_requests/138
[sd-boot split]
https://salsa.debian.org/systemd-team/systemd/-/merge_requests/132
[Draft: Prepare for EFI signing]
Am Mi., 27. Apr. 2022 um 16:13 Uhr schrieb Zbigniew Jędrzejewski-Szmek
:
>
> On Wed, Apr 27, 2022 at 08:55:55AM -0400, Neal Gompa wrote:
> > On Wed, Apr 27, 2022 at 6:47 AM Zbigniew Jędrzejewski-Szmek
> > wrote:
> > >
> > > On Tue, Apr 26, 2022 at 07:12:02PM -0400, Neal Gompa wrote:
> > > > Hey all,
> > >
> > > Hi Neal,
> > >
> > > thank you for starting the discussion. I think it'd be good to figure
> > > out what are the high-level options we have as a community…
> > >
> > > > Some of you might know about the recent discussion in Fedora about
> > > > dropping BIOS support[1][2]. While the end result for now is that
> > > > we're not dropping it[3], several side discussions involved enabling
> > > > systemd-boot as an option in Fedora in the future.
> > > >
> > > > While I *personally* am not a huge fan of systemd-boot itself
> > >
> > > You mentioned that a few times, and we (at least Lennnart and I) have
> > > asked for details. If there's something important missing from sd-boot,
> > > we would like to know.
> > >
> >
> > I think this is probably a distraction from this discussion, but sure
> > I guess I can answer: I fundamentally dislike systemd-boot because I
> > feel it's not a very user-friendly boot manager because of its
> > absolutely spartan interface. I'm much more of a fan of rEFInd[1],
> > which provides a feature-rich and user-friendly EFI boot manager[2].
>
> Ah, icons and graphics ;) Yeah, we don't provide that, and I don't think
> there are any plans to work on this. Instead of trying to make the menu
> better, we can follow the example of windows: integrate the boot menu
> directly in the graphical environment. We already have command-line access
> to this: bootctl reboot-to-firmware, bootctl set-oneshot,
> systemctl reboot --boot-loader-entry=. With a little bit of integration
> users should be able to select the system / kernel to boot directly
> from the reboot dialogue.
>
> Rebooting from the DE has advantages: nice UI without much work, l10n,
> accessibility, help, integration with normal auth mechanisms (e.g. polkit
> auth for non-default boot entries or firmware setup), no need to
> fiddle with pressing keys at the exactly right time.
>
> > Essentially, the Koji build channel for secure boot is made up of three
> > things:
>
> [snip]
>
> Thanks for the description. If the limitation that only RH folks can
> build the official package is true, it'd be annoying, but not such a big
> issue. In the recent times, I made the most builds, with Adam Williamson
> and Yu Watanabe also doing an occasional build, i.e. all RH employees. It
> is important that people can do pull requests for dist-git, but limiting the
> offical builds to a few people wouldn't be a big issue. (Don't get me wrong:
> I would prefer to keep the current state where a bunch of maintainer *and*
> any proven packager or relengee can build systemd, but in practice it doesn't
> happen much.)
>
> > For sd-boot, it'd be making sure the package is "ExclusiveArch:
> > %{efi}", then in %install, you'd do:
> >
> > pushd %{buildroot}%{_prefix}/lib/systemd/boot/efi
> > %pesign -s -i systemd-boot%{efi_arch}.efi -o
> > systemd-boot%{efi_arch}.efi.signed
> > popd
>
> https://src.fedoraproject.org/rpms/systemd/pull-request/77
> does the deed. PTAL. (Though it just conditionalized the build on
> %ifarch %efi, instead of limiting where the package is built. In mock
> this produces an additional .signed thingy that 'bootctl update'
> should pick up automatically.)
>
> Zbyszek
Re: [systemd-devel] Splitting sd-boot from systemd/bootctl for enabling sd-boot in Fedora
On Wed, Apr 27, 2022 at 08:55:55AM -0400, Neal Gompa wrote:
> On Wed, Apr 27, 2022 at 6:47 AM Zbigniew Jędrzejewski-Szmek
> wrote:
> >
> > On Tue, Apr 26, 2022 at 07:12:02PM -0400, Neal Gompa wrote:
> > > Hey all,
> >
> > Hi Neal,
> >
> > thank you for starting the discussion. I think it'd be good to figure
> > out what are the high-level options we have as a community…
> >
> > > Some of you might know about the recent discussion in Fedora about
> > > dropping BIOS support[1][2]. While the end result for now is that
> > > we're not dropping it[3], several side discussions involved enabling
> > > systemd-boot as an option in Fedora in the future.
> > >
> > > While I *personally* am not a huge fan of systemd-boot itself
> >
> > You mentioned that a few times, and we (at least Lennnart and I) have
> > asked for details. If there's something important missing from sd-boot,
> > we would like to know.
> >
>
> I think this is probably a distraction from this discussion, but sure
> I guess I can answer: I fundamentally dislike systemd-boot because I
> feel it's not a very user-friendly boot manager because of its
> absolutely spartan interface. I'm much more of a fan of rEFInd[1],
> which provides a feature-rich and user-friendly EFI boot manager[2].
Ah, icons and graphics ;) Yeah, we don't provide that, and I don't think
there are any plans to work on this. Instead of trying to make the menu
better, we can follow the example of windows: integrate the boot menu
directly in the graphical environment. We already have command-line access
to this: bootctl reboot-to-firmware, bootctl set-oneshot,
systemctl reboot --boot-loader-entry=. With a little bit of integration
users should be able to select the system / kernel to boot directly
from the reboot dialogue.
Rebooting from the DE has advantages: nice UI without much work, l10n,
accessibility, help, integration with normal auth mechanisms (e.g. polkit
auth for non-default boot entries or firmware setup), no need to
fiddle with pressing keys at the exactly right time.
> Essentially, the Koji build channel for secure boot is made up of three
> things:
[snip]
Thanks for the description. If the limitation that only RH folks can
build the official package is true, it'd be annoying, but not such a big
issue. In the recent times, I made the most builds, with Adam Williamson
and Yu Watanabe also doing an occasional build, i.e. all RH employees. It
is important that people can do pull requests for dist-git, but limiting the
offical builds to a few people wouldn't be a big issue. (Don't get me wrong:
I would prefer to keep the current state where a bunch of maintainer *and*
any proven packager or relengee can build systemd, but in practice it doesn't
happen much.)
> For sd-boot, it'd be making sure the package is "ExclusiveArch:
> %{efi}", then in %install, you'd do:
>
> pushd %{buildroot}%{_prefix}/lib/systemd/boot/efi
> %pesign -s -i systemd-boot%{efi_arch}.efi -o
> systemd-boot%{efi_arch}.efi.signed
> popd
https://src.fedoraproject.org/rpms/systemd/pull-request/77
does the deed. PTAL. (Though it just conditionalized the build on
%ifarch %efi, instead of limiting where the package is built. In mock
this produces an additional .signed thingy that 'bootctl update'
should pick up automatically.)
Zbyszek
Re: [systemd-devel] Splitting sd-boot from systemd/bootctl for enabling sd-boot in Fedora
On Mi, 27.04.22 09:44, Neal Gompa (ngomp...@gmail.com) wrote: > > To be frank, it seems to not actually bring much to the table that > > might be interesting from a modern Linux userspace perspective, > > i.e. doesn't implement boot laoder spec or boot loader interface, no > > boot counting, no random seed stuff, no certificate enrolling, no tpm > > measurement stuff, no devicetree loading, and so on. Functionalitywise > > it appears to be quite a step back from both sd-boot and in fact > > grub. I'd really prefer if we'd not complicate the boot loader > > question further, by throwing even more half-baked options into the > > mix. I mean, it's fine if this is an option, but more? > > Can anything actually *use* that stuff today? Depends on what you mean by "that" stuff. Different people use different features. I personally probably use the majority but not all (i only have x86 machines, so devictree is not something i care about, and suchlike). Distros have integreated things to different levels. Arch is pretty good in making the best of it. Stuff like the random seed stuff generally just is there, no need to "use" it really, it just happens there is no UI for that. > Like those are all nice > things to have, but as you've mentioned on the Fedora mailing lists > before, most of those things don't work or provide anything in the OS > because no enablement has been done there. I think in that case you are referring specifically to the UI integration of "boot into windows", "boot into boot loader entry", "boot into firmware" and so on in gdm? While integrating that in gdm would be fantastic, all the stuff is exposed on the command line just fine. i.e. i use "systemctl reboot --boot-loader-entry=…" and things like that all the time. So, yeah, I am pretty sure "that stuff" is pretty well used, but in some scenarios more and and in others less. > > I think this is unrealistic to be frank. Ignoring this refind thing > > (which I have not much clue about), for grub installation is a lot more > > complex than just dropping a bunch of files+dirs into the ESP. They > > have stages, partitions, boot scripts that need to be generated. I > > think the complexities this involves is a major problem, and certainly > > not something we should make *our* problem. > > At least in Fedora's grub2 (which is BLS-enabled), this doesn't apply. > All of that is static and we just use BLS snippets. My understanding > is that it *will* be upstreamed, but getting it upstreamed is slow > since the Red Hat grub2 patch set is *huge* and there's not enough > reviewers to go through and get patches into the tree. are you saying grub installation on fedora is just dropping some files and dirs into the ESP now? are you *sure* about that? i am pretty sure that's not the case, i.e. the weird boot counting stuff grub is doine actually works with an expicit file that needs to be created with specific properties, no? Lennart -- Lennart Poettering, Berlin
Re: [systemd-devel] Splitting sd-boot from systemd/bootctl for enabling sd-boot in Fedora
On Mi, 27.04.22 09:31, Neal Gompa (ngomp...@gmail.com) wrote: > > > Some of you might know about the recent discussion in Fedora about > > > dropping BIOS support[1][2]. While the end result for now is that > > > we're not dropping it[3], several side discussions involved enabling > > > systemd-boot as an option in Fedora in the future. > > > > > > While I *personally* am not a huge fan of systemd-boot itself, I > > > *am* > > > > I asked this elsewhere already, without getting a reply: why aren't > > you though? Can you elaborate on what crucial thing you are missing? > > It is not very friendly when you're in a failure scenario or have to > deal with boot stuff. But how does refind help you in failure scenario? Now you piqued my interest... > Eventually, I'd like to have a comparable experience to Windows or > macOS on Fedora, and I just don't see that happening with systemd-boot > as it stands today. Hmm, you mean this thing on macos: https://support.apple.com/library/content/dam/edam/applecare/images/en_US/macos/macos-sierra-startup-disk.png is that correct? I mean, does it actually offer you to do more than pick one of these icons and go for it? that's actually less of a UI than sd-boot if you so will… what else can it do? i know the windows boot loader can do more, but I think it's fairly in the territory of having gone too far. (it has the wifi selector, but i doubt this is relevant for us, so ignore that). > > Note that the commands in the last section of the "bootctl --help" > > text are something we never can make generic really. They are > > specifically about installing sd-boot in the ESP, I see no avenue > > about making this a grub installer, sorry... > > Well, if you're installing and managing EFI binaries (as bootctl > does), you can design a scheme to allow bootctl to know what to do in > those circumstances. As mentioned elswhere, grub needs to generate a boot script, and has multiple stages and a separate boot partition iirc. This is not stuff I want to see in bootctl. > As a back of the napkin example, say you offer the user three EFI boot > manager options: rEFInd, sd-boot, or GRUB. The distribution may prefer > GRUB, sd-boot, or rEFInd, and define that as a config file for bootctl > to read. But if the user wants to override this choice, they could > pass --bootloader= to the install subcommand. That would write > out a config file in /etc/systemd/boot declaring which bootloader the > user chose as the "default" for future bootctl invocations and allow > the commands to work. I think we systematically disagree on one point here: I am pretty sure picking a boot loader is genuinely someting a distro should be doing, and not the admin really. I mean, yes, I personally of course switched away from Fedora's default choice of grub to use sd-boot, and of course I'd prefer if it wasn't such a mess to do so. But also: we should not advertise this as something people should actually do and should make easy to do. > The bootloaders themselves could be stored in > /usr/lib/systemd/boot/efi//, where is the bootloader name > passed to bootctl. It would then know to copy all the files from that > directory into the ESP. We kinda do that already. But for the "bootctl update" stuff we actually want to know the version of the boot loaders we install, but only sd-boot implements a logic for that, how we can know it from the binary. > As I illustrated in another email[5], decoupling the lifecycle of the > EFI boot manager code from the rest of systemd would be ideal to not > make the constraints around building sd-boot with secure boot support > painful. But why would that be painful? I really don't follow. how we organize our git tree or what we put together in a tarball should not matter to the build processs. it's totally ok to decouple, desynchronize the build process of the sd-boot side and the rest, and that should be all you need? > It's a perception that based on your integration of sd-boot with > bootctl that there is no point for them to do it. The dynamics change > a lot when bootctl(1) is fully decoupled from sd-boot. [citation needed] > I don't have great answers here. > > But I really do think that being able to separate those lifecycles is > key to allowing Fedora to adopt it as a workable option. I don't understand why upstream release lifecycles should have to propagate 1:1 into RPM build cycles. It is absolutely, totally fine if Fedora builds userspace systemd RPM packages for 251 today and the systemd-boot parts 3 weeks later. The builds can be entirely independent and decoupled. In fact it's even OK if systemd-boot for example skips a few upstream releases. Our code is tightly coupled at build time, but at runtime as very losely coupled only, and it is our explicit goal to ensure that old userspace can work with new sd-boot and vice versa. Anf in fact work with other boot loaders, if they'd implement the specs... Lennart -- Lennart Poettering, Berlin
Re: [systemd-devel] Splitting sd-boot from systemd/bootctl for enabling sd-boot in Fedora
On Wed, Apr 27, 2022 at 9:31 AM Lennart Poettering wrote: > > On Mi, 27.04.22 08:55, Neal Gompa (ngomp...@gmail.com) wrote: > > > > You mentioned that a few times, and we (at least Lennnart and I) have > > > asked for details. If there's something important missing from sd-boot, > > > we would like to know. > > > > I think this is probably a distraction from this discussion, but sure > > I guess I can answer: I fundamentally dislike systemd-boot because I > > feel it's not a very user-friendly boot manager because of its > > absolutely spartan interface. I'm much more of a fan of rEFInd[1], > > which provides a feature-rich and user-friendly EFI boot manager[2]. > > Dunno. sd-boot UI was originally based on GNOME designers > ideas. It's a simple UI doing exactly what is needed, that's how > designs really should be I believe. Minimal, effective, and given this > is low-level nerd stuff that people should normally never see I think > that's already way above what is actually needed. > > I personally find the refind UI (judging by the screenshots) > stylistically problematic, but I'll certainly accept that other people > have other tastes on that... > > So, ignoring how precisely it looks, what exactly is the > "feature-rich" stuff that you prefer you appear to indicate that > exists? > > To be frank, it seems to not actually bring much to the table that > might be interesting from a modern Linux userspace perspective, > i.e. doesn't implement boot laoder spec or boot loader interface, no > boot counting, no random seed stuff, no certificate enrolling, no tpm > measurement stuff, no devicetree loading, and so on. Functionalitywise > it appears to be quite a step back from both sd-boot and in fact > grub. I'd really prefer if we'd not complicate the boot loader > question further, by throwing even more half-baked options into the > mix. I mean, it's fine if this is an option, but more? > Can anything actually *use* that stuff today? Like those are all nice things to have, but as you've mentioned on the Fedora mailing lists before, most of those things don't work or provide anything in the OS because no enablement has been done there. But this is pretty much why I didn't want to talk about this, because it diverts from the point I was trying to make around enabling sd-boot as an option. > > > > The first problem is mostly because I think bootctl(1) is a fantastic > > > > interface to manage *any* Bootloader Spec[4] (BLS) conformant boot > > > > manager, and I would love for that tool to be useful for doing so. > > > > Being able to do things like install/upgrade/swap GRUB 2, > > > > systemd-boot, or any other registered BLS-enabled boot manager would > > > > make it tremendously useful and valuable as a "building block" tool. I > > > > feel it would make sense to offer some kind of configuration to teach > > > > bootctl(1) about these boot managers so that it can work for them, and > > > > not just systemd-boot. > > > > > > bootctl could be taught to install other EFI stuff. It'd probably be a > > > matter of specifying a different glob when looking for files to copy. > > > I'm not sure if we want to get into the business of installing non-EFI > > > stuff… What exactly do you have in mind? > > > > I'm purely talking about EFI boot managers. I would like bootctl(1) to > > be able to lifecycle any BLS-conformant EFI boot manager. > > I think this is unrealistic to be frank. Ignoring this refind thing > (which I have not much clue about), for grub installation is a lot more > complex than just dropping a bunch of files+dirs into the ESP. They > have stages, partitions, boot scripts that need to be generated. I > think the complexities this involves is a major problem, and certainly > not something we should make *our* problem. > At least in Fedora's grub2 (which is BLS-enabled), this doesn't apply. All of that is static and we just use BLS snippets. My understanding is that it *will* be upstreamed, but getting it upstreamed is slow since the Red Hat grub2 patch set is *huge* and there's not enough reviewers to go through and get patches into the tree. And that means there are already two implementations. And there could be more in the future. -- 真実はいつも一つ！/ Always, there's only one truth!
Re: [systemd-devel] Splitting sd-boot from systemd/bootctl for enabling sd-boot in Fedora
On Wed, Apr 27, 2022 at 7:41 AM Lennart Poettering
wrote:
>
> On Di, 26.04.22 19:12, Neal Gompa (ngomp...@gmail.com) wrote:
>
> > Hey all,
> >
> > Some of you might know about the recent discussion in Fedora about
> > dropping BIOS support[1][2]. While the end result for now is that
> > we're not dropping it[3], several side discussions involved enabling
> > systemd-boot as an option in Fedora in the future.
> >
> > While I *personally* am not a huge fan of systemd-boot itself, I
> > *am*
>
> I asked this elsewhere already, without getting a reply: why aren't
> you though? Can you elaborate on what crucial thing you are missing?
>
It is not very friendly when you're in a failure scenario or have to
deal with boot stuff. I know it more or less looks like Fedora's GRUB
is configured today, but Fedora is an outlier among Linux
distributions in that it doesn't theme GRUB or provide more
user-friendly boot configure out of the box. I don't like it and I'd
like to change this someday.
Eventually, I'd like to have a comparable experience to Windows or
macOS on Fedora, and I just don't see that happening with systemd-boot
as it stands today.
I'm a fan of rEFInd[1], which provides a feature-rich and
user-friendly EFI boot manager[2] that I feel can offer that
experience. It even supports the Discoverable Partitions Spec[3], and
I hope that they'll support the Boot Loader Spec[4] eventually.
[1]: https://www.rodsbooks.com/refind/
[2]: https://www.rodsbooks.com/refind/features.html
[3]: https://systemd.io/DISCOVERABLE_PARTITIONS/
[4]: https://systemd.io/BOOT_LOADER_SPECIFICATION/
> > * bootctl(1) appears to be tightly coupled to sd-boot
>
> Well, kinda. But also not. So if you type "bootctl --help" then you'll
> see three sections of commands. The last section "systemd-boot
> Commands" shows commands that only really make sense for systemd-boot
> as they install/remove/update the boot loader itself. They are the
> only things inherently tied to sd-boot.
>
> The other two sections are useful outside of sd-boot: the first
> section ("Generic EFI Firmware/Boot Loader Commands") is useful on any
> kind of UEFI section, the second section ("Boot Loader Specification
> Commands") on all boot loaders that implement the (upstream) boot loader
> spec/boot loader interface, as documented.
>
> Note though that at this point grub does not implement either of the
> specs properly, or has any upstream work in that area to my
> knowledge.
>
> > The first problem is mostly because I think bootctl(1) is a fantastic
> > interface to manage *any* Bootloader Spec[4] (BLS) conformant boot
> > manager, and I would love for that tool to be useful for doing so.
> > Being able to do things like install/upgrade/swap GRUB 2,
> > systemd-boot, or any other registered BLS-enabled boot manager would
> > make it tremendously useful and valuable as a "building block" tool.
>
> As mentioned the commands in the first two sections of the --help
> blurb should work just fine with other loaders, and the reason the
> sections exist in the --help text is to help making this clear.
>
> > I feel it would make sense to offer some kind of configuration to
> > teach bootctl(1) about these boot managers so that it can work for
> > them, and not just systemd-boot.
>
> So the commands in the first two sections already should work with any
> boot loader implementing these specs. I am pretty sure that bootctl
> should not be adjusted to specific boot loaders needleslly, instead
> the bootloaders should just implement the specs...
>
> Implementing the specs after all is not something that just is
> relevant for bootctl only. After all there's a lot of hookup to the
> two specs in logind too (for example, reboot into Windows, reboot into
> boot loader entry xyz, and so on), or in the kexec-based reboot
> functionality. And a lot of other stuff as well…
>
> So we kept these things reasonably generic via making this specs so
> that any other boot loader can be plugged in there. But I think real
> interest in adding support for the specs in those other boot loaders
> appears to be minimal unfortunately.
>
> Note that the commands in the last section of the "bootctl --help"
> text are something we never can make generic really. They are
> specifically about installing sd-boot in the ESP, I see no avenue
> about making this a grub installer, sorry...
>
Well, if you're installing and managing EFI binaries (as bootctl
does), you can design a scheme to allow bootctl to know what to do in
those circumstances.
As a back of the napkin example, say you offer the user three EFI boot
manager options: rEFInd, sd-boot, or GRUB. The distribution may prefer
GRUB, sd-boot, or rEFInd, and define that as a config file for bootctl
to read. But if the user wants to override this choice, they could
pass --bootloader= to the install subcommand. That would write
out a config file in /etc/systemd/boot declaring which bootloader the
user chose as the "default" for future bootctl invocat
Re: [systemd-devel] Splitting sd-boot from systemd/bootctl for enabling sd-boot in Fedora
On Mi, 27.04.22 08:55, Neal Gompa (ngomp...@gmail.com) wrote: > > You mentioned that a few times, and we (at least Lennnart and I) have > > asked for details. If there's something important missing from sd-boot, > > we would like to know. > > I think this is probably a distraction from this discussion, but sure > I guess I can answer: I fundamentally dislike systemd-boot because I > feel it's not a very user-friendly boot manager because of its > absolutely spartan interface. I'm much more of a fan of rEFInd[1], > which provides a feature-rich and user-friendly EFI boot manager[2]. Dunno. sd-boot UI was originally based on GNOME designers ideas. It's a simple UI doing exactly what is needed, that's how designs really should be I believe. Minimal, effective, and given this is low-level nerd stuff that people should normally never see I think that's already way above what is actually needed. I personally find the refind UI (judging by the screenshots) stylistically problematic, but I'll certainly accept that other people have other tastes on that... So, ignoring how precisely it looks, what exactly is the "feature-rich" stuff that you prefer you appear to indicate that exists? To be frank, it seems to not actually bring much to the table that might be interesting from a modern Linux userspace perspective, i.e. doesn't implement boot laoder spec or boot loader interface, no boot counting, no random seed stuff, no certificate enrolling, no tpm measurement stuff, no devicetree loading, and so on. Functionalitywise it appears to be quite a step back from both sd-boot and in fact grub. I'd really prefer if we'd not complicate the boot loader question further, by throwing even more half-baked options into the mix. I mean, it's fine if this is an option, but more? > > > The first problem is mostly because I think bootctl(1) is a fantastic > > > interface to manage *any* Bootloader Spec[4] (BLS) conformant boot > > > manager, and I would love for that tool to be useful for doing so. > > > Being able to do things like install/upgrade/swap GRUB 2, > > > systemd-boot, or any other registered BLS-enabled boot manager would > > > make it tremendously useful and valuable as a "building block" tool. I > > > feel it would make sense to offer some kind of configuration to teach > > > bootctl(1) about these boot managers so that it can work for them, and > > > not just systemd-boot. > > > > bootctl could be taught to install other EFI stuff. It'd probably be a > > matter of specifying a different glob when looking for files to copy. > > I'm not sure if we want to get into the business of installing non-EFI > > stuff… What exactly do you have in mind? > > I'm purely talking about EFI boot managers. I would like bootctl(1) to > be able to lifecycle any BLS-conformant EFI boot manager. I think this is unrealistic to be frank. Ignoring this refind thing (which I have not much clue about), for grub installation is a lot more complex than just dropping a bunch of files+dirs into the ESP. They have stages, partitions, boot scripts that need to be generated. I think the complexities this involves is a major problem, and certainly not something we should make *our* problem. Lennart -- Lennart Poettering, Berlin
Re: [systemd-devel] Splitting sd-boot from systemd/bootctl for enabling sd-boot in Fedora
On Wed, Apr 27, 2022 at 8:55 AM Neal Gompa wrote: > > On Wed, Apr 27, 2022 at 6:47 AM Zbigniew Jędrzejewski-Szmek > wrote: > > > > On Tue, Apr 26, 2022 at 07:12:02PM -0400, Neal Gompa wrote: > > > Hey all, > > > > Hi Neal, > > > > thank you for starting the discussion. I think it'd be good to figure > > out what are the high-level options we have as a community… > > > > > Some of you might know about the recent discussion in Fedora about > > > dropping BIOS support[1][2]. While the end result for now is that > > > we're not dropping it[3], several side discussions involved enabling > > > systemd-boot as an option in Fedora in the future. > > > > > > While I *personally* am not a huge fan of systemd-boot itself > > > > You mentioned that a few times, and we (at least Lennnart and I) have > > asked for details. If there's something important missing from sd-boot, > > we would like to know. > > > > I think this is probably a distraction from this discussion, but sure > I guess I can answer: I fundamentally dislike systemd-boot because I > feel it's not a very user-friendly boot manager because of its > absolutely spartan interface. I'm much more of a fan of rEFInd[1], > which provides a feature-rich and user-friendly EFI boot manager[2]. > > While I get the idea that people shouldn't spend a lot of time at the > boot menu, it's very clear that other platforms (Windows and macOS) > spend a great deal of time making their boot environments friendly and > useful so that when they have to be there, it's not a bad place. I > like having the same for my Linux environments too. > > I use rEFInd on my MacBooks running Fedora Linux and I really like it. > A couple of other Linux distributions have also adopted rEFInd as an > option (notably Mageia did). One of the things I'd like to have as a > positive outcome from this would be to have the infrastructure in > place to make it appealing for rEFInd to support the Bootloader > Spec[3] in addition to the Discoverable Partition Spec[4] it already > supports today. > > [1]: https://www.rodsbooks.com/refind > [2]: https://www.rodsbooks.com/refind/features.html > [3]: https://systemd.io/BOOT_LOADER_SPECIFICATION/ > [4]: https://systemd.io/DISCOVERABLE_PARTITIONS/ > > > > I *am* a fan of a lot of the mechanisms around it, and I think it > > > would be valuable for us to adopt more of that in Fedora. To that > > > end, that means making it easier for people to fully adopt > > > systemd-boot on their systems in Fedora with minimal effort (ideally > > > just a kickstart or Anaconda flag if desired). > > > > Yeah. While I don't think we're ready to propose it as the default, > > we definitely would like it to be trivial to switch to it if somebody > > wants to. > > > > > From my point of view as someone working on several Fedora variants > > > and would like to provide more optionality around this, there are a > > > couple of issues: > > > > > > * bootctl(1) appears to be tightly coupled to sd-boot > > > > This is a misunderstanding of the our development goals of systemd > > (the project) and sd-boot. As far as possible, generic interfaces are used. > > > > Starting from the bottom: the boot loader specification is designed > > to be completely generic and independent of the boot loader and independent > > of the userspace tools used to configure the boot loader. Similarly, > > most bootctl commands are implemented using generic functionality > > (either EFI or any bootloader implementing the boot loader specification). > > And stuff like 'bootctl status' and 'systemd-analyze blame' use interfaces > > that are completely generic and any bootload can provide the appropriate > > information to the operating system (see > > https://systemd.io/BOOT_LOADER_INTERFACE/). > > So bootctl is NOT coupled to sd-boot, except for some specific parts > > explicitly documented to be about sd-boot, currently the > > install/update/uninstall verbs and random-seed support. > > > > > * sd-boot is part of the systemd source tree > > > > > > The first problem is mostly because I think bootctl(1) is a fantastic > > > interface to manage *any* Bootloader Spec[4] (BLS) conformant boot > > > manager, and I would love for that tool to be useful for doing so. > > > Being able to do things like install/upgrade/swap GRUB 2, > > > systemd-boot, or any other registered BLS-enabled boot manager would > > > make it tremendously useful and valuable as a "building block" tool. I > > > feel it would make sense to offer some kind of configuration to teach > > > bootctl(1) about these boot managers so that it can work for them, and > > > not just systemd-boot. > > > > bootctl could be taught to install other EFI stuff. It'd probably be a > > matter of specifying a different glob when looking for files to copy. > > I'm not sure if we want to get into the business of installing non-EFI > > stuff… What exactly do you have in mind? > > > > I'm purely talking about EFI boot managers. I would like bootctl(1) to > be ab
Re: [systemd-devel] Splitting sd-boot from systemd/bootctl for enabling sd-boot in Fedora
On Wed, Apr 27, 2022 at 6:47 AM Zbigniew Jędrzejewski-Szmek wrote: > > On Tue, Apr 26, 2022 at 07:12:02PM -0400, Neal Gompa wrote: > > Hey all, > > Hi Neal, > > thank you for starting the discussion. I think it'd be good to figure > out what are the high-level options we have as a community… > > > Some of you might know about the recent discussion in Fedora about > > dropping BIOS support[1][2]. While the end result for now is that > > we're not dropping it[3], several side discussions involved enabling > > systemd-boot as an option in Fedora in the future. > > > > While I *personally* am not a huge fan of systemd-boot itself > > You mentioned that a few times, and we (at least Lennnart and I) have > asked for details. If there's something important missing from sd-boot, > we would like to know. > I think this is probably a distraction from this discussion, but sure I guess I can answer: I fundamentally dislike systemd-boot because I feel it's not a very user-friendly boot manager because of its absolutely spartan interface. I'm much more of a fan of rEFInd[1], which provides a feature-rich and user-friendly EFI boot manager[2]. While I get the idea that people shouldn't spend a lot of time at the boot menu, it's very clear that other platforms (Windows and macOS) spend a great deal of time making their boot environments friendly and useful so that when they have to be there, it's not a bad place. I like having the same for my Linux environments too. I use rEFInd on my MacBooks running Fedora Linux and I really like it. A couple of other Linux distributions have also adopted rEFInd as an option (notably Mageia did). One of the things I'd like to have as a positive outcome from this would be to have the infrastructure in place to make it appealing for rEFInd to support the Bootloader Spec[3] in addition to the Discoverable Partition Spec[4] it already supports today. [1]: https://www.rodsbooks.com/refind [2]: https://www.rodsbooks.com/refind/features.html [3]: https://systemd.io/BOOT_LOADER_SPECIFICATION/ [4]: https://systemd.io/DISCOVERABLE_PARTITIONS/ > > I *am* a fan of a lot of the mechanisms around it, and I think it > > would be valuable for us to adopt more of that in Fedora. To that > > end, that means making it easier for people to fully adopt > > systemd-boot on their systems in Fedora with minimal effort (ideally > > just a kickstart or Anaconda flag if desired). > > Yeah. While I don't think we're ready to propose it as the default, > we definitely would like it to be trivial to switch to it if somebody > wants to. > > > From my point of view as someone working on several Fedora variants > > and would like to provide more optionality around this, there are a > > couple of issues: > > > > * bootctl(1) appears to be tightly coupled to sd-boot > > This is a misunderstanding of the our development goals of systemd > (the project) and sd-boot. As far as possible, generic interfaces are used. > > Starting from the bottom: the boot loader specification is designed > to be completely generic and independent of the boot loader and independent > of the userspace tools used to configure the boot loader. Similarly, > most bootctl commands are implemented using generic functionality > (either EFI or any bootloader implementing the boot loader specification). > And stuff like 'bootctl status' and 'systemd-analyze blame' use interfaces > that are completely generic and any bootload can provide the appropriate > information to the operating system (see > https://systemd.io/BOOT_LOADER_INTERFACE/). > So bootctl is NOT coupled to sd-boot, except for some specific parts > explicitly documented to be about sd-boot, currently the > install/update/uninstall verbs and random-seed support. > > > * sd-boot is part of the systemd source tree > > > > The first problem is mostly because I think bootctl(1) is a fantastic > > interface to manage *any* Bootloader Spec[4] (BLS) conformant boot > > manager, and I would love for that tool to be useful for doing so. > > Being able to do things like install/upgrade/swap GRUB 2, > > systemd-boot, or any other registered BLS-enabled boot manager would > > make it tremendously useful and valuable as a "building block" tool. I > > feel it would make sense to offer some kind of configuration to teach > > bootctl(1) about these boot managers so that it can work for them, and > > not just systemd-boot. > > bootctl could be taught to install other EFI stuff. It'd probably be a > matter of specifying a different glob when looking for files to copy. > I'm not sure if we want to get into the business of installing non-EFI > stuff… What exactly do you have in mind? > I'm purely talking about EFI boot managers. I would like bootctl(1) to be able to lifecycle any BLS-conformant EFI boot manager. > > The second problem is because having sd-boot in the systemd source > > tree means that in order for Fedora to sign the boot manager EFI > > binaries, we have to lock down the systemd source
Re: [systemd-devel] Splitting sd-boot from systemd/bootctl for enabling sd-boot in Fedora
On Di, 26.04.22 19:12, Neal Gompa (ngomp...@gmail.com) wrote:
> Hey all,
>
> Some of you might know about the recent discussion in Fedora about
> dropping BIOS support[1][2]. While the end result for now is that
> we're not dropping it[3], several side discussions involved enabling
> systemd-boot as an option in Fedora in the future.
>
> While I *personally* am not a huge fan of systemd-boot itself, I
> *am*
I asked this elsewhere already, without getting a reply: why aren't
you though? Can you elaborate on what crucial thing you are missing?
> * bootctl(1) appears to be tightly coupled to sd-boot
Well, kinda. But also not. So if you type "bootctl --help" then you'll
see three sections of commands. The last section "systemd-boot
Commands" shows commands that only really make sense for systemd-boot
as they install/remove/update the boot loader itself. They are the
only things inherently tied to sd-boot.
The other two sections are useful outside of sd-boot: the first
section ("Generic EFI Firmware/Boot Loader Commands") is useful on any
kind of UEFI section, the second section ("Boot Loader Specification
Commands") on all boot loaders that implement the (upstream) boot loader
spec/boot loader interface, as documented.
Note though that at this point grub does not implement either of the
specs properly, or has any upstream work in that area to my
knowledge.
> The first problem is mostly because I think bootctl(1) is a fantastic
> interface to manage *any* Bootloader Spec[4] (BLS) conformant boot
> manager, and I would love for that tool to be useful for doing so.
> Being able to do things like install/upgrade/swap GRUB 2,
> systemd-boot, or any other registered BLS-enabled boot manager would
> make it tremendously useful and valuable as a "building block" tool.
As mentioned the commands in the first two sections of the --help
blurb should work just fine with other loaders, and the reason the
sections exist in the --help text is to help making this clear.
> I feel it would make sense to offer some kind of configuration to
> teach bootctl(1) about these boot managers so that it can work for
> them, and not just systemd-boot.
So the commands in the first two sections already should work with any
boot loader implementing these specs. I am pretty sure that bootctl
should not be adjusted to specific boot loaders needleslly, instead
the bootloaders should just implement the specs...
Implementing the specs after all is not something that just is
relevant for bootctl only. After all there's a lot of hookup to the
two specs in logind too (for example, reboot into Windows, reboot into
boot loader entry xyz, and so on), or in the kexec-based reboot
functionality. And a lot of other stuff as well…
So we kept these things reasonably generic via making this specs so
that any other boot loader can be plugged in there. But I think real
interest in adding support for the specs in those other boot loaders
appears to be minimal unfortunately.
Note that the commands in the last section of the "bootctl --help"
text are something we never can make generic really. They are
specifically about installing sd-boot in the ESP, I see no avenue
about making this a grub installer, sorry...
> The second problem is because having sd-boot in the systemd source
> tree means that in order for Fedora to sign the boot manager EFI
> binaries, we have to lock down the systemd source package to the
> secure boot Koji build channel. This is unequivocally unacceptable
> from my point of view, as the restrictions around the secure boot
> channel make it realistically impossible for community contribution
> and maintenance of the package.
I don't follow. Where's the problem using the same source package for
two independently built RPM packages?
If Fedora wants to build systemd userspace packages one way, and
systemd-boot another way, that's entirely fine actually. they can just
do that…
> Realistically, I think if we want to make movement on making
> systemd-boot fully supported in Fedora, the systemd-boot boot manager
> code itself should be split out into its own repository with its own
> release cadence, while bootctl(1) and related infrastructure remains
> in the systemd source tree and evolves to be able to manage arbitrary
> BLS-conformant boot managers.
Why though? I don't follow? as long as we provide you with a tarball
you can build your stuff from it shouldn't really matter if there's
more stuff in it you are not immediately interested in.
I mean, if you like I could do a "cp systemd-251.tar.gz
systemd-boot-251.tar.gz" for you if you want two tarballs instead of
one, but I don't see the point?
> This would also (hopefully) encourage other boot managers to support
> the Bootloader Spec configuration model, making it succeed as a
> semi-universal abstraction for boot manager configuration.
I don't think grub developers really care about bootctl. They probably
never heard of it I figure ;-).
I am not sure what the reason of t
Re: [systemd-devel] Q: non-ASCII in syslog
On Mi, 27.04.22 09:09, Ulrich Windl (ulrich.wi...@rz.uni-regensburg.de) wrote: > Hi! > > Having written an RFC 3164 compatible syslog daemon, I noticed that systemd > created syslog messages with non-ASCII characters. > The problem is that a remote syslogd can hardly guess the correct character > set (I'm using rsyslog to forward local messages to a remote > server). It's 2022. I think at this point, software should always assume the charset is UTF-8 if it doesn't have an reason to believe otherwise. It's kinda what we started to do all across our codebase really. We'll use UTF-8 for everything by default. For some things where people complain sufficeintly loudly we'll conditionalize them so that we have some fallback in place if we know for sure UTF-8 is not OK, but the default we do is always and everywhere UTF-8. > Example of such message: > systemd-tmpfiles[3311]: [/usr/lib/tmpfiles.d/svnserve.conf:1] Line references > path below legacy directory /var/run/, updating /var/run/svnserve → > /run/svnserve; please update the tmpfiles.d/ drop-in file accordingly. > > (The arrow is encoded as three bytes (\xe2\x86\x92)) > > RFC 5425 syslog messages require the use of a BOM (%xEF.BB.BF) at the > beginning of a message if the message used UTF-8: We do not implement RFC 5425, as glibc doesn't support that. In fact we don't even implement RFC 3164 in full, since glibc generates the messages in a very specific format only. > > MSG = MSG-ANY / MSG-UTF8 > MSG-ANY = *OCTET ; not starting with BOM > MSG-UTF8= BOM UTF-8-STRING > BOM = %xEF.BB.BF > > Wouldn't it make sense to add such a BOM for RFC 3164 syslog messages also if > non-ASCII (i.e.: UTF-8) encoded characters are used? There's plenty software that doesn't support RFC 5425, and putting a BOM first is certainly not implemented in any of those. I think BOM is hideous and defaulting to UTF-8 generally safe. If we'd put BOM first, these messages would likely not be compatible with a large variety of consumers anymore, because they can't handle BOM. This would be worse than the status quo I am sure, since if we just send UTF-8 things should generally just work fine for any software that either a) also defaults to UTF-8 when encountering an 8bit char or b) is agonistic to charsets and just passes data thorugh. So, yeah, we might be stretching stdandards and tradition a bit, but it actually works out quite well so far. Lennart -- Lennart Poettering, Berlin
Re: [systemd-devel] Splitting sd-boot from systemd/bootctl for enabling sd-boot in Fedora
On Tue, Apr 26, 2022 at 07:12:02PM -0400, Neal Gompa wrote: > Hey all, Hi Neal, thank you for starting the discussion. I think it'd be good to figure out what are the high-level options we have as a community… > Some of you might know about the recent discussion in Fedora about > dropping BIOS support[1][2]. While the end result for now is that > we're not dropping it[3], several side discussions involved enabling > systemd-boot as an option in Fedora in the future. > > While I *personally* am not a huge fan of systemd-boot itself You mentioned that a few times, and we (at least Lennnart and I) have asked for details. If there's something important missing from sd-boot, we would like to know. > I *am* a fan of a lot of the mechanisms around it, and I think it > would be valuable for us to adopt more of that in Fedora. To that > end, that means making it easier for people to fully adopt > systemd-boot on their systems in Fedora with minimal effort (ideally > just a kickstart or Anaconda flag if desired). Yeah. While I don't think we're ready to propose it as the default, we definitely would like it to be trivial to switch to it if somebody wants to. > From my point of view as someone working on several Fedora variants > and would like to provide more optionality around this, there are a > couple of issues: > > * bootctl(1) appears to be tightly coupled to sd-boot This is a misunderstanding of the our development goals of systemd (the project) and sd-boot. As far as possible, generic interfaces are used. Starting from the bottom: the boot loader specification is designed to be completely generic and independent of the boot loader and independent of the userspace tools used to configure the boot loader. Similarly, most bootctl commands are implemented using generic functionality (either EFI or any bootloader implementing the boot loader specification). And stuff like 'bootctl status' and 'systemd-analyze blame' use interfaces that are completely generic and any bootload can provide the appropriate information to the operating system (see https://systemd.io/BOOT_LOADER_INTERFACE/). So bootctl is NOT coupled to sd-boot, except for some specific parts explicitly documented to be about sd-boot, currently the install/update/uninstall verbs and random-seed support. > * sd-boot is part of the systemd source tree > > The first problem is mostly because I think bootctl(1) is a fantastic > interface to manage *any* Bootloader Spec[4] (BLS) conformant boot > manager, and I would love for that tool to be useful for doing so. > Being able to do things like install/upgrade/swap GRUB 2, > systemd-boot, or any other registered BLS-enabled boot manager would > make it tremendously useful and valuable as a "building block" tool. I > feel it would make sense to offer some kind of configuration to teach > bootctl(1) about these boot managers so that it can work for them, and > not just systemd-boot. bootctl could be taught to install other EFI stuff. It'd probably be a matter of specifying a different glob when looking for files to copy. I'm not sure if we want to get into the business of installing non-EFI stuff… What exactly do you have in mind? > The second problem is because having sd-boot in the systemd source > tree means that in order for Fedora to sign the boot manager EFI > binaries, we have to lock down the systemd source package to the > secure boot Koji build channel. This is unequivocally unacceptable > from my point of view, as the restrictions around the secure boot > channel make it realistically impossible for community contribution > and maintenance of the package. Do you have more information about "secure boot Koji build channel"? I was asking around, and I learnt that I need to read up on "pesign", but not much beyond that. What restrictions does it place on the build? > Realistically, I think if we want to make movement on making > systemd-boot fully supported in Fedora, the systemd-boot boot manager > code itself should be split out into its own repository with its own > release cadence, while bootctl(1) and related infrastructure remains > in the systemd source tree and evolves to be able to manage arbitrary > BLS-conformant boot managers. This is not so simple. systemd-boot shared source code with the bootctl and other parts of the tree (see src/fundamental/), and in general we want to move towards not having seperate implementations, but to share as much code as possible. Having this all in one tree makes things much easier. Splitting out to a separate repo would make development much harder and is somethign that we'd very much want to avoid. > This would also (hopefully) encourage other boot managers to support > the Bootloader Spec configuration model, making it succeed as a > semi-universal abstraction for boot manager configuration. We could > then teach our tooling in Fedora to interact with bootctl(1) to do > bootloader things, rather than having to create multiple tools and >
Re: [systemd-devel] Splitting sd-boot from systemd/bootctl for enabling sd-boot in Fedora
On Tue, 2022-04-26 at 19:12 -0400, Neal Gompa wrote: > Hey all, > > Some of you might know about the recent discussion in Fedora about > dropping BIOS support[1][2]. While the end result for now is that > we're not dropping it[3], several side discussions involved enabling > systemd-boot as an option in Fedora in the future. > > While I *personally* am not a huge fan of systemd-boot itself, I *am* > a fan of a lot of the mechanisms around it, and I think it would be > valuable for us to adopt more of that in Fedora. To that end, that > means making it easier for people to fully adopt systemd-boot on their > systems in Fedora with minimal effort (ideally just a kickstart or > Anaconda flag if desired). > > From my point of view as someone working on several Fedora variants > and would like to provide more optionality around this, there are a > couple of issues: > > * bootctl(1) appears to be tightly coupled to sd-boot > * sd-boot is part of the systemd source tree > > The first problem is mostly because I think bootctl(1) is a fantastic > interface to manage *any* Bootloader Spec[4] (BLS) conformant boot > manager, and I would love for that tool to be useful for doing so. > Being able to do things like install/upgrade/swap GRUB 2, > systemd-boot, or any other registered BLS-enabled boot manager would > make it tremendously useful and valuable as a "building block" tool. I > feel it would make sense to offer some kind of configuration to teach > bootctl(1) about these boot managers so that it can work for them, and > not just systemd-boot. > > The second problem is because having sd-boot in the systemd source > tree means that in order for Fedora to sign the boot manager EFI > binaries, we have to lock down the systemd source package to the > secure boot Koji build channel. This is unequivocally unacceptable > from my point of view, as the restrictions around the secure boot > channel make it realistically impossible for community contribution > and maintenance of the package. > > Realistically, I think if we want to make movement on making > systemd-boot fully supported in Fedora, the systemd-boot boot manager > code itself should be split out into its own repository with its own > release cadence, while bootctl(1) and related infrastructure remains > in the systemd source tree and evolves to be able to manage arbitrary > BLS-conformant boot managers. > > This would also (hopefully) encourage other boot managers to support > the Bootloader Spec configuration model, making it succeed as a > semi-universal abstraction for boot manager configuration. We could > then teach our tooling in Fedora to interact with bootctl(1) to do > bootloader things, rather than having to create multiple tools and > scripts to deal with this. > > The alternative, of course, is to build sd-boot by having a second > source package of the systemd code and setting it up to only build the > boot stuff. This is painful for a variety of reasons: it guarantees we > need to have some kind of synchronization point to ensure fixes and > improvements are carried between the two. It is more work from a > maintenance perspective (especially around security stuff), and it > doesn't really help with pushing the adoption of the Bootloader Spec > as a whole. > > What do you all think? I can't comment on the first point as I don't use/work on bootctl/sd- boot, but the second one is going to be challenging, after all sd-boot was merged in the tree explicitly to keep it with the rest of the system. We like our monorepos here :-) But having a separate build shouldn't be too much of hassle I think? On OBS it would work great as a _multibuild package. With Meson 0.60 you can even have separate install "tags", so the following is very very quick, you don't need to rebuild the world (and hence have dependency tracking in the spec file, etc): $ meson foo <...> $ cd foo $ ninja src/boot/efi/linuxx64.efi.stub [21/21] Generating src/boot/efi/linuxx64.efi.stub with a custom command $ ninja src/boot/efi/systemd-bootx64.efi [10/10] Generating src/boot/efi/systemd-bootx64.efi with a custom command $ DESTDIR=/tmp/foo meson install --tags sd-boot --no-rebuild Installing src/boot/efi/systemd-bootx64.efi to /tmp/foo/usr/lib/systemd/boot/efi Installing src/boot/efi/linuxx64.elf.stub to /tmp/foo/usr/lib/systemd/boot/efi Installing src/boot/efi/linuxx64.efi.stub to /tmp/foo/usr/lib/systemd/boot/efi (note 'install --tags' requires https://github.com/systemd/systemd/pull/23204 ) Would this be a feasible approach? -- Kind regards, Luca Boccassi signature.asc Description: This is a digitally signed message part
Re: [systemd-devel] Q: non-ASCII in syslog
On Wed, Apr 27, 2022 at 10:09 AM Ulrich Windl < ulrich.wi...@rz.uni-regensburg.de> wrote: > Hi! > > Having written an RFC 3164 compatible syslog daemon, I noticed that systemd > created syslog messages with non-ASCII characters. > The problem is that a remote syslogd can hardly guess the correct character > set (I'm using rsyslog to forward local messages to a remote server). > > Example of such message: > systemd-tmpfiles[3311]: [/usr/lib/tmpfiles.d/svnserve.conf:1] Line > references > path below legacy directory /var/run/, updating /var/run/svnserve → > /run/svnserve; please update the tmpfiles.d/ drop-in file accordingly. > > (The arrow is encoded as three bytes (\xe2\x86\x92)) > > RFC 5425 syslog messages require the use of a BOM (%xEF.BB.BF) at the > beginning of a message if the message used UTF-8: > > MSG = MSG-ANY / MSG-UTF8 > MSG-ANY = *OCTET ; not starting with BOM > MSG-UTF8= BOM UTF-8-STRING > BOM = %xEF.BB.BF > > Wouldn't it make sense to add such a BOM for RFC 3164 syslog messages also > if > non-ASCII (i.e.: UTF-8) encoded characters are used? > RFC 3164 over a local socket from journald to local rsyslogd? The local rsyslogd already knows if messages are UTF-8 because the system's $LANG (well, nl_langinfo) says so. And if rsyslog can't trust that for some reason (e.g. because a user might have a different locale), then systemd-journald won't be able to trust it either, so it won't know whether it could add the BOM. RFC 3164 over the network to a remote server? Outside the scope for systemd, since it doesn't generate the network packets; your local rsyslogd forwarder does. (Also, why RFC 3164 and not 5425?) Generally, if a message successfully decodes as UTF-8 then it's most likely actual UTF-8 (and if UTF-8 decode fails then you fall back to ISO8859-1). Various old systems get away with this without needing a UTF-8 BOM. -- Mantas Mikulėnas
[systemd-devel] Q: non-ASCII in syslog
Hi! Having written an RFC 3164 compatible syslog daemon, I noticed that systemd created syslog messages with non-ASCII characters. The problem is that a remote syslogd can hardly guess the correct character set (I'm using rsyslog to forward local messages to a remote server). Example of such message: systemd-tmpfiles[3311]: [/usr/lib/tmpfiles.d/svnserve.conf:1] Line references path below legacy directory /var/run/, updating /var/run/svnserve → /run/svnserve; please update the tmpfiles.d/ drop-in file accordingly. (The arrow is encoded as three bytes (\xe2\x86\x92)) RFC 5425 syslog messages require the use of a BOM (%xEF.BB.BF) at the beginning of a message if the message used UTF-8: MSG = MSG-ANY / MSG-UTF8 MSG-ANY = *OCTET ; not starting with BOM MSG-UTF8= BOM UTF-8-STRING BOM = %xEF.BB.BF Wouldn't it make sense to add such a BOM for RFC 3164 syslog messages also if non-ASCII (i.e.: UTF-8) encoded characters are used? systemd in use is systemd-228-157.38.4 of SLES12 SP5... Regards, Ulrich
