If we're Going to Have Alternate Init Systems, we need to Understand Apt Dependencies
> "Ansgar" == Ansgar writes: Ansgar> Adam Borowski writes: >> * dependencies on "systemd | other" rather than "other | >> systemd"; this is a no-op on a systemd system (installed by >> debootstrap before any non-base packages) but causes apt to force >> an init+rc switch elsewhere Ansgar> It's very likely not a no-op on systemd systems as Ansgar> significantly more people somehow got systemd-shim installed Ansgar> than had sysvinit-core, see for example [1] which shows that Ansgar> somehow the "no-op" results in systemd-shim getting Ansgar> installed (which caused problems in the past). Ansgar> Just because you don't observe unwanted behavior happening Ansgar> right now on your system doesn't imply it doesn't exist. Ansgar> And the unwanted behavior that you say wouldn't happen (as Ansgar> it is supposedly a "no-op") happens on a scale larger than Ansgar> the entire sysvinit user base here... Ansgar, you keep bringing this issue up. And it keeps coming up as "Stuff might happen that we don't really understand." That's deeply unsatisfying to hear. And I think it's deeply unsatisfying to you when you hear that people talk about playing alternative games and assuming it's just going to work out. But this issue has kind of reached the level of FUD on both sides. In that it's really hard to respond to "bad stuff might happen," and yet you've also presented evidence that there's something that needs to be considered. I think the way forward is to actually try and get people to explore what the issues are and to write them up for all of us. And once we've done that, assuming that we've done a credible job of trying to research it, trust our conclusions. Yeah, we might introduce bugs and have to revise those conclusions. But saying "something bad might happen but we don't know what," is really frustrating to hear. I do understand it's also frustrating when you keep bringing up a real concern and it is dismissed. have stron If one of the options that promotes alternatives wins, I think it's important to do this work. I think the work would be valuable regardless of which option wins, but especially if we're going to continue to support alternate init systems.
Re: If we're Going to Have Alternate Init Systems, we need to Understand Apt Dependencies
Hi Sam, On Mon, 2019-12-02 at 15:25 -0500, Sam Hartman wrote: > > > > > > "Ansgar" == Ansgar writes: > > Ansgar> Adam Borowski writes: > >> * dependencies on "systemd | other" rather than "other | > >> systemd"; this is a no-op on a systemd system (installed by > >> debootstrap before any non-base packages) but causes apt to force > >> an init+rc switch elsewhere > > Ansgar> It's very likely not a no-op on systemd systems as > Ansgar> significantly more people somehow got systemd-shim installed > Ansgar> than had sysvinit-core, see for example [1] which shows that > Ansgar> somehow the "no-op" results in systemd-shim getting > Ansgar> installed (which caused problems in the past). > > Ansgar> Just because you don't observe unwanted behavior happening > Ansgar> right now on your system doesn't imply it doesn't exist. > Ansgar> And the unwanted behavior that you say wouldn't happen (as > Ansgar> it is supposedly a "no-op") happens on a scale larger than > Ansgar> the entire sysvinit user base here... > > Ansgar, you keep bringing this issue up. Yes, I bring this up as at is one of the few concrete examples where we have alternatives that must be chosen at build time. I would like to see if the proposed resolution would help if similar issues would come up again. > And it keeps coming up as "Stuff might happen that we don't really > understand." For one of the problems (apt making unexpected decisions) that is pretty close to what is the case. We do find such issues again and again, including too late, i.e. only after a stable release, also for other packages that aren't related to init systems. > That's deeply unsatisfying to hear. > And I think it's deeply unsatisfying to you when you hear that people > talk about playing alternative games and assuming it's just going to > work out. Yes, I now understand that the proponent of proposal D would just dismiss concerns as "theoretical degradations" without even trying to understand the problem or even getting to a point where one would look at weighting different trade-offs to get a compromise solution. In this discussion I provided a popcon graph which shows that systemd- shim has significantly more installations than sysvinit-core; if you prefer inline numbers: for stretch there are 3335 systemd-shim installations and 775 sysvinit-core installations. So it looks like something isn't quite as wanted. (And likely not all 775 sysvinit-core installations even have systemd-shim, so the unwanted case is even a bit larger; I'm too lazy to check that right now.) But even getting some people to acknowledge this seems very hard. Please note there was also a tech-ctte bug about the "no-op" dependency on "systemd-shim | systemd-sysv" causing problems beyond theoretical degradations previously (#883573). But still this is given as an example of "changes I view as done with spite, which bring no or negligible benefit to systemd yet large detriment to other rc systems"[1]. So I believe proposal D wouldn't help with resolving conflicts at all and doesn't provide improvements on this front. I also tried some related things, including some proposals to improve systemd support via Policy proposals that don't even degrade support for sysvinit such as recommending to always include native systemd units where approporiate. That now went back to someone questioning whether one should recommend to (almost) *never* include native systemd units instead... That is not very motivating and I fear that proposal D will result in that happening for any "non-init-related declarative systemd facility" as well. That is no improvement over the status quo, but additional obstacles such as mandatory waiting times. The "being excellent to each other" part of the proposal also leaves a sour impression if the proponent doesn't seem interested to hold himself to his own requirements as you pointed out in [2] (and other people did so before), but asks these to be uphold by others. Altogether this convinced me to likely rate "D" and some other choices below "FD". I don't think they'll bring us forward, even if on paper they might look like they do. That's a change from my position in the 2014 GR where I ranked the option that included "Debian packages may require a specific init system [...] if [...] and no patches or other derived works exist in order to support other init systems in such a way to render software usable to the same extent" first (but which was defeated by the option having less requirements to not provide support for sysvinit). Ansgar [1]: https://lists.debian.org/debian-vote/2019/11/msg00386.html [2]: https://lists.debian.org/debian-vote/2019/12/msg00064.html
Re: If we're Going to Have Alternate Init Systems, we need to Understand Apt Dependencies
Hi, On Wed, Dec 04, 2019 at 04:43:39PM +0100, Ansgar wrote: > For one of the problems (apt making unexpected decisions) that is > pretty close to what is the case. We do find such issues again and > again, including too late, i.e. only after a stable release, also for > other packages that aren't related to init systems. The experience is the same on the other side of the pond. I think the problem is the transition process that forces us to make dependencies weaker than they should be to have a satisfactory result. One of the options I had in my original proposal was that we could drop the requirement for transitions through apt, and instead provide transition scripts that use dpkg's --force options to go through an invalid state instead of requiring all intermediate states to be valid. > In this discussion I provided a popcon graph which shows that systemd- > shim has significantly more installations than sysvinit-core; if you > prefer inline numbers: for stretch there are 3335 systemd-shim > installations and 775 sysvinit-core installations. So it looks like > something isn't quite as wanted. (And likely not all 775 sysvinit-core > installations even have systemd-shim, so the unwanted case is even a > bit larger; I'm too lazy to check that right now.) But even getting > some people to acknowledge this seems very hard. Could these be upstart based? IIRC installing systemd-sysv should deinstall systemd-shim, so the obvious broken configuration is locked out. > I also tried some related things, including some proposals to improve > systemd support via Policy proposals that don't even degrade support > for sysvinit such as recommending to always include native systemd > units where approporiate. That now went back to someone questioning > whether one should recommend to (almost) *never* include native systemd > units instead... I'd be in favour of always including units along with init scripts, ideally as a strong requirement so we can disable the init script compatibility layer in systemd, and the same for cron files and timer units. Guaranteeing that the system will only ever consume one of the two files would allow us to drop a lot of ugly hacks in individual packages, but at the price of a big ugly hack in the compatibility generators (which we still need to provide for users who want to run non-Debian software). Simon
Re: If we're Going to Have Alternate Init Systems, we need to Understand Apt Dependencies
On Wed, Dec 04, 2019 at 06:04:19PM +0100, Simon Richter wrote: > One of the options I had in my original proposal was that we could drop the > requirement for transitions through apt, and instead provide transition > scripts that use dpkg's --force options to go through an invalid state > instead of requiring all intermediate states to be valid. I think apt already does that sometimes? > > In this discussion I provided a popcon graph which shows that systemd- > > shim has significantly more installations than sysvinit-core; if you > > prefer inline numbers: for stretch there are 3335 systemd-shim > > installations and 775 sysvinit-core installations. So it looks like > > something isn't quite as wanted. (And likely not all 775 sysvinit-core > > installations even have systemd-shim, so the unwanted case is even a > > bit larger; I'm too lazy to check that right now.) But even getting > > some people to acknowledge this seems very hard. > > Could these be upstart based? I doubt that, looking at the popcon data for upstart itself. Though I don't know how to get the data for stretch specifically. > > I also tried some related things, including some proposals to improve > > systemd support via Policy proposals that don't even degrade support > > for sysvinit such as recommending to always include native systemd > > units where approporiate. That now went back to someone questioning > > whether one should recommend to (almost) *never* include native systemd > > units instead... > > I'd be in favour of always including units along with init scripts, ideally > as a strong requirement so we can disable the init script compatibility > layer in systemd, and the same for cron files and timer units. The relevant policy bug is #941198. -- WBR, wRAR signature.asc Description: PGP signature
Re: If we're Going to Have Alternate Init Systems, we need to Understand Apt Dependencies
Hi, On Wed, Dec 04, 2019 at 10:24:40PM +0500, Andrey Rahmatullin wrote: > > One of the options I had in my original proposal was that we could drop the > > requirement for transitions through apt, and instead provide transition > > scripts that use dpkg's --force options to go through an invalid state > > instead of requiring all intermediate states to be valid. > I think apt already does that sometimes? Yes, but as far as I've understood that is a different resolver than the regular dependency solver. The regular solver avoids including operations in the solution that require too many follow-up operations to get back into a valid state, because searching the solution space there takes a long time. The aptitude resolver can do that, but it can go into a state where it needs to make a lot of attempts to find a solution. This is what I described in my earlier mail to -devel about libgtk pulling in systemd-sysv through dbus-user-session. The alternative using dbus-x11 exists, but the regular apt resolver doesn't find it, while aptitude does after about thirty seconds. The resolver that avoids dpkg errors reorders operations to remove invalid states, and when it finds that it cannot avoid going through one, it grumbles and does so -- which is why once you give apt the solution "maybe if you install dbus-x11", it will come up with a working plan. > > I'd be in favour of always including units along with init scripts, ideally > > as a strong requirement so we can disable the init script compatibility > > layer in systemd, and the same for cron files and timer units. > The relevant policy bug is #941198. I see no problem with going forward with this change, because it is not going to be affected by GR outcome. We want to be able to have systemd unit files for services in any case. Simon
Re: If we're Going to Have Alternate Init Systems, we need to Understand Apt Dependencies
On 2019/12/04 19:11, Svante Signell wrote: > I've purposely kept out of this discussion, hoping that you all can > behave in a civil manner. Obviously not. I don't rank you mail > defective, there have bee several other on this list. Anyway, this > whole GR is about systemd or sysvinit, and everybody pretends they > don't know about alternatives, like OpenRC, initng, runit, monit, s6, > daemontools, and especially Shepherd. Are you all blind to Free > Software progressing steadily, in spite of something that would hurt > Debian as a distribution for many years to come? I don't believe that that kind of tone is welcome on this list. I understand how you could feel that way, but if you read a bit closer you would see that openrc, runit and other init systems have come up multiple times on this list and on debian-devel recently. A few people have mentioned that sysvinit scripts come up in discussion so much because they tend to be a common denominator that can be used across init systems as a fallback, the people who refer to sysvinit scripts in such a fashion do not intend to imply that the alternative to systemd should be sysvinit per sé. If you look at the current proposals[1], none of the options explicitly mention sysvinit, it talks about systemd and other init systems, I doubt it's at all necessary to mention all of them by name. Anyone who cares about init systems other than systemd probably already uses one or more of those. -Jonathan [1] https://www.debian.org/vote/2019/vote_002 -- ⢀⣴⠾⠻⢶⣦⠀ Jonathan Carter (highvoltage) ⣾⠁⢠⠒⠀⣿⡁ Debian Developer - https://wiki.debian.org/highvoltage ⢿⡄⠘⠷⠚⠋ https://debian.org | https://jonathancarter.org ⠈⠳⣄ Be Bold. Be brave. Debian has got your back.
Re: If we're Going to Have Alternate Init Systems, we need to Understand Apt Dependencies
On Wed, 2019-12-04 at 18:04 +0100, Simon Richter wrote: > On Wed, Dec 04, 2019 at 04:43:39PM +0100, Ansgar wrote: > > For one of the problems (apt making unexpected decisions) that is > > pretty close to what is the case. We do find such issues again and > > again, including too late, i.e. only after a stable release, also for > > other packages that aren't related to init systems. > > The experience is the same on the other side of the pond. Well, as I said it's a problem with packages entire unrelated to init systems as well (and the problems also don't involve init systems there). The dependencies are just very complex and depending on the set of installed packages apt might give very unexpected (and unwanted) results. To avoid unwanted switching between already installed init systems, adding "Important: yes" to init systems (systemd-sysv, sysvinit-core) has been suggested, but this would cause apt to be very unhappy about intentionally switching to a different init system (similar to uninstalling essential packages) and wasn't implemented for that reason. Currently "init" has this flag to prevent unintentional removing all supported init systems. > I think the problem is the transition process that forces us to make > dependencies weaker than they should be to have a satisfactory result. Which transition? Intentionally switching init systems in an installation? > > In this discussion I provided a popcon graph which shows that systemd- > > shim has significantly more installations than sysvinit-core; if you > > prefer inline numbers: for stretch there are 3335 systemd-shim > > installations and 775 sysvinit-core installations. So it looks like > > something isn't quite as wanted. (And likely not all 775 sysvinit-core > > installations even have systemd-shim, so the unwanted case is even a > > bit larger; I'm too lazy to check that right now.) But even getting > > some people to acknowledge this seems very hard. > > Could these be upstart based? IIRC installing systemd-sysv should deinstall > systemd-shim, so the obvious broken configuration is locked out. No, upstart has popcon 33 across all releases. Practically everyone has systemd-sysv or sysvinit-core (99.5% in stretch; some users might not have any init system installed and/or installed one outside the package system; some popcon submissions are also just corrupted). systemd-sysv doesn't have a conflict with systemd-shim in stretch. That was only added in buster (after systemd-shim got incompatible with systemd and was eventually removed). You can see the bend in systemd- shim's popcon release after the buster release. It also doesn't look like people transitioning from sysvinit-core to systemd-sysv that kept -shim installed as systemd-shim popcon increases after sysvinit-core already shrunk and stayed stagnant for a while (see [1]). [1]: https://qa.debian.org/popcon-graph.php?packages=systemd-shim+sysvinit-core&show_installed=on&want_legend=on&want_ticks=on&date_fmt=%25Y-%25m&beenhere=1 > I'd be in favour of always including units along with init scripts, ideally > as a strong requirement so we can disable the init script compatibility > layer in systemd, and the same for cron files and timer units. It would be nice to always have systemd service files for various reasons. I don't think we can remove systemd's sysv-compat layer soon (if we wanted to, probably bookwork would be earliest realistic target). For cron/.timer files one could extend crontab syntax to include some marker that some things should not be run under systemd, same for scripts in cron.daily. But it's always a bit of a hack... Anyway, a bit far from the current topic. Ansgar
Re: If we're Going to Have Alternate Init Systems, we need to Understand Apt Dependencies
> "Svante" == Svante Signell writes: Svante> Jonathan, FYI: From a mail From Uoti Urpala: Svante> https://lists.debian.org/debian-vote/2019/12/msg00054.html That mail had unfortunate tone and several people replied to the thread indicating that the approach taken was not appropriate for Debian lists. >> I don't believe that that kind of tone is welcome on this list. I Svante> Again, systemd versus sysvinit is not the real issue. It is Svante> about systemd versus _any_ alternative. Thank you for this point. It is appreciated, as was your long list of init systems to think about. Svante> And don't talk about Svante> tone, look at this mail list archive, one contribution Svante> quoted above. This however is not welcome, nor was >I've purposely kept out of this discussion, hoping that you all can >behave in a civil manner. Obviously not. I appreciate that you'd like to talk about alternatives other than sysvinit. That's great. We welcome that *if* you can do so in a respectful manner. There are several of us who would be happy to help you figure out how to do that more effectively. --Sam
Re: If we're Going to Have Alternate Init Systems, we need to Understand Apt Dependencies
[2019-12-04 18:11] Svante Signell > Hello, Hello. > I've purposely kept out of this discussion, hoping that you all can > behave in a civil manner. Obviously not. I don't rank you mail > defective, there have bee several other on this list. Anyway, this > whole GR is about systemd or sysvinit, and everybody pretends they > don't know about alternatives, like OpenRC, initng, runit, monit, s6, > daemontools, and especially Shepherd. Are you all blind to Free > Software progressing steadily, in spite of something that would hurt > Debian as a distribution for many years to come? We are not blind. We know of alternatives, and I even work on advancing one of them. But right now, there is only two systems in Debian that are actually supported by packages that provide daemons -- systemd and sysvinit. Other systems, openrc/runit in particular, in most cases just fall-back on init.d scripts. If we succeed at protecting init.d scripts, it will be feasible to develop support for other init systems gradually, package after package. Should we fail, introduction of new init system will require either introduction of native support into ~1300 packages at same time or use of systemd files as fallback, which means inheriting huge complexity. In other words, death of init.d scripts means end of all hope. -- Note, that I send and fetch email in batch, once in a few days. Please, mention in body of your reply when you add or remove recepients.
Re: If we're Going to Have Alternate Init Systems, we need to Understand Apt Dependencies
On December 7, 2019 7:26:14 PM UTC, Dmitry Bogatov wrote: > >If we succeed at protecting init.d scripts, it will be feasible to >develop support for other init systems gradually, package after >package. > >Should we fail, introduction of new init system will require either >introduction of native support into ~1300 packages at same time or use >of systemd files as fallback, which means inheriting huge complexity. > >In other words, death of init.d scripts means end of all hope. There are definitely advantages to having each daemon ship its own startup scripts/control files, and that's how Debian has done it for as long as I can remember, but it seems like there is an alternative — have them provided by a different package. Probably one package providing quite a few of them. It'd need some way to only try to start installed daemons, but that sounds solvable. Advantage being that partial support (200 common daemons, not all 1300) would be possible. And it'd be independent (so anyone running systemd would just ignore its presence in the archive), and wouldn't need cooperation from the maintainers of the other packages. Wouldn't help with programs depending on e.g., systemd-logind, of course. But possibly there is hope for other init systems even if one of the systemd-all-the-way options wins.
Re: If we're Going to Have Alternate Init Systems, we need to Understand Apt Dependencies
On Sun, 08 Dec 2019 at 09:37:07 +, Anthony DeRobertis wrote: > it seems like there is an alternative — have them provided by > a different package. Probably one package providing quite a few of > them. It'd need some way to only try to start installed daemons, but > that sounds solvable. Not only solvable, but already required to be solved: any LSB init script that doesn't start with something like test -x "$DAEMON" || exit 0 (or use init-d-script(5) which does this for you) is already (RC-?)buggy, because LSB init scripts are conffiles, and conffiles usually get left behind when a package is removed but not purged. smcv
Re: Re: If we're Going to Have Alternate Init Systems, we need to Understand Apt Dependencies
Hello, I've purposely kept out of this discussion, hoping that you all can behave in a civil manner. Obviously not. I don't rank you mail defective, there have bee several other on this list. Anyway, this whole GR is about systemd or sysvinit, and everybody pretends they don't know about alternatives, like OpenRC, initng, runit, monit, s6, daemontools, and especially Shepherd. Are you all blind to Free Software progressing steadily, in spite of something that would hurt Debian as a distribution for many years to come? Thank you all, (Sam you do really rush things unnecessarily too much)
Re: Re: If we're Going to Have Alternate Init Systems, we need to Understand Apt Dependencies
Jonathan, FYI: From a mail From Uoti Urpala: https://lists.debian.org/debian-vote/2019/12/msg00054.html fact: There is in practice no development of new alternative init systems happening, and no clear reason to believe that if it hypothetically did occur, there would be particular problems. Certainly there are no concrete problems in need of resolution. Did you see this mail? > I don't believe that that kind of tone is welcome on this list. I > understand how you could feel that way, but if you read a bit closer > you would see that openrc, runit and other init systems have come up > multiple times on this list and on debian-devel recently. A few > people have mentioned that sysvinit scripts come up in discussion so > much because they tend to be a common denominator that can be used > across init systems as a fallback, the people who refer to sysvinit > scripts in such a fashion do not intend to imply that the alternative > to systemd should be sysvinit per sé. Again, systemd versus sysvinit is not the real issue. It is about systemd versus _any_ alternative. And don't talk about tone, look at this mail list archive, one contribution quoted above. > If you look at the current proposals[1], none of the options > explicitly mention sysvinit, it talks about systemd and other init > systems, I doubt it's at all necessary to mention all of them by > name. Anyone who cares about init systems other than systemd probably > already uses one or more of those. Again, see above. And don't insult me, that is not polite. I've been a user, supporter and contributor to Debian for a very long time. Just take some time to search (in different forums), if you find the time to do that. Thanks!
