Proposal for next steps for systemd-related policy

[Russ Allbery]

Hi all,

This is a tentative proposal for next steps from a Policy standpoint given
the result of .  I thought it
might be helpful to lay out a possible way to sequence the work.

1. Downgrade the requirement to include an init script in a package with a
   systemd unit to "encouraged."  This is the direct outcome of the GR, so
   I think we should make this change before the next normative upload,
   since there's no point in Policy being inconsistent with the GR result.

   I think there's some discussion to be had here about whether the
   correct level is "encouraged" or "optional."  I'd also like to revise
   and merge my change that adds "encouraged" first, although if we decide
   "optional" is correct, that sequencing is, well, optional.

2. Do nothing further before January 6th.  It's still the holidays, and
   subsequent steps are going to require some discussion, and I think it's
   worth taking a breath.

3. Start a discussion on debian-devel to see if we can come up with some
   idea for how to declare dependencies on availability of system
   services.

   My thought process here is that while the GR permits packages to start
   using systemd facilities directly, doing that without somehow declaring
   that requirement in package metadata seems likely to cause bugs and
   upgrade issues, so we should try to provide some better facilities.  I
   think there's an obvious gap here where we need a mechanism to declare
   a dependency on a system facility (as distinct from a package that may
   be installed but not running) such as tmpfiles.d, sysusers.d, the
   ability to rely on DynamicUser without creating a user for that purpose
   in maintainer scripts, systemd D-Bus facilities, socket activation, or
   so forth.

   This would be a way to provide better UI when someone on a sysvinit
   system tries to install a package that requires a systemd facility (via
   an upgrade scenario, for example).  It also exposes in the archive what
   facilities are blocking use of packages on non-systemd systems, and
   thus provides a prioritized list of work for anyone who is pursuing
   exploration per paragraph two of the GR.

   This obviously is going to require some hashing out, and may well
   require support from the dpkg, apt, and aptitude teams.

4. Parallel to point 3, start fleshing out Policy recommendations for best
   practices for systemd units.  Some initial candidates for security
   hardening:

   - DynamicUser (*without* removing useradd, see below)
   - ProtectSystem
   - ProtectHome
   - PrivateTmp
   - PrivateDevices
   - SystemCallFilter

   We probably also want to recommend Type=notify where possible and
   Type=exec where not, over Type=forking, when the daemon supports that.
   OOMScoreAdjust may also be worth setting some policy around.  We
   should, of course, have an open discussion of other things that people
   would like to see documented as best practices.

   These are chosen because they only affect the unit file and don't add
   any additional assumptions on the availability of other services.

5. As the outcome of point 3 concludes, start documenting use of other
   desirable services that would allow simplification of Debian packages
   and allow package maintainers to use those services rather than the
   ones that Policy currently requires.  My personal priority list looks
   like (roughly in order):

   - DynamicUser without useradd of a system user
   - tmpfiles.d
   - sysusers.d
   - Timer units
   - Socket activation

   but we should have an open discussion of what other facilities people
   think Debian would benefit from.  I think we should prioritize the
   cases where it would be relatively straightforward for other init
   systems to provide the same functionality; for example, tmpfiles.d and
   sysusers.d can be supported by simply arranging to run the relevant
   systemd binaries at appropriate times.  DynamicUser is a bit harder,
   but I think the minimum functionality to let the package keep working
   could be added to start-stop-daemon or some wrapper that it execs.

   As time goes on, we'll get a better feel for how much work folks will
   be doing going forward on supporting other init systems, and thus on
   how quickly we should move versus giving them time to determine how
   they want to support equivalent functionality.

-- 
Russ Allbery (r...@debian.org)  

Re: Bug#944920: Revise terminology used to specify requirements

[Russ Allbery]

Paul Gevers  writes:
> On 21-11-2019 13:59, Paul Gevers wrote:

>> [Disclaimer: the words below are as a member of the release team, but
>> not necessarily those of the team. We haven't discussed this yet.]

> We have had a discussion, and there were no objections against my vision
> below.

>> I can envision that if Policy carries such a summary list, our policy
>> would mention the version of Policy it was based on, to make sure that
>> Policy doesn't suddenly change what we as the RT agreed on.

> So, yes, we would welcome the Policy to maintain a summary list that we
> could reference. We already acknowledge that there will be items in the
> Policy text that we balance differently for RC-ness, so there will be
> exceptions maintained by us.

Thanks, Paul!  This is now on me to compose this list, and I'll let you
know when that's done.  Once that's complete, we can do a reconciliation.
I'm inclined to downgrade Policy musts that the release team does not
consider likely to be release-critical in the future, for instance.

-- 
Russ Allbery (r...@debian.org)  

Re: Proposal for next steps for systemd-related policy

[Simon McVittie]

On Sun, 29 Dec 2019 at 10:47:44 -0800, Russ Allbery wrote:
>We probably also want to recommend Type=notify where possible and
>Type=exec where not, over Type=forking, when the daemon supports that.

I'm not sure what you mean by "where possible" - it'll usually be
possible to implement Type=notify (even if a libsystemd dependency is
unacceptable or not feasible in the implementation language, the protocol
is designed to be reimplementable) but it won't always be useful.

Type=exec doesn't provide a way for the daemon to signal that it has
opened listening sockets or done whatever else is necessary for units that
depend on it to be able to start without error, so daemons that might be
depended on by other units will normally need to use either Type=notify,
Type=dbus or Type=forking, then make sure they do basic setup (listen on
sockets, etc.) before they notify, request their D-Bus names or do the
traditional double-fork daemonization trick (whichever is appropriate
for their Type). However, I don't think this is necessarily in-scope
for Policy? Implementing readiness-signalling if required seems like
it might be the sort of "don't be buggy" requirement that shouldn't
necessarily need to be said explicitly.

I think perhaps a better recommendation might be that it's preferred for
systemd services that run daemons to run them as a foreground process
(without forking or "daemonizing" during startup), with a footnote or
parenthetical note saying that this recommendation is incompatible with
Type=forking, which should therefore only be used if readiness-signalling
is required and alternatives like the preferred Type=notify or Type=dbus
are unavailable?

I say "when run from a systemd service" so that services like dbus-daemon
meet this recommendation - by default it double-forks for compatibility
with LSB-style init scripts and its own historical behaviour, but the
systemd unit dbus.service runs "dbus-daemon --nofork" which disables the
double-fork code path, and I think that's a perfectly good implementation
of what we want.

>- DynamicUser without useradd of a system user

One warning for this one is that users referred to in dbus-daemon
system.d/*.conf snippets, typically ,
currently need to be pre-created by sysusers.d, useradd or equivalent
(because dbus-daemon resolves usernames to uids during its own
startup/reload, to populate internal data structures that are all designed
in terms of uids). Other references to users in non-D-Bus configuration
might well behave similarly.

smcv

Re: Proposal for next steps for systemd-related policy

[Russ Allbery]

Simon McVittie  writes:
> On Sun, 29 Dec 2019 at 10:47:44 -0800, Russ Allbery wrote:

>>We probably also want to recommend Type=notify where possible and
>>Type=exec where not, over Type=forking, when the daemon supports
>>that.

> I'm not sure what you mean by "where possible" - it'll usually be
> possible to implement Type=notify (even if a libsystemd dependency is
> unacceptable or not feasible in the implementation language, the
> protocol is designed to be reimplementable) but it won't always be
> useful.

I mean something relatively modest.  If upstream already supports
systemd's notification protocol, I think we should recommend that as the
best choice.  (I'm dubious about asking Debian packagers to add support.)

> Type=exec doesn't provide a way for the daemon to signal that it has
> opened listening sockets or done whatever else is necessary for units
> that depend on it to be able to start without error, so daemons that
> might be depended on by other units will normally need to use either
> Type=notify, Type=dbus or Type=forking, then make sure they do basic
> setup (listen on sockets, etc.) before they notify, request their D-Bus
> names or do the traditional double-fork daemonization trick (whichever
> is appropriate for their Type).

Ah, yes, it's a good point that Type=forking when properly implemented is
somewhat better than Type=exec on getting dependency ordering correct.
But this is the sort of nuance that I think we should document somewhere.

Getting Type=forking right is surprisingly complicated.  The best
documentation for how to do that I've seen is, ironically, in the systemd
documentation, in daemon(7).  If the daemon double-forks and exits and
then binds to network sockets and so forth, which is common, the
advantages over Type=exec are basically lost.

> However, I don't think this is necessarily in-scope for Policy?
> Implementing readiness-signalling if required seems like it might be the
> sort of "don't be buggy" requirement that shouldn't necessarily need to
> be said explicitly.

Right, what I think is in scope for Policy is advising packagers on which
readiness signaling mechanism to use if upstream supports several.  If one
is relatively new to packaging daemons, this may not be something on one's
radar to look at, but there are substantial, if subtle, advantages to
using Type=notify if upstream already supports it.  (Type=dbus is fine, of
course, and we probably shouldn't doument that as well, although my guess
is that things that register with D-Bus are more likely to already know
about these subtleties.)

> I say "when run from a systemd service" so that services like
> dbus-daemon meet this recommendation - by default it double-forks for
> compatibility with LSB-style init scripts and its own historical
> behaviour, but the systemd unit dbus.service runs "dbus-daemon --nofork"
> which disables the double-fork code path, and I think that's a perfectly
> good implementation of what we want.

Yes, completely agreed.

>>- DynamicUser without useradd of a system user

> One warning for this one is that users referred to in dbus-daemon
> system.d/*.conf snippets, typically ,
> currently need to be pre-created by sysusers.d, useradd or equivalent
> (because dbus-daemon resolves usernames to uids during its own
> startup/reload, to populate internal data structures that are all
> designed in terms of uids). Other references to users in non-D-Bus
> configuration might well behave similarly.

Right, there are a lot of subtleties for when DynamicUser can and can't be
used that we'll need to hash out and document.

-- 
Russ Allbery (r...@debian.org)  

Re: Proposal for next steps for systemd-related policy

[Russ Allbery]

Russ Allbery  writes:

> Right, what I think is in scope for Policy is advising packagers on
> which readiness signaling mechanism to use if upstream supports several.
> If one is relatively new to packaging daemons, this may not be something
> on one's radar to look at, but there are substantial, if subtle,
> advantages to using Type=notify if upstream already supports it.
> (Type=dbus is fine, of course, and we probably shouldn't doument that as
> well, although my guess is that things that register with D-Bus are more
> likely to already know about these subtleties.)

That was supposed to be "we probably should document that as well."  I
think I edited that sentence too much.  Sorry about the confusion.

-- 
Russ Allbery (r...@debian.org)