Bug#1039102: debian-policy: make systemd units mandatory for packages shipping system services

[Benda Xu]

Hi Luca,

Luca Boccassi  writes:

>> On Sun, 2023-06-25 at 18:51 +0100, Luca Boccassi wrote:
>> > Patch attached and pushed to
>> > https://salsa.debian.org/bluca/policy/-/commits/mandatory_units?ref_type=hea
>>
>> There is one part I think should not be changed: we currently don't
>> require integration with service management at all. That should
>> probably not be changed.  So please consider reverting
>>
>> +---
>> | Packages that include system services must include ``systemd`` service
>> +---
>>
>> to the original
>>
>> +---
>> | Packages that include system services should include ``systemd`` service
>> +---
>
> How about:
>
> -Packages that include system services should include ``systemd`` service
> -units to start or stop those services.
> +Packages shipping system services should integrate with service management.
> +If they choose to do so, they must include ``systemd`` service units to start
> +or stop those services.

I take care of packages that does not meet the proposed policy. And I
don't have a systemd test environment.  I am curious what is the
recommended way to go forward.

- upload a generator-converted .service without any test;
- set low-NMU to encourage interested party to upload a .service;
- leave the lack-of-systemd-service bug open until some one submit a
  patch or a merge request;
- you name it.

Cheers,
Benda

Bug#1039102: debian-policy: make systemd units mandatory for packages shipping system services

[Luca Boccassi]

On Sun, 25 Jun 2023 at 19:24, Luca Boccassi  wrote:
>
> On Sun, 25 Jun 2023 at 19:21, Russ Allbery  wrote:
> >
> > Luca Boccassi  writes:
> >
> > > systemd upstream will drop support for the transitional sysv generator
> > > in the near future. The transition is long finished, it's been at least
> > > a decade, and it's time for the tail of packages still shipping only
> > > init scripts but not units to be updated.
> >
> > Has there already been a mass bug filing for packages that ship init
> > scripts but not systemd unit files?
>
> It has not, I'll get that sorted first then, thanks

Done and done:

https://lists.debian.org/debian-devel/2023/06/msg00291.html
https://bugs.debian.org/cgi-bin/pkgreport.cgi?users=bl...@debian.org;tag=missing-systemd-service

Kind regards,
Luca Boccassi

Re: Bug#945269: debian-policy: packages should use tmpfiles.d(5) to create directories below /var

[RL]

Some grammer thoughts. Especially i suggest avoiding the word
'integrate' wherever possible

Luca Boccassi  writes:

> +Service Directories

> +
> +Init systems other than ``systemd`` should allow providing the same

"allow providing" should be "provide" (or "support"? or "allow [who] to
[do what]").

> +functionality as appropriate for each system, for example managing the
> +directories from the appropriate service startup configuration.
> +

does the first "system" mean "installation", "service" or is this "init
system"?


> +``tmpfiles.d`` snippets should be integrated in packages using
> automated shared

does "integrated in packages" mean "provided by packages"? (or "shipped"
is used below, but that is also jargony)

> +tooling rather than by manually writing package-specific code in maintainers
> +scripts. For example, packages built using ``debhelper`` should make use of 
> the
> +``dh_installtmpfiles`` addon.

Could be more direct, eg "packages should create ``tmpfiles.d`` snippets
using existing tooling instead of writing bespoke code in maintainer
scripts. For example, packages that use ``debhelper`` should use
``dh_installtmpfiles``."


> +Init systems are required to integrate with ``tmpfiles.d`` and run
> the service

does this use of "integrate" mean "support" (or "implement something
equivalent to"?)

(And - is this sentence a set of requirements for init systems rather
than an individual package?  - if so, it might be better in a separate
subsection as would have different audience.)

> +that applies them on boot, and regularly for cleanup purposes, depending on 
> the
> +appropriate package providing the appropriate implementation that best
> +integrates with each system.

i didnt understand this part - i wonder if this is trying to say several
requirements in one sentence: the init system needs to provide an
implementation of tmpfiles; the init system needs to run it on boot; the
init system needs to do 'cleanup' (of what?). 

> For example, ``systemd`` will make the reference
> +implementation available when its main package is installed. The 
> documentation
> +for the reference implementation, `systemd-tmpfiles,
> +`_
> +explains how to call the program so that the appropriate ``tmpfiles.d`` 
> snippets
> +are applied at the appropriate time.
> +

i got lost reading this bitwq, wasnt sure quite what "the program" and
"the reference implemetation" referring to here? and "the appropriate
time" maybe needs some more explanation.

> --- a/policy/ch-maintainerscripts.rst
> +++ b/policy/ch-maintainerscripts.rst


> +Instead, :ref:`s-tmpfiles.d` snippets should be shipped, with the ones 
> provided
> +by the upstream sources, if any, to be preferred over Debian-specific ones 
> when
> +possible.

does "with" mean "instead of" or "as well as" here?

Bug#1039102: debian-policy: make systemd units mandatory for packages shipping system services

[Simon McVittie]

On Sun, 25 Jun 2023 at 18:51:24 +0100, Luca Boccassi wrote:
> Tentatively this should happen within Trixie's development cycle. Of
> course it's free software and generators are not that difficult to
> maintain, so if someone wanted to lift the sysv generator out of the
> systemd repository and adapt it to be a standalone binary there's
> nothing stopping them. But I wouldn't want the systemd package to
> depend on such a backward compat tool, so packages needing this
> hyptothetical package should depend explicitly on it.

I think it's maybe worth mentioning (not in Policy, but in the mass
bug filing about deprecating the generator) that there is an immediate
concrete benefit of moving from the generated unit to a maintainer-written
unit: it lets systemd know whether the service is meant to be a one-time
action like /etc/init.d/sudo (Type=oneshot, RemainAfterExit=yes), or
a long-running daemon like /etc/init.d/ssh (Type=forking or similar,
RemainAfterExit=no).

systemd's generator can't know which one of those each service is meant
to be (because LSB init scripts don't distinguish), so it has to make
pessimistic assumptions that work but are non-ideal in both cases.

smcv

Bug#1039102: debian-policy: make systemd units mandatory for packages shipping system services

[Luca Boccassi]

On Sun, 25 Jun 2023 at 19:26, Ansgar  wrote:
>
> Hi,
>
> On Sun, 2023-06-25 at 18:51 +0100, Luca Boccassi wrote:
> > Patch attached and pushed to
> > https://salsa.debian.org/bluca/policy/-/commits/mandatory_units?ref_type=hea
>
> I support this as using the compat layer with systemd-sysv-generator
> has some limitations that confuse users (e.g., behavior of "start" as
> the unit stays running due to RemainAfterExit=yes set).  We should
> really aim at providing native systemd units.
>
> There is one part I think should not be changed: we currently don't
> require integration with service management at all. That should
> probably not be changed.  So please consider reverting
>
> +---
> | Packages that include system services must include ``systemd`` service
> +---
>
> to the original
>
> +---
> | Packages that include system services should include ``systemd`` service
> +---

How about:

-Packages that include system services should include ``systemd`` service
-units to start or stop those services.
+Packages shipping system services should integrate with service management.
+If they choose to do so, they must include ``systemd`` service units to start
+or stop those services.

Kind regards,
Luca Boccassi
From 73ff234d23464acdcc08c78312c267d51749c64e Mon Sep 17 00:00:00 2001
From: Luca Boccassi 
Date: Sun, 25 Jun 2023 18:42:29 +0100
Subject: [PATCH] system services: make systemd units mandatory

systemd upstream will drop support for the transitional sysv generator
in the near future. The transition is long finished, and it's time for
the tail of packages still shipping only init scripts but not units to
be updated.
---
 policy/ch-opersys.rst | 29 +
 1 file changed, 13 insertions(+), 16 deletions(-)

diff --git a/policy/ch-opersys.rst b/policy/ch-opersys.rst
index 207b3c0..af4b159 100644
--- a/policy/ch-opersys.rst
+++ b/policy/ch-opersys.rst
@@ -328,15 +328,12 @@ Starting system services
 Introduction
 
 
-Packages that include system services should include ``systemd`` service
-units to start or stop those services.  See :manpage:`systemd.service(5)`
-for details on the syntax of a service unit file.  In the common case that
-a package includes a single system service, the service unit should have
-the same name as the package plus the ``.service`` extension.
-
-If the package does not include a service unit (if, for example, no one
-has yet written one), including an init script, as described below, to
-start the service is encouraged.
+Packages shipping system services should integrate with service management.
+If they choose to do so, they must include ``systemd`` service units to start
+or stop those services.  See :manpage:`systemd.service(5)` for details on the
+syntax of a service unit file.  In the common case that a package includes a
+single system service, the service unit should have the same name as the
+package plus the ``.service`` extension.
 
 Packages including a service unit may optionally include an init script to
 support other init systems.  In this case, the init script should have the
@@ -345,13 +342,13 @@ it and use the service unit instead.  Packages may also support other init
 systems by including configuration in the native format of those init
 systems.
 
-If a service unit is not present, ``systemd`` uses dependency information
-contained within the init scripts and symlinks in ``/etc/rcn.d`` to decide
-which scripts to run and in which order.  The ``sysv-rc`` runlevel system
-for ``sysvinit`` uses the same symlinks in ``/etc/rcn.d`` to decide which
-scripts to run and in which order at boot time and when the init state (or
-"runlevel") is changed.  See the ``README.runlevels`` file shipped with
-``sysv-rc`` for implementation details.  Other alternatives might exist.
+``systemd`` uses dependency and ordering information contained within the
+enabled unit files to decide which services to run and in which order.
+The ``sysv-rc`` runlevel system for ``sysvinit`` uses symlinks in
+``/etc/rcn.d`` to decide which scripts to run and in which order at boot
+time and when the init state (or "runlevel") is changed.  See the
+``README.runlevels`` file shipped with ``sysv-rc`` for implementation details.
+Other alternatives might exist.
 
 The sections below describe how to write those scripts and configure those
 symlinks.
-- 
2.39.2

Bug#1039102: debian-policy: make systemd units mandatory for packages shipping system services

[Ansgar]

Hi,

On Sun, 2023-06-25 at 18:51 +0100, Luca Boccassi wrote:
> Patch attached and pushed to
> https://salsa.debian.org/bluca/policy/-/commits/mandatory_units?ref_type=hea

I support this as using the compat layer with systemd-sysv-generator
has some limitations that confuse users (e.g., behavior of "start" as
the unit stays running due to RemainAfterExit=yes set).  We should
really aim at providing native systemd units.

There is one part I think should not be changed: we currently don't
require integration with service management at all. That should
probably not be changed.  So please consider reverting

+---
| Packages that include system services must include ``systemd`` service
+---

to the original

+---
| Packages that include system services should include ``systemd`` service
+---

Ansgar

Bug#1039102: debian-policy: make systemd units mandatory for packages shipping system services

[Luca Boccassi]

On Sun, 25 Jun 2023 at 19:21, Russ Allbery  wrote:
>
> Luca Boccassi  writes:
>
> > systemd upstream will drop support for the transitional sysv generator
> > in the near future. The transition is long finished, it's been at least
> > a decade, and it's time for the tail of packages still shipping only
> > init scripts but not units to be updated.
>
> Has there already been a mass bug filing for packages that ship init
> scripts but not systemd unit files?

It has not, I'll get that sorted first then, thanks

Kind regards,
Luca Boccassi

Bug#1039102: debian-policy: make systemd units mandatory for packages shipping system services

[Russ Allbery]

Luca Boccassi  writes:

> systemd upstream will drop support for the transitional sysv generator
> in the near future. The transition is long finished, it's been at least
> a decade, and it's time for the tail of packages still shipping only
> init scripts but not units to be updated.

Has there already been a mass bug filing for packages that ship init
scripts but not systemd unit files?

> Tentatively this should happen within Trixie's development cycle. Of
> course it's free software and generators are not that difficult to
> maintain, so if someone wanted to lift the sysv generator out of the
> systemd repository and adapt it to be a standalone binary there's
> nothing stopping them. But I wouldn't want the systemd package to
> depend on such a backward compat tool, so packages needing this
> hyptothetical package should depend explicitly on it. This is just
> mentioned for completeness, it's been at least a decade and writing a
> unit file is beyond trivial so there shouldn't be any issue adding the
> few remaining ones.

> Once the policy is updated I plan to ask Lintian to bump the severity
> of the existing check:

> https://salsa.debian.org/lintian/lintian/-/merge_requests/407

Assuming the mass bug filing hasn't already happened and I missed it, I
think this is the wrong order.  This sort of large-scale breaking change
should always start with a mass bug filing against all affected packages.
I think the right process is:

* Raise this in debian-devel and propose a mass bug filing requiring all
  packages to add systemd unit files if they currently only have init
  scripts.  This gives people the opportunity to object or take over
  maintenance of the unit file generator and document how to depend on it
  if they wish to do that instead.  (I don't think that's a good idea, but
  we should let the discussion happen.)

* Unless something surprising happens in that discussion, do a mass bug
  filing for this transition and bump the Lintian severity at the same
  time.

* Once that has consensus and is underway, *then* change Policy to reflect
  this project decision.

If the mass bug filing already happened and I just didn't notice, my
apologies.

-- 
Russ Allbery (r...@debian.org)  

Bug#1039102: debian-policy: make systemd units mandatory for packages shipping system services

[Luca Boccassi]

Package: debian-policy
X-Debbugs-Cc: pkg-systemd-maintain...@lists.alioth.debian.org

systemd upstream will drop support for the transitional sysv generator
in the near future. The transition is long finished, it's been at least
a decade, and it's time for the tail of packages still shipping only
init scripts but not units to be updated.

Tentatively this should happen within Trixie's development cycle. Of
course it's free software and generators are not that difficult to
maintain, so if someone wanted to lift the sysv generator out of the
systemd repository and adapt it to be a standalone binary there's
nothing stopping them. But I wouldn't want the systemd package to
depend on such a backward compat tool, so packages needing this
hyptothetical package should depend explicitly on it. This is just
mentioned for completeness, it's been at least a decade and writing a
unit file is beyond trivial so there shouldn't be any issue adding the
few remaining ones.

Once the policy is updated I plan to ask Lintian to bump the severity
of the existing check:

https://salsa.debian.org/lintian/lintian/-/merge_requests/407

Patch attached and pushed to
https://salsa.debian.org/bluca/policy/-/commits/mandatory_units?ref_type=heads

-- 
Kind regards,
Luca Boccassi
From ea524f52d256e37b4e4747d7e6ac4f316c4805a8 Mon Sep 17 00:00:00 2001
From: Luca Boccassi 
Date: Sun, 25 Jun 2023 18:42:29 +0100
Subject: [PATCH] system services: make systemd units mandatory

systemd upstream will drop support for the transitional sysv generator
in the near future. The transition is long finished, and it's time for
the tail of packages still shipping only init scripts but not units to
be updated.
---
 policy/ch-opersys.rst | 20 
 1 file changed, 8 insertions(+), 12 deletions(-)

diff --git a/policy/ch-opersys.rst b/policy/ch-opersys.rst
index 207b3c0..3134783 100644
--- a/policy/ch-opersys.rst
+++ b/policy/ch-opersys.rst
@@ -328,16 +328,12 @@ Starting system services
 Introduction
 
 
-Packages that include system services should include ``systemd`` service
+Packages that include system services must include ``systemd`` service
 units to start or stop those services.  See :manpage:`systemd.service(5)`
 for details on the syntax of a service unit file.  In the common case that
 a package includes a single system service, the service unit should have
 the same name as the package plus the ``.service`` extension.
 
-If the package does not include a service unit (if, for example, no one
-has yet written one), including an init script, as described below, to
-start the service is encouraged.
-
 Packages including a service unit may optionally include an init script to
 support other init systems.  In this case, the init script should have the
 same name as the ``systemd`` service unit so that ``systemd`` will ignore
@@ -345,13 +341,13 @@ it and use the service unit instead.  Packages may also support other init
 systems by including configuration in the native format of those init
 systems.
 
-If a service unit is not present, ``systemd`` uses dependency information
-contained within the init scripts and symlinks in ``/etc/rcn.d`` to decide
-which scripts to run and in which order.  The ``sysv-rc`` runlevel system
-for ``sysvinit`` uses the same symlinks in ``/etc/rcn.d`` to decide which
-scripts to run and in which order at boot time and when the init state (or
-"runlevel") is changed.  See the ``README.runlevels`` file shipped with
-``sysv-rc`` for implementation details.  Other alternatives might exist.
+``systemd`` uses dependency and ordering information contained within the
+enabled unit files to decide which services to run and in which order.
+The ``sysv-rc`` runlevel system for ``sysvinit`` uses symlinks in
+``/etc/rcn.d`` to decide which scripts to run and in which order at boot
+time and when the init state (or "runlevel") is changed.  See the
+``README.runlevels`` file shipped with ``sysv-rc`` for implementation details.
+Other alternatives might exist.
 
 The sections below describe how to write those scripts and configure those
 symlinks.
-- 
2.39.2



signature.asc
Description: This is a digitally signed message part

Bug#945269: debian-policy: packages should use tmpfiles.d(5) to create directories below /var

[Luca Boccassi]

On Sun, 25 Jun 2023 at 16:52, Ansgar  wrote:
>
> Hi Luca,
>
> On Tue, 2023-06-20 at 22:53 +0100, Luca Boccassi wrote:
> > Russ, anything I've missed that you want me to change from the most
> > recent revision at
> > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=945269#160 ?
>
> I suggest a minor change:
>
> In the paragraph
>
> +---
> | Init systems other than ``systemd`` should allow providing the same
> | functionality as appropriate for each system, for example managing the
> | directories from the init script shipped by the package.
> +---
>
> change "init script shipped by the package" to "appropriate service
> startup configuration".
>
> This should address concerns raise on d-devel@ that some packages might
> not ship an init script. It also better covers alternative init systems
> that do something more interesting than just starting the same sysvinit
> scripts of old (not sure if any do).

Sure, updated as suggested.

Kind regards,
Luca Boccassi
From 8538f41dc235e9d9be4101595d469d389db693ef Mon Sep 17 00:00:00 2001
From: Luca Boccassi 
Date: Tue, 9 May 2023 01:38:13 +0100
Subject: [PATCH] Define service directories and tmpfiles.d interfaces and
 usage

---
 policy/ch-files.rst | 64 +
 policy/ch-maintainerscripts.rst |  7 
 2 files changed, 71 insertions(+)

diff --git a/policy/ch-files.rst b/policy/ch-files.rst
index b34c183..830f69c 100644
--- a/policy/ch-files.rst
+++ b/policy/ch-files.rst
@@ -722,6 +722,70 @@ The name of the files and directories installed by binary packages
 outside the system PATH must be encoded in UTF-8 and should be
 restricted to ASCII when it is possible to do so.
 
+.. _s-service-directories:
+
+Service Directories
+---
+
+Services might need auxiliary directories under ``/var/``, ``/run/`` or
+``/etc``. Instead of shipping empty directories in packages, or creating them
+with custom code in maintainer scripts, services should use ``systemd``'s native
+settings to ensure the required directories are created regardless of the
+privilege level under which the services are running. The relevant settings are
+`RuntimeDirectory=, StateDirectory=, CacheDirectory=, LogsDirectory= and
+ConfigurationDirectory=
+`_,
+covering respectively ``/run/``, ``/var/lib/``, ``/var/cache/``, ``/var/log/``
+and ``/etc/`` for system services, and the equivalent XDG-defined location for
+user services.
+
+Init systems other than ``systemd`` should allow providing the same
+functionality as appropriate for each system, for example managing the
+directories from the appropriate service startup configuration.
+
+.. _s-tmpfiles.d:
+
+tmpfiles.d
+--
+
+Packages might need additional files or directories to implement their
+functionality. Directories that are located under ``/var/`` or ``/etc/``, and
+files that are located under ``/var/``, should not be created manually via
+maintainer scripts, but instead be declaratively defined via the `tmpfiles.d
+`_ interface.
+Files and directories under ephemeral filesystems such as ``/tmp/`` may also be
+created and managed via ``tmpfiles.d`` snippets.
+
+When ownership of a directory can be clearly tied to a specific service,
+``Service Directories`` should be preferred to ``tmpfiles.d`` snippets.
+
+The ``tmpfiles.d`` file format is defined by the ``systemd`` project, and is
+guaranteed to be stable. Details about the syntax and installation paths are
+defined by its `reference implementation's documentation,
+`_ and will
+not be redefined here.
+
+``tmpfiles.d`` snippets should be usable on systems that do not boot (such as a
+very minimal chroot image), and also on systems booting with init systems other
+than ``systemd``.
+
+``tmpfiles.d`` snippets should be integrated in packages using automated shared
+tooling rather than by manually writing package-specific code in maintainers
+scripts. For example, packages built using ``debhelper`` should make use of the
+``dh_installtmpfiles`` addon. Packages shipping ``tmpfiles.d`` snippets should
+depend on the appropriate virtual packages in the following order:
+``default-systemd-tmpfiles | systemd-tmpfiles``.
+
+Init systems are required to integrate with ``tmpfiles.d`` and run the service
+that applies them on boot, and regularly for cleanup purposes, depending on the
+appropriate package providing the appropriate implementation that best
+integrates with each system. For example, ``systemd`` will make the reference
+implementation available when its main package is installed. The documentation
+for the reference implementation, `systemd-tmpfiles,
+`_
+explains how to call the program so that the appropriate ``tmpfiles.d`` snippets
+are applied at the

Bug#945269: debian-policy: packages should use tmpfiles.d(5) to create directories below /var

[Ansgar]

Hi Luca,

On Tue, 2023-06-20 at 22:53 +0100, Luca Boccassi wrote:
> Russ, anything I've missed that you want me to change from the most
> recent revision at
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=945269#160 ?

I suggest a minor change:

In the paragraph

+---
| Init systems other than ``systemd`` should allow providing the same
| functionality as appropriate for each system, for example managing the
| directories from the init script shipped by the package.
+---

change "init script shipped by the package" to "appropriate service
startup configuration".

This should address concerns raise on d-devel@ that some packages might
not ship an init script. It also better covers alternative init systems
that do something more interesting than just starting the same sysvinit
scripts of old (not sure if any do).

Ansgar