On Wed, 7 Jun 2023 at 11:46, Luca Boccassi <bl...@debian.org> wrote: > > On Wed, 7 Jun 2023 at 11:29, Simon McVittie <s...@debian.org> wrote: > > > > On Tue, 06 Jun 2023 at 20:40:52 -0700, Russ Allbery wrote: > > > Luca Boccassi <bl...@debian.org> writes: > > > > +Packages might need additional files or directories to implement their > > > > +functionality. Directories that are located under ``/var/`` or > > > > +``/etc/``, and files that are located under ``/var/``, must not be > > > > +created manually via maintainer scripts, but instead be declaratively > > > > +defined via the `tmpfiles.d > > > > +<https://www.freedesktop.org/software/systemd/man/tmpfiles.d.html>`_ > > > > +interface. > > > > > > This is an oddly specific list of directories and not at all the > > > directories that I would have expected to be handled by tmpfiles.d. > > > > Sorry, in my previous reading of this bug I had been concentrating on > > the mechanics of how to make tmpfiles.d(5) something that maintainers can > > rely on if it's convenient/helpful, and I'd missed that Luca is asking > > for its use to be mandatory in some cases. > > > > I would personally be inclined to concentrate on making tmpfiles.d(5) > > something that we can rely on and encourage the use of where appropriate, > > even on non-systemd systems, so that (upstream and downstream) maintainers > > can move towards it of their own accord because it's more convenient > > than other options, and put aside the question of making it generally a > > "should" or "must" for the moment. > > > > I believe (please correct me if I'm wrong) that Luca's intention here > > is that this is drawing a line between: > > > > - the static files of the OS: /usr and the /usr-like top-level directories > > (the ones that get merged by the /usr merge), which should be statically > > shipped in packages and managed by dpkg (or on "immutable" systems > > that use image-based/tree-based upgrades, maybe by ostree or casync > > or similar, from an tree originally constructed from dpkg packages) > > > > - this specific system's persistent state: /var and parts of /etc > > > > with the intention of eventually enabling functionality like being > > able to do a "factory reset" to the equivalent of a freshly installed > > system by deleting (most of) /etc and /var, rebooting, and letting the > > OS re-create them from a template below /usr; or doing the equivalent > > for individual packages by deleting only their part of /etc and /var. > > > > /etc is somewhere between static files and state, because traditionally > > it has been a mixture of files that the sysadmin or installer must provide > > (like /etc/passwd); configuration files that are shipped by a package and > > can be edited by the sysadmin (like /etc/systemd/logind.conf); and > > integration glue that links up one package with another, can in principle > > be edited by the sysadmin, but in practice is rarely edited > > (like /etc/profile.d/flatpak.sh). > > > > Various upstream projects including systemd have been trying to reduce > > the extent to which /etc and /var are included in the data.tar.* of a .deb > > or other packaging systems' equivalents, by moving the integration glue > > to a /usr-like directory (/lib/udev/rules.d, /usr/share/dbus-1/system.d), > > reserving the corresponding /etc directory for sysadmin configuration > > (/etc/udev/rules.d, /etc/dbus-1/system.d), and providing a way for the > > sysadmin to "mask" any integration files they want the system to ignore. > > > > If we disregard conffiles and configuration files in /etc for the > > moment, there are basically three ways for a package to get a file onto > > the running system: > > > > - ship it in the data.tar.* of a .deb > > - create it from a maintainer script and also during boot > > - maybe via tmpfiles.d(5) > > - or maybe open-coded > > - have the package create it at runtime, on-demand > > - this clearly doesn't work if the package's code runs unprivileged > > and relies on root having created a directory for it already > > > > For /usr and the /usr-like directories, shipping files in the .deb is by > > far the most common, although a few packages need to create files here > > via maintainer scripts or triggers (for example > > /usr/lib/x86_64-linux-gnu/gio/modules/giomodule.cache which is a summary > > of files created by multiple packages, and is updated whenever those > > packages are added, removed or changed). > > > > For /run, /tmp and /var/tmp, I think there's consensus that shipping files > > in those directories in the .deb is a bug, because at the next reboot, > > the file will be deleted, leaving the files that dpkg thinks it's managing > > out of sync with the files that actually exist. At the moment, these are > > variously created by maintainer scripts, systemd units/init scripts, or the > > daemons themselves, with some duplication, and no good way to get an > > overview of which packages "own" which locations: dpkg doesn't know anything > > about them, and systemd knows about some but not all of them. > > > > tmpfiles.d seems like a good way to keep track of who "owns" those > > transient files and directories. I think a Policy "must" is probably too > > strong here, but a "should" might be reasonable? > > > > For the persistent parts of /var, several packages ship regular files > > (as opposed to directories) in the .deb. I think there might be rough > > consensus that doing so is at least a "code smell", but quite a lot of > > packages do this, so a Policy "must" certainly seems too strong at this > > stage. Legacy policy files for polkitd (<< 0.106) are a notable example. > > They're no longer necessary with bookworm's polkitd, and I'm hoping to > > get rid of them during the trixie cycle; but historically /var/lib was > > the only place supported by polkitd other than /etc, so it was functionally > > necessary to ship these files. > > > > Looking at my /var, the TeX family of packages seem to be the heaviest > > users of regular files in /var, with /var/lib/tex-common/**/*.{cfg,cnf}. > > > > I think we can safely say that creating the top-level directory > > of a package's /var, /run, etc. subdirectory declaratively is more > > self-documenting than creating it imperatively, and tmpfiles.d seems > > like a perfectly good way to achieve that. > > > > I don't think it's desirable to have wording that could be taken to imply > > that packages would be wrong to create files inside those top-level > > directories at runtime - that would seem silly (for example I don't > > want anyone arguing that because the mariadb maintainer script starts > > the server, it's somehow a bug for mariadb to create arbitrary database > > tables below /var/lib/mysql). > > > > I also don't think Policy should forbid maintainer scripts using files > > inside those tmpfiles-managed directories for their own state. For example, > > maintainer scripts should be allowed to create files like > > /var/lib/mysql/debian-10.11.flag and > > /var/lib/systemd/deb-systemd-helper-enabled/*, even if we want to require > > /var/lib/mysql and /var/lib/systemd/deb-systemd-helper-enabled to be > > registered in tmpfiles.d(5) so that the sysadmin can discover where they > > come from. > > Yes I agree, it was not my intention to forbid or discourage such > cases, quite the opposite. What I wanted to say is: if you are > creating directories/files by hand in a maintainer script, please use > tmpfiles.d instead, or if your service/tool/code can do it instead, > that's fine too. Among the goals that you correctly described above, > there's also the desire to remove as much custom maintainer script > code as possible, and that's what I was aiming at. > > > > Packages that need to create files or directories in file systems that > > > may be deleted on each reboot (for example, ``/run``, ``/tmp``, and > > > ``/var/tmp``) should do so via configuration files in the > > > ``/usr/lib/tmpfiles.d`` directory. The syntax of those files is > > > defined by the `systemd tmpfiles.d documentation > > > <https://www.freedesktop.org/software/systemd/man/tmpfiles.d.html>`__. > > > > > > However, reading that documentation, it sounds like most of the cases for > > > other directories are handled by other systemd unit configuration > > > directives. We should say that explicitly here and reproduce the list of > > > other directories that should be handled directly by the unit file if > > > that's what we want people to do. > > > > Hmm, yes. Which of these two policies do the systemd maintainers want? > > > > - If a service has RuntimeDirectory= etc. in its unit, then redundantly > > registering those directories in tmpfiles.d(5) is not required unless > > there is some technical reason to do so > > > > - If a service has RuntimeDirectory= etc. in its unit, then it must > > redundantly register those same directories with tmpfiles.d(5) > > > > Reasons we might want the first of those: "don't repeat yourself", and > > letting systemd create the directories as late as possible before starting > > the service, and clean them up as early as possible after stopping it. > > > > Reasons we might want the second: if we had the first policy, a sysadmin > > wanting to find out who "owns" /var/lib/mystery will need to check both > > tmpfiles.d and systemd units; if we had the second, in principle they only > > need to check tmpfiles.d. Also, non-systemd init systems won't read > > RuntimeDirectory= etc., so if the directory is functionally required for > > an LSB init script, it needs to be created some other way, and tmpfiles.d > > is a reasonable choice for that other way. > > I would tend toward the first one, for the DynamicUser case - for > clarity, this currently means RuntimeDir and friends are recursively > chowned on the fly. I hope for Trixie we'll get uid mapping done > instead and it will be moot, but it's not implemented yet so trying to > be careful. > > Also, note that as mentioned earlier these settings work well when > there is a clear and obvious owner, to whose lifecycle the directory > can be tied to, but there are cases where there's either no service at > all or no clear owner.
New revision attached, reworded a bunch of stuff following suggestions, and also I've added a section specifically about the service directories settings.
From a9437eef805b24bd26fbc95e01c52cb19b8ef8ba Mon Sep 17 00:00:00 2001 From: Luca Boccassi <bl...@debian.org> Date: Tue, 9 May 2023 01:38:13 +0100 Subject: [PATCH] Define service directories and tmpfiles.d interfaces and usage --- policy/ch-files.rst | 64 +++++++++++++++++++++++++++++++++ policy/ch-maintainerscripts.rst | 7 ++++ 2 files changed, 71 insertions(+) diff --git a/policy/ch-files.rst b/policy/ch-files.rst index b34c183..073677d 100644 --- a/policy/ch-files.rst +++ b/policy/ch-files.rst @@ -722,6 +722,70 @@ The name of the files and directories installed by binary packages outside the system PATH must be encoded in UTF-8 and should be restricted to ASCII when it is possible to do so. +.. _s-service-directories: + +Service Directories +------------------- + +Services might need auxiliary directories under ``/var/``, ``/run/`` or +``/etc``. Instead of shipping empty directories in packages, or creating them +with custom code in maintainer scripts, services should use ``systemd``'s native +settings to ensure the required directories are created regardless of the +privilege level under which the services are running. The relevant settings are +`RuntimeDirectory=, StateDirectory=, CacheDirectory=, LogsDirectory= and +ConfigurationDirectory= +<https://freedesktop.org/software/systemd/man/systemd.exec.html#RuntimeDirectory=>`_, +covering respectively ``/run/``, ``/var/lib/``, ``/var/cache/``, ``/var/log/`` +and ``/etc/`` for system services, and the equivalent XDG-defined location for +user services. + +Init systems other than ``systemd`` should allow providing the same +functionality as appropriate for each system, for example managing the +directories from the init script shipped by the package. + +.. _s-tmpfiles.d: + +tmpfiles.d +---------- + +Packages might need additional files or directories to implement their +functionality. Directories that are located under ``/var/`` or ``/etc/``, and +files that are located under ``/var/``, should not be created manually via +maintainer scripts, but instead be declaratively defined via the `tmpfiles.d +<https://www.freedesktop.org/software/systemd/man/tmpfiles.d.html>`_ interface. +Files and directories under ephemeral filesystems such as ``/tmp/`` may also be +created and managed via ``tmpfiles.d`` snippets. + +When ownership of a directory can be clearly tied to a specific service, +``Service Directories`` should be preferred to ``tmpfiles.d`` snippets. + +The ``tmpfiles.d`` file format is defined by the ``systemd`` project, and is +guaranteed to be stable. Details about the syntax and installation paths are +defined by its `reference implementation's documentation, +<https://www.freedesktop.org/software/systemd/man/tmpfiles.d.html>`_ and will +not be redefined here. + +``tmpfiles.d`` snippets should be usable on systems that do not boot (such as a +very minimal chroot image), and also on systems booting with init systems other +than ``systemd``. + +``tmpfiles.d`` snippets should be integrated in packages using automated shared +tooling rather than by manually writing package-specific code in maintainers +scripts. For example, packages built using ``debhelper`` should make use of the +``dh_installtmpfiles`` addon. Packages shipping ``tmpfiles.d`` snippets should +depend on the appropriate virtual packages in the following order: +``default-systemd-tmpfiles | systemd-tmpfiles``. + +Init systems are required to integrate with ``tmpfiles.d`` and run the service +that applies them on boot, and regularly for cleanup purposes, depending on the +appropriate package providing the appropriate implementation that best +integrates with each system. For example, ``systemd`` will make the reference +implementation available when its main package is installed. The documentation +for the reference implementation, `systemd-tmpfiles, +<https://www.freedesktop.org/software/systemd/man/systemd-tmpfiles.html>`_ +explains how to call the program so that the appropriate ``tmpfiles.d`` snippets +are applied at the appropriate time. + .. [#] If you are using GCC, ``-fPIC`` produces code with relocatable position independent code, which is required for most architectures diff --git a/policy/ch-maintainerscripts.rst b/policy/ch-maintainerscripts.rst index 724074c..3734deb 100644 --- a/policy/ch-maintainerscripts.rst +++ b/policy/ch-maintainerscripts.rst @@ -50,6 +50,13 @@ absolute pathname. Maintainer scripts should also not reset the appending package-specific directories. These considerations really apply to all shell scripts. +Maintainer scripts should not be used to create or remove auxiliary files and/or +directories that packages may need, such as those in ``/var/`` or ``/etc/``. +Instead, :ref:`s-tmpfiles.d` snippets should be shipped, with the ones provided +by the upstream sources, if any, to be preferred over Debian-specific ones when +possible. For more details about the ``tmpfiles.d`` interface, see +:ref:`s-tmpfiles.d`. + .. _s-idempotency: Maintainer scripts idempotency -- 2.39.2