Bug#1069256: debian-policy: clarify requirement for use of Static-Built-Using

[Simon McVittie]

On Wed, 21 Aug 2024 at 23:45:30 +0200, Guillem Jover wrote:
> On Sat, 2024-04-27 at 17:40:49 +0800, Maytham Alsudany wrote:
> > +A package statically linked with the libraries contained in the
> > +librust-gtk4-dev and librust-pulsectl-rs-dev binary packages, where
> > +the latter is licensed under GPL-3+ (a license that requires full
> > +source code to be available), would have these fields in its control
> > +file:
> > +
> > +::
> > +
> > +Built-Using: rust-pulsectl-rs (= 0.3.2-1+b1)
> > +Static-Built-Using: rust-gtk4 (= 0.7.3-3), rust-pulsectl-rs (= 
> > 0.3.2-1+b1)

With this interpretation of the relationship between the two fields,
is there any situation where a source package would validly appear in
Built-Using but not in Static-Built-Using? (Other than the obvious
"this package has not yet been updated to implement Static-Built-Using")

Thanks,
smcv

Bug#872587: Document the Protected field

[Simon McVittie]

On Wed, 27 Mar 2024 at 14:43:40 +0200, Martin-Éric Racine wrote:
> ke 27. maalisk. 2024 klo 14.00 Andrey Rakhmatullin (w...@debian.org) 
> kirjoitti:
> > "Essential: yes" are always installed. Tools and dependencies assume they
> > are installed.  Bootstrapping tools install them implicitly. Package
> > management tools refuse to remove them.
> >
> > "Protected: yes" are nothing like that. Package management tools refuse to
> > remove them and that's all.

Another way to look at this is that if a Protected package is already
installed, then package management behaves as though it's Essential,
but if a Protected package is merely available from a configured apt
source, then nothing special happens.

> Protected: yes|no
> This field prevents a package from getting auto-removed by dpkg
> without using one of the force options.

True so far...

> It is intended for custom
> local packages not meant for upload to the Debian repository.

... but this isn't the whole story. Protected: yes is also appropriate
for non-local packages that are required for system boot, or for package
management. The design principle is that if it would be hard to recover
from removing a package once it has been installed, then it's Protected.

An example of Protected: yes on boot packages is that the init metapackage
is Protected: yes, to make sure you don't accidentally remove the init
system and make the system unbootable (which is hard to recover from
because before you can install a new init system, you need to be able to
boot). It might also make sense for bootloaders like grub to be Protected:
yes, although currently they are not.

An example of Protected: yes on package management packages is that it
would be appropriate to put Protected: yes on apt (although in fact apt
is hard-coded to behave that way even without Protected: yes), to avoid
accidentally getting into a situation where you have removed apt
(which is hard to recover from because now there's no package manager,
and no easy-to-validate trust chain to check that a manually-downloaded
apt_*.deb is authentic).

smcv

Re: Does iproute2 moving config files to /usr/lib violate section 10.7.2?

[Simon McVittie]

On Sun, 17 Sep 2023 at 02:03:52 +0200, Santiago Vila wrote:
> If the files are intended to be modified "in place" by the
> system admin, we call them configuration files (and we try hard
> to put them in /etc). If they are not intended to be modified by
> the system admin, we don't call them configuration files (and
> we try hard not to put them in /etc).

Perhaps an interesting example of this is that in packages originally
developed on an OS with less elaborate conffile handling than dpkg's
(like dbus, originally developed on/by Red Hat), there used to be a
common pattern of having:

1. a file in /etc that sysadmins are specifically asked *not* to edit,
   for which non-dpkg package managers would be asked to default to
   the equivalent of dpkg's "use the maintainer's version" (for example
   %config in RPM), like /etc/dbus-1/system.conf

2. a separate file for local changes, which might or might not exist with
   empty or template contents by default, for which non-dpkg package
   managers would be asked to default to the equivalent of dpkg's "keep
   the locally modified version" (for example %config(noreplace) in RPM),
   like /etc/dbus-1/system-local.conf

/etc/dbus-1/system.conf was traditionally a dpkg conffile because
of its location, but it wasn't really a configuration file by
the definition Santiago gave here (and it has now been moved to
/usr/share/dbus-1/system.conf as a result): it's more like part of the
implementation of dbus-daemon --system that, as an implementation detail,
happens to have been written in the dbus-daemon XML configuration language
rather than in C.

/etc/dbus-1/system-local.conf isn't created by default, but sysadmins
can create it if they want to, and it's a "true" configuration file
(although in practice I'd recommend "drop-ins" in /etc/dbus-1/system.d
as a better version of /etc/dbus-1/system-local.conf).

smcv

Bug#945269: debian-policy: packages should use tmpfiles.d(5) to create directories below /var

[Simon McVittie]

On Tue, 12 Sep 2023 at 22:17:44 -0700, Russ Allbery wrote:
> >> Maybe the right way to do this is just have two examples, one as the
> >> default and another if you're using tmpfiles.d functionality added in a
> >> specific version of systemd that's newer than the version shipped with
> >> the stable version of Debian prior to the one you're targeting.

Yes, I think that reads better than what I initially suggested. Thanks!

The key piece of information that was missing from your previous proposal
was that systemd-tmpfiles interface versions match upstream systemd
version numbers. As a concrete example, if someone wants to upload an
implementation other than the one from systemd, it cannot have

Provides: systemd-tmpfiles (= 254)

until it has at least a basic implementation of the new "X", "C+" and
"--graceful" features introduced in systemd 254.

> +All packages that ship ``tmpfiles.d`` configuration should declare a
> +dependency on::
> +
> +default-systemd-tmpfiles | systemd-tmpfiles

If the package benefits from running tmpfiles.d but does not strictly
require it (for example dbus-daemon, where /run/dbus/containers is needed
for some optional functionality), would a Recommends or Suggests be
allowed by this wording, or are we intending for this to be a mandatory
hard dependency?

smcv

Bug#567033: Decide if we should continue recommending /usr/games

[Simon McVittie]

On Mon, 11 Sep 2023 at 10:19:13 -0700, Russ Allbery wrote:
> I am inclined to agree [with no longer recommending /usr/games];
> it's just one more thing for people to think about
> while packaging things, and I don't think it serves much of a useful
> purpose.  However, the bug log has a couple of concrete objections.

I have another possible reason to separate /usr/games, which I think is
potentially more compelling than either of the ones in the bug (I'm sorry,
I thought it was already common knowledge):

Games are often written for performance more than correctness, and
frequently do non-ideal things or have unfixed security issues. If we
separate them into /usr/games and avoid putting that directory in root's
PATH, then tab completion mistakes can't result in root accidentally
running a game.

Similarly, I think (although I have no evidence) it's more common to
install non-free games, and non-free game managers like Steam, than
non-free utility/application software. If we put those in /usr/games, the
root can't accidentally run those as a result of a tab-completion mistake.

Entertainment programs that are not strictly games (the ones we might
classify as "toys") have similar considerations.

Is this enough to justify the existence of /usr/games? I don't know; but
I think it's more likely to be enough to justify /usr/games than the other
reasons previously cited here?

In recent versions of Debian's Steam packaging (formerly the steam
package, now the steam-installer package) I've put a safety-catch on it:
if some important files in ~/.steam/root don't exist (indicating a new
installation), the wrapper script doesn't actually install or run any
proprietary code until the user has confirmed in a GUI dialog that they
actually wanted to install it. Similarly, the binary-only games that
game-data-packager can install have a wrapper script with a confirmation
dialog before the first run, similar to a clickthrough license, to avoid
accidents. That mitigates these being in privileged users' PATHs.

Valve's official Steam packaging in their third-party apt repository
(the steam-launcher package, currently maintained by me with a different
hat on) installs to /usr/bin and has no such safety-catch, but it does
require adding an apt source that will result in running Valve-supplied
maintainer scripts as root, so if you add that apt source you're already
completely trusting Valve anyway.

> Axel Beckert objected because games may conflict with other tools
> installed in /usr/bin.  I feel like this is already a bug and merging the
> two namespaces to force us to deal with that bug may be a feature in
> disguise, because having two binaries with entirely different purposes on
> the user's PATH is a recipe for confusion and problems.

I agree. I think it's silly that a PATH search for pacman can result in
running either a 2D maze game or Arch Linux's package manager, especially
if the two packages are co-installable!

I know we have a Policy requirement that packages do not contain both
/bin/foo and /usr/bin/foo, to ensure that merged-/usr is possible.

I thought we also had a requirement for names of executables in the
PATH to be unique between /{usr/,}bin and /{usr/,}sbin, but I can't find
it now. I know some packages like iproute2 install both /{usr/,}sbin/ip
and /{usr/,}bin/ip, which I think is OK if they are functionally equivalent,
but would be a bug if they were functionally different.

> This is similar the old multiuser timeshare use case for separating games
> back when they competed for resources with other uses of the system and
> administrators wanted to be able to stop people from running them until
> after hours.  I feel like this use case is exceptionally rare at this
> point, and I'm not sure it's worth the packaging thought to maintain a
> separation just for that.

Unlike Axel's namespace-splitting use-case, I think this is a positive
thing, but I'm unsure whether its small benefit is really worth its cost.

> Alexandre also requested keeping games data separate so that it could be
> moved out of the /usr partition because it could be quite large.

Again, I think this is a positive, but I'm unsure whether its benefit is
worth its cost. Perhaps it would be proportionate to say that games MAY
install into /usr/share/games, and leave it up to maintainers whether they
do so, with the suggestion that small games shouldn't bother but
maintainers of large games might want to consider it?

Disclosure: I am a co-maintainer with Alexandre of the game-data-packager
package, which installs proprietary game data into /usr/share/games, some
of it much larger than the vast majority of games we package in Debian; and
I think converting game-data-packager and its various supported game engines
for a transition from /usr/share/foo to /usr/share/games/foo would be quite
annoying to achieve, without providing any significant benefit.

smcv

Bug#945269: debian-policy: packages should use tmpfiles.d(5) to create directories below /var

[Simon McVittie]

In general I support this direction.

On Sun, 25 Jun 2023 at 16:55:44 +0100, Luca Boccassi wrote:
> Packages shipping ``tmpfiles.d`` snippets should
> +depend on the appropriate virtual packages in the following order:
> +``default-systemd-tmpfiles | systemd-tmpfiles``.

I think it's worth saying explicitly that if the package relies on
functionality added in systemd version n, where n is newer than some
reasonable cutoff, it should depend on
default-systemd-tmpfiles (>= n) | systemd-tmpfiles (>= n)
instead.

If a package is targeting testing/unstable, then it can drop the version
constraint in the very common case where its tmpfiles.d snippet is processed
correctly by stable's systemd-tmpfiles. Similarly if it's targeting stable,
it can drop the version constraint if oldstable's systemd would have been
enough.

Perhaps this?

Packages shipping ``tmpfiles.d`` snippets should
depend on the appropriate virtual packages in the following order:
``default-systemd-tmpfiles (>= v) | systemd-tmpfiles (>= v)``,
where *v* is a version of systemd that provides all ``tmpfiles.d``
features that are required. The version constraint may be
omitted if it is satisfied by all implementations of the
``systemd-tmpfiles`` virtual package supported in the previous stable
release.

(If debhelper generates the dependency, in practice it's probably enough
for it to generate the unversioned dependency, and in the rare case
where a tmpfiles.d snippet needs new features, maintainers can add
the versioned dependency themselves.)

smcv

Bug#945269: debian-policy: packages should use tmpfiles.d(5) to create directories below /var

[Simon McVittie]

On Sat, 09 Sep 2023 at 19:51:50 -0700, Russ Allbery wrote:
> Luca, am I right that service directories are specific to, well, services?
> If so, what would you think of moving them to Policy 9.3 alongside the
> other discussion of systemd units?  They feel out of place here, since
> packages that do not use services cannot use this functionality

I'm not Luca, but I think you're correct here.

> Luca Boccassi  writes:
> > +Packages might need additional files or directories to implement their
> > +functionality. Directories that are located under ``/var/`` or
> > +``/etc/``, and files that are located under ``/var/``, should not be
> > +created manually via maintainer scripts, but instead be declaratively
> > +defined via the `tmpfiles.d
> > +`_
> > +interface.  Files and directories under ephemeral filesystems such as
> > +``/tmp/`` may also be created and managed via ``tmpfiles.d`` snippets.
> 
> I understand the empty directory part, but I don't believe "files that are
> located under /var" is correct unless you specifically mean *empty* files
> (and even then, I'm not clear on precisely when this would be needed).
> For example, /var/lib/gnubg/gnubg_ts0.bd is created by the gnubg package
> maintainer script, and I can see no possible way that action could (or
> should) be handled by the tmpfiles.d mechanism.

In general tmpfiles.d handles files that exist only as metadata: symbolic
links (for which the target is the only interesting fact), empty files
(for which the existence and ownership/permissions are the only interesting
facts), directories (ditto) and so on.

It can also handle files that have static initial contents that do not
vary between systems, but can change in a system-specific way later,
with initial contents either hard-coded in the tmpfiles.d snippet (for
short text strings) or copied from somewhere below /usr (canonically
/usr/share/factory).

Files generated by non-trivial imperative code, like machine-specific
initial contents (/var/lib/dbus/machine-id) or some sort of compiler
(/var/lib/gnubg/gnubg_ts0.bd, as far as I can tell), are out of scope for
tmpfiles.d, so I think you're right to be concerned that Luca's wording
as written is asking gnubg to do something that is unimplementable.
ch-maintainerscripts.rst has the same issue.

Perhaps "files with trivial contents that are located under /var" would be
a good wording that is not overly specific about implementation details,
covers the 90% case, and leaves room for exceptions by declaring packages
like dbus and gnubg to be non-trivial?

If /var/lib/gnubg/gnubg_ts0.bd is deterministically compiled from
files shipped in the .deb as a time/space trade-off, is only written
during package management operations, and is otherwise read-only, then
perhaps it should live in /usr, but that's orthogonal to wanting to use
tmpfiles.d where feasible. (Prior art for similar situations includes
Python bytecode, glibc locales, GLib gschemas.compiled and giomodule.cache,
and so on.)

smcv

Bug#1051371: debian-policy: stop referring to legacy filesystem paths for script interpreters

[Simon McVittie]

On Thu, 07 Sep 2023 at 00:50:54 +0100, Luca Boccassi wrote:
> On Thu, 7 Sept 2023 at 00:45, Sam Hartman  wrote:
> > I.E. in the cases you adjust, I think it is already
> > not a bug, and it would be inappropriate to use existing policy language
> > to complain about which interpreter path people use.
> 
> In practice though they are used as normative, for example Lintian
> raises an error (not even a warning, an actual error).

In general, if Lintian is diagnosing paths as an error when they are
allowed by Policy (therefore not a serious bug) and work in practice
on all relevant systems (therefore not a grave bug), then the place to
change that would be Lintian, not Policy.

However, because our official buildds for trixie/sid/experimental
are still using the transitional non-merged-/usr layout at the time
of writing, it is still valuable for Lintian to care about whether
interpreters are specified with their /usr path or not: so I think Lintian
is still entirely correct to encourage #!/bin/sh over #!/usr/bin/sh,
and so on.

Also, the same version of Lintian is used to check packages targeting
any supported suite (including bookworm-backports, bookworm, and even
bullseye), and Lintian does not always have enough information to know
which suite is being targeted (consider UNRELEASED packages). bookworm
buildds are likely to remain non-merged-/usr indefinitely, to ensure we
have a working upgrade path from bullseye, so using the non-traditional
path to an interpreter (for example /usr/bin/sh or /bin/perl) is in
practice still RC for bookworm if the script might be invoked during
build. So I think it would be premature to change Lintian for this,
even after our official buildds for suites >= trixie become merged-/usr.

Using the non-traditional path to an interpreter is also *definitely*
RC for *bullseye*, because bullseye systems that were upgraded from
a sufficiently old release are non-merged-/usr unless the user has
specifically taken steps to merge it.

I think it would be appropriate to change Lintian to accept /bin/sh
and /usr/bin/sh as interchangeable after trixie is released, at which
point bullseye-backports will have been discontinued, bullseye will be
LTS-maintained, and bookworm-backports will only be receiving targeted
stable-release-suitable changes which are unlikely to regress the choice
of interpreter path.

smcv

Bug#1051371: debian-policy: stop referring to legacy filesystem paths for script interpreters

[Simon McVittie]

On Wed, 06 Sep 2023 at 16:51:10 -0600, Sam Hartman wrote:
> I'd consider it a non-RC bug if someone were manually writing
> #!/usr/bin/sh

As long as our official buildds are non-merged-/usr, I would consider use
of that path in scripts that get run at package build time to be
potentially RC, in fact. I have tried to arrange for our official buildds
for suites >= trixie to become merged-/usr (#1050868, #1025708) but that
work is awaiting review.

Until that change lands, or our official buildds for
trixie/sid/experimental become merged-/usr some other way, encouraging
maintainers to use interpreters that were historically in /bin via their
/usr/bin paths is actively harmful and we should not do it. Please don't
make the transition to merged-/usr-everywhere more contentious than it
already is.

> > "Luca" == Luca Boccassi  writes:
> Luca> /bin/sh is not universally compatible with non-Linux OSes.
> 
> I claim it is more compatible.

Hard-coding "#!/bin/sh" is not compatible with every operating system,
but it's much more widely compatible than hard-coding "#!/usr/bin/sh",
particularly for scripts that are meant to be backportable to older
versions of LTS distributions (notably Debian!). Even NixOS, an operating
system that is notable for the extent to which it intentionally avoids
using FHS paths, has made the pragmatic choice to provide /bin/sh.

I would not want to encourage Debian maintainers to use
"#!/usr/bin/env sh" for shell scripts, even though it's more broadly
portable than assuming that /bin/sh is a POSIX shell.
This is analogous to how we encourage (and sometimes require) Debian
maintainers to use "#!/usr/bin/perl" and "#!/usr/bin/python3" in
preference to indirection via "#!/usr/bin/env perl" or similar, even
if their upstream uses "#!/usr/bin/env perl" for wider portability.

We certainly shouldn't require shell scripts in Debian to go through
the same contortions as Autoconf-generated configure scripts to find
a POSIX shell (see `info autoconf 'Portable shell'`) because that's
unnecessary runtime and maintenance overhead on any GNU/Linux system,
any of the non-Linux ports that Debian has attempted to support in the
past, and hopefully any free or proprietary Unix that is still relevant
to Free Software authors this decade.

smcv

Bug#1050322: Partial versus complete replacement of a package by another

[Simon McVittie]

On Wed, 23 Aug 2023 at 09:22:41 +0200, julien.pu...@gmail.com wrote:
> In src:mathcomp-analysis 0.6.4-2, I declared that libcoq-mathcomp-
> classical Breaks libcoq-mathcomp-analysis (<< 0.6.4) and closed the
> bug. It was swiftly re-opened because I hadn't used Breaks+Replaces
> according to Policy 7.6.1. But I don't want to use Replaces as libcoq-
> mathcomp-classical isn't a *complete* replacement for libcoq-mathcomp-
> analysis (<< 0.6.4) -- it only provides a small part of it!

It seems you've misunderstood what Replaces means: it doesn't mean
"is a complete drop-in replacement for". (That would need a Provides too.)

Policy 7.6.1 talks about an old package foo which has been split into
new packages foo and foo-data, which seems like your situation with the
names changed?

> The problem is with section 7.6 on how to use Replaces: indeed, the
> very first paragraph of 7.6 mentions "two distinct purposes", partial
> and complete replacement... but the rest of the section is all about
> complete replacement

What I see in that section is:

7.6. Overwriting files and replacing packages - Replaces
7.6.1. Overwriting files in other packages
7.6.2. Replacing whole packages, forcing their removal

7.6.2 is about complete replacement, so by a process of elimination,
7.6.1 ought to be about partial replacement (and it is: it talks about
"The new version of the package foo", which wouldn't exist if foo-data
had completely superseded it).

> In fact subsection 7.6.1 has an example which looks exactly like
> #1050027, but it looks wrong: if foo 1.2-2 is installed on a system and
> foo-data 1.2-3's installation is requested, then because of Breaks dpkg
> will know about the file conflicts and beware, but because of Replaces,
> it will think it's a complete replacement -- foo will disappear if I
> understand well.

No, the central misunderstanding here is that you think Replaces will
have the effect of instructing dpkg to remove the replaced package
completely, which is not the case.

smcv

Bug#1039102: debian-policy: make systemd units mandatory for packages shipping system services

[Simon McVittie]

On Sun, 25 Jun 2023 at 18:51:24 +0100, Luca Boccassi wrote:
> Tentatively this should happen within Trixie's development cycle. Of
> course it's free software and generators are not that difficult to
> maintain, so if someone wanted to lift the sysv generator out of the
> systemd repository and adapt it to be a standalone binary there's
> nothing stopping them. But I wouldn't want the systemd package to
> depend on such a backward compat tool, so packages needing this
> hyptothetical package should depend explicitly on it.

I think it's maybe worth mentioning (not in Policy, but in the mass
bug filing about deprecating the generator) that there is an immediate
concrete benefit of moving from the generated unit to a maintainer-written
unit: it lets systemd know whether the service is meant to be a one-time
action like /etc/init.d/sudo (Type=oneshot, RemainAfterExit=yes), or
a long-running daemon like /etc/init.d/ssh (Type=forking or similar,
RemainAfterExit=no).

systemd's generator can't know which one of those each service is meant
to be (because LSB init scripts don't distinguish), so it has to make
pessimistic assumptions that work but are non-ideal in both cases.

smcv

Re: 6.1.3. Multiple binary packages question

[Simon McVittie]

On Sun, 18 Jun 2023 at 23:18:24 +, Holger Levsen wrote:
> On Sun, Jun 18, 2023 at 11:19:06PM +0200, Bill Allombert wrote:
> > The drawback of dh_install is that it requires more diskspace to build than
> > dh_movefiles but is less error prone.
> > So unless your package is very large, it is safer to use dh_install.
> 
> interesting, I've never heard of dh_movefiles before.

It's the tool that was used before dh_install existed. Its man page says:

Note: dh_install is a much better program, and you are recommended
to use it instead of dh_movefiles.

> what does "more diskspace" mean, however? kilobytes? terabytes? 100% of
> the diskspace used by the files installed? 200%?

The cost is 100% of the disk space used by the files being installed,
if your filesystem doesn't support reflinks; or 0% if it does.

dh_install does something similar to

cp --reflink=auto debian/tmp/usr/bin/foo debian/foo-bin/usr/bin/foo

whereas dh_movefiles is more like

mv debian/tmp/usr/bin/foo debian/foo-bin/usr/bin/foo

On btrfs and other reflink-capable filesystems, dh_install should be
basically zero-cost, as a result of --reflink=auto.

I'd personally only consider using dh_movefiles for the largest of "data"
packages. Things like openarena-data and nexuiz-data (hundreds of MiB)
are fine with dh_install, but 0ad-data (about 3 GiB) would maybe benefit?

(But if I maintained 0ad-data, I'd probably still use dh_install, and just
make sure that I built it on btrfs or another reflink-capable filesystem
if time and space were a concern.)

> "my" largest package was less than 100mb in size, which can be a bit annoying 
> for
> uploads, but for diskspace during build, several gigabytes (or more) are
> not uncommon, so I'm having a somewhat hard time imaginening that the
> above is a relavant argument in most situations.

If your package's total Installed-Size is 100M and we assume the compressed
size is (say) 60M, then I would expect the build to need:

quite a lot  ./configure && make
+ 100M   make install DESTDIR=debian/tmp
+ 100M   dh_install
+ 60Mdh_builddeb

At best, dh_movefiles saves you the second "+ 100M".

smcv

Bug#945269: debian-policy: packages should use tmpfiles.d(5) to create directories below /var

[Simon McVittie]

On Tue, 06 Jun 2023 at 20:40:52 -0700, Russ Allbery wrote:
> Luca Boccassi  writes:
> > +Packages might need additional files or directories to implement their
> > +functionality. Directories that are located under ``/var/`` or
> > +``/etc/``, and files that are located under ``/var/``, must not be
> > +created manually via maintainer scripts, but instead be declaratively
> > +defined via the `tmpfiles.d
> > +`_
> > +interface.
> 
> This is an oddly specific list of directories and not at all the
> directories that I would have expected to be handled by tmpfiles.d.

Sorry, in my previous reading of this bug I had been concentrating on
the mechanics of how to make tmpfiles.d(5) something that maintainers can
rely on if it's convenient/helpful, and I'd missed that Luca is asking
for its use to be mandatory in some cases.

I would personally be inclined to concentrate on making tmpfiles.d(5)
something that we can rely on and encourage the use of where appropriate,
even on non-systemd systems, so that (upstream and downstream) maintainers
can move towards it of their own accord because it's more convenient
than other options, and put aside the question of making it generally a
"should" or "must" for the moment.

I believe (please correct me if I'm wrong) that Luca's intention here
is that this is drawing a line between:

- the static files of the OS: /usr and the /usr-like top-level directories
  (the ones that get merged by the /usr merge), which should be statically
  shipped in packages and managed by dpkg (or on "immutable" systems
  that use image-based/tree-based upgrades, maybe by ostree or casync
  or similar, from an tree originally constructed from dpkg packages)

- this specific system's persistent state: /var and parts of /etc

with the intention of eventually enabling functionality like being
able to do a "factory reset" to the equivalent of a freshly installed
system by deleting (most of) /etc and /var, rebooting, and letting the
OS re-create them from a template below /usr; or doing the equivalent
for individual packages by deleting only their part of /etc and /var.

/etc is somewhere between static files and state, because traditionally
it has been a mixture of files that the sysadmin or installer must provide
(like /etc/passwd); configuration files that are shipped by a package and
can be edited by the sysadmin (like /etc/systemd/logind.conf); and
integration glue that links up one package with another, can in principle
be edited by the sysadmin, but in practice is rarely edited
(like /etc/profile.d/flatpak.sh).

Various upstream projects including systemd have been trying to reduce
the extent to which /etc and /var are included in the data.tar.* of a .deb
or other packaging systems' equivalents, by moving the integration glue
to a /usr-like directory (/lib/udev/rules.d, /usr/share/dbus-1/system.d),
reserving the corresponding /etc directory for sysadmin configuration
(/etc/udev/rules.d, /etc/dbus-1/system.d), and providing a way for the
sysadmin to "mask" any integration files they want the system to ignore.

If we disregard conffiles and configuration files in /etc for the
moment, there are basically three ways for a package to get a file onto
the running system:

- ship it in the data.tar.* of a .deb
- create it from a maintainer script and also during boot
- maybe via tmpfiles.d(5)
- or maybe open-coded
- have the package create it at runtime, on-demand
- this clearly doesn't work if the package's code runs unprivileged
  and relies on root having created a directory for it already

For /usr and the /usr-like directories, shipping files in the .deb is by
far the most common, although a few packages need to create files here
via maintainer scripts or triggers (for example
/usr/lib/x86_64-linux-gnu/gio/modules/giomodule.cache which is a summary
of files created by multiple packages, and is updated whenever those
packages are added, removed or changed).

For /run, /tmp and /var/tmp, I think there's consensus that shipping files
in those directories in the .deb is a bug, because at the next reboot,
the file will be deleted, leaving the files that dpkg thinks it's managing
out of sync with the files that actually exist. At the moment, these are
variously created by maintainer scripts, systemd units/init scripts, or the
daemons themselves, with some duplication, and no good way to get an
overview of which packages "own" which locations: dpkg doesn't know anything
about them, and systemd knows about some but not all of them.

tmpfiles.d seems like a good way to keep track of who "owns" those
transient files and directories. I think a Policy "must" is probably too
strong here, but a "should" might be reasonable?

For the persistent parts of /var, several packages ship regular files
(as opposed to directories) in the .deb. I think there might be rough
consensus that doing so is at least a "code smell", but quite a lot

Bug#945269: debian-policy: packages should use tmpfiles.d(5) to create directories below /var

[Simon McVittie]

On Tue, 06 Jun 2023 at 11:37:51 +0100, Sean Whitton wrote:
> On Sun 04 Jun 2023 at 02:56PM +01, Simon McVittie wrote:
> > Another possible mitigation which I haven't previously seen proposed
> > is giving *elogind* a Depends or Recommends on systemd-*-standalone.
> > I think that would work to mitigate the failure mode with (1.) and (B.),
> > and the installed-size argument seems less interesting here because the
> > sort of systems that require elogind are already much larger anyway.
> > Would the elogind maintainers be willing to consider this? Does anyone
> > see a reason why it wouldn't work?
> 
> So to confirm, you think that if the elogind maintainers did this, then
> default-systemd-tmpfiles could point at systemd rather than
> systemd-standalone-tmpfiles, which the systemd maintainers prefer, but
> in addition, there aren't any scenarios in which people's systems are
> likely to be re-arranged when they don't want them to be?

Exactly. My hope is that if we had:

Package: systemd
Architecture: linux-any
Provides: default-systemd-tmpfiles, systemd-tmpfiles
Conflicts: systemd-tmpfiles
Replaces: systemd-tmpfiles

Package: systemd-standalone-tmpfiles
Architecture: linux-any
Provides: systemd-tmpfiles
Conflicts: systemd-tmpfiles
Replaces: systemd-tmpfiles

Package: elogind
Depends: systemd-standalone-tmpfiles# or Recommends?

Package: foo-service # any package that requires tmpfiles.d(5)
Depends: default-systemd-tmpfiles | systemd-tmpfiles

# optionally, if someone does the work
Package: openrc-tmpfiles# any other implementation
Architecture: hurd-any kfreebsd-any
Provides: default-systemd-tmpfiles, systemd-tmpfiles
Conflicts: systemd-tmpfiles
Replaces: systemd-tmpfiles

then the right thing (or at least *a* right thing) would happen in
all cases:

* install foo-service on a systemd-booted system:
  systemd is already installed and the dependency is satisfied

* install foo-service on a sysvinit-booted desktop system with elogind:
  elogind is already installed, therefore systemd-standalone-tmpfiles is
  already installed and the dependency is satisfied, avoiding #1016006 etc.

* install foo-service on a sysvinit-booted headless system with no elogind:
  systemd gets installed as a dependency by default, which is what the
  systemd maintainers would prefer to happen when there are no compelling
  space constraints; but the user can specifically ask for
  systemd-standalone-tmpfiles if that's what they'd prefer

* install foo-service in a container with no init system at all:
  systemd gets installed as a dependency by default, which is what the
  systemd maintainers would prefer to happen when there are no compelling
  space constraints; but the user can specifically ask for
  systemd-standalone-tmpfiles if that's what they'd prefer

* (optionally) install foo-service on hurd-i386:
  openrc-tmpfiles (or whatever) gets installed as a dependency

smcv

Bug#945269: debian-policy: packages should use tmpfiles.d(5) to create directories below /var

[Simon McVittie]

On Mon, 05 Jun 2023 at 01:36:25 +0100, Luca Boccassi wrote:
> On Sun, 4 Jun 2023 at 14:56, Simon McVittie  wrote:
> > So I think the only realistic options for packages that hard-require
> > this functionality (not all do) are:
> >
> > 1. Depends: systemd | systemd-tmpfiles
> > 2. Depends: systemd-tmpfiles-standalone | systemd-tmpfiles
> > 3. Depends: default-systemd-tmpfiles | systemd-tmpfiles

In case it's not obvious, (2.) should say
systemd-standalone-tmpfiles | systemd-tmpfiles (and the same everywhere
else that I mentioned systemd-tmpfiles-standalone).

> Our time is worth more than 80K or whatever it is of disk space in a
> throw-away container.

I agree that the systemd maintainers' time is a limited resource that we
should not waste, but that size estimate is off by a couple of orders of
magnitude. On amd64, aptitude tells me systemd-standalone-tmpfiles and
-sysusers are about 700K of Uncompressed Size between them, while the
full systemd and libsystemd-shared packages add up to about 15M. For
genuinely throwaway containers, yes, it's not worth optimizing this,
but for containers that will be archived in a registry and/or kept
running longer-term, that seems like enough that maintainers of Docker
containers, etc. will want to use the standalone binaries if they are
sufficient for the container's needs.

(This is ignoring any extra library dependencies that might be required by
systemd and libsystemd-shared but unnecessary for the standalone binaries;
if there are any, then obviously the effective size delta increases.)

smcv

Bug#945269: debian-policy: packages should use tmpfiles.d(5) to create directories below /var

[Simon McVittie]

On Mon, 05 Jun 2023 at 01:36:25 +0100, Luca Boccassi wrote:
> If it is useful, adding a "default-tmpfiles" or so virtual package
> would be fine by me - but with the kfreebsd port being retired soon,
> and i386 (for hurd) going the way of the dodo, I'm not sure it would
> be very useful? I don't think it would be a problem to add it, if it
> turns out to be of use though.

What this would look like, in src:systemd:

Package: systemd
Provides: default-systemd-tmpfiles, systemd-tmpfiles

Package: systemd-standalone-tmpfiles
Provides: systemd-tmpfiles

(or maybe the other way round, depending what conclusion we come to on
the choice between (1.) and (2.)), and then in dependent packages:

Package: foo-service
Depends: default-systemd-tmpfiles | systemd-tmpfiles

The benefits of that over having foo-service depend directly on
"systemd | systemd-tmpfiles" or
"systemd-standalone-tmpfiles | systemd-tmpfiles" are fairly minor, but the
cost is also fairly minor; and if we want this, we should set it up
*before* lots of packages start adding a dependency on the -tmpfiles or
-sysusers interfaces. The benefits I see are:

- if we find out that the dependency we first added is a practical problem
  for whatever reason, we can swap the default with a src:systemd upload,
  without needing multiple maintainers to touch all the dependent packages;

- if we get a useful non-Linux port (admittedly this looks increasingly
  unlikely) which cannot compile src:systemd, then their reimplementation
  of these interfaces can have Provides: default-systemd-tmpfiles on that
  architecture, without affecting Linux architectures
  (the same way dbus-x11 Provides default-dbus-session-bus on non-Linux
  ports, even though it's dbus-user-session that Provides the virtual
  package on Linux)

smcv

Bug#945269: debian-policy: packages should use tmpfiles.d(5) to create directories below /var

[Simon McVittie]

(Newly cc'd elogind maintainers: Please see #945269 for context)

On Sun, 04 Jun 2023 at 12:15:41 +0100, Luca Boccassi wrote:
> On Sun, 4 Jun 2023 at 12:02, Sean Whitton  wrote:
> > On Tue 09 May 2023 at 01:44AM +01, Luca Boccassi wrote:
> > > For now I've kept only a mention of the 'systemd-tmpfiles' virtual
> > > package. As maintainers we would really prefer if the 'main'
> > > implementation is pulled in whenever possible. When a minimal
> > > installation is desired (ie, a minbase), it is possible to manually
> > > specify the -standalone variant.
> > >
> > > This was a controversial point last year, see:
> > >
> > > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1017441
> >
> > Hmm.  I don't have personal experience with this sort of thing, but
> > based on some of the examples in that bug, it seems like doing this
> > could cause apt to change people's systems around in ways they strongly
> > disprefer.  What you propose seems like it could cause unpleasant
> > surprises.
> 
> Only due to bugs in said other packages, or if the wrong commands are
> passed to apt/aptitude/etc.

I think one way or another, if anyone is going to set a package-level
dependency on systemd-tmpfiles, the first (preferred) dependency needs to
be on either a concrete provider (systemd or systemd-tmpfiles-standalone
in this case), or a default-systemd-tmpfiles virtual package
that only has one provider per architecture (which is the way
{default-,}dbus-{system,session}-bus are handled). Otherwise, you
can get a non-deterministic choice of default implementation, which
seems strictly worse than either depending on systemd or depending on
systemd-tmpfiles-standalone - if you're unlucky, it can have all the
disadvantages of either one of those.

So I think the only realistic options for packages that hard-require
this functionality (not all do) are:

1. Depends: systemd | systemd-tmpfiles
2. Depends: systemd-tmpfiles-standalone | systemd-tmpfiles
3. Depends: default-systemd-tmpfiles | systemd-tmpfiles

(where the third one is equivalent to one of the first two, depending on
how default-systemd-tmpfiles was implemented), possibly with some more
less-preferred options between the first "|" and the virtual-package
fallback.

I also think that Policy shouldn't be recommending this interface without
also being able to give guidance on what an appropriate dependency would
look like, because if some packages choose (1.) and some choose (2.),
again we'll get a non-deterministic result, depending on which packages
you happen to install first and what their maintainers think; and I don't
want individual services' maintainers to be required to constantly argue
whether (1.) or (2.) is better.

In #1017441 the debhelper maintainers declined to generate such a
dependency, but even if they had wanted to generate it centrally, we'd
have the same distro-integration considerations about saying what the
right dependency would look like.

Before describing pros and cons of those options in different scenarios,
I want to reiterate that installing the systemd package (which contains
the default/recommended implementation of the systemd-tmpfiles interface)
does not imply using systemd as an init system, and everywhere in this
message that I have said "systemd", I specifically mean the systemd
package and not systemd-as-pid-1.

>From the point of view of init systems, I think the interesting scenarios
are:

A. systemd package installed (typically via systemd-sysv)
B. non-systemd init system (typically sysvinit) and no systemd package
C. no init system at all (typically in Docker or a chroot)

For (A.), there is no ambiguity: systemd is installed and provides the
tmpfiles interface, and everything is fine. Any dependency sequence ((1.),
(2.) or (3.) above) should give us the result we want. The only problem
scenario I can think of for (A.) with (2.) is:

- a package (let's say foo-service) has chosen (2.) and so depends on
  systemd-tmpfiles-standalone | systemd-tmpfiles
- we install a base system with neither foo-service nor systemd
  - this must be a minbase debootstrap or similar, because a default
debootstrap already includes systemd-sysv
- we install foo-service first, without systemd
  - systemd-tmpfiles-standalone is pulled in
- afterwards, we install systemd (via the init metapackage and
  systemd-sysv, or directly)
  - desired result: systemd-tmpfiles-standalone is removed and replaced by
systemd
  - actual result: apt's heuristic might have difficulty realising that
it needs to do that

I would have expected that anyone wanting an environment with an init
system is more likely to include it in the bootstrap or in the first
batch of post-bootstrap additions than to install it later, but maybe
that's wrong? Or are there other problem scenarios here?

I'll come back to (B.), non-default init, in a moment.

No init system at all, (C.), can only happen when starting with a
minbase debootstrap or equivalent (because a default debootstr

Re: nocheck (don't run) vs nodoc (don't build)

[Simon McVittie]

On Wed, 26 Apr 2023 at 16:48:43 -0600, Sam Hartman wrote:
> Simon> - the nocheck option SHOULD NOT alter the contents of any
> Simon> binary package
> 
> I agree this is true--possibly even as a MUST--for the nocheck build
> profile, but
> I think DEB_BUILD_OPTIONS are allowed to modify the contents of binary
> packages.
> As an example nostrip certainly modifies the size of things, and noopt
> can modify the behavior and performance.

Yes, I'm not making a general statement about DEB_BUILD_OPTIONS here:
some of the other options clearly do alter binary packages.

I think that as a design principle, DEB_BUILD_OPTIONS=nocheck,
specifically, should not affect the contents of binary packages (because
like the build-profile of the same name, it's about whether to run tests,
and not about what to package).

> My assumption was that if you were specifying the nocheck build option,
> you don't want to run tests, possibly because they are flaky and/or you
> are trying a build for some local reason even though you know the tests
> will fail.
> 
> If you actually want to avoid building the tests, use the nocheck build
> profile (assuming that can be done without modifying binary packages).

I agree, although if the package has installed-tests (like dbus-tests,
fwupd-tests, systemd-tests) then you would also need to turn those off
with the noinsttest build profile to get the desired effect.

> If my reasoning is correct, my interpretation is that nocheck option
> can build tests or not, depending on whatever is convenient.

Yes, I have the same interpretation.

smcv

Re: nocheck (don't run) vs nodoc (don't build)

[Simon McVittie]

On Wed, 26 Apr 2023 at 23:24:58 +0200, Christian Kastner wrote:
> On 2023-04-26 20:42, Russ Allbery wrote:
> > It's just
> > less common (although certainly not unheard of) for test suites to have
> > test-suite-only build dependencies (as opposed to test-only runtime
> > dependencies, which are very common in at least the Perl world).
> 
> We (the ROCm team) have a few instances of such test suites, namely
> header-only libraries. Upstream ships tests that require compilation,
> and we'd like to ship these tests for autopkgtest purposes.

I think test-suite-only build dependencies are actually quite common.
At least, I see them a lot.

But DEB_BUILD_OPTIONS are not a mechanism to reduce build-dependencies:
they *cannot* reduce build dependencies. If you want to reduce
build-dependencies, you have to use build profiles.

nocheck is both a build option and a build profile, with a dependency
between them: using the build profile documented as requiring also using
the option (in other words, if you try to build with DEB_BUILD_OPTIONS=""
and DEB_BUILD_PROFILES=nocheck, that's undefined behaviour and it's OK
for packages to FTBFS). So the three valid things are:

DEB_BUILD_OPTIONS="" DEB_BUILD_PROFILES="" - normal

DEB_BUILD_OPTIONS="nocheck" DEB_BUILD_PROFILES="" - don't run
the test suite to save some time or work around known failures

DEB_BUILD_OPTIONS="nocheck" DEB_BUILD_PROFILES="nocheck" - same, but
also try to avoid installing test-only build-dependencies where feasible

Any of these can be combined with DEB_BUILD_PROFILES += " noinsttest"
to disable the build of "as-installed" tests like fwupd-tests.

> Strictly speaking, [the build-profiles spec] says build dependencies
> for the purposes of
> _running_ should be ignored, which makes perfect sense of course. But
> what if there is another purpose, specifically shipping autopkgtests?
> Like packages 'systemd-tests', or 'fwupd-tests'?

Then you probably need the same logic as in src:dbus and src:glib2.0,
which is this pseudocode, so that you'll compile with tests enabled
whenever there is at least one reason to want them:

Package: foobar
Build-Depends: thing-needed-to-compile-tests  ,
   thing-needed-to-run-tests 

Package: foobar-tests
Build-Profiles: 

# note: OPTIONS not PROFILES
want build-time tests := not (nocheck in DEB_BUILD_OPTIONS)

# note: PROFILES not OPTIONS
want installed-tests := not (noinsttest in DEB_BUILD_PROFILES)
# or equivalently and often clearer:
# want installed-tests := (foobar-tests in dh_listpackages output)

override_dh_auto_configure:
if want build-time tests or want installed-tests:
--enable-tests or equivalent
else:
--disable-tests or equivalent

# recent dh_auto_test does this internally, but you might still need
# this logic if the tests have non-trivial setup/teardown
override_dh_auto_test:
if want build-time tests:
test setup
dh_auto_test
test teardown
else:
do nothing

I can never remember without looking it up whether
  or  is the right syntax
for this, but I'm reasonably sure I got it right in dbus and glib2.0 (by
testing all four combinations), so it must be  .

Any build-profile is OK to leave unimplemented (as long as nobody is
relying on it to avoid cyclic dependencies during bootstrapping, and
if they are, they'll tell you); so if you don't care about minimizing
build-dependencies, it's OK to simplify this by behaving as though
noinsttest was never used. But in that case you would always have to
build your tests (or at least the subset of them that end up installed
in foobar-tests), even if they will not be run.

> But that will take me a while, as I clearly need to think this through a
> bit more. Because it just dawned me that for a header-only library, the
> argument could be made that autopkgtests should always build the tests
> from scratch anyways. Or maybe even both (run, and build-and-run).

For the special case of a header-only library, yes I think the autopkgtest
should ideally rebuild the test executable every time, because that's
what real consumers of your API have to do, and it confirms that if your
header-only library is doing some tricky C++ thing, it hasn't been broken
by a newer toolchain. (But for autopkgtest, it should really be compiled
against the header-only library from /usr/include, and not against the copy
of hopefully the same headers in the source tree!)

For the more common case of a shared library, I think it's desirable to
*not* rebuild the tests every time, as a way to verify that the ABI of
the new shared library does not cause pre-existing binaries to regress.
If the test was rebuilt every time, it would silently accept and ignore
ABI breaks, by always being compiled with the new header that describes
the new ABI.

Header-only libraries are special because they don't have ABI, only AP

Re: nocheck (don't run) vs nodoc (don't build)

[Simon McVittie]

On Wed, 26 Apr 2023 at 18:59:46 +0200, Christian Kastner wrote:
> Policy 4.9.1 states that (emphases mine):
>   * "[nocheck] says not to *run* any build-time test suite"
>   * "[nodoc] says to skip any *build* steps"
> 
> My reading with regards to 'nocheck' was that where tests were available
> and needed to be built, then they should always be built, just not run.
> 
> A typical example might be a C library package that builds those tests
> and ships them as autopkgtests, maybe even in a dedicated package.
> 
> I thought this line of reasoning was sound, but then I remembered the
> 'nodoc' tag and now I am no longer sure. Maybe I'm taking the 'nocheck'
> description too literally.

With RFC 2119 terms, my opinion would be:

- packages built with the nocheck option SHOULD NOT run tests
- the nocheck option SHOULD NOT alter the contents of any binary package
- packages built with the nocheck option MAY avoid building the tests at
  all, but only if that doesn't change the binary packages

The nocheck option (in Policy) is related to the nocheck build-profile
(not part of Policy). https://wiki.debian.org/BuildProfileSpec documents
that DEB_BUILD_PROFILES=nocheck SHOULD NOT change either the contents of
any binary package or the set of binary packages that are compiled, and
also documents that DEB_BUILD_PROFILES=nocheck MUST be accompanied by
DEB_BUILD_OPTIONS=nocheck.

For the use-case of choosing whether to build tests and ship them
in a .deb for autopkgtest or manual testing, we have the noinsttest
build-profile ("Disable binary packages consisting entirely of automated
tests, manual tests, example/demo programs and test tools").

For example, see dbus, which uses GLib in many of its automated tests,
but not in its production code. It build-depends on

libglib2.0-dev  

which means it usually needs GLib, but if it is built with *both* nocheck
and noinsttest (meaning don't run build-time tests, *and* don't build
dbus-tests_*.deb for later use by autopkgtest) then GLib isn't required.

smcv

Bug#1026231: debian-policy: document droppage of support for legacy locales

[Simon McVittie]

On Fri, 20 Jan 2023 at 09:54:21 -0700, Anthony Fok wrote:
> supposedly some older Chinese websites are still using "GBK" as
> encoding, probably something like:
> 
>  
> 
> which has less than 30,000 characters and thus a very limited subset
> of Unicode.  And, presumably not everyone has the know how to convert
> to UTF-8, the Chinese government wants those unable to at least change
> that meta tag to:
> 
>  

Sure, but neither of those actually require us to support GBK or GB
18030 as a system locale, only as something that iconv() (or whatever
browsers actually use, which is probably their own thing) can convert
into their preferred internal representation (which is almost certainly
UTF-8, UTF-16 or UCS-4).

Analogously, we've never supported using Windows-1252 (Microsoft's
legacy Latin-1 variant) as a system locale encoding in some hypothetical
locale like en_US.windows-1252, but HTML documents with
text/html;charset=windows-1252 still work fine.

> I have the feeling that many tech-savvy Chinese have already switched
> to UTF-8, but then perhaps in some circles there are lots of legacy
> GB2312/GBK documents or systems that made GB18030 a necessity, as an
> intermediate step to Unicode.

That doesn't seem so far away from how in some English-speaking circles
there are lots of legacy ISO-8859-1, ISO-8859-15 or (more likely)
Windows-1252 documents, and we can cope OK with those via transcoding,
even in UTF-8 system locales.

smcv

Bug#1026231: debian-policy: document droppage of support for legacy locales

[Simon McVittie]

On Wed, 18 Jan 2023 at 16:30:46 -0700, Anthony Fok wrote:
> In their mind, GB 18030 encompasses a lot more than just
> a character encoding mapping table.  It is the full support package
> (including fonts, display, printing, input methods, etc.) for Han
> Chinese and all other minority languages used in China.

If I'm reading correctly, the character encoding part of GB 18030-2022 is
a subset of a sufficiently new version of Unicode, in the same way that
(say) ISO-8859-15 is a subset of Unicode: for every character representable
in GB 18030-2022, you can point at an equivalent Unicode character and say
"this is the GB 18030-2022 encoding of U+4E00" or similar? Is that true?

If that's the case, then supporting text files written in GB 18030
does not *necessarily* require the internal representation or the
system locale to be GB 18030, the same way I can still work with legacy
en_GB.ISO-8859-15 files on my en_GB.UTF-8 system: it could equally well
be done by using iconv() or equivalent to transcode to UTF-8, UTF-16 or
UCS-4 on input, doing all text editing operations on that Unicode, and
then transcoding back into GB 18030 on output. Most language frameworks
already do this as a matter of API: Qt, Java and Windows tend to work
with UTF-16 internally, while GLib/GTK uses UTF-8 internally.

iconv() seems very unlikely to drop support for GB 18030, ISO-8859-15 and
other non-Unicode encodings altogether. What this bug report is about is
dropping support for locales whose associated encoding is non-Unicode,
such as en_GB.ISO-8859-15 and zh_CN.GB18030, so that the data stream
between a CLI program and the terminal emulator will be assumed to be UTF-8
instead of ISO-8859-15 or GB18030.

The main thing I can see that would be a problem for GB 18030 users
if the zh_CN.GB18030 locale was dropped is that various programs might
assume that the locale encoding is the right one to assume when loading
existing files and unable to guess the encoding, or the right one to
write into new files by default - and so users who have moved from
zh_CN.GB18030 to zh_CN.UTF-8 might find themselves unintentionally
producing new UTF-8 files.

Preferring to use Unicode does seem to be the direction that all of
computing is going in, as a simplifying assumption - for example W3C
advice for HTML is "You should always use the UTF-8 character encoding"[1]
- and as we know, things that aren't tested usually don't work. So I
think the level of functionality for non-UTF-8 locales and encodings in
the software we package is going to decline over time, whether Debian
wants it to or not.

smcv

[1] https://www.w3.org/International/questions/qa-html-encoding-declarations

Bug#1026231: debian-policy: document droppage of support for legacy locales

[Simon McVittie]

On Wed, 21 Dec 2022 at 18:15:11 +0100, Adam Borowski wrote:
> On Mon, Dec 19, 2022 at 07:08:09PM +0000, Simon McVittie wrote:
> > On Fri, 16 Dec 2022 at 19:21:37 +0100, Adam Borowski wrote:
> > > * The execution environment (usually init system or a container) must
> > >   default to UTF-8 encoding unless explicitly configured otherwise.
> > 
> > Is this already true? This seems like the sort of thing which should be
> > fixed in at least the major init systems and container managers before it
> > goes into Policy, in the interests of not making those init systems and
> > container managers retroactively buggy.
> 
> I'm less knowledgeable about containers, but they appear to work.  It might
> be due to copying variables from the host or having template defaults...

There are three major categories of container that I'm aware of:

1. Full-system containers like lxc/lxd run a complete init system like
   systemd or sysv-rc for the container (they aim to behave like a
   lighter-weight alternative to VMs), so their init system would be
   responsible for making this work. This seems non-problematic for your
   proposed requirement: if an init system does the right thing on bare
   metal or in a VM, it will also do the right thing in lxc containers.

2. Per-service containers like the upstream-recommended use of Docker either
   have no init system at all, or a minimal reaper process like tini
   (they aim to behave like a heavier-weight alternative to chroots).
   chroot managers that have subsequently gained namespace functionality,
   like some uses of pbuilder and schroot, also work like this.
   I think these are the category that is most likely to have trouble
   complying with the requirement you propose, because the container manager
   is intentionally "hands-off" (mechanism more than policy), while the
   processes run inside the container are not under Debian's control (they
   are chosen by whoever wrote the Dockerfile or equivalent, and might come
   from either Debian or another distribution).

3. Per-app containers like Flatpak and Snap are not intended to emulate a
   whole system, so they are expected to inherit locale settings from the
   host system like a non-sandboxed app would. They shouldn't be a problem
   here, as long as your proposed requirement is worded in such a way that
   it is valid for these container managers to rely on the host system
   locale being correct (in other words, if someone using the legacy en_GB
   locale reports a bug "flatpak: does not set a UTF-8 locale", I should be
   able to close it with "This is not flatpak's job, set your host system
   locale to en_GB.UTF-8 instead").

podman and systemd-nspawn can be used as either the first category
(like lxc) or the second (like Docker), depending how they were invoked.

> Anyway, my aim is more to tell packages that they are allowed to misbehave
> when the settings are missing than to hunt misuse scenarios.  But, if such
> a scenario is found, with the current Policy there is no recourse, while
> if this rule is added it would be a bug.

Not every bug necessarily needs to be a Policy violation.

> I just tested Windows 11 notepad.exe: it defaults to UTF-8, and when
> saving it allows "ANSI" "UTF-16 LE" "UTF-16 BE" "UTF-8" (default) and
> "UTF-8 with BOM".

Yes, that's the sort of UX that I think needs to be allowed. I would
personally not expose that choice in the UI of an intentionally simple
text editor like Notepad or gnome-text-editor, but I would expect similar
behaviour in an editor with more elaborate programmer-oriented features,
like vim, emacs, gnome-builder or kate.

If iconv(1) or a similar program has an option for "UTF-8 with BOM" then
that also needs to not be accidentally declared to be a Policy violation.

smcv

Bug#1026231: debian-policy: document droppage of support for legacy locales

[Simon McVittie]

On Fri, 16 Dec 2022 at 19:21:37 +0100, Adam Borowski wrote:
> As of Bookworm, legacy locales are no longer officially supported.

For clarity, I think when you say "legacy locales" you mean locales
whose character encoding is either explicitly or implicitly something
other than UTF-8 ("legacy national encodings"), like en_US (implicitly
ISO-8859-1 according to /usr/share/i18n/SUPPORTED) and en_GB.ISO-8859-15
(explicitly ISO-8859-15 in its name). True?

Many of the non-UTF-8 encodings are single-byte encodings in the
ISO-8859 family, but if I understand correctly, your reasoning applies
equally to multi-byte east Asian encodings like BIG5, GB18030 and EUC-JP.
Also true?

Meanwhile, locales with a UTF-8 character encoding, like en_AG
(implicitly UTF-8 according to /usr/share/i18n/SUPPORTED) or en_US.UTF-8
(explicitly UTF-8), are the ones you are considering to be non-legacy.
Also true?

I think for Policy use, this would have to say something more precise,
like "locales with a non-UTF-8 character encoding". I wouldn't want to
get en_US speakers trying to argue that en_GB.UTF-8 is a legacy locale,
or en_GB speakers like me trying to argue that en_US.UTF-8 is a legacy
locale :-)

When you say "officially supported" here, do you refer to the extent
to which they are supported by the glibc maintainers, or some other
group? Or are you describing a change request that they *should not*
be officially supported by Debian - something that is not necessarily
true yet, but in this bug you are asking for it to become true?

> * Software may assume they always run in an UTF-8 locale, and emit or
>   require UTF-8 input/output without checking.

I suspect this is already common: for example, ikiwiki is strictly
UTF-8-only and ignores locales' character sets, which is arguably a bug
right now but would become a non-bug with your proposed policy.

This is a "may" so it can't possibly make a package gain bugs. It might
make packages have fewer bugs.

> * The execution environment (usually init system or a container) must
>   default to UTF-8 encoding unless explicitly configured otherwise.

Is this already true? This seems like the sort of thing which should be
fixed in at least the major init systems and container managers before it
goes into Policy, in the interests of not making those init systems and
container managers retroactively buggy.

> * Legacy locales are no longer officially supported, and packages may
>   drop support for them and/or exclude them from their testsuites.
> * Packages may retain support for legacy locales, but related bug reports
>   (unless security related) are considered to be of wishlist severity.

Is the C (aka POSIX) locale still a non-UTF-8 locale (if I remember
correctly its character encoding is officially 7-bit ASCII), or has it
been redefined to be UTF-8? Given the special status of the C locale in
defaults and standards, it might be necessary to say that it's the only
supported locale with a non-UTF-8 character encoding.

> * Filesystems may be configured to reject file names that are not valid
>   printable UTF-8 encoded Unicode.

To put this in terms of the requirements that Policy puts on packages,
is this really a should/must in disguise: packages should/must not
assume that they can successfully read/write filenames that are not valid
printable UTF-8-encoded Unicode?

This seems like a change with a wider scope: not only is it excluding
filenames in Latin-1 or whatever, it's also excluding filenames with
non-printable characters (tabs, control characters etc.), or with
the UTF-8 representation of a noncharacter like U+FDEF. Perhaps that
should be a change orthogonal to de-supporting the non-UTF-8 locales?

> * Human-readable files outside of packages' private data must be encoded
>   in UTF-8.  This applies especially to files in /usr/share/doc and /etc
>   but applies to eg. executable scripts in /bin or /sbin as well.

It's not immediately obvious to me what "human-readable files" means here.
Text files? Text files in ASCII-compatible encodings? Files intended to be
read and written by standard text editors?

I assume the intention here is to make it a policy violation to ship
documentation, scripts, configuration files, etc. encoded in something
like ISO-8859-1 or EUC-JP?

Is this intended to make it a policy violation to ship documentation, etc.
encoded in UTF-16?

> * So-called BOM (U+FEFF) must not be added to plain-text output, and if
>   present, editors/viewers customarily used for editing code should not
>   hide its presence.

This seems to me like it should perhaps be out-of-scope here, and treated
as a separate change: UTF-8 is still UTF-8, whether it starts with U+FEFF
or not, and I think deprecating en_GB in favour of en_GB.UTF-8 (and so on)
is orthogonal to deprecating the use of a U+FEFF prefix on UTF-8 text.

I think "UTF-8 output" is probably a better scope for this than
"plain-text output": my understanding is that when emitting UTF-16, UCS-2
or UCS-4 it's conv

Bug#1015784: source-only upload requirement not documented

[Simon McVittie]

On Thu, 21 Jul 2022 at 10:44:26 +0200, Marc Haber wrote:
> for a few years now, the Debian archive wants to see source-only
> uploads. This is not documented in the Developer's Reference and also
> now in the New Maintainer's Guide. It should be there.
> 
> Also, it should be documented that the first upload of a new package to
> debian MUST be a source+binary upload. Arch all packages need another
> source-only upload after being accepted into the archive to be allowed
> to migrate to testing.

I've had the attached sitting in my outbox for a while and I think it's at
least a good start towards what Marc requests?

I have deliberately not documented the precise meaning of needing to
include binary packages for NEW, on the basis that the conservative thing
to do is to include a complete set (debuild --full). My understanding is
that *technically*, the upload does not need to include *every* binary
package, but that the ftp team would prefer uploads to NEW to include
everything from debuild --full, except in rare special cases such as
the kernel, whose maintainers already know what all the tradeoffs are.

Similarly, I have deliberately been a bit vague about whether uploads
that will add a package to a suite other than unstable/experimental
need binaries, because I don't know whether they do or
not. unstable/experimental NEW definitely needs binaries, I *think*
backports-NEW also needs binaries but I'm not sure, but I think new
additions to -security can/should be source-only?

I have also not attempted to improve §5.10 "Porting and being ported",
which seems to have been written from a circa 1998 perspective where all
maintainers uploaded source+i386, binaries for other architectures were
often hand-built by porting teams, and the ability for a package to be
autobuilt successfully was somewhere between "optional but recommended"
and "newly required". It could probably benefit from restructuring or a
rewrite, but I don't think I'm the right writer for that.

> I THINK that arch any packages get an automatic binNMU to avoid that.

My understanding is that the release team often schedule a binNMU to
be helpful, but it is not automatic. We can give simpler advice if we
ignore these binNMUs, which seems better to me anyway - maintainers of
source packages with at least one "Architecture: all" binary package
have to do a sourceful upload regardless, and I'd prefer to encourage
maintainers to be responsible for their packages' migration to testing
rather than centralising that responsibility into the release team.

smcv
>From 96ea2e40f43ce32895d3d2a30e1b5c3319aa1540 Mon Sep 17 00:00:00 2001
From: Simon McVittie 
Date: Mon, 1 Nov 2021 23:30:12 +
Subject: [PATCH] Recommend source-only uploads when possible, and describe
 when they're not

Closes: #1015784
---
 source/pkgs.rst | 58 +
 1 file changed, 58 insertions(+)

diff --git a/source/pkgs.rst b/source/pkgs.rst
index 11f28cd..7f35e77 100644
--- a/source/pkgs.rst
+++ b/source/pkgs.rst
@@ -336,6 +336,64 @@ details.
 Uploading a package
 
 
+Source and binary uploads
+
+
+Each upload to Debian consists of a signed ``.changes`` file describing
+the requested change to the archive, plus the source and binary package
+files that are referenced by the ``.changes`` file.
+
+If possible, the version of a package that is uploaded should be a
+source-only changes file.
+These are typically named ``*_source.changes``, and reference the source
+package, but no binary ``.deb`` or ``.udeb`` packages.
+All of the corresponding architecture-dependent and architecture-independent
+binary packages, for all architectures, will be built automatically by
+the build daemons in a controlled and predictable environment
+(see :ref:`wanna-build` for more details).
+However, there are several situations where this is not possible.
+
+The first upload of a new source package (see :ref:`newpackage`)
+must include binary packages, so that they can be reviewed by the
+archive administrators before they are added to Debian.
+
+If new binary packages are added to an existing source package, then the
+first upload that lists the new binary packages in ``debian/control``
+must include binary packages, again so that they can be reviewed by the
+archive administrators before they are added to Debian.
+It is preferred for these uploads to be done via the ``experimental``
+suite.
+
+Uploads that will be held for review in other queues, such as packages
+being added to the ``*-backports`` suites, might also require inclusion
+of binary packages.
+
+The build daemons will 

Bug#968226: Move documentation of Build-Depends alternative selection out of footnote

[Simon McVittie]

On Thu, 22 Sep 2022 at 19:11:38 -0700, Russ Allbery wrote:
> I also reworded the paragraph about backports to hopefully address
> Holger's reading.  It's just trying to say that backports uses aptitude in
> the normal way and doesn't do anything special to transform the
> alternative.

It's perhaps worth mentioning that experimental does something similar
(it has used the aptitude and aspcud resolvers at various times, but
I'm not sure which one is currently in use).

smcv

Bug#975631: debian-policy: window manager: remove reference to Debian menu

[Simon McVittie]

On Tue, 20 Sep 2022 at 21:45:28 -0700, Russ Allbery wrote:
> I just found https://bugs.debian.org/838777, which says packages that only
> provide a window manager without a mechanism for launching programs should
> not register as x-window-manager

See also https://bugs.debian.org/1004522 in which I proposed adding a
wayland-session virtual package, and in the process, ended up also proposing
x-session for /usr/share/xsessions/*.desktop.

In particular https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1004522#30
has a brief survey of what display managers actually do, and
in https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1004522#66 I tried to
document existing practice (which contradicts #838777).

> try to document what x-window-manager is really for in the new X world

This is part of what I tried to do in #1004522, but I was trying to
document what it is now (including literally-only-a-window-manager window
managers like mutter, not just tiny-desktop-environment window managers
like twm or openbox), rather than what it should be.

If we redefine x-window-manager to mean something that is more strict than
it is now, then we'll have a mixture of x-window-manager implementations
that meet the new requirement, and (perhaps essentially unmaintained)
x-window-manager implementations that don't; so I'm not sure that's such a
good idea. That's why I proposed adding a new virtual package instead.

I personally think the fully-integrated-system approach should be
that packages that want to be considered to be a session that you
can log into (potentially ranging from GNOME/KDE all the way down to
twm) should register themselves in /usr/share/xsessions/*.desktop
(or /usr/share/wayland-sessions/*.desktop), user-friendly display
managers like gdm/lightdm/sddm should list those and only those,
and people who want to construct their own tiny desktop environment
out of a window manager and some xterms should enable it by installing
something resembling
https://salsa.debian.org/gnome-team/gdm/-/blob/debian/master/debian/custom-x11-session.desktop
into /etc/X11/sessions. gdm3 installs that file into
/usr/share/doc/gdm3/examples for sysadmin convenience, but intentionally
does not install it in the search path.

Rationale: people who want to piece together their own desktop environment
programmatically are welcome to do so, but that setup inherently requires
configuration. Trying to list ~/.xsession in UIs is confusing to novice
users, because the display manager cannot know what it will result in,
so its user-facing name has to be either very technical ("Xsession")
or hopelessly vague (older gdm's "System X11 Default") or both; so we
should not inflict that UX on users who have not done the necessary
setup to get it to behave as they want it to. For a setup that already
requires configuration before it will be usable, adding one more piece
of configuration doesn't seem like an undue burden.

gdm3 (>= bookworm), sddm, slim and lxdm already follow what I'm advocating
here.

smcv

Bug#696185: [copyright-format] Use short names from SPDX

[Simon McVittie]

Please keep the subject line in place when replying to bugs, to give
readers some context (maintainers will often be seeing bug mail as a
single message among many unrelated messages).

On Sat, 03 Sep 2022 at 16:22:46 +0500, Akbarkhon Variskhanov wrote:
> FSF[1] as well as SPDX[2] request using the suffixes "-only" or
> "-or-later" with GNU licenses:
> 
> > Therefore, when you use SPDX license indicators, please use these:
> > GPL-2.0-only or GPL-2.0-or-later
> 
> DEP-5 uses the bare form, i.e. "GPL-3" or "GPL-3+". I added this
> difference to the wiki page.
> 
> What's not clear is how are we going to approach this discrepancy?
> Shall we a) ignore this, b) adopt SPDX/FSF naming, or c) suggest SPDX
> to stick to uniform naming convention, which is using "+" to denote
> later versions of a license?

Sorry, (c) seems very unlikely: earlier versions of SPDX had the same
convention as DEP-5, but later versions moved to "GPL-2.0-only" and
"GPL-2.0-or-later", which I think was the result of a request from the
FSF to make it clearer whether the "or later" clause of the {A,L,}GPL
family was allowed or excluded.

Forms like GPL-2.0, LGPL-3.0+ and so on are still listed in
https://spdx.org/licenses/ as deprecated equivalents of GPL-2.0-only,
LGPL-3.0-or-later and so on.

I would personally be in favour of (b) as our long-term direction,
but for now the status quo is basically a variation of (a): keep using
the Debian-specific names where they exist, but where there is no
Debian-specific name for a license, the SPDX name is as good a name as
any other.

> [2] https://spdx.dev/ids/

I believe the canonical reference for the SPDX license identifiers is
https://spdx.org/licenses/ which also lists all the deprecated forms.

smcv

Bug#1004522: debian-policy: Proposing new virtual packages: wayland-session, x-session

[Simon McVittie]

On Fri, 18 Feb 2022 at 09:15:24 -0700, Sean Whitton wrote:
> Some time has passed without objections so I would be happy to add these
> two virtual packages.  My only concern is that there are going to be
> quite a few virtual packages in this area, now, and the virtual packages
> list will contain only terse descriptions of each one.  Someone
> packaging a new session/display manager/WM might have a hard time
> figuring out what to use, unless they happen to stumble across the
> discussion in this bug.

I'm slightly regretting opening this can of worms now, but here's my
best attempt at Policy wording.

§(Packages providing an X session manager) might be incomplete: it's not
clear to me whether there are any particular requirements for an
x-session-manager (does it have to implement XSMP?) or whether it's as
vague as "anything that is a larger X11 session than an x-window-manager".

smcv
>From 0ddc90382dda161060459f286f81fe26ecaffc4e Mon Sep 17 00:00:00 2001
From: Simon McVittie 
Date: Fri, 18 Feb 2022 20:34:33 +
Subject: [PATCH] Describe how to register X11 and Wayland sessions

Closes: #1004522
---
 policy/ch-customized-programs.rst | 136 +-
 virtual-package-names-list.yaml   |   6 +-
 2 files changed, 139 insertions(+), 3 deletions(-)

diff --git a/policy/ch-customized-programs.rst b/policy/ch-customized-programs.rst
index bf8dabb..85a5e79 100644
--- a/policy/ch-customized-programs.rst
+++ b/policy/ch-customized-programs.rst
@@ -274,8 +274,8 @@ configuration.
 
 .. _s11.8:
 
-Programs for the X Window System
-
+Programs for the X Window System and Wayland
+
 
 .. _s11.8.1:
 
@@ -363,6 +363,19 @@ That alternative should have a slave alternative for
 ``/usr/share/man/man1/x-window-manager.1.gz`` pointing to the
 corresponding manual page.
 
+If the window manager acts as a desktop environment in its own right,
+including mechanisms to launch programs, exit the session and so on,
+so that it would be appropriate for it to appear as a desktop environment
+that can be chosen by users in a display manager such as gdm or sddm,
+then it may install a ``.desktop`` file in ``/usr/share/xsessions``
+(see :ref:`s-x-session` for details).
+
+Conversely, if the window manager is intended to be used as part of a
+larger desktop environment session, then it should not install a file into
+``/usr/share/xsessions``, and should not provide ``x-session``.
+Instead, the metapackage or session manager for the larger desktop
+environment session should be responsible for that integration.
+
 .. _s11.8.5:
 
 Packages providing fonts
@@ -502,6 +515,125 @@ name. Other X Window System applications should use the ``/etc/``
 directory unless otherwise mandated by policy (such as for
 :ref:`s-appdefaults`).
 
+.. _s-x-session:
+
+Packages providing an X Window System session
+~
+
+Packages that provide a user-facing X Window System graphical environment
+should install a ``.desktop`` file into ``/usr/share/xsessions``,
+and declare in their ``Provides`` control field that they provide the
+virtual package ``x-session``.
+
+The package providing ``x-session`` may be either a window manager in
+its own right (for example ``twm``), or a session manager that will
+launch multiple components including a window manager (for example
+``gnome-session``). If it meets the requirements for an
+``x-window-manager`` (see :ref:`s11.8.4`)
+and/or an ``x-session-manager`` (see :ref:`s-x-session-manager`), then
+it should also behave as described in the appropriate sections.
+
+As with Wayland sessions (see :ref:`s-wayland-session`),
+the format of the session files in ``/usr/share/xsessions`` is similar
+to the format defined in the
+`freedesktop.org Desktop Entry Specification
+<https://specifications.freedesktop.org/desktop-entry-spec/latest/>`_,
+although many of the fields normally used for applications
+(see :ref:`s-menus`) are not applicable in session files.
+
+The session file must contain at least the ``Name`` and ``Exec`` fields.
+Login managers are expected to either launch a new X server or reuse
+an existing X server, with no window manager active, and then launch
+the ``Exec`` program with its ``DISPLAY`` environment variable set to
+use that X server.
+
+In addition to the fields from the Desktop Entry Specification, graphical
+environments may provide a ``DesktopNames`` field in the
+``[Desktop Entry]`` group. If present, this field's value is a
+semicolon-separated list of graphical environment names, as used for the
+``OnlyShowIn`` and ``NotShowIn`` fields. As described in the Desktop Entry
+Specification, the login manager should set the ``XDG_CURRENT_DESKTOP``
+environment variable to contain the same names, but as a colon-separated
+list, before launching the ``Exec`` program.
+
+Login managers that provide

Bug#1004522: debian-policy: Proposing new virtual packages: wayland-session, x-session

[Simon McVittie]

On Sat, 29 Jan 2022 at 20:12:21 +, Simon McVittie wrote:
> I propose this entry for virtual-package-names-list.yaml:
> 
> - name: wayland-session
>   description: a Wayland desktop session 
> (/usr/share/wayland-sessions/*.desktop)

Having looked more closely at display managers, I think we should also
consider adding:

- name: x-session
  description: an X11 desktop session registered via 
/usr/share/xsessions/*.desktop

because there are three common ways for an x-display-manager to choose what
sessions to offer:

1. Older or simpler display managers (xdm, nodm) unconditionally run
   /etc/X11/Xsession or equivalent, which results in running ~/.xsessionrc,
   /etc/alternatives/x-session-manager or various similar things

2. Newer display managers (gdm3, sddm, slim, lxdm) look for Wayland
   sessions and/or X11 sessions represented by .desktop files in various
   locations[1], of which the most suitable for distribution packagers are
   /usr/share/xsessions and /usr/share/wayland-sessions

3. Some of the .desktop-based display managers (lightdm and historically
   gdm3) additionally install their own .desktop file to make the
   xdm-style /etc/X11/Xsession or equivalent (from 1.) appear in the menu
   of .desktop files (from 2.)

If a display manager is in category 1, it would be appropriate for it to
depend on x-session-manager | x-window-manager | x-terminal-emulator.

However, if a display manager is in category 2, then that dependency is
actually incorrect, because not every x-window-manager is designed to
be usable as a desktop session in its own right (some of them are just
components of a larger desktop environment, like KDE's kwin-x11 as a
component of the Plasma desktop), and therefore not every x-window-manager
has (or should have!) a file in /usr/share/xsessions/*.desktop.

Strictly speaking I don't think there's any guarantee that every
x-session-manager has /usr/share/xsessions/*.desktop either, although
it seems to be 90% true for current packages in unstable[2].

So I think x-display-manager implementations in category 2 should be
depending on [their preferred session] | wayland-session | x-session
(omitting wayland-session if they are X11-only), perhaps with
x-session-manager as an additional alternative for backwards compatibility.

Alternatively, display managers could relax that dependency to a Recommends
or Suggests, or omit it completely (in fact this seems to be what most
display managers do at the moment). However, gdm3 has historically had a
dependency on something resembling a desktop environment, and I would not
want to drop that dependency unless there is consensus that installing
gdm3, without also installing a desktop environment that it can log into,
is a sysadmin error rather than a bug in gdm3.

I tested all x-display-manager implementations in sid and it looks like
they break down like this:

1. traditional /etc/X11/Xsession: xdm, nodm
1a. /etc/X11/Xsession plus a hard-coded list of extra options?: wdm
2. xsessions/*.desktop: gdm3, sddm, slim, lxdm
3. combining (1.) and (2.): lightdm

I think category 3 leads to a bad user experience, which is why I removed
the equivalent from gdm3: the display manager cannot predict what will
happen if /etc/X11/Xsession is chosen, so it has to name the option
something very general, like lightdm's "Default Xsession" or older gdm3
versions' "System X11 Default". This is particularly misleading if the
display manager actually has a different default, as gdm3 has done since
the switch to Wayland-by-default. (See also lightdm bug #1004559.)

smcv

[1] For example gdm3 currently searches /usr/{,local/}share/xsessions,
/etc/X11/sessions, /etc/gdm3/Sessions, /usr/share/gdm/BuiltInSessions
and /usr/{,local/}share/wayland-sessions, some of which are gdm-specific
[2] cinnamon-session Provides x-session-manager, but
/usr/share/xsessions/cinnamon.desktop is in cinnamon-common, a
"larger" package.
lxsession Provides x-session-manager, but
/usr/share/xsessions/LXDE.desktop is in openbox-lxde-session, again
a "larger" package.

Bug#998063: debian-policy: New virtual package: {default-,}dbus-system-bus

[Simon McVittie]

On Thu, 23 Dec 2021 at 21:26:53 -0700, Sean Whitton wrote:
> On Fri 29 Oct 2021 at 11:12AM +01, Simon McVittie wrote:
> > it seems like a good time to introduce {default-,}dbus-system-bus
> > virtual packages, mirroring {default-,}dbus-session-bus.
> 
> virtual-package-names-list.yaml says that proposed new virtual package
> names are meant to be sent to d-devel, not just filed as a bug against
> debian-policy, so perhaps you could do that and we'll give it a week,
> then I'll go ahead and add these?

I sent this to -devel in December. There were no objections, and one
positive reply.

Thanks,
smcv

Bug#1004522: debian-policy: Proposing new virtual package: wayland-session

[Simon McVittie]

Package: debian-policy
Version: 4.6.0.1
Severity: wishlist
X-Debbugs-Cc: debian-de...@lists.debian.org

GNOME's gdm3 and KDE's sddm both enumerate possible Wayland sessions in
/usr/{,local/}share/wayland-sessions/*.desktop and make them available
as desktop sessions that users can choose, in addition to listing the
X11 sessions that they traditionally did.

At the moment, installing gdm3 pulls in either gnome-session (a minimal
GNOME desktop), or some sort of X11 thing (usually a session manager,
but sometimes a window manager or an xterm), but it should ideally
be possible to install gdm3 as a login prompt from which to launch a
non-GNOME Wayland session like weston or sway.

I propose this entry for virtual-package-names-list.yaml:

- name: wayland-session
  description: a Wayland desktop session (/usr/share/wayland-sessions/*.desktop)

According to `apt-file search`, it should initially be provided by these:

gnome-session: /usr/share/wayland-sessions/gnome.desktop
phosh: /usr/share/wayland-sessions/phosh.desktop
plasma-workspace-wayland: /usr/share/wayland-sessions/plasmawayland.desktop
sway: /usr/share/wayland-sessions/sway.desktop
weston: /usr/share/wayland-sessions/weston.desktop

and perhaps also (I don't know how practical this one is for actual use):

mir-demos: /usr/share/wayland-sessions/mir-shell.desktop

Rationale for not using the names people are probably going to suggest:

- wayland-compositor would be wrong, because it's too low-level. Some
  Wayland compositors are a somewhat complete desktop environment in their
  own right, but for example plasma-workspace-wayland and gnome-session
  are larger components that merely *depend on* a Wayland compositor, plus
  the additional components needed to get a practical desktop environment;
  meanwhile, kwin-wayland and gnome-shell are Wayland compositors, but
  are not desktop environments on their own.

- wayland-session-manager seems like it would be misleading, because an X
  session manager has specific functional expectations (XSMP) separating
  it from an mere x-window-manager, but there's no such thing in Wayland.

smcv

Bug#998165: debian-policy: document and allow Description in the source paragraph

[Simon McVittie]

On Mon, 27 Dec 2021 at 15:08:03 -0700, Nicholas D Steeves wrote:
> The following is only Informational level, but the existence of
> Lintian's "duplicate-long-description" tag suggests that producing
> duplicate bin:Descriptions in bin:libfoo and bin:foo packages is not
> ideal, thus a straight copy from src:Description is not ideal.  I'm not
> sure what the best way to solve this is, but substvar looks like a good
> solution.

I believe the intention is to automate this pattern, which a lot of
packages with shared libraries are already using:

Source: dbus

Package: dbus
Description: simple interprocess messaging system (system message bus)
 D-Bus is a message bus, used for sending messages between applications.
 [the real Description goes into more detail here]
 .
 This package provides a fully-functional D-Bus system bus [etc.]

Package: libdbus-1-3
Description: simple interprocess messaging system (library)
 D-Bus is a message bus, used for sending messages between applications.
 [the real Description goes into more detail here]
 .
 This package provides the runtime library for use by applications.

Package: libdbus-1-dev
Description: simple interprocess messaging system (development files)
 D-Bus is a message bus, used for sending messages between applications.
 [the real Description goes into more detail here]
 .
 This package provides development headers and a static library.

by turning it into something like this:

Source: dbus
Description: simple interprocess messaging system
 D-Bus is a message bus, used for sending messages between applications.
 [the real Description goes into more detail here]

Package: dbus
Description: simple interprocess messaging system (system message bus)
 ${source:Description}
 .
 This package provides a fully-functional D-Bus system bus [etc.]

Package: libdbus-1-3
Description: simple interprocess messaging system (library)
 ${source:Description}
 .
 This package provides the runtime library for use by applications.

Package: libdbus-1-dev
Description: simple interprocess messaging system (development files)
 ${source:Description}
 .
 This package provides development headers and a static library.

which eliminates a lot of the duplication. Is that correct?

> This appears to conflict with the "duplicate-long-description" tag.  Of
> course, Lintian isn't Policy

Lintian tags/hints aren't a reason to do something on their own: the
causality should be the other way round (Lintian should remind maintainers
about things that are already undesirable, rather than something being
undesirable solely because Lintian says so).

However, the rationale given in the long descriptions of Lintian
tags/hints should point to a reason why it's better to avoid the tagged
behaviour, and that reason is the thing to pay attention to.

In the case of binary package descriptions, I believe the reasoning is:
the Description of a package should tell you whether you might want to
install it. If the Description is identical, then by definition it can't
tell you why you would want to install dbus but not libdbus-1-dev, or
vice versa; and if there is no reason why you would want to install one
but not the other, then they should usually be combined into one larger
package (although multiarch and Architecture: any vs. all sometimes mean
that things need to be split for technical reasons even though there is
no user-facing reason for them to be separate).

smcv

Bug#998063: debian-policy: New virtual package: {default-,}dbus-system-bus

[Simon McVittie]

On Fri, 29 Oct 2021 at 12:40:34 +0200, Adam Borowski wrote:
> On Fri, Oct 29, 2021 at 11:12:03AM +0100, Simon McVittie wrote:
> > * dbus, the portable reference implementation
> > * dbus-broker, a Linux-specific reimplementation
> > 
> > so it seems like a good time to introduce {default-,}dbus-system-bus
> > virtual packages, mirroring {default-,}dbus-session-bus.
> 
> > * default-dbus-system-bus | dbus-system-bus:
> >   the well-known system bus, as required by e.g. Avahi, polkit, udisks2
> 
> I wonder, perhaps it would be better to hijack the name "dbus", renaming
> the current bin:dbus package?  That'd allow avoid changing around a hundred
> packages.

"Depends: dbus" implies all the functionality of what's now referred
to as dbus-system-bus, dbus-bin and dbus-daemon, which is a strict superset
of what I am proposing that dbus-system-bus implies. Packages with
Depends: dbus have always been able to assume that they can run dbus-send,
or dbus-run-session, or dbus-daemon, among other executable entry points,
but dbus-system-bus does not guarantee that functionality.

We should not require reimplementations like dbus-broker to provide
(or more likely, depend on) dbus-daemon(1), dbus-send(1) and all the
other executables that "Depends: dbus" has historically given you.

This is not a "hard" transition, and I don't think changing those packages
will ever need to be RC: dbus-broker and dbus can coexist (dbus-broker
will "win" in this case), so the worst-case scenario is that an excessive
dependency wastes some disk space.

Some of the packages that depend on dbus could benefit from some review
anyway, because some of them seem to be GUI applications for which
the dbus package is neither necessary nor sufficient: packages like
rhythmbox and evolution probably don't strictly need the system bus, but
probably *do* need the well-known session bus (represented by
default-dbus-session-bus | dbus-session-bus, as implemented by
dbus-user-session | dbus-launch) which is not implied by dbus.

smcv

Bug#998063: debian-policy: New virtual package: {default-,}dbus-system-bus

[Simon McVittie]

Package: debian-policy
Version: 4.6.0.1
Severity: wishlist

We now have two implementations of the D-Bus well-known system bus
available in Debian:

* dbus, the portable reference implementation
* dbus-broker, a Linux-specific reimplementation

so it seems like a good time to introduce {default-,}dbus-system-bus
virtual packages, mirroring {default-,}dbus-session-bus.

At the moment, dbus is the default for all architectures. It is possible
that dbus-broker might take over as the default on Linux architectures
in some future release (but it is explicitly not portable, so dbus will
probably always be the default on kFreeBSD and Hurd, similar to how we
choose dbus-user-session vs. dbus-launch).

Packages depending on "dbus" can currently count on having most aspects of
the reference implementation available (except for the session bus, which
requires either dbus-user-session or dbus-launch), but I would prefer to
move towards packages explicitly declaring a dependency on one or more of:

* default-dbus-system-bus | dbus-system-bus:
  the well-known system bus, as required by e.g. Avahi, polkit, udisks2
* dbus-daemon: ability to run the dbus-daemon(1) and dbus-run-session(1)
  executables, or rely on having the D-Bus machine ID /var/lib/dbus/machine-id
  (dbus-session-bus and dbus-system-bus both imply some sort of machine
  ID, but it might be systemd's /etc/machine-id)
* dbus-bin: ability to run assorted CLI tools such as dbus-send(1) and
  dbus-monitor(1)

Proposed wording attached.

Thanks,
smcv
>From cc65839b65e9a41ca0e9e633ac32a085cec66fa2 Mon Sep 17 00:00:00 2001
From: Simon McVittie 
Date: Fri, 29 Oct 2021 10:58:13 +0100
Subject: [PATCH] virtual-package-names-list: Add dbus-system-bus,
 default-dbus-system-bus

This is the same as dbus-session-bus, but for the well-known system bus
described in the D-Bus Specification[1].

dbus, the reference implementation of D-Bus, Provides these since
bullseye. dbus-broker, an independent reimplementation, also Provides
dbus-system-bus in unstable.

The mention of "including service activation" is intended to make it
clearer that some implementation of /usr/share/dbus-1/system-services/
activation is required, for example via dbus' setuid helper and
conditional handoff to systemd, or dbus-broker's unconditional handoff
to systemd.

[1] https://dbus.freedesktop.org/doc/dbus-specification.html

Signed-off-by: Simon McVittie 
---
 virtual-package-names-list.yaml | 4 
 1 file changed, 4 insertions(+)

diff --git a/virtual-package-names-list.yaml b/virtual-package-names-list.yaml
index 2a9857a..eb5ace2 100644
--- a/virtual-package-names-list.yaml
+++ b/virtual-package-names-list.yaml
@@ -118,6 +118,10 @@ virtualPackages:
description: provides the D-Bus well-known session bus for most or all user login sessions
  - name: default-dbus-session-bus
description: Debian's preferred implementation of dbus-session-bus, possibly architecture-specific
+ - name: dbus-system-bus
+   description: provides the D-Bus well-known system bus as a system service, including service activation
+ - name: default-dbus-system-bus
+   description: Debian's preferred implementation of dbus-system-bus, possibly architecture-specific
  - name: logind
description: an org.freedesktop.login1 D-Bus API implementation (versioned)
  - name: default-logind
-- 
2.33.1

Bug#994008: debian-policy: Clarify relationship between source and binary packages' archive areas

[Simon McVittie]

On Thu, 09 Sep 2021 at 21:15:06 +0200, Ansgar wrote:
> On Thu, 2021-09-09 at 17:39 +0100, Simon McVittie wrote:
> > In the form of a table, the allowed source/binary combinations are:
> > 
> >   |    binary   |
> >   | main  contrib  non-free |
> >  -|-|
> >  main |  yes    yes  -  |
> >  source  contrib  |   - yes  -  |
> >  non-free |   -  -  yes |
> > 
> > ftp team: is this correct?
> 
> Yes. But source packages in main must also produce at least one binary
> package in main[1].

Here are some updated patches for Policy, incorporating this requirement.

I have not attempted to incorporate the corner case involving
build-profiles. I think if we were going to do that, it would require
documenting build-profiles first (#757760), and maybe even then it's too
much of a corner-case to be documenting unless/until it actually happens.

> I personally would prefer if we would avoid using this feature too much
> if possible.

I've added some wording to try to express that.

smcv
>From a332e4e787837cac0856c9c36d6e87e9f19197e2 Mon Sep 17 00:00:00 2001
From: Simon McVittie 
Date: Thu, 9 Sep 2021 15:43:20 +0100
Subject: [PATCH 1/2] archive: Point out that mixed main/contrib source
 packages can exist

Most source packages produce only binary packages in the same archive
area, but a few source packages in main (such as bumblebee) produce
a mixture of main and contrib binary packages.

If an upstream project is in this situation (for example a program with
optional plugins that have non-free dependencies) it isn't entirely
obvious how to package it; clarify that a single source package in main
is considered to be appropriate in this case, as long as no non-free
build-dependencies are required.

Signed-off-by: Simon McVittie 
Closes: #994008
---
 policy/ch-archive.rst | 21 +
 1 file changed, 21 insertions(+)

diff --git a/policy/ch-archive.rst b/policy/ch-archive.rst
index ab04261..3d40f55 100644
--- a/policy/ch-archive.rst
+++ b/policy/ch-archive.rst
@@ -130,6 +130,27 @@ In addition, the packages in *main*
 
 - must meet all policy requirements presented in this manual.
 
+If a source package is in the *main* archive area, then at least one of
+the binary packages that it produces must be in the *main* archive area,
+and each of the remaining packages must be in either the *main* or *contrib*
+archive area. Each binary package's archive area is indicated by its
+``Section`` field: see :ref:`s-subsections`.
+
+Source packages in *main* with a mixture of *main* and *contrib* binary
+packages should be limited to situations where it would be inconvenient
+to split the source package. If it is straightforward to split the source
+package into a *main* part and a *contrib* part that are compiled
+separately, then those parts should be represented as separate source
+packages.
+
+When a *main* source package has a mixture of *main* and *contrib*
+binary packages, the source package and the *main* binary packages must
+follow the requirements for *main* packages, but the *contrib* binary
+packages may follow the weaker requirements for *contrib* packages.
+In particular, build-dependencies outside *main* are not allowed in
+these source packages, but the *contrib* binary packages may have runtime
+dependencies outside *main*.
+
 .. _s-contrib:
 
 The contrib archive area
-- 
2.33.1

>From 14cd80454fc2ef8122315a1edcc05eed43106583 Mon Sep 17 00:00:00 2001
From: Simon McVittie 
Date: Thu, 9 Sep 2021 15:53:20 +0100
Subject: [PATCH 2/2] archive: Clarify binaries produced by contrib and
 non-free source

A source package outside main cannot produce main binary packages, because
we want main to be self-contained: if you download all main source
packages, that should give you the source code of all main binary
packages.

A source package in contrib cannot produce non-free binary packages,
because by definition contrib only contains free software (with non-free
dependencies, but those are not part of the source code).

A source package in non-free cannot produce contrib binary packages,
because we want main + contrib to be self-contained: if you download
all main or contrib source packages, that should give you the source
code of all main and contrib binary packages.

Signed-off-by: Simon McVittie 
---
 policy/ch-archive.rst | 8 
 1 file changed, 8 insertions(+)

diff --git a/policy/ch-archive.rst b/policy/ch-archive.rst
index 3d40f55..0979d87 100644
--- a/policy/ch-archive.rst
+++ b/policy/ch-archive.rst
@@ -177,6 +177,10 @@ Examples of packages which would be included in *contrib* are:
 -  wrapper packages or other sorts of free accessories for non-free
programs.
 
+If a source package is in the *contrib* archive area, then each of the

Bug#984511: debian-policy: please clarify how archive areas can be combined in source packages

[Simon McVittie]

Control: forcemerge 994008 984511

On Thu, 04 Mar 2021 at 13:09:04 +, Simon McVittie wrote:
> Package maintainers (including me, most of the time) tend to assume that
> each source package has to exist in exactly one archive area, and all
> of its binary packages have to go into that same archive area. However,
> Ansgar tells me this is not actually true.

Looks like I opened this bug twice, 6 months apart. #994008 has more
discussion so let's treat that one as the main bug.

smcv

Bug#994008: debian-policy: Clarify relationship between source and binary packages' archive areas

[Simon McVittie]

Package: debian-policy
Version: 4.6.0.1
Severity: wishlist
Tags: patch
X-Debbugs-Cc: ftpmas...@debian.org

If you have an upstream project consisting of Free source code with a
mixture of Free and non-Free dependencies, it isn't currently clear how
that should be packaged.

For example, if gnome-software added a steam plugin that was itself Free
Software, but had Depends: steam, then it would be in this situation:
the program and the flatpak and snap plugins would still be suitable
for main, but the steam plugin would need to go in contrib.

Based on the precedent seen in the bumblebee package, I believe the
ftp team's intention is that packages like this are allowed to be a
single source package in main, which builds binary packages for main
and contrib, even though this means the contrib binary packages don't
match their source package's archive area.

I think I remember talking to a ftp team member about this in person
and coming to the conclusion that there are good reasons for none of
the other possible source/binary archive area mismatches to be allowed.

In the form of a table, the allowed source/binary combinations are:

  |binary   |
  | main  contrib  non-free |
 -|-|
 main |  yesyes  -  |
 source  contrib  |   - yes  -  |
 non-free |   -  -  yes |

ftp team: is this correct?

If it is, I attach proposed patches for Policy. Their commit messages
attempt to capture the rationale for why the other situations are not
allowed; corrections/clarifications welcome.

Thanks,
smcv
>From eda0f325301bd514e5ac94328dd4b5a01960634a Mon Sep 17 00:00:00 2001
From: Simon McVittie 
Date: Thu, 9 Sep 2021 15:43:20 +0100
Subject: [PATCH 1/2] archive: Point out that mixed main/contrib source
 packages can exist

Most source packages produce only binary packages in the same archive
area, but a few source packages in main (such as bumblebee) produce
a mixture of main and contrib binary packages.

If an upstream project is in this situation (for example a program with
optional plugins that have non-free dependencies) it isn't entirely
obvious how to package it; clarify that a single source package in main
is considered to be appropriate in this case, as long as no non-free
build-dependencies are required.

Signed-off-by: Simon McVittie 
---
 policy/ch-archive.rst | 13 +
 1 file changed, 13 insertions(+)

diff --git a/policy/ch-archive.rst b/policy/ch-archive.rst
index ab04261..7829601 100644
--- a/policy/ch-archive.rst
+++ b/policy/ch-archive.rst
@@ -130,6 +130,19 @@ In addition, the packages in *main*
 
 - must meet all policy requirements presented in this manual.
 
+If a source package is in the *main* archive area, then each of the
+binary packages that it produces must be in either the *main* or *contrib*
+archive area. Each binary package's archive area is indicated by its
+``Section`` field: see :ref:`s-subsections`.
+
+When a *main* source package has a mixture of *main* and *contrib*
+binary packages, the source package and the *main* binary packages must
+follow the requirements for *main* packages, but the *contrib* binary
+packages may follow the weaker requirements for *contrib* packages.
+In particular, build-dependencies outside *main* are not allowed in
+these source packages, but the *contrib* binary packages may have runtime
+dependencies outside *main*.
+
 .. _s-contrib:
 
 The contrib archive area
-- 
2.33.0

>From 0b855a4a0dbb348269388985fc45c7887376a245 Mon Sep 17 00:00:00 2001
From: Simon McVittie 
Date: Thu, 9 Sep 2021 15:53:20 +0100
Subject: [PATCH 2/2] archive: Clarify binaries produced by contrib and
 non-free source

A source package outside main cannot produce main binary packages, because
we want main to be self-contained: if you download all main source
packages, that should give you the source code of all main binary
packages.

A source package in contrib cannot produce non-free binary packages,
because by definition contrib only contains free software (with non-free
dependencies, but those are not part of the source code).

A source package in non-free cannot produce contrib binary packages,
because we want main + contrib to be self-contained: if you download
all main or contrib source packages, that should give you the source
code of all main and contrib binary packages.

Signed-off-by: Simon McVittie 
---
 policy/ch-archive.rst | 8 
 1 file changed, 8 insertions(+)

diff --git a/policy/ch-archive.rst b/policy/ch-archive.rst
index 7829601..35b44bb 100644
--- a/policy/ch-archive.rst
+++ b/policy/ch-archive.rst
@@ -169,6 +169,10 @@ Examples of packages which would be included in *contrib* are:
 -  wrapper packages or other sorts of free accessories for non-free
programs.
 
+If a source package is in the *contrib* archive area, then each of the
+binary packages 

Bug#907051: Finding rough consensus on level of vendoring for large upstreams

[Simon McVittie]

On Fri, 03 Sep 2021 at 02:46:20 +0200, Jonas Smedegaard wrote:
> I suspect that it helps if separating reasons for _encouraging_ 
> embedding (tiny upstream projects and deeply integrated sets of 
> upstreams, I guess) from reasons for _discouraging_ embdding (all other 
> cases, I guess).

If the embedded project is specifically not maintained as API- or
ABI-stable, then I think that's also a common and valid reason to embed
it, perhaps even more so than the ones you mentioned here. This can mean
library APIs, but also command-line options, output behaviour or some
other machine interface that the embedding project might require.
(Or perhaps you intended "deeply integrated" to cover this, but I think
it's worth being explicit about the API stability factor.)

I'm mainly thinking here of "copylibs" like gnulib, libiberty, libglnx,
stb. If an upstream tells us their code will break API/ABI without
notice, then I think we should believe them - which means the only
long-term-maintainable way to depend on it is for the projects that need
it to vendor a known-compatible copy, and be responsible for updating
that copy and the code that calls it at the same time if a serious bug
is found. This is not new - gnulib and libiberty have been like this
for decades, and vendoring those is not really the same thing as vendoring
the likes of zlib or libjpeg.

I know we do have a libstb package that builds a shared library, but the
existence of a stb build system (let alone a shared library build system)
is a Debian-specific patch, which does not fill me with confidence
that it has the necessary API- and ABI-stability to be correct as a
shared library.

Outside the scope of libraries, a few packages in GNOME are currently
using gi-docgen (a gtk-doc-like API documentation generator) as a vendored
project, because it is intended to have a stable interface in future but
has not got there yet. There's a gi-docgen package in NEW, but it will
stay in experimental and will not be used in build-dependencies until
it has stabilized enough to be valid to use that way.

Of course, if an upstream decides that their project *is* ready to be a
sufficiently-stable separate project, then we should usually prefer to
package it separately (for example, bubblewrap and xdg-dbus-proxy are
vendored into flatpak, and the vendored copies were used in Debian early
in its lifetime, but both are now stable enough and independent enough
that we have moved to system copies).

smcv

Bug#989581: autopkgtest: ADTTMP is now obsolete

[Simon McVittie]

On Fri, 20 Aug 2021 at 14:32:27 -0700, Sean Whitton wrote:
> I believe that it's been AUTOPKGTEST_TMP for a very long time now

Just over 5 years, FYI (autopkgtest 4.0, 2016-06-08).

smcv

Bug#992601: Allow non-64-bit packages to install to /usr/lib64/ again

[Simon McVittie]

On Fri, 20 Aug 2021 at 14:45:01 -0700, Sean Whitton wrote:
> diff --git a/policy/ch-opersys.rst b/policy/ch-opersys.rst
> index 83b71b1..131b450 100644
> --- a/policy/ch-opersys.rst
> +++ b/policy/ch-opersys.rst
> @@ -48,8 +48,8 @@ Debian Policy. The following exceptions to the FHS apply:
>  libraries must not install these libraries to
>  ``/usr/lib/i386-linux-gnu``.  [#]_
> 
> -Packages must not install files in ``/usr/lib64`` or in a subdirectory
> -of it.
> +Packages for 64-bit architectures must not install files in 
> ``/usr/lib64``
> +or in a subdirectory of it.
> 
>  The requirement for C and C++ headers files to be accessible through
>  the search path ``/usr/include/`` is amended, permitting files to be

Seconded.

smcv

Bug#542288: debian-policy: Version numbering: native packages, NMU's, and binary only uploads

[Simon McVittie]

On Thu, 01 Apr 2021 at 18:17:59 -0700, Russ Allbery wrote:
> +- ``upstream_version`` components in native packages or
> +  ``debian_revision`` components in non-native packages ending in
> +  ``~debNuX`` also indicate a stable update, but of a different type.
> +  This version convention indicates that the stable version of the package
> +  was updated to a new upstream release, as distinct from the ``+debNuX``
> +  convention that indicates additional changes were applied to the
> +  existing stable release.  ``N`` and ``X`` mean the same as with
> +  ``+debNuX``.

I don't think this is quite it.

For me, ~debNu1 implies that a stable update was made by taking a newer
Debian package (typically from unstable) - not necessarily a new upstream
release! - and backporting it in its entirety, mechanically equivalent to
a ~bpo version but released to all users of stable rather than just those
who have opted-in to backports.

src:pango1.0 in Debian 10 is an example of this: we fixed CVE-2019-1010238
in testing/unstable by applying a patch (1.42.4-6 was vulnerable,
1.42.4-7 was fixed). We could have continued by fixing CVE-2019-1010238
in stable as 1.42.4-6+deb10u1, but the result would have been functionally
equivalent to 1.42.4-7, so it seemed clearer to just take 1.42.4-7 and
rebuild it for Debian 10 as 1.42.4-7~deb10u1. We later repeated the
process for 1.42.4-8 with some non-security fixes.

If the stable version of the package was updated to a new upstream release
instead, there are two ways that can happen. One way is to take the version
from testing/unstable and rebuild it in stable, like src:openjdk-11 in
Debian 10: 11.0.9.1+1-1 was uploaded to unstable first, then rebuilt
in Debian 10 as 11.0.9.1+1-1~deb10u1.

The other way is to repackage the new upstream release from first
principles, either because it's a new release from an upstream branch
that's older than the one in testing/unstable (like src:flatpak in Debian
10), or because testing/unstable already has packaging changes that aren't
suitable for stable (like src:dbus in Debian 10). In this case it would be
misleading to use a version number like flatpak_1.2.5-1~deb10u1 (because
1.2.5-1 never existed) or dbus_1.12.20-1~deb10u1 (because 1.12.20-1 did
exist, but the version of dbus in stable was not based on it, and does
not have the downstream changes from that version). Instead, the convention
that has emerged is to use 0+deb10u1, by analogy with the convention that
an NMU introducing a new upstream release has revision number 0.1.

~debNuX for X > 1 indicates subsequent stable-specific changes applied
on top of to a ~debNu1 version.

(Sorry, I'm not sure how to express all this in a paragraph or two...)

smcv

Bug#984511: debian-policy: please clarify how archive areas can be combined in source packages

[Simon McVittie]

Package: debian-policy
Version: 4.5.1.0
Severity: normal
X-Debbugs-Cc: ftpmas...@debian.org

Package maintainers (including me, most of the time) tend to assume that
each source package has to exist in exactly one archive area, and all
of its binary packages have to go into that same archive area. However,
Ansgar tells me this is not actually true.

Some of the terms in use here are overloaded, so please be careful. At the
dpkg level, there is a Section field in d/control and in binary package
metadata. Despite its name, this field actually combines the user-facing
section (admin, games, libdevel and so on) and the archive area (main,
contrib or non-free). In this mail I will use Section (titlecase) for the
field, and section (lower-case) for the user-facing section.

.dsc files do not include a Section field, but both source and binary
packages have to exist in an archive area, so when accepting uploaded
packages the archive software has to guess which archive area the
source package ought to be in, based on the Section of the various
binary packages in its Package-List field.

The Section in the first stanza of d/control is a default for all binary
packages that do not explicitly specify their Section, but is not copied
into the .dsc file - although perhaps it should be?

My understanding is that the ftp team allows the following situations:

* A source package builds binary packages that are all in main
  (e.g. dpkg). The source package is Free and gets put in main.

  Example d/control (simplified down to just the relevant parts):

  Source: dpkg
  # this implicitly means archive area: main, section: admin
  Section: admin

  Package: dpkg

  Package: dpkg-dev
  # this implicitly means archive area: main, section: utils
  Section: utils

* A source package builds binary packages that are all in contrib
  (e.g. src:game-data-packager). The source package is Free, and gets put
  in contrib too.

  Example d/control (simplified down to just the relevant parts):

  Source: game-data-packager
  # this means archive area: contrib, section: games
  Section: contrib/games

  Package: game-data-packager

  Package: quake

* A source package builds binary packages that are all in non-free
  (e.g. src:steam). The source package is at least partially non-Free
  and gets put in non-free.

  Example d/control (simplified down to just the relevant parts):

  Source: steam
  # this means archive area: non-free, section: games
  Section: non-free/games

  Package: steam

  Package: steam-devices

* A source package builds a mixture of binary packages that are
  suitable for main and binary packages suitable for contrib
  (e.g. src:cpl-plugin-amber, where cpl-plugin-amber-calib is contrib
  and the rest is in main). The source package is Free, even though
  some of its binary packages have external and/or non-free dependencies,
  so it can be put in main.

  Example d/control (simplified down to just the relevant parts):

  Source: cpl-plugin-amber
  # this implicitly means archive area: main, section: science
  Section: science

  Package: cpl-plugin-amber

  Package: cpl-plugin-amber-calib
  # this means archive area: contrib, section: science
  Section: contrib/science

In particular, a source package is not allowed to combine non-free binary
packages with main or contrib binary packages: the important parts of
steam-devices.deb are actually Free (MIT-licensed), but if I wanted to put
those in main, I would have to split the source package into an entirely
Free part to build steam-devices.deb, and a non-Free part to build steam.deb.

>From the Policy text, it is not clear that the last one of those four
situations (as used in cpl-plugin-amber) is allowed. It would be useful for
Policy to explicitly say that it is, so that it's available as a tool for
maintainers to use when appropriate.

Questions for the ftp team:

* Is my understanding of this correct?

* Are source packages like cpl-plugin-amber, that mix main and contrib
  binary packages, considered to be something that is entirely valid and
  should be used whenever it is the closest representation of reality;
  or is this considered to be deprecated feature that should be avoided,
  with maintainers preferring to split the source package into a part
  that builds 100% main binary packages and a part that builds 100%
  contrib binary packages?

One concrete place where this could be useful is that src:pipewire
optionally builds a plugin that uses fdk-aac (in non-free) for better
Bluetooth support. This is currently disabled via a Build-Conflicts in
Debian, but users of some Bluetooth headsets would prefer to be able to
use it. If pipewire could build everything from a single source package in
main, and produce binary packages that are mostly main but also include
a contrib plugin for fdk-aac, then that would be a lot easier than a
separate pipewire-contrib source package.

smcv

Bug#980069: Better documentation of x-terminal-emulator

[Simon McVittie]

On Wed, 13 Jan 2021 at 22:49:27 +, Bastien Roucariès wrote:
> gnome-terminal -e sleep 60

The upstream-provided gnome-terminal executable does not claim to be an
implementation of the x-terminal-emulator alternative. It has its own
command-line interface, which uses GNU-style long-option parsing (unlike
xterm). If we patched it to provide the x-terminal-emulator interface,
that modification would make it incompatible with upstream gnome-terminal,
and with other packages that know how the gnome-terminal interface works.

The Debian-specific script /usr/bin/gnome-terminal.wrapper is what gets
registered as an implementation of x-terminal-emulator if you have the
gnome-terminal package installed. Its command-line interface is almost
a subset of xterm's (it does additionally support --help, which xterm
does not).

If you want gnome-terminal behaviour, run gnome-terminal and pass
gnome-terminal arguments to it; if you want generic x-terminal-emulator
behaviour, run x-terminal-emulator and pass x-terminal-emulator
arguments to it. They are not the same, and programs can only rely on
x-terminal-emulator behaviour if what they run is the x-terminal-emulator
alternative. If they run a specific terminal directly, then they'll get
that specific terminal's behaviour, whatever that might be.

> Moreover some packages noticed that x-terminal-emulator does not wait by 
> default

I'm not sure whether waiting for the executed command to exit is
actually part of the specification for x-terminal-emulator, but
/usr/bin/gnome-terminal.wrapper passes --wait to gnome-terminal, so that
it will behave like xterm in this respect.

Other implementations might not.

> I plan to create a sensible-x-terminal-emulator and thus I need to document;
> * the -e behavior, that  should behave like xterm and do not need to use 
> quotes

https://www.debian.org/doc/debian-policy/ch-customized-programs.html#packages-providing-a-terminal-emulator
describes what you can rely on from an implementation of the
x-terminal-emulator alternative. Anything not written there cannot be
relied on.

Policy promises that "x-terminal-emulator -e sleep 60" does what you think
it does.

Policy does not say anything either way on what
"x-terminal-emulator -e 'sleep 60'" will do. It happens to work with xterm
and with /usr/bin/gnome-terminal.wrapper, but that is not guaranteed.

smcv

Bug#940144: developers-reference: document self-service givebacks in wanna-build section

[Simon McVittie]

On Fri, 13 Sep 2019 at 10:16:25 +0800, Paul Wise wrote:
> https://debblog.philkern.de/2019/08/alpha-self-service-buildd-givebacks.html
...
>As an alpha trial I implemented self-service givebacks as a web
>script. As SSO for Debian developers is now a thing, it is trivial
>to add authentication in a way that a role account can use to act on
>your behalf. While at work this would all be an RPC service, I
>figured that a little CGI script would do the job just as well. So
>lo and behold, accessing
>
> https://buildd.debian.org/auth/giveback.cgi?pkg=&suite=&arch=
>with the right parameters set:
> 
>   You are authenticated as pkern. ✓
>   Working on package fife, suite sid and architecture mipsel. ✓
>   Package version 0.4.2-1 in state Build-Attempted, can be given back. ✓
>   Successfully given back the package. ✓
> 
>Note that you need to be a Debian developer with a valid SSO client
>certificate to access this service.

On Tue, 21 Jan 2020 at 21:20:54 +0100, Philipp Kern wrote:
> Yeah, so Julien helpfully just created auth.buildd.debian.org (thanks
> for that!).

To be completely clear about this for those using this bug report as a
stand-in for the requested documentation in devref (like me), it's now at:
https://auth.buildd.debian.org/auth/giveback.cgi?pkg=&suite=&arch=

smcv

Bug#949690: debian-policy: "service unit should have the same name as the package" seems too strong

[Simon McVittie]

Package: debian-policy
Version: 4.5.0.0
Severity: minor
X-Debbugs-Cc: syst...@packages.debian.org

If a package has a single system service with a systemd service unit,
and the system service's name does not match the package's name, current
Policy implies that this is probably a (non-RC) bug.

I think that's too strong. In particular, because the name of the service
unit is part of the "API surface" of the system service, aligning the
name of the service unit with the name used by upstream is usually
more important than aligning it with the name of the Debian package,
unless the name used by upstream results in conflicts or is otherwise
poorly-chosen. This is analogous to the names of executables in the PATH
and the SONAMEs of libraries, both of which we try not to change.

For example, flatpak contains the flatpak-system-helper.service unit
(not flatpak.service). I don't think that should be considered to be a
bug in flatpak: there is no system service that can/should be called
"the Flatpak service", only a system service that implements a small
non-essential part of flatpak (the ability for a suitably privileged
user to install Flatpak apps and runtimes system-wide), so it would be
misleading to have a flatpak.service unit.

The one time it would be a bug for the systemd service unit not to match
the package's name is when the systemd service unit needs to shadow a
pre-existing init script, and the init script was named according to
the Policy suggestion that its name should match that of the package.
In this case, I think the right thing to do is to add an alias (symlink),
similar to what was done in gdm3:

- /etc/init.d/gdm3 is a regular file
- /lib/systemd/system/gdm.service is a regular file
- /lib/systemd/system/gdm3.service is a symlink to gdm.service

However, I don't think this should apply to all single-system-service
packages, only those with pre-existing init scripts: it would be silly to
require flatpak to contain a symlink
/lib/systemd/system/flatpak.service -> flatpak-system-helper.service.

Flatpak's system service is started on-demand via D-Bus activation,
not on boot, so it never had a corresponding LSB init script. Strictly
speaking, if it had been started on boot and hence needed a LSB init
script, Policy version 4.4 and older would have implied that it was a
bug for the init script not to be named /etc/init.d/flatpak, which I
think would also have been too strong. However, LSB init scripts don't
have quite the same "it's API" argument as systemd services, because
they have historically been extremely distro-specific - indeed, one of
systemd's goals was to encourage service units to be provided by upstream
developers and shared by all distros, unlike LSB init scripts.

smcv

Bug#948275: is Debian POSIX compliant?

[Simon McVittie]

On Mon, 06 Jan 2020 at 12:12:23 +0100, Harald Dunkel wrote:
> I haven't found it mentioned in the policy manual, so I wonder if
> Debian is supposed to be POSIX compliant (unless noted otherwise)?

I suspect what we aim for is "approximately POSIX". Neither the
Linux kernel nor the various user-space components within the scope
of POSIX (mostly GNU, but probably also some non-GNU packages like
util-linux) aim to be pedantically POSIX-compliant: they will implement
non-POSIX-compliant behaviours if their maintainers think there is a
good reason to do so.

For example, GNU grep has command-line parsing behaviour that is
technically not POSIX-compliant (option arguments given after filenames
are interpreted as options, but POSIX says they are filenames), unless
the environment variable POSIXLY_CORRECT is set:

$ grep nobody /etc/passwd -v
... all of /etc/passwd, except nobody ...
$ POSIXLY_CORRECT=1 grep nobody /etc/passwd -v
/etc/passwd:nobody:x:65534:65534:nobody:/nonexistent:/usr/sbin/nologin
grep: -v: No such file or directory

Another classic example is that POSIX requires df and du to count in
512-byte blocks, but the GNU tools default to counting 1 KiB blocks.

smcv

Re: Guidance on solving the username namespacing problem

[Simon McVittie]

On Sun, 05 Jan 2020 at 17:16:58 +0100, Philipp Kern wrote:
> On 1/4/2020 5:08 PM, Simon McVittie wrote:
> > It's also worth noting that the 61184-65519 uid range used for DynamicUser
> > by default collides with the rarely-used 6-64999 uid range for system
> > users that are "globally allocated by the Debian project, but only created
> > on demand".
> > 
> > The uids in range 6-64999 that are really allocated in practice are
> > in the sub-range 64000-64062, except for 63434 'netplan' which appears
> > to be for an orphaned package that isn't in testing; so it might be
> > feasible to reconfigure the DynamicUser range to 6-63999 or similar
> > in Debian's systemd?
> 
> It looks like the range must be contiguous, as it is compiled in[1].

Yes.

> What are the preexisting ones apart from netplan that you have in mind?

https://sources.debian.org/src/base-passwd/3.5.47/README/#L28

As you can see, apart from netplan (63434, which I suspect may have been
imported from a previous ad-hoc allocation) in practice they are all in
the 640xx range. Most of the newer allocations seem to be cluster/cloud
services where having the same uid across a whole cluster is
desirable: the vast majority of system services are fine with the
dynamically-allocated 100-999 range.

> It feels like systemd's boundaries are pretty arbitrary (0xEF00 to
> 0xFFEF) but at the same time we might want to reserve a range for it in
> policy - given that right now it is effectively squatting in that range?

Yes, I think so.

The exact range used seems to be arbitrary, and having a slightly smaller
range in Debian would probably be OK, as long as it isn't orders of
magnitude smaller?

> The main argument presented by upstream is that it needs to be within
> 64bit for user namespacing to work properly.

I assume you meant within the 16-bit range (first 64k uids), but yes. The
big idea is that container number 0x1234 can have uids ranging from
0x1234 (root in the container) to 0x1234fffe ('nobody' in the
container, where 'nobody' is 65534 = 0xfffe), which avoids the problem
seen in e.g. Docker where uid 1000 in container A can sometimes
unintentionally get privileges over processes owned by uid 1000 in
container B, because they are both uid 1000 on the host system.

Managing uid and gid allocations for containers seems likely to be
much, much easier if each container gets a linear block of 65535 uids
within which it can use the entire legacy 16-bit uid space, rather than
having a discontinuous uid range so that some uids are unavailable to
the container.

smcv

Re: Guidance on solving the username namespacing problem

[Simon McVittie]

On Sat, 04 Jan 2020 at 13:52:51 +0100, Philipp Kern wrote:
> now that we are talking again about standardizing user creation using
> sysusers, I wonder if you could give me any guidance on how to attack
> the Debian system user namespacing problem.

It's a good reminder, but I think the naming convention for system users
is mostly orthogonal to whether they're created by adduser, useradd,
systemd-sysusers or something else. (But see below for DynamicUser.)

> OpenBSD rather successfully standardized on the underscore prefix to
> eliminate this conflict altogether. I would like that we recommend the
> same thing.

I agree that's a good convention for new system users (better than
debian- or Debian-, and a lot better than having no particular prefix),
particularly now that apt creates _apt on basically every Debian-derived
system.

> The main question that has been raised was how to manage the migration.
> I think the priority should be on stopping the bleeding and new users
> should follow a consistent scheme

I agree, and I don't think the absence of a migration path for existing
system users like messagebus and systemd-coredump should prevent us
from encouraging the OpenBSD underscore thing for new system users
like _apt and _flatpak.

I maintain several game servers that use undesirable usernames like
Debian-openarena, which might make good test subjects for automatic
migration to names like _openarena in relatively unimportant packages
(we shouldn't be using more important packages like dbus as our test
subjects). After successfully prototyping it in a couple of packages,
the right place for it would probably be debhelper or init-system-helpers.

The thing to do during migration would probably be to add a second user
with the same uid and other details, but a different username (like
useradd --non-unique); then optionally remove the original user record,
leaving only the new name.

On Sat, 04 Jan 2020 at 13:55:40 +0100, Philipp Kern wrote:
> A more bold move would be to tell daemon packagers to use DynamicUser
> where feasible and only allocate more permanent users if there's a need
> for it.

I think this is *almost* orthogonal to how those users are named.

The only way in which DynamicUser affects the naming policy is that if
foo-bar.service uses DynamicUser=yes and does not specify a User, the
default is foo-bar. /lib/systemd/system/fwupd-refresh.service is currently
the only example on my laptop.

For units with a long, namespaced name like fwupd-refresh.service,
flatpak-system-helper.service and quake2-server.serice, this is probably
good enough in practice, even though in principle these names might collide.

Possible routes:

- leave these units with behaviour that does not follow the recommendation,
  reasoning that if they have fairly long names it won't actually be a
  practical problem

- recommend that units with DynamicUser=yes should specify a User and Group
  that fit the naming convention, for example DynamicUser=yes,
  User=_fwupd-refresh

- ask the systemd Debian maintainers to patch systemd so that it defaults
  to the equivalent of User=_fwupd-refresh instead (note that this
  behaviour change is in principle an API break which could conceivably
  make services that work in other distributions fail to work in Debian,
  although probably it would work fine in practice)

- ask systemd upstream to make that change (they could probably be persuaded
  that the OpenBSD underscore thing is a good convention; same notes about
  this being a behaviour change)

- avoid DynamicUser=yes

It's also worth noting that the 61184-65519 uid range used for DynamicUser
by default collides with the rarely-used 6-64999 uid range for system
users that are "globally allocated by the Debian project, but only created
on demand". systemd avoids uids that are already in use when allocating
a DynamicUser, so the only time this is a practical problem is if systemd
has already used a uid for DynamicUser, and then the package to which that
uid is allocated in base-passwd gets installed before the next reboot.

The uids in range 6-64999 that are really allocated in practice are
in the sub-range 64000-64062, except for 63434 'netplan' which appears
to be for an orphaned package that isn't in testing; so it might be
feasible to reconfigure the DynamicUser range to 6-63999 or similar
in Debian's systemd?

On Sat, 04 Jan 2020 at 09:00:37 -0500, Sam Hartman wrote:
> I think you could certainly do  usernames created by packages are
> encouraged to start with an underscore.
...
> You could also do something more complex like
>¯
> When maintainers choose a new hard-coded or dynamically generated username
> for packages to use, they should start this username with an underscore.

The latter sounds better to me, at least for now.

This should probably go somewhere near §9.2.2 "UID and GID classes":
it applies to future allocations in the 100-999 and 6-64999 ranges,
and maybe to future allocations in the 0-99 

Re: Proposal for next steps for systemd-related policy

[Simon McVittie]

On Sun, 29 Dec 2019 at 10:47:44 -0800, Russ Allbery wrote:
> 3. Start a discussion on debian-devel to see if we can come up with some
>idea for how to declare dependencies on availability of system
>services.

Do you mean "systemd features", or do you mean system services more
generally?

If the latter, we've had this class of bugs for years, particularly in
chroots, even before systemd existed: if a package relies on a system
service (perhaps most frequently the D-Bus system bus, but also things
like avahi-daemon, BlueZ, firewalld and nslcd), package dependencies
can ensure that the system service is *installed*, but cannot ensure
that it is *running*.

For example, I've seen this recently with packages adding build-time tests
that fail when run in a chroot (or a Docker container) that does not have
a D-Bus system bus running, because "Depends: dbus" isn't sufficient to
make that facility available in a chroot. Such tests need to either run
a mock system bus or be skipped gracefully when run by Debian buildds,
and if skipped at build-time they should preferably run as autopkgtests
instead. This is not unique to systemd: the same situation would exist
on pre-systemd Debian versions, or in Devuan (assuming Devuan builds in
a minimal chroot like Debian does).

I agree that having both systemd and sysvinit does make this situation
more problematic, because expectations for a bootable system with a
non-default init system like sysvinit are higher than expectations for
a chroot or Docker-style container with no init system at all.

autopkgtest documents the isolation-container restriction as implying
that services *can* be started, although it doesn't actually document
that restriction as implying that installed services *will* be
started. Amusingly, this means a Docker virtualization backend for
autopkgtest will probably be unable to advertise isolation-container,
because Docker containers usually don't run an init system like sysvinit
or systemd.

smcv

Re: Proposal for next steps for systemd-related policy

[Simon McVittie]

On Sun, 29 Dec 2019 at 10:47:44 -0800, Russ Allbery wrote:
>We probably also want to recommend Type=notify where possible and
>Type=exec where not, over Type=forking, when the daemon supports that.

I'm not sure what you mean by "where possible" - it'll usually be
possible to implement Type=notify (even if a libsystemd dependency is
unacceptable or not feasible in the implementation language, the protocol
is designed to be reimplementable) but it won't always be useful.

Type=exec doesn't provide a way for the daemon to signal that it has
opened listening sockets or done whatever else is necessary for units that
depend on it to be able to start without error, so daemons that might be
depended on by other units will normally need to use either Type=notify,
Type=dbus or Type=forking, then make sure they do basic setup (listen on
sockets, etc.) before they notify, request their D-Bus names or do the
traditional double-fork daemonization trick (whichever is appropriate
for their Type). However, I don't think this is necessarily in-scope
for Policy? Implementing readiness-signalling if required seems like
it might be the sort of "don't be buggy" requirement that shouldn't
necessarily need to be said explicitly.

I think perhaps a better recommendation might be that it's preferred for
systemd services that run daemons to run them as a foreground process
(without forking or "daemonizing" during startup), with a footnote or
parenthetical note saying that this recommendation is incompatible with
Type=forking, which should therefore only be used if readiness-signalling
is required and alternatives like the preferred Type=notify or Type=dbus
are unavailable?

I say "when run from a systemd service" so that services like dbus-daemon
meet this recommendation - by default it double-forks for compatibility
with LSB-style init scripts and its own historical behaviour, but the
systemd unit dbus.service runs "dbus-daemon --nofork" which disables the
double-fork code path, and I think that's a perfectly good implementation
of what we want.

>- DynamicUser without useradd of a system user

One warning for this one is that users referred to in dbus-daemon
system.d/*.conf snippets, typically ,
currently need to be pre-created by sysusers.d, useradd or equivalent
(because dbus-daemon resolves usernames to uids during its own
startup/reload, to populate internal data structures that are all designed
in terms of uids). Other references to users in non-D-Bus configuration
might well behave similarly.

smcv

Bug#945269: debian-policy: packages should use tmpfiles.d(5) to create directories below /var

[Simon McVittie]

On Sat, 30 Nov 2019 at 10:58:23 -0800, Russ Allbery wrote:
> Guillem Jover  writes:
> > I don't mind much how this could end up being hooked into various init
> > systems and chroot/container managers. I can see how it could be done by
> > the respective imeplementations themselves or provided by dpkg itself,
> > I'm open to any of these TBH.
> 
> One possibly interesting option would be for dpkg to take responsibility
> for writing the tmpfiles.d configuration file for a package based on its
> own metadata

I think the other way round could also make sense: having dpkg create
variable/transient directories, and track them as owned by the package,
using its tmpfiles.d fragment(s) as input (either directly, or via a
debhelper step that takes tmpfiles.d as input and registers it with dpkg
as output).

That way, in the hopefully-common case where the upstream developer
of a system service installs a tmpfiles.d fragment for the files that
their package owns (most likely generated at build-time with appropriate
Autoconf/etc. @substitutions@), the packaged version will do the right
thing, without the packager having to take action specifically.

Prior art/naming: I think RPM "ghost files" are files tracked as owned
but not actually shipped in the RPM?

smcv

Bug#945269: debian-policy: packages should use tmpfiles.d(5) to create directories below /var

[Simon McVittie]

On Fri, 22 Nov 2019 at 14:05:50 +0100, Ansgar wrote:
> I'm fairly sure that systemd-tmpfiles doesn't require systemd as pid-1

It doesn't, but its dh_installsystemd integration currently does, so
maintainer scripts relying on it would currently be buggy. I think it
would be premature to recommend systemd-tmpfiles in Policy before it can
be used in a non-buggy way.

I had also thought that, in the current state of non-systemd-init-world,
having a dependency on a package containing systemd-tmpfiles was likely
to be practically problematic because it would indirectly conflict
with elogind, via libsystemd0 and its elogind reimplementation - but
looking more closely, systemd-tmpfiles and systemd-sysusers depend on
/lib/systemd/libsystemd-shared-243.so (and not on libsystemd0) so that
shouldn't actually be a problem.

smcv

Bug#945269: debian-policy: packages should use tmpfiles.d(5) to create directories below /var

[Simon McVittie]

On Fri, 22 Nov 2019 at 09:03:29 +0100, Ansgar wrote:
> I would like to recommend packages to use tmpfiles.d(5) to manage
> creating directories in locations such as /var or /etc instead of
> maintainer scripts.

Using tmpfiles.d(5) seems like a good thing to encourage, but using
them *instead of* maintainer scripts is less simple. My understanding
is that in current Debian policies and status, if a package relies
on tmpfiles.d(5) for important functionality and does not depend on a
tmpfiles implementation, that would usually be a grave bug.

The only tmpfiles implementation in Debian right now (that I know of) is
the combination of systemd-tmpfiles(8), systemd-tmpfiles-setup.service
(for creation of tmpfiles during boot), the maintainer script
fragments generated by dh_installsystemd (for creation of tmpfiles
during installation), and systemd as pid 1. I'm not entirely sure why
systemd doesn't have a trigger on tmpfiles.d, but perhaps it's because
the sequencing would be wrong: a package's tmpfiles generally need to be
created before its postinst reaches the point where services are started,
and I don't think triggers would guarantee that.

( is an alternative implementation,
but is not in Debian and would require adjustments to dh_installsystemd.)

It is of course fine to ship tmpfiles.d(5) fragments (like dbus does)
as long as the package either depends on systemd as pid 1, or has some
redundant way to create enough tmpfiles that it can work without systemd
as pid 1; and I think it might be good for Policy to recommend that
required directories in /run, /var and /etc are listed in tmpfiles.d(5)
fragments, particularly /run and /var/cache.

Ways to avoid solely relying on tmpfiles.d(5) include creating
directories from maintainer scripts (possibly redundantly), shipping
directories (other than in /run!) in the .deb (possibly redundantly),
and creating directories on a just-in-time basis in init-scripts or
init-script-equivalent constructs like systemd services (possibly
redundantly).

Note that I am deliberately saying "without systemd as pid 1" rather than
"with a non-systemd init", because there is a subtle difference. If you
install (for example) dbus into a chroot or container that does not run an
init system at all, then the current dh_installsystemd fragment will see
that /run/systemd/system doesn't exist, so /usr/lib/tmpfiles.d/dbus.conf
will not be processed and /var/lib/dbus will not be created. (Reference:
look in /var/lib/dpkg/info/dbus.postinst for "Automatically added
by dh_installsystemd".) This doesn't matter for the D-Bus system
service, because the D-Bus system service will not be started in those
chroots/containers (there is no init to start it), but it does matter
for dbus-x11 (which also wants to see /var/lib/dbus/machine-id for
the autolaunching protocol) and anything else that is using the D-Bus
machine ID.

This means that even if GR 2019-002 resolves that we will stop supporting
non-systemd init systems, there will still be two cases to consider:
systemd vs. no init at all.

Another potential difficulty is that some tmpfiles.d(5) fragments might
assume that systemd is installed: certain guarantees/constraints,
such as /etc/machine-id behaving as documented in machine-id(5),
are known to be true on systems with systemd as pid 1, but not on
Debian systems in general. dbus' tmpfiles fragment creates a symlink
/var/lib/dbus/machine-id -> /etc/machine-id based on the assumption that
machine-id(5) is as documented, which is not necessarily true without
systemd (again, this can mean either sysvinit as pid 1, or no init
at all).

For some packages these issues could be solved by a dependency on
systemd (in this case not systemd-sysv the init system, just systemd
the collection of executables) and suitable adjustments to the
dh_installsystemd maintainer script fragment; or by a dependency on
systemd | opentmpfiles and suitable adjustments to dh_installsystemd,
if someone packages .

> This has advantages over maintainer scripts

I agree, and I would like to seem more use of tmpfiles.d(5), but I think
it needs to be thought through some more.

> As far as I understand the current GR is unrelated to adopting things
> like systemd-tmpfiles.

I don't think that's true: as long as the only working implementation
of tmpfiles.d(5) in Debian requires systemd as pid 1, and Debian aims to
support pid 1 other than systemd, we can't rely on tmpfiles.d(5). Some of
the possible results of the current GR would change those constraints.
Someone packaging opentmpfiles could also change those constraints.

smcv

Bug#941198: initscripts: packages should ship systemd units

[Simon McVittie]

On Fri, 01 Nov 2019 at 12:21:43 +0100, Ansgar wrote:
> Dmitry Bogatov writes:
> > Does it mean that lack of systemd unit file is RC-critical bug? Or I
> > will be able to waive it with "you are welcome to contribute a patch"
> > response?

I think in general the answer is that it should be a non-RC bug to have
an LSB ("sysvinit") init script /etc/init.d/foo without a corresponding
systemd service /lib/systemd/system/foo.service (but maintainers should
accept good patches).

Notable exceptions that shouldn't be considered to be bugs:

- if foo is provided in some other way on systemd systems, then the init
  script should be masked (symlink
  /lib/systemd/system/foo.service -> /dev/null), either by the package that
  provides it (e.g. screen and sudo both do this), or by systemd itself
  (as is currenly done for "core" init scripts in packages like sysv-rc
  and x11-common; but perhaps responsibility for doing this should move
  to the non-systemd package, coordinating that transition with the
  systemd maintainers, now that systemd is the default and has been for
  a while?)

- if foo is provided under a different name on systemd systems, then
  /lib/systemd/system/foo.service should be a symlink to the
  systemd name for it (udev.service -> systemd-udevd.service and
  network-manager.service -> NetworkManager.service are good examples)

- packages that cannot be co-installed with systemd-sysv should get an
  exemption from this, as the opposite of the rule that says it's OK that
  systemd-cron doesn't have LSB init scripts :-)

> Or should we consider making shipping a sysvinit script, but no systemd
> unit a RC bug?

Not in general, I don't think, but there are certainly situations where
it should be made RC on a case-by-case basis:

- rcS scripts nearly always need a native systemd service, because the units
  that systemd generates for LSB init scripts always end up in the
  equivalent of rc2

- more generally, if the conservative assumptions that systemd has to make
  about LSB init scripts result in broken behaviour in practice (notably,
  apache2 in the initial jessie release had this problem) then the broken
  behaviour might be a RC bug, to be fixed by either adding a native
  systemd service (like apache2 in stretch) or carefully overriding some
  aspects of the generated systemd service (like apache2 in jessie updates).
  In this case I'd say the absence of a systemd service is not RC, but the
  *practical effect of* the absence of a systemd service can be RC.

smcv

Bug#942051: debian-policy: [4.9] requirement to write only to /tmp, /var/tmp, ${TMPDIR} is too strict

[Simon McVittie]

On Wed, 09 Oct 2019 at 17:51:53 +0200, Ansgar Burchardt wrote:
> +---
> | 4.9
> |Required targets must not write outside of the unpacked source
> |package tree, except for TMPDIR, /tmp and /var/tmp.
> +---

If you are interested in this sort of thing, 
is perhaps also relevant.

smcv

Bug#942051: debian-policy: [4.9] requirement to write only to /tmp, /var/tmp, ${TMPDIR} is too strict

[Simon McVittie]

On Wed, 09 Oct 2019 at 18:08:09 +0200, Bill Allombert wrote:
> On Wed, Oct 09, 2019 at 05:51:53PM +0200, Ansgar Burchardt wrote:
> > There are
> > other paths that should be fine to be written to during the build
> > process, for example /dev/shm
> 
> Do you have example of packages that does that ?

Anything that uses the POSIX shared memory API (shm_open()), and notably
anything that is built using fakeroot with its default fakeroot-sysv
backend.

smcv

Bug#932704: debian-policy: Don't force sysvinit compatibility if e.g. alternate init required

[Simon McVittie]

On Mon, 22 Jul 2019 at 13:39:31 +0200, Ansgar wrote:
> What sort of dependencies are we talking about? Package-level
> dependencies (e.g. Depends: systemd-sysv directly or indirectly)?

Probably yes. systemd-cron and dbus-user-session are examples of packages
that rely on the systemd service manager being pid 1.

Strictly speaking Depends: systemd-sysv is neither necessary nor
sufficient to make the systemd manager pid 1 (a system without
systemd-sysv could boot with init=/lib/systemd/systemd, or a system
with systemd-sysv could boot with init=/some-other-init), so I used
Depends: systemd, Recommends: systemd-sysv in dbus-user-session; but
perhaps I should switch that to Depends: systemd-sysv, because it would
make the intention clearer.

smcv

Bug#172436: Updated BROWSER proposal

[Simon McVittie]

On Sun, 21 Jul 2019 at 21:08:15 -0700, Russ Allbery wrote:
> I think it's clear that we're not making forward progress here, and the
> lack of a clear specification for the BROWSER environment variable doesn't
> seem to be causing a lot of noticable ongoing pain.  I'm therefore going
> to close this bug as something on which we've just not reached consensus.

As a note for anyone who comes back to this, before the bug gets archived:

freedesktop.org has developed a system of URI-scheme handlers since this
bug was opened, which as far as I know is used in all the major desktop
environments (this is definitely the case for GNOME and the various forks
based on older GNOME releases, and I'm fairly sure it's also true for KDE,
XFCE, LXDE). In that system, the equivalent of Debian's x-www-browser
alternative is to register a MIME type handler for the pseudo-MIME-types
"x-scheme-handler/http" and "x-scheme-handler/https". A similar approach
works for other URI schemes, for example "x-scheme-handler/mailto".

Desktop-oriented libraries should already have a way to launch the default
handler for a URI, for example g_app_info_launch_default_for_uri_async()
in GLib or bool QDesktopServices::openUrl(const QUrl&) in Qt. This is
often a better choice than running a subprocess.

Applications that specifically want a command-line/subprocess-based
interface can launch a URI in a browser with something like:

- xdg-open http://example.com (xdg-utils)
  - a 1000+-line shell script with many special cases; uses $BROWSER when
not in an X11 environment

- gio open http://example.com (libglib2.0-bin)
  - CLI wrapper for g_app_info_launch_default_for_uri_async()
  - requires GLib, but in practice all mainstream desktop environments
are going to have that anyway, e.g. via Firefox or Qt; does not attempt
to handle the non-graphical case

Regards,
smcv

Re: Bug#919507: Reboot required patch for Debian policy

[Simon McVittie]

On Sat, 20 Jul 2019 at 20:27:04 -0700, Russ Allbery wrote:
> "Karl O. Pinc"  writes:
> > diff --git a/policy/ch-opersys.rst b/policy/ch-opersys.rst
> > index 59c92ec..8276bfe 100644
> > --- a/policy/ch-opersys.rst
> > +++ b/policy/ch-opersys.rst
> > @@ -1040,3 +1040,33 @@ Debian, so this section has been removed.
> > activate the trigger. In that case, it can be done by calling
> > ``dpkg-trigger --no-await /usr/lib/mime/packages`` from the
> > maintainer script after creating, modifying, or removing the file.
> > +
> > +.. index::
> > +   pair: signaling; reboot
> > +
> > +.. _s-signalingreboot
> > +
> > +Signaling that a reboot is required
> > +---
> > +
> > +.. index::
> > +   single: reboot-required
> > +   single: reboot-required.pkgs
> > +
> > +Programs can signal that a reboot is required by ``touch``\ing
> > +``/run/reboot-required``.  It is conventional to add the name of the
> > +package(s) requiring the reboot to
> > +``/run/reboot-required.pkgs``. Programs should not add a package name
> > +to ``/run/reboot-required.pkgs`` if it is already present there.
> > +
> > +.. index:
> > +   single: postinst
> > +
> > +The ``/run/reboot-required`` mechanism is used when a reboot is
> > +needed to fully apply the changes introduced by package
> > +installation or upgrade.  Typically it is the ``postinst``
> > +maintainer script that touches ``/run/reboot-required``, at the end
> > +of a successful configuration of the package.
> > +
> > +There are no guarantees provided by the ``/var/reboot-required``
> > +convention as to when or whether the requested reboot will occur.
> 
> This looks good to me as well.  Seconded.

Normative changes seconded (and at some point I'll add
reboot-required.pkgs handling to dbus, which is one of the packages that
requires a reboot).

One non-normative fix:

> There are no guarantees provided by the ``/var/reboot-required``

should say /run/reboot-required.

Thanks,
smcv

Bug#761219: debian-policy: document versioned Provides

[Simon McVittie]

On Thu, 06 Jun 2019 at 21:54:40 +0100, Dominic Hargreaves wrote:
> This is a fair comment. The wording was potentially misleading. How about
> the attached instead?

This mostly looks good, just one thing that I would add:

> -If a relationship field has a version number attached, only real
> -packages will be considered to see whether the relationship is satisfied
> -(or the prohibition violated, for a conflict or breakage). In other
> -words, if a version number is specified, this is a request to ignore all
> -``Provides`` for that package name and consider only real packages. The
> -package manager will assume that a package providing that virtual
> -package is not of the "right" version. A ``Provides`` field may not
> -contain version numbers, and the version number of the concrete package
> -which provides a particular virtual package will not be considered when
> -considering a dependency on or conflict with the virtual package name.
> -[#]_
> +A ``Provides`` field may contain version numbers, and such a version number
> +will be considered when considering a dependency on or conflict with the
> +virtual package name.  [#]_ For example, given the following packages:
> +
> +::
> +
> +Package: foo
> +Depends: bar (>= 1.0)
> +
> +Package: bar
> +Version: 0.9
> +
> +Package: bar-plus
> +Provides: bar (= 1.0)
> +
> +the ``bar-plus`` package will again satisfy the dependency for
> +the ``foo`` package.

I think it's still worth saying explicitly that an unversioned Provides
doesn't satisfy any versioned dependencies or violate any versioned
conflicts:

---
... with the virtual package name.  [#]_ If the ``Provides`` field does
not specify a version number, it will not satisfy versioned dependencies
or violate versioned ``Conflicts`` or ``Breaks``. For example, given the
following packages:

::

Package: foo
Depends: bar (>= 1.0)

Package: bar
Version: 0.9

Package: bar-plus
Provides: bar (= 1.0)

Package: bar-clone
Provides: bar

the ``bar-plus`` package will satisfy the dependency for the ``foo``
package, but the ``bar-clone`` package will not.
---

I wasn't sure what happens for versioned Conflicts and Breaks on
an unversioned Provides, so I tried it, and the answer is that an
unversioned Provides is ignored by versioned Breaks: for example ack
Breaks ack-grep (<< some version), and I was able to install a test
package with "Provides: ack-grep" without removing ack.  It would be
possible to expand the example to demonstrate this:

---
For example, given the following packages:

::

Package: foo
Depends: bar (>= 1.0)

Package: bar
Version: 0.9

Package: bar-plus
Provides: bar (= 1.0)

Package: bar-clone
Provides: bar

Package: no-old-bar
Breaks: bar (<< 2.0)

the ``bar-plus`` package will satisfy the dependency for the ``foo``
package, but the ``bar-clone`` package will not. Similarly, installing the
``no-old-bar`` package will not allow the ``bar-plus`` package to be
configured, but the ``no-old-bar`` and ``bar-clone`` packages can be
installed simultaneously without error.
---

but perhaps that's sufficiently niche as to make the example more
rather than less confusing?

smcv

Bug#924401: base-files fails postinst when base-passwd is unpacked

[Simon McVittie]

On Thu, 14 Mar 2019 at 10:21:30 +0100, Santiago Vila wrote:
> The reason I'm often asked to add hacks to base-files.postinst is only
> that base-files is usually configured in the second place

I think it's also fair to say that base-files is exactly a collection of
the miscellaneous files and bootstrapping actions that no other package
seems particularly responsible for, so if a new miscellaneous file or a
new bootstrapping action appears, base-files seems a reasonably natural
home for it (or at least, more natural than a package that contains
non-bootstrapping code, for example bash or debianutils).

smcv

Bug#924401: base-files fails postinst when base-passwd is unpacked

[Simon McVittie]

It would probably be good for the overall robustness of the system if
we try to solve this from multiple angles.

On Tue, 12 Mar 2019 at 16:17:10 +0100, Helmut Grohne wrote:
> A. /etc/passwd is part of base-passwd's interface and base-files is
>right in relying on it working at all times. Then base-passwd is rc
>buggy for violating a policy must. Fixing this violation is
>technically impossible.

If it isn't implementable then it's probably the wrong design.

Strictly speaking, I think /etc/passwd *is* part (or maybe all?) of
base-passwd's Essential interface, but then the Policy requirement that
it provide this interface even when unconfigured is unimplementable, and
we can't do unimplementable things.

I think it would be reasonable to say that Essential packages are *not*
entitled to assume that base-passwd provides /etc/passwd, even though
non-Essential packages are entitled to assume it.

The lateral-thinking approach to solving this would be to have an
Essential or transitively-Essential fallback NSS module that provides
uids and gids 0 to 99 and 65534 without needing /etc/passwd. After all,
they're part of the ABI of a Debian system, so deleting them is not a
supported action (although reconfiguring their shells, etc. in /etc/passwd
is supported, especially for uid 0, so nsswitch.conf would need to have
either files or compat before this module). However, that's probably
a bigger hammer than people really want, and is annoying for multiarch
(we'd want it installed for each foreign architecture).

libnss-systemd partially solves this: it guarantees to provide uid and
gid 0 and 65534, but doesn't provide the 1-99 range, isn't Essential,
isn't portable to non-Linux, depends on systemd, and would cause political
objections.

> B. /etc/passwd is not part of base-passwd's interface and base-files
>wrongly relies on its presence rendering base-files rc buggy.

Perhaps base-files should use chown 0:0, etc.? That would be more robust.

In addition to the root user and group, it would have to hard-code
the numeric group IDs of staff, mail and utmp, AFAICS. That doesn't
seem awful? They're all in the statically-allocated range.

The postinst is already a built file (a .in file with sed substitutions),
so it could take the numeric staff, mail and utmp gids at build time
and hard-code them in the binary package without also having to hard-code
them in the source package, if preferred.

Other possibilities in this direction include:

- complete the staff-group-for-usr-local transition and stop using the
  staff group
- use systemd-tmpfiles (or something compatible with tmpfiles.d(5)
  on non-systemd systems) to create /var/log/*tmp, /var/log/lastlog,
  /run/utmp, /var/mail and any other /var files that are needed during boot
  (in fact systemd already has tmpfiles snippets for all the files I named,
  except /var/mail)

>Given that
>we have debootstrap, cdebootstrap, multistrap, and mmdebstrap, it
>seems like specifying the bootstrap interface would be a good idea.
>Unfortunately, I don't exactly understand the bootstrap interface at
>present. In practise, you cannot run postinsts of essential packages
>in arbitrary order.

This is certainly more fragile than I'd hope: I've seen debootstrap fail
in Open Build Service chroots when presented with a modified Essential
set (in a Debian derivative targeting containers that are not multi-user
systems and never run on bare metal, which doesn't need everything that a
"real" Debian system does).

If we rely on bootstrap implementations having out-of-band knowledge
of the right order to configure the Essential set, the risk is that
they need to have different out-of-band knowledge for different target
distributions, leading to the bootstrap implementation becoming relatively
tightly coupled to the target distribution.

Maybe the rule should be to retry configuration of each unconfigured
package until either they all succeed, or forward progress stops being
made? Pythonesque pseudocode:

to_configure = set(["base-files", "base-passwd", ...])

while to_configure is non-empty:
failed = set()

for p in to_configure:  # can be in arbitrary order
if run_postinst(p) fails:
failed.add(p)

if len(failed) == len(to_configure):
# Nothing succeeded since the last iteration of the outer loop.
# Assume that trying again isn't going to help.
fatal("Could not configure Essential packages:", failed)
else:
# Progress has been made, so retry any failures in the hope that
# the progress we made has unblocked them, or terminate the loop
# if there were none
to_configure = failed

assert(to_configure is empty)
log success

Regards,
smcv

Bug#917431: debian-policy: virtual packages: logind, default-logind

[Simon McVittie]

On Sun, 30 Dec 2018 at 15:39:58 +0100, Adam Borowski wrote:
> On Sat, Dec 29, 2018 at 02:07:25PM +, Sean Whitton wrote:
> > Ideally, this would be reviewed and seconded by people working on init
> > stuff, so I'm not going to second it myself unless we don't get interest.
> 
> I asked around, and got the following remark:
> 
> Mark Hindley:
> } Could you just replace "providing logind API (over D-Bus and /run/)"
> } with something like "providing logind APIs (D-Bus and sd-login)" in the
> } upgrading checklist?
> }
> } /run/ is really not part of the API and nobody should be using it directly.
> 
> Which makes sense -- directing people to wrappers and libraries instead of
> the files would be preferred.  The less ad-hoc undocumented APIs are there,
> the better.

Maybe "via D-Bus and " or "via D-Bus and sd-login(3)"?

If a logind provider such as elogind doesn't result in the libsystemd
sd-login APIs working (at least as well as they would with the provided
logind version), would you consider that to be a bug in the provider?
I asked this earlier in the bug and didn't notice an answer (my apologies
if there has been one that I missed).

I ask because libsystemd users like dbus and policykit-1 cannot be linked
to both libsystemd and libelogind without compiling the daemon twice and
somehow arranging to run the right one for the installed system, which
I think is an undesirable level of complexity; for Debian, we should be
preferring the library that matches our default installation, which is
libsystemd. Derivatives like Devuan that don't have libsystemd at all
are of course free to link their libsystemd users to libelogind instead.

smcv

Bug#917431: debian-policy: virtual packages: logind, default-logind

[Simon McVittie]

On Fri, 28 Dec 2018 at 02:52:09 +0900, Ansgar Burchardt wrote:
> Adam Borowski writes:
> > Thus, the wording would be (as proposed by fsateler):
> >
> > logind: an org.freedesktop.login1 D-Bus API implementation
> >
> > default-logind: should be provided by the distribution's default logind
> > provider (currently pam-systemd)
> 
> So any provider of logind would have to provide the full interface of
> the current interface (including future updates)?

Presumably a package that Provides: logind (= 123) would be expected to
provide the same D-Bus APIs that the logind in systemd 123 did.

Some of logind's APIs are not D-Bus APIs: in addition to D-Bus, it
has an ad-hoc form of IPC using files in /run. I think the files and
their contents themselves are not considered to be public API, but the
functions in libsystemd that read those files are. This is used by other
packages, notably dbus-daemon (which should not normally use D-Bus APIs
itself because that would be a circular dependency) for its deprecated
at_console policy, and policykit-1 for many of its policies.

Are packages that provide logind expected to create the same files in
/run so that libsystemd APIs will work as intended?

> You can start logind without libpam-systemd, it just wouldn't know about
> any session.  So it is about providing hooks to register sessions with
> logind.

A *working* org.freedesktop.login1 implementation, then, perhaps? (If an
implementation of logind doesn't know about login sessions then it can't
really be said to be working correctly.)

smcv

Bug#833401: debian-policy: virtual packages: dbus-session-bus, default-dbus-session-bus

[Simon McVittie]

On Fri, 07 Dec 2018 at 19:26:50 -0700, Sean Whitton wrote:
> diff --git a/virtual-package-names-list.yaml b/virtual-package-names-list.yaml
> + - name: dbus-session-bus
> +   description: provides the D-Bus well-known session bus for most or all 
> user login sessions
> + - name: default-dbus-session-bus
> +   description: Debian's preferred implementation of dbus-session-bus, 
> possibly architecture-specific

Seconded, thanks.

smcv

Bug#833401: debian-policy: virtual packages: dbus-session-bus, default-dbus-session-bus

[Simon McVittie]

On Sat, 03 Nov 2018 at 14:07:18 -0700, Sean Whitton wrote:
> On Thu 04 Aug 2016 at 01:43PM +0100, Simon McVittie wrote:
> >> Other options
>
> I note that no such binary package exists right now.  Does this issue
> remain unresolved?  I also note recent discussion on d-devel.  Kindly
> update this bug with the present state of play.

The routes that I described as "other options" were not taken.

The current situation is that we have the two virtual packages that I
proposed in the original bug report
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=833401#5
modulo the rename of dbus-default-session-bus to default-dbus-session-bus
as mentioned in
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=833401#10 (trying to
continue/establish a convention that the preferred implementation of a
virtual package foo is default-foo). So:

* dbus-session-bus: anything providing the D-Bus well-known session bus
  for most or all user login sessions. If dbus-session-bus is
  installed, then programs in at least graphical login sessions can
  rely on seeing the $DBUS_SESSION_BUS_ADDRESS environment variable,
  the $XDG_RUNTIME_DIR/bus Unix socket, or some other way to discover a
  session bus that is supported by all the major D-Bus client libraries.

* default-dbus-session-bus: Debian's preferred implementation of
  dbus-session-bus, possibly architecture-specific.

Programs and desktop environments that require a D-Bus session bus and
cannot work without one should normally declare:

Depends: default-dbus-session-bus | dbus-session-bus

unless they have more specific requirements. Programs with weaker
dependencies can use a Recommends or a Suggests.

(Implementation detail: default-dbus-session-bus is currently provided
by dbus-user-session on Linux, and by dbus-x11 on non-Linux ports. On
Linux, dbus-x11 is a non-default implementation of dbus-session-bus,
to be used by people who prefer to have one session bus per X11 display,
or who don't want systemd or systemd-logind.)

smcv

Bug#824495: debian-policy: Source packages "can" declare relationships

[Simon McVittie]

On Sat, 10 Nov 2018 at 19:29:03 -0700, Sean Whitton wrote:
> On Thu 08 Nov 2018 at 02:51PM GMT, Ian Jackson wrote:
> >(ii) output packages with additional features or functionality.
> >Such additional features MAY imply additional functional runtime
> >dependencies, which then SHOULD be represented in the output
> >packages' metadata.  In this case the additional packages
> >SHOULD NOT be declared in Build-Conflicts.
> 
> Do we really want (ii)?  It seems like a recipe for all sorts of
> confusion.  Do any packages currently work like that?

I don't have any concrete examples, and I'd argue that they are not
best-practice, but they almost certainly exist in the archive.

This will happen whenever an Autoconf package has a feature enabled by an
"automagic" dependency, the maintainer has not enabled the feature for all
Debian users by adding its required package to B-D or explicitly enabling
it with --with-foo or --enable-foo (either deliberately, or because they
didn't notice the feature being added), and the maintainer has also not
explicitly disabled it with --without-foo or --disable-foo.

For instance, dbus has optional support for using AppArmor policy to
decide which messages to deliver. If its maintainer hadn't noticed that
feature being added, then it wouldn't have B-D: libapparmor-dev or
./configure --enable-apparmor, but it also wouldn't have
./configure --disable-apparmor, resulting in the upstream default
behaviour (AppArmor-related features enabled if and only if
libapparmor-dev is installed).

I think the best practice is for maintainers to notice new features being
added, make a deliberate decision on what to enable (in an architecture-
or kernel-specific way if appropriate), and use the equivalent of
--disable-apparmor for any feature we don't (yet?) want to enable.

smcv

Bug#911165: debian-policy: drop requirement to ship sysvinit init script with same name

[Simon McVittie]

On Tue, 16 Oct 2018 at 20:31:54 +0200, Andreas Henriksson wrote:
> I'm still leaning towards thinking just dropping this section is
> better than doing a direct translation of it to the current systemd
> reality which might just end up being confusing and help noone.

One part of this section that seems valuable to rescue is:

If you have an LSB (or "sysvinit") init script /etc/init.d/foo, and
systemd unit(s) that are intended to be used instead of the LSB init
script on systemd-booted systems, then the package must either include
foo.service, or mask foo.service.

(This is a consequence of how systemd's backwards compatibility for LSB
init scripts is set up. It's a much simpler and easier requirement than
the wording we used to have about how to make Upstart coexist with LSB
init scripts, but it's still a requirement.)

smcv

Bug#904248: Beginnings of a patch to add netbase to build-essential

[Simon McVittie]

On Fri, 12 Oct 2018 at 13:45:06 +0200, Mattia Rizzolo wrote:
> On the note of /etc/hosts, I'm fixing the original bug rised by Simon
> McVittie (that was triggered by tests.reproducible-builds.org not
> resolving localhost) within pbuilder (#905307) btw.

I think this is a better solution to my original bug report. If my unit
tests need to talk to a service listening on localhost (as allowed by
Policy already) then let them resolve localhost: no more, no less.

> I think we also need a wider request for /etc/services and
> /etc/protocols.  Till now I only heard Ian asking for it, so I believe
> there needs to be a wider request.

I too am not so sure that these files are something that the
build-essential set should guarantee. I think resolvability of localhost
falls into the same sort of "always true on a sane system" guarantees
as the lo device having 127.0.0.1 assigned to it, but I'm a lot less
sure that "freeciv can be resolved to port 5556" is part of a minimal
reasonable system. (Admittedly, I'm deliberately choosing a slightly
ridiculous example to make a point, and a lot of what's in /etc/services
is more reasonable.)

smcv

Re: Bug#905817: UID range of DyanmicUser overlaps with existing definitions in debian-policy

[Simon McVittie]

On Sat, 15 Sep 2018 at 08:47:19 -0700, Sean Whitton wrote:
> On Fri 10 Aug 2018 at 08:23AM +0200, Michael Biebl wrote:
> > There is also:
> > 65536-4294967293:
> >  Dynamically allocated user accounts. By default adduser will not
> >  allocate UIDs and GIDs in this range, to ease compatibility with legacy
> >  systems where uid_t is still 16 bits.
> >
> > I'm not sure if it would be more suitable to pick the DynamicUser ids
> > from this range.
> 
> This strikes me as suitable.  We could either just change systemd's
> configuration, or allocate a section of that range to systemd.

Beware that if systemd dynamic users are above the 16-bit boundary, then
they won't work well with systemd-nspawn and other container systems that
give a 16-bit uid range to each container (mapping uids N+0 to N+65535
in the top-level uid namespace to uids 0 to 65535 in the container's
uid namespace, where N is large; so when systemd inside the container
thinks it's allocating uid 64923 to a service, it's really uid N+64923
in the top-level init namespace). That's a useful technique because
it assigns a unique uid to each process that ought to be protected from
other processes, which protects both the host system and other containers
from a compromised container.

smcv

Bug#906949: Clarify documentation location in a Python2-less distribution

[Simon McVittie]

On Thu, 23 Aug 2018 at 15:43:06 +0100, Ian Jackson wrote:
> There are two reasons for the standardised paths in /usr/share/doc:
>  1. So that the user can find the right documentation.
>  2. So that you avoid clashes with other packagkes.
> 
> Not moving the docs about assists 1.  As does providing a link
> /usr/share/doc/python3-foo -> python-foo.

It would be non-Policy-compliant for python3-foo to have a symlink
/usr/share/doc/python3-foo -> python-foo unless it Depends on
python-foo (= ${binary:Version}), because python3-foo is required to
contain or provide a corresponding /usr/share/doc/python3-foo/copyright
and .../changelog.Debian.gz.

However, it would be wrong for python3-foo to depend on python-foo
or vice versa, because they are libraries for different interpreters
and do not really have any dependency relationship. That's why the
documentation is usually split out into python-foo-doc.deb.

> Alternatively, when dropping the python[2] package, move the docs but
> leave a symlink the other way:
>  /usr/share/doc/python-foo -> python3-foo

Similarly, you can't do that unless python3-foo Conflicts: python-foo,
which seems like overly-aggressive deprecation :-)

Symlinks *inside* the per-package documentation directory are OK though,
for example /usr/share/doc/python3-foo/html -> ../python-foo/html, or
perhaps more realistically
/usr/share/doc/python{,3}-foo/html -> ../python-foo-doc/html,
which is how this works in src:dbus-python for example.

smcv

Bug#906949: Clarify documentation location in a Python2-less distribution

[Simon McVittie]

On Thu, 23 Aug 2018 at 01:20:44 +1000, Stuart Prescott wrote:
> d) revert to /usr/share/doc/python-foo-doc: do we ignore policy's 
> recommendation, overriding (or changing) dh_installdocs? While 
> /usr/share/doc/main-package is only a recommendation in policy, 700 
> python-foo-doc packages deliberately ignoring it seems wrong.

In dbus-python I left the documentation in /usr/share/doc/python-dbus-doc
(overriding dh_installdocs), but put symlinks to it in
/usr/share/doc/python{,3}-dbus so that users can find it by following
the Policy-recommended route. Maybe that's a reasonable compromise?

smcv

Bug#883950: Next steps on "[GPL-3+]" proposal

[Simon McVittie]

On Sun, 12 Aug 2018 at 13:52:57 +0200, Wouter Verhelst wrote:
> The obvious objection to that would be the fact that the SPDX
> identifiers are not set in stone; a future update of the SPDX
> identifiers might then conflict with one of the identifiers that we add.
> Either we'd need [...], or a rule to not have non-SPDX identifiers.
> 
> Personally, I have a preference towards the latter; it seems simpler,
> and there is benefit to be had to encourage creating a new SPDX
> identifier over having a "local" fix.

I don't think we should have a policy that forbids packaging useful
software whose license happens to be absent from the SPDX list. It can
be harder, it can need special cases, but it should not be impossible.

For example, if I was a maintainer of the SPDX license list, I wouldn't
want to include the license of the Return to Castle Wolfenstein engine
(see src:iortcw), which is GPLv3 with additional terms that will probably
never be used as-is by any other upstream; adding instances of license
proliferation to the list just because they are used by a leaf package
really doesn't scale. However, it's a valid GPLv3-compatible free software
license, and we should be able to include software under that license
in Debian (iortcw is in contrib because it needs proprietary game data,
but if someone extracts or derives code from iortcw that does not need
the proprietary game data, it should be allowed in main).

I am not defending license proliferation, but as a downstream maintainer
I don't get to decide whether my upstream participates in it.

smcv

Bug#904729: Policy 12.5: Must the license grant be included in debian/copyright?

[Simon McVittie]

On Wed, 01 Aug 2018 at 19:23:09 -0700, Jonathan Nieder wrote:
> Simon McVittie wrote:
> 
> >   ( ) the full text of the license, *and* the license grant
> >   (unless the license *is* the license grant, like BSD-style licenses)
> 
> This wording confuses me.  All licenses are license grants.

In the past, it has been asserted that maintainers are required to
paste the text written by upstream that tells the consumer that they
may redistribute the package under a specified license, verbatim,
into the copyright file. That's what I meant whenever I said "license
grant" on this bug. (Not to be confused with the text you can find in
/usr/share/common-licenses, which tells you what the terms of the GPL
are, but does not tell you that you can distribute any particular piece
of software under those terms.)

This might be in the form

"""
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.
"""

or

"""
This file is licensed under the Creative Commons Attribution-Share
Alike 3.0 Unported license.
"""

but equally it might be

"""
SPDX-License-Identifier: GPL-2+
"""

or even

"""
<http://icculus.org/pipermail/quake3/2007-September/001979.html>:
  http://f0rqu3.deviantart.com/art/quake3-icon-64264660
  this is the one I made from the original logo
  and there is another in ioquake3 svn
<http://icculus.org/pipermail/quake3/2007-September/001980.html>:
  This is awesome, can I use it?
<http://icculus.org/pipermail/quake3/2007-September/001981.html>:
  I think he ment to use it under the GPL in ioq3 :)
  For normal use there is no need to ask...
  http://creativecommons.org/licenses/by-nc-sa/2.5/
<http://icculus.org/pipermail/quake3/2007-September/001984.html>:
  yes. I cant choose GPL as license in DA so I use CC :/
"""

> | Files: *
> | License: GPL-2+
> |  On Debian systems the full text of the GPL-2 can be found in
> |  /usr/share/common-licenses/GPL-2
> 
> in combination with
> 
> | Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
> 
> makes the intention very clear; the only potential icing on the top
> would be if there were some file for users explaining how to interpret
> this file.

The reason I am being so pedantic about this is that previous statements
from the ftp team have implied that paraphrasing the license grant text
provided by upstream (for example simplifying "This program is free
software; etc." into "License: GPL-2+") is not acceptable, and I want
to be sure that this rule has intentionally been changed.

(For clarity, I think what you said is a very valuable simplification,
and I would love to be able to stop copying and pasting upstreams'
license grant text.)

smcv

Bug#228692: User/group creation/removal in package maintainer scripts

[Simon McVittie]

On Tue, 31 Jul 2018 at 17:53:50 +0200, Andreas Henriksson wrote:
> previously created users should *not* (ever) be removed

There has been a suggestion in the past that these users should be locked
on package removal and unlocked on reinstallation, as implemented in
(for example) openarena-server. It is not entirely clear to me what
technical benefit this has, given that these users normally have a
disabled password anyway.

> Packages commonly check if user/group already exists before calling
> adduser to create them.

I have seen it suggested elsewhere that this is a bug or misunderstanding,
because adduser --system is already meant to exit successfully if the
requested user or group already exists in the system range. Calling
adduser conditionally prevents adduser from detecting whether the user
or group exists but is outside the system range.

> Writing manual mantainerscript code should always be avoided, because
> it's a common source of bugs.

Some alternatives to open-coding this:

systemd-sysusers(8) creates system users from declarative text files,
either at package installation or during early boot (part of a wider goal
for it to be feasible to boot a stateless or generic system after emptying
/etc and /var), in a way that is feasible to reimplement outside systemd
if people want to (but has not been reimplemented, as far as I'm aware).

dh-sysuser encapsulates maintainer script code into a single command,
although imperative rather than declarative. It uses useradd directly,
so it might be NIHing adduser(8).

> An example of a mechanism that
> allows not creating static system users/groups is unit file option
> DynamicUser=yes from systemd (and likely many others that I'm not aware
> of). For further information see:
> https://www.freedesktop.org/software/systemd/man/systemd.exec.html#DynamicUser=
> http://0pointer.net/blog/dynamic-users-with-systemd.html

But note that:

- this doesn't work if some other daemon needs to know about your
  system user ahead of time: in particular, dbus-daemon system.d snippets
  cannot currently refer to dynamic users

- this is systemd-specific (suitable for systemd-systems-only software like
  systemd-cron, but not suitable for general daemons, unless Debian drops
  support for non-systemd init systems and non-Linux kernels)

Regards,
smcv

Bug#883950: Next steps on "[GPL-3+]" proposal

[Simon McVittie]

On Sun, 29 Jul 2018 at 00:50:34 +1000, Stuart Prescott wrote:
> Let us consider this proposed syntax in terms of what someone unfamiliar 
> with the format is going to see

Along these lines, it might be helpful for people with an interest in
pushing this forward to convert some d/copyright files to their proposed
syntax, as concrete examples of what they're proposing. The ftp team
might also find it easier to approve or reject proposals when looking at
a concrete example?

Having redone its copyright file recently, I can suggest mutter as a
potential test case for this that is non-trivial, but not so verbose
as to be opaque: it has a mixture of various GPL-, LGPL- and BSD-based
licenses, including GPL-2+ with the Autoconf exception.

adwaita-icon-theme is a more pathological example with many Creative
Commons licenses, including at least one that is considered to be
non-Free (but is included in a triple-license with a Free option).

smcv

Bug#883950: Bug#904729: Policy 12.5: Must the license grant be included in debian/copyright?

[Simon McVittie]

On Fri, 27 Jul 2018 at 17:18:42 +0800, Sean Whitton wrote:
> If we want to implement anything like what's being discussed
> in #883950, we need section 12.5 to state explicitly that the license
> grant need not be included.

 suggests
that the license grant is no longer required, which appears to be a
change in ftp team requirements since the previous official statement
,
which said: "Its not enough to have the following two-liner ["On Debian
systems", etc.]. There are license headers [license grants], you should
use those".

ftp team: has the requirement to reproduce license grants for (some?)
licenses in d/copyright changed since 2006, or am I misunderstanding
the statement in #883950?

To make things easier, here are some possible policies that the ftp team
might reasonably choose:

* For common-licenses, d/copyright must include

  ( ) at least one of: a well-known license identifier like "GPL-2+"
  or a reference to common-licenses ("On Debian systems", etc.)
  ( ) a reference to common-licenses, and also at least one of:
  a well-known license identifier like "GPL-2+", or a license grant
  ( ) both a license grant and a reference to common-licenses
  ( ) something else (please specify)

* For licenses not available in common-licenses, d/copyright must include

  ( ) the full text of the license
  ( ) the full text of the license, *and* the license grant
  (unless the license *is* the license grant, like BSD-style licenses)
  ( ) something else (please specify)

Thanks,
smcv

Bug#459427: Patch seeking seconds on changelog vs. NEWS handling

[Simon McVittie]

On Wed, 25 Jul 2018 at 20:57:41 -0700, Jonathan Nieder wrote:
> a) The work must carry prominent notices stating that you modified
>it, and giving a relevant date.

I don't think this is normally interpreted as requiring that *all*
modifications be listed, only that the existence of modifications is
obvious, or (to borrow wording from another popular license) the software
is not misrepresented as being the unmodified original source. Remember
that when the GPL was written, the expectation was that people would
receive Free Software on tapes or similar and would not be able to apply
the now-more-pragmatic policy of "if you want upstream's unmodified
version of GNU Hello, go and download it from GNU".

Debian packages have a Debian changelog and (when the source package
is included, without which we are not GPL-compliant anyway) a Debian
diff/tar, so there is certainly a prominent notice that we have modified
it. I am not a lawyer, but I believe this is enough for the letter and
spirit of the license: it tells a user "if you are looking for an
unmodified GNU Hello, this is not it".

smcv

Bug#804018: options to avoid service startup on package installation

[Simon McVittie]

On Wed, 25 Jul 2018 at 05:11:15 +0100, Ian Jackson wrote:
> ISTM that in days gone by this would have been (should have been) a
> wishlist item for invoke-rc.d.  The interface was defined in
> invoke-rc.d(8) and README.policy-rc.d.gz.
...
> Nowadays, there is systemd too.  I have no idea whether
> policyrcd-script-zg2 works with systemd.

It should. Packages that have an LSB init script (with or without a
parallel systemd unit) run update-rc.d and invoke-rc.d in their postinst,
as they did before systemd. Packages that have only a systemd unit run
deb-systemd-helper and deb-systemd-invoke instead. Both invoke-rc.d and
deb-systemd-invoke run policy-rc.d to decide whether they should continue.
All of this now lives in the Essential init-system-helpers package, which
took over the init-agnostic scripts from sysvinit.

Some examples:
* LSB init script only: cpufrequtils, ikiwiki-hosting-web, sysfsutils
* Both: apt-cacher-ng, avahi-daemon, uuid-runtime, many more
* systemd only: quake4-server, systemd-cron

policyrcd-script-zg2 is an adapter between the invoke-rc.d-defined API
(which says the policy-rc.d script must be in /usr/sbin) and what that
API should maybe have been all along (additionally looking for the script
in a configurable location, /usr/local/sbin, and /etc).

The policy-rc.d interface seems to have at least two overall objectives,
which shouldn't necessarily be conflated.

Some sysadmins want to stick to the usual Debian setup in general
(services that are installed get started), but disable individual
services: for instance I have various game dedicated servers like
openarena-server installed, but I don't want them to run when my laptop
boots, only on-demand. policy-rc.d is inconvenient for this use-case
because there is only one policy-rc.d script, so the list of services to
run is centralized and written in an imperative rather than declarative
language (unless you write a local policy-rc.d implementation that reads
a ".d"-style configuration directory, but you're still responsible for
writing the script that pulls it all together, because no such script
is packaged).

Other sysadmins (particularly those working with chroots) want a more
Red-Hat-like setup where services are not, in general, started just
because they are installed (possibly with a whitelist of exceptions to
that general policy, which is what happens in Red Hat). policy-rc.d is
fine for this, although somewhat over-engineered for the common trivial
cases "start everything" and "start nothing". sbuild-createchroot is one
example of a tool that provides a policy-rc.d that doesn't start anything,
although this is somewhat obsoleted by the recent change that /sbin/init
is non-Essential and invoke-rc.d doesn't start services when there is
no /sbin/init.

> If systemd has a different way of achieving the same overall objective
> then maybe the answer is just that the admin must configure this
> differently depending on what init system they have.

systemd does have an equivalent of policy-rc.d ("presets",
systemd.preset(5)) with what I personally think is a better configuration
language (declarative files in a .d-style directory instead of a single
imperative script), but it isn't fully supported by Debian packages:
it looks as though deb-systemd-helper (the equivalent of update-rc.d
for systemd units with no LSB equivalent) does respect presets, but
update-rc.d itself does not.

The problem with systemd-specific interfaces (and sysv-rc-specific
interfaces, and other-init-specific interfaces) is that it's hard to
make them meet the goal that switching between init systems preserves
"enabledness". I suspect policy-rc.d is going to be the only "official
Debian" interface to this as long as that goal exists.

An implementation of policy-rc.d that reads systemd presets and applies
them to any Debian-supported init system might be an interesting way to
get declarative configuration for this, if full support for sysvinit
and init-system-switching continues to be a goal.

smcv

Bug#795402: base-files: Please add Creative Commons license texts

[Simon McVittie]

On Mon, 09 Jul 2018 at 15:16:27 +0100, Jonathan Dowland wrote:
> CC-BY-SA3.0 303

Note that some of those will be CC-BY-SA-3.0-US, some will be CC-BY-SA-3.0
Unported, and some might be a different (legally distinct) localization
(there are many).

Some uses of Creative Commons licenses are probably on dual- or
triple-licensed content, for packages whose maintainers have assumed that
Policy requires the entire license terms to be reproduced in d/copyright,
including non-DFSG elements of a multiple-license that is DFSG-compliant
when taken as a whole (it isn't entirely clear what Policy §12.5 or
the ftp team's rules require for such licenses).

smcv

Bug#787816: Replace FHS 2.3 by FHS 3.0 in the Policy.

[Simon McVittie]

How about the attached patch?

Complete patch series (including non-normative) updated here:
https://salsa.debian.org/smcv/policy/merge_requests/1/diffs

On Thu, 28 Jun 2018 at 14:04:28 +0100, Sean Whitton wrote:
> On Thu, Jun 28 2018, Simon McVittie wrote:
> > On 64-bit architectures, only the dynamic linker and libc are allowed to
> > install files in /lib64.
> 
> Let's drop the "On 64-bit architectures..." because it makes the
> requirement simpler.

Done

> So how about adding a bullet point that says that the requirement is
> relaxed to a recommendation?  Then it is a normal severity bug at most,
> if not lower.

Done

> > As an exception to the requirement for there to be no subdirectories
> > in ``/usr/bin``, the ``mh`` mail-handling suite may create
> > ``/usr/bin/mh/``, as was allowed in FHS version 2.3. Other
> > subdirectories are not allowed.
> 
> LGTM.

Applied

> > I'll check that the version I imported from a web server matches what's
> > in that bzr repository.

It does (I removed an extra file that was only on the web server but
not in bzr, containing links to the various generated documents)

Thanks,
    smcv
>From 5205d0a50465cf422f1040d9395d5ea83dbfde5f Mon Sep 17 00:00:00 2001
From: Simon McVittie 
Date: Thu, 14 Jun 2018 11:43:04 +0100
Subject: [PATCH 3/3] Update to FHS v3.0

Notable changes that this causes:

* A GNU-style /usr/libexec is now allowed.

Notable non-changes:

* /usr/games, /usr/share/games and /usr/lib/games were optional in
  FHS 2.3, and are still optional. It is up to us whether we want
  to keep using those directories.

Drop our special exception for /run, and replace it with a requirement
that /var/run and /var/lock are symlinks to /run and /run/lock
respectively. The FHS does not mandate that these directories are
symlinked or bind-mounted (although I think it should) and some
misguided distributions make /run and /var/run non-equivalent, but they
have always been equivalent on Debian.

Drop our special exception for /sys, which is now standardized in the
FHS.

Relax the requirement to create /usr/local/share/color to a
recommendation to avoid making 6 packages immediately noncompliant.

Add a special exception for /usr/bin/mh/ for now, restoring the FHS
2.3 situation (dropping this might be a good idea, but should be
disussed with the nmh and mailutils-mh maintainers if desired).

Signed-off-by: Simon McVittie 
Closes: #787816
---
 policy/ch-opersys.rst | 33 +
 1 file changed, 13 insertions(+), 20 deletions(-)

diff --git a/policy/ch-opersys.rst b/policy/ch-opersys.rst
index f9f878f..cded697 100644
--- a/policy/ch-opersys.rst
+++ b/policy/ch-opersys.rst
@@ -12,7 +12,7 @@ File System Structure
 ~
 
 The location of all files and directories must comply with the
-Filesystem Hierarchy Standard (FHS), version 2.3, with the exceptions
+Filesystem Hierarchy Standard (FHS), version 3.0, with the exceptions
 noted below, and except where doing so would violate other terms of
 Debian Policy. The following exceptions to the FHS apply:
 
@@ -34,9 +34,8 @@ Debian Policy. The following exceptions to the FHS apply:
 is recommended the configuration files not start with the '.'
 character.
 
-3.  The requirement for amd64 to use ``/lib64`` for 64 bit binaries is
-removed.  Only the dynamic linker and libc are allowed to use this
-directory.
+3.  Only the dynamic linker and libc are allowed to install files
+in ``/lib64``.
 
 4.  The requirement for object files, internal binaries, and libraries,
 including ``libc.so.*``, to be located directly under ``/lib{,32}``
@@ -76,20 +75,13 @@ Debian Policy. The following exceptions to the FHS apply:
 ``/etc``, or at least are symlinked there, is relaxed to a
 recommendation.
 
-8.  The additional directory ``/run`` in the root file system is
-allowed. ``/run`` replaces ``/var/run``, and the subdirectory
-``/run/lock`` replaces ``/var/lock``, with the ``/var`` directories
-replaced by symlinks for backwards compatibility. ``/run`` and
-``/run/lock`` must follow all of the requirements in the FHS for
-``/var/run`` and ``/var/lock``, respectively, such as file naming
-conventions, file format requirements, or the requirement that files
-be cleared during the boot process. Files and directories residing
-in ``/run`` should be stored on a temporary file system.
+8.  ``/var/run`` is required to be a symbolic link to ``/run``,
+and ``/var/lock`` is required to be a symbolic link to ``/run/lock``.
 
-9.  The ``/sys`` directory in the root filesystem is additionally
-allowed.  [#]_
+9.  The ``/var/www`` directory is additionally allowed.
 
-10. The ``/var/www`` directory is additionally allowed.
+10. The requirement for ``/usr/local/share/color`` to exist if
+``/usr/share/color`

Bug#787816: Replace FHS 2.3 by FHS 3.0 in the Policy.

[Simon McVittie]

On Sat, 23 Jun 2018 at 21:05:07 +0100, Sean Whitton wrote:
> 1.  FHS 3.0 allows distributions to create directory hierarchies under
> user's home directories conforming to the XDG Base Directories or
> the GLib conventions on user directory contents.
> 
> We don't permit packages to install to home directories or
> maintscripts to touch home directories, so maybe we need to add an
> exception saying that packages can't actually do this (of course,
> programs installed by those packages can do it).

OK. Something like this?

Packages must not contain files in /home, and packages' maintainer
scripts must not write to users' home directories. The programs in
those packages may create directory hierarchies as described in
§3.8.3 "Home Directory Specifications and Conventions" when run
by a user.

I'm not so sure whether this belongs in the FHS section? I think it's a
point about how our packages are required to behave, rather than about the
directories that can exist and their purposes. The directory hierarchies
are still the same, regardless of how they're created.

> 2. The requirement for amd64 libraries to be installed to /lib64 have
>been removed, so we can drop/reword our exception for that (point 3 in
>9.1.1 in current Policy)

OK, so more like this?

On 64-bit architectures, only the dynamic linker and libc are allowed to
install files in /lib64.

(Or perhaps without the "On 64-bit architectures" prefix, but I don't
know whether we have any 32-bit architectures with 64-bit multilib
that uses /lib64 other than for the dynamic linker and libc. lib64z1
is a multilib variant of a library that normally lives on the rootfs,
and it uses /usr/lib64, so perhaps that situation doesn't actually exist.)

> 3. We are violating the new requirements that /usr/local/share/color
>must exist if /usr/share/color exists.  I guess we just need to file
>a bug after policy is updated.

apt-file says we would need to file bugs against argyll-ref, colord-data,
icc-profiles, icc-profiles-free, krita-data and libgs9-common. I don't
have krita-data installed so I didn't inspect it, but none of the others
have maintainer scripts, except for argyll-ref which does some trivial
symlink-to-dir conversion.

I can't help thinking that adding maintainer scripts to those
packages is a lot of effort to go to to have an extra directory in
/usr/local/share, particularly with the elaborate dance involving
/etc/staff-group-for-usr-local that dh_usrlocal is now required to
perform. Can't sysadmins create that directory themselves if they want it?
Might it be better to add an exception so that everything we do now is
still allowed, in the same spirit as the /usr/bin/mh thing?

We could even extend the current point

The requirement for /usr/local/libqual to exist if /libqual or
/usr/libqual exists (where libqual is a variant of lib such as lib32
or lib64) is removed.

and turn it into

All requirements for /usr/local/foo to exist if /foo or /usr/foo
exists (where foo is any directory specified by the FHS) are removed.

After all, the code with fewest bugs is the code that isn't there :-)
(This doesn't mean packages can't create those directories if they want
to: Policy §9.1.2 says they still can.)

I'm not really sure why the FHS has requirements of the form "if
/usr/foo exists then /usr/local/foo must exist" anyway. I think what
they really mean might be "if an OS component reads /usr/foo, then it
must read /usr/local/foo, if it exists, with higher precedence" but
neither actually implies the other.

> > +8.  The ``/var/run`` and ``/var/lock`` directories are required to be
> > +made equivalent to ``/run`` and ``/run/lock`` respectively, for
> > +example using symbolic links or bind-mounts.
> 
> The change here is not just to update to FHS 3.0 but also to allow bind
> mounting instead of symlinking, and indeed to allow any other way of
> making them "equivalent".  The previous version of the text required
> symlinking.  And FHS 3.0 only explicitly mentions symlinking.

OK, I'll require symlinking too. I was vaguely remembering that the
transition path between /var/run and /run had involved bind mounts under
at least some circumstances, but it'll be simpler to require that /var/run
is a symlink to /run and /var/lock is a symlink to /run/lock, as it is
on any reasonable system these days.

> > +11. The requirement for there to be no subdirectories in ``/usr/bin``
> > +is relaxed to allow the ``mh`` mail-handling suite to create
> > +``/usr/bin/mh/``, as was allowed in FHS version 2.3.
> 
> I think this needs to be worded more strongly so that it is clear that
> the mh case is the /only/ exception.

OK; that was my intention, but it could have been clearer. How about:

As an exception to the requirement for there to be no subdirectories
in ``/usr/bin``, the ``mh`` mail-handling suite may create
``/usr/bin/mh/``, as was allowed in FHS

Bug#846970: Patch to document Build-Indep-Architecture field

[Simon McVittie]

On Fri, 15 Jun 2018 at 18:16:47 +0100, Ian Jackson wrote:
> > > +Specification of architectures on which the architecture-independent
> > > +binary packages are known to be buildable and/or not buildable.  If
> > > +this field is not specified, it defaults to ``any``, matching all
> > > +Debian machine architectures.  If specified, it should be either
> > > +
> > > +-  A unique single word identifying a Debian machine architecture as
> > > +   described in :ref:`s-arch-spec`.
> > > +
> > > +-  An architecture wildcard identifying a set of Debian machine
> > > +   architectures, see :ref:`s-arch-wildcard-spec`.

> I'm afraid I don't understand this.

Was the intention that the Build-Indep-Architecture is a space-separated
list of words, each of which is an architecture or architecture wildcard
as defined above? That would make more sense (and would almost match
Architecture in debian/control or in a .dsc file, except that 'all'
doesn't make sense here and so is disallowed, which as far as I can
work out makes the only difference between those two Architecture fields
disappear).

I know the intention was "it's the same as Architecture because we
already have to know how to parse that", but Architecture has different
specifications in debian/control, in a .dsc file, in a .changes file,
and in a binary package's DEBIAN/control...

smcv

Bug#880920: Document Rules-Requires-Root field

[Simon McVittie]

On Fri, 15 Jun 2018 at 18:02:39 +0100, Sean Whitton wrote:
> Here is the complete new diff for seconding

Seconded (as below).
smcv

> > diff --git a/debian/changelog b/debian/changelog
> > index 2dea331..b89816e 100644
> > --- a/debian/changelog
> > +++ b/debian/changelog
> > @@ -1,5 +1,11 @@
> > -debian-policy (4.1.4.2) UNRELEASED; urgency=medium
> > +debian-policy (4.1.5.0) UNRELEASED; urgency=medium
> >
> > +  * Policy: Document Rules-Requires-Root
> > +Wording: Niels Thykier 
> > +Wording: Guillem Jover 
> > +Wording: Sean Whitton 
> > +Seconded: ...
> > +Closes: #880920
> >* Fix URL to the alioth lists service in footnote (Closes: #896749).
> >  Thanks Martin Zobel-Helas for the report.
> >
> > diff --git a/policy/ch-controlfields.rst b/policy/ch-controlfields.rst
> > index 0771346..3afba4c 100644
> > --- a/policy/ch-controlfields.rst
> > +++ b/policy/ch-controlfields.rst
> > @@ -129,6 +129,8 @@ package) are:
> >
> >  -  :ref:`Testsuite `
> >
> > +-  :ref:`Rules-Requires-Root `
> > +
> >  The fields in the binary package paragraphs are:
> >
> >  -  :ref:`Package ` (mandatory)
> > @@ -1020,6 +1022,118 @@ This field is automatically added to Debian source 
> > control files
> >  field may also be used in source package control files
> >  (``debian/control``) if needed in other situations.
> >
> > +.. _s-f-Rules-Requires-Root:
> > +
> > +``Rules-Requires-Root``
> > +~~~
> > +
> > +Simple field that defines if the source package requires access to
> > +root (or fakeroot) during selected targets in the :ref:`Main building
> > +script: debian/rules `.
> > +
> > +The field can consist of exactly one of the following three items:
> > +
> > + - ``no``: Declares that neither root nor fakeroot is required.
> > +   Package builders (e.g. dpkg-buildpackage) may choose to invoke any
> > +   target in ``debian/rules`` with an unprivileged user.
> > +
> > + - ``binary-targets`` (default): Declares that the package will need
> > +   the root (or fakeroot) when either of the ``binary``,
> > +   ``binary-arch`` or ``binary-indep`` targets are called.  This is
> > +   how every tool behaved before this field was defined.
> > +
> > + - A space separated list of keywords described below.  These keywords
> > +   must always contain a forward slash, which sets them apart from the
> > +   other possible values of ``Rules-Requires-Root``.  When this list
> > +   is provided, the builder must provide a `gain root command` (as
> > +   defined in :ref:`debian/rules and Rules-Requires-Root
> > +   `) *or* pretend that the value was set
> > +   to ``binary-targets``, and both the builder and the package's
> > +   ``debian/rules`` script must downgrade accordingly (see below).
> > +
> > +If the package builder supports the ``Rules-Requires-Root`` field and
> > +wants to enable the feature, then it must set the environment variable
> > +``DEB_RULES_REQUIRES_ROOT`` when invoking the package building script
> > +``debian/rules``.  The value of ``DEB_RULES_REQUIRES_ROOT`` should be
> > +one of:
> > +
> > + * The value of ``Rules-Requires-Root`` if the builder can support
> > +   that value.  The builder may trim unnecessary whitespace used to
> > +   format the field for readability.
> > +
> > + * The value ``binary-targets`` if it cannot support the value of
> > +   ``Rules-Requires-Root``.
> > +
> > +A compliant builder may also leave ``DEB_RULES_REQUIRES_ROOT`` unset
> > +or set it to ``binary-targets`` if it has been requested to test
> > +whether the package it builds correctly implements the fall-back for
> > +legacy builders.
> > +
> > +Remarks
> > +^^^
> > +
> > +All packages and builders must support ``binary-targets`` as this was
> > +the historical behaviour prior to the introduction of this field.
> > +
> > +Any tool (particularly older versions of them) may be unaware of this
> > +field and behave like the field was set to ``binary-targets``.  The
> > +package build must gracefully cope with this and produce a
> > +semantically equivalent result.
> > +
> > +This field intentionally does not enable a package to request a true
> > +root over fakeroot.
> > +
> > +Definition of the keywords
> > +^^
> > +
> > +The keywords have the format ``/``, where:
> > +
> > + *  must consist entirely of printable ASCII characters
> > +   except for any whitespace and the forward slash (``/``).  It must
> > +   consist of at least 2 characters.
> > +
> > + * ``/`` (between  and ) is a single ASCII
> > +   forward slash.
> > +
> > + *  must consist entirely of printable ASCII characters
> > +   except for any whitespace.  It must consist of at least 2
> > +   characters.
> > +
> > +These keywords define where the package build script ``debian/rules``,
> > +or the tools called by that script, will need access to root or
> > +fakeroot.
> > +
> > +In addition to the keywords defined in the next section, each tool or
> > +package may define keywords w

Bug#880920: Document Rules-Requires-Root field

[Simon McVittie]

On Fri, 15 Jun 2018 at 13:43:36 +0100, Sean Whitton wrote:
> On Thu, Jun 14 2018, Paul Gevers wrote:
> >> +This command
> >> +allows the ``debian/rules`` target to run particular subcommands under
> >
> >^^ lintian will tell you this should be "enables"
> 
> As a native speaker I find 'allows' more natural than 'enables'.
> 'Allows' is certainly not grammatically incorrect.  Can you point me to
> the Lintian tag?

(I am a native en_GB speaker)

I agree that the quoted diff was correct English.

Lintian's rather simplistic spelling and grammar checker complains
about the formulation "this utility *allows to* reticulate splines"
(my emphasis), which is a fairly common bit of en_DE. I assume it's
a literal translation of something that's correct in German?

That isn't correct English: it doesn't say who or what is given
the ability to reticulate splines. The Lintian spellchecker suggests
"this utility *allows one to* reticulate splines", which is correct and
general but maybe overly formal. "This utility allows a user to reticulate
splines", "this utility allows game developers to reticulate splines" or
"this utility allows simulation game engines to reticulate splines"[1]
would also be correct, but Lintian's spellchecker doesn't have enough
context to know who/what.

In this case, the sentence correctly says who/what is given the ability
to run subcommands: the ``debian/rules`` target.

> >> +The
> >> +`gain root command` must not rely on shell features because it need
> >> +not be used via a shell.
> >
> > I am not a native speaker, but isn't "doesn't need to" more natural?
> > Otherwise it should be "needs" I guess.
> 
> s/need/needs/ would not be grammatically correct.
> 
> I am a native speaker and I find the "need not" more idiomatic than
> "doesn't need to", but I have to admit that I do not know why :)
> 
> Here is a possible explanation, but please do not rely on this in the
> future unless some other native speakers are able to confirm it:
> 
> "The gain root command ... because it need not be used via a shell"
> <-- it /can/ be used without a shell and /might/ be used without a shell,
> /therefore/ it should not rely on shell features.
> 
> "The gain root command ... because it doesn't need to be used via a
> shell" <-- it /can/ be used without a shell,
> /therefore/ it should not rely on shell features.
> 
> i.e. the 'need not' carries the additional connotation that it might
> /actually/ be used without a shell, which is the reason for not relying
> on shell features.

This all seems valid to me, but it's relatively subtle. Perhaps you could
clarify this sentence by saying "The gain root command must (blah blah)
because it will not necessarily be used by a shell"? I think that's
maybe easier to understand?

smcv

[1] http://sims.wikia.com/wiki/Reticulating_splines

Bug#864615: please update version of posix standard for scripts (section 10.4)

[Simon McVittie]

On Fri, 15 Jun 2018 at 13:06:43 +0100, Sean Whitton wrote:
> On Thu, Jun 14 2018, Simon McVittie wrote:
> > I'd suggest replacing SUSv3 with POSIX.1-2017 or SUSv4 2018 edition
> > instead,
> 
> Please find a revised patch below; hopefully Gunnar will renew his
> second, and perhaps you'll second too, Simon.

Seconded (full text of what I'm seconding below).

> > or perhaps replacing SUSv3 with POSIX and clarifying that we use POSIX
> > to refer to the latest version of the POSIX.1 standard.
> 
> This is an interesting suggestion.  So far as I can see the only
> advantage to this is that we don't need Policy bugs to bump the version
> of the standard we target.  That does not strike me as a large enough
> advantage to justify giving up explicit control over the version of the
> standard that applies to Debian -- do you see other advantages?

(Note that I have seconded the version that says POSIX.1-2017; I don't
want the perfect to be the enemy of the good here.)

Having explicit control over the version of the standard we target
doesn't seem to have brought us any particular benefits, and it has
taken us 9½ years (SUSv4 was published in December 2008) to update
Policy from SUSv3 to SUSv4. I would hope that the upstream and Debian
maintainers of the shells and command-line utilities that we ship already
consider unexplained divergence from POSIX to be a (possibly minor) bug,
and they are almost certainly thinking of the latest version of POSIX
rather than the semi-arbitrary version mentioned in Debian Policy :-)

(I say "unexplained divergence" because some GNU stuff is deliberately
non-POSIX, in areas where its maintainers can justify that decision.)

I find it difficult to imagine a future in which the maintainers of POSIX
make changes that Debian doesn't want to follow, and in such a future
we would have the escape route of either writing exceptions into Policy,
or ceasing to track new POSIX versions at that time and "freezing" at
the last "good" version of POSIX, analogous to what we'd do if the
maintainers of an important upstream package like Linux or glibc went
in directions we strongly objected to.

But perhaps explicit is better than implicit, and hopefully next time
we have a change like this it'll take significantly less than a decade.
If people like my proposed patches for FHS 3.0 (#787816) it might take
a mere 3 years to adopt FHS 3.0...

> Patch:
> 
> > diff --git a/policy/ch-files.rst b/policy/ch-files.rst
> > index 90ae58a..f31a3b4 100644
> > --- a/policy/ch-files.rst
> > +++ b/policy/ch-files.rst
> > @@ -203,9 +203,9 @@ may instead be easier to check the exit status of 
> > commands directly. See
> >  Every script should use ``set -e`` or check the exit status of *every*
> >  command.
> >  
> > -Scripts may assume that ``/bin/sh`` implements the SUSv3 Shell Command
> > +Scripts may assume that ``/bin/sh`` implements the POSIX.1-2017 Shell 
> > Command
> >  Language  [#]_ plus the following additional features not mandated by
> > -SUSv3.. [#]_
> > +POSIX.1-2017.. [#]_
> >  
> >  -  ``echo -n``, if implemented as a shell built-in, must not generate a
> > newline.
> > @@ -238,13 +238,13 @@ SUSv3.. [#]_
> > which are the same as for ``kill`` above, 13 (SIGPIPE) must be
> > allowed.
> >  
> > -If a shell script requires non-SUSv3 features from the shell interpreter
> > +If a shell script requires non-POSIX.1-2017 features from the shell 
> > interpreter
> >  other than those listed above, the appropriate shell must be specified
> >  in the first line of the script (e.g., ``#!/bin/bash``) and the package
> >  must depend on the package providing the shell (unless the shell package
> >  is marked "Essential", as in the case of ``bash``).
> >  
> > -You may wish to restrict your script to SUSv3 features plus the above
> > +You may wish to restrict your script to POSIX.1-2017 features plus the 
> > above
> >  set when possible so that it may use ``/bin/sh`` as its interpreter.
> >  Checking your script with ``checkbashisms`` from the devscripts package
> >  or running your script with an alternate shell such as ``posh`` may help
> > @@ -762,10 +762,10 @@ restricted to ASCII when it is possible to do so.
> > complicated and difficult to manage.
> >  
> >  .. [#]
> > -   Single UNIX Specification, version 3, which is also IEEE 1003.1-2004
> > -   (POSIX), and is available on the World Wide Web from `The Open
> > -   Group <http://www.unix.org/version3/online.html>`_ after free
> > -   registration.
> > +   The Open Group Base Specifications Issue 7, 2018 Edition, 

Bug#787816: Replace FHS 2.3 by FHS 3.0 in the Policy.

[Simon McVittie]

On Fri, 15 Jun 2018 at 14:37:04 +0200, Bill Allombert wrote:
> There are already 28 /usr/lib/TUPLE/*/bin directories in unstable.
> There are probably other directories with binaries not named bin.
> 
> They are candidates for being moved to /usr/libexec, but they should
> probably go to /usr/libexec/TUPLE at least to avoid file conflicts.

Packages are free to continue to use /usr/lib/TUPLE/PACKAGE/bin if they
want to, even after /usr/libexec exists, and I think that should be
best-practice for any packages that would otherwise collide.

FHS 3.0 explicitly says that it's OK for packages to install helper
binaries into either /usr/lib{,64} or /usr/libexec, although they should
choose one or the other, not both; so we are not making these packages
immediately buggy by adopting FHS 3.0. One of the design principles
for FHS 3.0 was that most FHS 2.3 systems should be obviously FHS
3.0 compliant without changes. (The only counter-example I'm aware of
is /usr/bin/mh/, for which I added an exemption in my proposed patch
for now.)

I would anticipate that the way we move to /usr/libexec would be:

* maintainers who want it explicitly use
  --libexecdir='${exec_prefix}/libexec' (some of these maintainers
  are currently using --libexecdir='${libdir}', as I did in src:ostree,
  for the reasons outlined in #859724)

* some future debhelper compat level (perhaps 12 or 13) changes the
  dh_auto_configure defaults from --libexecdir='${libdir}' to the
  Autoconf upstream default --libexecdir='${exec_prefix}/libexec'
  (and maybe the same for non-Autotools build systems like CMake)

* when maintainers change their packages to that compat level, in
  addition to adjusting their debian/*.install files, they are expected
  to check that they aren't introducing multiarch non-coinstallability
  in any Multi-Arch: same packages, and if necessary specify
  --libexecdir='${libdir}' explicitly (I expect this to be rare in
  practice)

* packages that install their helper binaries to ${libdir} or a
  subdirectory are unaffected, and will continue to install them in
  /usr/lib or a subdirectory indefinitely

Expanding on that last point, there are two ways an upstream can
install libexec-style helper binaries with Automake: they can put
them in ${libdir} (or more likely ${pkglibdir}) or they can put them
in ${(pkg)libexecdir}.  For the former, the binaries would remain in
/usr/lib/TUPLE by default and there's no multiarch collision.

For the latter, if there was going to be a collision, then the maintainer
would have to deal with this at the time that they bump debhelper compat
level to opt-in to the changed behaviour; but I don't think this problem
case will happen much in practice, because the same upstream package would
already have been troublesome to install on Red-Hat-like biarch systems
where ${libdir} is /usr/lib{,64} but ${libexecdir} is always /usr/libexec,
so any affected upstream projects have probably already been fixed by
users of Red-Hat-like distros.

When switching from libexecdir=${libdir} to
libexecdir=${exec_prefix}/libexec, maintainers already need to check
that their package isn't breaking the expectations of other packages
(or older versions of their library that might still be in RAM), just
like they did at the transition from libexecdir=${libdir}/${DEB_SOURCE}
to libexecdir=${libdir} when moving from debhelper compat level 8 to 9
(or from cdbs to dh 9 for packages that used to use cdbs).

smcv

Bug#787816: Replace FHS 2.3 by FHS 3.0 in the Policy.

[Simon McVittie]

On Fri, 05 Jun 2015 at 20:43:10 +0900, Charles Plessy wrote:
> Le Thu, Jun 04, 2015 at 07:09:00PM -0700, Russ Allbery a écrit :
> > I have not looked at this at all, but this list should be aware that it
> > exists.
> 
> > The LSB workgroup is happy to announce the release of FHS 3.0.
> ... 
> > Release notes can be found here:
> > 
> > https://wiki.linuxfoundation.org/en/FHSReleaseNotes30

Unfortunately those release notes have vanished, but a copy can be found at:
https://web.archive.org/web/20160624022300/wiki.linuxfoundation.org/en/FHSReleaseNotes30

I've prepared patches for this, which are available here:
https://salsa.debian.org/smcv/policy/merge_requests/1/diffs

I've attached them here, except for the patch that replaces the bundled
copy of FHS v2.3 with a bundled copy of FHS v3.0, which is inconveniently
large. I think patch 0002 is the only normative one?

The only thing I found that would make Debian non-compliant is the
fact that the /usr/bin/mh/ directory used to be allowed, but is not
allowed any more. For the moment I think this should be documented as a
departure from FHS v3.0, and if anyone cares enough about removing that
exception, it should be discussed with the maintainers of nmh (which ships
/usr/bin/mh/) and mailutils-mh (which is technically non-Policy-compliant,
although does comply with the spirit of the policy) on a separate bug.

I got my copy of the FHS source code from
http://refspecs.linuxfoundation.org/FHS_3.0/ (which contains source and
compiled copies), not from
http://bzr.linuxfoundation.org/loggerhead/lsb/devel/fhs-spec/changes
(which I only found later), and I didn't include the bundled copy of
docbook-xsl or the outdated FHS in draft.sgml. I used a directory
instead of a tarball, for better transparency.

> By the way, I wonder if the debian-policy package is the best place for
> shipping a copy of the FHS.

Probably not, but let's not delay its adoption by another 3 years while
we paint that particular bike shed :-)

smcv
>From 8867b0b88d311739fd360f1f6dc945406c39ed54 Mon Sep 17 00:00:00 2001
From: Simon McVittie 
Date: Thu, 14 Jun 2018 11:43:04 +0100
Subject: [PATCH 2/3] Update to FHS v3.0

Notable changes that this causes:

* A GNU-style /usr/libexec is now allowed.

Notable non-changes:

* /usr/games, /usr/share/games and /usr/lib/games were optional in
  FHS 2.3, and are still optional. It is up to us whether we want
  to keep using those directories.

Drop our special exception for /run, and replace it with a requirement
that the pairs (/run, /var/run) and (/run/lock, /var/lock) are made
equivalent via symlinks or bind mounts. The FHS does not mandate this
(although I think it should) and some misguided distributions make
/run and /var/run non-equivalent, but they have always been equivalent
on Debian.

Drop our special exception for /sys, which is now standardized in the
FHS.

Add a special exception for /usr/bin/mh/ for now, restoring the FHS
2.3 situation (dropping this might be a good idea, but should be
disussed with the nmh and mailutils-mh maintainers if desired).

Signed-off-by: Simon McVittie 
Closes: #787816
---
 policy/ch-opersys.rst | 29 ++---
 1 file changed, 10 insertions(+), 19 deletions(-)

diff --git a/policy/ch-opersys.rst b/policy/ch-opersys.rst
index 7d85c00..b18e2c2 100644
--- a/policy/ch-opersys.rst
+++ b/policy/ch-opersys.rst
@@ -12,7 +12,7 @@ File System Structure
 ~
 
 The location of all files and directories must comply with the
-Filesystem Hierarchy Standard (FHS), version 2.3, with the exceptions
+Filesystem Hierarchy Standard (FHS), version 3.0, with the exceptions
 noted below, and except where doing so would violate other terms of
 Debian Policy. The following exceptions to the FHS apply:
 
@@ -76,25 +76,20 @@ Debian Policy. The following exceptions to the FHS apply:
 ``/etc``, or at least are symlinked there, is relaxed to a
 recommendation.
 
-8.  The additional directory ``/run`` in the root file system is
-allowed. ``/run`` replaces ``/var/run``, and the subdirectory
-``/run/lock`` replaces ``/var/lock``, with the ``/var`` directories
-replaced by symlinks for backwards compatibility. ``/run`` and
-``/run/lock`` must follow all of the requirements in the FHS for
-``/var/run`` and ``/var/lock``, respectively, such as file naming
-conventions, file format requirements, or the requirement that files
-be cleared during the boot process. Files and directories residing
-in ``/run`` should be stored on a temporary file system.
+8.  The ``/var/run`` and ``/var/lock`` directories are required to be
+made equivalent to ``/run`` and ``/run/lock`` respectively, for
+example using symbolic links or bind-mounts.
 
-9.  The ``/sys`` directory in the root filesystem is additionally
-allowed.  [#]_
+9.  The ``/var/www`` directory is additionally allowed.
 
-10. The 

Bug#567033: Decide if we should continue recommending /usr/games

[Simon McVittie]

Control: unmerge 567033

On Fri, 11 Aug 2017 at 07:07:49 -0700, Sean Whitton wrote:
> The latest version of the FHS does not have /usr/games, so merging this
> with the bug about updating our FHS version.

Removing the games directories was considered in

and , but there
was no consensus. The FHS 3.0 as published by the Linux Foundation says
essentially the same things games as FHS 2.3:

> [/usr/]games  Games and educational binaries (optional)
> ...
> [/usr/share/]gamesStatic data files for /usr/games (optional)
> ...
> Similarly, a /usr/lib/games hierarchy may be used in addition to the
> /usr/share/games hierarchy if the distributor wishes to place some game
> data there.

(/usr/local/games is technically also mandatory, although I'm sure
that's an oversight and it was meant to be optional...)

Debian can choose to put games in the /.../games directories, or in the
standard directories /usr/bin, /usr/share etc., or any mixture of our
choice, orthogonal to whether/when we move to FHS 3.0.

smcv

Bug#864615: please update version of posix standard for scripts (section 10.4)

[Simon McVittie]

On Sat, 14 Oct 2017 at 15:28:04 -0700, Sean Whitton wrote:
> On Sat, Oct 14 2017, Adam D. Barratt wrote:
> > The 2016 edition is Technical Corrigendum 2. I'm not sure that it's
> > conventional to use versioning such as 4.2 in such cases, however. I'd
> > expect it to be referred to as SUSv4, SUSv4TC2, or SUSv4 2016 edition;
> > the latter seems to be more common.

http://pubs.opengroup.org/onlinepubs/9699919799/ has now been replaced
(see #900882, for which I'm preparing a patch) with POSIX.1-2017, which
is variously labelled as:

* POSIX.1-2017
* IEEE Std 1003.1-2017
* The Open Group Technical Standard Base Specifications, Issue 7
* The Open Group Base Specifications Issue 7, 2018 Edition

and no longer has visible "Single Unix Specification" branding at all.
However, the downloadable tarballs at
http://pubs.opengroup.org/onlinepubs/9699919799/download/
have basename susv4-2018. According to Wikipedia it "incorporates Singles
UNIX Specification TC1 and TC2, and is technically identical to the
2016 edition."

I'd suggest replacing SUSv3 with POSIX.1-2017 or SUSv4 2018 edition
instead, or perhaps replacing SUSv3 with POSIX and clarifying that we
use POSIX to refer to the latest version of the POSIX.1 standard.

smcv

Bug#813471: network access to the loopback device should be allowed

[Simon McVittie]

On Thu, 10 May 2018 at 17:51:19 +0100, Ian Jackson wrote:
> Simon McVittie writes ("Bug#813471: network access to the loopback device 
> should be allowed"):
> > Is it legitimate for a package
> > to resolve the reserved name "localhost" during build, and assume that
> > it will get 127.0.0.1 and/or ::1 back?
> 
> Yes, assuming that it uses gethostbyname() from the libc.

When you say gethostbyname() I hope you're using that as shorthand
for anything that goes through the libc nsswitch mechanism, like
gethostbyname(), getaddrinfo(), `getent hosts` and so on? (gethostbyname()
is deprecated in favour of getaddrinfo(), because it isn't thread-safe
and can't return a mixture of IPv4 and IPv6 addresses.)

> > - yes, but only if it Build-Depends on libnss-myhostname and/or netbase
> 
> I don't know what these things are but I think an system being used
> for builds ought to contain an /etc/hosts giving an IP address for
> `localhost' and enough libc to read it via gethostbyname().

libnss-myhostname is an nsswitch hosts plugin (a backend for
gethostbyname(), getaddrinfo() and friends) which programmatically
resolves localhost, localhost.localdomain and whatever is returned by
gethostname() to plausible IP[v4 and v6] addresses, even if /etc/hosts
is missing or empty. The idea is that it makes the right thing happen
on unconfigured systems, while gracefully staying out of the way if
overridden by configuration (it is canonically configured as the lowest
priority backend in /etc/nsswitch.conf, whereas /etc/hosts is part of
libnss_files which is highest-priority.)

netbase is the Priority: important package that owns files like
/etc/services, and it creates a minimal /etc/hosts (localhost and related
names) in its postinst. For systems installed with d-i, I think d-i
is also involved in the creation of /etc/hosts (it adds the configured
hostname as in the section of hostname(1) that you cited), but not all
systems were installed with d-i, particularly minimal chroots.

> > Similarly, can it assume it can resolve $(hostname) always, or only if
> > it B-D on libnss-myhostname, or never?
> 
> I think `resolve $(hostname)' is a bit ambiguous.

Sorry, what I meant is: take the result of gethostname(), and pass it to
libc resolver calls like gethostbyname() or getaddrinfo().

> > At the moment, schroot/sbuild is very likely to make both localhost and
> > $(hostname) resolvable (/etc/hosts from the host system is copied into
> > the chroot, and that file is not strictly guaranteed to make localhost or
> > $(hostname) resolvable but probably does), but pbuilder with its default
> > USENETWORK=no configuration does not necessarily have a hosts file or a
> > working resolv.conf. dbus currently FTBFS on reproducible-builds (#897662)
> > because one of its automated tests assumes localhost is resolvable.
> 
> I think this is a bug in pbuiler.

Correction, schroot/sbuild *might* copy in /etc/hosts, but it might not:
as well as /etc/hosts being a configuration file on the host, the list
of files to copy is configurable in each schroot "profile". By default,
the sbuild profile only copies in /etc/resolv.conf, whereas the buildd
profile additionally copies /etc/hosts and /etc/networks. I think official
Debian buildds use the buildd profile, while developers are encouraged
to use the sbuild profile.

This seems increasingly like a trap, with no particular guarantees
currently given to software running in the chroot...

> > I started to implement a feature in pbuilder to make it create a trivial
> > /etc/hosts that can resolve localhost and friends (the same as is created
> > by netbase.postinst) whenever it locks down resolv.conf, but then realised
> > that netbase isn't Essential, so it isn't completely clear whether the
> > resolvability of localhost is part of the basic "API" of a Debian system.
> 
> I think it must be part of the basic API for a build environment.  I'm
> a bit surprised that netbase isn't build-essential.  Certainly IMO an
> /etc/hosts with the entries you describe above should be implied by
> build-essential, one way or another.

The rest of netbase is fairly niche (I suspect few packages need
/etc/protocols or /etc/rpc, and most of the packages that depend on it
do so because they want /etc/services), but it happens to be one of the
few ways to guarantee that you have some sort of /etc/hosts.

smcv

Bug#813471: network access to the loopback device should be allowed

[Simon McVittie]

On Wed, 04 Oct 2017 at 14:09:53 +0200, Bill Allombert wrote:
> On Tue, Oct 03, 2017 at 01:00:14PM -0500, Gunnar Wolf wrote:
> > Jérémy Lal dijo [Tue, Oct 03, 2017 at 07:46:43PM +0200]:
> > > It might be a good idea to make policy more explicit about downloads 
> > > during
> > > build.
> > 
> > I completely agree. This led me to look at #813471 ("network access to
> > the loopback device should be allowed"), and... Well, it seems to set
> > the stage to the issue we are tackling now: #813471 is opened as a
> > reaction against #770016 ("Clarify network access for building
> > packages in main").
> 
> I want to clarify that I never intended the prohibition of network access
> to apply to the loopback device, and I expect the other seconders to
> think the same, given the rationale for the change.
> 
> To my mind, using the loop backdevice is not performing network access.

An interesting question related to this: Is it legitimate for a package
to resolve the reserved name "localhost" during build, and assume that
it will get 127.0.0.1 and/or ::1 back? Possible answers include:

- yes, always
- yes, but only if it Build-Depends on libnss-myhostname and/or netbase
- no, and it must not attempt to resolve that name because that might
  be network access in corner cases

Similarly, can it assume it can resolve $(hostname) always, or only if
it B-D on libnss-myhostname, or never?

At the moment, schroot/sbuild is very likely to make both localhost and
$(hostname) resolvable (/etc/hosts from the host system is copied into
the chroot, and that file is not strictly guaranteed to make localhost or
$(hostname) resolvable but probably does), but pbuilder with its default
USENETWORK=no configuration does not necessarily have a hosts file or a
working resolv.conf. dbus currently FTBFS on reproducible-builds (#897662)
because one of its automated tests assumes localhost is resolvable.

I started to implement a feature in pbuilder to make it create a trivial
/etc/hosts that can resolve localhost and friends (the same as is created
by netbase.postinst) whenever it locks down resolv.conf, but then realised
that netbase isn't Essential, so it isn't completely clear whether the
resolvability of localhost is part of the basic "API" of a Debian system.

smcv

Bug#850156: Please firmly deprecate vendor-specific series files [and 1 more messages]

[Simon McVittie]

On Wed, 18 Apr 2018 at 21:11:11 -0700, Steve Langasek wrote:
> The examples given are for series.ubuntu, which is certainly the case I've
> seen in the wild.  Ubuntu, as a project, did not ask for this.  As an Ubuntu
> developer, it has never benefitted me.  I have only ever seen it used by
> Debian developers who for some reason considered it useful to "merge" an
> Ubuntu delta without actually merging it.  This is a development
> antipattern; the source package that is then automatically synced to Ubuntu
> from Debian is then neither under the guidance of an Ubuntu developer caring
> for its quality, nor benefitting from the testing of the package in Debian,
> because the source is no longer the same.

I tried asking codesearch how large this problem is:
https://codesearch.debian.net/search?q=%28diff%7Cpatch%29+path%3Adebian%2Fpatches%2F%28.*%29%5C.series&perpkg=1
and the very first hit (at least for me) happens to be an example of it
being a real, practical problem (#894175). In ddccontrol/0.4.3-1:

% diff -u debian/patches/series debian/patches/ubuntu.series
--- debian/patches/series   2018-01-26 09:48:53.0 +
+++ debian/patches/ubuntu.series2018-01-26 09:48:53.0 +
@@ -0,0 +1 @@
+0001-ubuntu_desktop_file_use_gksu.patch
% cat debian/patches/0001-ubuntu_desktop_file_use_gksu.patch 
Description:
 In Ubuntu, Use gksu in gddccontrol.desktop to get access to the DDC/CI device.
...
-Exec=gddccontrol
+Exec=gksu gddccontrol

... but the world has (thankfully) moved on from using gksu, and
https://launchpad.net/ubuntu/+source/gksu says gksu is no longer present
in Ubuntu bionic. So this is Ubuntu divergence that, because it was not
checked by an Ubuntu maintainer, is no longer valid (and in fact an
Ubuntu maintainer has had to *add* divergence by reverting this change).

smcv

Bug#850156: Please firmly deprecate vendor-specific series files

[Simon McVittie]

On Thu, 19 Apr 2018 at 08:00:28 +, Mike Gabriel wrote:
> One example, where the vendor.series file is really helpful is:
> https://anonscm.debian.org/cgit/pkg-mate/mate-terminal.git/tree/debian/patches/2001_fix-find-next-previous.patch

That one-line change could easily be guarded by
#ifdef UBUNTU_PATCHED_LIBVTE or something, with
CPPFLAGS += -DUBUNTU_PATCHED_LIBVTE in debian/rules whenever dpkg-vendor
says you're compiling for Ubuntu.

Alternatively, looking at the Ubuntu vte patch, this pseudo-patch should
work either way:

-can_search = vte_terminal_search_get_regex (VTE_TERMINAL (screen)) != NULL;
+can_search = (vte_terminal_search_get_regex (VTE_TERMINAL (screen)) != NULL ||
+  vte_terminal_search_get_gregex (VTE_TERMINAL (screen)) != NULL);

... and indeed the Ubuntu side of the #ifdef should really be something
more like that anyway, because otherwise, if Ubuntu add pcre2 to main
and stop patching libvte, mate-terminal search on Ubuntu will stop working
again (this time for the opposite reason).

smcv

Bug#850156: Please firmly deprecate vendor-specific series files

[Simon McVittie]

On Wed, 18 Apr 2018 at 14:36:14 +0200, Mike Gabriel wrote:
> On Wed, 4 Jan 2017 13:41:53 + Ian Jackson
>  wrote:
> > But [vendor.series] is quite wrong, because it means that the same
> > source package has different "contents" on different computers.
> 
> The source package is always the same. The bin:pkgs' contents is different.
> But that is different anyway, even if built against different versions of
> Debian. You have to rebuild the package for your system anyway, so what is
> your point here? (addressing Ian with this question)

This isn't as clear-cut as you might think. There are two frequently-seen
possibilities for what you think the contents of a 3.0 (quilt) source
package are:

- the upstream source, with debian/ deleted if it exists, the
  contents of debian.tar.* unpacked into debian/, and patches from
  debian/patches/ applied
  (the "patches-applied" state, as produced by dpkg-source -x and
  stored in git by git-dpm and dgit)

- the upstream source, with debian/ deleted if it exists, and the
  contents of debian.tar.* unpacked into debian/
  (the "patches-unapplied" state, as stored in git by gbp import-dsc
  and gbp pq)

Based on his work on dgit, I believe Ian considers the canonical contents
of the source package to be the patches-applied state. Developers who
agree with this point of view would say that applying patches is part of
unpacking the source package, and any source package with vendor-specific
series gets different contents depending where it was unpacked, which
seems strange - steps described as "unpacking" are normally deterministic.

Conversely, developers who consider the canonical contents of the
source package to be the patches-unapplied state would say that applying
patches is more like part of the process of building a binary package
(converting a source package into a binary package), which would imply
that a source package with vendor-specific series always has the same
contents (but those contents result in different vendors building different
patched source code).

I prefer to manage the patches-unapplied state in version control,
but despite that I'm not sure that I can disagree with Ian about the
patches-applied state being the "contents" of the package.

smcv

Bug#881431: proposed wording

[Simon McVittie]

On Wed, 04 Apr 2018 at 11:47:09 -0700, Sean Whitton wrote:
> > §3.2.2 Uniqueness of version numbers
> >
> > The part of the version number after the epoch must not be reused for
> > a version of the package with different contents once the package has
> > been accepted into the archive, even if the version of the package
> > previously using that part of the version number is no longer present
> > in any archive suites.
> >
> > This uniqueness requirement applies to the version numbers of source
> > packages and of binary packages, even if the source package producing
> > a given binary package changes.  Thus the version numbers which a
> > binary package must not reuse includes the version numbers of any
> > versions of the binary package ever accepted into the archive, under
> > any source package.
> >
> > Additionally, for non-native packages, the upstream version must not
> > be reused for different upstream source code, so that for each source
> > package name and upstream version number there exists exactly one
> > original source archive contents [reference to defintiion of that].
> >
> > The reason for these restrictions is as follows.  Epochs are not
> > included in the names of the files that compose source packages, or in
> > the filenames of binary packages, so reusing a version number, even if
> > the epoch differs, results in identically named files with different
> > contents.  This can cause various problems.
> >
> > If you find yourself wanting to reuse the part of a version number
> > after the epoch, you can just increment the Debian revision, which
> > doesn't need to start at 1 or be consecutive.

Seconded, thanks.

Bug#881431: proposed wording

[Simon McVittie]

On Thu, 29 Mar 2018 at 08:12:15 -0700, Sean Whitton wrote:
> Seeking seconds:
> 
> > §3.2.2 Uniqueness of version numbers

This has lost the part of Adam's wording where he explicitly said that
this applies to all three of these namespaces:

* (source package name, source version without epoch)
* (binary package name, binary version without epoch)
* (non-native source package name, upstream version without epoch or
  Debian revision)

and I think that's valuable information.

Perhaps adding a paragraph like this would address that?

"""
This uniqueness requirement applies to source package names and versions,
and to binary package names and versions (even if the binary package was
previously produced by a different source package). Additionally, for
non-native packages, the upstream version without the epoch (excluding
the Debian revision) must not be reused for different content, so that
each (source package name, upstream version without epoch) pair refers
to the same original source archive (see _Source packages as archives_).
"""

(This could be interpreted as either allowing or forbidding replacement of
foo_1.2.3.orig.tar.gz with foo_1.2.3.orig.tar.xz if they uncompress
to the same foo_1.2.3.orig.tar, or even to an equivalent but not
byte-identical foo_1.2.3.orig.tar - I'd be tempted to say that's silly
and should not be allowed either, but I'm not sure how to make that more
explicit.)

> > If you find yourself wanting to reuse the part of a version number
> > after the epoch, you can just bump the Debian revision, which doesn't
> > need to start at 1 or be consecutive.

"increase the Debian revision" might be clearer and is certainly more
formal.

smcv

Bug#747320: Mandate "type" in /bin/sh

[Simon McVittie]

Control: retitle -1 Mandate "type" and/or "command -v" in /bin/sh

On Sat, 07 Jun 2014 at 21:52:34 +0200, Jakub Wilk wrote:
> * Ian Jackson , 2014-05-07, 14:32:
> > It seems to me that given that dash and bash both provide `type', and
> > the underlying functionality necessarily exists in the shell, it would
> > be better to mandate that the shell expose it.
> 
> Alternatively, we could bless "command -v", which is a nicer interface IMO,
> and it's also supported by bash, dash and mksh.

command -v appears to be supported by bash, dash, busybox sh, zsh (not
otherwise a fully-POSIX shell), ksh, mksh and yash. It is not supported
by posh or sash. Are there other shells that should be considered to be
supported as /bin/sh?

command -v is also in the Single Unix Specification, version 3 (aka
POSIX:2001), as part of the User Portability Utilities section (which
Policy doesn't currently say scripts may assume, but perhaps it should).

It's also in SUSv4 (aka POSIX:2008) without any such qualifiers, so if
#864615 is resolved with the suggested patch, command -v will implicitly
be mandated by Policy (which would incidentally make sash a
non-Policy-compliant choice for /bin/sh).

For completeness, type is supported by all of those shells except posh,
and is standardized in SUSv3 and SUSv4 as an optional XSI extension.
Policy doesn't mandate XSI extensions in general, although a couple of
specific XSI extensions to the kill and trap commands are called out
as mandatory.

smcv

Re: Javascript team policy and rejection of node-three binary package [and 1 more messages]

[Simon McVittie]

On Sun, 11 Mar 2018 at 14:48:40 +0530, Pirate Praveen wrote:
> On വെള്ളി 09 മാർച്ച് 2018 08:39 വൈകു, Simon McVittie wrote:
> > And for executables, perhaps something like this:
...
> > * should not be named node-* without a suffix like -bin or -tools (?)
> 
> I don't think there is any particular benefit in this restriction.
> Having a uniform was to map node module name -> package name without
> having extra effort to check if it is an executable or library is
> helpful in automation (especially because we are talking about a large
> number of modules here). Manually finding if a module is already
> packaged vs automatically finding.

The reason I suggested that restriction was to avoid having contradictory
requirements: if node-foo is the naming convention for the module
that lets nodejs users require('foo'), and node-foo is also the naming
convention for a nodejs executable that upstream refer to as foo where
foo is too generic for Debian, then we might get a collision between
the requirements for libraries and the requirements for executables.

But perhaps I'm being overly paranoid there, because the main extra
requirement for executables with nodejs as an interpreter is "must depend
on nodejs", and for a package named node-* that's hardly a burden.

smcv