Bug#1079967: should policy and dpkg agree on allowed versions?

[Guillem Jover]

Control: reassign -1 debian-policy

Hi!

On Wed, 2024-08-28 at 18:11:23 +0200, Helmut Grohne wrote:
> Package: dpkg-dev,debian-policy
> Severity: wishlist
> X-Debbugs-Cc: po...@debian.org

> Emilio and me noticed that policy and dpkg have subtly different ideas
> of what is a version. While man deb-version says
> 
> | The upstream-version may contain only alphanumerics (“A-Za-z0-9”)
> | and the characters . + - : ~ (full stop, plus, hyphen, colon, tilde)
> | and should start with a digit.
> 
> Debian policy section 5.6.1 says
> 
> | The upstream_version must contain only alphanumerics 6 and the
> | characters . + - ~ (full stop, plus, hyphen, tilde) and should start
> | with a digit. If there is no debian_revision then hyphens are not
> | allowed.
> 
> Technically speaking, it is fine for policy to forbid things that dpkg
> allows. Other distributions based on dpkg may use a different policy and
> allow using multiple colons. Still is is an odd aspect and may cause
> confusion. Is this difference intentional? If yes, would it make sense
> to add a footnote to policy hinting that it is more restrictive than
> dpkg? I also checked packages in unstable and found no packages with a
> version containing two colons (i.e. all packages are policy-compliant in
> this regard).

Yes, this is the result of both #792853 and #971023. Reassigning to
Debian Policy in case someone wants to clarify it there.

Thanks,
Guillem

Bug#1057238: debian-policy: Take dpkg-build-api into account for Rules-Requires-Root

[Guillem Jover]

Hi!

On Sun, 2024-08-25 at 10:04:33 +0800, Sean Whitton wrote:
> On Thu 15 Aug 2024 at 11:07am +02, Niels Thykier wrote:
> > How about adding
> >
> > """
> > The default for Rules-Requires-Root depends on the dpkg-build-api level (the
> > dpkg-build-api is defined in "man 7 dpkg-build-api"). At level 0 (or when 
> > not
> > declared), Rules-Requires-Root defaults to binary-targets. In later 
> > versions,
> > it defaults to no.
> > """
> >
> > in 5.6.31 and then removing the `(default)` next `binary-targets` in 5.6.31?
> >
> > I think that would do wonders.
> 
> Based on this I came up with the following, which I think is slightly
> cleaner than adding a whole paragraph.  Would you mind checking it?


> -- >8 --
> Subject: [PATCH] Update specification of the default value for
>  Rules-Requires-Root
> 
> ---
>  debian/changelog|  2 ++
>  policy/ch-controlfields.rst | 17 ++---
>  2 files changed, 12 insertions(+), 7 deletions(-)
> 
> diff --git a/debian/changelog b/debian/changelog
> index c1fc3c3..2d5d204 100644
> --- a/debian/changelog
> +++ b/debian/changelog
> @@ -4,6 +4,8 @@ debian-policy (4.7.0.1) UNRELEASED; urgency=medium
>  Thanks to Paul Gevers for the report.
>* Fix markup problem in 3.9 (Closes: #1074040).
>  Thanks to Ke Zhang for the report & patch.
> +  * Update specification of the default value for Rules-Requires-Root.
> +See #1057238.
> 
>   -- Sean Whitton   Sun, 07 Apr 2024 15:34:08 +0800
> 
> diff --git a/policy/ch-controlfields.rst b/policy/ch-controlfields.rst
> index ecccbe9..530755a 100644
> --- a/policy/ch-controlfields.rst
> +++ b/policy/ch-controlfields.rst
> @@ -1204,14 +1204,14 @@ script: debian/rules `.
> 
>  The field can consist of exactly one of the following three items:
> 
> - - ``no``: Declares that neither root nor fakeroot is required.
> -   Package builders (e.g. dpkg-buildpackage) may choose to invoke any
> -   target in ``debian/rules`` with an unprivileged user.
> + - ``no`` (default at dpkg-build-api level >=1): Declares that neither root
> +   nor fakeroot is required.  Package builders (e.g. dpkg-buildpackage) may
> +   choose to invoke any target in ``debian/rules`` with an unprivileged user.
> 
> - - ``binary-targets`` (default): Declares that the package will need
> -   the root (or fakeroot) when either of the ``binary``,
> -   ``binary-arch`` or ``binary-indep`` targets are called.  This is
> -   how every tool behaved before this field was defined.
> + - ``binary-targets`` (default at dpkg-build-api level 0): Declares that the
> +   package will need the root (or fakeroot) when either of the ``binary``,
> +   ``binary-arch`` or ``binary-indep`` targets are called.  This is how every
> +   tool behaved before this field was defined.
> 
>   - A space separated list of keywords described below.  These keywords
> must always contain a forward slash, which sets them apart from the
> @@ -1222,6 +1222,9 @@ The field can consist of exactly one of the following 
> three items:
> to ``binary-targets``, and both the builder and the package's
> ``debian/rules`` script must downgrade accordingly (see below).
> 
> +The default depends on the dpkg-build-api level (see dpkg-build-api(7)).
> +If the dpkg-build-api level is undeclared, it is equivalent to level 0.
> +
>  If the package builder supports the ``Rules-Requires-Root`` field and
>  wants to enable the feature, then it must set the environment variable
>  ``DEB_RULES_REQUIRES_ROOT`` when invoking the package building script

Ah, indeed, this looks better to me too!

Thanks,
Guillem

Bug#1069256: debian-policy: clarify requirement for use of Static-Built-Using

[Guillem Jover]

Hi!

On Sat, 2024-04-27 at 17:40:49 +0800, Maytham Alsudany wrote:
> Thanks for your input and suggestions. I've attached an updated patch with
> several changes, including improving making the description of the field more
> specific, adding another example that is not Go/Rust related, and improving 
> the
> Rust example to show the simultaneous use of Static-Built-Using and 
> Built-Using.

Thanks for the update!

> From 06cf64756ff1ee66d845e86dcf5c9dafd4a84b39 Mon Sep 17 00:00:00 2001
> From: Maytham Alsudany 
> Date: Thu, 18 Apr 2024 22:29:01 +0300
> Subject: [PATCH] Require use of Static-Built-Using to declare
>  statically-linked libraries
> 
> ---
>  policy/ch-relationships.rst | 60 +++--
>  1 file changed, 58 insertions(+), 2 deletions(-)
> 
> diff --git a/policy/ch-relationships.rst b/policy/ch-relationships.rst
> index fb9dae8..636e2a5 100644
> --- a/policy/ch-relationships.rst
> +++ b/policy/ch-relationships.rst

> @@ -710,6 +713,59 @@ requirements to retain the referenced source packages.  
> It should not
>  be added solely as a way to locate packages that need to be rebuilt
>  against newer versions of their build dependencies.
>  
> +``Static-Built-Using``
> +~~
> +
> +This ``Static-Built-Using`` field must list source packages who's

I'm not sure “who” would be idiomatic to refer to packages contents?

> +contents (like source code or data) were incorporated into the binary
> +package during the build, including an "exactly equal" ("=") version
> +relation on the version that was used to build that version of the
> +incorporating binary package.

I find the placement/phrasing after the “, including …” a bit
confusing, because I read it as a continuation from the “were
incorporated into the binary package during build“, instead of
”must list source packages”. It is also perhaps not clear that the
exact version restriction is mandatory.

> +Cases where this field may be used include (but are not limited to)
> +linking against static libraries in other packages, builds for
> +source-centered languages such as Go and Rust, usage of header-only
> +C/C++ libraries and injecting data blobs into code.
> +
> +This is useful to track whether the package might need to be rebuilt
> +when source packages listed here have been updated. This is important
> +to stay ahead of the package failing to build from source (FTBFS) with
> +the updated versions of the listed source packages, or security
> +updates in the listed source packages.
> +
> +Unlike Built-Using, the Debian archive will **not** retain the
> +versions of the source packages listed in the Static-Built-Using
> +field. This means that any package listed in Static-Built-Using who's

Same comment about “who“.

> +license requires its source code to be available must also
> +simultaneously be listed in the Built-Using field.
> +
> +A package that needs domain name suffix data from the publicsuffix
> +binary package would list it in the ``Static-Built-Using`` field like
> +so:

Perhaps reword these or preface with some text to make them more clear
they are just some examples for how the field could be used? I mean, I
guess this is implicit with the “would“, but perhaps making this
explicit is preferable in a document like this?

(I'm also always a bit conflicted with examples that are based on real
current package data, because while this are then extremely clear right
now, they can quickly become obsolete or seem stale or odd after some
time has passed. :/ But I'm not sure what would be an alternative, and
I think this is something for the policy editors to weight in if at all.)

> +
> +::
> +
> +Static-Built-Using: publicsuffix (= 20231001.0357-0.1)
> +
> +A package statically linked with a library from the
> +golang-github-mattn-go-xmpp-dev binary package would have this field
> +in its control file:
> +
> +::
> +
> +Static-Built-Using: golang-github-mattn-go-xmpp (= 0.2.0-1)
> +
> +A package statically linked with the libraries contained in the
> +librust-gtk4-dev and librust-pulsectl-rs-dev binary packages, where
> +the latter is licensed under GPL-3+ (a license that requires full
> +source code to be available), would have these fields in its control
> +file:
> +
> +::
> +
> +Built-Using: rust-pulsectl-rs (= 0.3.2-1+b1)
> +Static-Built-Using: rust-gtk4 (= 0.7.3-3), rust-pulsectl-rs (= 
> 0.3.2-1+b1)
> +
>  .. [#]
> The relations ``<`` and ``>`` were previously allowed, but they were
> confusingly defined to mean earlier/later or equal rather than

Beside these comments, which to me seem more on the editorial side,
I'm now happy with the essence of the text to second it, once it has
seen some wordsmithing. :)

Thanks,
Guillem

Bug#1057238: debian-policy: Take dpkg-build-api into account for Rules-Requires-Root

[Guillem Jover]

Hi!

On Thu, 2024-08-15 at 11:07:43 +0200, Niels Thykier wrote:
> How about adding
> 
> """
> The default for Rules-Requires-Root depends on the dpkg-build-api level (the
> dpkg-build-api is defined in "man 7 dpkg-build-api"). At level 0 (or when
> not declared), Rules-Requires-Root defaults to binary-targets. In later
> versions, it defaults to no.
> """
> 
> in 5.6.31 and then removing the `(default)` next `binary-targets` in 5.6.31?

That would look good to me.

Thanks,
Guillem

Bug#1064454: debian-policy: Restrict deb822 field names more

[Guillem Jover]

eived a bug report
> against debhelper long ago about being able to do path traversal via
> questionable package names. Largely, this was considered a non-issue because
> dpkg in its default configuration would abort and debhelper has a "run
> arbitrary code via the upstream build system" feature anyway.

Do you have a reference on hand to that report? Because if dpkg fails
I'd assume that might be a side effect from something else, and it
might make sense to make sure this is handled explicitly.

Thanks,
Guillem
From 9b284029b29b3f27cc478198f7de73dfed61cf6d Mon Sep 17 00:00:00 2001
From: Guillem Jover 
Date: Fri, 16 Aug 2024 19:59:51 +0200
Subject: [PATCH] Dpkg::Control::HashCore: Restrict field names

FIXME: discuss, update manual pages, extend commit message.
---
 scripts/Dpkg/Control/HashCore.pm |  4 ++--
 scripts/t/Dpkg_Control.t | 28 +++-
 2 files changed, 29 insertions(+), 3 deletions(-)

diff --git a/scripts/Dpkg/Control/HashCore.pm b/scripts/Dpkg/Control/HashCore.pm
index 08f35c57b..cea329cf4 100644
--- a/scripts/Dpkg/Control/HashCore.pm
+++ b/scripts/Dpkg/Control/HashCore.pm
@@ -221,8 +221,8 @@ sub parse {
 my ($name, $value) = split /\s*:\s*/, $_, 2;
 if (defined $name and $name =~ m/^\S+?$/) {
 	$parabody = 1;
-if ($lead eq '-') {
-		$self->parse_error($desc, g_('field cannot start with a hyphen'));
+if ($name !~ m{^[_]*[A-Za-z]+[A-Za-z0-9-]*[A-Za-z0-9]+$}) {
+$self->parse_error($desc, g_('field name %s uses invalid format'), $name);
 	}
 	if (exists $self->{$name}) {
 		unless ($$self->{allow_duplicate}) {
diff --git a/scripts/t/Dpkg_Control.t b/scripts/t/Dpkg_Control.t
index 29311b8c8..8df78c227 100644
--- a/scripts/t/Dpkg_Control.t
+++ b/scripts/t/Dpkg_Control.t
@@ -20,7 +20,7 @@ use Test::More;
 use Test::Dpkg qw(:needs :paths);
 
 BEGIN {
-plan tests => 24;
+plan tests => 35;
 
 use_ok('Dpkg::Control');
 use_ok('Dpkg::Control::Info');
@@ -116,6 +116,32 @@ Architecture: all
 Depends: hello
 ', "Dump of second binary package of $datadir/control-1");
 
+# Check parsing deb822 data.
+my %deb822_bogus = (
+no_separator => 'SomethingNotAField = with a value',
+no_field => ': only value',
+bad_start_hyphen => '-Bad-Field: hyphen-value',
+bad_cont_line => ' continuation line outside of field',
+bad_dup_field => "Duped-Field: value-1\n" .
+ "Duped-Field: value-2\n",
+bad_short_field => 'A: short',
+bad_end_hyphen => 'Field-: value-',
+bad_end_underscore => 'Field-End-Underscore_: value',
+bad_prefix_number => '0field: 0value',
+bad_prefix_special => '$field: $value',
+bad_middle_special => 'Field$name: value$mixed',
+);
+
+while (my ($name, $data) = each %deb822_bogus) {
+open $io, '<', \$data or die "canno open io string\n";
+$c = Dpkg::Control::Hash->new();
+eval {
+$c->parse($io, "parse bogus deb822 data $name");
+};
+ok($@, "failed to parse bogus deb822 data $name");
+close $io;
+}
+
 # Check OpenPGP armored signatures in source control files
 
 my $dsc;
-- 
2.45.2

Re: Why do we have both locales/ and policy/locale/ ?

[Guillem Jover]

Hi!

On Wed, 2024-05-08 at 16:39:16 +0100, Sean Whitton wrote:
> 'make update-po' changes files under locales/.
> Our translators Hideki, Fei Ding and Ke Zhang work under policy/locale/.
> 
> This seems wrong.  As far as I can tell, when the English is updated, we
> are not updating the .po files that are actually being translated.
> 
> Looking at
> 
> % git log --stat -10 --author=henrich debian/4.2.0.1
> 
> shows that around the time when Hideki internationalised Policy, he was
> editing files under both directories.  This suggests that it wasn't that
> Ian, Russ or I accidentally introduced a duplicate set of .po files when
> we were later working on the internationalisation.

When the i18n support was merged I briefly looked into it, but the
sphinx i18n support looked a bit alien along its directory hierarchy,
so stopped looking.

Doing some brief checks now, it seems to confirm that locales/
contains no major translations, while policy/locale/ does:

  for po in policy/locale/ja/LC_MESSAGES/*.po; do
echo "== $po =="
msgfmt --output=/dev/null --statistics $po
  done

vs

  for po in locales/ja/LC_MESSAGES/*.po; do
echo "== $po =="
msgfmt --output=/dev/null --statistics $po
  done

In addition, re-running the above msgfmt with --check, and also running
«i18nspector **/*.po», both emit multiple things that could be fixed,
or improved.

Thanks,
Guillem

Bug#1065643: debian-policy: Refer to «dpkg-buildtree clean» for dpkg generated files

[Guillem Jover]

Hi!

On Thu, 2024-03-28 at 09:58:29 +0800, Sean Whitton wrote:
> On Thu 07 Mar 2024 at 11:22pm +01, Guillem Jover wrote:
> > diff --git a/policy/ch-source.rst b/policy/ch-source.rst
> > index 4307e89..2fb05cd 100644
> > --- a/policy/ch-source.rst
> > +++ b/policy/ch-source.rst
> > @@ -685,7 +685,7 @@ variables are also available.
> >
> >  The ``debian/substvars`` file is usually generated and modified
> >  dynamically by ``debian/rules`` targets, in which case it must be
> > -removed by the ``clean`` target.
> > +removed by the ``clean`` target (for example with ``dpkg-buildtree 
> > clean``).
> >
> >  See :manpage:`deb-substvars(5)` for full details about source variable
> >  substitutions, including the format of ``debian/substvars``.
> > @@ -725,8 +725,9 @@ building packages to record which files are being 
> > generated.
> >
> >  It should not exist in a shipped source package, and so it (and any
> >  backup files or temporary files such as ``files.new``)  [#]_ should be
> > -removed by the ``clean`` target. It may also be wise to ensure a fresh
> > -start by emptying or removing it at the start of the ``binary`` target.
> > +removed by the ``clean`` target (for example with ``dpkg-buildtree 
> > clean``).
> > +It may also be wise to ensure a fresh start by emptying or removing it at 
> > the
> > +start of the ``binary`` target.
> >
> >  When ``dpkg-gencontrol`` is run for a binary package, it adds an entry
> >  to ``debian/files`` for the ``.deb`` file that will be created when
> 
> Instead of "It may also be wise ..." can you use one of the sets of
> magic words from Policy 1.1, please?

This text was already part of policy and the proposed patch did not
really touch it, except for wrapping it into a new line. I think
modifying it feels a bit out-of-scope for this request? But if you
think it's relevant, and the sentence should be improved as part of
this, then I'll try to provide some wording. :)

Thanks,
Guillem

Bug#1069139: developers-reference: out-of-date section "Make transition packages deborphan compliant"

[Guillem Jover]

Hi!

On Wed, 2024-04-17 at 04:24:16 +0200, Vincent Lefevre wrote:
> Package: developers-reference
> Version: 13.5
> Severity: normal

> Now that the deborphan package has been removed from unstable,
> the section "Make transition packages deborphan compliant" in
> "Best Packaging Practices" is out of date and should be updated.
> 
> See https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1065312
> where "apt-mark auto ..." (for autoremove) is suggested as a
> replacement. But with it, putting transition packages to oldlibs
> is even more necessary.

While I fully support properly marking obsolete packages by putting
them in the (unfortunately misnamed :) oldlibs section (well excluding
library-like depended on packages that get dropped as a mater of course).
I wanted to note that I've received some pushback from the archive
maintainers about this being considered unnecessary churn (paraphrasing
from what ISTR). So it would be nice to clarify this with them before
creating and proposing a procedure that might end up generating social
friction.

Thanks,
Guillem

Bug#1069256: debian-policy: clarify requirement for use of Static-Built-Using

[Guillem Jover]

Hi!

On Thu, 2024-04-18 at 23:29:11 +0300, Maytham Alsudany wrote:
> Package: debian-policy
> Version: 4.7.0.0
> Severity: normal
> X-Debbugs-Cc: debian-de...@lists.debian.org

> In early 2022, Guillem added support for a new Static-Built-Using field to
> dpkg, encouraging packagers to use it over Built-Using to specify
> statically-linked dependencies [2]. The commit message states the following:
> 
>   This field mimics the previous Built-Using field semantics, but is
>   specifically intended for shadow dependencies stemming from static builds.
>   
>   In Debian, the Rust and Go teams agreed to use this language agnostic field,
>   instead of one for each of the languages. This means it can be easily
>   supported by dpkg, and can be used by other languages and run-times.
> 
> This was also added to the deb-control(5) manpage, specifically 
> differentiating
> it from Built-Using as a field to be used "for static building purposes (for
> example linking against static libraries, builds for source-centered languages
> such as Go or Rust[...])".

I think this (and the policy proposal) is omitting several important
parts for the intended use for the field, as it was considered on its
addition. Quoting from deb-control(5) for context, which I think
covers the concerns from the mails from Simon and Chris:

   Built-Using: 
   This dependency field lists extra source packages that were used
   during the build of this binary package, for license compliance
   purposes.  This is an indication to the archive maintenance
   software that these extra source packages must be kept whilst this
   binary package is maintained.  This field must be a comma-separated
   list of source package names with strict ‘=’ version relationships
   enclosed within parenthesis.  Note that the archive maintenance
   software is likely to refuse to accept an upload which declares a
   Built-Using relationship which cannot be satisfied within the
   archive.

   Static-Built-Using: 
   This dependency field lists extra source packages that were used
   during the build of this binary package, for static building
   purposes (for example linking against static libraries, builds for
   source-centered languages such as Go or Rust, usage of header-only
   C/C++ libraries, injecting data blobs into code, etc.).  This is
   useful to track whether this package might need to be rebuilt when
   source packages listed here have been updated, for example due to
   security updates.  This field must be a comma-separated list of
   source package names with strict ‘=’ version relationships enclosed
   within parenthesis.

   Supported since dpkg 1.21.3.

> The proposed change is to clarify and formally mandate the requirement to
> declare statically-linked libraries used to build packages containing binaries
> in Static-Built-Using. Attached is a draft patch of the proposed change.

Thanks for starting this!

As mentioned before I think the currently proposed description is too
restrictive and specific to Go and Rust, and it should be expanded. The
example usage seems also too specific, and might lead people to think
this is the only valid use, and their placement feels a bit odd, perhaps
these should be given as a footnote? (But this is really for the Policy
editors to decide.)

Thanks,
Guillem

Bug#1065643: debian-policy: Refer to «dpkg-buildtree clean» for dpkg generated files

[Guillem Jover]

Package: debian-policy
Version: 4.6.2.1
Severity: wishlist

Hi!

Currently the Debian policy lists several files that are generated by
dpkg tools, but because there was no matching dpkg tool to clean them
up, it delegates that responsibility to packagers. With dpkg 1.22.3 a
new dpkg-buildtree tool was introduced that supports a «clean» action
precisely to take care of removing all these files. In the future I'd
like to change the location of some of these files, but that requires
for packages that do not use a helper to be updated to match the dpkg
behavior. While I expect this to be controlled via dpkg-build-api, it
would be best if such packages switched to use dpkg-buildtree to have
automatic handling when that time comes.

I've worded the changes as an example, because I don't want to impose
that packages should be using this if they are using a helper or that
helpers necessarily have to call it. If you'd prefer me to spell that
out more explicitly I'm happy to try to come up with some wording. It
intends to be more informational than a requirement, as I am going to
be filing wishlist bugs with patches as time permits, anyway.

I'm attaching a patch to add references to that new tool.

Thanks,
Guillem
From afac52fa956087eb737c123682f634fc739c7e20 Mon Sep 17 00:00:00 2001
From: Guillem Jover 
Date: Tue, 27 Feb 2024 23:37:06 +0100
Subject: [PATCH] =?UTF-8?q?Add=20references=20to=20=C2=ABdpkg-buildtree=20?=
 =?UTF-8?q?clean=C2=BB=20for=20debian/{substvars,files}?=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

These files are generated by dpkg tools (and in some cases by helpers),
but the maintainer was responsible for cleaning them up. There is now
a new command that will take care of cleaning these (and any other
future) files that the dpkg suite might end up generating, making their
introduction easier as the responsibility to remove them shifts back
where it belongs.
---
 policy/ch-source.rst | 7 ---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/policy/ch-source.rst b/policy/ch-source.rst
index 4307e89..2fb05cd 100644
--- a/policy/ch-source.rst
+++ b/policy/ch-source.rst
@@ -685,7 +685,7 @@ variables are also available.
 
 The ``debian/substvars`` file is usually generated and modified
 dynamically by ``debian/rules`` targets, in which case it must be
-removed by the ``clean`` target.
+removed by the ``clean`` target (for example with ``dpkg-buildtree clean``).
 
 See :manpage:`deb-substvars(5)` for full details about source variable
 substitutions, including the format of ``debian/substvars``.
@@ -725,8 +725,9 @@ building packages to record which files are being generated.
 
 It should not exist in a shipped source package, and so it (and any
 backup files or temporary files such as ``files.new``)  [#]_ should be
-removed by the ``clean`` target. It may also be wise to ensure a fresh
-start by emptying or removing it at the start of the ``binary`` target.
+removed by the ``clean`` target (for example with ``dpkg-buildtree clean``).
+It may also be wise to ensure a fresh start by emptying or removing it at the
+start of the ``binary`` target.
 
 When ``dpkg-gencontrol`` is run for a binary package, it adds an entry
 to ``debian/files`` for the ``.deb`` file that will be created when
-- 
2.43.0

Bug#1057199: debian-policy: express more clearly that Conflicts to not reliably prevent concurrent unpacks

[Guillem Jover]

Hi!

On Wed, 2024-01-03 at 15:04:01 -0700, Sam Hartman wrote:
> >>>>> "Guillem" == Guillem Jover  writes:
> Guillem> At least the dpkg behavior seems entirely
> Guillem> correct to me and required for safe upgrades (
> 
> Can you help me understand the sentence above?
> Where is the case where this behavior is needed for safe upgrades?
> (I am asking out of curiosity; I'm guessing it's some corner case with
> essential packages, but I would like to understand.)

I should probably have qualified that statement, where safe upgrades
is restricted to the specific scenarios at hand. In any case, I don't
think this is specific to essential packages, this seems more general.
But of course the effects of not supporting such safe upgrades for
essential packages would be potentially more severe (although an
essential package can never be a conflictor as dpkg would refuse to
remove it).

In essence this involves a cyclic restriction where a set of packages
are stating they cannot be unpacked at the same time, but later
versions might be. There are several subcases for this depending on
the strength of the dependencies from each side, whether these are
versioned, and also whether the package manager front-end or the user
decides whether to fully evict a conflictor or wants to upgrade to a
latest version that can co-exist.

What the current behavior permits is to safely intertwine an unpack,
a conflictors files transfer and its removal in the same step, so that
the front-end or the user does not need to apply --force-* options to
forcibly remove the conflictor while breaking the dependency system
for an undetermined amount of time, to be able to proceed with such
upgrade.

I don't see any other way around this kind of upgrade that does not
break the dependency system besides the current behavior (if someone
does, I'm happy to hear it). Of course there's always the option to
prohibit those kinds of relationships, which means the behavior never
is brought up, and there's no need either to be concerned about whether
it needs to be supported or not. But that seems overly restrictive,
because once or if you need to express that kind of relationship, then
that path would be closed.


All this being said, when Helmut brought this up, I noticed that dpkg
is expecting to be hand held by giving to it the proper order for these
operations (as front-ends do), or it will either fail to upgrade
depending on the scenario, or unnecessarily use the conflictor eviction
behavior, which are not ideal (but do not break the dependency system).
I've got some test cases (Conflicts vs Breaks, Conflicts vs Conflicts),
which I should expand to add addition restrictions in the form of
Depends and Pre-Depends, unversioned relationships and similar, after
which I'll look into improving the way dpkg schedules its processing
queue in these cases so that it is a bit smarter about them and does
not fail so easily or uses this behavior unnecessarily when there's a
path forward at hand. But this seems besides the point of the conflictor
eviction behavior.

Thanks,
Guillem

Bug#1057199: debian-policy: express more clearly that Conflicts to not reliably prevent concurrent unpacks

[Guillem Jover]

Hi!

On Fri, 2023-12-15 at 16:40:09 +, Sean Whitton wrote:
> On Fri 01 Dec 2023 at 02:11pm +01, Helmut Grohne wrote:
> > §7.4 currently starts with:
> >
> > When one binary package declares a conflict with another using a
> > Conflicts field, dpkg will refuse to allow them to be unpacked on
> > the system at the same time.
> >
> > I believe this is technically wrong. There are situations where dpkg
> > will allow such unpacks to temporarily co-exist. §6.6 goes into further
> > detail and is accurate.
> 
> Thank you for the detailed report.
> 
> Do the dpkg and apt people think that the bug here is just in Policy, or
> are there any code changes under consideration in response to this work?

I think it is just a documentation issue in the Debian Policy, yes. At
least the dpkg behavior seems entirely correct to me and required for safe
upgrades (and definitely not something like an accidental regression as it
has behaved that way since pretty much the beginning of its git history).

In addition I think the paragraph in §7.4 that states:

,---
A package will not cause a conflict merely because its configuration
files are still installed; it must be at least “Half-Installed”.
`---

could also be clarified that what is stated here does not apply either
to conflicting packages that are being removed, as those will be in
half-installed state. Perhaps as part of this, also make this state
change explicit in §6.6.2.3.

Thanks,
Guillem

Bug#1057238: debian-policy: Take dpkg-build-api into account for Rules-Requires-Root

[Guillem Jover]

Package: debian-policy
Version: 4.6.2.0
Severity: wishlist

Hi!

Starting with dpkg 1.22.0, it implements a dpkg-build-api mechanism
similar in concept to the debhelper-compat levels.

You can check its documentation in the dpkg-build-api(7) and
dpkg-buildapi(1) manual pages.

I think at least the part that involves the Rules-Requires-Root field
which in level 1 defaults to value «no» instead of «binary-targets»
should be documented in the Debian policy.

I'm ambivalent on whether documenting the general mechanism in Debian
policy makes sense though.

[ I noticed I didn't update the deb-src-control(5) manual page,
  so I did that locally and will be part of the next dpkg release. ]

Thanks,
Guillem

Bug#1057057: debian-policy: Please make Checksums-Sha1 optional

[Guillem Jover]

Hi!

On Tue, 2023-11-28 at 14:57:10 -0800, Russ Allbery wrote:
> Dimitri John Ledkov  writes:
> > Dak currently requires Checksums-Sha1, but I am happy to facilitate in
> > patching dak to make Checksums-Sha1 optional if this bug report is
> > accepted.
> 
> The field is documented as mandatory precisely because DAK requires it,
> which makes it mandatory for Debian packages.  As soon as DAK doesn't
> require it, I'm happy to make it optional (and indeed it would arguably be
> a bug in Policy if it's optional in the archive but Policy claims it's
> mandatory).

I'd like to drop those from .changes and .dsc (among other things),
but demoting these which are currently marked as required to me implies
a major format version bump. And I don't recall ever demoting required
fields, only promoting fields from optional to required.

For .changes, I've got this among other cleanups that would be nice to
do to the format:

  https://wiki.debian.org/Teams/Dpkg/Spec/ChangesFormat2.0

but there did not seem to be much enthusiasm when I proposed this some
time ago:

  https://lists.debian.org/debian-devel/2016/04/msg00326.html

For .dsc, there's the problem that, very confusingly the Format is used
not for the file format, but for the source format, which I think was
a mistake at the time, but here we are, see the .dsc section at:

  https://wiki.debian.org/Teams/Dpkg/TimeTravelFixes

Thanks,
Guillem

Bug#945269: debian-policy: packages should use tmpfiles.d(5) to create directories below /var

[Guillem Jover]

Hi!

On Tue, 2023-09-12 at 22:17:44 -0700, Russ Allbery wrote:
> Russ Allbery  writes:
> > Russ Allbery  writes:
> >> Maybe the right way to do this is just have two examples, one as the
> >> default and another if you're using tmpfiles.d functionality added in a
> >> specific version of systemd that's newer than the version shipped with
> >> the stable version of Debian prior to the one you're targeting.
> 
> > Here's an updated version with that change plus some other minor fixes.
> 
> Er, right, helps to rebase first.  Here's the actual patch.

> diff --git a/policy/ch-files.rst b/policy/ch-files.rst
> index b34c183..fa3e5be 100644
> --- a/policy/ch-files.rst
> +++ b/policy/ch-files.rst
> @@ -722,6 +722,70 @@ The name of the files and directories installed by 
> binary packages
>  outside the system PATH must be encoded in UTF-8 and should be
>  restricted to ASCII when it is possible to do so.
>  
> +.. _s-tmpfiles.d:
> +
> +Volatile and temporary files (``tmpfiles.d``)
> +-
> +
> +Some packages require empty directories in ``/var`` or ``/etc``, or

Not shipping these empty directories in the .deb seems like a
regression or a disservice to me. Even for things that might get
deleted because things like our policy or the FHS allows for it (say
stuff under /var/cache), as «dpkg --verify» can be useful. Because of
course, these in addition, can be defined via tmpfiles.d, so that they
can possibly be recreated if needed (until dpkg provides its own
interfaces to do that).

> +symlinks or files with trivial content in ``/var``, to implement their
> +functionality.  Examples include directories under ``/var/cache`` that are
> +writable by the package as cache areas, an initially-empty directory in
> +``/etc`` intended for local overrides added by the local system
> +administrator, or a file in ``/var`` that should default to a symlink
> +elsewhere on the system but may be changed later.
> +
> +Rather than include these symlinks, files, or directories in the binary
> +package or create them in package maintainer scripts, packages should use
> +the ``tmpfiles.d`` mechanism to specify the files and directories that
> +should be created.  This allows associating these files and directories
> +with specific packages (not currently possible when creating them in
> +maintainer scripts),

Well, this association would then only be indirect, instead of being able
to get at them via say «dpkg-query --search» or «dpkg-query --listfiles».

>   and allows local administrators to delete the
> +contents of directories such as ``/var/cache`` with the assurance that
> +``tmpfiles.d`` can recreate the necessary file structure without
> +reinstalling packages or re-running maintainer scripts.

Thanks,
Guillem

Bug#793499: debian-policy: The Installed-Size algorithm is out-of-date

[Guillem Jover]

Hi!

On Fri, 2015-07-24 at 18:04:41 +0200, Guillem Jover wrote:
> Package: debian-policy
> Version: 3.9.7.0
> Severity: wishlist

> As discussed in the debian-policy list, the Installed-Size algorithm
> as implemented in dpkg-gencontrol changed due to #650077. So the
> current wording is out-of-sync. Please see the thread starting at
> <https://lists.debian.org/debian-policy/2011/11/msg00079.html>,
> with the current implementation
> <https://lists.debian.org/debian-policy/2015/01/msg00044.html>.

How about the attached patch, based on the text in deb-substvars(5)?

Thanks,
Guillem
From 024d94daeb0ab0e7c447868a1b8f9ff953850404 Mon Sep 17 00:00:00 2001
From: Guillem Jover 
Date: Tue, 12 Sep 2023 22:47:27 +0200
Subject: [PATCH] Update Installed-Size algorithm used by dpkg since 1.18.0

The previous algorithm relied entirely on du(1) computing the used
size, but depended on the filesystem in use on the build system.

The new algorithm used by dpkg since 1.18.0 (implemented in
commit 9ed7d4d47b73ffe67e1f7d31f899a1dfd43d490b), guarantees a
constant and reproducible size regardless of the build system
filesystem being used. Although it is still an approximation of
the actual size that the package will use on the installed system.

Closes: #793499
---
 policy/ch-controlfields.rst | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/policy/ch-controlfields.rst b/policy/ch-controlfields.rst
index 45776ea..2871658 100644
--- a/policy/ch-controlfields.rst
+++ b/policy/ch-controlfields.rst
@@ -939,8 +939,9 @@ space required to install the named package. Actual installed size may
 vary based on block size, file system properties, or actions taken by
 package maintainer scripts.
 
-The disk space is given as the integer value of the estimated installed
-size in bytes, divided by 1024 and rounded up.
+The disk space is given as the accumulated size of each regular file and
+symlink rounded to 1 KiB used units, and a baseline of 1 KiB for any other
+filesystem object type.
 
 .. _s-f-Files:
 
-- 
2.40.1

Bug#970234: consider dropping "No hard links in source packages"

[Guillem Jover]

Hi!

On Thu, 2022-09-22 at 19:20:00 -0700, Russ Allbery wrote:
> Russ Allbery  writes:
> > The fact that this has gone unnoticed in a source package in an existing
> > release makes a pretty strong argument that nothing in Debian cares and
> > we should just remove the constraint.
> 
> Here is a patch dropping the restriction on hard links in source packages
> that I think is ready for seconds.  I'm copying Guillem for his review, in
> case there's some dpkg concern.

I'm not really sure what the footnote really refers to, TBH, as I'm
not aware of any such check, or what would require a fair amount of
work. Besides the other potential issues mentioned on the bug, which
I agree we might not care much about, a case that comes to mind would
be that patching hard linked source files can end up being very
confusing, and might not really break the build but can end up not
producing what one might expect. I've quickly prepared the attached
tentative and completely untested patch, to add a warning in that case,
which I guess I might be targeting for dpkg 1.22.1 (once I've added
some tests).

But given that hard links in source packages do not seem prevalent
at all, and that the tooling or linters can be improved in that
direction I suppose it might make sense to lift this specific
restriction.

> >From 12b014c4b930577a728dfb1254b64aac6a5eb1e0 Mon Sep 17 00:00:00 2001
> From: Russ Allbery 
> Date: Thu, 22 Sep 2022 19:15:52 -0700
> Subject: [PATCH] Allow hard links in source packages
> 
> It's not clear why this restriction was in place, and Debian
> included a package containing hard links without anyone noticing
> in the last release.
> ---
>  policy/ch-source.rst | 11 ++-
>  1 file changed, 2 insertions(+), 9 deletions(-)
> 
> diff --git a/policy/ch-source.rst b/policy/ch-source.rst
> index c7415fc..a7df539 100644
> --- a/policy/ch-source.rst
> +++ b/policy/ch-source.rst
> @@ -282,8 +282,8 @@ source files in a package, as far as is reasonably 
> possible.  [#]_
>  Restrictions on objects in source packages
>  --
>  
> -The source package must not contain any hard links,  [#]_ device special
> -files, sockets or setuid or setgid files.. [#]_
> +The source package must not contain device special files, sockets, or
> +setuid or setgid files. [#]_
>  
>  .. _s-debianrules:
>  
> @@ -918,13 +918,6 @@ must not exist a file ``debian/patches/foo.series`` for 
> any ``foo``.
> would be nice if the modification time of the upstream source would
> be preserved.
>  
> -.. [#]
> -   This is not currently detected when building source packages, but
> -   only when extracting them.
> -
> -   Hard links may be permitted at some point in the future, but would
> -   require a fair amount of work.
> -
>  .. [#]
> Setgid directories are allowed.

Seconded.

Thanks,
Guillem
diff --git i/scripts/Dpkg/Source/Patch.pm w/scripts/Dpkg/Source/Patch.pm
index 0da35b9e5..99020885b 100644
--- i/scripts/Dpkg/Source/Patch.pm
+++ w/scripts/Dpkg/Source/Patch.pm
@@ -513,6 +513,11 @@ sub analyze {
 	error(g_("diff '%s' patches something which is not a plain file"),
 	  $diff);
 	}
+my $nlink = (stat _)[3];
+if ($nlink > 1) {
+warning(g_("diff '%s' patches hard link %s which can have " .
+   "unintended consequences"), $diff, $fn);
+}
 
 	if ($filepatched{$fn}) {
 $filepatched{$fn}++;


signature.asc
Description: PGP signature

Bug#963524: debian-policy: Binary and Description fields not mandatory in .changes on source-only uploads

[Guillem Jover]

Hi!

On Sun, 2023-09-10 at 16:31:30 -0700, Russ Allbery wrote:
> Guillem Jover  writes:
> > Hmm, the "For this case" comes just after the "no binary packages" which
> > to me reads as being somewhat referring to it, perhaps the "no binary
> > packages" sentence should be put at the end of the paragraph to avoid
> > confusion, or the "For this case" moved instead after the "It is a
> > multiline field" one or something along those lines?
> 
> I knew I should have listened to my instincts and reworded that paragraph
> some more to make it explicit that the format of the field in the .changes
> file is different.  (Unfortunate, but oh well, too late now.)

No worries, that's what reviews are for! :D

> Here is an updated patch that restructures this paragraph to try to make
> this clearer.

Thanks, this is indeed clearer.

> >From 516d0a327e247c35bd1bb95ff2a9bfc773f87c21 Mon Sep 17 00:00:00 2001
> From: Russ Allbery 
> Date: Tue, 20 Sep 2022 21:17:55 -0700
> Subject: [PATCH] Binary and Description optional in .changes
> 
> In .changes files for source-only uploads, the Binary and
> Description fields are not present.  Document this, and be clearer
> in the description of the Description field for .changes files that
> only descriptions of binary packages are included.
> ---
>  policy/ch-controlfields.rst | 23 ++-
>  1 file changed, 14 insertions(+), 9 deletions(-)
> 
> diff --git a/policy/ch-controlfields.rst b/policy/ch-controlfields.rst
> index 4bab7df..d5c9d68 100644
> --- a/policy/ch-controlfields.rst
> +++ b/policy/ch-controlfields.rst
> @@ -278,7 +278,7 @@ The fields in this file are:
>  
>  -  :ref:`Source ` (mandatory)
>  
> --  :ref:`Binary ` (mandatory)
> +-  :ref:`Binary ` (mandatory in some cases)
>  
>  -  :ref:`Architecture ` (mandatory)
>  
> @@ -292,7 +292,7 @@ The fields in this file are:
>  
>  -  :ref:`Changed-By `
>  
> --  :ref:`Description ` (mandatory)
> +-  :ref:`Description ` (mandatory in some cases)
>  
>  -  :ref:`Closes `
>  
> @@ -812,12 +812,16 @@ See :ref:`s-descriptions` for further information on
>  this.
>  
>  In a ``.changes`` file, the ``Description`` field contains a summary of
> -the descriptions for the packages being uploaded. For this case, the
> -first line of the field value (the part on the same line as
> -``Description:``) is always empty. It is a multiline field, with one
> -line per package. Each line is indented by one space and contains the
> -name of a binary package, a space, a hyphen (``-``), a space, and the
> -short description line from that package.
> +the descriptions of the binary packages being uploaded. If no binary
> +packages are being uploaded, this field will not be present.
> +
> +When used inside a ``.changes`` file, the ``Description`` field has a
> +different format than in source or binary control files. It is a multiline
> +field with one line per binary package. The first line of the field value
> +(the part on the same line as ``Description:``) is always empty. Each
> +subsequent line is indented by one space and contains the name of a binary
> +package, a space, a hyphen (``-``), a space, and the short description
> +line from that package.
>  
>  .. _s-f-Distribution:
>  
> @@ -927,7 +931,8 @@ every architecture. The source control file doesn't 
> contain details of
>  which architectures are appropriate for which of the binary packages.
>  
>  When it appears in a ``.changes`` file, it lists the names of the binary
> -packages being uploaded, separated by whitespace (not commas).
> +packages being uploaded, separated by whitespace (not commas). If no
> +binary packages are being uploaded, this field will not be present.
>  
>  .. _s-f-Installed-Size:
>  

Seconded.

Thanks,
Guillem


signature.asc
Description: PGP signature

Bug#963524: debian-policy: Binary and Description fields not mandatory in .changes on source-only uploads

[Guillem Jover]

Hi!

On Sun, 2023-09-10 at 10:30:41 -0700, Russ Allbery wrote:
> diff --git a/policy/ch-controlfields.rst b/policy/ch-controlfields.rst
> index 4bab7df..904fa52 100644
> --- a/policy/ch-controlfields.rst
> +++ b/policy/ch-controlfields.rst
> @@ -812,10 +812,11 @@ See :ref:`s-descriptions` for further information on
>  this.
>  
>  In a ``.changes`` file, the ``Description`` field contains a summary of
> -the descriptions for the packages being uploaded. For this case, the
> -first line of the field value (the part on the same line as
> -``Description:``) is always empty. It is a multiline field, with one
> -line per package. Each line is indented by one space and contains the
> +the descriptions of the binary packages being uploaded. If no binary
> +packages are being uploaded, this field will not be present. For this
> +case, the first line of the field value (the part on the same line as
> +``Description:``) is always empty.

Hmm, the "For this case" comes just after the "no binary packages"
which to me reads as being somewhat referring to it, perhaps the "no
binary packages" sentence should be put at the end of the paragraph to
avoid confusion, or the "For this case" moved instead after the "It is
a multiline field" one or something along those lines?

>   It is a multiline field, with one line
> +per binary package. Each line is indented by one space and contains the
>  name of a binary package, a space, a hyphen (``-``), a space, and the
>  short description line from that package.
>  

Thanks,
Guillem

Bug#1035733: debian -policy: packages must not use dpkg-divert to override default systemd configuraton files

[Guillem Jover]

On Mon, 2023-05-08 at 08:48:49 -0700, Russ Allbery wrote:
> […] I suspect Policy should say something stronger and more general,
> namely that no package in Debian should divert a file from another package
> unless this is arranged cooperatively between the packages to solve some
> specific (unusual) problem.

,--- §3.9 ---
You should not use "dpkg-divert" on a file belonging to another
package without consulting the maintainer of that package first. When
adding or removing diversions, package maintainer scripts must provide
the "--package" flag to "dpkg-divert" and must not use "--local".
`---

> To me, this feels similar to the case of one
> package modifying the configuration files of another package, where we
> explicitly prohibit one package modifying the configuration of another
> package except through an interface provided by the package whose
> configuration is being modified.

,--- §10.7.4 ---
If the configuration file cannot be shared as described above, the
packages must be marked as conflicting with each other. Two packages
that specify the same file as a "conffile" must conflict. This is an
instance of the general rule about not sharing files. Neither
alternatives nor diversions are likely to be appropriate in this case;
in particular, "dpkg" does not handle diverted "conffile"s well.
`---

Regards,
Guillem

Bug#1029831: debian-policy: Make required packages build-essential

[Guillem Jover]

On Sat, 2023-01-28 at 14:07:06 +0100, Ansgar wrote:
> Timo Röhling writes:
> > * Andreas Henriksson  [2023-01-28 12:50]:
> >>Policy is not a religion. Policy has many bugs. Policy is very outdated.
> >>[...]
> >>Here's an example you could follow:
> >>https://lists.debian.org/debian-policy/2022/12/msg00023.html
> > Your argument cuts both ways. Right now, Policy says that
> > the bugs are RC, and if you believe that should be different, why
> > don't you propose such a change and seek consensus yourself?
> 
> I don't think it does.  Policy doesn't specify what packages actually
> are build-essential; it only refers to an "informational list" in
> Section 4.2.

It does, but in a way that does not require encoding package names in
policy to leave breathing room to toolchain maintainers.

Policy says this:

  ,---
  It is not necessary to explicitly specify build-time relationships on
  a minimal set of packages that are always needed to compile, link and
  put in a Debian package a standard “Hello World!” program written in C
  or C++. The required packages are called *build-essential*, and an
  informational list can be found in "/usr/share/doc/build-
  essential/list" (which is contained in the "build-essential" package).
  [1]
  `---

With that footnote explaining the rationale:

  ,---
  [1] Rationale:

* This allows maintaining the list separately from the policy
  documents (the list does not need the kind of control that the
  policy documents do).

* Having a separate package allows one to install the build-
  essential packages on a machine, as well as allowing other
  packages such as tasks to require installation of the build-
  essential packages using the depends relation.

* The separate package allows bug reports against the list to be
  categorized separately from the policy management process in the
  BTS.
  `---

And further down it says this:

  ,---
  If build-time dependencies are specified, it must be possible to build
  the package and produce working binaries on a system with only
  essential and build-essential packages installed and also those
  required to satisfy the build-time relationships (including any
  implied relationships). […]
  `---

> I think discussion in #1027832 suggested that required packages should
> be build-essential as well. Maybe this should be made explicit, for
> example by changing
> 
> +---
> | The required packages are called build-essential, and an informational
> | list can be found in /usr/share/doc/build-essential/list (which is
> | contained in the build-essential package).
> +---[ Section 4.2 ]
> 
> to something like
> 
> +---
> | The required packages are called build-essential, and include packages
> | with Priority "required" and additional packages. An informational
> | list of additional packages can be found in
> | /usr/share/doc/build-essential/list (which is contained in the
> | build-essential package).
> +---
> 
> This only documents existing practice as practically all systems have
> required packages installed.

If the point of this proposal is to include tzdata by proxy, then that
makes no sense, because tzdata does not have the properties to keep
being a "required" package. And if Priority "required" packages are
a way to denote the pseudo-essential set via priorities (even though we
do not even mark depended libraries as such anymore, so it's at most a
subset) instead of the Essential:yes field and dependencies of those,
then this is already covered by the "essential" mention above.

So adding this would only confuse things more than help anything. And
would be only a definition hack to preserve tzdata in that set only
temporarily anyway.

Guillem

Bug#1020248: marked as done (debian-policy: Clarifying nomenclature for control file names)

[Guillem Jover]

Hi!

On Sat, 2022-12-17 at 17:24:57 -0700, Sean Whitton wrote:
> On Sat 17 Dec 2022 at 04:43PM +01, Guillem Jover wrote:
> > Sorry, probably my fault! As I tend to use «Fixes:» git pseudo-fields
> > for things that fix part of a bug, but are not intended yet to close it,
> > for which I use «Closes:».
> >
> > And for some reason I think I also got the impression, even though
> > the stanza changes had been committed, they could still be backed out.
> > (BTW I've now gone over the wiki and updated all paragraph references
> > that applied to stanza.)
> >
> > In any case, I've sat down and gone over the meat of the original
> > report. See below.
>·
> We're sticking with 'stanza', and in light of that, could you confirm
> that the bug is reopened in order to make additional fixes, rather than
> back anything else out?

In case my other replies to Russ didn't make this clear. This comment
was in reference to the various replies in the sub-thread started by
Charles, where it looked to me like whether to back that out was still
an open question for the editors.

In any case, as I mentioned, given the changes being included in the
release, I took that as indeed, sticking with the term, and that's why
I reopened and submitted the actual changes this original report was
intending on requesting. :)

As an aside I've since updated the dpkg code and docs to refer to
these as stanzas everywhere applicable.

Thanks,
Guillem

Bug#1020248: marked as done (debian-policy: Clarifying nomenclature for control file names)

[Guillem Jover]

On Sat, 2022-12-17 at 08:35:02 -0800, Russ Allbery wrote:
> (That said, and this is only personal preference and I don't feel that
> strongly about it, I usually err on the side of creating lots of bugs so
> that there can be roughly one bug per patch.  It can make it a bit harder
> to track things if there's one bug following a bunch of semi-related but
> separable changes.  Unfortunately, the BTS doesn't support the concept of
> a hierarchy with a tracking issue and a bunch of underlying implementaton
> issue very well.)

(Right, personally I don't think I'd split one bug per patch, as long
as the patch is covering the same thing, perhaps. In this case I
pondered about opening a new one, but given the initial discussion
and context was here it seemed best to keep it together. Should have
perhaps split the stanza stuff into another report, though. :)

(In any case, hope this is all not too inconvenient!)

> Guillem Jover  writes:
> > And for some reason I think I also got the impression, even though
> > the stanza changes had been committed, they could still be backed out.
> > (BTW I've now gone over the wiki and updated all paragraph references
> > that applied to stanza.)
> 
> I'm personally happy to stick with stanza.

Sorry, rereading that paragraph it seems not clear on what I was
trying to convey. :) I meant that because I thought this could be backed
out until an upload, I guess subconsciously postponed further changes
for this (both in dpkg and here) based on that. Should have asked.

> > I also recalled another term that has always seemed very confusing in
> > context: «control information files» or «control information area». For
> > example in a sentence such as “the control file is a control information
> > file in the control information area in a .deb archive”. :) This also
> > seems confusing when some of the files in the .deb control member are
> > not really “control files” with a deb822(5) format.
> 
> > My thinking has been going into calling these as the «metadata files»,
> > and being located in either the  «metadata part of the .deb archive» or
> > explicitly the «control member of the .deb archive», in contrast to the
> > filesystem part. In dpkg I'd be eventually switching to meta/metadata
> > and fsys/filesystem, from control or info and data. I've added a patch
> > with the proposed change, but again nothing set in stone, and I'm again
> > open to discussing pros/cons of this.
> 
> I like metadata file, but I think I prefer talking about the "control
> member" than the "metadata part" because it more closely matches what one
> sees if one takes the *.deb file apart.  But I haven't looked at your
> diffs yet.

Probably more clear currently, yeah. To expand on the above, my thinking
has been for example that if we ever have a deb 3.0 format, I'd probably
like to name the members, something like: «meta.tar.*» and «fsys.tar.*»,
and that's also what I've kind of been moving the dpkg internals to
(functions, structs and similar). Also as part of the file metadata work,
I've also have pending splitting the dpkg db info/ dir into a dir that
contains things shipped by the .deb, and a dir for stuff generated by
dpkg itself, so that they cannot conflict or get overwritten or need
to be taken into account doing any of that.

Thanks,
Guillem

Bug#1020248: marked as done (debian-policy: Clarifying nomenclature for control file names)

[Guillem Jover]

Control: reopen -1

Hi!

Sorry, probably my fault! As I tend to use «Fixes:» git pseudo-fields
for things that fix part of a bug, but are not intended yet to close it,
for which I use «Closes:».

And for some reason I think I also got the impression, even though
the stanza changes had been committed, they could still be backed out.
(BTW I've now gone over the wiki and updated all paragraph references
that applied to stanza.)

In any case, I've sat down and gone over the meat of the original
report. See below.

On Sat, 2022-12-17 at 03:09:10 +, Debian Bug Tracking System wrote:
> Date: Sun, 18 Sep 2022 22:28:00 +0200
> From: Guillem Jover 
> To: sub...@bugs.debian.org
> Subject: debian-policy: Clarifying nomenclature for control file names
> 
> Package: debian-policy
> Version: 4.6.1.1
> Severity: wishlist

> This is a followup from my comment at:
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=998165#43
> 
> To summarize, we have IMO confusing naming and nomenclature for the
> various control files and paragraphs/stanzas, and this is even
> confusing me when having to deal with dpkg code, so I'd like to give
> these more clear and unambiguous new names, and I'd very strongly
> prefer to agree on the same naming for Debian policy and dpkg, to
> avoid further and worse confusion (even though they currently do not
> match exactly anyway, but I'd prefer to not make it worse…).
> 
> Just for reference and to give some context, I've got the following
> WIP branches, trying to clarify the names in documentation and in the
> API on, which I'll probably rework (split/merge) and reword as needed,
> so do not take them as anything set in stone:
> 
>   
> https://git.hadrons.org/git/debian/dpkg/dpkg.git/log/?h=next/clarify-control-filenames
>   
> https://git.hadrons.org/git/debian/dpkg/dpkg.git/log/?h=next/deb822-field-types
> 
> 
> File descriptions
> -
> 
> For example we have:
> 
>   * debian/control:
> policy → «Source package control file»
> dpkg   → «Debian source packages' master control file»
> 
>   * .dsc:
> policy → «Debian source control file»
> dpkg   → «Debian source packages' control file»
> 
>   * DEBIAN/control
> policy → «Binary package control files»
> dpkg   → «Debian binary packages' master control file»

Seems I missed another file:

  * .changes:
policy → «upload control file» / «Debian changes file»
dpkg   → «upload control file» / «.changes control file» /
 «Debian .changes file» / «Debian changes file»

> These are quite confusingly close.
> 
> I've been considering naming debian/control something like
> «Debian template source package control file», as that is used to
> generate both the source and binary control files. And always
> prefixing with Debian, so that would end up as:
> 
>   * debian/control: «Debian source package template control file»
>   * .dsc:   «Debian source package control file»
>   * DEBIAN/control: «Debian binary package control file»

For changes I think something like the following might be a more clear
option (and has the minor bonus of aligning perfectly on the first
words! :), with it mentioning explicitly this is about changes being
uploaded, and that it is a control file (but I'm not sure I'm entirely
convinced about it):

* .changes:   «Debian upload changes control files»

> This also removes the «master» usage in dpkg, for me for the same
> reasons as I covered at
> <https://lists.debian.org/debian-dpkg/2021/03/msg2.html>.

> File contents
> -
> 
> We have references to the various parts being called as «paragraphs»,
> «stanza», «blocks», but this seems to be more of an issue with dpkg, as
> the usage in the Debian policy is quite clear and uniform now, so I'll
> at least try to remove the «block» usage there, stanza has the nice
> property of being shorter and policy already mentions that this is
> currently a common alias, so I might keep paragraph and stanza for now
> in dpkg.

I've also found instances of «record» and «section» referring to fields
or stanzas.

> The other thing affecting dpkg and debian-policy is how the parts
> within the control files are referred to. We have for example:
> 
>   dpkg   → «general section of control info file»
>«source stanza»
>   policy → «general paragraph»
> 
>   dpkg   → «package's section of control info file»
>   policy → «binary package paragraphs»
> 
> 
> So, how does «source package paragraph» and «binary package paragraph»
> (of the «template control file») sound instead?

> If I've missed any other problematic nomenclature, I'm happy to
> discuss and update 

Bug#1020267: Essential packages only provide functionality after being configured

[Guillem Jover]

Hi!

On Sun, 2022-09-18 at 20:27:46 -0700, Russ Allbery wrote:
> Helmut Grohne  writes:
> > […] It can be made explicit in section 3.8 quite easily:
> 
> >  Since dpkg will not prevent upgrading of other packages while an
> >  ``essential`` package is in an unconfigured state, all ``essential``
> >  packages must supply all of their core functionality even when
> > -unconfigured. If the package cannot satisfy this requirement it must not
> > +unconfigured after being configured at least once.
> > +If the package cannot satisfy this requirement it must not
> >  be tagged as essential, and any packages depending on this package must
> >  instead have explicit dependency fields as appropriate.

Seconded.

Thanks,
Guillem


signature.asc
Description: PGP signature

Bug#998282: Please make Section a required field for the source paragraph in d/control

[Guillem Jover]

Hi!

On Tue, 2022-09-20 at 19:36:36 -0700, Russ Allbery wrote:
> Felix Lechner  writes:
> > The installable stanzas in d/control (called "binary package paragraphs"
> > in policy) inherit the Section field from the source paragraph. There is
> > no reason to provide inheritance the other way around.
> 
> Huh, this pointed out to me that I don't know what the current behavior
> is.  I think I've always put a Section field in the first stanza and then
> overridden it as necessary, or at least I don't remember thinking about
> this before.

If you omit the field in the source stanza, then both dpkg-genchanges
and lintian will complain about it with a warning.

> The current Policy description of the Section field is rather cryptic:
> 
> This field specifies an application area into which the package has
> been classified. See Sections.
> 
> When it appears in the debian/control file, it gives the value for the
> subfield of the same name in the Files field of the .changes file. It
> also gives the default for the same field in the binary packages.
> 
> I understand what it's trying to say, but that's a very mechanical
> definition that isn't clear about the relationship between the Section
> field in the source stanza and the Section field in the binary stanzas.
> (It also claims Section is about an "application area," a term that I
> don't think we use anywhere else in Policy.)
> 
> So, what happens today if you put Section in a binary stanza but not in
> the source stanza?  Is it inherited from the binary stanza by the source
> stanza (I agree that seems weird)?  If so, from which binary stanza is it
> inherited, if there are several?  The definition of the Files field
> implies that the section may just be - in some cases.

Some of the fields found in the debian/control binary stanzas inherit
their values (if omitted) from the source stanza, otherwise they get
overridden.

The binary packages use a combination of information from
debian/changelog, the source stanza and their own binary stanza.

For non-binary packages (sources) or artifacts (say .buildinfo, or
byhand objects) there are no binary stanzas, so no information is
taken from them.

If you specify a Section field for every binary stanza, and omit it
from the source stanza, then dpkg-genchanges will have to fill it with
«-» when generating the .changes file.

If you omit the Section field from both the source stanza and one of
the binary stanzas, then that binary package will contain no Section
field, dpkg-genchanges will warn about both missing fields, and it
will fill the section value as «-» for that binary package on the
.changes file.

Thanks,
Guillem

Bug#953911: debian-policy: clarify documentation of "Closes: #NNNNNN" changelog syntax

[Guillem Jover]

Hi!

On Sun, 2022-09-18 at 21:42:37 -0700, Russ Allbery wrote:
> "Daniel Shahaf"  writes:
> > Here's a revision of the patch incorporating the feedback so far:
> 
> Thank you for this patch!  I confirmed that your description matches the
> regular expression.  This has now been applied for the next release of
> Debian Policy.

I also committed a changed based off this patch to dpkg's
deb-changelog(5) man page, which I mentioned some time ago I'd do,
but probably didn't as I thought the patch here still had the issue
discussed at the time.

Thanks,
Guillem

Bug#1020248: [Git][dbnpolicy/policy][master] 2 commits: Use stanza to refer to deb822 parts instead of paragraph

[Guillem Jover]

Hi!

On Thu, 2022-09-22 at 14:26:38 +0900, Charles Plessy wrote:
> Le Tue, Sep 20, 2022 at 06:08:16PM -0700, Russ Allbery a écrit :
> > I do find the use of paragraph the way we were previously using it to
> > be confusing, particularly given that the paragraphs contain fields
> > which in turn contain actual paragraphs in the normal sense of the
> > term.

Idem.

> > I don't want to keep using paragraph, but I'd be open to some other term
> > that Guillem was also open to (I think matching the terminology in dpkg is
> > very important).  Section or block are commonly used for things like this,
> > but aren't very precise, so I'm not that enthused by them.
> 
> In the spec, the word "paragraph" is only used in the specified context,
> so I always felt that there is no ambiguity.  But of course, it can
> create opportunities for misunderstanding when discussing about the
> spec.  So point taken about "paragraph", although interestingly, the
> Simple English definition of "paragraph" is quite spot on if one would
> replace "sentence" with "field": ”one or more sentences that are written
> together with no line breaks separating them.  Usually they are
> connected by a single idea.” ()

In the end nothing will match exactly, and we need to choose some
terminology. In this case, as previously mentioned, «stanza» has the
good properties of not usually applying to prose, being short, distinct
from the other terms and the less ambiguous of them all. It also makes
constructing sentences to describe things less cumbersome.

> I do not mind the word "section".  It is the term used in the manual
> page "systemd.syntax" that describes systemd's unit files, which means
> that readers may be already familiar with the concept.  One could argue
> that its definition in Simple English
> (, “A section of a thing or
> place is a part of it”) would allow a reader to think that a Field is
> also a section, but I feel it is unlikely to happen.  This said, one big
> disadvantage of "section" is that when searching for this word in a
> document, there may be a lot of noisy hits such as "refer to section xyz
> for details".

The problems with section, is that as you mention is not very
searchable, but worse we already have a field with the same name!

> I understand about avoiding ambiguity, but in my opinion it is the price
> to pay to be able to translate information into simple words from
> English to non-European languages.  Although the Policy itself is not
> going to be translated, I think that it can be advantageous if its
> contents can be discussed in simple words in people's native languages.

As a non-native speaker, and a translator, I agree having clear
wording in the original text is important, as otherwise that tends to
make translation work harder. But then, part of that work is to find
or create terminology, in many cases not existing yet in the
translated language, that might be suitable there, trying several terms
that might not necessarily be direct translations.

For a translation anecdote related to finding the right terms, when
triggers got introduced, and having to translate them to Catalan, we
initially used «gallets» (which would be the direct translation). But
when reading them that was bothering several of us as it sounded weird,
it could be read as “small roosters” («gall» being rooster, and «ets»
forming the plural diminutive), or being too close to «galets» which is
a type of pasta used for example in «sopa de galets» ("galets" soup). We
then switched to «activadors» which sounds way nicer, even though it's
not a direct translation. But if we had to translate the spec today,
that would be annoying as it uses «activating» all over the place, so
perhaps using «disparador» would be better. So, in the end this is a
process too, and terms can be changed if they are deemed confusing or
not helping convey the meaning. And in some others, you just need to
simply create new terminology, and describe what it means in specific
contexts.


For example for Catalan/Spanish «stanza» is simply «estrofa» which
seems like a nice term to use here.

Thanks,
Guillem

Bug#963524: debian-policy: Binary and Description fields not mandatory in .changes on source-only uploads

[Guillem Jover]

Hi!

On Tue, 2022-09-20 at 21:21:23 -0700, Russ Allbery wrote:
> Here is a patch to fix this wording in Policy.  I think it's ready for
> seconds.

> >From c98654d7effa875c6e11da16159ac3feded8f763 Mon Sep 17 00:00:00 2001
> From: Russ Allbery 
> Date: Tue, 20 Sep 2022 21:17:55 -0700
> Subject: [PATCH] Binary and Description optional in .changes
> 
> In .changes files for source-only uploads, the Binary and
> Description fields are not present.  Document this, and be clearer
> in the description of the Description field for .changes files that
> only descriptions of binary packages are included.
> ---
>  policy/ch-controlfields.rst | 20 +++-
>  1 file changed, 11 insertions(+), 9 deletions(-)
> 
> diff --git a/policy/ch-controlfields.rst b/policy/ch-controlfields.rst
> index 428b8a7..f85f75d 100644
> --- a/policy/ch-controlfields.rst
> +++ b/policy/ch-controlfields.rst
> @@ -278,7 +278,7 @@ The fields in this file are:
>  
>  -  :ref:`Source ` (mandatory)
>  
> --  :ref:`Binary ` (mandatory)
> +-  :ref:`Binary `

For deb-changes(5) I switched from “(required)” to “(required in
context)”, but I don't think there's any similar precedent in the
Debian policy. Just noting here mostly for reference or as possible
inspiration :), and not as any kind of blocker or conditional on
seconding.

>  
>  -  :ref:`Architecture ` (mandatory)
>  
> @@ -292,7 +292,7 @@ The fields in this file are:
>  
>  -  :ref:`Changed-By `
>  
> --  :ref:`Description ` (mandatory)
> +-  :ref:`Description `
>  
>  -  :ref:`Closes `
>  
> @@ -809,12 +809,13 @@ See :ref:`s-descriptions` for further information on
>  this.
>  
>  In a ``.changes`` file, the ``Description`` field contains a summary of
> -the descriptions for the packages being uploaded. For this case, the
> -first line of the field value (the part on the same line as
> -``Description:``) is always empty. It is a multiline field, with one
> -line per package. Each line is indented by one space and contains the
> -name of a binary package, a space, a hyphen (``-``), a space, and the
> -short description line from that package.
> +the descriptions of the binary packages being uploaded. (``.changes``
> +files for uploads containing only source packages will not have this
> +field.)

Hmm, I guess that's fine, but perhaps it would be more precise and/or
future-proof to state instead that the field is only present if there
are binary packages included (or will be missing if they are not
present, but based on the binary packages instead of the source
packages)? Things that cross my mind might be say .changes that include
only byhand entries or similar, dunno.

> […] For this case, the first line of the field value (the part on the
> +same line as ``Description:``) is always empty. It is a multiline field,
> +with one line per binary package. Each line is indented by one space and
> +contains the name of a binary package, a space, a hyphen (``-``), a space,
> +and the short description line from that package.
>  
>  .. _s-f-Distribution:
>  
> @@ -924,7 +925,8 @@ every architecture. The source control file doesn't 
> contain details of
>  which architectures are appropriate for which of the binary packages.
>  
>  When it appears in a ``.changes`` file, it lists the names of the binary
> -packages being uploaded, separated by whitespace (not commas).
> +packages being uploaded, separated by whitespace (not commas). If only
> +source packages are being uploaded, this field will not be present.

Ditto.

>  
>  .. _s-f-Installed-Size:

Thanks,
Guillem

Bug#823256: debian-policy: Update maintscript arguments with dpkg >= 1.18.5

[Guillem Jover]

Hi!

On Tue, 2022-09-20 at 18:52:22 -0700, Russ Allbery wrote:
> Here is a patch that I believe implements that, and which I think is ready
> for seconds.

> >From 2260f7a3aafe93282860aad07b7d8c1544bcf0ce Mon Sep 17 00:00:00 2001
> From: Russ Allbery 
> Date: Tue, 20 Sep 2022 18:49:04 -0700
> Subject: [PATCH] new-version now passed to more maintainer scripts
> 
> Starting with dpkg 1.18.5, several maintainer script actions
> involving a new package version get that version as an argument
> after the old version. Update Policy's maintainer script
> documentation accordingly.
> ---
>  policy/ch-maintainerscripts.rst | 32 
>  1 file changed, 16 insertions(+), 16 deletions(-)
> 
> diff --git a/policy/ch-maintainerscripts.rst b/policy/ch-maintainerscripts.rst
> index 709aabf..724074c 100644
> --- a/policy/ch-maintainerscripts.rst
> +++ b/policy/ch-maintainerscripts.rst
> @@ -107,8 +107,8 @@ old version of a package that is being upgraded from or 
> downgraded from.
>  The ``preinst`` script may be called in the following ways:
>  
>  | ``new-preinst`` install
> -| ``new-preinst`` install *old-version*
> -| ``new-preinst`` upgrade *old-version*
> +| ``new-preinst`` install *old-version* *new-version*
> +| ``new-preinst`` upgrade *old-version* *new-version*
>  
>  The package will not yet be unpacked, so the ``preinst`` script
>  cannot rely on any files included in its package. Only essential
> @@ -168,10 +168,10 @@ The ``prerm`` script may be called in the following 
> ways:
>  where dependencies are only "Half-Installed" due to a partial
>  upgrade.
>  
> -``new-prerm`` failed-upgrade *old-version*
> -Called during error handling when ``prerm upgrade`` fails. The new 
> package will not yet be
> -unpacked, and all the same constraints as for ``preinst upgrade``
> -apply.
> +``new-prerm`` failed-upgrade *old-version* *new-version*
> +Called during error handling when ``prerm upgrade`` fails. The new
> +package will not yet be unpacked, and all the same constraints as for
> +``preinst upgrade`` apply.
>  
>  The ``postrm`` script may be called in the following ways:
>  
> @@ -189,7 +189,7 @@ The ``postrm`` script may be called in the following ways:
>  the package's dependencies if those dependencies are unavailable.
>  [#]_
>  
> -``new-postrm`` failed-upgrade *old-version*
> +``new-postrm`` failed-upgrade *old-version* *new-version*
>  Called when the old ``postrm upgrade`` action fails. The new package
>  will be unpacked, but only essential packages and pre-dependencies
>  can be relied on. Pre-dependencies will either be configured or will
> @@ -197,8 +197,8 @@ The ``postrm`` script may be called in the following ways:
>  configured and was never removed.
>  
>  | ``new-postrm`` abort-install
> -| ``new-postrm`` abort-install *old-version*
> -| ``new-postrm`` abort-upgrade *old-version*
> +| ``new-postrm`` abort-install *old-version* *new-version*
> +| ``new-postrm`` abort-upgrade *old-version* *new-version*
>  
>  Called before unpacking the new package as part of the error
>  handling of ``preinst`` failures. May assume the same state as
> @@ -229,7 +229,7 @@ These are the "error unwind" calls listed below.
>  
> .. parsed-literal::
>  
> -   new-prerm failed-upgrade *old-version*
> +   new-prerm failed-upgrade *old-version* *new-version*
>  
> If this works, the upgrade continues. If this does not work, the
> error unwind:
> @@ -305,13 +305,13 @@ These are the "error unwind" calls listed below.
>  
> .. parsed-literal::
>  
> -   *new-preinst* upgrade *old-version*
> +   *new-preinst* upgrade *old-version* *new-version*
>  
> If this fails, we call:
>  
> .. parsed-literal::
>  
> -   *new-postrm* abort-upgrade *old-version*
> +   *new-postrm* abort-upgrade *old-version* *new-version*
>  
> i.  If that works, then
>  
> @@ -331,13 +331,13 @@ These are the "error unwind" calls listed below.
>  
> .. parsed-literal::
>  
> -   *new-preinst* install *old-version*
> +   *new-preinst* install *old-version* *new-version*
>  
> Error unwind:
>  
> .. parsed-literal::
>  
> -   *new-postrm* abort-install *old-version*
> +   *new-postrm* abort-install *old-version* *new-version*
>  
> If this fails, the package is left in a "Half-Installed" state,
> which requires a reinstall. If it works, the packages is left in
> @@ -397,7 +397,7 @@ These are the "error unwind" calls listed below.
>  
> .. parsed-literal::
>  
> -   *new-postrm* failed-upgrade *old-version*
> +   *new-postrm* failed-upgrade *old-version* *new-version*
>  
> If this works, installation continues. If not, Error unwind:
>  
> @@ -410,7 +410,7 @@ These are the "error unwind" calls listed below.
>  
> .. parsed-literal::
>  
> -   

Bug#1020248: debian-policy: Clarifying nomenclature for control file names

[Guillem Jover]

Hi!

On Sun, 2022-09-18 at 17:34:57 -0700, Russ Allbery wrote:
> Sean Whitton  writes:
> > On Mon 19 Sep 2022 at 12:45AM +02, Guillem Jover wrote:
> >> So, personally, I'd be happy to fully switch to stanza TBH, because it
> >> seems more specific to our use, probably easier to search for, and
> >> it's shorter.
> 
> > I think this is fine for Policy to do.
> 
> I vote for switching to stanza.  Paragraph is going to be confusing when
> talking about package descriptions, which often have multiple paragraphs
> in the normal English meaning of the term.

Ok, I've prepared the attached incremental patch, which only switches
from paragraph(s) to stanza(s) all over the place.

I've updated all the specs for consistency. I've updated the footnote
to swap the preference and to mention paragraph is now discouraged
nomenclature. I've also updated all «id»s out of consistency, which
might break links, so I can revert that if you'd prefer. And I've
preserved the (upper) casing for one of the titles (“Stand-alone
License Stanza”, although that was not consistent with the other
titles, such as “Files stanza”, I'm happy to lower case that one).

I've gone one by one, but please review carefully as I might have
perhaps switched in excess!

Thanks,
Guillem
From 6d02f28eb1f0cd2f7afa75b04691265425122366 Mon Sep 17 00:00:00 2001
From: Guillem Jover 
Date: Mon, 19 Sep 2022 22:33:40 +0200
Subject: [PATCH] Use stanza to refer to deb822 parts instead of paragraph
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The «stanza» name is a commonly used and understood term when referring
to deb822 blocks. Although «paragraph» is commonly used it has the
problem of being confusing as it then makes it hard to distinguish
actual text paragraphs in prose, while «stanza» is a very specific
term that is not applied anywhere else in the deb822 context, so it's
always more clear and specific.

In addition «stanza» is shorter, which is always a nice attribute
on code for example.

The references in dpkg documentation and code, will be updated shortly,
so that there is uniform nomenclature used.

Fixes: #1020248
---
 autopkgtest.md |   2 +-
 copyright-format-1.0.xml   | 116 -
 policy/ch-controlfields.rst|  46 ++---
 policy/upgrading-checklist.rst |   8 +--
 4 files changed, 87 insertions(+), 85 deletions(-)

diff --git a/autopkgtest.md b/autopkgtest.md
index bc7bdaf..74d6885 100644
--- a/autopkgtest.md
+++ b/autopkgtest.md
@@ -219,7 +219,7 @@ debian/control by adding
 
 XS-Testsuite: autopkgtest
 
-in the `Source:` paragraph.
+in the `Source:` stanza.
 
 Implicit test control file for known package types
 --
diff --git a/copyright-format-1.0.xml b/copyright-format-1.0.xml
index d5d2bbe..954a65b 100644
--- a/copyright-format-1.0.xml
+++ b/copyright-format-1.0.xml
@@ -115,17 +115,17 @@
   The syntax of the file is the same as for other Debian control files, as
   specified in the Debian Policy Manual.  See its https://www.debian.org/doc/debian-policy/ch-controlfields#s-controlsyntax";>section
-  5.1 for details. Extra fields can be added to any paragraph.  No
+  5.1 for details. Extra fields can be added to any stanza.  No
   prefixing is necessary or desired, but please avoid names similar to
   standard ones so that mistakes are easier to catch.  Future versions of
   the debian/copyright specification will attempt to
   avoid conflicting specifications for widely used extra fields.
 
 
-  The file consists of two or more paragraphs.  At minimum, the file
-  must include one header
-  paragraph and one Files
-  paragraph.
+  The file consists of two or more stanzas.  At minimum, the file
+  must include one header
+  stanza and one Files
+  stanza.
 
 
   There are four types of fields.  The definition for each field in this
@@ -184,22 +184,22 @@
 
   
 
-  
-Paragraphs
+  
+Stanzas
 
-  There are three kinds of paragraphs.  The first paragraph in the file
-  is called the header paragraph.
-  Every other paragraph is either a Files paragraph or a stand-alone License
-  paragraph.  This is similar to source and binary package
-  paragraphs in debian/control files.
+  There are three kinds of stanzas.  The first stanza in the file
+  is called the header stanza.
+  Every other stanza is either a Files stanza or a stand-alone License
+  stanza.  This is similar to source and binary package
+  stanzas in debian/control files.
 
 
-
-  Header paragraph (once)
+
+  Header stanza (once)
   
-The following fields may be present in a header paragraph.
+The following fields may be present in a header 

Bug#1020241: debian-policy: copyright-format: Formatting improvements/changes

[Guillem Jover]

On Sun, 2022-09-18 at 18:01:38 -0700, Russ Allbery wrote:
> Guillem Jover  writes:
> 
> > Oh! I've completely missed this all this time, I think because that
> > feels very weird given that it has no synopsis and the text is added
> > already on the first line on the spec. :/
> 
> Other formatted fields with the same semantics are Source, Disclaimer, and
> Comment.  I don't think there are any fields in debian control files with
> those semantics (Description is the only formatted field and it has a
> synopsis), but there are several of them in copyright files.
> 
> Source is another ongoing minor problem, since it's *usually* a URL but is
> not required to be one, and sometimes a textual description of the source
> is needed.  Here too, a structured format would have been nicer, so that
> you could have something like:
> 
> source:
>   urls:
> - https://example.com/foo
> - https://example.org/foo
>   comment: >-
> The foo-rewrite script was originally posted to comp.unix.sources in
> 1992 but otherwise has no source other than the Debian package.

I think Disclaimer and Comment do not seem as problematic because they
tend to contain descriptive prose. For Source it's true that it's
weird as it seems to indeed want to have two different semantics
depending on the content, and considering the current deb822 format,
perhaps having used two different field names might have been better
as you alluded with your YAML example, say Source-Urls (line-based list),
and Source-Comment (formatted text). Such split still seems feasible
and backwards compatible right now though.

But see below.

> > Right, the problem I see is that applying this formatting to a field
> > that has no special treatment for the first line just after the field
> > name seems very unintuitive, because the first line does not contain an
> > additional prefixing space, or if it does no one is adding it!
> 
> > It feels very weird to me that all these would be equivalent:
> 
> >   Copyright: Something long that might trigger some wrapping behavior
> > Other thing very long that might not be clear behaves as the above
> > More
> 
> > and
> 
> >   Copyright:  Something long that might trigger some wrapping behavior
> > Other thing very long that might not be clear behaves as the above
> > More
> 
> > and
> 
> >   Copyright:
> > Something long that might trigger some wrapping behavior
> > Other thing very long that might not be clear behaves as the above
> > More
> 
> I think my brain just assumes that all whitespace after the colon of a
> field name and before the first non-whitespace character is ignored, so
> doesn't have a problem with that, but I can see why it would be confusing.

Just to try to clarify to make sure we are on the same page (if we are,
sorry for the obvious!). What I find confusing is that the semantics
of the field imply different line-wrapping semantics depending on leading
spaces, and because there's no synopsis, the first line is supposed to
act just like the rest, but if spaces are ignored, then how do you
select either of the line-wrapping behaviors for the first line? Also
because adding such spaces after the colon look like typographic errors
to me somehow.

So I think what seems most confusing to me is that for formatted-text
fields with no synopsis, the first line is being used at all, because
that messes with the intuition on how the Description field operates.

> > Otherwise, if the current semantics are retained, at least for me, the
> > first line behavior really needs to be clarified.
> 
> Yes, we should distinguish between formatted text with synopsis and
> formatted text without synopsis more clearly.

Yes.

> Or, you know, just propose
> a new YAML format which would make it trivial to clean up all of these
> problems *and* would provide first-class editor support and easy parsing
> in every major programming language.  :)  But that's WAY bigger than this
> bug.

Ahem, yeah. :)

> > If we end up switching the field semantics, that seems it might cause
> > unnecessary modification churn, given that I (not sure whether other
> > people have done this before than me as well) at least have "instigated"
> > unintentionally this type of change in several places (packages I
> > maintain, golang/prometheus team), including tooling (AFAIR dh-make and
> > dh-make-golang), and other people might have also picked this up too. :/
> 
> I think making the field a line-based list is the obviously correct thing
> to do.  It's just not backward-compatible, so we will have to face the
> question of how we handle a version bump in the c

Bug#1020241: debian-policy: copyright-format: Formatting improvements/changes

[Guillem Jover]

On Sun, 2022-09-18 at 18:04:00 -0700, Russ Allbery wrote:
> Guillem Jover  writes:
> > BTW, just to make this clear, if this feels like it might not be decided
> > quickly on the Debian policy side, then I'll prepare/commit changes to
> > revert this behavior from tooling that I've previously introduced, until
> > and if this changes again. Otherwise if the feeling is that this might
> > get decided quickly, then I'd prefer to avoid the flip-flopping behavior
> > change (but not to be taken as "current-practice" pressure to swindle
> > the decision! And I'm happy to do it in this case anyway if that makes
> > people feel better about it).
> 
> I have an entirely unpredictable schedule for Debian work, unfortunately,
> so I truly don't have any idea how long this will take.

No problem. Then in that case I guess I'll try to prepare/commit such
revert patches for the tooling during this week or so.

Thanks,
Guillem

Bug#1020248: debian-policy: Clarifying nomenclature for control file names

[Guillem Jover]

On Sun, 2022-09-18 at 14:53:30 -0700, Sean Whitton wrote:
> On Sun 18 Sep 2022 at 10:28PM +02, Guillem Jover wrote:
> 
> > So, how does «source package paragraph» and «binary package paragraph»
> > (of the «template control file») sound instead?
> 
> Can we standardise on 'stanza', please?
>
> I thought that was already standard, and "paragraph" is for prose.

I was also thinking about whether I'd prefer paragraph or stanza, and
the latter seems more specific to deb822 "blocks", and as you say
paragraph seems more for prose.

I went for paragraph, because dpkg has some instances of it already in
docs and code (and stanza only in code), and mainly because the Debian
policy uses almost exclusively paragraph for this with a single
mention of "stanza" in a footnote to mention it's a common alias or
similar.

So, personally, I'd be happy to fully switch to stanza TBH, because
it seems more specific to our use, probably easier to search for, and
it's shorter.

Thanks,
Guillem

Bug#1020241: debian-policy: copyright-format: Formatting improvements/changes

[Guillem Jover]

Hi!

On Sun, 2022-09-18 at 22:56:16 +0200, Guillem Jover wrote:
> On Sun, 2022-09-18 at 10:58:20 -0700, Russ Allbery wrote:
> > Russ Allbery  writes:
> > > I would happily apply a version of 0002 that only changes Files and
> > > leaves Copyright alone.
> 
> I can split that up, for an incremental update yes. Will send later.

Attached.

> > Or, perhaps even better, one that changes Copyright the way that you did,
> > but just adds an extra space.  I think that's the simplest compromise
> > between what you're trying to accomplish and the field definition.
> 
> If we end up switching the field semantics, that seems it might cause
> unnecessary modification churn, given that I (not sure whether
> other people have done this before than me as well) at least have
> "instigated" unintentionally this type of change in several places
> (packages I maintain, golang/prometheus team), including tooling
> (AFAIR dh-make and dh-make-golang), and other people might have also
> picked this up too. :/

BTW, just to make this clear, if this feels like it might not be
decided quickly on the Debian policy side, then I'll prepare/commit
changes to revert this behavior from tooling that I've previously
introduced, until and if this changes again. Otherwise if the feeling
is that this might get decided quickly, then I'd prefer to avoid the
flip-flopping behavior change (but not to be taken as "current-practice"
pressure to swindle the decision! And I'm happy to do it in this case
anyway if that makes people feel better about it).

Thanks,
Guillem
From 6c74ff53624595267215405edaf09ab3146d5b93 Mon Sep 17 00:00:00 2001
From: Guillem Jover 
Date: Sun, 18 Sep 2022 18:10:59 +0200
Subject: [PATCH] copyright-format: Wrap and sort -sat Files field

Do not place entries on the first line with the field name, and place
one item per line, so that adding or removing entries generates as
minimal a diff as possible, and avoids modifying unrelated lines. Use
a single space so that the indentation is always uniform among all
fields.
---
 copyright-format-1.0.xml | 42 ++--
 debian/copyright | 18 +++--
 2 files changed, 40 insertions(+), 20 deletions(-)

diff --git a/copyright-format-1.0.xml b/copyright-format-1.0.xml
index 0f86a76..d5d2bbe 100644
--- a/copyright-format-1.0.xml
+++ b/copyright-format-1.0.xml
@@ -316,19 +316,23 @@ Upstream-Contact: John Doe <john@example.com>
 
   
 Example files paragraphs
-Files: *
+Files:
+ *
 Copyright: 1975-2010 Ulla Upstream
 License: GPL-2+
 
-Files: debian/*
+Files:
+ debian/*
 Copyright: 2010 Daniela Debianizer
 License: GPL-2+
 
-Files: debian/patches/fancy-feature
+Files:
+ debian/patches/fancy-feature
 Copyright: 2010 Daniela Debianizer
 License: GPL-3+
 
-Files: */*.1
+Files:
+ */*.1
 Copyright: 2010 Manuela Manpager
 License: GPL-2+
 
@@ -401,12 +405,14 @@ License: LGPL-2.1
 
   
 recurrent license
-Files: src/js/editline/*
+Files:
+ src/js/editline/*
 Copyright: 1993, John Doe
1993, Joe Average
 License: MPL-1.1
 
-Files: src/js/fdlibm/*
+Files:
+ src/js/fdlibm/*
 Copyright: 1993, J-Random Corporation
 License: MPL-1.1
 
@@ -1261,7 +1267,8 @@ Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
 Source: ftp://ftp.example.com/pub/games
 Upstream-Name: X Solitaire
 
-Files: *
+Files:
+ *
 Copyright: 1998 John Doe 
1998 Jane Smith 
 License: GPL-2+
@@ -1298,39 +1305,46 @@ Source: https://www.example.com/code/venus
 Upstream-Name: Planet Venus
 Upstream-Contact: John Doe 
 
-Files: *
+Files:
+ *
 Copyright: 2008, John Doe 
2007, Jane Smith 
2007, Joe Average 
2007, J. Random User 
 License: PSF-2
 
-Files: debian/*
+Files:
+ debian/*
 Copyright: 2008, Dan Developer 
 License: permissive
  Copying and distribution of this package, with or without modification,
  are permitted in any medium without royalty provided the copyright notice
  and this notice are preserved.
 
-Files: debian/patches/theme-diveintomark.patch
+Files:
+ debian/patches/theme-diveintomark.patch
 Copyright: 2008, Joe Hacker 
 License: GPL-2+
 
-Files: planet/vendor/compat_logging/*
+Files:
+ planet/vendor/compat_logging/*
 Copyright: 2002, Mark Smith 
 License: MIT
  [LICENSE TEXT]
 
-Files: planet/vendor/httplib2/*
+Files:
+ planet/vendor/httplib2/*
 Copyright: 2006, John Brown 
 License: MIT2
  Unspecified MIT style license.
 
-Files: planet/vendor/feedparser.py
+Files:
+ planet/vendor/feedparser.py
 Copyright: 2007, Mike Smith 
 License: PSF-2
 
-Files: planet/vendor/htmltmpl.py
+Files:
+ planet/vendor/htmltmpl.py
 Copyright: 2004, Thomas Brown 
 License: GPL-2+
 
diff --git a/debian/copyright b/debian/copyright
index 357ae48..21a514b 100644
--- a/debian/copyright
+++ b/debian/copyright
@@ -7,7 +7,8 @@ Comment: Complete copyright notic

Bug#1020241: debian-policy: copyright-format: Formatting improvements/changes

[Guillem Jover]

Hi!

On Sun, 2022-09-18 at 10:42:28 -0700, Russ Allbery wrote:
> Guillem Jover  writes:
> 
> > These are the set of changes I keep doing to the debian/copyright files
> > I end up touching. While some could be characterized as a subjective
> > style issue, I've tried to give as close as possibly objective :)
> > rationale for every and each of the changes in the commit messages which
> > I'll summarize here.
> 
> Thanks!  I have applied these changes for the next release except for
> patch 0002 (applying wrap-and-sort -sat).
> 
> I agree with patch 0002 for Files, but unfortunately I believe it's
> incorrect for Copyright.  You're treating Copyright as if it were a
> line-based list field, but for better or worse (I think probably for
> worse) the current specification says that it's a formatted field.

Oh! I've completely missed this all this time, I think because that
feels very weird given that it has no synopsis and the text is added
already on the first line on the spec. :/

> This means that your change to, for example:
> 
> Copyright:
>  2008 John Doe 
>  2007 Jane Smith 
>  2007 Joe Average 
>  2007 J. Random User 
> 
> means that the semantic content of that field is now:
> 
>  2008 John Doe  2007 Jane Smith 
>  2007 Joe Average  2007 J. Random User
>  
> 
> and a format-aware display engine should show it as such.  This is the
> reason why lines are indented: that makes them verbatim lines per the
> formatting specification in
> https://www.debian.org/doc/debian-policy/ch-controlfields#s-f-description

Right, the problem I see is that applying this formatting to a field
that has no special treatment for the first line just after the field
name seems very unintuitive, because the first line does not contain
an additional prefixing space, or if it does no one is adding it!

It feels very weird to me that all these would be equivalent:

  Copyright: Something long that might trigger some wrapping behavior
Other thing very long that might not be clear behaves as the above
More

and

  Copyright:  Something long that might trigger some wrapping behavior
Other thing very long that might not be clear behaves as the above
More

and

  Copyright:
Something long that might trigger some wrapping behavior
Other thing very long that might not be clear behaves as the above
More

> I would agree with changing the definition of Copyright to a line-based
> list, although in order to do so we'd have to figure out the implications
> of a format change in the specification, and we should also flesh out the
> definition of a line-based list to explain how to handle a line that's
> longer than the normal wrap length.

Right, I'd prefer this too, but obviously that is more involved than
what this report intended to be. So I guess it should be detangled
into another request.

Otherwise, if the current semantics are retained, at least for me, the
first line behavior really needs to be clarified.

On Sun, 2022-09-18 at 10:58:20 -0700, Russ Allbery wrote:
> Russ Allbery  writes:
> > I would happily apply a version of 0002 that only changes Files and
> > leaves Copyright alone.

I can split that up, for an incremental update yes. Will send later.

> Or, perhaps even better, one that changes Copyright the way that you did,
> but just adds an extra space.  I think that's the simplest compromise
> between what you're trying to accomplish and the field definition.

If we end up switching the field semantics, that seems it might cause
unnecessary modification churn, given that I (not sure whether
other people have done this before than me as well) at least have
"instigated" unintentionally this type of change in several places
(packages I maintain, golang/prometheus team), including tooling
(AFAIR dh-make and dh-make-golang), and other people might have also
picked this up too. :/

Thanks,
Guillem

Bug#1020248: debian-policy: Clarifying nomenclature for control file names

[Guillem Jover]

Package: debian-policy
Version: 4.6.1.1
Severity: wishlist

Hi!

This is a followup from my comment at:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=998165#43

To summarize, we have IMO confusing naming and nomenclature for the
various control files and paragraphs/stanzas, and this is even
confusing me when having to deal with dpkg code, so I'd like to give
these more clear and unambiguous new names, and I'd very strongly
prefer to agree on the same naming for Debian policy and dpkg, to
avoid further and worse confusion (even though they currently do not
match exactly anyway, but I'd prefer to not make it worse…).

Just for reference and to give some context, I've got the following
WIP branches, trying to clarify the names in documentation and in the
API on, which I'll probably rework (split/merge) and reword as needed,
so do not take them as anything set in stone:

  
https://git.hadrons.org/git/debian/dpkg/dpkg.git/log/?h=next/clarify-control-filenames
  
https://git.hadrons.org/git/debian/dpkg/dpkg.git/log/?h=next/deb822-field-types


File descriptions
-

For example we have:

  * debian/control:
policy → «Source package control file»
dpkg   → «Debian source packages' master control file»

  * .dsc:
policy → «Debian source control file»
dpkg   → «Debian source packages' control file»

  * DEBIAN/control
policy → «Binary package control files»
dpkg   → «Debian binary packages' master control file»

These are quite confusingly close.

I've been considering naming debian/control something like
«Debian template source package control file», as that is used to
generate both the source and binary control files. And always
prefixing with Debian, so that would end up as:

  * debian/control: «Debian source package template control file»
  * .dsc:   «Debian source package control file»
  * DEBIAN/control: «Debian binary package control file»

This also removes the «master» usage in dpkg, for me for the same
reasons as I covered at
.


File contents
-

We have references to the various parts being called as «paragraphs»,
«stanza», «blocks», but this seems to be more of an issue with dpkg, as
the usage in the Debian policy is quite clear and uniform now, so I'll
at least try to remove the «block» usage there, stanza has the nice
property of being shorter and policy already mentions that this is
currently a common alias, so I might keep paragraph and stanza for now
in dpkg.

The other thing affecting dpkg and debian-policy is how the parts
within the control files are referred to. We have for example:

  dpkg   → «general section of control info file»
   «source stanza»
  policy → «general paragraph»

  dpkg   → «package's section of control info file»
  policy → «binary package paragraphs»


So, how does «source package paragraph» and «binary package paragraph»
(of the «template control file») sound instead?


If I've missed any other problematic nomenclature, I'm happy to
discuss and update those on the dpkg side.

Thanks,
Guillem

Bug#1020243: debian-policy: Use OpenPGP instead of PGP

[Guillem Jover]

Package: debian-policy
Version: 4.6.1.1
Severity: minor
Tags: patch

Hi!

Another minor patch I had laying around, switching references to the
OpenPGP specification instead of to the specific PGP implementation.

Thanks,
Guillem
From 4dd6c1ef7c487fe2cd293e8fbddfdc898e0e9199 Mon Sep 17 00:00:00 2001
From: Guillem Jover 
Date: Thu, 23 Dec 2021 07:11:55 +0100
Subject: [PATCH] Use OpenPGP instead of PGP

The standard is called OpenPGP, PGP instead is a specific
implementation. And while depending on the context (such as filename
extensions) using .pgp is better and more neutral than using some
other implementation specific extension (such as .gpg), in this context
the text refers to the generic OpenPGP signatures.
---
 policy/ch-controlfields.rst | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/policy/ch-controlfields.rst b/policy/ch-controlfields.rst
index daff424..533abba 100644
--- a/policy/ch-controlfields.rst
+++ b/policy/ch-controlfields.rst
@@ -212,7 +212,7 @@ The fields in this file are:
 Debian source control files -- ``.dsc``
 ---
 
-This file consists of a single paragraph, possibly surrounded by a PGP
+This file consists of a single paragraph, possibly surrounded by an OpenPGP
 signature. The fields of that paragraph are listed below. Their syntax
 is described above, in :ref:`s-controlsyntax`.
 
@@ -261,7 +261,7 @@ Debian changes files -- ``.changes``
 
 The ``.changes`` files are used by the Debian archive maintenance
 software to process updates to packages. They consist of a single
-paragraph, possibly surrounded by a PGP signature. That paragraph
+paragraph, possibly surrounded by a OpenPGP signature. That paragraph
 contains information from the ``debian/control`` file and other data
 about the source package gathered via ``debian/changelog`` and
 ``debian/rules``.
-- 
2.37.2

Bug#1020241: debian-policy: copyright-format: Formatting improvements/changes

[Guillem Jover]

Package: debian-policy
Version: 4.6.1.1
Severity: wishlist
Tags: patch

Hi!

These are the set of changes I keep doing to the debian/copyright
files I end up touching. While some could be characterized as a
subjective style issue, I've tried to give as close as possibly
objective :) rationale for every and each of the changes in the
commit messages which I'll summarize here.

The main drive is for uniformity in formatting (compared to the
recommended GPL notice example), minimization of diff delta on changes
(the same we seem to be converging with wrap-and-sort -sat), making
the notice future-proof (by using an URL instead of a postal address),
and moving the Debian-specific filesystem location reference into a
Comment to not confuse with the actual upstream/in-code notice. Then
there's the placement of the Source field, which I guess is the
possibly more style/subjective one, but that's one that seems to also
be triggering some OCDish button or similar. :)

This change was implemented on top of the spacing and typographical
patches and seems to depend on changes in there.

Thanks,
Guillem
From 600aabb1a2235396db5fce4240ac0751258fcf7f Mon Sep 17 00:00:00 2001
From: Guillem Jover 
Date: Sun, 18 Sep 2022 18:05:57 +0200
Subject: [PATCH 1/6] copyright-format: Place Source field after Format field

These both contain URLs, are the most important references, and nicely
align, so it's nice to get them as early as possible in the file.
---
 copyright-format-1.0.xml | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/copyright-format-1.0.xml b/copyright-format-1.0.xml
index d29c490..3d5c672 100644
--- a/copyright-format-1.0.xml
+++ b/copyright-format-1.0.xml
@@ -271,9 +271,9 @@
   
 Example header paragraph
 Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
+Source: https://www.example.com/software/project
 Upstream-Name: SOFTware
-Upstream-Contact: John Doe <john@example.com>
-Source: https://www.example.com/software/project
+Upstream-Contact: John Doe <john@example.com>
   
 
 
@@ -1265,8 +1265,8 @@ also delete it here.
 package):
 
   
 
@@ -1339,10 +1329,9 @@ Files:
 Copyright:
  2008 Dan Developer 
 License: permissive
- Copying and distribution of this package, with or without
- modification, are permitted in any medium without royalty
- provided the copyright notice and this notice are
- preserved.
+ Copying and distribution of this package, with or without modification,
+ are permitted in any medium without royalty provided the copyright notice
+ and this notice are preserved.
 
 Files:
  debian/patches/theme-diveintomark.patch
@@ -1380,26 +1369,22 @@ License: PSF-2
  [LICENSE TEXT]
 
 License: GPL-2+
- This program is free software; you can redistribute it
- and/or modify it under the terms of the GNU General Public
- License as published by the Free Software Foundation; either
- version 2 of the License, or (at your option) any later
- version.
+ This program is free software; you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation; either version 2 of the License, or
+ (at your option) any later version.
  .
- This program is distributed in the hope that it will be
- useful, but WITHOUT ANY WARRANTY; without even the implied
- warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR
- PURPOSE.  See the GNU General Public License for more
- details.
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ GNU General Public License for more details.
  .
- You should have received a copy of the GNU General Public
- License along with this package; if not, write to the Free
- Software Foundation, Inc., 51 Franklin St, Fifth Floor,
- Boston, MA  02110-1301 USA
+ You should have received a copy of the GNU General Public License along
+ with this package; if not, write to the Free Software Foundation, Inc.,
+ 51 Franklin St, Fifth Floor, Boston, MA  02110-1301 USA.
  .
- On Debian systems, the full text of the GNU General Public
- License version 2 can be found in the file
- '/usr/share/common-licenses/GPL-2'.]]>
+ On Debian systems, the full text of the GNU General Public License
+ version 2 can be found in the file '/usr/share/common-licenses/GPL-2'.]]>
   
 
   
diff --git a/debian/copyright b/debian/copyright
index eef116a..d80be52 100644
--- a/debian/copyright
+++ b/debian/copyright
@@ -99,15 +99,15 @@ Copyright:
 License: GPL-2+
 
 License: GPL-2+
- This manual is free software; you may redistribute it and/or modify it
- under the terms of the GNU General Public License as published by the
- Free Software Foundation; either version 2 of the License, or (at your
- option) any later version.
+ This manual is free software; yo

Bug#1020238: debian-policy: Spacing an typographical cleanups

[Guillem Jover]

Package: debian-policy
Version: 4.6.1.1
Severity: minor

Hi!

I'm attaching a few patches fixing spacing and typographical issues.
For the quotes fix, I personally highly prefer UTF-8 character pairs
such as «», “”, ‘’, but went with the conservative ASCII ones '' as
that tends to cause less opposition. But I'm happy to convert these to
some of the UTF-8 ones if you prefer.

Thanks,
Guillem
From a367e8cd6dd50c4304978c07d3823826bfb61365 Mon Sep 17 00:00:00 2001
From: Guillem Jover 
Date: Wed, 5 Jan 2022 02:49:28 +0100
Subject: [PATCH 1/3] Remove trailing whitespace

---
 Makefile|  2 +-
 README.md   |  2 +-
 debconf/commands.xml|  2 +-
 debconf/priorities.xml  |  2 +-
 debian/changelog| 48 
 historical/README.shlibdeps | 20 +-
 historical/libc6-migration.txt  | 56 ++--
 policy/ap-license.rst   |  2 +-
 policy/index.rst|  2 +-
 virtual-package-names-list.yaml | 66 -
 10 files changed, 101 insertions(+), 101 deletions(-)

diff --git a/Makefile b/Makefile
index 5640fb8..0c8765d 100644
--- a/Makefile
+++ b/Makefile
@@ -299,4 +299,4 @@ $(XML_FILES:=.txt) $(XML_SINGLE_FILES:=.txt) $(XML_SPLIT_FILES:=.txt): \
 .DELETE_ON_ERROR:
 
 # No default suffixes work here, don't waste time on them.
-.SUFFIXES: 
+.SUFFIXES:
diff --git a/README.md b/README.md
index 2cf6637..fcbd8b9 100644
--- a/README.md
+++ b/README.md
@@ -113,7 +113,7 @@ as possible for others to review your work.
for your change.  If your change is particularly large, it might be
more readable not to use `--word-diff=plain`, but usually the word
diff is better.
-   
+
 Do not quote the output -- many people have mail readers which
 will colorise the diff if it is left unmodified.
 
diff --git a/debconf/commands.xml b/debconf/commands.xml
index ca73e28..490e1be 100644
--- a/debconf/commands.xml
+++ b/debconf/commands.xml
@@ -78,7 +78,7 @@
   Using a template has the advantage that titles are translatable and
   that they can be maintained in the same place as other text
   displayed to users.
- 
+
   
   
 
diff --git a/debconf/priorities.xml b/debconf/priorities.xml
index 114a9c7..7730491 100644
--- a/debconf/priorities.xml
+++ b/debconf/priorities.xml
@@ -24,7 +24,7 @@
   
 	high
 	
-  Items that don't have a reasonable 
+  Items that don't have a reasonable
 	  default.
 	
   
diff --git a/debian/changelog b/debian/changelog
index bc08838..dcb9de8 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1366,7 +1366,7 @@ debian-policy (3.9.6.0) unstable; urgency=low
 Seconded: Tony Mancill 
 Seconded: Bill Allombert 
 Closes: #754876
-  * virtual-package-names-list: Add httpd-wsgi 
+  * virtual-package-names-list: Add httpd-wsgi
 Wording: Bill Allombert 
 Seconded: Jonas Smedegaard 
 Seconded: Piotr Ożarowski 
@@ -1955,9 +1955,9 @@ debian-policy (3.9.0.0) unstable; urgency=low
 Seconded:  Julien Cristau  
 Seconded:  Gregor Herrmann 
 Closes: #566220
-  * extend UID range of user accounts by removing the 3-5 reserved 
+  * extend UID range of user accounts by removing the 3-5 reserved
 ranges.
-Proposed by Santiago Vila  
+Proposed by Santiago Vila
 Seconded:  Russ Allbery
 Seconded:  Luk Claes   
 Seconded:  Raphael Hertzog 
@@ -2093,25 +2093,25 @@ debian-policy (3.8.4.0) unstable; urgency=low
   [ Manoj Srivastava ]
   * [b270d2d]: Typo fix: relayed -> related. Thanks to Matt Kraai for
 pointing this out.
-  * [c74ac74]: 
+  * [c74ac74]:
 Policy: Grant an FHS exception for the multiarch library directories
 Wording: Steve Langasek.
 Seconded: Aurelien Jarno
 Seconded: Julien Cristau
 Seconded: Kurt Roeckx
 Closes: #542865
-  * [7ac3ee6]: 
+  * [7ac3ee6]:
 virtual package list: Added Doom virtual packages
 Wording: Manoj Srivastava
-Seconded: Russ Allbery 
+Seconded: Russ Allbery
 Seconded: Giacomo A. Catenazzi
 Closes: #518199
   * [8fd91a0]
 README Process upgrading-checklist: Created/converted to org-mode
 Wording: Manoj Srivastava
-Seconded: Russ Allbery 
+Seconded: Russ Allbery
 Closes: #545548
-  * [4da0692]: [typo-fixes]: 
+  * [4da0692]: [typo-fixes]:
 policy: Fix a number of grammatical or typographical errors
 wording: Eric Dantan Rzewnicki
 Seconded: Manoj Srivastava
@@ -2145,7 +2145,7 @@ debian-policy (3.8.4.0) unstable; urgency=low
  - Support of crontab entries with names for days and months,
ranges, step values
  - Correct execution of /etc/cron.{hourly,daily,weekly,monthly}
-
+
   [ Russ Allbery ]
   * Policy: Clarify policy on named pipes in packages
 Wording: Russ Allbery 
@@ -2286,8 +2286,8 @@ debian-policy (3.8.2.0) unstable; urgency=low
  

Bug#1002626: debian-policy: building packages should not require to be root

[Guillem Jover]

On Sat, 2021-12-25 at 18:45:08 -0800, Russ Allbery wrote:
> Vincent Lefevre  writes:
> > On 2021-12-25 14:48:33 -0800, Russ Allbery wrote:
> >> Vincent Lefevre  writes:
> >>> Here, the build via "debuild" is failing even when fakeroot is
> >>> available (installed on the machine). Note that Rules-Requires-Root
> >>> has been set to "no". IMHO, the policy should say that when
> >>> Rules-Requires-Root is set to "no", being root or using fakeroot
> >>> should not be required.
> 
> >> It does already.
> 
> >> no: Declares that neither root nor fakeroot is required. Package
> >> builders (e.g. dpkg-buildpackage) may choose to invoke any target in
> >> debian/rules with an unprivileged user.
> 
> >> Am I missing something?
> 
> > According to Sean, this is just advisory (and Scott Kitterman seemed
> > to assume that a build failure as non-root[*] was not a RC bug).
> 
> I don't understand what "advisory" means here.  This field controls the
> behavior of the package building software.  If the package says that root
> isn't required, the package will be built without root.  If root turns out
> to be required, the package will FTBFS.  There's nothing "advisory" about
> having inaccurate package metadata that causes FTBFS, surely?

I did not understand Sean reply either TBH. Also rereading the policy
description of this, seems to me has somewhat lost some of the nuance
from the spec in dpkg (/usr/share/doc/dpkg/rootless-builds.txt.gz).

Neither I understood the comment about that being related to the field
being new.

> Presumably the question is about the severity of the bug, but I don't
> think the severity question has anything to do with the Policy wording or
> would change if we worded Policy differently.  The package says that you
> don't have to run it as root, so an autobuilder that knows about
> Rules-Requires-Root won't run the build as root, the build will fail, and
> that's a FTBFS bug, regardless of what Policy says.  Presumably Lucas
> would report it as such if his builder supports Rules-Requires-Root.

But, the builder in this context is the program driving debian/rules
and not any external wrappers, in this case dpkg-buildpackage, which
has honored the field since it got implemented in 1.19.0. We drafted
it as "the builder" to allow for other potential drivers, because we
are still considering debian/rules the canonical entry point (even
though I still think we should ideally stop supporting calling it
directly, and instead should make dpkg-buildpackage the only supported
interface).

> […] It looks like that's not the case, so I think this was
> just a bog-standard FTBFS, only a bit surprising because it was triggered
> by honoring Rules-Requires-Root, which I'm not sure the buildds do (yet).

The buildds have "honored" R³ since dpkg-buildpackage does.

Thanks,
Guillem

Bug#998165: debian-policy: document and allow Description in the source paragraph

[Guillem Jover]

On Tue, 2021-12-21 at 17:53:31 -0700, Sean Whitton wrote:
> On Sun 12 Dec 2021 at 06:47PM +01, Mattia Rizzolo wrote:
> 
> > |--- a/policy/ch-controlfields.rst
> > |+++ b/policy/ch-controlfields.rst
> > |@@ -652,9 +654,14 @@ orderings.  [#]_
> > | ~~~
> > |
> > | In a source or binary control file, the ``Description`` field contains a
> > |-description of the binary package, consisting of two parts, the synopsis
> > |-or the short description, and the long description. It is a multiline
> > |-field with the following format:
> > |+description of the package, consisting of two parts, the synopsis or the 
> > short
> > |+description, and the long description.
> > |+
> > |+When used in a source control file in the general paragraph (i.e., the 
> > first
> > |+one, for the source package), the text in this field is relevant for all 
> > binary
> > |+packages built by the given source package.
> 
> Is there really no name for the first paragraph other than "general
> paragraph"?

That's how the dpkg documentation (man and perl modules POD) refers to
it (or first block of information, which is even worse), but I agree
it's rather suboptimal, and I'd like to get a better name for it. See
below.

> Maybe "the source package's stanza"?

Something like this might work, which is probably what we have been
calling it for some time now, but that ties up with something that has
been bothering me for some time now, and that is that I've found our
naming of the various stanzas and the various control filenames rather
confusingly similar, which for dpkg I'd really like to clear up (and
have few tentative commits already), as that's even affecting its perl
API currently. :/

For example we have «Debian source control file» or «Debian source
packages' control file» for .dsc, then we have «Source package control
file» or in dpkg «Debian source packages' master control file» for
debian/control. Which are almost the same. I've been considering naming
debian/control something like «Debian template source control file», as
that is used to generate both the source and binary control files.

But I think I'll open a new bug to cover and discuss that.

> Also, how about "the text in this field describes all binary packages
> which do not have their own Description: fields" ?

I'm not sure whether you are (or the text would then) imply this; but
the Description in the source stanza does not get inherited by the
binary stanzas when generating the binary package control file, one
needs to add references to it via substvars.

Thanks,
Guillem

Bug#983304: please document "Protected" field

[Guillem Jover]

Hi!

On Mon, 2021-02-22 at 11:30:08 +0100, Julian Andres Klode wrote:
> On Mon, Feb 22, 2021 at 09:23:00AM +0100, Tomas Pospisek wrote:
> > Source: debian-policy
> > Version: 4.5.1.0
> > Severity: wishlist

> > In Julian Andres Klode's blog I've [1] glimpsed:
> > 
> > > New features
> > > [...]
> > > The Protected field is now supported. It replaces the previous Important
> > > field and is like Essential, but only for installed packages (some minor
> > > more differences maybe in terms of ordering the installs).
> > 
> > So I've tried to find out what the "Protected" field is for. The only
> > info about it that I could find is from `man 1 dpkg`:

> You can also read the spec [1]
> 
> [1] https://wiki.debian.org/Teams/Dpkg/Spec/ProtectedField

I've updated that to point to the RFC thread on debian-devel and to
the file shipped by dpkg (/usr/share/doc/dpkg/protected-field.txt),
which are both better resources. In general once these get implemented
in dpkg, the wiki spec pages have stopped being the core canonical
source of information, and are kept for historical purposes or to
keep information that is out-of-scope for dpkg itself (such as
Debian-specific policy or implementation tracking, etc). But I guess
that should be clearly reflected there or the pages kept in sync. I'll
try to do both, and perhaps incorporate any of the following paragraphs,
that might no be already covered from information in the pre-existing
docs once merged.

> What you use them for is obviously not fixed. What we have so far are:
> 
> ## Essential for some use cases
> 
> The init package uses the old Important: yes to be harder to remove so
> you don't end up bricking your system into an unbootable one.
> 
> That's the "Essential only for some use cases" use case, because init
> obviously should not be Essential because we don't want it in an app
> container, but we do want it to be Essential when it's already installed.
> 
> The same applies to e2fsprogs, which is also marked like that, because
> duh, you do not want to be able to remove e2fsprogs on a real system,
> but again you don't want it installed automatically in a container.
> 
> 
> ## Weird hacks in low-level packages
> 
> As another example, we marked libgcc-s1 as Protected, because it Replaces:
> libgcc1 but can't Conflict with it, so removing libgcc-s1 would remove
> the library entirely, so we had to protect it.
> 
> ## Local metapackages
> 
> The original use case was local system configuration meta packages,
> where you define a my-machine-foo meta package, and have it depend on
> all packages in the system and mark that as Protected: yes[2]; back then
> we used XB-Important: yes (well we still do, you gotta set both for
> backwards compatibility with stable atm really). Because you then mark
> all other packages auto and you then absolutely do not want to lose your
> meta package :D
> 
> [2] 
> https://blog.jak-linux.org/2012/01/24/managing-system-package-selections-using-custom-meta-packages/
> 
> ## De-essentializing Essential packages
> 
> Packages that were Essential can be downgraded to Protected and
> installed systems won't show any change in behavior (avoiding
> regressions), whereas the package will no longer be "essential"
> on new systems.

Thanks,
Guillem

Bug#980825: debian-policy: Historical sign off dates in d/changelog and "single digit" day of the month

[Guillem Jover]

Hi!

On Fri, 2021-01-22 at 22:15:24 +0100, Niels Thykier wrote:
> Package: debian-policy
> Version: 4.5.0.0
> Severity: minor
> 
> This is a bit of a nit pick, but I think it is a special case worth
> mentioning in Policy.
> 
> I am basing this on
> https://www.debian.org/doc/debian-policy/ch-source.html#debian-changelog-debian-changelog
> where the date format of the changelog signoff line is described as:
> 
> 
> > The date has the following format 7 (compatible and with the same semantics 
> > of RFC 2822 and RFC 5322):
> > 
> > day-of-week, dd month  hh:mm:ss +
> > 
> > where:
> > 
> > day-of-week is one of: Mon, Tue, Wed, Thu, Fri, Sat, Sun
> > 
> > dd is a one- or two-digit day of the month (01-31)
> > [...]
> 
> I find that "single-digit day" is a bit underspecified here in.
> Basically there are two options, either the leading zero is replaced
> with a leading space or the leading zero is simply omitted.
> 
> Sadly, neither RFC 2822 nor RFC 5322 are helpful in clearing this up as
> they both assume "two-digit" days.
> 
> My understanding is that the reason for "single-digit" days is to
> support historical changelogs, where Debian omitted the leading zero.
> The samples I have found[1], the leading zero is replaced with a single
> space as in:
> 
> >  -- Joey Hess   Thu,  3 Dec 1998 23:31:56 -0800
> 
> 
> 
> This is relatively prevalent.  As an (un)scientific example, this
> alternative variant accounts for:
> 
>  * ~21% of all signoff dates in debhelper (202/927)
>  * ~10% of all signoff dates in apt (49/480)
> 
> 
> I applaud policy for highlighting the correct and preferred example, so
> I propose that we restrain this amendment to a footnote (or another note
> of equal low importance) that informs the reader that this alternative
> format may be found in older changelog entries and that this variant is
> still accepted but that the two-digit format with leading zero should be
> preferred in every new entry.

Isn't this report a duplicate of #971977?

(I clarified the other report in deb-changelog(5) with
.)

Thanks,
Guillem

Bug#976301: Fix invalid `changelog` format example

[Guillem Jover]

On Mon, 2021-01-18 at 18:25:55 -0700, Sean Whitton wrote:
> On Thu 03 Dec 2020 at 05:08AM +03, Anatoli Babenia wrote:
> > diff --git a/policy/ch-source.rst b/policy/ch-source.rst
> > index edae8c1..1265c5e 100644
> > --- a/policy/ch-source.rst
> > +++ b/policy/ch-source.rst
> > @@ -126,7 +126,7 @@ That format is a series of entries like this:
> >   [blank line(s), included in output of dpkg-parsechangelog]
> >   * even more change details
> >   [optional blank line(s), stripped]
> > []{+[space]--+} maintainer name [two [-spaces]
> > date-]{+spaces]date+}
> 
> I do not believe that what is already there is invalid or unclear.
> 
> Please explain your motivations.

Oh, but it does look invalid. :) Notice that the trailing maintainer
line starts at the same column as the header line, but the trailer must
be indented by one space. Also the «two spaces» are then followed by
two explicit spaces, which seems confusing.

I guess Anatoli proposed to make the whitespace requirements explicit
to avoid this kind of subtle problems in the future.

Thanks,
Guillem

Bug#975250: clarify gathering together of copyright information

[Guillem Jover]

On Thu, 2020-11-26 at 08:55:21 +, Holger Levsen wrote:
> AIUI the first year of contributions and the last year of contributions are
> important data points for each contributor for a project, and mostly only
> the last year as that might be used to calculate when a project becomes
> public domain after the dead of an author.

The first year might also be relevant in infringement cases where
provenance might be disputed, but probably less so with a VCS at hand.

> So if I have contributed to something in 2018 and 2020 I find it ok to claim
> 'Copyright 2018-2020 Holger Levsen'.

Personally I see a big distinction between someone doing it for their
own copyright claims, and doing that for someone else's. For example
as was pointed out in the thread, GNU projects tend to do a global
copyright year bump commit at the beginning of the year updating all
the FSF notices, even for things that might otherwise end up not being
changed that year (but which I don't practice as it makes me a bit
uncomfortable).

While I've had no qualms whatsoever on coalescing claimed copyright
years into ranges, for a long time now, even for licenses that state
the copyright notice needs to be preserved (such as BSD), I've always
understood that to preserve the intent of the license, and I've
considered this pretty much accepted practice in the project. But I'd
be very uncomfortable claiming (unclaimed) copyright years on someone
else's behalf (say «© 2008, 20010-2015» into «© 2008-2015»), and
that's something I'd not consider doing.

> (Also because I might not have commited
> something in 2019 but you have no idea how much I prepared my 2020 commits
> in 2019...)

AFAIUI when and what you have done before publication does not count when
it comes to copyright, only the publication date does.

Thanks,
Guillem

Bug#954794: New packages must not declare themselves Essential

[Guillem Jover]

On Sat, 2020-11-07 at 13:30:13 -0700, Sean Whitton wrote:
> Could I ask you to explain your wanting to reduce the Essential set for
> the sake of small installation size in more detail, including some
> numbers, please?  It would be good to get to the bottom of Bill's worry
> about this change, but in addition, I would like to see a stronger
> positive case.

I'm not sure about Josh, but I think the main reasons for wanting to
reduce the essential set are:

  - Making chroots/containers slimmer, which can have a substantial
impact when needing lots of them, where even few MiB can make a
difference.
  - Making bootstrapping (build and installation) in general easier,
even though for the former these packages also need to then
be ideally removed from the build-essential set too.

Then there are these:

  - Making the packaging system less opaque, as the full implications
of Essential:yes do not seem generally clear to all maintainers?
  - Making it easier to use alternative implementations for some of
these packages, as then it's easier to know what depends on what,
instead of having to unconditionally provide the entire interface
surface for anything.
  - Making it easier to handle some multiarch problems, as then
dependencies can be explicitly marked as needed.

Thanks,
Guillem

Bug#954794: New packages must not declare themselves Essential

[Guillem Jover]

On Wed, 2020-09-30 at 18:34:06 -0700, Jonathan Nieder wrote:
> Josh Triplett wrote:
> > Jonathan Nieder wrote:
> > > Josh Triplett wrote:
> > > > This change does not propose eliminating the concept of Essential, nor
> > > > does it propose that any specific package become non-Essential.
> > >
> > > I think I'd be more supportive of this change if it did.  Freezing the
> > > current essential set in time feels oddly unpragmatic.  If we had a
> > > plan, even one that would take a while, to shrink the essential set,
> > > then it would be more likely to feel worth the cognitive load.

I'm in full support of trying to reduce the essential set, and this is
something we have already been trying to do for a while now:

  https://wiki.debian.org/Proposals/EssentialOnDiet
  https://wiki.debian.org/BusterPriorityRequalification

I also do not see the need to disallow adding new stuff to it if need
be. I think there's enough resistance in place that proposing that on
debian-devel would be met with quite some skepticism, *but* in case it
is needed, it would seem odd to do it anyway, even if policy disallows
it.

> > Long-term, I'd like to see that happen. But I'm a huge fan of
> > incremental steps; defining the problem as "eliminate Essential" makes
> > it both difficult enough and controversial enough to make it unlikely to
> > happen at all. Right now, the first step is "let's not let the problem
> > get any worse, and let's ensure that any new package that might have
> > otherwise used Essential must instead get packages to add a dependency".

> Even so, some *rough* consensus on the plan is very useful for helping
> people evaluate that first step.  It also gives people confidence that
> it can still move forward even if some people involved no longer have
> time to carry it forward themselves.  In this example, I think we
> agree about the ideal first (freeze the current Essential set) and
> last (eliminate Essential) steps but we don't have a clear picture of
> what happens in between.  That makes it hard to be comfortable with
> the first step because there's no reason to be confident that we'll
> get any further than that.

While the idea of completely getting rid of the Essential:yes concept
has been floated around in the bootstrapping and multiarch circles,
and seems like an interesting idea, I think it currently has too many
problems to be entertained.

Several of the Essential:yes properties [P] would need to be preserved
somehow for the packaging system to be able to operate as of now. Making
that marking implicit (some kind of lore) seems worse, than the current
explicit one, because the Essential field does not get involved much in
the package system operations anyway, it's more of a contract to be
honored by *maintainers*! Having to add Pre-Depends to tons of packages,
would probably substantially strain our dependency resolvers. Maintainer
scripts in at least the essential set are another problematic area,
so they would need to be reduced substantially, or eliminated
completely [B][D].

 [P] 
 [B] 
 [D] 

> If we don't have a rough plan, then the current state already seems
> okay: whenever someone proposes adding new Essential functionality, we
> can discuss it on debian-devel and the most likely outcome is that it
> gets rejected.

I think there are some rough plans for some of the problematic pieces,
but otherwise this seems rather premature, and I concur that the
current state seems fine.

> Some next steps that would make me more comfortable with an explicit
> freeze on Essential:
> 
> - document how to mark a package as "not safe in normal circumstances
>   to uninstall" (such as apt and systemd-sysv).  Is "Priority:
>   important" enough or should one use more?  As a package maintainer,
>   how do I mark a package as something that a user would never want to
>   deinstall in normal circumstances?

This is already supported with the Protected (old Important) fields,
and was implemented precisely to make it possible to split the
essential set into its different facets.

  

> - some rough idea about how to find undeclared dependencies.  Do we
>   do an NMU to add currently Essential packages as Pre-Depends on all
>   packages and then unmark them as Essential and let package
>   maintainers sort it out?  Is there some autodetection we can do
>   using autopkgtest?

Not all packages need Pre-Depends on currently Essential:yes packages,
but this would require careful consideration. But as mentioned above,
having to increase the amount of Pre-Depends relationships could make
our dependency resolvers and upgrades unhappy.

Trying to detect dependencies is going to be tricky, given that most
of the essential set are callable interfaces, which need to be checked
withi

Bug#954794: New packages must not declare themselves Essential

[Guillem Jover]

On Sun, 2020-10-18 at 11:43:18 +0200, Bill Allombert wrote:
> On Thu, Oct 15, 2020 at 11:56:19AM -0700, Jonathan Nieder wrote:
> > More specifically, it's the right first three steps.
> > 
> > https://www.debian.org/doc/debian-policy/ch-binary.html#dependencies
> > currently says
> > 
> > Packages are not required to declare any dependencies they
> > have on other packages which are marked Essential (see below),
> > and should not do so unless they depend on a particular
> > version of that package.[4]
> > 
> > [4] [...] If packages add unnecessary dependencies on packages
> > in this set, the chances that there will be an unresolvable
> > dependency loop caused by forcing these Essential packages to
> > be configured first before they need to be is greatly
> > increased.
> > 
> > I'd propose that as a first step we change that to
> > 
> > Packages are not required to declare any dependencies they
> > have on other packages which are marked Essential (see below),
> > but are permitted to do so even if they do not depend on a
> > particular version of that package.[4]
> 
> This is very dangerous with respect to upgrade between stable releases.
> The issue is at the time a package is made for a stable release, the
> state of Debian and Essential: yes packages is not known. It is
> unrealistic to expect Debian to plan so far in advance.
> Requiring changes to Essential packages to take into account spurious
> dependencies is too fragile.

This could refer to concerns with two opposite situations, I guess:

  - When adding new Essential:yes marks, with omitted dependencies (but
that is already a problem with the current text). In that case
the dependencies on the new Essential:yes should only be dropped
after the next stable release, or packages might assume the
presence of functionality that might not be there.
  - When removing old Essential:yes marks, with adding "excess"
dependencies, which could possibly introduce dependency loops,
that could break ordering assumptions in maintainer script
execution. This can be problematic if the requirements for the
behavior of an Essential:yes package for the affected package are
dropped before the next stable release (work even when unpacked,
Pre-Depends usage, etc), but otherwise it should be fine I guess.

Thanks,
Guillem

Bug#971977: debian-policy: debian/changelog date syntax description inconsistent/ambiguous wrt. to day of month

[Guillem Jover]

Hi!

On Mon, 2020-10-12 at 11:35:22 +0200, Axel Beckert wrote:
> Guillem Jover wrote:
> > Right. I've clarified this now locally for deb-changelog(5) as follows:
> >   +Is a one- or two-digit day of the month (B<01>-B<31>), where the heading
>   ^^^
> >   +zero is optional, but conventionally does not get omitted.
> [...]
> >Any line that consists entirely (i.e., no leading whitespace) of B<#>
>^^^
> >or B style comments or RCS keywords.
> 
> You once use "heading" and once "leading". Is this on purpose? At
> least for "whitespace" the term "leading whitespace" seems to be the
> common one, so I'm not sure if "heading zero" is really a proper
> term. (But then again, English is not my mother tongue and I might be
> wrong here.)

Ah, thanks! I did actually doubt about that word usage, but didn't
notice the other instance after a very brief scan. :) I've amended this
now locally.

Thanks,
Guillem

Bug#971977: debian-policy: debian/changelog date syntax description inconsistent/ambiguous wrt. to day of month

[Guillem Jover]

On Sun, 2020-10-11 at 00:36:00 +0200, Axel Beckert wrote:
> Package: debian-policy
> Version: 4.5.0.3
> Severity: minor

> Triggered by writing https://bugs.debian.org/971975 against lintian
> (which actually was triggered by writing another bug report, #971974
> :-), I noticed that in
> 
> https://www.debian.org/doc/debian-policy/ch-source.html#debian-changelog-debian-changelog
> 
> the description of the date format is inconsistent or at least ambiguous
> with regards to how many digits a day of the month can have and if there
> should be a zero or a space or nothing prepended to single-digit day of
> months.
> 
> Especially this item made me rise an eyebrow:
> 
> > * dd is a one- or two-digit day of the month (01-31)
> 
> Now what? Are one-digit day of months allowed or must we use "01" to
> "09" for single-digit day of months? And the commonly used leading space
> for one-digit days of months are not mentioned anywhere.
> 
> At least the "01-31" in parentheses clashes with "one-[…]digit day of
> month" in the beginning of the item. (Or does "one-digit" refer to that
> the number of the day of the months can be (obviously) one digit, but
> does not refer to how it's actually written? If so, this should be
> clarified.)

Right. I've clarified this now locally for deb-changelog(5) as follows:

  ,---
  --- i/man/deb-changelog.pod
  +++ w/man/deb-changelog.pod
  @@ -127,7 +127,8 @@ B, B, B, B, B, B, B.
   
   =item I
   
  -Is a one- or two-digit day of the month (B<01>-B<31>).
  +Is a one- or two-digit day of the month (B<01>-B<31>), where the heading
  +zero is optional, but conventionally does not get omitted.
   
   =item I
  @@ -168,6 +169,9 @@ The “trailer” line with the maintainer and date details 
must be
   preceded by exactly one space (U+0020 B).
   The maintainer details and the date must be separated by exactly two
   spaces (U+0020 B).
  +Each part of the I can be separated by one or more spaces
  +(U+0020 B), except after the comma where it can be separated
  +by zero or more spaces (U+0020 B).
   
   Any line that consists entirely (i.e., no leading whitespace) of B<#>
   or B style comments or RCS keywords.
 `---

Thanks,
Guillem

Re: Bug#971023: Version field (5.6.12) and colons

[Guillem Jover]

On Wed, 2020-09-30 at 13:56:47 +0200, Mattia Rizzolo wrote:
> On Wed, Sep 30, 2020 at 11:23:43AM +0200, Christian Kastner wrote:
> > To be honest, as a reader, I found that to be the opposite. The "If
> > [epoch] is omitted" makes it sound as if there were an alternative
> > handling if it's not omitted.
> > 
> > So the text
> > 
> >  If it is omitted then the upstream_version may not contain any colons
> > 
> > actually means
> > 
> >  The upstream_version may not contain any colons
> 
> If my memory serves correctly¹, this is just a historic remnant, as
> colons used to be allowed if the epoch was present (i.e., a version
> string "1:2.3:abc" used to be valid).
> 
> 
> ¹ and I think it does: 
> https://salsa.debian.org/dbnpolicy/policy/-/commit/918cac858424739a5af269d993e4cadfab285b29
> 
> 
> So, yes.  I think it would be good to make the wording just clearer,
> instead of carrying over some previous syntax from when the rules were
> different.

Yes, more so given that this is a Debian policy specific restriction,
whereas dpkg does still treat such versions as valid.

Thanks,
Guillem

Bug#967857: debian-policy: [Files/Permissions and owners] files installed by package manager should not be writable

[Guillem Jover]

On Wed, 2020-08-05 at 00:58:27 +0200, Ansgar wrote:
> On Tue, 2020-08-04 at 23:50 +0200, Guillem Jover wrote:
> > On Tue, 2020-08-04 at 13:56:45 -0700, Russ Allbery wrote:
> > > Ansgar  writes:
> > > > 10.9 Permissions and owners currently says
> > > > > Files should be owned by root:root, and made writable only by the
> > > > > owner and universally readable (and executable, if appropriate),
> > > > > that is mode 644 or 755."
> > > > However most files shouldn't be modified as modifications will just be
> > > > lost (e.g. everything installed by the package manager that isn't
> > > > handled as a conffile).  It also gives more permissions than the minimum
> > > > needed.
> > 
> > I'm not sure why we need to protect the local sysadmin(s) from this? Also
> > root can just write to any file regardless of the permissions.
> 
> The same reason that `rm -rf /` gives an error these days? :)

That might have been a good argument if root honored those missing
write-perms, but it does not. :)

> People modify files managed by dpkg from time to time and wonder why
> these modifications suddenly disappear on upgrades.  Or some package
> might assume a file is writable and use it, then data is lost on
> upgrade.  (I'm fairy certain to remember bug reports about that for
> files shipped in /var.)

Well, I guess they'll eventually understand what's going on?

> Various files are read-only anyway (such as `/bin/bash` when a shell is
> running).

But, that's just the standard Unix behavior of ETXTBSY, where the
file has been mmap()ed so you don't want to allow writing to it,
otherwise you'd end up modifying running code, ending up in at least
crashes. You can of course still unlink or rename these files though.

> > Countless
> > times I've modified local files, for example, to fix an issue that is
> > pending upload. And while that does not require write perms if done as
> > root, both of the above would seem to counter this as a good reason
> > for this change?
> 
> You can still do that if you really want to; editors will warn users if
> they modify read-only files, even when they provide an override to
> force writing the file.

Actually, editors need explicit support for this when running as root,
and this pretty much mostly applies to text files. For example nano
has no such protection.

So, I'm still failing to see what kind of protection this gives
anyway, because it does not really protect from root itself.

> > > Maybe it
> > > would make sense for Guillem to weigh in first and indicate whether this
> > > would be a problem on the dpkg side and if he sees any concerns.  Copying
> > > debian-dpkg@lists for that.
> > 
> > Thanks! Was meaning to comment anyway. :)
> > 
> > This would break installations as non-root, as those users will not
> > have enough privs to create the objects to extract. So that alone seems
> > like a non-starter.
> 
> Why would non-root users not be able to create files with 444
> permissions instead of 644?

First, due to the missing perms in directories. If we ignored that and
only removed write perms from regular files, then that would still
fail due to the needed delayed syncing required by new filesystem
so that they can behave in a sensible way when it comes to
performance. And while, f.ex., at least Linux does support passing a
read-only fd to fsync(2), this is a non-portable assumption.

Thanks,
Guillem

Bug#967857: debian-policy: [Files/Permissions and owners] files installed by package manager should not be writable

[Guillem Jover]

On Tue, 2020-08-04 at 13:56:45 -0700, Russ Allbery wrote:
> Ansgar  writes:
> > 10.9 Permissions and owners currently says
> 
> > | Files should be owned by root:root, and made writable only by the
> > | owner and universally readable (and executable, if appropriate),
> > | that is mode 644 or 755."
> 
> > However most files shouldn't be modified as modifications will just be
> > lost (e.g. everything installed by the package manager that isn't
> > handled as a conffile).  It also gives more permissions than the minimum
> > needed.

I'm not sure why we need to protect the local sysadmin(s) from this? Also
root can just write to any file regardless of the permissions. Countless
times I've modified local files, for example, to fix an issue that is
pending upload. And while that does not require write perms if done as
root, both of the above would seem to counter this as a good reason
for this change?

(I could also see this being performed by one of the other "admin" roles
in the system that is not root.)

> > I think static files should not be writable instead, so every file under
> > /usr (and /bin, /sbin, /lib*; or everything dpkg installs that is not a
> > conffile) should have 444 (or 555).
> 
> I assume this is in support of systems, containers, or jails where UID 0
> may not have CAP_FOWNER?

If that's the reason, it certainly was not clear from the original
report. :)

> The basic argument makes sense to me, but this is the sort of change where
> we'll need to figure out a transition strategy coordinated across multiple
> packages, since this behavior is encoded in a lot of places.  Maybe it
> would make sense for Guillem to weigh in first and indicate whether this
> would be a problem on the dpkg side and if he sees any concerns.  Copying
> debian-dpkg@lists for that.

Thanks! Was meaning to comment anyway. :)

This would break installations as non-root, as those users will not
have enough privs to create the objects to extract. So that alone seems
like a non-starter.

Thanks,
Guillem

Re: Bug#963524: debian-policy: Binary and Description fields not mandatory in .changes on source-only uploads

[Guillem Jover]

Control: reassign -1 debian-policy
Control: retitle -1 debian-policy: Binary and Description fields not mandatory 
in .changes on source-only uploads

On Mon, 2020-06-22 at 18:51:21 -0700, Felix Lechner wrote:
> Package: dpkg
> Severity: normal
> X-Debbugs-CC: debian-lint-ma...@lists.debian.org

> Starting with an upcoming release, Lintian will check for the presence
> of required and recommended fields in various packaging control files.
> Our methods are probably not perfect, but it was brought to my
> attention that 'dpkg-buildpackage -S' produces *.changes files without
> 'Binary' and 'Description' fields.

This is due to a fix in dpkg 1.19.3 prompted by #818618, which brought up
an inconsistency in the handling of these fields
(commit 4a4619831de8b8972f86b489660dc98f187cfa34 in dpkg.git). DAK got
also fixed to accept these .changes files.

> Policy 5.5 states that both fields are mandatory. [1]
> 
> [1] 
> https://www.debian.org/doc/debian-policy/ch-controlfields.html#debian-changes-files-changes
> 
> You may be able to find details about an example (by building Lintian)
> at the link below.
> 
> https://salsa.debian.org/lintian/lintian/-/commit/54a3c2437eadb0684f6762a81a82163f36562d3e#note_176583
> 
> Please note that I filed this bug with normal severity, even though as
> a policy violation, it should be serious. I did so because I believe
> the policy is at least partially in error (with respect to the
> 'Binary' field).

The deb-changes(5) and policy state that these fields contain
information for binary packages being uploaded. On a source-only upload,
there are no such binaries, so the definition was internally inconsistent.
I think the only thing missing is policy clarifying that this field is only
mandatory on non-source-only uploads.

Regards,
Guillem

Bug#955005: Relax requirements to copy copyright notices into d/copyright

[Guillem Jover]

On Tue, 2020-04-07 at 17:18:27 -0700, Sean Whitton wrote:
> On Wed 08 Apr 2020 at 01:18AM +02, Guillem Jover wrote:
> >> +The copyright information for files in a package must be copied
> >> +verbatim into ``/usr/share/doc/package/copyright``, when
> >   ^ Shouldn't this and other instances
> > of "package" be marked as replaceable text?
> 
> Possibly, though that's an issue with the existing Policy text not this
> patch -- perhaps I should just find and replace after applying the patch
> from this bug?

Ah right, thought this was specific to this drafting. Sounds good.

> > I'm assuming the entire list is supposed to hold at the same time? If
> > so perhaps adding an «and» here would make this completely unambiguous.
> 
> Hmm, thanks for the feedback, but I don't think "a; b; and c" is
> ambiguous in English, and "a; and b; and c" would be an irregular usage.

I guess what I found ambiguous is that "; and" in English does not
necessarily have a logic connotation. So one can read it as "item a;
item b; and item c" where the and is just there to introduce the next
item instead of specifying the content is ANDed. The “when” should
make it somewhat clear, but on a quick read it just made me doubt.

Take the example list in ch-source.rst
“Main building script: ``debian/rules``”:

  ,---
  There are sometimes good reasons to use a different approach.  For
  example, the standard tools for packaging software written in some
  languages may use another tool; some rarer packaging patterns, such as
  multiple builds of the same software with different options, are easier to
  express with other tools; and a packager working on a different packaging
  helper might want to use their tool.
  `---

Which I'd take it as an “and” for the list, not for its contents holding
true at the same time. :)

With the context I guess it is somewhat clearish, but I'd really like
to see text that is completely unambiguous for stuff that is normative.

> If this really does need clarification it would be better to add "all of
> the following" or something before the list.

Yes, clarifying before the list starts would work too, and I thought I
mentioned it in my reply, but apparently not.

Thanks,
Guillem

Bug#955005: Relax requirements to copy copyright notices into d/copyright

[Guillem Jover]

Hi!

On Sun, 2020-04-05 at 17:54:01 -0700, Sean Whitton wrote:
> Here's a patch for seconding, and for the FTP Team to approve.  Thanks
> to Scott for prompting the "all copies" amendation.

> diff --git a/policy/ch-archive.rst b/policy/ch-archive.rst
> index b8ba081..4217dd4 100644
> --- a/policy/ch-archive.rst
> +++ b/policy/ch-archive.rst
> @@ -184,15 +184,32 @@ Copyright considerations
>  
[…]
> +The copyright information for files in a package must be copied
> +verbatim into ``/usr/share/doc/package/copyright``, when
  ^ Shouldn't this and other instances
of "package" be marked as replaceable text?

> +
> +#. the distribution license for those files requires that copyright
> +   information be included in all copies and/or binary distributions;

I'm assuming the entire list is supposed to hold at the same time? If
so perhaps adding an «and» here would make this completely unambiguous.

> +#. the files are shipped in the binary package, either in source or
> +   compiled form; and
> +
> +#. the form in which the files are present in the binary package does
> +   not include a plain text version of their copyright notices.
> +
> +Thus, the copyright information for files in the source package which
> +are only part of its build process, such as autotools files, need not
> +be included in ``/usr/share/doc/package/copyright``, because those
> +files do not get installed into the binary package.  Similarly, plain
> +text files which include their own copyright information and are
> +installed into the binary package unmodified need not have that
> +copyright information copied into ``/usr/share/doc/package/copyright``
> +
> +However, the copyright notices for any files which are compiled into
> +the object code shipped in the binary package must all be included in
> +d/copyright when the license requires that copyright information be

I'm don't think abbreviating debian/ as d/ is appropriate in policy?
(Personally I fint it annoying also on debian/changelog, because then
you need to search using multiple variants of the filenames, but meh. :)

Thanks,
Guillem

Bug#953911: debian-policy: clarify documentation of "Closes: #NNNNNN" changelog syntax

[Guillem Jover]

Hi!

On Sat, 2020-03-14 at 21:49:12 +, Daniel Shahaf wrote:
> Sean Whitton wrote on Sat, 14 Mar 2020 20:39 +00:00:
> > On Sat 14 Mar 2020 at 08:09PM +00, Daniel Shahaf wrote:
> > > Daniel Shahaf wrote on Sat, 14 Mar 2020 18:14 +00:00:
> > >> -   ::
> > >> -
> > >> -   /closes:\s*(?:bug)?\#?\s?\d+(?:,\s*(?:bug)?\#?\s?\d+)*/i
> > >> -
> > >> +   All of the bug numbers listed must be given on the same physical line
> > >> +   as the word ``closes:``.
> > >
> > > Is this newly-added sentence correct?
> > >
> > > Whether it correctly states the semantics of the regexp depends on
> > > whether the regexp is matched against the changelog entry as a single
> > > multiline string or line-by-line.  In the former case, the list of bug
> > > numbers would be allowed to span multiple lines, but in the latter case
> > > it wouldn't.
> > >
> > > Currently, the patch assumes all bug numbers should be on the same line,
> > > but /usr/share/vim/vim81/syntax/debchangelog.vim allows bug numbers to
> > > span multiple lines.  I'm guessing the syntax file is right and the patch 
> > > should
> > > be changed?
> > 
> > In PCREs \s matches the newline character so I believe your text is
> > incorrect.
> 
> I agree that it is probably incorrect, but that doesn't follow from the
> semantics of /\s/.  Even if /\s/ matched any single byte/character, the
> semantics would still depend on what haystack the regexp is matched
> against: the entire changelog entry, or one line thereof at a time.

It is definitely incorrect. The canonical implementation is currently
in libdpkg-perl's Dpkg::Changelog::Entry::Debian::find_closes(), where
the entire changelog entry is passed as a single string, so something
like this is perfectly fine:

Closes:
  #12345,
  #67890,
  #5

AFAIK, DAK only operates based on the Closes field in the .changes
file. I'll clarify the regex also in the deb-changelog(5) man page.

Thanks,
Guillem

Bug#950440: debian-policy: Confusing conflation of Essential:yes w/ Priority:required

[Guillem Jover]

Package: debian-policy
Version: 4.5.0.0
Severity: normal

Hi!

This was brought up on debian-devel, and I think it needs to be
updated/corrected in the policy manual:

On Fri, 2020-01-17 at 12:21:11 +0100, Guillem Jover wrote:
> On Fri, 2020-01-17 at 11:12:50 +0100, Ansgar wrote:
> > Johannes Schauer writes:
> > > My advice would also be to switch from debootstrap to mmdebstrap because 
> > > the
> > > latter is able to create a chroot with only Essential:yes packages in it 
> > > while
> > > debootstrap includes Priority:required packages as well. Alternatively,
> > > debootstrap could gain a variant only installing Essential:yes packages 
> > > and
> > > their dependencies (why doesn't it have that already?).
> > 
> > Debian doesn't support systems that don't have "required" packages
> > installed. So buildds should have them installed.
> 
> If these statements are based on the policy quote below, then I do
> not agree. I don't see why e2fsprogs would need to be installed on
> a chroot, as long as there's nothing depending on it, for example.
> 
> > Policy states:
> > "Removing a required package may cause your system to become totally
> > broken and you may not even be able to use dpkg to put things back, so
> > only do so if you know what you are doing."
> 
> That seems wrong, or inaccurate at best. dpkg should never depend on
> anything that is not part of the pseudo-essential set (strictly
> speaking only Essential:yes + awk-virtual), or that it depends on
> explicitly. So as long as a package has not been forced out, dpkg
> must work.
> 
> Removing a required package (that is not Essential:yes) might indeed
> render your system broken, but what broken means or in what context is
> not qualified there. This could apply to systems that get booted for
> example, but not to chroots. A package that relies on another package
> that is Priority:required and not Essential:yes, and that it does not
> depend on, is just broken.
> 
> Here's the current list of these packages on my system:
> 
>   $ aptitude -F '%p' search '~prequired !~E'
>   debconf
>   e2fsprogs
>   gcc-10-base
>   gcc-9-base
>   libpam-modules
>   libpam-modules-bin
>   libpam-runtime
>   mawk
>   mount
>   passwd
>   tzdata
> 
> And most of these are actually part of the pseudo-essential set
> anyway, and cannot be removed w/o force.
> 
> That passage in policy might have made sense some time ago, when
> Priority:required almost matched the pseudo-essential set, and when we
> didn't have a separation of "dpkg-essential" and "boot-essential".

Regards,
Guillem

Bug#944920: Revise terminology used to specify requirements

[Guillem Jover]

On Wed, 2020-01-29 at 14:42:08 -0700, Sean Whitton wrote:
> On Sun 26 Jan 2020 at 03:48AM +01, Guillem Jover wrote:
> > I think one of the nice things about RFC2119 is that it uses uppercase
> > versions for the normative keywords, so that these are very clearly
> > distinguished both when writing and readin, from sentences that may
> > use some of the common words but that do not carry any normative
> > meaning. Was there any consideration in using uppercased keywords?
> 
> Well, Russ has now gone through and eliminated non-normative use of the
> keywords, so I think the question is moot.

Ok, let me try again, because I'm not sure I expressed my concern
clearly.

Some of the words chosen to convey normative meaning are glue words
used to build pretty mundane sentences, so having them appear around
while they might not convey normative meaning seems rather confusing,
and is a mental overhead when reading or drafting new text. Probably
more for non-native speakers.

For example in ch-scope.rst in the entry describing the *may* keyword
the following sentence uses also «may». :) The ch-archive.rst contains
several «may» instances that do not feel like the normative *may*, at
least the ones in the DFSG?

Perhaps instead of the potentially ugly uppercased keywords, marking
them as *keyword* like in the definition of the terms would go a long
way?


BTW I guess the instances of «might» and «shall» need to be converted,
or added to the keyword list? What about «may not», or «can», «can't»,
«cannot» and «could»? And I'm probably missing other words. :)

Thanks,
Guillem

Bug#944920: Revise terminology used to specify requirements

[Guillem Jover]

On Fri, 2020-01-03 at 20:43:14 -0800, Russ Allbery wrote:
> Russ Allbery  writes:
> > I agree, but let's also fix existing incorrect wording.  I reviewed
> > every instance of may and optional in Policy, and I think this larger
> > diff will make wording (mostly) consistent.  I've tried not to change
> > the sense of any of these Policy statements (even though a few are
> > questionable and should probably be revisited).
> 
> Here is an updated version of this patch, including the upgrading
> checklist entry. […]

> Most of this diff is changing normative "may not" phrasings to "must not"
> or some other equivalent, and changing other uses of "may" to "could" or
> other wordings.

I think one of the nice things about RFC2119 is that it uses uppercase
versions for the normative keywords, so that these are very clearly
distinguished both when writing and readin, from sentences that may
use some of the common words but that do not carry any normative
meaning. Was there any consideration in using uppercased keywords?

(I'm asking this irrespective of the actual words being used.)

Thanks,
Guillem

Bug#941198: In support of mandatory unit files

[Guillem Jover]

On Sun, 2019-12-08 at 15:55:45 -0800, Russ Allbery wrote:
> Guillem Jover  writes:
> > But here you do have another option, but I'm not sure it might be
> > described as nicer TBH, :) something like this, or variations on this
> > theme:
> 
> >   [Service]
> >   Type=simple
> >   EnvironmentFile=/etc/default/service-static-vars
> >   EnvironmentFile=-/run/service-dynamic-vars
> >   ExecStartPre=-/bin/sh -c 'echo NAME=$(hostname) 
> > >/run/service-dynamic-vars'
> >   ExecStart=/usr/bin/daemon --option ${NAME}
> 
> This is a nice approach, but I don't think it quite preserves the original
> behavior.  As you wrote it above, if someone changed the setting in the
> /etc/default file, that would have no effect and the default would still
> be used.  If you reverse the order of EnvironmentFile, it avoids that
> problem, but now the legacy setting with $(hostname) will be used if it
> hasn't been changed, and that will result in a literal $(hostname) in the
> value.

Right, it was more about showing the concept than a proper
implementation, but it's true that as is, it's not helpful. :)

I guess the following which starts to get a bit into ugly territory
would do instead:

   [Service]
   Type=simple
   EnvironmentFile=-/run/service-dynamic-vars
   ExecStartPre=-/usr/bin/env -i /bin/sh -a -c '. 
/etc/default/service-static-vars && env -uPWD >/run/service-dynamic-vars'
   ExecStart=/usr/bin/daemon --option ${NAME}

Thanks,
Guillem

Bug#941198: In support of mandatory unit files

[Guillem Jover]

On Sun, 2019-12-08 at 11:15:57 -0800, Russ Allbery wrote:
> > Sure, help fir that would be nice. Thanks for the offer.  (Probably
> > should have an own bug for that.) Nethertheless, this is the line that
> > causes my problems and needs to be transferred:
> > https://salsa.debian.org/debian/gmrender-resurrect/blob/master/debian/gmediarender.default#L8
> 
> Ah, I see, the problem is the $(hostname) part, which isn't supported by
> EnvironmentFile in systemd (which is the normal way to solve this
> problem).
> 
> After looking at this for a bit, my inclination if I were you would be to
> write a tiny shell script that loads /etc/default/gmediarender, constructs
> the command line, and then execs the daemon; install that script as
> /usr/share/gmediarender/start; and then invoke that script via
> start-stop-daemon in the init script and via ExecStart in the systemd unit
> file.  It's a little bit overkill because most of what the /etc/default
> file is doing is simple, but it looks like the easiest way to handle
> $(hostname).

I'd avoid using the wrapper for the init script TBH, because even
though that will make this a bit WET, it would otherwise make it more
complex there (having to use --startas and --exec to cope with the
intermediate interpreter usage).

I think shared wrappers make more sense when used to offload some kind of
pre/post "hook" work, which gets called from Exec{Start,Stop}{Pre,Post}
systemd service file directives (and init scripts).

But here you do have another option, but I'm not sure it might be
described as nicer TBH, :) something like this, or variations on this
theme:

  [Service]
  Type=simple
  EnvironmentFile=/etc/default/service-static-vars
  EnvironmentFile=-/run/service-dynamic-vars
  ExecStartPre=-/bin/sh -c 'echo NAME=$(hostname) >/run/service-dynamic-vars'
  ExecStart=/usr/bin/daemon --option ${NAME}

Thanks,
Guillem

Bug#945269: debian-policy: packages should use tmpfiles.d(5) to create directories below /var

[Guillem Jover]

Hi!

On Fri, 2019-11-29 at 09:13:47 -0800, Russ Allbery wrote:
> Guillem Jover  writes:
> > As I mentioned on debian-devel, I think major parts of this and of the
> > sysuser stuff fall under dpkg realm. And my plan is to implement this
> > kind of functionality natively in dpkg, regardless of whatever the
> > outcome of that GR is.
> 
> > Of course some parts of the functionality needed to manage/track
> > temporary pathnames requires integration or hooking into an init system
> > or chroot/container manager, but that's true regardless of the
> > implementation.
> 
> Could you say more about how you think dpkg would be able to handle
> systemd-tmpfiles?  As you mention, this is something that has to be done
> at every system boot, which seems pretty far afield from dpkg's domain.
> Do you want to maintain custom dpkg plugins for every init system that a
> dpkg system may support?

The way I see it, any pathname "owned" by a package is within the
realm of dpkg, which is in charge of tracking and managing the
filesystem contents for system software. I've, for example, always
found it a big defficiency that when querying dpkg about system
pathnames it cannot give a proper answer in many cases. The same with
verification of system pathnames. But this goes beyond temporary files,
this includes things like log or cache files, or other volatile
pathnames, for example.

These kinds of pathnames are also potential package cruft, that should
be cleaned up on package remove/purge (ideally in a configurable way,
as in “I want logs to be left behind after purge”), and w/o requiring
maintainer scripts.

  <https://wiki.debian.org/Teams/Dpkg/Spec/MetadataTracking>

I don't mind much how this could end up being hooked into various init
systems and chroot/container managers. I can see how it could be done by
the respective imeplementations themselves or provided by dpkg itself,
I'm open to any of these TBH.

> (My recollection is that dpkg supports other OSes like Solaris.)

  <https://wiki.debian.org/Teams/Dpkg/Porting>
  <https://wiki.debian.org/Teams/Dpkg/Downstream>

> (systemd-sysusers is another matter and a bit more obviously doable with
> dpkg, although if Sam's option three wins, I'd ask everyone in the
> project, including you as dpkg maintainer, to consider the possible
> benefits of using cross-distribution mechanisms instead of Debian-specific
> mechanisms.)

  <https://wiki.debian.org/Teams/Dpkg/Spec/SysUser>

To me that seems like an extremely unsatisfying and restrictive way
to characterize cross-distribution. This implies GNU/Linux-only ones,
not even things like musl-based, for example. I consider portability
of paramount importance, and I'll not stop caring about it, even if
the Debian project decided otherwise, less so as an upstream. (See my
GR amendment. :)

I find integration within Debian and with the wider dpkg ecosystem to
be way more interesting, and interfaces in dpkg that can work anywhere
where it would be available. That's why I'll work on this no matter
what, even if Debian decided to not use these features (which I'd find
rather unfortunate), I'd spend time on this at the expense of other
stuff.

Thanks,
Guillem

Bug#945269: debian-policy: packages should use tmpfiles.d(5) to create directories below /var

[Guillem Jover]

On Fri, 2019-11-22 at 10:12:06 -0800, Russ Allbery wrote:
> Ansgar  writes:
> > I think no option says we shouldn't use services that don't rely on
> > systemd as pid-1 (which also includes widely used things like udev).
> 
> I agree, but if, say, Sam's option 3 wins, we can directly incorporate
> this, whereas if, say, Ian's option wins, then we should have a
> conversation with the sysvinit maintainers to see if they believe they can
> adopt systemd-tmpfiles directly, and we should give them at least six
> months (obviously shortened if it happens sooner) to update the packaging
> and set up a mechanism to run systemd-tmpfiles (or a replacement of their
> choice) at boot.

As I mentioned on debian-devel, I think major parts of this and of
the sysuser stuff fall under dpkg realm. And my plan is to implement
this kind of functionality natively in dpkg, regardless of whatever
the outcome of that GR is.

Of course some parts of the functionality needed to manage/track
temporary pathnames requires integration or hooking into an init system
or chroot/container manager, but that's true regardless of the
implementation.

Thanks,
Guillem

Bug#944296: debian-policy: Source provenance requirement is WET

[Guillem Jover]

On Sat, 2019-11-09 at 08:55:23 -0700, Sean Whitton wrote:
> On Thu 07 Nov 2019 at 09:00AM -08, Russ Allbery wrote:
> > I'm in favor of dropping this information from debian/copyright and
> > instead writing some language saying that packages should include this
> > information in Homepage in debian/control and, if there's a substantial
> > non-obvious difference between the package home page and how to download
> > it, put download information in debian/watch.
> 
> For a lot of packages I've put some sort of git web view URL into the
> Source: field and a descriptive homepage into the Homepage: field.

If upstream does not release tarballs, then I guess that usage of the
Source: field seems appropriate, otherwise it looks a bit like a
stretch?

> I don't think this is a very strong reason not to drop the Source:
> field, since the descriptive homepage will usually have a link to the
> git web view.  But I thought I'd note my experience.

Just to clarify, I didn't intend to propose drop the field, just for
example to make it optional when the same information is elswhere in
the packaging.

Thanks,
Guillem

Bug#944331: debian-policy: Spurious space after dash in plain text output

[Guillem Jover]

Package: debian-policy
Version: 4.4.1
Severity: minor

Hi!

Noticed the following instances of spurious spaces after a dash, while
skimming over the plain text policy document:

  - index, rendered as:

,---
* 7.8. Additional source packages used to build the binary -
"Built- Using"
`---

  - chapter 4, footnote [11], rendered as:

,---
Another common way to do this is for "build" to depend on
"build- stamp" and to do nothing else, and for the "build-stamp"
`---

  - chapter 6, footnote [2], rendered as:

,---
This can happen if the new version of the package no longer
pre- depends on a package that had been partially upgraded.
`---

  - chapter 7, footnote [1], rendered as:

,---
While "Build-Depends", "Build-Depends-Indep" and "Build-
Depends- Arch" permit the use of alternative dependencies, these
`---

  - chapter 8, footnote [7], rendered as:

,---
This wording allows the development files to be split into
several packages, such as a separate architecture-independent
libraryname- headers, provided that the development package
`---

  - chapter 8, footnote [13], rendered as:

,---
separate "substvars" files for each binary package and calling
"dpkg- gencontrol" with the appropriate flags.
`---

Thanks,
Guillem

Bug#944332: debian-policy: Broken markup in policy source

[Guillem Jover]

Source: debian-policy
Source-Version: 4.4.1.1
Severity: normal

Hi!

Found this markup issue while going over the policy:

  - chapter 4, footnote [6], rendered as:

,---
listed in the :ref:"`Maintainer" <#s-f-Maintainer` or "`Uploaders"
` control fields of the package), the first line of
`---

source:

,---
:ref:```Maintainer`` <#s-f-Maintainer` or
```Uploaders`` ` control fields of the package),
`---

Thanks,
Guillem

Bug#944330: debian-policy: Hyphenation damage on plain text output

[Guillem Jover]

Package: debian-policy
Version: 4.4.1
Severity: minor

Hi!

The rendering to plain text contain many technical terms, program
names, email addresses, make rule names, etc., that have been cut
at their hypen at the end of line. This makes copy&paste more
difficult, and it reads confusingly.

I started collecting a list of these, but there are quite many, so
before wasting time, I guess it's best to canvas whether you agree
this should be fixed. :)

Thanks,
Guillem

Bug#944329: debian-policy: Unclear text about password files modifications

[Guillem Jover]

Package: debian-policy
Version: 4.4.1
Severity: normal

Hi!

There's this text in section §9.2.1:

  ,---
  Packages other than "base-passwd" must not modify "/etc/passwd",
  "/etc/shadow", "/etc/group" or "/etc/gshadow".
  `---

It's not clear to me, whether this refers to the packaging or any
program provided by that package. Depending on the reading this
would make the passwd package buggy. So it might be worth clarifying
probably by adding "passwd" to the exception.

Thanks,
Guillem

Bug#944296: debian-policy: Source provenance requirement is WET

[Guillem Jover]

On Thu, 2019-11-07 at 09:00:29 -0800, Russ Allbery wrote:
> Guillem Jover  writes:
> > This means that when using a debian/watch file one has to duplicate
> > the information in two places, with the possibility of this getting
> > out-of-sync, etc.
> 
> > In addition the machine readable debian/copyright format, specifies
> > the Source field as optional, which could perhaps be interpreted as
> > contradict what policy says.
> 
> > IMO, ideally the requirement in policy would be lifted by clarifying
> > that the information should be provided in *either* debian/copyright
> > or debian/watch.
> 
> Personally, I usually find they're not the same thing.  debian/watch wants
> a very specific technical URL (the path to the download location), whereas
> I usually use the Source file to specify a higher-level view of the
> project.
> 
> That's not an argument against your point that this is duplicative; it's
> just that I find Source to more normally duplicate Homepage in
> debian/control than duplicate debian/watch.

Hmm, right, that just depends on the upstream project. I guess these
tend to converge in the following way:

  debian/watch → Source: → Homepage:

As in you could have the three being distinct, all the same (well the
URL in debian/watch being extended by the filename part), or two from
each side being equal.

> Anyway, I have also found this an odd fit for debian/copyright if one
> views debian/copyright as being for the legally-mandated notices plus
> license information for Debian package users.  I suspect that it's a
> combination of that Policy text predating both Homepage and uscan.
> 
> I'm in favor of dropping this information from debian/copyright and
> instead writing some language saying that packages should include this
> information in Homepage in debian/control and, if there's a substantial
> non-obvious difference between the package home page and how to download
> it, put download information in debian/watch.

I would not even really mind keeping the Source: information (even if
we considered debian/copyright not to be the best place, and planned to
eventually move it elsewhere), as long as it's different from the other
two locations.

So I guess the core of the problem I see is that the Source field is
always required.

Thanks,
Guillem

Bug#944296: debian-policy: Source provenance requirement is WET

[Guillem Jover]

Package: debian-policy
Version: 4.4.1.1
Severity: wishlist

Hi!

We currently require (with a must) in section §12.5, to add to the
debian/copyright, where the upstream source was obtained from:

  ,---
  In addition, the copyright file must say where the upstream sources
  (if any) were obtained, …
  `---

This means that when using a debian/watch file one has to duplicate
the information in two places, with the possibility of this getting
out-of-sync, etc.

In addition the machine readable debian/copyright format, specifies
the Source field as optional, which could perhaps be interpreted as
contradict what policy says.

IMO, ideally the requirement in policy would be lifted by clarifying
that the information should be provided in *either* debian/copyright
or debian/watch.

The other option would be, I guess, to turn the Source field into a
required one. If that's not desired to be done from the
copyright-format itself (because the format can be considered more
general and there's no need for this requirement from the parsing PoV)
then this requirement could perhaps be added from the policy side.

I just noticed recently I've been creating non-compliant packages for
a while. :/ Once there's a decision either way we could add lintian
tags so that we do not miss this kind of problem.

Thanks,
Guillem

Bug#288822: marked as done (developers-reference: "Bugs" control field not documented)

[Guillem Jover]

On Tue, 2019-10-08 at 10:33:46 +, Holger Levsen wrote:
> On Tue, Oct 08, 2019 at 12:30:50PM +0200, Guillem Jover wrote:
> > > I don't really understand "#288822: developers-reference: "Bugs" control 
> > > field
> > > not documented" and I'm not sure it's really an issue still.
> > This would be the Bugs field documented now in both deb-control(5) and
> > deb-src-control(5), since dpkg 1.14.7 (2007-10-08), ref #173463.
> 
> thanks for that information! do you agree there's nothing to be added to
> dev-ref?

Yeah, looks like it.

Thanks,
Guillem

Bug#288822: marked as done (developers-reference: "Bugs" control field not documented)

[Guillem Jover]

Hi!

On Tue, 2019-10-08 at 10:09:08 +, Debian Bug Tracking System wrote:
> Date: Tue, 8 Oct 2019 10:05:05 +
> From: Holger Levsen 
> To: 288822-d...@bugs.debian.org
> Subject: bug purpose vague and unclear
> Message-ID: <20191008100505.grkghleotjlxn...@layer-acht.org>

> I don't really understand "#288822: developers-reference: "Bugs" control field
> not documented" and I'm not sure it's really an issue still.

This would be the Bugs field documented now in both deb-control(5) and
deb-src-control(5), since dpkg 1.14.7 (2007-10-08), ref #173463.

Thanks,
Guillem

Bug#941803: debian-policy: dependencies on font packages

[Guillem Jover]

Hi!

On Sat, 2019-10-05 at 21:44:25 +0200, Stephen Kitt wrote:
> Package: debian-policy
> Version: 4.4.1.1
> Severity: normal

> Policy section 11.8.5, point 1 says
> 
> > If one or more of the fonts so packaged are necessary for proper
> > operation of the package with which they are associated the font
> > package may be Recommended; if the fonts merely provide an
> > enhancement, a Suggests relationship may be used. Packages must not
> > Depend on font packages.
> 
> The associated footnote explains that
> 
> > This is because the X server may retrieve fonts from the local file
> > system or over the network from an X font server; the Debian package
> > system is empowered to deal only with the local file system.
> 
> While this is still technically true,

Actually, I don't think this is true nowadays. Font server support was
disabled in libxfont 1:1.4.7-1 (in 2014-01-07).

> it seems rather irrelevant
> nowadays: most GUI programs directly render fonts obtained locally,
> and even for “traditional” X fonts, the vast majority of systems will
> obtain the fonts locally. Debian hasn’t had xfs for 5.5 years
> (); there is another font server
> available, xfstt, but that only handles TrueType fonts.

I've kept xfstt in the archive mostly for two reasons:

  - sentimental; it's the first package I adopted in Debian (at the
same time of taking over as upstream :).
  - remote usage; external non-Debian systems where their X server
still has font server support can still use these.

 But it has crossed my mind few times already, dropping it from Debian
 and probably also from upstream.

> It’s common for packages to strongly depend on non-X fonts they need;
> see for example the reverse dependencies of fonts-dejavu. While
> lintian objects to X font depencencies
> (),
> it doesn’t have anything to say about non-X fonts (rightly so).
> 
> Wouldn’t it make sense to relax the constraints on X font
> dependencies?

Looks like it to me, yes.

Thanks,
Guillem

Bug#940234: debian-policy: add a section about source reproducibility

[Guillem Jover]

On Sat, 2019-09-14 at 08:58:21 -0700, Sean Whitton wrote:
> On Sat 14 Sep 2019 at 02:01PM +00, Holger Levsen wrote:
> > On Sat, Sep 14, 2019 at 01:34:49PM +0200, Aurelien Jarno wrote:
> >> There is already a section about reproducibility in the debian-policy,
> >> but it only mentions the binary packages. It might be a good idea to
> >> add a new requirement that repeatedly building the source package in
> >> the same environment produces identical .dsc file modulo the GPG
> >> signature.
> >>
> >> I haven't checked how many packages do not fulfill this condition
> >
> > please do check. last (and only) time we (=r-b) looked, it wasn't
> > practical at all. this was around 5 years ago, but I don't remember any
> > work done on improving this.
> 
> Right.  While we can all agree that it would be nice for source package
> builds to reproducible, I think our current source package formats make
> it quite a hard problem, so it would be good to have some data before we
> spend any time discussing this further.

Back when we were fixing the binary package reproducible problems
within dpkg, I also checked the source side, and fixed a few
problematic cases. Assuming the same tools installed as defined in
the .buildinfo file, and the same content in the unpacked source
tree, dpkg-source should be producing the same output source packages.
If this does not hold, I'd consider it a bug to be fixed.

Thanks,
Guillem

Re: Bug#884999: debhelper: Please default Rules-Require-Root to no

[Guillem Jover]

Hi!

On Fri, 2019-05-24 at 13:42:28 +0900, Hideki Yamane wrote:
> > > In summary: The debhelper fundamentally cannot affect whether
> > > Rules-Requires-Root: no is default or not.  The debhelper compat level
> > > system is the wrong interface to do this as well.
> > > 
> > > That said, in a distant future, I hope we can flip the default of
> > > Rules-Requires-Root.  But when that comes, it will not be via a
> > > debhelper compat level AFAICT.
> 
>  If we want to implement "Rules-Requires-Root: no" mandatory,
>  is it dpkg-dev and policy issue?

The current thinking is that it would be acceptable to switch the
default by introducing a new dpkg-dev-specific compat level. See
.

I think the argument put forward by Adrian Bunk, about this possibly
being just another transition among many we do that can break
non-smallish parts of the archive makes sense, except that I fear in
this case it might cause silent breakage, or at least breakage not
detectable w/o an archive-wide rebuild, with several permutations,
such as binary-indep-only, binary-arch-only, etc. And this still does
not cover external users. And in any case I've not heard anyone
expressing interest in handling such transition, so that's pretty much
a non-starter.

Thanks,
Guillem

Re: Thinking about Delegating Decisions about Policy

[Guillem Jover]

Hi!

Thanks for sending this out Ian, part of this matches exactly what I've
been thinking for a long time, and the reason for my continued public
opposition and deep dissatisfaction with the tech-ctte as a body. I've
mentioned in the past [P] I'd put my thoughts in a more structured form,
but I always find this topic to be too exhausting and demotivating.

  [P]  

On Mon, 2019-05-13 at 15:28:11 +0100, Ian Jackson wrote:
> Rather than see the TC's role enhanced, I would like to abolish it and
> replace it with something completely different.

Full ack! And like Ian, I pretty much have a problem with the structure
and the body, not its members!

Some of the problems I see myself which I've probably covered in the
past are:

  * It has multiple conflicting roles (arbiter of disputes, mediator
of conflicts, advice giver, etc.), so there's always the worry that
something brought forward might be morphed/redirected/switched into
one of the other powers, which can bring unanticipated power
dynamics (f.ex. I don't think mediation can work well or at all when
the mediating party has also the power to adjudicate).

- If it was just an advisory board of elders/oracles or similar,
  with no powers of authority, besides the ones coming from knowledge,
  experience, track record, etc., it would seem perfectly fine for
  any party, including, say the policy editors, to request advice on
  contentious or uncertain issues from that advisory board, because
  it would be just another input to consider when doing the actual
  decision.

- If it was a social-ctte (not arbiters but mediators or resolvers
  of conflict, w/o authority) to deal with interpersonal conflicts,
  that would also seem fine, and my take is that the different
  parties might be more open to be mediated this way.

  * It has the power to impose final decisions into others:

- While not being involved (as a body) in the design nor the
  implementation, nor long-term maintenance, nor any consequence
  and fallout from those, which seems completely unfair within the
  confines of a volunteer project.

- Which also seems like a way to sidestep one of the core tenets of
  the project ("volunteers cannot be forced to do anything"), which
  might force them to do things they otherwise would not do, and
  giving them pretty much the options of complying or stepping down.

  * When conflicts are brought up, in most cases they are at a point
where the positions of the opposing factions/parties are so hardened,
and/or communication has broken down so badly, that a decision will
just be a win/lose scenario, which further destroys/erodes the social
fabric of the project, bringing with it possibly rancour and hard
to repair relationships between members in the project.

  * Reaching good decisions or deciding at all is important, but how
we reach them is IMO as important, or more (the ends do not justify
the means):

- Forced decisions coming out of the tech-ctte, by way of wielding
  authority instead of convincing arguments and wide adoption, means
  they are way less legitimate than ones that would have been reached
  by (rough) consensus, or incremental adoption.

  * It still has the very unfair power imbalance I pointed out some years
ago, about a GR requiring a super-majority to override its decisions.

  * This is a body composed of members that come and go, these might have
wide experience in Debian in general (although not necessarily) or
might had expertise in specific fields. The problem is that this body
gets unbounded topic issues about anything. You cannot expect anyone
w/ no prior experience to have "taste" or "intuition" about things
they have not experienced/practiced for a long time. This is not,
say, a java-ctte composed of Java experts.

  * It is a self-selected body, where things like being uncomfortable
with or not being able to work with other specific members, might
bias the selection process, while this body is supposed to serve
the project interests at large and not the individual body members'.
(And to be clear having a body with members that cannot or do not
enjoy working together would be pretty terrible, but given the purpose
of the body, having that limit its composition seems pretty bad too.)

  * Most decisions are not just technical decisions, in many/most cases
the decisions have answers that are all correct, but it just depends
on the weight of specific trade-offs. How those are weighted depends
heavily on each individual. This also seems rather unfair, as it's
taking the natural and expected biases of a small set of people in
the project and forcing them into the entire project.


Thanks,
Guillem

Bug#931975: dpkg-checkbuilddeps don't allow multiple Vcs-Git statements

[Guillem Jover]

On Sun, 2019-07-14 at 15:03:28 -0700, Russ Allbery wrote:
> In that case, should we increase the strength of this by changing the
> first sentence?  I'm not seeing much purpose served by developer
> discretion here, and this clarifies matters for tool developers.

Sure.

> diff --git a/policy/ch-controlfields.rst b/policy/ch-controlfields.rst
> index 81b3542..8124d64 100644
> --- a/policy/ch-controlfields.rst
> +++ b/policy/ch-controlfields.rst
> @@ -979,7 +979,10 @@ repository where the Debian source package is developed.
>  or ``hg clone`` command. If no branch is specified, the packaging
>  should be on the default branch.
>  
> -More than one different VCS may be specified for the same package.
> +A package control file must not have more than one ``Vcs-``
> +field.  If the package is maintained in multple version control
> +systems, the maintainer should specify the one that they would prefer
> +other people to use as the basis for proposing changes to the package.
>  
>  For both fields, any URLs given should use a scheme that provides
>  confidentiality (``https``, for example, rather than ``http`` or ``git``)

Seconded, too.

Thanks,
Guillem

Bug#931975: dpkg-checkbuilddeps don't allow multiple Vcs-Git statements

[Guillem Jover]

Hi!

On Sun, 2019-07-14 at 09:31:16 -0700, Russ Allbery wrote:
> Yeah, this just seems generally wrong to me.  I assume the idea was that a
> package may have mirrors of its packaging repository in multiple VCS
> systems and list all of them, but I'm dubious there's much point.  My
> leaning is to make the following change:

Right, the problem though is that I'm not even sure consumers of these
fields will behave properly and consistently when multiple Vcs-
fields are present? (I'd assume they do not handle that case
gracefully.)

> diff --git a/policy/ch-controlfields.rst b/policy/ch-controlfields.rst
> index 81b3542..d491d57 100644
> --- a/policy/ch-controlfields.rst
> +++ b/policy/ch-controlfields.rst
> @@ -979,7 +979,10 @@ repository where the Debian source package is developed.
>  or ``hg clone`` command. If no branch is specified, the packaging
>  should be on the default branch.
>  
> -More than one different VCS may be specified for the same package.
> +Only one ``Vcs-`` field should be given for a package.  If the
> +package is maintained in multple version control systems, the
> +maintainer should specify the one that they would prefer other people
> +to use as the basis for proposing changes to the package.
>  
>  For both fields, any URLs given should use a scheme that provides
>  confidentiality (``https``, for example, rather than ``http`` or ``git``)
> 
> Before we make that change it would be great if someone could check how
> many packages we would make buggy.  (I'm sure there's some good way to do
> this with standard tools, but I don't know off-hand how to do it.)

Yeah, that's something that crossed my mind too before reassigning.
I've just done it now:

  ,--- grep-deb-sources ---
  #!/bin/sh
  set -e
  set -u
  Sources=$(apt-get indextargets --format '$(FILENAME)' 'Identifier: Sources')
  /usr/lib/apt/apt-helper cat-file $Sources | grep-dctrl "$@"
  `---

  ,---
  $ grep-deb-sources -sPackage,Vcs-Arch,Vcs-Bzr,Vcs-Cvs,Vcs-Darcs,\
  Vcs-Git,Vcs-Hg,Vcs-Mtn,Vcs-Svn Vcs- \
| awk -F "\n" -v RS="" '{ if (NF > 2) print $0, "\n" }'
  Package: netselect
  Vcs-Git: git://github.com/apenwarr/netselect.git
  Vcs-Svn: svn://svn.debian.org/svn/collab-maint/deb-maint/netselect/trunk
  `---

And that one instance seems to be "bogus" anyway as the Vcs-Svn URL
does not exist anymore. So I'd say the above change looks perfectly
fine to me, thanks! :)

So, «Seconded».

Thanks,
Guillem

Re: Bug#931975: dpkg-checkbuilddeps don't allow multiple Vcs-Git statements

[Guillem Jover]

Control: reassign -1 debian-policy

Hi!

On Sat, 2019-07-13 at 10:27:24 +0200, Jörg Frings-Fürst wrote:
> Package: dpkg-dev
> Version: 1.19.7
> Severity: important

> With two Vsc-Git statements in debian/control I get:
> 
> dpkg-checkbuilddeps: error: syntax error in debian/control at line 14:
> duplicate field Vcs-Git found

Right, this is the expected behavior for any field. No duplicates are
allowed. I'm curious what's the reason you'd like to list two Vcs-Git
fields, is this to list multiple mirrors/alternative-URLs of the same
repo?

> From Debian policy 4.4.0 paragraph 5.6.26:
> 
> More than one different VCS may be specified for the same package.

Right, and apparently I seconded that change, with this very confused
wording :/, although my reading is different: as in diffferent VCS
types are allowed, which would be consistent with the current behavior.
But even then I'm not sure what's the point alogether. At a minimum
this sentences needs to be clarified, or maybe just entirely dropped,
as it looks very confusing?

Thanks,
Guillem

Re: Converting dev-ref to use rST

[Guillem Jover]

Hi!

On Mon, 2019-04-08 at 14:45:29 -0700, Sean Whitton wrote:
> - whether I should use the scripts that were used to convert
>   debian-policy Debian-SGML->docbook->rST+Sphinx, or instead write a
>   new Debian-SGML->rST+Sphinx converter; and

Hmm, but the devref appears to be already in docbook? :)

Thanks,
Guillem

Bug#924401: base-files fails postinst when base-passwd is unpacked

[Guillem Jover]

Hi!

On Fri, 2019-03-15 at 00:37:33 +0100, Santiago Vila wrote:
> Maybe, but this is neither a new miscellaneous file nor a new
> bootstrapping action. This is yet another bootstrapping tool
> forgetting the lessons learned from the other bootstrapping tools.

My impression though is that the general consensus is that the
lessons learned from the traditional bootstrapping tools, is that
they are doing it wrong. :)

> Helmut Grohne wrote:
> > How is a bootstrap tool supposed to know
> > that it must configure base-passwd before base-files?
> 
> In a theoretical level, because it's their job to know such things.

As it's been mentioned elsewhere, this is not a property that is
necessary. In the same way that bootstrapping an architecture should
not require manually tracking the order of the build-dependencies
and cycle breakings (which was the case before), etc, even if that
might still be partially the case now, while we have not fully added
build profiles and cross-building support everywhere necessary.

> In a practical level, because you already see what happens when
> you configure any package which uses users defined in /etc/passwd
> without a minimal /etc/passwd in place. Again in a practical level,
> once we know it, we can't pretend that we don't know it.

That's what the traditional bootstrapping tools have been doing, and
it's all kinds of wrong. For example whether a system even uses an
/etc/passwd is an implementation detail of that system. Some don't
even have one. If we added support for such a port, then in addition
to patching the relevant code that directly deals with this, we'd
need to also adapt all the bootstrappers to cope with this. Instead
of them just automatically just working out of the box, w/o needing
to hardcode package names, internal implementation details, etc.

> You want to create a common framework to reuse such knowledge and
> share it between different bootstrapping tools? Fine, maybe then new
> bootstrapping tools finally realize that there must be a valid
> /etc/passwd before configuring anything else.

That knowledge is already in each relevant package, the passwd stuff
in the base-passwd package (in Debian) or possibly elsewhere in other
systems.

> But lack of such a common framework is not an excuse to ditch the
> definition of essential and start doing things "just in case"
> some new bootstrapping tool forgets again that there must
> be a valid /etc/passwd before configuring anything else.

You seem to keep using the definition of essential to justify this,
but as I've mentioned elsehwere, the definition of essential does not
cover what happends before the system has been bootstrapped. If we'd
consider that policy already covers it, that would mean most essential
packages are RC buggy.

(And as I've mention on another reply, dpkg f.ex. is already falling
back to numeric UIDs "just in case". :)

> Please reassign to whatever bootstrapping tool is not doing its job
> properly. Sorry, but this is starting to annoy me.

I'm sorry to hear, but TBH, I'm a bit puzzled about your replies. That
might be due to the apparent conflation of trying to look for a better
solution to the problem presented, and how to incrementally transition
to such a bettter place smoothly already right now?

It's not clear to me whether you oppose the discussion about such
proposals because you consider the current bootstrapping method the
ideal solution, or whether you oppose any progressive move towards
those, as in an all or nothing scenario?

Thanks,
Guillem

Bug#924401: base-files fails postinst when base-passwd is unpacked

[Guillem Jover]

Hi!

On Tue, 2019-03-12 at 16:17:10 +0100, Helmut Grohne wrote:
> Package: base-passwd,base-files,debian-policy
> 
> Debian policy section 3.8 says:
> | Essential is defined as the minimal set of functionality that must be
> | available and usable on the system at all times, even when packages
> | are in the “Unpacked” state.

The behavior for Essential package is scattered around the policy
document, there's also this in § 6.5 for the preinst maintscript:

  ,---
  The package will not yet be unpacked, so the "preinst" script
  cannot rely on any files included in its package. Only essential
  packages and pre-dependencies ("Pre-Depends") may be assumed to be
  available. Pre-dependencies will have been configured at least
  once, but at the time the "preinst" is called they may only be in
  an “Unpacked” or “Half-Configured” state if a previous version of
  the pre-dependency was completely configured and has not been
  removed since then.
  `---

I do think the same things that currently apply to Pre-Depends,
apply to the pseudo-essential set here, even if this is probably
a bit implicit.

> Now we can make a choice:

On Tue, 2019-03-12 at 16:32:30 +, Simon McVittie wrote:
> On Tue, 12 Mar 2019 at 16:17:10 +0100, Helmut Grohne wrote:
> > A. /etc/passwd is part of base-passwd's interface and base-files is
> >right in relying on it working at all times. Then base-passwd is rc
> >buggy for violating a policy must. Fixing this violation is
> >technically impossible.

(This assumes that policy covers the bootstrapping process, which I
don't think it has ever done.)

> If it isn't implementable then it's probably the wrong design.

This is not possible right now, but I think it should be possible in
the future.

> Strictly speaking, I think /etc/passwd *is* part (or maybe all?) of
> base-passwd's Essential interface, but then the Policy requirement that
> it provide this interface even when unconfigured is unimplementable, and
> we can't do unimplementable things.

Yes and no (see above). I also agree these are part of the Essential
interface, but I do not agree it's specified when it is unconfigured
*and* has never been configured before.

> I think it would be reasonable to say that Essential packages are *not*
> entitled to assume that base-passwd provides /etc/passwd, even though
> non-Essential packages are entitled to assume it.

Taking this in its general form seems extremely restrictive to me,
when the only problematic case here is for when the Essential packages
have never been configured before. This would mean possibly more
painful or no logic at all for upgrades and similar.

> > B. /etc/passwd is not part of base-passwd's interface and base-files
> >wrongly relies on its presence rendering base-files rc buggy.

(I think this is also the wrong interpretation, and would mean no
other package could rely on these being always present.)

> Perhaps base-files should use chown 0:0, etc.? That would be more robust.

While that would indeed solve part of the problem, and it's actually
what dpkg is doing itself:

  <https://git.dpkg.org/cgit/dpkg/dpkg.git/tree/debian/dpkg.postinst#n25>

it's still a workaround to the more general problem presented here.

> >Given that
> >we have debootstrap, cdebootstrap, multistrap, and mmdebstrap, it
> >seems like specifying the bootstrap interface would be a good idea.
> >Unfortunately, I don't exactly understand the bootstrap interface at
> >present. In practise, you cannot run postinsts of essential packages
> >in arbitrary order.
> 
> This is certainly more fragile than I'd hope: I've seen debootstrap fail
> in Open Build Service chroots when presented with a modified Essential
> set (in a Debian derivative targeting containers that are not multi-user
> systems and never run on bare metal, which doesn't need everything that a
> "real" Debian system does).
> 
> If we rely on bootstrap implementations having out-of-band knowledge
> of the right order to configure the Essential set, the risk is that
> they need to have different out-of-band knowledge for different target
> distributions, leading to the bootstrap implementation becoming relatively
> tightly coupled to the target distribution.

Yes, off-loading this knowledge from the packages themselves into
external bootstrapping tools is bogus IMO, and something we should
try to fix.

> Maybe the rule should be to retry configuration of each unconfigured
> package until either they all succeed, or forward progress stops being
> made? Pythonesque pseudocode:

I think this would too be a non-ideal workaround.

On Tue, 2019-03-12 at 16:17:10 +0100, Helmut Grohne wrote:
> C. Guillem Jover hin

Bug#918438: orig tarball components with uppercase letters

[Guillem Jover]

Hi!

On Sun, 2019-01-06 at 00:34:30 +, Ian Jackson wrote:
> Package: debian-policy
> Version: 3.9.8.0

> dpkg-source format `3.0 (quilt)' supports what it calls `additional
> orig tarballs', named
>-.orig-.tar.
> 
> The documentation in dpkg-source(1) says
>component can only contain alphanumeric characters and hyphens

This is actually part of the “2.0” format, which got inherited by the
“3.0 (quilt)” format, which is based on the former.

See commit 3748a23e47c1af76f291f3c4caa98eefc45ff508 for the initial
introduction and commit ff03345b7a8d9dd0950dc581c5263373b2a0b406 for
its further restriction.

> This allows the possibility of uppercase letters [1].  But of course
> distinguishing case of letters is troublesome for some computers.
> This specification makes it possible for two different source package
> component files to exist with names which differ only in the case of
> some of the letters - perhaps, even two files which are part of the
> very same package version and must necessarily exist side by side.

Right, and this diverges from the prevalent principle that source and
binary package name components in filenames are lower-cased.

> It seems obvious to me that any reliance on case here is undesirable.

But, pondering about this now, this opens the hypothetical problem of
versions which can have the same exact problem! And we could also have:

  foo_1.0a_amd64.deb
  foo_1.0A_amd64.deb

:/

So when I first read this proposal, I was initially in favor of it and
ready to implement it in dpkg-source. But given the above, I'm now not
so sure whether it make sense anymore TBH.

> (I erroneously mishandled this case in dgit, due to not reading the
> spec properly.  This caused lossage to a Debian downstream.  #916926.)

(See below, I'll clarify this.)

> I would like to arrange, somehow, that our tools and policies ensure
> that filename clashes cannot occur even on case-insensitive
> filesystems.

> I don't think we can solve this in dpkg.  All it could is reject all
> uppercase letters.  That is not backward compatible: we don't want to
> add to the situations where it is not possible to edit and rebuild an
> old source package.

If this is desirable, I don't see why it cannot be done at the dpkg
level TBH. The requirement here is that dpkg-source must preserve the
ability to pack and unpack these packages, but that does not
necessarily mean it needs to allow that by default.

I could see introducing both a warning on upper-case letter usage, and
force options to both pack and unpack such source packages. Then…

> Eventually, when everything is converted, the restriction could be
> made firm (MUST; lintian error) and that would get us into a situation
> where we can't accidentally mess this up in the future.

…later on those warnings could be turned into dpkg-source errors.

> [1] Technically the spec also allows the possibility of non-ASCII
> letters and numbes but I doubt anyone would read that that was
> intended and I am confident that dpkg-source would actually reject
> those.

Yes, in the dpkg context when it comes to names as part of namespaces
and interfaces these tend to be restricted to ASCII characters. I'll
make that explicit in the man page, thanks.

Regards,
Guillem

Bug#913659: Document that not all bugs are policy violations

[Guillem Jover]

On Fri, 2018-11-16 at 12:22:35 -0200, Henrique de Moraes Holschuh wrote:
> How about also adding one that makes it clear that in *Debian*, policy
> follows practice, and not the other way around (which should also
> require seconds just to make sure people agree with this, even if it is
> a decades-long practice in debian-policy)...

Not always, two cases that quickly come to mind are the nocheck and
terse DEB_BUILD_OPTIONS.

Thanks,
Guillem

Bug#907313: Lack of guidelines on purging conffiles in stateless packages

[Guillem Jover]

On Sun, 2018-08-26 at 12:17:23 -0700, Russ Allbery wrote:
> Gioele Barabucci  writes:
> > For instance, apache (the application) is configured by some stub conf
> > in `/etc/apache` that loads *.conf files from directories such as
> > `/etc/apache2/sites-enabled/`. The real files are usually in
> > `/etc/apache2/sites-available/`.
> 
> > The purge process for the apache (the package) removes the configuration
> > files it has installed and the symlinks it has created but leaves the
> > configuration files written by the sysadmin alone.
> 
> Yeah, this is a very interesting example.  If the administrator puts a
> bunch of local configuration in /etc/apache2/sites-available and related
> directories, those are pretty clearly intended to be configuration files
> of Apache, but should we delete everything in those directories on purge?
> I can imagine someone being *quite* surprised by that.
> 
> Another thing that makes this less obvious is that this mechanism is
> frequently used for cross-package cooperation.  In a sense, everything
> under /etc/apache2 is a configuration file for Apache, but a bunch of
> other packages do install files into that hierarchy (including things that
> don't strongly depend on Apache), so running rm -rf in postrm purge isn't
> necessarily correct.

I think the distinguishing factor here is whether a pathname is a
configuration file or a configuration fragments directory. So, I'd
say:

 * configuration file → /etc/foo/foo.conf → remove on purge, even if
   the package did not ship a file there, because this is "virtually"
   owned by the program/package (and I can see in the future these
   being marked as ghost conffiles in the dpkg metadata manifest, for
   example).
 * configuration fragment directory → /etc/foo/foo.d/* → do not remove
   on purge, unless the package ships or creates these itself. This
   preserves local admin changes, and changes from 3rd party packages.

Thanks,
Guillem

Bug#850156: Please firmly deprecate vendor-specific series files [and 1 more messages]

[Guillem Jover]

Hi!

[
  Had this sitting here half drafted, but as I got poked privately also
  due to the apparent incoherence in the first part, I'm sending the
  reply for that now. And will handle the other part later on.

  Although, I guess, because it might be only partly on topic, I'm
  considering whether it might be better to do more extensive write-ups
  elsewhere on my website or similar, to articulate it properly, and
  then possibly link it here, as then I'd also have reference to point
  to. Because I do realize (or at least that seems apparent), mine is
  probably a very minority view within the project, and that people
  going on and on on this kind of tirades and ramblings become annoying
  and/or tiresome fast. So then I could try to stop interjecting on the
  subject, while evaluating how much these situation bother me until
  they become unbearable, at which point I'd just disengage and distance
  myself from work involving them.
]

On Tue, 2018-07-31 at 19:43:32 -0700, Russ Allbery wrote:
> Guillem Jover  writes:
> > If someone wants to see dpkg changed in some way related to this, I'd
> > request the same thing I did to Ian a couple of years ago, gather input
> > from derivatives and other current users, on their reasons for using it,
> > or start a discussion with them on whether they'd be satisfied with
> > potential alternatives, etc.
> 
> At least from a Policy perspective, I don't see any obvious need to remove
> the feature from dpkg regardless of the outcome of this discussion (nor do
> I think that's within the scope that Policy should even consider).  dpkg
> may support all kinds of things that Debian chooses not to use or even
> prohibit in the archive, and I think that division is healthy and good.
> dpkg is a more general tool than just Debian.

Indeed!

> dgit is *also* a more general tool than just Debian, of course, and I can
> see some reasons why the dgit maintainers may aspire to handling every
> package that dpkg can handle, even ones that wouldn't be uploaded to
> Debian, and thus may care about whether the feature is supported at all.
> But I think that's outside the scope of Policy and could be discussed
> betweeen dpkg and dgit maintainers if that's a concern.

Exactly!

> > That Ubuntu finds it to be a problem as a Debian downstream, with these
> > packages percolating into Ubuntu? Well, you could also have tried to
> > argue your case to the ftp-masters and lintian maintainers, that this
> > is making your life difficult, and whether they would reject packages
> > with debian/patches/ubuntu.series. Or convince the maintainers in
> > Debian using them that this is in fact not helping you.
> 
> > Apparently, you do not even need to do that anymore, you just need to
> > get a ctte to ban them from Debian.
> 
> I'm a bit confused about why you're upset with us asking the Technical
> Committee to make a decision when you're (apparently, maybe I'm
> misunderstanding) fine with asking ftp-master to reject such packages.  Of
> those two things, I think the Technical Committee discussion is more open,
> collaborative, and accountable than asking ftp-master to reject packages!
> (Indeed, if I were ftp-master, I would just bump the request to the
> Technical Committee anyway, since I wouldn't want to make that sort of
> decision on the basis of just the ftp-master delegation authority.)

The root problem is that AFAICS there's been at least three different
ways the interested parties have expressed a desire they wanted this
getting enforced, for quite different reasons, that have gotten mingled
in here:

  * Ban this everywhere at the dpkg level (dgit and Ubuntu interest).
  * Ban this for all Debian at the ftp-master/lintian level via policy
(Ubuntu interest, fallback dgit interest).
  * Ban this at the vendor level (Ubuntu interest)

Ideally? Yes, I'd like for any of the interested parties to get the users
of the feature and possibly convince them they are doing it wrong, and
if so why, and what they could do instead, or for the complaining part
to realize this might have some merit, and solve the apparent problems
they have with technical means.

But for the Ubuntu case, which is what I was explicitly discussing with
Steve, the reality is slightly different. The Ubuntu organization is
reigned by their own rules, and it's a different entity to Debian, and
how they reach their conclusions and policies is for them to decide, and
even if how they do that might not be my cup of tea, I'm just not involved
and I don't think it's my place to question their processes.

So, if Ubuntu, as an organization, decides that the ubuntu.series file is
not good for them, and they are going to ban it no matter what, probably
at their bo

Bug#883950: Next steps on "[GPL-3+]" proposal

[Guillem Jover]

On Thu, 2018-08-02 at 16:45:52 +0800, Markus Koschany wrote:
> Am 02.08.2018 um 16:27 schrieb gregor herrmann:
> > On Thu, 02 Aug 2018 15:13:26 +0800, Markus Koschany wrote:
> >> Nothing will break because no tool besides Lintian checks
> >> debian/copyright for copyright format 1.0 compatibility. 
> > 
> > This is not correct.
> > 
> > There are at least cme (with libconfig-model-dpkg-perl), decopy,
> > probably some of licensecheck/license-reconcile/cdbs (or whatever
> > consumes copyright_hints), dh-make-perl refresh,
> > libdebian-copyright-perl, and probably others which check and/or
> > create/update copyright-format 1.0 files.

libdpkg-perl also has basic parsing support, to stuff could be using
that as a foundation to parse the files.

> Ok, so you have to tell your Perl tools that you can write a license
> paragraph without also having to specify the license in a separate
> standalone license paragraph. This is not really an incompatible change
> but merely a _new_ way to document it. You still have to write new code
> regardless if there is a new copyright format version or not. I don't
> think this is a real issue

The point is that this spec is about a "machine readable format", if
you change the semantics in a backwards incompatible way, then there's
no way for parsers and validators to know whether the file is correct
or not according to the spec (which is generic and should not be tied
to Debian or whether some other package contains what). Parsers might
also start silently failing because the format didn't change, etc. And
users that might want to check whether their files have been updated
might not even be able to easily tell, so this is also making life
for humans more difficult.

> and it would still be better than to shift
> the work to everyone else in existence. We can also just keep allowing
> the boilerplate approach _together_ with the implementation of this
> proposal, so that nobody can complain about it?

As has been mentioned before, you should not need to bump the version
if you don't use the new format; if you do, you have aleady changed
the file anyway and might as well change the version digit.

Thanks,
Guillem

Bug#228692: User/group creation/removal in package maintainer scripts

[Guillem Jover]

Hi!

On Tue, 2018-07-31 at 17:53:50 +0200, Andreas Henriksson wrote:
> I'm going to attempt to first collect what I've picked up both from the
> previously mentioned mailinglist thread (and other similar ones) and
> what I've seen when reviewing maintainerscripts of packages in the
> archive. Hopefully others can speak up if they disagree or think I've
> missed a common convention. Later we can attempt to formulate a specific
> wording for policy.

Something I'd like to eventually consider doing, is make dpkg get
support for system user/group handling, in a declarative way. This
makes more sense once dpkg starts tracking all fsys metadata.

I thought I had sent this out, but apparently not, so I've publised
my local notes into an early draft of this:

  .

I also think that several of the contentious topics should just end
up being configurable by the admin, and then we'd pick the most
conservative default. Such as not removing the users on package
removal/purge, but let the admin configure to do so.

So I'd appreciate that any possible changes to policy would not make
something like the above unworkable.

Thanks,
Guillem

Bug#850156: Please firmly deprecate vendor-specific series files [and 1 more messages]

[Guillem Jover]

On Tue, 2018-07-31 at 17:23:31 -0700, Steve Langasek wrote:
> On Wed, Aug 01, 2018 at 02:12:13AM +0200, Guillem Jover wrote:
> > I'm detaching dpkg from this, I don't see anything constructive to do
> > out if this, TBH.
> 
> > If someone wants to see dpkg changed in some way related to this, I'd
> > request the same thing I did to Ian a couple of years ago, gather
> > input from derivatives and other current users, on their reasons for
> > using it, or start a discussion with them on whether they'd be
> > satisfied with potential alternatives, etc.
> 
> I will limit myself to pointing out the asymmetry of this requirement:
> Ubuntu as a derivative was not consulted before ubuntu.series was inflicted
> on us, but other derivatives who like this feature must be consulted before
> upstream will un-break it for Ubuntu.

I'm not sure why any possible asymmetry is surprising at all? Removing
or disabling features is always harder than adding them? More so if
there are known users happy with them? And also when the objections do
not seem to be generally applicable.

Should the addition of the feature have gathered more input from
derivatives? Perhaps. AIUI, for Ubuntu at least Colin was consulted.

That Ubuntu finds it to be a problem as a Debian downstream, with these
packages percolating into Ubuntu? Well, you could also have tried to
argue your case to the ftp-masters and lintian maintainers, that this
is making your life difficult, and whether they would reject packages
with debian/patches/ubuntu.series. Or convince the maintainers in
Debian using them that this is in fact not helping you.

Apparently, you do not even need to do that anymore, you just need to
get a ctte to ban them from Debian. I'm not sure how requiring input
from affected parties for a change in dpkg that would affect not only
Ubuntu but all derivatives and local usage, seems onerous at all, TBH…

Regards,
Guillem

Bug#850156: Please firmly deprecate vendor-specific series files [and 1 more messages]

[Guillem Jover]

Control: reassign -1 debian-policy 3.9.8.0

On Mon, 2018-07-30 at 06:15:42 +0200, Guillem Jover wrote:
> In any case, I discussed this in a private mail interchange with Ian
> a couple of years ago (AFAIR). My reply back then was that I don't
> personally feel very strongly about the feature, that I think it has
> some merit and that before even considering any kind of deprecation
> I'd like to get input from derivative developers (Ubuntu is obviously
> not the only derivative) and other people using it for their rationale,
> etc.
> 
> Personally I do have a preference for doing unconditional patching,
> that ideally does feature selection at configure/build/run-time. But I
> don't think this is generally applicable to all programming languages
> or environments, which might not have the notion of some meta language
> or preprocessor, or might not be able to portably select OS and then
> a vendor and similar. At least w/o creating a huge mess of code.
> 
> Also if any potential deprecation might imply that people will switch
> from declarative to vendor-conditionalized patching from within
> debian/rules, well, that looks like a regression to me.

In addition, there's the confounding of the source package as a
transport format, with the tools used to extract them, the process
to do so, and the end result on disk.

The vendor-specific series can be thought of just as a serialization
of multiple branches. How this ends up being unpacked could vary, the
unpacker could generate say a git repo, and create multiple branches,
and point HEAD to the current vendor. This is a tooling behavior issue.

If this is really causing problems to dgit, then it feels to me like
a self-inflicted consequence of its design.

> I'm definitely not even going to consider removal of extraction support,
> because that would break at least historic source unpacking. That's
> the price of adding these kinds of features into dpkg.
> 
> When it comes to deprecation of the packing, see above. When I saw this
> thread I initially though that at least adding options to forbid packing
> and unpacking this kind of source would be a nice compromise, but with
> the ctte being involved I've lost any motivation for that.

> In any case I'm not even sure why dpkg is any kind of blocker for this
> at all, because acceptance into the Debian archive is controlled by
> ftp-masters, f.ex via lintian and its auto-reject list. Well, it might
> be if there's some kind of intention to try to block this even for other
> unrelated derivatives…

I'm detaching dpkg from this, I don't see anything constructive to do
out if this, TBH.

If someone wants to see dpkg changed in some way related to this, I'd
request the same thing I did to Ian a couple of years ago, gather
input from derivatives and other current users, on their reasons for
using it, or start a discussion with them on whether they'd be
satisfied with potential alternatives, etc.

Thanks,
Guillem

Bug#904608: Support specifying upstream VCS location in debian/control

[Guillem Jover]

Hi!

On Wed, 2018-07-25 at 18:20:52 -0700, Jonathan Nieder wrote:
> Sean Whitton wrote:
> > On Wed 25 Jul 2018 at 05:14PM +0100, Iain Lane wrote:
> >> Some tools, like git-buildpackage, can support merging an upstream's
> >> version history into Debian packaging repositories. This enables more
> >> rich usage of (D)VCS when packaging - for example `git blame' works
> >> properly.
> >>
> >> Currently there's no canonical place to specify where upstream's VCS is
> >> located so people have to set this up manually themselves. If there were
> >> such a place, it would be possible for tools like `gbp clone' to
> >> configure the VCS to know about the upstream history when checking out a
> >> packaging repository.
> 
> I like this use case.  Thanks for bringing it up.
> 
> > In fact, there is: the Repository field in debian/upstream/metadata.[1]
> 
> Neat!
> 
> I would have expected to find this information in debian/copyright.  The
> Source field there sometimes names an upstream VCS but isn't required to
> do so; I'd be in favor of some tightening of the requirements in
> copyright-format based on how packages in the archive have been using
> the field (for example, encouraging a list of lines each of which has
> the same format as Vcs-* fields).

I've mentioned this before, and while I appreciate some of the good
properties that a structured format such as YAML provides, using it
for any metadata that could be of potential use by low-level package
tooling (such as dpkg-dev), means this information would not be usable
there as I'd really not like to impose a depedency on YAML modules and
shared libraries or similar.

So, from my PoV, it should end up in a deb822 formatted file.

> >> The request here is to ask whether this would be suitable for
> >> debian/control, along the lines of the Vcs-* fields specified in 5.6.26
> >> and the Homepage field in 5.6.23.
> 
> My feeling is that it doesn't belong in debian/control.

> The debian/control file is the source for control fields that appear
> in the binary package, Packages file, and Sources file.  If I
> understand correctly, the primary consumers of this field would
> already have a copy of the source (via Vcs-Git) so they can get the
> information from other files in the debian/ directory; they don't need
> to get it from the Sources file.

I think it could be argued that both debian/control and debian/copyright
might be appropriate candidates. Because both have some precedent
(Homepace, Source/Upstream-*). I also agree that a free form field would
not be good enough, and that a more structured field should be used
instead, similar to our Vcs- ones.

> With that in mind, Sean's suggestion of using debian/upstream/metadata[1]
> sounds good to me.  Would it work well for you?

From dpkg-dev PoV, that would pretty much mean the information is
unreacheable.

> >> If so, I'd be happy to propose wording for policy. I'm not set on any
> >> particular name, so please feel free to weigh in on that if you'd like.
> >
> > Even if we did want to add this to d/control files, we would want to see
> > it already used in d/control files in the archive before documenting
> > that in Policy.
> 
> On this subject more generally, I think there's a bit of a
> chicken-and-egg problem.  If we want new fields in the Packages or
> Sources file, it does make sense to coordinate a little with potential
> consumers, and it's not obvious to me where the right place to start
> that is (d...@packages.debian.org? a DEP? something else?).  So I
> understand why people ask policy team.
> 
> For the future, I'd like to have good advice to offer for this kind of
> case, even if that advice is as simple as "ask d...@packages.debian.org"
> or "ask ftp-master", say.

I think it depends, but yeah involving potential consumers would be
important. In some cases trying to get consensus on debian-devel might
also work (ahem :). Or bringing it up in policy and discussing there.
We did this for example for the nocheck build option.

In any case I don't mind dpkg being the entry point for this kind of
queries, but while in many/most cases I'll try to poke holes at the
proposals, with naming, location, purpose, etc, it must not be the
exclusive right of the dpkg maintainer to decide on this, and any
other relevant party should get involved and have a say.

Thanks,
Guillem

Bug#850156: Please firmly deprecate vendor-specific series files [and 1 more messages]

[Guillem Jover]

On Wed, 2018-04-18 at 09:15:25 -0700, Sean Whitton wrote:
> On Wed, Apr 18 2018, Ian Jackson wrote:
> > FAOD I feel very strongly about this.  The bug is over a year old.
> > Can the Policy Editors please tell me when it would be apprropiate to
> > escalate this to the TC ?

*Sigh*

> Sorry, I wrote my other e-mail before reading this.
> 
> ISTM that we can move towards consensus on this bug by developing
> tooling that allows downstreams to make better use of dgit.  That is, we
> offer them something that does the work that vendor-specific series
> files are doing.
> 
> We are actively working on the relevant processes and tools right now.
> Let's see what things look like once we reach the end of that work
> before escalating this bug anywhere.

[ Well, apparently not, because it was referred to the ctte anyway…
  I had pending a reply to this thread, but when I saw your mail
  I thought this was going to be solved properly by technical means,
  and deprioritized it, should have known better I guess. It seems
  we are going through the hammer of authority yet again. ]

In any case, I discussed this in a private mail interchange with Ian
a couple of years ago (AFAIR). My reply back then was that I don't
personally feel very strongly about the feature, that I think it has
some merit and that before even considering any kind of deprecation
I'd like to get input from derivative developers (Ubuntu is obviously
not the only derivative) and other people using it for their rationale,
etc.

Personally I do have a preference for doing unconditional patching,
that ideally does feature selection at configure/build/run-time. But I
don't think this is generally applicable to all programming languages
or environments, which might not have the notion of some meta language
or preprocessor, or might not be able to portably select OS and then
a vendor and similar. At least w/o creating a huge mess of code.

Also if any potential deprecation might imply that people will switch
from declarative to vendor-conditionalized patching from within
debian/rules, well, that looks like a regression to me.

I'm definitely not even going to consider removal of extraction support,
because that would break at least historic source unpacking. That's
the price of adding these kinds of features into dpkg.

When it comes to deprecation of the packing, see above. When I saw this
thread I initially though that at least adding options to forbid packing
and unpacking this kind of source would be a nice compromise, but with
the ctte being involved I've lost any motivation for that.

In any case I'm not even sure why dpkg is any kind of blocker for this
at all, because acceptance into the Debian archive is controlled by
ftp-masters, f.ex via lintian and its auto-reject list. Well, it might
be if there's some kind of intention to try to block this even for other
unrelated derivatives…

Thanks,
Guillem

Bug#902612: Packages should not touch users' home directories

[Guillem Jover]

On Thu, 2018-06-28 at 10:23:17 -0300, David Bremner wrote:
> Sean Whitton  writes:
> >> OK. Something like this?
> >>
> >> Packages must not contain files in /home, and packages' maintainer
> >> scripts must not write to users' home directories. The programs in
> >> those packages may create directory hierarchies as described in
> >> §3.8.3 "Home Directory Specifications and Conventions" when run by
> >> a user.
> >>
> >> I'm not so sure whether this belongs in the FHS section?
> >
> > Indeed.  I thought that the requirement that packages don't contain
> > files in /home was in Policy already, but it seems that it is not.
> >
> 
> > So filing as a separate bug.
> >
> >> I think it's a point about how our packages are required to behave,
> >> rather than about the directories that can exist and their
> >> purposes. The directory hierarchies are still the same, regardless of
> >> how they're created.
> >
> > The wording of FHS is not about directories that can exist -- it says
> > that distributions "can create" or "may create" the dirs.  That's what
> > we want to forbid.
> 
> By the way there are packages (gitolite3 is the one I know) that create
> user home directories in /var for some package dedicated user. I'm not
> sure how to distinguish cleanly between that case and the human users
> case. I guess having a home directory in /home is a reasonable proxy for 
> being human.

I think the distinction you are looking for is system users, as the
ones requested in the gitolite3 case with «adduser --system …».

Thanks,
Guillem

Bug#891216: seconded 891216: Requre d-devel consultation for epoch bump

[Guillem Jover]

On Thu, 2018-06-28 at 14:34:06 +0200, Wouter Verhelst wrote:
> On Thu, Jun 28, 2018 at 01:43:17PM +0200, Guillem Jover wrote:
> > On Thu, 2018-06-28 at 13:03:56 +0200, Wouter Verhelst wrote:
> > > On Thu, Jun 28, 2018 at 11:39:53AM +0100, Ian Jackson wrote:
> > > > The requirement to consult d-d has worked very well with Pre-Depends.
> > > > Many pointless and harmful Pre-Depends have been avoided this way,
> > > > with very low levels of project-wide effort.
> > > 
> > > Yes. There's a difference though.
> > 
> > Sure, but not in their harmfulness, though. :)
> 
> That's just a matter of opinion.

I don't think it is though. I'd even say bogus epochs bumps are
potentially more harmful than Pre-Depends! Pre-Depends are localized
in a single package, and their effects can be easily tested, but the
validity of the rationale for its introduction might be difficult or
impossible to check automatically, though. But the full effects of
epoch bumps are not easily testable at all and they are changes with
global effect, for all rdepends (in-archive and local), local forks,
etc.

From the categorization I did in the dpkg FAQ, there are two main
scenarios:

  * valid bumps due to version schema change, where even though the
version-space gets reset, the releases have continuity.
  * invalid bumps, either because they are completely unnecessary, or
because the releases break the continuity.

When breaking the release continuity, and forcing an invalidation of
all relationships, it means we might be breaking upgrade scenarios,
satisfiability checks, interface requirement checks, etc, this is a
silent and very harmful thing. As I mention on the FAQ the
relationships can be hidden in many places, not just in control
metadata, which makes checking for this way more difficult.

And just to be clear, epochs have their place, that's why they were
created. They might be somewhat ugly and confusing, but when used
correctly they are better than any other alternative, and any
ill-effects are unavoidable anyway. The problem is when those
ill-effects are caused by bogus usage of epochs.

> > > Incorrect pre-depends are actively harmful. They may cause dependency
> > > loops which dpkg cannot fix, and may therefore result in problems that
> > > go way beyond the package which introduced the incorrect pre-depends. In
> > > that context, I agree that trying to reach consensus on -devel before
> > > introducing a pre-depends is a good idea.
> > > 
> > > Incorrect epochs are a nuisance at best. There's a myth going around
> > > that they cause a "stigma", which I suspect is where this comes from,
> > > but in actual fact they're just a few extra characters in a version
> > > number with some special semantics, and nothing beyond that.
> ^^

BTW something I missed in my previous reply, the fact that it might
(should! :) be considered a stigma is IMO a good thing, because it
might make people think at least twice before introducing them. :)

> > No, they are not just a decorator for the version, they have a
> > semantic meaning,
> 
> I know that.

Sorry, my apologies, I missed that part you underline. I my defense I'd
say that over the years I've seen so many very experienced maintainers
using epochs incorrectly or trying to defend their non-problematic nature
and goodness, that I've reached a point where I just assume epochs are
generally missunderstood. I realize that can obviously end up coming
across as very condescending. :/

> > they just reset the sorting order of all previous versions and thus
> > invalidate any previous relationships.
> 
> Not any more than do upstream version numbers towards debian revisions.
> 
> If you consider epoch-zero versions to have "no epoch" (which is wrong),
> then yes, they imply a "reset". 
> 
> But really, an epoch is just prepending an extra version component
> before the version number. epochless versions have an implicit zero
> (okay, I shouldn't be telling you this).

See my comment above about release continuity.

> Honestly, if this is going to become a requirement, and I didn't want to
> be bothered with it, I would just use . rather than : as my epoch
> separator whenever I need to introduce an epoch. The result regarding
> upgrades etc is *exactly* the same.

Anything that has the same semantic effect of an epoch has potentially
the same ill-consequences. The +really convention is as harmful as
using an epoch, the main advantage is that any uncaught breakage in
relationships will eventually remedy itself once the package gets back
into its previous release continuity.

Using a convention that keeps appendin

Bug#891216: seconded 891216: Requre d-devel consultation for epoch bump

[Guillem Jover]

On Thu, 2018-06-28 at 13:03:56 +0200, Wouter Verhelst wrote:
> On Thu, Jun 28, 2018 at 11:39:53AM +0100, Ian Jackson wrote:
> > Wouter Verhelst writes ("Bug#891216: seconded 891216: Requre d-devel 
> > consultation for epoch bump"):
> > > I would oppose this change.
> > 
> > > Documenting why you should not use epochs in certain cases does make
> > > sense, but I think we should trust our developers to understand what
> > > they're being told, rather than requiring people uselessly email -devel
> > > (and clutter that mailinglist, which also causes harm).
> > 
> > My perception of our experience is that trying to get people to make
> > the right choice solely by writing things in documents is not
> > effective.
> > 
> > Epochs are frequently misunderstood and used where it would have been
> > better not to use them.  Proper usage of an epoch is sufficiently rare
> > that asking for a review is reasonable.  We do not have a central code
> > review board or anything; debian-devel is the closes thing we have.

I think I've spotted more wrong usage of epochs from reading
changelogs (when it's too late already), than correct ones, TBH.

> > The requirement to consult d-d has worked very well with Pre-Depends.
> > Many pointless and harmful Pre-Depends have been avoided this way,
> > with very low levels of project-wide effort.
> 
> Yes. There's a difference though.

Sure, but not in their harmfulness, though. :)

> Incorrect pre-depends are actively harmful. They may cause dependency
> loops which dpkg cannot fix, and may therefore result in problems that
> go way beyond the package which introduced the incorrect pre-depends. In
> that context, I agree that trying to reach consensus on -devel before
> introducing a pre-depends is a good idea.
> 
> Incorrect epochs are a nuisance at best. There's a myth going around
> that they cause a "stigma", which I suspect is where this comes from,
> but in actual fact they're just a few extra characters in a version
> number with some special semantics, and nothing beyond that.

No, they are not just a decorator for the version, they have a
semantic meaning, they just reset the sorting order of all previous
versions and thus invalidate any previous relationships.

For the valid use cases that's an unavoidable transition that one has
to handle, but for the invalid cases it's just unnecessary breakage in
the archive and out-of-archive, in most cases silent breakage!

Please see
.

> Yes, it's correct that you cannot get rid of an epoch, once it has been
> introduced. This has indeed sometimes caused issues when downstream
> people have introduced epochs in their versions of our packages, causing
> what in effect is an arms race -- but there really is no reason why
> asking on -devel will fix *that* particular issue; I don't think that
> downstreams who fight with us on epochs will do that anyway, so that
> just leaves the debian package maintainer to DTRT and bump the epoch
> there.

That's really not the right thing to do. That's the equivalent of
introducing bogus changes into our packages to be bug-compatible with
external entities. If a downstream unilaterally bumps an epoch, that's
their burden to carry.

> Yes, it's correct that epochs cause confusion, because some bits of our
> infrastructure drop the epoch in the filename. I submit that that is in
> fact a bug in that bit of infrastructure; epochs are a critical part of
> the version number, and they should not be dropped, ever.

I don't think this is a black/white thing, it's a matter of trade-offs,
IMO. Using colons in filenames can cause problems in various places,
filesystems, or even unintended network access from tar (w/o using
--force-local), for example, etc.

> But if we're going to introduce the *requirement* to ask on -devel for
> every nitty bitty thing that might possibly somewhere down the road
> cause some confusion in some inexperienced developer, then in the end
> the -devel mailinglist will devolve to a list where senior DDs come by
> to ask "can I please introduce a postinst to my package?" and that's
> just a waste of everyone's time.

The big difference is that introducing epochs is (currently) an
irreversible process. Even Pre-Depends can be fixed after the fact!
And I've never been bothered for having to send a notice to d-d, even
when I was pretty confident I was right, as in:

  "Hey I'll add Pre-Depends blah for package blur in X days. Please
   shout if you think this is a problem."

If I had to choose between the two, I'd remove the requirement for
Pre-Depends.

> That (and a feeling that I'll just balk at wasting my time by asking on
> -devel "please pretty please can I introduce an epoch please") is why
> I'm oposed to introducing this requirement.

I guess I don't see it as some kind of begging. For me it's a change that
has far wider consequences, just like any other transition, a

Bug#846970: Patch to document Build-Indep-Architecture field

[Guillem Jover]

On Fri, 2018-06-15 at 19:05:17 +0100, Sean Whitton wrote:
> Sean Whitton writes ("Bug#846970: Patch to document Build-Indep-Architecture 
> field"):
> > > +``Build-Indep-Architecture``
> > > +

> Zooming out a bit:
> 
> We do not normally add fields to Policy until they are already in use in
> the archive.
> 
> Looking at codesearch.debian.net reveals that only a single package is
> actually using this field at present.  I haven't checked, but presumably
> the field is not supported by the Debian autobuilders.
> 
> So I would like to suggest we just close this bug until those who would
> actually be involved in implementing support for the field chime in to
> say that it is needed, and exactly what's needed.

This has been discussed multiple times over the years [D]. I still
consider both the purpose of this field a workaround [W], and its
name to be potentially very confusing [N]. :)

Regarding the name, in addition to what I mention in [N], there's also
the fact that we already have Build--Indep and Build--Arch.
With the second being in a way a poor name, and while not Architecture,
it would make adding Build-Indep-Arch.* less than ideal.

I think a better approach for many of these cases might be
, but I'd
otherwise consider the more immediate and realistic cross-compiler
option to be way preferable (even the variant of building the
cross-compiler from within the package itself), than this field.

And I'm saying this as one of the perpetrators that maintained and/or
introduced many of the abominable packaging aberrations that abuse
the Architecture field and semantics in that way, and would need of
something like this. :)

Thanks,
Guillem

[D] <…other earlier discussions omitted…>
,
.
[W] 
[N] 

Bug#891216: Requre d-devel consultation for epoch bump

[Guillem Jover]

Hi!

On Fri, 2018-02-23 at 13:26:01 +, Ian Jackson wrote:
> Package: debian-policy
> Version: 4.1.3.0

> We had another thread on debian-devel recently, in which it once again
> became evident that epochs are misunderstood.  Epoch bumps should be
> rare and there are often better solutions.  I suggest that we should
> ask people to consult debian-devel.
> 
> Also we should encourage the +really convention rather than epoch
> bumps.

I also ended up writing a new dpkg FAQ entry, given that thread:

  


Thanks,
Guillem

Bug#888978: developers-reference: 5.6.5 queued logfile

[Guillem Jover]

Hi!

On Wed, 2018-01-31 at 22:29:30 +0100, Thorsten Alteholz wrote:
> Package: developers-reference
> Version: 3.4.19
> Severity: normal
> Tags: patch

> in paragraph 5.6.5 you recommend to login to ssh.debian.org to find the
> logfile for queued. This seems to be no longer true. Nevertheless the
> logfiles can be found on usper.

> diff --git a/common.ent b/common.ent
> index 1bbed6a..561af11 100644
> --- a/common.ent
> +++ b/common.ent
> @@ -48,6 +48,7 @@
>  
>  
>  
> +

I think using ssh.upload.debian.org here would be the more correct fix?

Thanks,
Guillem