Bug#986320: Stronger advice on when to use native packages

[Russ Allbery]

Package: debian-policy
Version: 4.5.1.0
Severity: wishlist

Currently, Debian Policy is silent on when it's appropriate to use a
native package, but there may be a project consensus aganist using
native packages when the software has an existence outside of Debian.

Even if that consensus does not exist, there is probably consensus
that native packages are a poor match for large packages (because of
the inefficiency of making small updates to the packaging of native
packages), and there may be other cases where we can give stronger
guidance.

Probe the project consensus here and see if Policy should say something
stronger.

(See #542288 for some of this discussion.)

-- System Information:
Debian Release: bullseye/sid
  APT prefers unstable
  APT policy: (990, 'unstable'), (500, 'unstable-debug'), (1, 'experimental')
Architecture: amd64 (x86_64)

Kernel: Linux 5.10.0-5-amd64 (SMP w/8 CPU threads)
Kernel taint flags: TAINT_PROPRIETARY_MODULE, TAINT_OOT_MODULE, 
TAINT_UNSIGNED_MODULE
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

debian-policy depends on no packages.

Versions of packages debian-policy recommends:
ii  libjs-sphinxdoc  3.4.3-2

Versions of packages debian-policy suggests:
pn  doc-base  

-- no debconf information

Processed: user debian-pol...@packages.debian.org, limit package to debian-policy, tagging 682347 ...

[Debian Bug Tracking System]

Processing commands for cont...@bugs.debian.org:

> user debian-pol...@packages.debian.org
Setting user to debian-pol...@packages.debian.org (was r...@debian.org).
> limit package debian-policy
Limiting to bugs with field 'package' containing at least one of 'debian-policy'
Limit currently set to 'package':'debian-policy'

> tags 682347 = patch
Bug #682347 [debian-policy] mark 'editor' virtual package name as obsolete
Ignoring request to alter tags of bug #682347 to the same tags previously set
> usertags 682347 = normative
Usertags were: normative proposal simple.
Usertags are now: normative.
> tags 542288 = patch
Bug #542288 [debian-policy] Versions for native packages, NMU's, and binary 
only uploads
Bug #850729 [debian-policy] debian-policy: Documenting special version number 
suffixes
Added tag(s) patch.
Added tag(s) patch.
> usertags 542288 = normative
Usertags were: normative proposal.
Usertags are now: normative.
> tags 932696 = patch
Bug #932696 [debian-policy] Please document Haskell team style Vcs-Git sytax
Ignoring request to alter tags of bug #932696 to the same tags previously set
> usertags 932696 = normative
Usertags were: normative proposal.
Usertags are now: normative.
> tags 944920 = patch
Bug #944920 [debian-policy] Revise terminology used to specify requirements
Added tag(s) patch.
> usertags 944920 = normative
There were no usertags set.
Usertags are now: normative.
> thanks
Stopping processing here.

Please contact me if you need assistance.
-- 
542288: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=542288
682347: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=682347
850729: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=850729
932696: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=932696
944920: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=944920
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems

Bug#875531: "editor +42 filename" -- accept or reject?

[Russ Allbery]

Adam Borowski  writes:

> Now that you folks are dealing with the "editor" virtual package, and,
> what interests me here, the alternative for /usr/bin/editor -- could you
> please process this proposal as well, and either accept or close it?

> My point is that, all but one (e3) current alternatives allow
> positioning the cursor on a given line, and various users take
> inconsistent approach as to when this feature can be used.

Thank you for doing the research to confirm this is supported by all the
editor alternatives!  That makes this fairly easy.  Attached is a proposed
diff.  This is on top of the diff for #682347; it's separable, but I'd
prefer to get that seconded and merged first.

Note that, based on your research, this will make e3 RC-buggy.  (Obviously
this wouldn't be RC for this upcoming release.)

-- 
Russ Allbery (r...@debian.org)  

diff --git a/policy/ch-customized-programs.rst b/policy/ch-customized-programs.rst
index 27abebd..d58e297 100644
--- a/policy/ch-customized-programs.rst
+++ b/policy/ch-customized-programs.rst
@@ -93,6 +93,14 @@ alternative should have a slave alternative for
 ``/usr/share/man/man1/pager.1.gz`` pointing to the corresponding manual
 page.
 
+Packages that register as an alternative for ``/usr/bin/editor`` must
+support the following features:
+
+- When invoked as ``editor ``, they open  for
+  editing.
+- When invoked as ``editor +N ``, they open  for
+  editing and position the cursor or equivalent at line ``N`` of the file.
+
 Packages that register as an alternative for ``/usr/bin/editor`` should
 also provide the virtual package ``editor`` by including it in the
 ``Provides`` control field. The package providing the current default

Bug#542288: debian-policy: Version numbering: native packages, NMU's, and binary only uploads

[Holger Levsen]

On Fri, Apr 02, 2021 at 10:26:47AM -0700, Russ Allbery wrote:
> I'll therefore propose that we move the discussion of whether to give
> stronger advice on when to use native packages to a separate bug.  Once
> this is merged, there will be some text in Policy defining native
> packages, so it will be easier to work on that.
> 
> Sound good?

sounds good, thanks!


-- 
cheers,
Holger

 ⢀⣴⠾⠻⢶⣦⠀
 ⣾⠁⢠⠒⠀⣿⡁   holger@(debian|reproducible-builds|layer-acht).org
 ⢿⡄⠘⠷⠚⠋⠀ PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C
 ⠈⠳⣄


signature.asc
Description: PGP signature

Bug#542288: debian-policy: Version numbering: native packages, NMU's, and binary only uploads

[Russ Allbery]

Holger Levsen  writes:

> I'm not sure if in this regard I would have liked the previous version
> better, as the paragraph about native packages is the only one which I
> would like to see extended to explain that it has been observed that
> packages we thought were native to Debian were not really native to
> Debian at all and require modifications in Debian downstream
> distributions, which are harder to do when the packages are native (eg
> modified orig.tar's despite no upstream changes).

This is generally my opinion as well, but I know there are other folks who
disagree, and I'm therefore not sure whether this has consensus.  (Native
packages are a lot simpler.)  I decided to take the easy way out and note
that this bug was originally about documenting NMU and stable upload
version conventions, so setting policy for when people should use native
packages is a bit out of scope.

I'll therefore propose that we move the discussion of whether to give
stronger advice on when to use native packages to a separate bug.  Once
this is merged, there will be some text in Policy defining native
packages, so it will be easier to work on that.

Sound good?

-- 
Russ Allbery (r...@debian.org)  

Bug#542288: debian-policy: Version numbering: native packages, NMU's, and binary only uploads

[Russ Allbery]

Simon McVittie  writes:

> The other way is to repackage the new upstream release from first
> principles, either because it's a new release from an upstream branch
> that's older than the one in testing/unstable (like src:flatpak in
> Debian 10), or because testing/unstable already has packaging changes
> that aren't suitable for stable (like src:dbus in Debian 10). In this
> case it would be misleading to use a version number like
> flatpak_1.2.5-1~deb10u1 (because 1.2.5-1 never existed) or
> dbus_1.12.20-1~deb10u1 (because 1.12.20-1 did exist, but the version of
> dbus in stable was not based on it, and does not have the downstream
> changes from that version). Instead, the convention that has emerged is
> to use 0+deb10u1, by analogy with the convention that an NMU introducing
> a new upstream release has revision number 0.1.

Ah, thanks for this.  I wasn't aware this convention existed.

Here's a new draft, which is unfortunately quite a bit longer because it's
hard to make this clear.  I also added the +really convention explicitly,
since it's mentioned immediately above, and noted that the stable update
and backport conventions can, in rare circumstances, stack.

-- 
Russ Allbery (r...@debian.org)  

diff --git a/policy/ch-controlfields.rst b/policy/ch-controlfields.rst
index a21a510..2fd6868 100644
--- a/policy/ch-controlfields.rst
+++ b/policy/ch-controlfields.rst
@@ -582,20 +582,17 @@ The three components here are:
 alphanumerics and the characters ``+`` ``.`` ``~`` (plus, full stop,
 tilde) and is compared in the same way as the ``upstream_version`` is.
 
-It is optional; if it isn't present then the ``upstream_version``
-may not contain a hyphen. This format represents the case where a
-piece of software was written specifically to be a Debian package,
-where the Debian package source must always be identical to the
-pristine source and therefore no revision indication is required.
+It is conventional to restart the ``debian_revision`` at ``1`` each
+time the ``upstream_version`` is increased.
 
-It is conventional to restart the ``debian_revision`` at ``1``
-each time the ``upstream_version`` is increased.
+The package management system will break the version number apart at
+the last hyphen in the string (if there is one) to determine the
+``upstream_version`` and ``debian_revision``. The absence of a
+``debian_revision`` is equivalent to a ``debian_revision`` of ``0``.
 
-The package management system will break the version number apart
-at the last hyphen in the string (if there is one) to determine
-the ``upstream_version`` and ``debian_revision``. The absence of a
-``debian_revision`` is equivalent to a ``debian_revision`` of
-``0``.
+Presence of the ``debian_revision`` part indicates this package is a
+non-native package (see :ref:`s-source-packages`).  Absence indicates
+the package is a native package.
 
 When comparing two version numbers, first the epoch of each are
 compared, then the ``upstream_version`` if epoch is equal, and then
@@ -625,6 +622,8 @@ These two steps (comparing and removing initial non-digit strings and
 initial digit strings) are repeated until a difference is found or both
 strings are exhausted.
 
+.. _s-avoid-epochs:
+
 Epochs should be used sparingly
 ^^^
 
@@ -639,13 +638,132 @@ Epochs should not be used when a package needs to be rolled back.
 In that case, use the ``+really`` convention: for example, if you
 uploaded ``2.3-3`` and now you need to go backwards to upstream 2.2,
 call your reverting upload something like ``2.3+really2.2-1``.
-Eventually, when we upload upstream 2.4, the +really part can go away.
+Eventually, when we upload upstream 2.4, the ``+really`` part can go away.
 
 Epochs are also not intended to cope with version
 numbers containing strings of letters which the package management
 system cannot interpret (such as ``ALPHA`` or ``pre-``), or with silly
 orderings.  [#]_
 
+Special version conventions
+^^^
+
+The following special version numbering conventions are used in the Debian
+archive:
+
+- The absence of ``debian_revision``, and therefore of a hyphen in the
+  version number, indicates that the package is native.
+
+- The presence of ``+really`` in the ``upstream_version`` component
+  indicates that a newer upstream version has been rolled back to an older
+  upstream version.  The part of the ``upstream_version`` component
+  following ``+really`` is the true upstream version.  See
+  :ref:`s-avoid-epochs` for an example of when this is used.
+
+Non-maintainer uploads:
+
+- ``debian_revision`` components ending in ``.`` (period) followed by a
+  number indicate this version of the non-native package was uploaded by
+  someone other than the maintainer (an NMU or non-maintainer upload).
+  This is used for a source package upload; for uploads 

Bug#875531: "editor +42 filename" -- accept or reject?

[Adam Borowski]

Hi!
Now that you folks are dealing with the "editor" virtual package, and,
what interests me here, the alternative for /usr/bin/editor -- could you
please process this proposal as well, and either accept or close it?

My point is that, all but one (e3) current alternatives allow positioning
the cursor on a given line, and various users take inconsistent approach
as to when this feature can be used.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀
⣾⠁⢠⠒⠀⣿⡁ According to recent spams, "all my email accounts are owned
⢿⡄⠘⠷⠚⠋⠀ by a hacker".  So what's the problem?
⠈⠳⣄

Bug#542288: debian-policy: Version numbering: native packages, NMU's, and binary only uploads

[Simon McVittie]

On Thu, 01 Apr 2021 at 18:17:59 -0700, Russ Allbery wrote:
> +- ``upstream_version`` components in native packages or
> +  ``debian_revision`` components in non-native packages ending in
> +  ``~debNuX`` also indicate a stable update, but of a different type.
> +  This version convention indicates that the stable version of the package
> +  was updated to a new upstream release, as distinct from the ``+debNuX``
> +  convention that indicates additional changes were applied to the
> +  existing stable release.  ``N`` and ``X`` mean the same as with
> +  ``+debNuX``.

I don't think this is quite it.

For me, ~debNu1 implies that a stable update was made by taking a newer
Debian package (typically from unstable) - not necessarily a new upstream
release! - and backporting it in its entirety, mechanically equivalent to
a ~bpo version but released to all users of stable rather than just those
who have opted-in to backports.

src:pango1.0 in Debian 10 is an example of this: we fixed CVE-2019-1010238
in testing/unstable by applying a patch (1.42.4-6 was vulnerable,
1.42.4-7 was fixed). We could have continued by fixing CVE-2019-1010238
in stable as 1.42.4-6+deb10u1, but the result would have been functionally
equivalent to 1.42.4-7, so it seemed clearer to just take 1.42.4-7 and
rebuild it for Debian 10 as 1.42.4-7~deb10u1. We later repeated the
process for 1.42.4-8 with some non-security fixes.

If the stable version of the package was updated to a new upstream release
instead, there are two ways that can happen. One way is to take the version
from testing/unstable and rebuild it in stable, like src:openjdk-11 in
Debian 10: 11.0.9.1+1-1 was uploaded to unstable first, then rebuilt
in Debian 10 as 11.0.9.1+1-1~deb10u1.

The other way is to repackage the new upstream release from first
principles, either because it's a new release from an upstream branch
that's older than the one in testing/unstable (like src:flatpak in Debian
10), or because testing/unstable already has packaging changes that aren't
suitable for stable (like src:dbus in Debian 10). In this case it would be
misleading to use a version number like flatpak_1.2.5-1~deb10u1 (because
1.2.5-1 never existed) or dbus_1.12.20-1~deb10u1 (because 1.12.20-1 did
exist, but the version of dbus in stable was not based on it, and does
not have the downstream changes from that version). Instead, the convention
that has emerged is to use 0+deb10u1, by analogy with the convention that
an NMU introducing a new upstream release has revision number 0.1.

~debNuX for X > 1 indicates subsequent stable-specific changes applied
on top of to a ~debNu1 version.

(Sorry, I'm not sure how to express all this in a paragraph or two...)

smcv

Bug#542288: debian-policy: Version numbering: native packages, NMU's, and binary only uploads

[Holger Levsen]

Hi Russ,

On Thu, Apr 01, 2021 at 06:17:59PM -0700, Russ Allbery wrote:
> Here is an updated diff that documents the most well-understood version
> conventions in the Debian archive.  More could certainly be added; this is
> just a first start that addresses this specific bug.

thank you for this, I've reviewed it and am generally very happy with these
changes.

> This revised patch is less aggressive about defining native packages to
> only mean packages with no existence external to Debian. 

I'm not sure if in this regard I would have liked the previous version better,
as the paragraph about native packages is the only one which I would like to
see extended to explain that it has been observed that packages we thought 
were native to Debian were not really native to Debian at all and require
modifications in Debian downstream distributions, which are harder to do
when the packages are native (eg modified orig.tar's despite no upstream 
changes).

src:piuparts is one example for this where this is even annoying within
Debian, as eg piuparts 1.1.2 was uploaded to sid and then that orig tarball
cannot be used for the upload to backports.

Also big/huge native packages are also annoying to maintain when one has
to do small packaging changes and each time has to upload huge orig.tar.
This was annoying when src:debian-edu-artwork was 50-100mb and native, today
that package is 8mb and non-native.

> I think this change is ready for seconds.

All that said, I'm still happy to second this! :)
 
> diff --git a/policy/ch-controlfields.rst b/policy/ch-controlfields.rst
> index a21a510..cd7daaa 100644
> --- a/policy/ch-controlfields.rst
> +++ b/policy/ch-controlfields.rst
> @@ -582,20 +582,17 @@ The three components here are:
>  alphanumerics and the characters ``+`` ``.`` ``~`` (plus, full stop,
>  tilde) and is compared in the same way as the ``upstream_version`` is.
>  
> -It is optional; if it isn't present then the ``upstream_version``
> -may not contain a hyphen. This format represents the case where a
> -piece of software was written specifically to be a Debian package,
> -where the Debian package source must always be identical to the
> -pristine source and therefore no revision indication is required.
> +It is conventional to restart the ``debian_revision`` at ``1`` each
> +time the ``upstream_version`` is increased.
>  
> -It is conventional to restart the ``debian_revision`` at ``1``
> -each time the ``upstream_version`` is increased.
> +The package management system will break the version number apart at
> +the last hyphen in the string (if there is one) to determine the
> +``upstream_version`` and ``debian_revision``. The absence of a
> +``debian_revision`` is equivalent to a ``debian_revision`` of ``0``.
>  
> -The package management system will break the version number apart
> -at the last hyphen in the string (if there is one) to determine
> -the ``upstream_version`` and ``debian_revision``. The absence of a
> -``debian_revision`` is equivalent to a ``debian_revision`` of
> -``0``.
> +Presence of the ``debian_revision`` part indicates this package is a
> +non-native package (see :ref:`s-source-packages`).  Absence indicates
> +the package is a native package.
>  
>  When comparing two version numbers, first the epoch of each are
>  compared, then the ``upstream_version`` if epoch is equal, and then
> @@ -646,6 +643,83 @@ numbers containing strings of letters which the package 
> management
>  system cannot interpret (such as ``ALPHA`` or ``pre-``), or with silly
>  orderings.  [#]_
>  
> +Special version conventions
> +^^^
> +
> +The following special version numbering conventions are used in the Debian
> +archive:
> +
> +- The absence of ``debian_revision``, and therefore of a hyphen in the
> +  version number, indicates that the package is native.
> +
> +- ``debian_revision`` components ending in ``.`` (period) followed by a
> +  number indicate this version of the non-native package was uploaded by
> +  someone other than the maintainer (an NMU or non-maintainer upload).
> +  This is used for a source package upload; for uploads of only binary
> +  packages without source changes, see the binary NMU convention below.
> +
> +- ``upstream_version`` components in native packages ending in ``+nmu``
> +  followed by a number indicate an NMU of a native package.  As with the
> +  convention for non-native packages, this is used for a source package
> +  upload, not for uploads of only binary packages without source changes.
> +
> +- ``upstream_version`` components in native packages or
> +  ``debian_revision`` components in non-native packages ending in ``+b``
> +  followed by a number indicate a binary NMU: an upload of a binary
> +  package without any source changes and hence without any corresponding
> +  source package upload or version change.
> +
> +- ``upstream_version``