Bug#542288: debian-policy: Version numbering: native packages, NMU's, and binary only uploads

[Bill Allombert]

On Sun, Jun 25, 2017 at 02:08:05PM -0700, Russ Allbery wrote:
> It's been a while since the last update to this thread and proposed
> wording about the special version numbering conventions in use in Debian,
> and in the meantime things have settled out a bit more and we have a
> pretty firm consensus on how to handle special versions.  I'd therefore
> like to resurrect this thread and see if we can agree on some wording.
> 
> The patch below adds a definition of native packages to our definitions
> section and documents the following version number conventions:
> 
> - Native packages
> - NMUs of native and non-native packages
> - binNMUs
> - Stable updates
> - Backports
> 
> I think these are all fairly consistent and widely agreed-on at this
> point.
> 
> Concerns, objections, seconds?

Maybe we should check whether this is consistent with the devref, since
it covers the same ground.

Cheers,
-- 
Bill. 

Imagine a large red swirl here. 

Bug#758234: debian-policy: allow packages to depend on packages of lower priority

[Russ Allbery]

Andreas Henriksson  writes:

> Can't help but wonder why not just remove the "extra" (and mentioning it
> as deprecated in upgrade notes) rather than explicitly documenting it as
> deprecated. I guess keeping it around is useful to avoid people
> mass-bug-filing RC-bugs for all current extra packages.

I'd like to keep the documentation of extra because, for some time to
come, there will be a lot of packages in the wild that will have that
priority in their control files (even if they've been overridden by
ftp-master in the archive metadata).  We therefore need to document that
the priority still exists.

Hm, it occurs to me that this wording should probably explicitly say that
the extra priority should be treated the same as optional if it appears
anywhere (although the archive-wide override change will mostly take care
of that).

-- 
Russ Allbery (r...@debian.org)   

Bug#758234: debian-policy: allow packages to depend on packages of lower priority

[Joerg Jaspert]

On 14717 March 1977, Andreas Henriksson wrote:
> Can't help but wonder why not just remove the "extra" (and mentioning it
> as deprecated in upgrade notes) rather than explicitly documenting it as
> deprecated. I guess keeping it around is useful to avoid people
> mass-bug-filing RC-bugs for all current extra packages.

As soon as this is more than just a discussion but gets reality, we will
ensure there *IS* no more extra package with a mass override change...


-- 
bye, Joerg

Re: Proposal: move forward to Sphinx

[Hideki Yamane]

On Sat, 24 Jun 2017 15:38:33 -0700
Russ Allbery  wrote:
> 1. We've always had reasonably good PDF output, and I want to maintain
>that.  I'm not sure how good Sphinx's pipeline is for that, and as
>noted in the slides that would need to be tested.  (ePub might be
>better than PDF at this point, though.)

 Let's check it.
 See http://www.ma-aya.to/~henrich/debian/policy/DebianPolicyManual.pdf


> 2. I don't think the Policy Editors probably have time to help a ton with
>the conversion, so it would need to be driven by someone else with time
>to rebase it repeatedly (probably by writing conversion scripts),
>similar to what was done with the DocBook conversion.  We'd also want
>all documents in Policy to be moved to restructured text (except the
>pure text ones) so that we are on a single format as a result of this
>change.

 I'll care it, of course :) But I'm not sure all conversions can be done
 at once, it takes time and some effort.


> 3. Historically, text-like formats have struggled with multiply-nested
>lists with multi-paragraph blocks and embedded literal blocks.  The
>output from Sphinx looks surprisingly good, so maybe its version of
>reStructured Text does address all of this.  But we'd want to verify
>that we can get reasonable results from fairly complex structure.
>(There seems to be something weird going on with fonts in definition
>lists in your sample output, for instance.)

 You mean 
http://www.ma-aya.to/~henrich/debian/policy/ch-controlfields.html#list-of-fields
 ?
 Some weird looking can be adjust with modifying Sphinx theme, I guess. 


> 4. We'd like to be able to divide documents into multiple source files for
>ease of editing (I kind of want to do that with Policy, and we're doing
>that now for the debconf specification).  I'm not sure if Sphinx
>supports this?

 I had splitted policydoc to restructuredtext on each chapter.
 https://github.com/henrich/policydoc/tree/restructure/source

 What does "multiple source" mean (Including other file)?
 If so, "include" directive would help. I'll test it later.
 http://docutils.sourceforge.net/docs/ref/rst/directives.html#include


> 5. I'd really like to move most of the comments into sidebars or some
>other form of embedded set-off text that we can clearly mark as
>non-normative, since reading footnotes is a kind of obnoxious
>experience in all of our output formats.  DocBook has native support
>for this in multiple ways (, , etc.), and I was looking
>forward to having that available (although the default formatting is
>perhaps a bit too aggressive and we'd need to find a more subtle way of
>rendering it).  Would we lose that in reStructured Text?

 Sphinx supports some directives like "tip", "warning", "note", etc.
 See http://docutils.sourceforge.net/docs/ref/rst/directives.html#tip

 And if you have good example for what you want, please share its URI.


> That said, the ease of editing improvement is *substantial*, and might be
> worth some regressions on the other points.

 :)


> Probably the biggest competitor would be to move to Publican, which also
> has similar translation support (at least so I am told) and better
> rendering, but continues to use DocBook as the underlying source format.
> That said, Publican and the DocBook toolchain in general are fairly
> complicated, so if Sphinx is easier to use, that would be another nice
> advantage.

 Yes, once I thought about Publican, it is also nice but little bit
 tough to use, perhaps we need some people to maintain it with lots
 effort. Sphinx is not so complicated and if other toolchain that can
 deal with restructuredtext is better than Sphinx, we can move to it.


-- 
Regards,

 Hideki Yamane henrich @ debian.or.jp/org
 http://wiki.debian.org/HidekiYamane

Bug#758234: debian-policy: allow packages to depend on packages of lower priority

[gregor herrmann]

On Wed, 28 Jun 2017 13:13:31 +0200, Andreas Henriksson wrote:

> I also think a really good upgrade notice for this is needed. Should be
> very clearly mentioned people adjusting their packages should not forget
> to also file a bug against ftp.debian.org to align the overrides. 

My guess is that the ftp-masters would run an SQL statement like
UPDATE packages SET priority='optional' WHERE priority='extra';
in the database behind dak just once (or something similar with the
same effect). Filing individual bugs and fixing them separately on
the archive side doesn't sound very efficient to me.


Cheers,
gregor

-- 
 .''`.  https://info.comodo.priv.at/ - Debian Developer https://www.debian.org
 : :' : OpenPGP fingerprint D1E1 316E 93A7 60A8 104D  85FA BB3A 6801 8649 AA06
 `. `'  Member of VIBE!AT & SPI, fellow of the Free Software Foundation Europe
   `-   BOFH excuse #264:  Your modem doesn't speak English. 

Bug#758234: debian-policy: allow packages to depend on packages of lower priority

[Andreas Henriksson]

Hi Russ,

Your suggestion looks good to me so just in case it's useful:

Seconded.


Can't help but wonder why not just remove the "extra" (and mentioning it
as deprecated in upgrade notes) rather than explicitly documenting it as
deprecated. I guess keeping it around is useful to avoid people
mass-bug-filing RC-bugs for all current extra packages.

I also think a really good upgrade notice for this is needed. Should be
very clearly mentioned people adjusting their packages should not forget
to also file a bug against ftp.debian.org to align the overrides. I
think the suggested new language in the policy itself is clear, but I'm
still sure many people will overlook that the Priority field in the
control file is not the canonical location for the actual priority used
by the debian archive. I think you've done a very good job for upgrade
notes keeping them short and to the point previously so I trust you'll
do an excellent job here as well, but if you want help with suggestions
please mention it and a non-native speaker like myself would make an
attempt.


Regards,
Andreas Henriksson


On Sun, Jun 25, 2017 at 03:54:00PM -0700, Russ Allbery wrote:
[...]
> diff --git a/policy.xml b/policy.xml
> index ace6a3b..be458cd 100644
> --- a/policy.xml
> +++ b/policy.xml
> @@ -837,11 +837,33 @@
>Priorities
>  
>
> -Each package should have a priority value,
> -which is included in the package's control
> -record (see ).  This
> -information is used by the Debian package management tools to
> -separate high-priority packages from less-important packages.
> +Each package must have a priority value,
> +which is set in the metadata for the Debian archive and is also
> +included in the package's control files (see  +linkend="s-f-Priority"/>).  This information is used to control
> +which packages are included in standard or minimal Debian
> +installations.
> +  
> +  
> +Most Debian packages will have a priority of
> +optional.  Priority levels other than
> +optional are only used for packages that should
> +be included by default in a standard installation of Debian.
> +  
> +  
> +The priority of a package is determined solely by the
> +functionality it provides directly to the user.  The priority of a
> +package should not be increased merely because another
> +higher-priority package depends on it; instead, the tools used to
> +construct Debian installations will correctly handle package
> +dependencies.  In particular, this means that C-like libraries
> +will almost never have a priority above
> +optional, since they do not provide
> +functionality directly to users.  However, as an exception, the
> +maintainers of Debian installers may request an increase of the
> +priority of a package to resolve installation issues and ensure
> +that the correct set of packages is included in a standard or
> +minimal install.
>
>
>  The following priority levels are recognized
> @@ -896,19 +922,22 @@
>installed by default if the user doesn't select anything
>else.  It doesn't include many large applications.
>  
> +
> +  No two packages that both have a priority of
> +  standard or higher may conflict with each
> +  other.
> +
>
>  
>  
>optional
>
>  
> -  (In a sense everything that isn't required is optional, but
> -  that's not what is meant here.) This is all the software
> -  that you might reasonably want to install if you didn't know
> -  what it was and don't have specialized requirements.  This
> -  is a much larger system and includes the X Window System, a
> -  full TeX distribution, and many applications.  Note that
> -  optional packages should not conflict with each other.
> +  This is the default priority for the majority of the
> +  archive.  Unless a package should be installed by default on
> +  standard Debian systems, it should have a priority of
> +  optional.  Packages with a priority of
> +  optional may conflict with each other.
>  
>
>  
> @@ -916,22 +945,21 @@
>extra
>
>  
> -  This contains all packages that conflict with others with
> -  required, important, standard or optional priorities, or are
> -  only likely to be useful if you already know what they are
> -  or have specialized requirements (such as packages
> -  containing only detached debugging symbols).
> +  This priority is 

Bug#758234: debian-policy: allow packages to depend on packages of lower priority

[Niels Thykier]

On Sun, 25 Jun 2017 15:54:00 -0700 Russ Allbery  wrote:
> Ansgar Burchardt  writes:
> 
> > I discussed this a bit on IRC with the other ftp-masters and we came to
> > this summary:
> 
> > 0) We would like to drop the requirement for packages to not depend on
> >packages of lower priority: it is better to declare only what we
> >actually want included in the installation (that is at priority >=
> >standard) rather than also the dependency closure.
> 
> > 1) We agree that the 'extra' priority can be dropped.
> 
> > 2) We wonder if the 'standard' priority can also be dropped: as far as
> >we know, it is used only by the "standard" task and it might make
> >sense to treat it the same as other tasks.
> >(Depending on what works better for the installer team.)
> 

Hi,

Thanks for this draft.

> Given KiBi's reply, I'll leave 2 out for now.
> 

Seems reasonable to me. :)

> [...]
> 
> Note that this also says that no two packages that both have a priority of
> standard or higher may conflict.  I think that's a logical consequence of
> the use of priorities, and didn't want to lose that completely when that
> requirement was dropped from optional.
> 

Agreed.

I second the change below.  Please notify me/the lintian maintainers
if/when it is merged, so we can update the relevant checks in lintian.

> diff --git a/policy.xml b/policy.xml
> index ace6a3b..be458cd 100644
> --- a/policy.xml
> +++ b/policy.xml
> @@ -837,11 +837,33 @@
>Priorities
>  
>
> -Each package should have a priority value,
> -which is included in the package's control
> -record (see ).  This
> -information is used by the Debian package management tools to
> -separate high-priority packages from less-important packages.
> +Each package must have a priority value,
> +which is set in the metadata for the Debian archive and is also
> +included in the package's control files (see  +linkend="s-f-Priority"/>).  This information is used to control
> +which packages are included in standard or minimal Debian
> +installations.
> +  
> +  
> +Most Debian packages will have a priority of
> +optional.  Priority levels other than
> +optional are only used for packages that should
> +be included by default in a standard installation of Debian.
> +  
> +  
> +The priority of a package is determined solely by the
> +functionality it provides directly to the user.  The priority of a
> +package should not be increased merely because another
> +higher-priority package depends on it; instead, the tools used to
> +construct Debian installations will correctly handle package
> +dependencies.  In particular, this means that C-like libraries
> +will almost never have a priority above
> +optional, since they do not provide
> +functionality directly to users.  However, as an exception, the
> +maintainers of Debian installers may request an increase of the
> +priority of a package to resolve installation issues and ensure
> +that the correct set of packages is included in a standard or
> +minimal install.
>
>
>  The following priority levels are recognized
> @@ -896,19 +922,22 @@
>installed by default if the user doesn't select anything
>else.  It doesn't include many large applications.
>  
> +
> +  No two packages that both have a priority of
> +  standard or higher may conflict with each
> +  other.
> +
>
>  
>  
>optional
>
>  
> -  (In a sense everything that isn't required is optional, but
> -  that's not what is meant here.) This is all the software
> -  that you might reasonably want to install if you didn't know
> -  what it was and don't have specialized requirements.  This
> -  is a much larger system and includes the X Window System, a
> -  full TeX distribution, and many applications.  Note that
> -  optional packages should not conflict with each other.
> +  This is the default priority for the majority of the
> +  archive.  Unless a package should be installed by default on
> +  standard Debian systems, it should have a priority of
> +  optional.  Packages with a priority of
> +  optional may conflict with each other.
>  
>
>  
> @@ -916,22 +945,21 @@
>extra
>
>  
> -  This contains all packages that conflict with others with
> -  required, important, standard or optional priorities, or are
> -  only likely to be 

Bug#865713: Please Start UTF-8 debian-policy Text Files with UTF-8 Signature

[Paul Wise]

On Tue, 2017-06-27 at 19:49 -0700, Paul Hardy wrote:

> 1) Serving debian-policy pages on Debian servers as UTF-8 documents,
> as an interim measure.

I think we would want to do this for all *.txt documents that are UTF-8 
and available on the website. First we need the list of UTF-8 encoded
text files and then we need an appropriate Apache configuration snippet
to be added to the template.

https://anonscm.debian.org/cgit/mirror/dsa-puppet.git/tree/modules/roles/templates/apache-www.debian.org.erb

I've run these commands on one of the static mirrors, results attached.

find . -iname *.txt -print0 | xargs -0 isutf8 --list --invert | sort > 
~/isutf8.txt
find . -iname *.txt -print0 | xargs -0 isutf8 --list | sort > ~/isnotutf8.txt
find . -iname *.txt -print0 | xargs -0 file --mime-encoding | sort > 
~/mime-encoding.txt

Could someone come up with an appropriate Apache configuration snippet?

-- 
bye,
pabs

https://wiki.debian.org/PaulWise


isnotutf8.txt.gz
Description: application/gzip


isutf8.txt.gz
Description: application/gzip


mime-encoding.txt.gz
Description: application/gzip


signature.asc
Description: This is a digitally signed message part