Bug#930666: Please document consensus on use of dh sequencer

[Bill Allombert]

On Sat, Jul 06, 2019 at 08:58:26PM -0700, Russ Allbery wrote:
> Here is an updated patch incorporating Sean's wording changes as well as
> other changes discussed elsewhere on the thread.  This reverts most of my
> unrelated informative changes for clarity (they can be discussed or made
> separately) and drops the sample dh makefile in favor of just referencing
> the debhelper documentation.  It does keep some minor wording and
> formatting fixes in the first two paragraphs.
> 
> This also removes the word "new" from the discussion of using another
> packaging helper as an example, to make it a bit more obvious that the
> cdbs maintainer using cdbs is one of the expected exceptions.
> 
> diff --git a/policy/ch-source.rst b/policy/ch-source.rst
> index ee9270d..93beb4a 100644
> --- a/policy/ch-source.rst
> +++ b/policy/ch-source.rst
> @@ -259,13 +259,33 @@ files, sockets or setuid or setgid files.. [#]_
>  Main building script: ``debian/rules``
>  --
>  
> -This file must be an executable makefile, and contains the
> -package-specific recipes for compiling the package and building binary
> -package(s) from the source.
> -
> -It must start with the line ``#!/usr/bin/make -f``, so that it can be
> -invoked by saying its name rather than invoking ``make`` explicitly.
> -That is, invoking either of ``make -f debian/rules args...`` or 
> ``./debian/rules args...`` must result in identical behavior.
> +This file must be an executable makefile.  It contains the
> +package-specific recipes for compiling the source (if required) and
> +constructing one or more binary packages.
> +
> +``debian/rules`` must start with the line ``#!/usr/bin/make -f``, so that
> +it can be invoked by saying its name rather than invoking ``make``
> +explicitly.  That is, invoking either of ``make -f debian/rules args...``
> +or ``./debian/rules args...`` must result in identical behavior.
> +
> +The recommended way to implement the build process of a Debian package, in
> +the absence of a good reason to use a different approach, is the ``dh``
> +tool.  This includes the contents of the ``debian/rules`` building script.
> +``dh`` is the most common packaging helper tool in Debian.  Using it will
> +usually save effort in complying with the rules in this document, because
> +``dh`` will automatically implement many of them without requiring
> +explicit instructions.
> +
> +There are sometimes good reasons to use a different approach.  For
> +example, the standard tools for packaging software written in some
> +languages may use another tool; some rarer packaging patterns, such as
> +multiple builds of the same software with different options, are easier to
> +express with other tools; and a packager working on a different packaging
> +helper might want to use their tool.  The recommendation to use ``dh``
> +does not always apply, and use of ``dh`` is not required.
> +
> +For more information about how to use ``dh``, see the documentation in the
> +debhelper package, most notably the dh(1) manual page.
>  
>  The following targets are required and must be implemented by
>  ``debian/rules``: ``clean``, ``binary``, ``binary-arch``,
> 

Thanks Russ, this is much nicer this way.

Cheers,
-- 
Bill. 

Imagine a large red swirl here. 

Bug#930666: Please document consensus on use of dh sequencer

[Sean Whitton]

Hello,

On Sat 06 Jul 2019 at 08:58PM -07, Russ Allbery wrote:

> diff --git a/policy/ch-source.rst b/policy/ch-source.rst
> index ee9270d..93beb4a 100644
> --- a/policy/ch-source.rst
> +++ b/policy/ch-source.rst
> @@ -259,13 +259,33 @@ files, sockets or setuid or setgid files.. [#]_
>  Main building script: ``debian/rules``
>  --
>
> -This file must be an executable makefile, and contains the
> -package-specific recipes for compiling the package and building binary
> -package(s) from the source.
> -
> -It must start with the line ``#!/usr/bin/make -f``, so that it can be
> -invoked by saying its name rather than invoking ``make`` explicitly.
> -That is, invoking either of ``make -f debian/rules args...`` or 
> ``./debian/rules args...`` must result in identical behavior.
> +This file must be an executable makefile.  It contains the
> +package-specific recipes for compiling the source (if required) and
> +constructing one or more binary packages.
> +
> +``debian/rules`` must start with the line ``#!/usr/bin/make -f``, so that
> +it can be invoked by saying its name rather than invoking ``make``
> +explicitly.  That is, invoking either of ``make -f debian/rules args...``
> +or ``./debian/rules args...`` must result in identical behavior.
> +
> +The recommended way to implement the build process of a Debian package, in
> +the absence of a good reason to use a different approach, is the ``dh``
> +tool.  This includes the contents of the ``debian/rules`` building script.
> +``dh`` is the most common packaging helper tool in Debian.  Using it will
> +usually save effort in complying with the rules in this document, because
> +``dh`` will automatically implement many of them without requiring
> +explicit instructions.
> +
> +There are sometimes good reasons to use a different approach.  For
> +example, the standard tools for packaging software written in some
> +languages may use another tool; some rarer packaging patterns, such as
> +multiple builds of the same software with different options, are easier to
> +express with other tools; and a packager working on a different packaging
> +helper might want to use their tool.  The recommendation to use ``dh``
> +does not always apply, and use of ``dh`` is not required.
> +
> +For more information about how to use ``dh``, see the documentation in the
> +debhelper package, most notably the dh(1) manual page.
>
>  The following targets are required and must be implemented by
>  ``debian/rules``: ``clean``, ``binary``, ``binary-arch``,

Seconded, and applied to 'next'.  Thank you!

-- 
Sean Whitton


signature.asc
Description: PGP signature

Bug#930666: Please document consensus on use of dh sequencer

[Sam Hartman]

My second still applies to the following diff; I agree this is
consistent with the discussion so far.

diff --git a/policy/ch-source.rst b/policy/ch-source.rst
index ee9270d..93beb4a 100644
--- a/policy/ch-source.rst
+++ b/policy/ch-source.rst
@@ -259,13 +259,33 @@ files, sockets or setuid or setgid files.. [#]_
 Main building script: ``debian/rules``
 --
 
-This file must be an executable makefile, and contains the
-package-specific recipes for compiling the package and building binary
-package(s) from the source.
-
-It must start with the line ``#!/usr/bin/make -f``, so that it can be
-invoked by saying its name rather than invoking ``make`` explicitly.
-That is, invoking either of ``make -f debian/rules args...`` or
``./debian/rules args...`` must result in identical behavior.
+This file must be an executable makefile.  It contains the
+package-specific recipes for compiling the source (if required) and
+constructing one or more binary packages.
+
+``debian/rules`` must start with the line ``#!/usr/bin/make -f``, so that
+it can be invoked by saying its name rather than invoking ``make``
+explicitly.  That is, invoking either of ``make -f debian/rules args...``
+or ``./debian/rules args...`` must result in identical behavior.
+
+The recommended way to implement the build process of a Debian package, in
+the absence of a good reason to use a different approach, is the ``dh``
+tool.  This includes the contents of the ``debian/rules`` building script.
+``dh`` is the most common packaging helper tool in Debian.  Using it will
+usually save effort in complying with the rules in this document, because
+``dh`` will automatically implement many of them without requiring
+explicit instructions.
+
+There are sometimes good reasons to use a different approach.  For
+example, the standard tools for packaging software written in some
+languages may use another tool; some rarer packaging patterns, such as
+multiple builds of the same software with different options, are easier to
+express with other tools; and a packager working on a different packaging
+helper might want to use their tool.  The recommendation to use ``dh``
+does not always apply, and use of ``dh`` is not required.
+
+For more information about how to use ``dh``, see the documentation in the
+debhelper package, most notably the dh(1) manual page.
 
 The following targets are required and must be implemented by
 ``debian/rules``: ``clean``, ``binary``, ``binary-arch``,



signature.asc
Description: PGP signature

Bug#930666: Please document consensus on use of dh sequencer

[Russ Allbery]

Here is an updated patch incorporating Sean's wording changes as well as
other changes discussed elsewhere on the thread.  This reverts most of my
unrelated informative changes for clarity (they can be discussed or made
separately) and drops the sample dh makefile in favor of just referencing
the debhelper documentation.  It does keep some minor wording and
formatting fixes in the first two paragraphs.

This also removes the word "new" from the discussion of using another
packaging helper as an example, to make it a bit more obvious that the
cdbs maintainer using cdbs is one of the expected exceptions.

diff --git a/policy/ch-source.rst b/policy/ch-source.rst
index ee9270d..93beb4a 100644
--- a/policy/ch-source.rst
+++ b/policy/ch-source.rst
@@ -259,13 +259,33 @@ files, sockets or setuid or setgid files.. [#]_
 Main building script: ``debian/rules``
 --
 
-This file must be an executable makefile, and contains the
-package-specific recipes for compiling the package and building binary
-package(s) from the source.
-
-It must start with the line ``#!/usr/bin/make -f``, so that it can be
-invoked by saying its name rather than invoking ``make`` explicitly.
-That is, invoking either of ``make -f debian/rules args...`` or 
``./debian/rules args...`` must result in identical behavior.
+This file must be an executable makefile.  It contains the
+package-specific recipes for compiling the source (if required) and
+constructing one or more binary packages.
+
+``debian/rules`` must start with the line ``#!/usr/bin/make -f``, so that
+it can be invoked by saying its name rather than invoking ``make``
+explicitly.  That is, invoking either of ``make -f debian/rules args...``
+or ``./debian/rules args...`` must result in identical behavior.
+
+The recommended way to implement the build process of a Debian package, in
+the absence of a good reason to use a different approach, is the ``dh``
+tool.  This includes the contents of the ``debian/rules`` building script.
+``dh`` is the most common packaging helper tool in Debian.  Using it will
+usually save effort in complying with the rules in this document, because
+``dh`` will automatically implement many of them without requiring
+explicit instructions.
+
+There are sometimes good reasons to use a different approach.  For
+example, the standard tools for packaging software written in some
+languages may use another tool; some rarer packaging patterns, such as
+multiple builds of the same software with different options, are easier to
+express with other tools; and a packager working on a different packaging
+helper might want to use their tool.  The recommendation to use ``dh``
+does not always apply, and use of ``dh`` is not required.
+
+For more information about how to use ``dh``, see the documentation in the
+debhelper package, most notably the dh(1) manual page.
 
 The following targets are required and must be implemented by
 ``debian/rules``: ``clean``, ``binary``, ``binary-arch``,

-- 
Russ Allbery (r...@debian.org)   

Bug#930666: Please document consensus on use of dh sequencer

[andreimpopescu]

On Vi, 21 iun 19, 13:09:29, Sean Whitton wrote:
> 
> There are sometimes good reasons to use a different approach.  For
> example, the standard tools for packaging software written in some
> languages may use another tool; some rarer packaging patterns, such
> as multiple builds of the same software with different options, are
> easier to express with other tools; and a packager working on a new
 ^
> packaging helper might want to use their new tool.  
  ^

Dropping "new" would cover both the case of existing (Jonas and cdbs) 
and new packaging helpers ;)

Kind regards,
Andrei
-- 
http://wiki.debian.org/FAQsFromDebianUser


signature.asc
Description: PGP signature

Bug#930666: Please document consensus on use of dh sequencer

[Sam Hartman]

I believe that both Russ's text and  Sean's revised text capture the
project level discussions.

I also believe that given those discussions the issues are well
understood enough for us to move forward relatively quickly if new
issues are not raised here.  I believe that Russ has adequately
responded to the issue that Bill raised.

As such, at least under my understanding of the policy process, I think
I've reached the necessary conclusions to second a proposal on this bug.
So, I second Sean's revisions to Russ's proposed wording.

--Sam


signature.asc
Description: PGP signature

Bug#930666: Please document consensus on use of dh sequencer

[Sean Whitton]

Hello,

On Thu 20 Jun 2019 at 08:51am -0700, Russ Allbery wrote:

> That said, this is way too large of a problem to solve in this bug.  I
> think we need to stay focused on one section of policy here with a few
> tactical fixes so that the text still reads cleanly and not confusingly
> (which is the goal above), and then look at what we can do elsewhere to
> spread the clarity once we've agreed on what we want to say here.

I agree that this is the way to proceed here.

-- 
Sean Whitton


signature.asc
Description: PGP signature

Bug#930666: Please document consensus on use of dh sequencer

[Sean Whitton]

Hello,

On Fri 21 Jun 2019 at 01:09PM +01, Sean Whitton wrote:

> packaging helper might want to use their new tool.  The
> recommendation to use ``dh`` does not always apply, and use of
> ``dh`` is therefore not required.
>
> - saying that use of ``dh`` is "not required" is strictly redundant
>   because 'required' is equivalent to 'must', and nothing in the
>   previous paragraph says that you must use dh
>
>   - I think I avoid any chance of misreading by reiterating that the
> recommendation has exceptions

Drop the final 'therefore' from my text, because the fact that it is not
a requirement is not a consequence of the fact that the recommendation
has exceptions.

-- 
Sean Whitton


signature.asc
Description: PGP signature

Bug#930666: Please document consensus on use of dh sequencer

[Sean Whitton]

Hello Russ, Sam,

On Thu 20 Jun 2019 at 09:01AM -07, Russ Allbery wrote:

> Sean Whitton  writes:
>
>> A tiny thing is that you seem to have switched "" for "",
>> which seems wrong.
>
> I'll put this back to args... since it just muddles the discussion, and we
> can talk about that separately some other time.  But, for the record,
> "" would imply that all the arguments have to be given as a single
> argument and would be later split, as opposed to " ...", which allows
> for one or more arguments.

Aha, I see.

>> However, any SHOULD requirement in Policy implicitly has that structure:
>> "you should X unless there is reason not to X".  Perhaps MUST
>> requirements don't have that implicit structure, because perhaps the
>> idea in such cases is that there is never reason not to X.
>
> I agree with Sam that this is not the way Policy is currently written.
> This *is* what RFC 2119 SHOULD means, but this is one of the places where
> Policy diverges from RFC 2119.
> [...]

Thank you both for the correction.  In hindsight, it would have been
wise of me to review our definitions of the magic words before making an
argument based on how they are defined.

>> Would it be right to say that what we are trying to express is the idea
>> that dh should be used when there aren't *package-specific* reasons not
>> to use dh, and for new packages, we're recommending a workflow of
>> starting with the assumption that no such package-specific reasons
>> exist?
>
> I don't think so, since this doesn't account for a maintainer who is
> writing a new packaging helper, which is not a package-specific reason
> (it's a maintainer-specific reason).  I also don't think the project
> consensus was to tell Jonas (the cdbs maintainer) to use dh for all new
> packages.

Good point.

> I understand the concern about making the language too weak, but I also
> don't think there's a need to be too firm here.  The goal, at least as I
> see it, is to push people who don't have strong opinions towards dh, not
> to try to convince the people who do have strong opinions.  (I think that
> convincing is worthwhile but I think it will happen organically and Policy
> isn't really the place.)

My concern is not about making the language too normatively *weak*.  I
agree with everything just quoted.  Rather, my concern is about the
language being normatively *imprecise*.  I wrote:

| I would like to try to make the first sentence normatively clearer.  I
| think that it would be difficult to understand exactly what the project
| consensus is from that sentence, had you not read Sam's d-d-a post.

and I still think that.  Let me say a bit more.  We want to push people
without strong opinions on the matter towards using dh.  We could do
that by writing something in Policy which does not use any of the magic
words (like some other tool recommendations in Policy), and in such a
case I would not be raising this concern.  However, we have decided that
we want to use the should/recommended words.  And I think that when we
use the magic words we should take special care to make it clear to the
reader what the normative content of the sentence is.

Looking at your patch again, I think that my concern could be allayed by
explicitly tying the "There are various situations ..." paragraph to the
"reason to use a different approach" clause.  This is the sort of thing
that I mean (I've also wordsmithed):

The recommended way to implement the build process of a Debian
package, in the absence of a good reason to use a different
approach, is the ``dh`` tool.  This includes the contents of the
``debian/rules`` building script.  ``dh`` is the most common
packaging helper tool in Debian.  Using it will usually save effort
in complying with the rules in this document, because ``dh`` will
automatically implement many of them without requiring explicit
instructions.

There are sometimes good reasons to use a different approach.  For
example, the standard tools for packaging software written in some
languages may use another tool; some rarer packaging patterns, such
as multiple builds of the same software with different options, are
easier to express with other tools; and a packager working on a new
packaging helper might want to use their new tool.  The
recommendation to use ``dh`` does not always apply, and use of
``dh`` is therefore not required.

Other notes on this:

- I changed "a reason"->"a good reason" because this is in fact what we
  mean

- saying that use of ``dh`` is "not required" is strictly redundant
  because 'required' is equivalent to 'must', and nothing in the
  previous paragraph says that you must use dh

  - I think I avoid any chance of misreading by reiterating that the
recommendation has exceptions

-- 
Sean Whitton


signature.asc
Description: PGP signature

Bug#930666: Please document consensus on use of dh sequencer

[Russ Allbery]

Sean Whitton  writes:

>> --- a/policy/ch-source.rst
>> +++ b/policy/ch-source.rst
>> @@ -259,18 +259,49 @@ files, sockets or setuid or setgid files.. [#]_
>>  Main building script: ``debian/rules``
>>  --
>>
>> -This file must be an executable makefile, and contains the
>> -package-specific recipes for compiling the package and building binary
>> -package(s) from the source.
>> -
>> -It must start with the line ``#!/usr/bin/make -f``, so that it can be
>> -invoked by saying its name rather than invoking ``make`` explicitly.
>> -That is, invoking either of ``make -f debian/rules args...`` or 
>> ``./debian/rules args...`` must result in identical behavior.
>> +This file must be an executable makefile.  It contains the
>> +package-specific recipes for compiling the source (if required) and
>> +constructing one or more binary packages.
>> +
>> +``debian/rules`` must start with the line ``#!/usr/bin/make -f`` so that
>> +it can be executed by running ``./debian/rules`` from the top of the
>> +source package.  Invoking either of ``make -f debian/rules  ...`` or
>> +``./debian/rules  ...`` must result in identical behavior.

> I must admit to being sad to see the demise of "invoked by saying its
> name", but this is probably a sensible change.

I probably shouldn't have muddled this patch with the wording
clarification, but I found the previous text a bit confusing.  It's a nice
turn of phrase, but I've never seen anyone elsewhere refer to running a
script via a relative path as "saying its name."

> A tiny thing is that you seem to have switched "" for "",
> which seems wrong.

I'll put this back to args... since it just muddles the discussion, and we
can talk about that separately some other time.  But, for the record,
"" would imply that all the arguments have to be given as a single
argument and would be later split, as opposed to " ...", which allows
for one or more arguments.

> However, any SHOULD requirement in Policy implicitly has that structure:
> "you should X unless there is reason not to X".  Perhaps MUST
> requirements don't have that implicit structure, because perhaps the
> idea in such cases is that there is never reason not to X.

I agree with Sam that this is not the way Policy is currently written.
This *is* what RFC 2119 SHOULD means, but this is one of the places where
Policy diverges from RFC 2119.  We say:

| Non-conformance with guidelines denoted by *should* (or *recommended*)
| will generally be considered a bug, but will not necessarily render a
| package unsuitable for distribution.

which isn't as clear as it could be, but doesn't leave clear room for
maintainer discretion or other reasons, just a lot of room for severity.
Compare to RFC 2119:

| This word, or the adjective "RECOMMENDED", mean that there may exist
| valid reasons in particular circumstances to ignore a particular item,
| but the full implications must be understood and carefully weighed
| before choosing a different course.

which is much weaker than Policy should.

> Would it be right to say that what we are trying to express is the idea
> that dh should be used when there aren't *package-specific* reasons not
> to use dh, and for new packages, we're recommending a workflow of
> starting with the assumption that no such package-specific reasons
> exist?

I don't think so, since this doesn't account for a maintainer who is
writing a new packaging helper, which is not a package-specific reason
(it's a maintainer-specific reason).  I also don't think the project
consensus was to tell Jonas (the cdbs maintainer) to use dh for all new
packages.

I understand the concern about making the language too weak, but I also
don't think there's a need to be too firm here.  The goal, at least as I
see it, is to push people who don't have strong opinions towards dh, not
to try to convince the people who do have strong opinions.  (I think that
convincing is worthwhile but I think it will happen organically and Policy
isn't really the place.)

> I find the expression "default content" odd in this context, I think
> because defaults are things that executable programs have, not source
> code.

> Can I suggest we talk instead about the "starting point for the content
> of d/rules" and say something like "this starting point is often
> sufficient to build a package satisfying most of the requirements below
> (and many others), and where it is not, package-specific instructions
> are usually expressed using override targets" ?

That sounds great to me.  I'll make that change in my draft.

-- 
Russ Allbery (r...@debian.org)   

Bug#930666: Please document consensus on use of dh sequencer

[Russ Allbery]

Bill Allombert  writes:
> On Tue, Jun 18, 2019 at 06:24:45PM -0700, Russ Allbery wrote:

>>  The following targets are required and must be implemented by
>>  ``debian/rules``: ``clean``, ``binary``, ``binary-arch``,
>> -``binary-indep``, ``build``, ``build-arch`` and ``build-indep``. These
>> -are the targets called by ``dpkg-buildpackage``.
>> +``binary-indep``, ``build``, ``build-arch`` and ``build-indep``.  (When
>> +using ``dh``, these are implemented using the wildcard pattern shown
>> +above.)  These are the targets called by ``dpkg-buildpackage``.

> Hello Russ,

> I find this layering violation rather confusing especially if dh should
> be seen as the rule and not the exception.

> There are maybe an hundred places where we could mention dh,
> with statement like "when using dh, do something else".

I think there are fewer of them than you might think (I had that
impression too going in, but found that there were a lot fewer than I had
expected), but I also think that making those changes going forward will
be valuable.  We've already started doing that with debhelper, largely in
footnotes, and having a more systematic way of structuring that would be
useful.  I think this may be usefully expressed in metadata: some way to
flag a section of Policy with "debhelper does this for you" (since in most
cases it doesn't matter if you're using debhelper directly or via dh).

That said, this is way too large of a problem to solve in this bug.  I
think we need to stay focused on one section of policy here with a few
tactical fixes so that the text still reads cleanly and not confusingly
(which is the goal above), and then look at what we can do elsewhere to
spread the clarity once we've agreed on what we want to say here.

> Maybe what we need is a separate 'dh packaging manual' where packaging
> is explained from dh point of view.

That already exists, I think?  That's the debhelper documentation.

Another option would be to leave out the minimal debian/rules example
entirely and just refer entirely to the dh manual.  Maybe that's better.

-- 
Russ Allbery (r...@debian.org)   

Bug#930666: Please document consensus on use of dh sequencer

[Bill Allombert]

On Tue, Jun 18, 2019 at 06:24:45PM -0700, Russ Allbery wrote:
>  The following targets are required and must be implemented by
>  ``debian/rules``: ``clean``, ``binary``, ``binary-arch``,
> -``binary-indep``, ``build``, ``build-arch`` and ``build-indep``. These
> -are the targets called by ``dpkg-buildpackage``.
> +``binary-indep``, ``build``, ``build-arch`` and ``build-indep``.  (When
> +using ``dh``, these are implemented using the wildcard pattern shown
> +above.)  These are the targets called by ``dpkg-buildpackage``.

Hello Russ,

I find this layering violation rather confusing especially if dh should
be seen as the rule and not the exception.

There are maybe an hundred places where we could mention dh,
with statement like "when using dh, do something else".

Maybe what we need is a separate 'dh packaging manual' where packaging
is explained from dh point of view.

Cheers,
-- 
Bill. 

Imagine a large red swirl here. 

Bug#930666: Please document consensus on use of dh sequencer

[Sam Hartman]

> "Sean" == Sean Whitton  writes:

Sean> Let me try to express what I think the problem is.  What the
Sean> first sentence says, given the equivalence of RECOMMENDED and
Sean> SHOULD noted above, is "you should use dh unless there is a
Sean> reason not to use dh".

Sean> However, any SHOULD requirement in Policy implicitly has that
Sean> structure: "you should X unless there is reason not to X".


That's not how I read policy at all.
>In the normative part of this manual, the words *must*, *should* and
>*may*, and the adjectives *required*, *recommended* and *optional*,
>are used to distinguish the significance of the various guidelines in
>this policy document. Packages that do not conform to the guidelines
>denoted by *must* (or *required*) will generally not be considered
>acceptable for the Debian distribution. Non-conformance with
>guidelines denoted by *should* (or *recommended*) will generally be
>considered a bug, but will not necessarily render a package unsuitable


No where in the above text do I see that if there is a good reason not
to do x, that x is not a non-RC bug in your package.
This is the key difference between how policy uses those terms and RFC
2119.

So, I think Russ is correct that what we're saying is that you should do
x unless there is an adequate reason not to.
If recommended in policy language actually meant that, we could say dh
is recommended, spend a bit of text giving examples of reasons why it
doesn't apply and be done.

I'd be in favor of changing policy so that should meant should x unless
there is a good reason not to do x, but that is a much much bigger
change.

Bug#930666: Please document consensus on use of dh sequencer

[Sean Whitton]

Hello Russ,

On Tue 18 Jun 2019 at 06:24PM -07, Russ Allbery wrote:

> For those reading this, note that "recommended" is a documented keyword in
> Policy, equivalent to "should."  However, it is intentionally weakened
> here with the "in the absence of a reason to use a different approach"
> language.  The goal here (and please judge whether I achieved it) is to
> concentrate on providing default advice to people with no preference,
> particularly people new to packaging, not to argue with maintainers who
> are familiar with the tradeoffs and are intentionally making a different
> choice.

This seems like the right goal but I'm not sure your patch quite
achieves it yet.

> diff --git a/policy/ch-source.rst b/policy/ch-source.rst
> index ee9270d..9ea2f5c 100644
> --- a/policy/ch-source.rst
> +++ b/policy/ch-source.rst
> @@ -259,18 +259,49 @@ files, sockets or setuid or setgid files.. [#]_
>  Main building script: ``debian/rules``
>  --
>
> -This file must be an executable makefile, and contains the
> -package-specific recipes for compiling the package and building binary
> -package(s) from the source.
> -
> -It must start with the line ``#!/usr/bin/make -f``, so that it can be
> -invoked by saying its name rather than invoking ``make`` explicitly.
> -That is, invoking either of ``make -f debian/rules args...`` or 
> ``./debian/rules args...`` must result in identical behavior.
> +This file must be an executable makefile.  It contains the
> +package-specific recipes for compiling the source (if required) and
> +constructing one or more binary packages.
> +
> +``debian/rules`` must start with the line ``#!/usr/bin/make -f`` so that
> +it can be executed by running ``./debian/rules`` from the top of the
> +source package.  Invoking either of ``make -f debian/rules  ...`` or
> +``./debian/rules  ...`` must result in identical behavior.

I must admit to being sad to see the demise of "invoked by saying its
name", but this is probably a sensible change.

A tiny thing is that you seem to have switched "" for "",
which seems wrong.

> +
> +In the absence of a reason to use a different approach, the recommended
> +way to implement the build process of a Debian package, including the
> +contents of the ``debian/rules`` building script, is the ``dh`` tool,
> +which is provided by the debhelper package.  This is the most common
> +packaging helper tool in Debian.  Using it will save effort in complying
> +with the rules in this document, since it will automatically implement
> +many of them without requiring explicit instructions.

I would like to try to make the first sentence normatively clearer.  I
think that it would be difficult to understand exactly what the project
consensus is from that sentence, had you not read Sam's d-d-a post.

Let me try to express what I think the problem is.  What the first
sentence says, given the equivalence of RECOMMENDED and SHOULD noted
above, is "you should use dh unless there is a reason not to use dh".

However, any SHOULD requirement in Policy implicitly has that structure:
"you should X unless there is reason not to X".  Perhaps MUST
requirements don't have that implicit structure, because perhaps the
idea in such cases is that there is never reason not to X.

Thus, your sentence seems to just be making explicit something which is
already implicit in any SHOULD requirement, i.e., "In the absence ..."
is strictly redundant.  And so I don't think the sentence succeeds in
expressing the nuanced project consensus which it aims to express,
because the consensus is more nuanced than just the implicit structure
had by any SHOULD requirement.

Would it be right to say that what we are trying to express is the idea
that dh should be used when there aren't *package-specific* reasons not
to use dh, and for new packages, we're recommending a workflow of
starting with the assumption that no such package-specific reasons
exist?  This captures the idea that dh makes things easier for other
contributors working on your package, which seems core to the project's
consensus.

> +
> +There are various situations in which ``dh`` is not the best choice.  For
> +example, the standard tools for packaging software written in some
> +languages may use another tool, some rarer packaging patterns (such as
> +multiple builds of the same software with different options) are easier to
> +express with other tools, or the packager may be working on a new
> +packaging helper and may therefore want to use that tool.  Using ``dh`` is
> +therefore not required.  But it is a wise default choice in most
> +situations.
> +
> +If the package uses ``dh``, the default content of ``debian/rules`` will
> +be::

I find the expression "default content" odd in this context, I think
because defaults are things that executable programs have, not source
code.

Can I suggest we talk instead about the "starting point for the content
of d/rules" and say something like "this starting point is 

Bug#930666: Please document consensus on use of dh sequencer

[Russ Allbery]

Sam Hartman  writes:

> I just published a consensus call on a discussion we had to canvas the
> project on the use of Debhelper's dh sequencer.
> https://lists.debian.org/msgid-search/tslmuif7pwy@suchdamage.org

> I'd like to ask the policy editors to facilitate using the normal
> process to document this consensus in policy.

> My understanding is that the editors already have some ideas about how
> that might work.

To kick off this discussion with some concrete wording that we can
discuss, here's a possible approach to documenting this consensus.  This
attempts to reflect the nuance of the consensus, including the
understanding that dh is not mandatory and there are various cases where
it is not the best choice.

For those reading this, note that "recommended" is a documented keyword in
Policy, equivalent to "should."  However, it is intentionally weakened
here with the "in the absence of a reason to use a different approach"
language.  The goal here (and please judge whether I achieved it) is to
concentrate on providing default advice to people with no preference,
particularly people new to packaging, not to argue with maintainers who
are familiar with the tradeoffs and are intentionally making a different
choice.

diff --git a/policy/ch-source.rst b/policy/ch-source.rst
index ee9270d..9ea2f5c 100644
--- a/policy/ch-source.rst
+++ b/policy/ch-source.rst
@@ -259,18 +259,49 @@ files, sockets or setuid or setgid files.. [#]_
 Main building script: ``debian/rules``
 --
 
-This file must be an executable makefile, and contains the
-package-specific recipes for compiling the package and building binary
-package(s) from the source.
-
-It must start with the line ``#!/usr/bin/make -f``, so that it can be
-invoked by saying its name rather than invoking ``make`` explicitly.
-That is, invoking either of ``make -f debian/rules args...`` or 
``./debian/rules args...`` must result in identical behavior.
+This file must be an executable makefile.  It contains the
+package-specific recipes for compiling the source (if required) and
+constructing one or more binary packages.
+
+``debian/rules`` must start with the line ``#!/usr/bin/make -f`` so that
+it can be executed by running ``./debian/rules`` from the top of the
+source package.  Invoking either of ``make -f debian/rules  ...`` or
+``./debian/rules  ...`` must result in identical behavior.
+
+In the absence of a reason to use a different approach, the recommended
+way to implement the build process of a Debian package, including the
+contents of the ``debian/rules`` building script, is the ``dh`` tool,
+which is provided by the debhelper package.  This is the most common
+packaging helper tool in Debian.  Using it will save effort in complying
+with the rules in this document, since it will automatically implement
+many of them without requiring explicit instructions.
+
+There are various situations in which ``dh`` is not the best choice.  For
+example, the standard tools for packaging software written in some
+languages may use another tool, some rarer packaging patterns (such as
+multiple builds of the same software with different options) are easier to
+express with other tools, or the packager may be working on a new
+packaging helper and may therefore want to use that tool.  Using ``dh`` is
+therefore not required.  But it is a wise default choice in most
+situations.
+
+If the package uses ``dh``, the default content of ``debian/rules`` will
+be::
+
+#!/usr/bin/make -f
+%:
+dh $@
+
+(the start of the ``dh`` line must be a literal tab), and any special
+rules for this package will be expressed as overrides.  ``dh`` will then
+take care most of the requirements below (and many others).  See the
+documentation in the debhelper package for more information.
 
 The following targets are required and must be implemented by
 ``debian/rules``: ``clean``, ``binary``, ``binary-arch``,
-``binary-indep``, ``build``, ``build-arch`` and ``build-indep``. These
-are the targets called by ``dpkg-buildpackage``.
+``binary-indep``, ``build``, ``build-arch`` and ``build-indep``.  (When
+using ``dh``, these are implemented using the wildcard pattern shown
+above.)  These are the targets called by ``dpkg-buildpackage``.
 
 Since an interactive ``debian/rules`` script makes it impossible to
 auto-compile that package and also makes it hard for other people to

-- 
Russ Allbery (r...@debian.org)   

Bug#930666: Please document consensus on use of dh sequencer

[Sam Hartman]

package: debian-policy

Dear policy team:

I just published a consensus call on a discussion we had to canvas the
project on the use of Debhelper's dh sequencer.
https://lists.debian.org/msgid-search/tslmuif7pwy@suchdamage.org

I'd like to ask the policy editors to facilitate using the normal
process to document this consensus in policy.

My understanding is that the editors already have some ideas about how
that might work.

Obviously I'm available to help as desired.

In the interest of setting expectations, if this issue is still open in
December, I plan to check in and see whether I think the approach of going
through the normal policy process still makes sense.



signature.asc
Description: PGP signature