Re: Bug#758234: Raising priority of Debian packages

[Ralf Treinen]

Hello,

On Sun, Sep 07, 2014 at 12:51:53PM +0200, Jakub Wilk wrote:
> * Paul Wise , 2014-09-07, 17:38:
> >>We should probably also monitor package conflicts. We made a big fuss
> >>about node vs nodejs (and rightly so); but I bet that we have lots of
> >>other package pairs in the archive that can't be co-installed for no
> >>good reason.
> >
> >We have this already:
> >
> >https://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=edos-file-overwrite;users=trei...@debian.org
> >https://qa.debian.org/dose/file-overwrites.html
> 
> Nah, I meant packages that do declare that they can't be co-installed.

Jakub is right, the bugs that I am tracking on [1] concern only cases where
there is no conflict declared in the metadata, yet the two packages fail
to install together.

Having said that, it might be possible to use dose-debcheck to do the check
you have in mind. Maybe you can spare me reading the 139 messages in the bug
report and tell me what precisely you need: Is it that every single package
can be installed using only packages of the same or higher priority ?

-Ralf.

[1] https://qa.debian.org/dose/file-overwrites.html


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20140907154514.gb1...@seneca.home.org

Re: Bug#758234: Raising priority of Debian packages

[Jakub Wilk]

* Paul Wise , 2014-09-07, 17:38:
We should probably also monitor package conflicts. We made a big fuss 
about node vs nodejs (and rightly so); but I bet that we have lots of 
other package pairs in the archive that can't be co-installed for no 
good reason.


We have this already:

https://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=edos-file-overwrite;users=trei...@debian.org
https://qa.debian.org/dose/file-overwrites.html


Nah, I meant packages that do declare that they can't be co-installed.

--
Jakub Wilk


--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20140907105153.ga5...@jwilk.net

Re: Detecting more undeclared conflicts (was: Re: Bug#758234: Raising priority of Debian packages)

[Paul Wise]

On Sun, Sep 7, 2014 at 6:36 PM, Johannes Schauer wrote:

> would it make sense to extend this test and not only check whether packages
> that share a file listed in Contents.gz can be co-installed but also packages
> which access/change/create the same files in their pre/post-install maintainer
> scripts do?

Could you provide an example of the kind of bug this test would detect?

[I'm subscribed, no need to CC]

-- 
bye,
pabs

https://wiki.debian.org/PaulWise


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
https://lists.debian.org/caktje6f4jnh2kmt01bod5f5v9kfbfsfz5o675sfnb5w5xmk...@mail.gmail.com

Detecting more undeclared conflicts (was: Re: Bug#758234: Raising priority of Debian packages)

[Johannes Schauer]

Hi,

Quoting Paul Wise (2014-09-07 11:38:27)
> On Fri, Sep 5, 2014 at 3:35 AM, Jakub Wilk wrote:
> 
> > We should probably also monitor package conflicts. We made a big fuss about
> > node vs nodejs (and rightly so); but I bet that we have lots of other
> > package pairs in the archive that can't be co-installed for no good reason.
> 
> We have this already:
> 
> https://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=edos-file-overwrite;users=trei...@debian.org
> https://qa.debian.org/dose/file-overwrites.html

would it make sense to extend this test and not only check whether packages
that share a file listed in Contents.gz can be co-installed but also packages
which access/change/create the same files in their pre/post-install maintainer
scripts do?

Is the information about which files are touched in any way by maintainer
scripts generated already somewhere? I think piuparts only takes two snapshots
before installation and after removal but not one in the middle and neither
runs a file system access tracer for all other changes to the filesystem?

If this could be useful, then I could run some tests locally, using fatrace
similar to how I used it to detect unneeded build dependencies.

Does this make sense?

cheers, josch


--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20140907103606.3685.16945@hoothoot

Re: Bug#758234: Raising priority of Debian packages

[Paul Wise]

On Fri, Sep 5, 2014 at 3:35 AM, Jakub Wilk wrote:

> We should probably also monitor package conflicts. We made a big fuss about
> node vs nodejs (and rightly so); but I bet that we have lots of other
> package pairs in the archive that can't be co-installed for no good reason.

We have this already:

https://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=edos-file-overwrite;users=trei...@debian.org
https://qa.debian.org/dose/file-overwrites.html

-- 
bye,
pabs

https://wiki.debian.org/PaulWise


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
https://lists.debian.org/caktje6fhf2qpkcr0lvtmok-wtoa4+z7pj3rwfa1qyfub7kx...@mail.gmail.com

Re: Bug#758234: Raising priority of Debian packages

[Jakub Wilk]

* Russ Allbery , 2014-08-31, 09:08:
If we want, as a project, to monitor the size of the required, 
important, and standard sets, I feel like we should do that directly: 
run a cron job somewhere that remembers the previous size, calculates 
the new transitive closure, and mails out a report once a week or month 
or something listing which packages were added or removed and their 
sizes, as well as the total installed size of each of those sets.  I 
think that would be more effective and more directly on point than 
mucking about with trying to monitor priority changes.


Yes, that would be a good idea.

We should probably also monitor package conflicts. We made a big fuss 
about node vs nodejs (and rightly so); but I bet that we have lots of 
other package pairs in the archive that can't be co-installed for no 
good reason.


--
Jakub Wilk


--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/20140904193554.ga3...@jwilk.net

Re: Raising priority of Debian packages

[Russ Allbery]

Paul Wise  writes:

> I am missing the justification for the last paragraph of Debian Policy
> section 2.5. I feel that it may have been due to a limitation of our
> tools in the past and that these days there are zero downsides to this
> situation (debootstrap/etc handle it fine) so we should probably remove
> the paragraph from policy.

> There is also a downside to raising priorities in this way; raising the
> priority of dependencies of higher-priority packages means that if
> implementations of higher-priority packages change and the (now higher
> priority) dependencies are no longer needed, we have to do work to
> revert those changes. I think they would end up as cruft that is
> forgotten about or just cause busywork that we can avoid in the first
> place by not raising priority of dependencies.

Yeah, that's one of the solutions to the broader problem that we were
discussing on debian-policy.  Gerrit was concerned that making this change
would make it even harder to track changes to the size of the required,
important, and standard sets than it is now, but I'm not convinced that
the priority concept is the right machinery for doing that.

What I think you're saying, and what I was thinking about after that
discussion, is whether we should recast priorities as applying to the
leaf packages that we're trying to get installed.  In other words, the
things we're specifically interested in having on the system would be
required, important, or standard.  Everything else (such as all the shared
libraries) would be optional.  Then calculating the size of required would
mean calculating the transitive closure of packages pulled in as
dependencies of required packages.

I feel like this is essentially what we've been doing in past releases;
we've just taken the bookkeeping step before the release of raising the
priority of all the packages in that dependency tree to match the priority
of the leaf package.  I'm not sure there's much point in doing that, since
the information is retrievable other ways.

If we want, as a project, to monitor the size of the required, important,
and standard sets, I feel like we should do that directly: run a cron job
somewhere that remembers the previous size, calculates the new transitive
closure, and mails out a report once a week or month or something listing
which packages were added or removed and their sizes, as well as the total
installed size of each of those sets.  I think that would be more
effective and more directly on point than mucking about with trying to
monitor priority changes.

-- 
Russ Allbery (r...@debian.org)   


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/87tx4sy73x@hope.eyrie.org

Re: Raising priority of Debian packages

[Michael Biebl]

Am 31.08.2014 17:58, schrieb Paul Wise:
> There is also a downside to raising priorities in this way; raising
> the priority of dependencies of higher-priority packages means that if
> implementations of higher-priority packages change and the (now higher
> priority) dependencies are no longer needed, we have to do work to
> revert those changes. I think they would end up as cruft that is
> forgotten about or just cause busywork that we can avoid in the first
> place by not raising priority of dependencies.

Completely agreed. That's why I believe it's counterproductive to raise
the priorities or library or helper packages.

Those packages should never be pulled on their own due to their priority.

I think policiy should really be changed in that regard.

-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?



signature.asc
Description: OpenPGP digital signature

Re: Raising priority of Debian packages

[Paul Wise]

On Sun, Aug 31, 2014 at 8:46 AM, Russ Allbery wrote:

> The problem that set off this entire discussion is the question of whether
> we want to raise the priority of perl, perl-modules, and
> init-system-helpers to important to match rsyslog, move the Perl modules
> used by init-system-helpers to perl-core (thereby adding them to
> Essential), or take some third course of action.  That's well worth
> discussing in a broader context, and I feel like trying to phrase that
> question as a general discussion of the role of priorities in Debian is
> not very helpful in starting that discussion.

I am missing the justification for the last paragraph of Debian Policy
section 2.5. I feel that it may have been due to a limitation of our
tools in the past and that these days there are zero downsides to this
situation (debootstrap/etc handle it fine) so we should probably
remove the paragraph from policy.

There is also a downside to raising priorities in this way; raising
the priority of dependencies of higher-priority packages means that if
implementations of higher-priority packages change and the (now higher
priority) dependencies are no longer needed, we have to do work to
revert those changes. I think they would end up as cruft that is
forgotten about or just cause busywork that we can avoid in the first
place by not raising priority of dependencies.

-- 
bye,
pabs

https://wiki.debian.org/PaulWise


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
https://lists.debian.org/caktje6gzfisnis__cex-rqqsznuvit6obugd3_xq4inh-_s...@mail.gmail.com

Re: Raising priority of Debian packages

[Russ Allbery]

Gerrit Pape  writes:
> On Sat, Aug 30, 2014 at 09:30:17AM -0700, Russ Allbery wrote:

>> Also, I'll reiterate what I said on debian-policy on this topic: the
>> current Policy discussion of priorities is deceptive, since it implies

> I don't think the discussion on the issue I raised is deceptive

The point that I'm making here is that you're talking about RC bugs
against packages, but, one, priority conflicts are not RC, and two,
priorities are not determined by packages.  They're determined by
ftp-master.  So I think you're barking up entirely the wrong tree by
trying to use the machinery of RC bugs to address the problem that you're
concerned with.

This feels very much to me like a specific problem (you disagree with
rsyslog depending on init-system-helpers) that has been abstracted into a
generalized issue with priorities.  There are general issues with
priorities, but I don't think they are as urgent as figuring out exactly
what we want to do about init-system-helpers and its additional Perl
module dependencies, since we've been living with those issues for a very
long time.

The problem that set off this entire discussion is the question of whether
we want to raise the priority of perl, perl-modules, and
init-system-helpers to important to match rsyslog, move the Perl modules
used by init-system-helpers to perl-core (thereby adding them to
Essential), or take some third course of action.  That's well worth
discussing in a broader context, and I feel like trying to phrase that
question as a general discussion of the role of priorities in Debian is
not very helpful in starting that discussion.

-- 
Russ Allbery (r...@debian.org)   


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/8738cczmoo@hope.eyrie.org

Re: Raising priority of Debian packages

[Tollef Fog Heen]

]] Gerrit Pape 

> When having such bugs filed, and the release team decides they shall not
> be release critical, this shall be made public, the severities of
> corresponding bugs adjusted, and everything is fine and transparent.
> Currently I'm not aware of such a release team's decision.

You might want to read https://release.debian.org/jessie/rc_policy.txt .  
It does not include anything about package priorities, from what I
can tell.  (It also seems to need updating with at least an
s/wheezy/jessie/ in a few places).

-- 
Tollef Fog Heen
UNIX is user friendly, it's just picky about who its friends are


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/87zjeksqbq@aexonyam.err.no

Re: Raising priority of Debian packages

[Gerrit Pape]

On Sat, Aug 30, 2014 at 09:30:17AM -0700, Russ Allbery wrote:
> Gerrit Pape  writes:
> > [0] Actually policy 2.5 requires to additionally file a RC bug to the
> > high priority package with the added dependency to prevent it from being
> > migrated to testing until the override decision has been made.
> 
> Policy does not address RC bugs at all.  The determination of whether a
> bug is RC is the province of the release team, not Policy.
> 
> Policy 2.5 says that appropriate priorities is a must, but does not say
> whether this would be a bug in the depending package or the package being
> depended on,

Me personally as a maintainer, when I plan to add a new dependency on a
lower priority package to a high priority package, I'm aware that this
will introduce a new violation of Debian policy into the archive.
Reading the BTS documentation on severities

  serious
  is a severe violation of Debian policy (roughly, it violates a must or
  required directive), or, in the package maintainer's or release
  manager's opinion, makes the package unsuitable for release.

I either first try to get the priorities of the involved packages
straight, or, when uploading with the new dependency added, feel that I
shall file the serious bug against my own new packages version.  After
all this very upload is the change to the Debian archive that introduces
the new priority conflict, and can wait a bit longer before migrating
into testing.  To me personally it's clear to which package the bug
report belongs, without having policy telling me.

But the RC bug thing actually is not what I'm after, let me iterate here
though that I have the feeling the Debian culture is changing to somehow
fear RC bugs, and also flames so that more and more things are done
without making the information publicly available.  I don't like this
change.

 Bug reports, including release critical ones, are a good thing!  They
 help us to improve our distribution and make high quality releases, we
 should utilize them where appropriate!

When having such bugs filed, and the release team decides they shall not
be release critical, this shall be made public, the severities of
corresponding bugs adjusted, and everything is fine and transparent.
Currently I'm not aware of such a release team's decision.


Back to my mail, the main content and not the footnote.  What I'd like
to see is more transparency (1) in the process of raising package
priorities and (2) in the information of existing priority conflicts in
the archive.

For (1) I share my view with fellows on this list, and suggest how this
can be achieved through single maintainer's actions.  For (2) I
suggested a change to policy that helps us to get a comprehensive view
on current priority conflicts that actually have an impact.  The
optional<->extra conflicts just dilute it.

 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=759260#62


> Also, I'll reiterate what I said on debian-policy on this topic: the
> current Policy discussion of priorities is deceptive, since it implies

I don't think the discussion on the issue I raised is deceptive

> that package maintainers are responsible for determining the priority of
> the package.  This is not the case; ftp-master determines the priority of
> packages, with input from package maintainers.  Policy discussion of
> priorities really needs some substantial revision to account for that, for
> the fact that conflict-free optional has not realistically been a project
> goal for some years, and to be clearer about just what we want to use
> priorities for.

and not connected to this, which sounds like an entirely different issue
that just happens to target the same area in the policy document.

Regards, Gerrit.


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
https://lists.debian.org/20140831074322.30727.qm...@da21230e36abc3.315fe32.mid.smarden.org

Re: Raising priority of Debian packages

[Russ Allbery]

Gerrit Pape  writes:

> [0] Actually policy 2.5 requires to additionally file a RC bug to the
> high priority package with the added dependency to prevent it from being
> migrated to testing until the override decision has been made.

Policy does not address RC bugs at all.  The determination of whether a
bug is RC is the province of the release team, not Policy.

Policy 2.5 says that appropriate priorities is a must, but does not say
whether this would be a bug in the depending package or the package being
depended on, and regardless:

Packages that do not conform to the guidelines denoted by must (or
required) will generally not be considered acceptable for the Debian
distribution.

The word "generally" is important, and we should probably highlight this
further.  The release team is free to decide whether Policy violations are
release-critical or not, and indeed there are must statements in Policy
that are not release-critical.

Obviously, this is to be avoided, but in the past it's frequently been
avoided by changing Policy when a must no longer makes sense.  And the
Policy change always trails the release team decision.

Also, I'll reiterate what I said on debian-policy on this topic: the
current Policy discussion of priorities is deceptive, since it implies
that package maintainers are responsible for determining the priority of
the package.  This is not the case; ftp-master determines the priority of
packages, with input from package maintainers.  Policy discussion of
priorities really needs some substantial revision to account for that, for
the fact that conflict-free optional has not realistically been a project
goal for some years, and to be clearer about just what we want to use
priorities for.

-- 
Russ Allbery (r...@debian.org)   


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: https://lists.debian.org/87vbpadjom@hope.eyrie.org

Raising priority of Debian packages

[Gerrit Pape]

Hi,

thanks to manual actions from Ansgar Burchardt, the process to raise
priority of Debian packages, after dependencies to packages of lower
priority have been added to high priority packages, is now a bit more
transparent

 
https://bugs.debian.org/cgi-bin/pkgreport.cgi?pkg=ftp.debian.org;dist=unstable#_0_1_4

Here we can see for example that amongst other packages, perl and
perl-modules are queued to be changed from standard to priority
important.

Still, that's not ideal.  IMHO maintainers of high (<= important)
priority packages should (1) think twice before adding new dependencies
to lower priority packages, and (2) file a corresponding bug against
ftp.debian.org when doing so to make it transparent[0].

Interesting parties then can track the list of override requests in the
BTS and comment when disired or necessary.  It might be that I'm the
only one interested in having a consistent and small, if not minimal,
set of packages in the higher priorities, but maybe there're others.

[0] Actually policy 2.5 requires to additionally file a RC bug to the
high priority package with the added dependency to prevent it from being
migrated to testing until the override decision has been made.

Regards, Gerrit.

 
On Fri, Aug 15, 2014 at 05:27:31PM +, Gerrit Pape wrote:
> On Fri, Aug 15, 2014 at 06:51:44PM +0200, Bill Allombert wrote:
> > Please do not report RC bug for this. Priorities are adjusted by the FTP 
> > master
> > team by batch using the overrides file. There is no need to report bugs 
> > against
> > the packages.
> 
> Hi, I filed three bugs for the extreme, where priority required depends
> on priority extra in current jessie.  No fear, I won't do mass filing.
> 
> I queried all the numbers on this beforehand.  And I don't think it's
> good to solve all this through ftp master's override[0].  This will
> bloat the installations using high priority packages more and more.
> 
> In stable we have the following violations (packages)
>  important: 8
>  required: 2
>  standard: 21
> In current testing:
>  important: 15
>  required: 5
>  standard: 43
> 
> It's not hundreds, or thousands.  Keep cool.
> rsyslog was fine when we raised its priority in wheezy.


-- 
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
https://lists.debian.org/20140830073110.28052.qm...@353a9a2a72ded5.315fe32.mid.smarden.org