Re: depcheck test (was Re: measuring success)

2010-07-13 Thread Luke Macken
On 07/13/2010 04:59 PM, Till Maas wrote: > On Tue, Jul 13, 2010 at 03:55:46PM -0400, Luke Macken wrote: >> This patch looks good at a first glance -- it's pretty much exactly what >> I was planning to do. The only tweak that is needed is to ensure that >> anonymous people can't pretend to be AutoQ

Re: depcheck test (was Re: measuring success)

2010-07-13 Thread Till Maas
On Tue, Jul 13, 2010 at 03:55:46PM -0400, Luke Macken wrote: > This patch looks good at a first glance -- it's pretty much exactly what > I was planning to do. The only tweak that is needed is to ensure that > anonymous people can't pretend to be AutoQA: > > -if comment.author == "a

Re: depcheck test (was Re: measuring success)

2010-07-13 Thread Luke Macken
On 07/06/2010 04:09 PM, Till Maas wrote: > On Tue, Jul 06, 2010 at 03:06:37PM -0400, Will Woods wrote: >> On Tue, 2010-07-06 at 19:21 +0200, Till Maas wrote: >>> On Tue, Jul 06, 2010 at 09:40:01AM -0700, Jesse Keating wrote: On 7/6/10 8:52 AM, Till Maas wrote: > IMHO it should not be a +1

Re: measuring success

2010-07-13 Thread Luke Macken
On 07/03/2010 06:50 AM, Till Maas wrote: > Also Bodhi does not allow to fix updates by other people than the update > submitter afaik. Anyone with commit privs to the rawhide branch of a package should be able to submit/edit updates for that package. Yes, it's not ideal, but that is how it is c

Re: depcheck test (was Re: measuring success)

2010-07-07 Thread Will Woods
On Tue, 2010-07-06 at 12:11 -0700, Adam Williamson wrote: > On Tue, 2010-07-06 at 11:34 -0400, Will Woods wrote: > > > If there are any other questions, feel free to ask. > > > > -w > > Did you get to look at the nss-softokn situation (details of which I > sent to autoqa-devel) yet? How hard wou

Re: depcheck test (was Re: measuring success)

2010-07-06 Thread Kevin Kofler
Till Maas wrote: > Btw. using the "path of least resistance" to implement policy > changes seems to be what makes the new workflows suck for package > maintainers, e.g. with the change in place using a auto-karma value of 1 > will become 0. That would be a good thing! It'd make all those requireme

Re: Requires --> BuildRequires (was: Re: measuring success)

2010-07-06 Thread Kevin Kofler
Michael Schwendt wrote: > And there would be drawbacks, too, for a hardcoded "Req => BR" rule. > It would make circular deps impossible: Pkg A requires Pkg A-extras, > and Pkg A-extras BR Pkg A. It would make bootstrapping a dist more > complicated. For some pkgs (e.g. leaf pkgs) it is fine if the

Re: measuring success [was Re: Bodhi 0.7.5 release]

2010-07-06 Thread Kevin Kofler
Michael Schwendt wrote: > One can quickly see that several (if not many) of them are due > to orphans/retired packages in Fedora 12. And due to violated upgrade > paths (e.g. compat-db): That just proves that we should avoid retiring packages, but try to keep them alive as long as we can, even if

Re: depcheck test (was Re: measuring success)

2010-07-06 Thread Till Maas
On Tue, Jul 06, 2010 at 10:09:34PM +0200, Till Maas wrote: > On Tue, Jul 06, 2010 at 03:06:37PM -0400, Will Woods wrote: > > On Tue, 2010-07-06 at 19:21 +0200, Till Maas wrote: > > > On Tue, Jul 06, 2010 at 09:40:01AM -0700, Jesse Keating wrote: > > > > On 7/6/10 8:52 AM, Till Maas wrote: > > > > >

Re: depcheck test (was Re: measuring success)

2010-07-06 Thread Till Maas
On Tue, Jul 06, 2010 at 11:25:01AM -0700, Jesse Keating wrote: > On 07/06/2010 10:21 AM, Till Maas wrote: > > Essentially using a different flag is just re-using the code used to > > flag a package as critpath-approved only with a different name. > > Therefore it should not need that much more effo

Re: depcheck test (was Re: measuring success)

2010-07-06 Thread Till Maas
On Tue, Jul 06, 2010 at 03:06:37PM -0400, Will Woods wrote: > On Tue, 2010-07-06 at 19:21 +0200, Till Maas wrote: > > On Tue, Jul 06, 2010 at 09:40:01AM -0700, Jesse Keating wrote: > > > On 7/6/10 8:52 AM, Till Maas wrote: > > > > IMHO it should not be a +1 karma but some different flag that is set

Re: depcheck test (was Re: measuring success)

2010-07-06 Thread Adam Williamson
On Tue, 2010-07-06 at 11:34 -0400, Will Woods wrote: > If there are any other questions, feel free to ask. > > -w Did you get to look at the nss-softokn situation (details of which I sent to autoqa-devel) yet? How hard would it be to catch that? -- Adam Williamson Fedora QA Community Monkey IRC

Re: depcheck test (was Re: measuring success)

2010-07-06 Thread Adam Williamson
On Tue, 2010-07-06 at 19:21 +0200, Till Maas wrote: > Essentially using a different flag is just re-using the code used to > flag a package as critpath-approved only with a different name. > Therefore it should not need that much more effort. > > Btw. using the "path of least resistance" to imple

Re: depcheck test (was Re: measuring success)

2010-07-06 Thread Will Woods
On Tue, 2010-07-06 at 19:21 +0200, Till Maas wrote: > On Tue, Jul 06, 2010 at 09:40:01AM -0700, Jesse Keating wrote: > > On 7/6/10 8:52 AM, Till Maas wrote: > > > IMHO it should not be a +1 karma but some different flag that is set for > > > updates that passed the tests. > > > > Using karma is vi

Re: depcheck test (was Re: measuring success)

2010-07-06 Thread Jesse Keating
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/06/2010 10:21 AM, Till Maas wrote: > Essentially using a different flag is just re-using the code used to > flag a package as critpath-approved only with a different name. > Therefore it should not need that much more effort. critpath approved i

Re: depcheck test (was Re: measuring success)

2010-07-06 Thread Till Maas
On Tue, Jul 06, 2010 at 11:34:25AM -0400, Will Woods wrote: > I'll attempt to give a brief summary here. First you need to understand > that there are three states for a package that has been built with the > hope of being pushed as an update: > * 'candidate': freshly-built packages intended for u

Re: depcheck test (was Re: measuring success)

2010-07-06 Thread Till Maas
On Tue, Jul 06, 2010 at 09:40:01AM -0700, Jesse Keating wrote: > On 7/6/10 8:52 AM, Till Maas wrote: > > On Tue, Jul 06, 2010 at 11:34:25AM -0400, Will Woods wrote: > > > >> Once we're satisfied that depcheck does the right thing, we will > >> probably set it up to start adding automatic +1 karma

Re: depcheck test (was Re: measuring success)

2010-07-06 Thread Jesse Keating
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 7/6/10 8:52 AM, Till Maas wrote: > On Tue, Jul 06, 2010 at 11:34:25AM -0400, Will Woods wrote: > >> Once we're satisfied that depcheck does the right thing, we will >> probably set it up to start adding automatic +1 karma from 'autoqa' when >> upda

Re: depcheck test (was Re: measuring success)

2010-07-06 Thread Till Maas
On Tue, Jul 06, 2010 at 11:34:25AM -0400, Will Woods wrote: > Once we're satisfied that depcheck does the right thing, we will > probably set it up to start adding automatic +1 karma from 'autoqa' when > updates pass the automated test suite (depcheck and possibly other tests > - rpmlint, rpmguard

depcheck test (was Re: measuring success)

2010-07-06 Thread Will Woods
On Sat, 2010-07-03 at 12:27 +0200, Michael Schwendt wrote: > [About automating this during the push process, it is possible to have > a depchecker simulate a --skip-broken and exclude packages which break > dependencies. There are different strategies. However, the procedure > must be backed up by

Re: measuring success [was Re: Bodhi 0.7.5 release]

2010-07-04 Thread Till Maas
On Sun, Jul 04, 2010 at 04:47:19PM +0200, Michael Schwendt wrote: > On Sun, 04 Jul 2010 14:40:20 +0200, Till wrote: > > > > It's fairly easy to verify other broken deps, too: > > > http://admin.fedoraproject.org/updates/compat-db-4.7.25-3.fc13 > > > > For me it is not that easy, because the infor

Re: measuring success [was Re: Bodhi 0.7.5 release]

2010-07-04 Thread Michael Schwendt
On Sun, 04 Jul 2010 14:40:20 +0200, Till wrote: > > It's fairly easy to verify other broken deps, too: > > http://admin.fedoraproject.org/updates/compat-db-4.7.25-3.fc13 > > For me it is not that easy, because the information is confusion (or not > clearly arranged) or not directly accessible, e.

Re: measuring success [was Re: Bodhi 0.7.5 release]

2010-07-04 Thread Till Maas
On Sun, Jul 04, 2010 at 02:06:08PM +0200, Michael Schwendt wrote: > On Sun, 04 Jul 2010 13:32:14 +0200, Till wrote: > > > On Sun, Jul 04, 2010 at 12:31:57PM +0200, Michael Schwendt wrote: > > > Broken deps in Fedora 13 + updates + updates-testing when > > > also enabling Fedora 12 + updates + u

Re: measuring success [was Re: Bodhi 0.7.5 release]

2010-07-04 Thread Michael Schwendt
On Sun, 04 Jul 2010 13:32:14 +0200, Till wrote: > On Sun, Jul 04, 2010 at 12:31:57PM +0200, Michael Schwendt wrote: > > Broken deps in Fedora 13 + updates + updates-testing when > > also enabling Fedora 12 + updates + updates-testing. > > > > One can quickly see that several (if not many) of t

Re: Requires --> BuildRequires (was: Re: measuring success)

2010-07-04 Thread Michael Schwendt
On Sun, 04 Jul 2010 13:22:16 +0200, Till wrote: > On Sun, Jul 04, 2010 at 12:06:39PM +0200, Michael Schwendt wrote: > > > And there would be drawbacks, too, for a hardcoded "Req => BR" rule. > > It would make circular deps impossible: Pkg A requires Pkg A-extras, > > and Pkg A-extras BR Pkg A. I

Re: measuring success [was Re: Bodhi 0.7.5 release]

2010-07-04 Thread Till Maas
On Sun, Jul 04, 2010 at 12:31:57PM +0200, Michael Schwendt wrote: > Broken deps in Fedora 13 + updates + updates-testing when > also enabling Fedora 12 + updates + updates-testing. > > One can quickly see that several (if not many) of them are due > to orphans/retired packages in Fedora 12. And

Re: Requires --> BuildRequires (was: Re: measuring success)

2010-07-04 Thread Till Maas
On Sun, Jul 04, 2010 at 12:06:39PM +0200, Michael Schwendt wrote: > And there would be drawbacks, too, for a hardcoded "Req => BR" rule. > It would make circular deps impossible: Pkg A requires Pkg A-extras, > and Pkg A-extras BR Pkg A. It would make bootstrapping a dist more > complicated. For

Re: Requires --> BuildRequires (was: Re: measuring success)

2010-07-04 Thread Michael Schwendt
On Sun, 4 Jul 2010 12:33:56 +0200, Michel wrote: > This is not semantically part of building, though. I see two possible > solutions: > 1. Koji should check the explicitly-listed Requirements and refuse to > build a package if these >are not available As I wrote in the previous msg, it would

Re: Requires --> BuildRequires (was: Re: measuring success)

2010-07-04 Thread Michel Alexandre Salim
On Sun, Jul 4, 2010 at 12:06 PM, Michael Schwendt wrote: > On Sat, 03 Jul 2010 13:35:20 +0200, Till wrote: > >> > 1) To make such run-time deps BuildRequires would be helpful (because it >> > doesn't make much sense to build something for a target that is missing >> > something). Several broken de

Re: measuring success [was Re: Bodhi 0.7.5 release]

2010-07-04 Thread Michael Schwendt
Broken deps in Fedora 13 + updates + updates-testing when also enabling Fedora 12 + updates + updates-testing. One can quickly see that several (if not many) of them are due to orphans/retired packages in Fedora 12. And due to violated upgrade paths (e.g. compat-db): Summary of broken packages

Requires --> BuildRequires (was: Re: measuring success)

2010-07-04 Thread Michael Schwendt
On Sat, 03 Jul 2010 13:35:20 +0200, Till wrote: > > 1) To make such run-time deps BuildRequires would be helpful (because it > > doesn't make much sense to build something for a target that is missing > > something). Several broken deps in old dist branches have been because > > of a discrepancy b

Re: measuring success [was Re: Bodhi 0.7.5 release]

2010-07-03 Thread Adam Williamson
On Sat, 2010-07-03 at 20:40 +0200, Michael Schwendt wrote: > > That only handles a subset of the 'broken dependencies' problem. We've > > already had an example this year of a dependency issue the proposed > > autoqa depcheck test wouldn't catch, and Michael's script didn't - the > > nss-softokn

Re: measuring success [was Re: Bodhi 0.7.5 release]

2010-07-03 Thread Michael Schwendt
On Sat, 03 Jul 2010 10:05:07 -0700, Adam wrote: > On Fri, 2010-07-02 at 18:24 +0200, Ralf Corsepius wrote: > > On 07/02/2010 06:20 PM, Will Woods wrote: > > > > > > > The main reasons we want to perform testing are things like: to avoid > > > pushing updates with broken dependencies, or updates

Re: measuring success [was Re: Bodhi 0.7.5 release]

2010-07-03 Thread Adam Williamson
On Fri, 2010-07-02 at 18:24 +0200, Ralf Corsepius wrote: > On 07/02/2010 06:20 PM, Will Woods wrote: > > > > The main reasons we want to perform testing are things like: to avoid > > pushing updates with broken dependencies, or updates that cause serious > > regressions requiring manual intervent

Re: measuring success [was Re: Bodhi 0.7.5 release]

2010-07-03 Thread Kevin Kofler
Adam Miller wrote: > If there are any discrepancy with the proventesters critpath policy then > please feel free to file a ticket with FESCo and allow our elected > officials decide the fate of this. There isn't any such discrepancy, it's the policy which is broken and FESCo which refuses to unde

Re: measuring success

2010-07-03 Thread Till Maas
On Sat, Jul 03, 2010 at 01:03:41PM +0200, Michael Schwendt wrote: > On Sat, 03 Jul 2010 12:50:15 +0200, Till wrote: > > Also Bodhi does not allow to [...] > > Bodhi ought to meet the package maintainers' requirements, not vice versa. > If you determine a problem with the typical work-flow, how ab

Re: measuring success

2010-07-03 Thread Till Maas
On Sat, Jul 03, 2010 at 01:03:41PM +0200, Michael Schwendt wrote: > On Sat, 03 Jul 2010 12:50:15 +0200, Till wrote: > > > This is not true, because there can be runtime dependencies on another > > update in -testing that is not build dependency, e.g. if an python app > > requires a newer version o

Re: measuring success

2010-07-03 Thread Michael Schwendt
On Sat, 03 Jul 2010 12:50:15 +0200, Till wrote: > This is not true, because there can be runtime dependencies on another > update in -testing that is not build dependency, e.g. if an python app > requires a newer version of a python module. 1) To make such run-time deps BuildRequires would be hel

Re: measuring success

2010-07-03 Thread Till Maas
On Sat, Jul 03, 2010 at 12:27:50PM +0200, Michael Schwendt wrote: > On Fri, 02 Jul 2010 20:12:26 +0200, Till wrote: > > > Btw. on a related issue:How do provenpackagers properly test for broken > > deps manually? > > Every packager can [configure and] run repoclosure from yum-utils. > Enable upda

Re: measuring success

2010-07-03 Thread Michael Schwendt
On Fri, 02 Jul 2010 20:12:26 +0200, Till wrote: > Btw. on a related issue:How do provenpackagers properly test for broken > deps manually? Every packager can [configure and] run repoclosure from yum-utils. Enable updates-testing, and optionally add a local repo for your own candidate builds. It s

Re: measuring success

2010-07-03 Thread Till Maas
On Sat, Jul 03, 2010 at 06:31:35AM +0200, Ralf Corsepius wrote: > On 07/02/2010 08:12 PM, Till Maas wrote: > > > Btw. on a related issue:How do provenpackagers properly test for broken > > deps manually? > Like ordinary packagers should do ;) > > The only difference between provenpackagers and "o

Re: measuring success [was Re: Bodhi 0.7.5 release]

2010-07-03 Thread Adam Miller
If there are any discrepancy with the proventesters critpath policy then please feel free to file a ticket with FESCo and allow our elected officials decide the fate of this. -AdamM (From Android) On Jul 2, 2010 8:16 PM, "Kevin Kofler" wrote: Will Woods wrote: > The main reasons we want to perf

Re: measuring success

2010-07-02 Thread Ralf Corsepius
On 07/02/2010 08:12 PM, Till Maas wrote: > Btw. on a related issue:How do provenpackagers properly test for broken > deps manually? Like ordinary packagers should do ;) The only difference between provenpackagers and "ordinary packagers" is them having write access to packages they do not own.

Re: measuring success [was Re: Bodhi 0.7.5 release]

2010-07-02 Thread Kevin Kofler
Will Woods wrote: > The main reasons we want to perform testing are things like: to avoid > pushing updates with broken dependencies The right way to prevent that is to get AutoQA completed, which will, if it works as intended, automatically detect and throw out updates with broken dependencies

Re: measuring success

2010-07-02 Thread Till Maas
On Fri, Jul 02, 2010 at 12:20:21PM -0400, Will Woods wrote: > Therefore: I propose that we choose a few metrics ("turnaround time on > security updates", "average number of live updates with broken > dependencies per day", etc.). Then we begin measuring them (and, if > possible, collect historical

Re: measuring success [was Re: Bodhi 0.7.5 release]

2010-07-02 Thread Ralf Corsepius
On 07/02/2010 06:20 PM, Will Woods wrote: > The main reasons we want to perform testing are things like: to avoid > pushing updates with broken dependencies, or updates that cause serious > regressions requiring manual intervention / emergency update > replacements. That sort of thing. > Should b

measuring success [was Re: Bodhi 0.7.5 release]

2010-07-02 Thread Will Woods
On Thu, 2010-07-01 at 06:33 +0200, Kevin Kofler wrote: > Fedora Legacy has shown how well this works… not! > > I completely agree with Ralf Corsepius and Tom Lane on this subject: this > policy is very unhelpful, and applying it to security updates is just > totally insane. We're going to see m