Re: Requiring package test instructions (was: Re: Too fast karma on Bodhi updates)

On Wed, 2016-07-13 at 10:21 +0100, Jonathan Wakely wrote: > On 13/07/16 08:21 +0530, Siddhesh Poyarekar wrote: > > > > On Tue, Jul 12, 2016 at 03:45:54PM -0700, Adam Williamson wrote: > > > > > > Bodhi works at the source package level, not binary package level. > > That's irrelevant. If a

Re: Requiring package test instructions (was: Re: Too fast karma on Bodhi updates)

On 13/07/16 08:21 +0530, Siddhesh Poyarekar wrote: On Tue, Jul 12, 2016 at 03:45:54PM -0700, Adam Williamson wrote: Bodhi works at the source package level, not binary package level. That's irrelevant. If a source package only provides a library for other packages to link against then testing

Re: Requiring package test instructions (was: Re: Too fast karma on Bodhi updates)

On Tue, Jul 12, 2016 at 11:52:45PM -0700, Adam Williamson wrote: > FWIW, as someone who is working on this, I don't think we can > realistically aim to do distribution-level automated testing with per- > package granularity. We actually have all the bits in place to do > something like that if we

Re: Requiring package test instructions (was: Re: Too fast karma on Bodhi updates)

On Tue, 2016-07-12 at 23:52 -0700, Adam Williamson wrote: > > Of course, we don't *have* to pick one thing or the other necessarily; > we can certainly provide all the appropriate hooks for packages to do > automated update testing, this is something folks are already looking > at, and there's no

Re: Requiring package test instructions (was: Re: Too fast karma on Bodhi updates)

On Wed, 2016-07-13 at 11:08 +0530, Siddhesh Poyarekar wrote: > On Tue, Jul 12, 2016 at 10:26:20PM -0700, Adam Williamson wrote: > > > > It would not be 'a lot of work', it would be a gigantic, totally > > unsustainable burden. I honestly think you're shooting *way* too high > > here. Even with

Re: Requiring package test instructions (was: Re: Too fast karma on Bodhi updates)

On Tue, Jul 12, 2016 at 10:26:20PM -0700, Adam Williamson wrote: > It would not be 'a lot of work', it would be a gigantic, totally > unsustainable burden. I honestly think you're shooting *way* too high > here. Even with all the recent volunteers, we have like a couple dozen I agree it is a

Re: Requiring package test instructions (was: Re: Too fast karma on Bodhi updates)

On Wed, 2016-07-13 at 10:55 +0530, Siddhesh Poyarekar wrote: > An individual wanting to get started in Fedora packaging has to prove > their competence and understanding of the packaging guidelines by > commenting on package review requests in addition to submitting their > own package for

Re: Requiring package test instructions (was: Re: Too fast karma on Bodhi updates)

On Wed, 2016-07-13 at 08:21 +0530, Siddhesh Poyarekar wrote: > On Tue, Jul 12, 2016 at 03:45:54PM -0700, Adam Williamson wrote: > > > > Bodhi works at the source package level, not binary package level. > > I think Jon's point was with respect to the scope of testing. With > glibc (or libstdc++

Re: Requiring package test instructions (was: Re: Too fast karma on Bodhi updates)

On Tue, Jul 12, 2016 at 09:12:25PM -0700, Gerald B. Cox wrote: > Instead of concentrating on testers, what about the packagers who don't > even test their > applications before throwing them over the wall to bodhi. I've seen > packages that didn't even > get past a simple dnf requisite test

Re: Requiring package test instructions (was: Re: Too fast karma on Bodhi updates)

On Tue, Jul 12, 2016 at 7:47 PM, Siddhesh Poyarekar wrote: > ... which is not enough. The definition of 'generally functional' is > vague, as all of us agree and we have seen examples of that being > misused in the past. Requiring devel to document their packages is >

Re: Requiring package test instructions (was: Re: Too fast karma on Bodhi updates)

On Tue, Jul 12, 2016 at 03:45:54PM -0700, Adam Williamson wrote: > Bodhi works at the source package level, not binary package level. I think Jon's point was with respect to the scope of testing. With glibc (or libstdc++ that Jon would be concerned with), an ideal set of sanity tests would cover

Re: Requiring package test instructions (was: Re: Too fast karma on Bodhi updates)

On Tue, Jul 12, 2016 at 12:16:59PM -0700, Adam Williamson wrote: > This is setting far too high a bar for a project like Fedora. We take > the feedback we can get, we are not in a position to demand all update > testers perform comprehensive testing of all possible facets of an > update. It is

Re: Requiring package test instructions (was: Re: Too fast karma on Bodhi updates)

On Tue, 2016-07-12 at 23:32 +0100, Jonathan Wakely wrote: > On 12/07/16 16:26 -0600, Kevin Fenzi wrote: > > > > Some packages are definitely going to be harder than others... > > For example, a libfoo, and its libfoo-devel subpackage. Bodhi works at the source package level, not binary package

Re: Requiring package test instructions (was: Re: Too fast karma on Bodhi updates)

On Tue, 2016-07-12 at 16:26 -0600, Kevin Fenzi wrote: > On Tue, 12 Jul 2016 09:49:14 -0700 > Adam Williamson wrote: > > > > > So I've been discussing this with various people in the last few days, > > and one specific idea has come out of that which I'd like to

Re: Requiring package test instructions (was: Re: Too fast karma on Bodhi updates)

On 12/07/16 16:26 -0600, Kevin Fenzi wrote: Some packages are definitely going to be harder than others... For example, a libfoo, and its libfoo-devel subpackage. -- devel mailing list devel@lists.fedoraproject.org https://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: Requiring package test instructions (was: Re: Too fast karma on Bodhi updates)

On Tue, 12 Jul 2016 09:49:14 -0700 Adam Williamson wrote: > So I've been discussing this with various people in the last few days, > and one specific idea has come out of that which I'd like to float. > > We've been hesitant to suggest this before as we thought

Re: Requiring package test instructions (was: Re: Too fast karma on Bodhi updates)

On Wed, 2016-07-13 at 00:18 +0530, Siddhesh Poyarekar wrote: > On Tue, Jul 12, 2016 at 11:38:01AM -0700, Adam Williamson wrote: > > > > This isn't really correct, because there is no simple relationship > > between 'bugs claimed to be fixed actually are fixed' and 'update > > should be released'.

Re: Requiring package test instructions (was: Re: Too fast karma on Bodhi updates)

On Tue, Jul 12, 2016 at 11:38:01AM -0700, Adam Williamson wrote: > This isn't really correct, because there is no simple relationship > between 'bugs claimed to be fixed actually are fixed' and 'update > should be released'. Both of these are possible: > > 1) an update which fixes the bugs it

Re: Requiring package test instructions (was: Re: Too fast karma on Bodhi updates)

On Tue, 2016-07-12 at 23:57 +0530, Siddhesh Poyarekar wrote: > On Tue, Jul 12, 2016 at 09:49:14AM -0700, Adam Williamson wrote: > > > > To be clear, the idea would be to have general-purpose instructions for > > basic functionality testing of each package, not requiring new 'how to > > test' text

Re: Requiring package test instructions (was: Re: Too fast karma on Bodhi updates)

On Tue, Jul 12, 2016 at 09:49:14AM -0700, Adam Williamson wrote: > To be clear, the idea would be to have general-purpose instructions for > basic functionality testing of each package, not requiring new 'how to > test' text to be written for every individual package update, > specifically

Re: Requiring package test instructions (was: Re: Too fast karma on Bodhi updates)

2016-07-12 10:49 GMT-06:00 Adam Williamson : > On Sun, 2016-07-10 at 21:30 +0530, Sayan Chowdhury wrote: > > Hi, > > > > I recently packaged and pushed an update for > fedmsg-meta-fedora-infrastructure > > to bodhi and exactly 40 secs[1] later I got a +1 to the update.

Requiring package test instructions (was: Re: Too fast karma on Bodhi updates)

On Sun, 2016-07-10 at 21:30 +0530, Sayan Chowdhury wrote: > Hi, > > I recently packaged and pushed an update for fedmsg-meta-fedora-infrastructure > to bodhi and exactly 40 secs[1] later I got a +1 to the update. I am sure that > testing a package surely takes more than 40 secs. This makes me