Re: [gentoo-dev] Re: taking a break from arches stabilization

[Pacho Ramos]

El mié, 12-07-2017 a las 09:13 -0500, William Hubbs escribió:
> On Wed, Jul 12, 2017 at 02:30:34PM +0200, Kristian Fiskerstrand wrote:
> > On 07/12/2017 01:59 PM, Michael Palimaka wrote:
> > > If it's not a stable candidate then why do you use this as an example
> > > against build testing-based stabilisations? If there are known issues it
> > > should never reach the arch teams in the first place.
> > 
> > This might be the crux of things, as long as automatic stabilization is
> > not triggered by some set of rules (e.g 30 days in ~arch) , and still
> > requires manual trigger by, preferably, the maintainer there is likely
> > no issue.
> 
> This doesn't make sense. If I have to trigger automatic stabilization, it
> isn't automatic any more.
> 
> I think with an appropriate set of rules automatic stabilization would
> be fine. For example:
> 
> - foo-2.0 has been in ~arch for 30 days
> - there are no open bugs against foo-2.0
> - an older version of foo is stable
> - all of the dependencies of foo-2.0 are stable
> 
> If those conditions are met, in theory there shouldn't be any problem
> with stabilizing foo-2.0.
> 
> If foo-2.0 is not a stabilization candidate, there probably should be an
> open bug in bugzilla stating why it isn't.
> 
> Thanks,
> 
> William
> 

Also please note that, when we were filling that automatic bug reports, there
were still an additional 60 days timeout (I think it was 60 days.. but I don't
remember if even 90 days) to allow maintainers to react. Only if nothing was
noted in relevant bug reports, arches were CCed automatically by the script 

Re: [gentoo-dev] Re: taking a break from arches stabilization

[William Hubbs]

On Wed, Jul 12, 2017 at 02:30:34PM +0200, Kristian Fiskerstrand wrote:
> On 07/12/2017 01:59 PM, Michael Palimaka wrote:
> > If it's not a stable candidate then why do you use this as an example
> > against build testing-based stabilisations? If there are known issues it
> > should never reach the arch teams in the first place.
> 
> This might be the crux of things, as long as automatic stabilization is
> not triggered by some set of rules (e.g 30 days in ~arch) , and still
> requires manual trigger by, preferably, the maintainer there is likely
> no issue.

This doesn't make sense. If I have to trigger automatic stabilization, it
isn't automatic any more.

I think with an appropriate set of rules automatic stabilization would
be fine. For example:

- foo-2.0 has been in ~arch for 30 days
- there are no open bugs against foo-2.0
- an older version of foo is stable
- all of the dependencies of foo-2.0 are stable

If those conditions are met, in theory there shouldn't be any problem
with stabilizing foo-2.0.

If foo-2.0 is not a stabilization candidate, there probably should be an
open bug in bugzilla stating why it isn't.

Thanks,

William



signature.asc
Description: Digital signature

Re: [gentoo-dev] Re: taking a break from arches stabilization

[Marek Szuba]

On 2017-07-12 00:26, Kristian Fiskerstrand wrote:

>> Question is what's more a problem: Having an outdated stable
>> package because nobody cared about stabilizing a new version (in
>> most cases this will end with a rushed stabilization once a
>> security bug was fixed in the package) or move a package in time
>> from ~ARCH to ARCH and deal with the fallout sometimes.
> Easy, keep the working package any time
Seconded.

That said, let me repeat something I mentioned during the last
discussion about stabilisation procedures: for me at least the problem
with stabilisation has never really been with version upgrades, it's
with packages which do not even have a single stable version. For
reference, as of right now my "version bump" keyword file lists 8 atoms
(half of them referring to GIMP-2.9, which should not be stabilised
anyway) - whereas my "not in stable" file lists 61.

-- 
MS



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] Re: taking a break from arches stabilization

[Kristian Fiskerstrand]

On 07/12/2017 01:59 PM, Michael Palimaka wrote:
> If it's not a stable candidate then why do you use this as an example
> against build testing-based stabilisations? If there are known issues it
> should never reach the arch teams in the first place.

This might be the crux of things, as long as automatic stabilization is
not triggered by some set of rules (e.g 30 days in ~arch) , and still
requires manual trigger by, preferably, the maintainer there is likely
no issue.

-- 
Kristian Fiskerstrand
OpenPGP keyblock reachable at hkp://pool.sks-keyservers.net
fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3



signature.asc
Description: OpenPGP digital signature

[gentoo-dev] Re: taking a break from arches stabilization

[Michael Palimaka]

On 07/12/2017 07:26 AM, Kristian Fiskerstrand wrote:> That presumes that
the maintainer is the one calling for the
> stabilization, and it is not an automated procedure simply due to 30
> days in ~arch. In this particular case, look for the number of bug
> reports filed in Gentoo for the issue.

All of the past automated stabilisation request initiatives did look for
open bugs against a package before filing any request.

> 
> But the main risk is certainly not built testing, it is breaking
> operational live stable systems.

I'm not sure exactly what you mean by this. Build testing is a process
and not a risk. When ebuilds are moved to stable, there's risk of
breakage at both build time and runtime.

Most runtime issues will either be known by the maintainer (who can then
block the stabilisation) or smoked out during the usual ~arch waiting
period.

Build time issues are more tricky. One of the reasons we have arch
testers is to ensure that something that built fine in ~arch will still
build fine on an otherwise stable system.

I've encountered a number of ebuilds marked stable that failed to build
on an all-stable system but built fine on an all-testing system. I've
never encountered a single ebuild that failed to run correctly on an
all-stable system but ran fine on an all-testing system.

I don't think it's practical to demand detailed runtime testing by an
arch tester for every stabilisation request (which we know doesn't
happen in practice anyway). There's also many packages where it may be
prudent to do more than just build testing.

I think the answer lies somewhere in the middle. We need to recognise
that we have very limited arch testing resources and focus them where it
will have the most impact. We have a "runtime testing required" field on
stable bugs now - let's use it, and let's be smart about it. Looking at
the current open bugs, I see some good examples where runtime testing
was requested (eg. sys-devel/gcc-config) and some examples where I
really question what value we get from an arch tester's time (eg.
app-text/htmldoc).

> Nowhere was it claimed that the arc
> testers are responsible for it, but it certainly doesn't coincide, at
> any point, with "The main risk of breakage of a package moving from
> testing to stable is always at build time anyway."
In a previous post you stated:
> currently gnupg 2.1.21 scdaemon bug will
> happily sign a third party public keyblock's UID using signature subkey
> on smartcard, which results in useless signature that doesn't have any
> effect, but the application builds fine.
>
> This means gnupg 2.1.21 is not a candidate for stabilization, but it
certainly builds fine.

If it's not a stable candidate then why do you use this as an example
against build testing-based stabilisations? If there are known issues it
should never reach the arch teams in the first place.

Re: [gentoo-dev] Re: taking a break from arches stabilization

[Sam Jorna (wraeth)]

On 12/07/17 03:16, William Hubbs wrote:
> On Tue, Jul 11, 2017 at 11:47:32PM +1000, Michael Palimaka wrote:
>> On 07/11/2017 11:06 PM, Rich Freeman wrote:
>>> On Tue, Jul 11, 2017 at 8:59 AM, Michael Palimaka  
>>> wrote:
 On 07/11/2017 09:29 AM, Andrew Savchenko wrote:
>
> Even if such stabilization is allowed, there are unanswered
> questions here:
> - is following seciton 4.1 from wg recommendations is sufficient?
> - should developer test each stabilization candidate on an
> up-to-date stable setup?

 The guidelines from that document are ripped straight out of the
 devmanual and are a good starting point but rather generic. You can find
 some more detailed suggestions on things to consider while testing on
 the wiki: https://wiki.gentoo.org/wiki/Package_testing

>>>
>>> I think that in practice arch teams don't do have the stuff on that
>>> wiki page.  Maybe some people do, but back when I was an amd64 AT I
>>> don't think anybody went testing multiple USE combinations for a
>>> typical package.
>>
>> Everything on that page is deliberately a suggestion only, and not
>> necessarily specific to stabilisation testing.
>>
>> In the end, we've never been able to reach any consensus on what exactly
>> an arch tester should do. Personally, I think we should just switch to
>> fully-automated, build-only testing for stabilistions unless the
>> maintainer opts otherwise (something that largely happens in practice
>> already). The main risk of breakage of a package moving from testing to
>> stable is always at build time anyway.
> 
> I would not be opposed to this. As a maintainer, I am as guilty as the
> next guy of not filing stable requests or not stabilizing packages.

As the next guy, I also +1 this.

As is often explained in #gentoo, "stable" and "testing" for upstream
have a different meaning to "stable" and "testing" for Gentoo - in fact,
beyond ensuring it builds, any testing performed by a downstream distro
is additional testing to what upstream have already released.

I can understand the concern with automatically stabilising a package
that has some flaw affecting users, but the two things i see are that
upstream have already released said flawed package, and Gentoo simply
doesn't have the manpower to perform comprehensive runtime testing of
all packages.

If a maintainer is aware of a significant issue with a package that
should prevent it from being marked as stable, then a bug should be
filed acting as a block to the automated stabilisation. Users aware of
an issue are just as likely to file a bug as well, also preventing
stabilisation of packages with some known issue. Therefore packages with
known issues don't get stabilised automatically.

Similarly, if the maintainer believes more comprehensive testing should
be done (eg. for critical base-system packages) a note could be made
somewhere of that requirement (metadata.xml?), meaning significantly
reduced workload for people like ago (if the maintainer doesn't
stabilise it beforehand).
-- 
Sam Jorna (wraeth)
GnuPG ID: D6180C26



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] Re: taking a break from arches stabilization

[Mart Raudsepp]

Ühel kenal päeval, K, 12.07.2017 kell 00:13, kirjutas Thomas
Deutschmann:
> Let's try Debian's testing
> approach and move packages to ARCH in time and don't wait for some
> magical appearing bug reports because someone really tested a package
> in
> ~ARCH. Severe problems will be reported anyways...

You should realize that this is what was happening.
The problem is, that no-one is doing keywording and stabilization queue
got stalled due to stabilization bugs depending on keywording bugs as
is due process and proper bugzilla sanity.

Re: [gentoo-dev] Re: taking a break from arches stabilization

[Kristian Fiskerstrand]

On 07/12/2017 12:13 AM, Thomas Deutschmann wrote:
> Question is what's more a problem: Having an outdated stable package
> because nobody cared about stabilizing a new version (in most cases this
> will end with a rushed stabilization once a security bug was fixed in
> the package) or move a package in time from ~ARCH to ARCH and deal with
> the fallout sometimes.

Easy, keep the working package any time

-- 
Kristian Fiskerstrand
OpenPGP keyblock reachable at hkp://pool.sks-keyservers.net
fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] Re: taking a break from arches stabilization

[Thomas Deutschmann]

>>> Anecdotal evidence against, currently gnupg 2.1.21 scdaemon bug will
>>> happily sign a third party public keyblock's UID using signature subkey
>>> on smartcard, which results in useless signature that doesn't have any
>>> effect, but the application builds fine.
>>>
>>> This means gnupg 2.1.21 is not a candidate for stabilization, but it
>>> certainly builds fine.
>>>
>>
>> Stop trolling - you know perfectly well that this sort of issue would
>> never ever be caught during arch testing. Nor should it be - it's called
>> *arch* testing for a reason.

Question is what's more a problem: Having an outdated stable package
because nobody cared about stabilizing a new version (in most cases this
will end with a rushed stabilization once a security bug was fixed in
the package) or move a package in time from ~ARCH to ARCH and deal with
the fallout sometimes.

Having a real AT doing real arch testing work would be ideal. But face
it: We don't have the required man power. Let's try Debian's testing
approach and move packages to ARCH in time and don't wait for some
magical appearing bug reports because someone really tested a package in
~ARCH. Severe problems will be reported anyways...


-- 
Regards,
Thomas



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] Re: taking a break from arches stabilization

[Kristian Fiskerstrand]

On 07/11/2017 04:21 PM, Michael Palimaka wrote:
> On 07/12/2017 12:15 AM, Kristian Fiskerstrand wrote:
>> On 07/11/2017 04:13 PM, Kristian Fiskerstrand wrote:
>>> On 07/11/2017 03:47 PM, Michael Palimaka wrote:
 The main risk of breakage of a package moving from testing to
 stable is always at build time anyway.
>>>
>>> citation needed
>>>
>>
>> Anecdotal evidence against, currently gnupg 2.1.21 scdaemon bug will
>> happily sign a third party public keyblock's UID using signature subkey
>> on smartcard, which results in useless signature that doesn't have any
>> effect, but the application builds fine.
>>
>> This means gnupg 2.1.21 is not a candidate for stabilization, but it
>> certainly builds fine.
>>
> 
> Stop trolling - you know perfectly well that this sort of issue would
> never ever be caught during arch testing. Nor should it be - it's called
> *arch* testing for a reason.

That presumes that the maintainer is the one calling for the
stabilization, and it is not an automated procedure simply due to 30
days in ~arch. In this particular case, look for the number of bug
reports filed in Gentoo for the issue.

But the main risk is certainly not built testing, it is breaking
operational live stable systems. Nowhere was it claimed that the arch
testers are responsible for it, but it certainly doesn't coincide, at
any point, with "The main risk of breakage of a package moving from
testing to stable is always at build time anyway."

-- 
Kristian Fiskerstrand
OpenPGP keyblock reachable at hkp://pool.sks-keyservers.net
fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] Re: taking a break from arches stabilization

[William Hubbs]

On Tue, Jul 11, 2017 at 11:47:32PM +1000, Michael Palimaka wrote:
> On 07/11/2017 11:06 PM, Rich Freeman wrote:
> > On Tue, Jul 11, 2017 at 8:59 AM, Michael Palimaka  
> > wrote:
> >> On 07/11/2017 09:29 AM, Andrew Savchenko wrote:
> >>>
> >>> Even if such stabilization is allowed, there are unanswered
> >>> questions here:
> >>> - is following seciton 4.1 from wg recommendations is sufficient?
> >>> - should developer test each stabilization candidate on an
> >>> up-to-date stable setup?
> >>
> >> The guidelines from that document are ripped straight out of the
> >> devmanual and are a good starting point but rather generic. You can find
> >> some more detailed suggestions on things to consider while testing on
> >> the wiki: https://wiki.gentoo.org/wiki/Package_testing
> >>
> > 
> > I think that in practice arch teams don't do have the stuff on that
> > wiki page.  Maybe some people do, but back when I was an amd64 AT I
> > don't think anybody went testing multiple USE combinations for a
> > typical package.
> 
> Everything on that page is deliberately a suggestion only, and not
> necessarily specific to stabilisation testing.
> 
> In the end, we've never been able to reach any consensus on what exactly
> an arch tester should do. Personally, I think we should just switch to
> fully-automated, build-only testing for stabilistions unless the
> maintainer opts otherwise (something that largely happens in practice
> already). The main risk of breakage of a package moving from testing to
> stable is always at build time anyway.

I would not be opposed to this. As a maintainer, I am as guilty as the
next guy of not filing stable requests or not stabilizing packages.



signature.asc
Description: Digital signature

Re: [gentoo-dev] Re: taking a break from arches stabilization

[Rich Freeman]

On Tue, Jul 11, 2017 at 10:35 AM, Michael Palimaka
 wrote:
> On 07/12/2017 12:25 AM, James Le Cuirot wrote:
>> On Tue, 11 Jul 2017 16:15:51 +0200
>> Kristian Fiskerstrand  wrote:
>>
>>> On 07/11/2017 04:13 PM, Kristian Fiskerstrand wrote:
 On 07/11/2017 03:47 PM, Michael Palimaka wrote:
> The main risk of breakage of a package moving from testing to
> stable is always at build time anyway.

 citation needed

>>>
>>> Anecdotal evidence against, currently gnupg 2.1.21 scdaemon bug will
>>> happily sign a third party public keyblock's UID using signature
>>> subkey on smartcard, which results in useless signature that doesn't
>>> have any effect, but the application builds fine.
>>>
>>> This means gnupg 2.1.21 is not a candidate for stabilization, but it
>>> certainly builds fine.
>>
>> This is a good opportunity to remind ourselves what stable means. Are
>> we referring exclusively to our packaging or are upstream issues taken
>> into account too? 30 days seems like a reasonable time for any upstream
>> issues to be reported. Unfortunately security issues mean that new
>> releases sometimes get stabilised immediately. Ideally these releases
>> would carry just the security fixes but that isn't always the case.
>>
>
> I think we should consider both our packaging as well as upstream
> issues, and I agree that for most packages 30 days in ~arch is enough
> time to smoke out upstream issues.
>

Agree.  Arch testing is really more of a sanity test.  I think there
is some value in runtime testing, but perhaps it is a bit of a luxury
compared to build testing.

It isn't going to detect obscure bugs.  The only way to do something
like that is to have an extensive testing protocol for each package,
and almost nobody can afford to do that let alone Gentoo.  Upstream
might be able to do it in some cases, though unfortunately not in our
environment.  To the extent that upstream supplies working automated
tests we can try to leverage those.

-- 
Rich

[gentoo-dev] Re: taking a break from arches stabilization

[Michael Palimaka]

On 07/12/2017 12:25 AM, James Le Cuirot wrote:
> On Tue, 11 Jul 2017 16:15:51 +0200
> Kristian Fiskerstrand  wrote:
> 
>> On 07/11/2017 04:13 PM, Kristian Fiskerstrand wrote:
>>> On 07/11/2017 03:47 PM, Michael Palimaka wrote:  
 The main risk of breakage of a package moving from testing to
 stable is always at build time anyway.  
>>>
>>> citation needed
>>>   
>>
>> Anecdotal evidence against, currently gnupg 2.1.21 scdaemon bug will
>> happily sign a third party public keyblock's UID using signature
>> subkey on smartcard, which results in useless signature that doesn't
>> have any effect, but the application builds fine.
>>
>> This means gnupg 2.1.21 is not a candidate for stabilization, but it
>> certainly builds fine.
> 
> This is a good opportunity to remind ourselves what stable means. Are
> we referring exclusively to our packaging or are upstream issues taken
> into account too? 30 days seems like a reasonable time for any upstream
> issues to be reported. Unfortunately security issues mean that new
> releases sometimes get stabilised immediately. Ideally these releases
> would carry just the security fixes but that isn't always the case.
> 

I think we should consider both our packaging as well as upstream
issues, and I agree that for most packages 30 days in ~arch is enough
time to smoke out upstream issues.

Re: [gentoo-dev] Re: taking a break from arches stabilization

[James Le Cuirot]

On Tue, 11 Jul 2017 16:15:51 +0200
Kristian Fiskerstrand  wrote:

> On 07/11/2017 04:13 PM, Kristian Fiskerstrand wrote:
> > On 07/11/2017 03:47 PM, Michael Palimaka wrote:  
> >> The main risk of breakage of a package moving from testing to
> >> stable is always at build time anyway.  
> > 
> > citation needed
> >   
> 
> Anecdotal evidence against, currently gnupg 2.1.21 scdaemon bug will
> happily sign a third party public keyblock's UID using signature
> subkey on smartcard, which results in useless signature that doesn't
> have any effect, but the application builds fine.
> 
> This means gnupg 2.1.21 is not a candidate for stabilization, but it
> certainly builds fine.

This is a good opportunity to remind ourselves what stable means. Are
we referring exclusively to our packaging or are upstream issues taken
into account too? 30 days seems like a reasonable time for any upstream
issues to be reported. Unfortunately security issues mean that new
releases sometimes get stabilised immediately. Ideally these releases
would carry just the security fixes but that isn't always the case.

-- 
James Le Cuirot (chewi)
Gentoo Linux Developer

[gentoo-dev] Re: taking a break from arches stabilization

[Michael Palimaka]

On 07/12/2017 12:15 AM, Kristian Fiskerstrand wrote:
> On 07/11/2017 04:13 PM, Kristian Fiskerstrand wrote:
>> On 07/11/2017 03:47 PM, Michael Palimaka wrote:
>>> The main risk of breakage of a package moving from testing to
>>> stable is always at build time anyway.
>>
>> citation needed
>>
> 
> Anecdotal evidence against, currently gnupg 2.1.21 scdaemon bug will
> happily sign a third party public keyblock's UID using signature subkey
> on smartcard, which results in useless signature that doesn't have any
> effect, but the application builds fine.
> 
> This means gnupg 2.1.21 is not a candidate for stabilization, but it
> certainly builds fine.
> 

Stop trolling - you know perfectly well that this sort of issue would
never ever be caught during arch testing. Nor should it be - it's called
*arch* testing for a reason.

Ensuring that a package's functionality is of merchantable quality is
the maintainer's responsibility (that's you!).

[gentoo-dev] Re: taking a break from arches stabilization

[Michael Palimaka]

On 07/12/2017 12:13 AM, Kristian Fiskerstrand wrote:
> On 07/11/2017 03:47 PM, Michael Palimaka wrote:
>> The main risk of breakage of a package moving from testing to
>> stable is always at build time anyway.
> 
> citation needed
> 

Based on my experience doing package testing in stabilisation work. Feel
free to provide citations (or complete sentences) to the contrary.

Re: [gentoo-dev] Re: taking a break from arches stabilization

[Kristian Fiskerstrand]

On 07/11/2017 04:13 PM, Kristian Fiskerstrand wrote:
> On 07/11/2017 03:47 PM, Michael Palimaka wrote:
>> The main risk of breakage of a package moving from testing to
>> stable is always at build time anyway.
> 
> citation needed
> 

Anecdotal evidence against, currently gnupg 2.1.21 scdaemon bug will
happily sign a third party public keyblock's UID using signature subkey
on smartcard, which results in useless signature that doesn't have any
effect, but the application builds fine.

This means gnupg 2.1.21 is not a candidate for stabilization, but it
certainly builds fine.

-- 
Kristian Fiskerstrand
OpenPGP keyblock reachable at hkp://pool.sks-keyservers.net
fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] Re: taking a break from arches stabilization

[Kristian Fiskerstrand]

On 07/11/2017 03:47 PM, Michael Palimaka wrote:
> The main risk of breakage of a package moving from testing to
> stable is always at build time anyway.

citation needed

-- 
Kristian Fiskerstrand
OpenPGP keyblock reachable at hkp://pool.sks-keyservers.net
fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3



signature.asc
Description: OpenPGP digital signature

[gentoo-dev] Re: taking a break from arches stabilization

[Michael Palimaka]

On 07/11/2017 11:06 PM, Rich Freeman wrote:
> On Tue, Jul 11, 2017 at 8:59 AM, Michael Palimaka  
> wrote:
>> On 07/11/2017 09:29 AM, Andrew Savchenko wrote:
>>>
>>> Even if such stabilization is allowed, there are unanswered
>>> questions here:
>>> - is following seciton 4.1 from wg recommendations is sufficient?
>>> - should developer test each stabilization candidate on an
>>> up-to-date stable setup?
>>
>> The guidelines from that document are ripped straight out of the
>> devmanual and are a good starting point but rather generic. You can find
>> some more detailed suggestions on things to consider while testing on
>> the wiki: https://wiki.gentoo.org/wiki/Package_testing
>>
> 
> I think that in practice arch teams don't do have the stuff on that
> wiki page.  Maybe some people do, but back when I was an amd64 AT I
> don't think anybody went testing multiple USE combinations for a
> typical package.

Everything on that page is deliberately a suggestion only, and not
necessarily specific to stabilisation testing.

In the end, we've never been able to reach any consensus on what exactly
an arch tester should do. Personally, I think we should just switch to
fully-automated, build-only testing for stabilistions unless the
maintainer opts otherwise (something that largely happens in practice
already). The main risk of breakage of a package moving from testing to
stable is always at build time anyway.

Re: [gentoo-dev] Re: taking a break from arches stabilization

[Rich Freeman]

On Tue, Jul 11, 2017 at 8:59 AM, Michael Palimaka  wrote:
> On 07/11/2017 09:29 AM, Andrew Savchenko wrote:
>>
>> Even if such stabilization is allowed, there are unanswered
>> questions here:
>> - is following seciton 4.1 from wg recommendations is sufficient?
>> - should developer test each stabilization candidate on an
>> up-to-date stable setup?
>
> The guidelines from that document are ripped straight out of the
> devmanual and are a good starting point but rather generic. You can find
> some more detailed suggestions on things to consider while testing on
> the wiki: https://wiki.gentoo.org/wiki/Package_testing
>

I think that in practice arch teams don't do have the stuff on that
wiki page.  Maybe some people do, but back when I was an amd64 AT I
don't think anybody went testing multiple USE combinations for a
typical package.

However, to directly answer one of Andrew's questions, yes, you
definitely should test on a stable system/chroot/container.  I've seen
a few package updates over the years that wouldn't even build and it
was probably because whoever keyworded it never even tried building it
with stable dependencies.  That is pretty rare though.

-- 
Rich

[gentoo-dev] Re: taking a break from arches stabilization

[Michael Palimaka]

On 07/11/2017 09:29 AM, Andrew Savchenko wrote:
> On Mon, 10 Jul 2017 22:17:34 +0200 Kristian Fiskerstrand wrote:
>> On 07/10/2017 10:02 PM, Rich Freeman wrote:
>>> On Mon, Jul 10, 2017 at 3:57 PM, Andrew Savchenko  
>>> wrote:

 On Mon, 10 Jul 2017 13:49:40 -0400 Rich Freeman wrote:

> In the case of amd64 we already
> encourage individual package maintainers to stabilize their own
> packages

 Huh? Have our rules changed? As per devmanual[1] and GLEP 40[2]
 stabilization must be carried out by arch teams, unless a special
 arrangement is done between a developer and a team.

>>>
>>> The docs are probably out of date - I'm not sure if the policy is
>>> documented anywhere.  However it has been a fairly longstanding policy
>>> at this point that amd64 allows individual maintainers to stabilize
>>> their own packages.
>>>
>>
>> We looked after it for wg-stable (which died out as a result of rather
>> low participation, maybe it should be rebooted if people feel like
>> discussing this again), there isn't any authoritative policy allowing
>> it, GLEP:40 explicitly removes the possibility to do it for x86. That
>> said, for a number of packages maintainer stabilization can likely make
>> sense, the opposite view is four-eyes principle, it is always good to
>> have someone else build-test etc, but this is greatly helped by
>> tinderboxing efforts (thanks toralf) etc. So one likely output if
>> wg-stable is to come up with something would be a replacement GLEP for
>> 40 that matches the current state, and also kernel auto-stabiliation (as
>> discussed in [section 3.2 (Kernel)]
> 
> So, am I understanding this correctly that right now a package
> stabilization by maintainer without explicit permit from an arch
> team will be the violation of active and approved policies?

As Rich pointed out, amd64 team has long allowed maintainers to perform
their own stabilisations. I've asked x86 team about this in the past,
and they too were OK with maintainer stabilisations.

It would be nice to improve documentation of this, but it is certainly
not a policy violation just because some ancient document was never updated.

> Despite the maintainer-driven stabilization seems to be "a fairly
> longstanding policy" I'm reluctant to do such stabilization myself,
> because anyone may point out later that such action is a violation
> of the written policies and I will have nothing to defend me.
> 
> Even if such stabilization is allowed, there are unanswered
> questions here:
> - is following seciton 4.1 from wg recommendations is sufficient?
> - should developer test each stabilization candidate on an
> up-to-date stable setup?

The guidelines from that document are ripped straight out of the
devmanual and are a good starting point but rather generic. You can find
some more detailed suggestions on things to consider while testing on
the wiki: https://wiki.gentoo.org/wiki/Package_testing