Re: [gentoo-dev] Auto adding packages to world was -> Sets vs Meta ebuilds

[William L. Thomson Jr.]

On Wed, 12 Jul 2017 15:19:32 +1000
"Sam Jorna (wraeth)"  wrote:

> On 12/07/17 15:14, William L. Thomson Jr. wrote:
> > Is it in system?
> > Is it in a set?
> > Is it in world?
> > If no to all, its a dep, warn!  
> 
> All this says is whether the package was explicitly installed and
> recorded in world, or is a member of @system. The target package may
> or may not be a dependency, may or may not have been --oneshot'd.

Then when the user sees the warning they can discard knowing they
merged the package with --oneshot.

What harm does a warning do?

> It may have been installed as a dependency but the requiring package
> was removed,

Then the person failed to run --depclean and maintain their system.
Either way, did the person install the package directly?

If the package was not installed by the user. Should they not be warned
about removing something they did not install?

> or may have been installed as an orphan but is now a
> dependency. 

Now being a dependency the warning would be valid.

>Assuming that if it's not in a set it must be a dependency  is
>incorrect and misleading.

Again there are various ways. There cannot be that much overhead to
check if a single package depends on one being removed. If you cannot
use simpler methods as mentioned.

Clearly people have objections to warnings about packages users did not
install themselves

-- 
William L. Thomson Jr.


pgp9MrrKBm3bB.pgp
Description: OpenPGP digital signature

Re: [gentoo-dev] Auto adding packages to world was -> Sets vs Meta ebuilds

[Sam Jorna (wraeth)]

On 12/07/17 15:14, William L. Thomson Jr. wrote:
> Is it in system?
> Is it in a set?
> Is it in world?
> If no to all, its a dep, warn!

All this says is whether the package was explicitly installed and
recorded in world, or is a member of @system. The target package may or
may not be a dependency, may or may not have been --oneshot'd.

It may have been installed as a dependency but the requiring package was
removed, or may have been installed as an orphan but is now a
dependency. Assuming that if it's not in a set it must be a dependency
is incorrect and misleading.

-- 
Sam Jorna (wraeth)
GnuPG ID: D6180C26



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] Auto adding packages to world was -> Sets vs Meta ebuilds

[William L. Thomson Jr.]

On Wed, 12 Jul 2017 15:00:30 +1000
"Sam Jorna (wraeth)"  wrote:
> 
> My point was that --unmerge is not intended to be dependency-aware.
> --depclean is. As far as I can tell, that is the point others have
> been trying to make as well, when pointing out the differences
> between -c and -C.

Sure but that is easily addressed.

How does emerge know a package is in system profile or a set?
Similar methods or others can be used to determine if a user installed
a package.

In a clean scenario, with a world file that ONLY contains stuff the
user merged directly. Then emerge could simply check that file, against
the stuff it already checks now.

Is it in system?
Is it in a set?
Is it in world?
If no to all, its a dep, warn!

It is really NOT complex, nor does it require complex depgraph or any
of the function of --depclean/-c.

Thus I say once again, mentioning anything to do with depclean is not
relevant and side tracks the discussion. There is no need. If you did
want to actually see if any deps existed. All you need is to see if
there is 1 installed package that needs the one a user is trying to
install. No need for a complete depgrah etc.

There are several ways to go about this that are not complex.

-- 
William L. Thomson Jr.


pgp_HMN_6hxYC.pgp
Description: OpenPGP digital signature

Re: [gentoo-dev] Auto adding packages to world was -> Sets vs Meta ebuilds

[Sam Jorna (wraeth)]

On 12/07/17 14:43, William L. Thomson Jr. wrote:
>>> It is not the same as -C, which is remove a package directly.
>>>
>>>  --unmerge (-C)  
>> Correct, --unmerge will remove a package without considering
>> dependencies (give or take a few special cases). It is usually (or, at
>> least, should generally be) reserved for those taking a hammer to a
>> problem or with a particular desire to recover a broken system.
>>
>> Again, it's doing exactly what it's supposed to - removing a package
>> you've told it to remove (unless it's one of the few
>> almost-always-critical packages).
> Yes and if you see bug. All I am saying is adding warnings when someone
> goes to remove a dependency. Since a dependency IS a critical package :)
> 
> Add warning message when -C/--unmerge a dependency like system,
> profile, and set files.
> https://bugs.gentoo.org/show_bug.cgi?id=624630

My point was that --unmerge is not intended to be dependency-aware.
--depclean is. As far as I can tell, that is the point others have been
trying to make as well, when pointing out the differences between -c and -C.

-- 
Sam Jorna (wraeth)
GnuPG ID: D6180C26



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] Auto adding packages to world was -> Sets vs Meta ebuilds

[William L. Thomson Jr.]

On Wed, 12 Jul 2017 14:17:30 +1000
"Sam Jorna (wraeth)"  wrote:
>
> --depclean is doing exactly what it is supposed to. If called with no

Problem is I was talking about removing packages directly. It served no
purpose in this discussion.

Since I use --depclean, not -c. I was assuming -c was unmerge like -C,
but it is not. Others brought that up and sidetracked things. I just
did not catch and mistakenly went down that wormhole.

> 
> > It is not the same as -C, which is remove a package directly.
> > 
> >  --unmerge (-C)  
> 
> Correct, --unmerge will remove a package without considering
> dependencies (give or take a few special cases). It is usually (or, at
> least, should generally be) reserved for those taking a hammer to a
> problem or with a particular desire to recover a broken system.
> 
> Again, it's doing exactly what it's supposed to - removing a package
> you've told it to remove (unless it's one of the few
> almost-always-critical packages).

Yes and if you see bug. All I am saying is adding warnings when someone
goes to remove a dependency. Since a dependency IS a critical package :)

Add warning message when -C/--unmerge a dependency like system,
profile, and set files.
https://bugs.gentoo.org/show_bug.cgi?id=624630


-- 
William L. Thomson Jr.


pgp5mukQUK6hy.pgp
Description: OpenPGP digital signature

Re: [gentoo-dev] Auto adding packages to world was -> Sets vs Meta ebuilds

[Sam Jorna (wraeth)]

On 12/07/17 10:05, William L. Thomson Jr. wrote:
> I should have caught that sooner. -c does not remove a package, it just
> removes its deps.
> 
> -c == --depclean.

--depclean is doing exactly what it is supposed to. If called with no
arguments, it searches for any unneeded dependencies and removes them,
however if called with a package as an argument, it will remove that
package *if it is not a dependency of another package*. Reporting
"nothing to remove" is precisely what it's supposed to do, and using
--verbose will tell you what is depending on the package.

To be clear, running '--depclean foo' does not remove dependencies of
foo, it removes foo provided it is not a dependency. It can be seen as a
dependency-aware (and thus, generally safe) --unmerge.

Making --depclean _always_ give you more information should just be a
case of adding --verbose to EMERGE_DEFAULT_OPTS.

> It is not the same as -C, which is remove a package directly.
> 
>  --unmerge (-C)

Correct, --unmerge will remove a package without considering
dependencies (give or take a few special cases). It is usually (or, at
least, should generally be) reserved for those taking a hammer to a
problem or with a particular desire to recover a broken system.

Again, it's doing exactly what it's supposed to - removing a package
you've told it to remove (unless it's one of the few
almost-always-critical packages).

-- 
Sam Jorna (wraeth)
GnuPG ID: D6180C26



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] Auto adding packages to world was -> Sets vs Meta ebuilds

[William L. Thomson Jr.]

On Wed, 12 Jul 2017 04:43:41 +0100
"M. J. Everitt"  wrote:
> 
> Of course, you can do what Poettering did, and write your own solution
> .. or fork portage to do things the way -you- want .. but don't
> reinvent the wheel for the rest of us .. that's what Choice and
> Gentoo is all about ...


Its not for me. It is not re-inventing the wheel. It is safe guard
stuff that benefits all. See bugs.

-- 
William L. Thomson Jr.


pgp9uZ61eHSJZ.pgp
Description: OpenPGP digital signature

Re: [gentoo-dev] Auto adding packages to world was -> Sets vs Meta ebuilds

[William L. Thomson Jr.]

On Tue, 11 Jul 2017 23:22:12 -0400
"Walter Dnes"  wrote:
> 
>   Step back for a minute, and relax.  There is a reason you're getting
> blowback.  You're asking for changes that would affect everybody else.
> This is similar in principle to what Lennart Poettering did, and
> you're getting the same reaction he got.  I understand that *YOU*
> want changes to how Portage works on *YOUR* machine.  No problem.  Set
> EMERGE_DEFAULT_OPTS in make.conf.  If you want excruciating detail on
> --depclean then check the small script I posted elsewhere in this
> thread.  Portage has many customization options; use them.

I am relaxed, and if you understand what I am proposing. It will only
help everyone. There is no harm in adding warmings that provide
additional information.

Preventing stuff from being added to world is moot, as that stuff comes
from something else, profile, set, etc. It being added to world serves
no purpose, Just can cause issues down the road. Stuff remaining that
may have not been wanted, but ended up in world so persists and gets
updated, etc.

What purpose does system profile packages saved in world serve?

These changes are NOT for me... I can edit and code myself. This is for
others per this discussion. Others brought up sets accidentally
removing system stuff, etc. Thus these ideas came up as ways to prevent
others from shooting themselves in the foot.

The blowback is mostly because its me, and people are misunderstanding
things. Like the mention of -c/--depclean. Which does not have the same
function as -C/--unmerge. That sidetracked things and added nothing.

>   If you can't be bothered to use available customization options to
> set up your machine to your liking, but ask for a change of defaults
> that also affects everybody else, don't be surprised at the negative
> reaction. You would've gotten a much better reception, if you had
> gone to the gentoo-user list and asked "How do I tweak Portage to do
> this, that, and the other".

How many times do I have to say I use -1, and others like -O.  People
do not pay attention...

Again I do much of this via ansible and profiles. I am not even using a
world file, or sets even. I did use sets before my custom profiles. Did
I always use -1 for the past over a decade no? Should all users have to
in order to prevent needless stuff from being recorded in world.

Please do not assume what I am or am not doing and problems I am not
having. This is stuff for others. I am seeing problems that OTHERS can
run into per the discussion on sets. From things OTHERS mentioned as
issues with using sets.

Bugs are filed.

-- 
William L. Thomson Jr.


pgp9XVKcFMdK4.pgp
Description: OpenPGP digital signature

Re: [gentoo-dev] Auto adding packages to world was -> Sets vs Meta ebuilds

[M. J. Everitt]

On 12/07/17 04:22, Walter Dnes wrote:
>
>   Step back for a minute, and relax.  There is a reason you're getting
> blowback.  You're asking for changes that would affect everybody else.
> This is similar in principle to what Lennart Poettering did, and you're
> getting the same reaction he got.  I understand that *YOU* want changes
> to how Portage works on *YOUR* machine.  No problem.  Set
> EMERGE_DEFAULT_OPTS in make.conf.  If you want excruciating detail on
> --depclean then check the small script I posted elsewhere in this
> thread.  Portage has many customization options; use them.
>
>   If you can't be bothered to use available customization options to set
> up your machine to your liking, but ask for a change of defaults that
> also affects everybody else, don't be surprised at the negative reaction.
> You would've gotten a much better reception, if you had gone to the
> gentoo-user list and asked "How do I tweak Portage to do this, that, and
> the other".
>
+1

Of course, you can do what Poettering did, and write your own solution
.. or fork portage to do things the way -you- want .. but don't reinvent
the wheel for the rest of us .. that's what Choice and Gentoo is all
about ...



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] Auto adding packages to world was -> Sets vs Meta ebuilds

[Walter Dnes]

On Tue, Jul 11, 2017 at 04:57:21PM -0400, William L. Thomson Jr. wrote
> On Tue, 11 Jul 2017 13:27:57 -0700
> Daniel Campbell  wrote:
> 
> > Portage's fault. If you don't want a package added to a set or world,
> > you'll need to use the -1 (--oneshot) option.
> 
> Did you even read above? I clearly state WITHOUT -1 option
> 
> > I added it to my default
> > emerge options in make.conf for exactly that reason (clean world);
> 
> The point is people should not have to do such. Or remember to always
> use -1 when re-emerging a dep, system, world, or set package.

  Step back for a minute, and relax.  There is a reason you're getting
blowback.  You're asking for changes that would affect everybody else.
This is similar in principle to what Lennart Poettering did, and you're
getting the same reaction he got.  I understand that *YOU* want changes
to how Portage works on *YOUR* machine.  No problem.  Set
EMERGE_DEFAULT_OPTS in make.conf.  If you want excruciating detail on
--depclean then check the small script I posted elsewhere in this
thread.  Portage has many customization options; use them.

  If you can't be bothered to use available customization options to set
up your machine to your liking, but ask for a change of defaults that
also affects everybody else, don't be surprised at the negative reaction.
You would've gotten a much better reception, if you had gone to the
gentoo-user list and asked "How do I tweak Portage to do this, that, and
the other".

-- 
Walter Dnes 
I don't run "desktop environments"; I run useful applications

Re: [gentoo-dev] Sets vs Meta ebuilds

[William L. Thomson Jr.]

On Sat, 8 Jul 2017 18:20:54 -0400
"William L. Thomson Jr."  wrote:

> For anyone interested in such, I opened a feature request bug for
> allowing use of sets in profile packages. 
> 
> https://bugs.gentoo.org/show_bug.cgi?id=624300
> 

Subsequent bugs from the discussion

portage should not add system, world, profile, set, or dependent
packages to world, like -1/--oneshot
https://bugs.gentoo.org/show_bug.cgi?id=624628

Add warning message when -C/--unmerge a dependency like system,
profile, and set files.
https://bugs.gentoo.org/show_bug.cgi?id=624630

Thank you for all's input on this topic and related. This should
conclude the thread now, or at least my part. :)

-- 
William L. Thomson Jr.


pgp9kxXmMa3WO.pgp
Description: OpenPGP digital signature

Re: [gentoo-dev] Auto adding packages to world was -> Sets vs Meta ebuilds

[William L. Thomson Jr.]

On Tue, 11 Jul 2017 14:32:27 -0700
Daniel Campbell  wrote:
>
> I understand where you're coming from, I just thought to give a few
> tips to make life a bit easier for you since it works out pretty well
> for myself. I think your idea has merit, just unsure of where the
> functionality goes, since I'm not a Portage developer.

I have been using the -1 option myself for some time. I have also moved
away from having anything in world file. I have my own profiles and
such. Just not committed to my public overlay yet.

This is mostly for others. I do what ever I need directly for myself if
it really becomes an issue for me. But I appreciate the thought!

> I think the hard part of implementing this will be detecting whether a
> command-line-given package is (a) in a set/profile/world and (b) part
> of a dependency tree (i.e. shouldn't be removed).

I do not think it will be that complex. It already does this now for
system, world, and set files. It must be looking at files already.
Having it look at say /var/lib/world is just another file/location.

> -c already traverses the dependency tree. This one message may mean
> iterating through the list of candidate packages and comparing them
> against a set/profile/world *per package*, unless the vdb/cache makes
> this lookup cheap in terms of cycles. The message would be helpful,
> though again, eix-test-obsolete might be the right tool to build that
> feature into. I'd be interested in following the bug discussion, if
> you've already filed it.

The looking at say world file is more an issue for -C than -c. -c knows
there are deps. Thus all it needs is an additional minimal message. It
already says to see deps use -v. It just does not say, why it took no
action. But actually now that I looked at -c, it is completely wrong.

I should have caught that sooner. -c does not remove a package, it just
removes its deps.

-c == --depclean.

It is not the same as -C, which is remove a package directly.

 --unmerge (-C)

Not even sure why anyone suggested -c. That explains why I use -C and
not -c, but I do use --depclean. This whole thread and topic really got
sidetracked big time with -c vs -C.

-c should never be mentioned. I was about to file a bug when I noticed
such.

emerge -c gcc, would never remove gcc, only run depclean
emerge -C will remove gcc

> If you're really interested in the message from -C, it could be done
> by checking the argument list for packages in sets or profiles. But
> it's going to incur similar overhead that -c has because it
> necessarily has to check some sort of list before producing the
> message.

Yes this is just about -C, and as stated previously. Other stuff
already hits files, this is not different really.

> I do not think this message belongs in -C output, however; -C is
> intentionally meant for operations where you don't care what happens
> to the dependency tree

Not true, as I have shown, -C will warn on removing system, world, and
set packages.

-C will NOT worn on dependencies, which it should. Using the world file
and others to know if a package is a dep or in one of those files.

> Please don't interpret this e-mail as aggressive or dismissive. I'm
> simply trying to offer explanation and guidance to help you make this
> happen. It's clear that you care about it, so I'm sure there's a way
> for this to go forward.

I do not, long as it is not insulting which it does not contain
anything of the sort. Though the discussion or mention of -c/--depclean
is really off topic. That confused and side tracked things.

-- 
William L. Thomson Jr.


pgpusktgEpxqE.pgp
Description: OpenPGP digital signature

Re: [gentoo-dev] Re: taking a break from arches stabilization

[Sam Jorna (wraeth)]

On 12/07/17 03:16, William Hubbs wrote:
> On Tue, Jul 11, 2017 at 11:47:32PM +1000, Michael Palimaka wrote:
>> On 07/11/2017 11:06 PM, Rich Freeman wrote:
>>> On Tue, Jul 11, 2017 at 8:59 AM, Michael Palimaka  
>>> wrote:
 On 07/11/2017 09:29 AM, Andrew Savchenko wrote:
>
> Even if such stabilization is allowed, there are unanswered
> questions here:
> - is following seciton 4.1 from wg recommendations is sufficient?
> - should developer test each stabilization candidate on an
> up-to-date stable setup?

 The guidelines from that document are ripped straight out of the
 devmanual and are a good starting point but rather generic. You can find
 some more detailed suggestions on things to consider while testing on
 the wiki: https://wiki.gentoo.org/wiki/Package_testing

>>>
>>> I think that in practice arch teams don't do have the stuff on that
>>> wiki page.  Maybe some people do, but back when I was an amd64 AT I
>>> don't think anybody went testing multiple USE combinations for a
>>> typical package.
>>
>> Everything on that page is deliberately a suggestion only, and not
>> necessarily specific to stabilisation testing.
>>
>> In the end, we've never been able to reach any consensus on what exactly
>> an arch tester should do. Personally, I think we should just switch to
>> fully-automated, build-only testing for stabilistions unless the
>> maintainer opts otherwise (something that largely happens in practice
>> already). The main risk of breakage of a package moving from testing to
>> stable is always at build time anyway.
> 
> I would not be opposed to this. As a maintainer, I am as guilty as the
> next guy of not filing stable requests or not stabilizing packages.

As the next guy, I also +1 this.

As is often explained in #gentoo, "stable" and "testing" for upstream
have a different meaning to "stable" and "testing" for Gentoo - in fact,
beyond ensuring it builds, any testing performed by a downstream distro
is additional testing to what upstream have already released.

I can understand the concern with automatically stabilising a package
that has some flaw affecting users, but the two things i see are that
upstream have already released said flawed package, and Gentoo simply
doesn't have the manpower to perform comprehensive runtime testing of
all packages.

If a maintainer is aware of a significant issue with a package that
should prevent it from being marked as stable, then a bug should be
filed acting as a block to the automated stabilisation. Users aware of
an issue are just as likely to file a bug as well, also preventing
stabilisation of packages with some known issue. Therefore packages with
known issues don't get stabilised automatically.

Similarly, if the maintainer believes more comprehensive testing should
be done (eg. for critical base-system packages) a note could be made
somewhere of that requirement (metadata.xml?), meaning significantly
reduced workload for people like ago (if the maintainer doesn't
stabilise it beforehand).
-- 
Sam Jorna (wraeth)
GnuPG ID: D6180C26



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] Re: taking a break from arches stabilization

[Mart Raudsepp]

Ühel kenal päeval, K, 12.07.2017 kell 00:13, kirjutas Thomas
Deutschmann:
> Let's try Debian's testing
> approach and move packages to ARCH in time and don't wait for some
> magical appearing bug reports because someone really tested a package
> in
> ~ARCH. Severe problems will be reported anyways...

You should realize that this is what was happening.
The problem is, that no-one is doing keywording and stabilization queue
got stalled due to stabilization bugs depending on keywording bugs as
is due process and proper bugzilla sanity.

Re: [gentoo-dev] Re: taking a break from arches stabilization

[Kristian Fiskerstrand]

On 07/12/2017 12:13 AM, Thomas Deutschmann wrote:
> Question is what's more a problem: Having an outdated stable package
> because nobody cared about stabilizing a new version (in most cases this
> will end with a rushed stabilization once a security bug was fixed in
> the package) or move a package in time from ~ARCH to ARCH and deal with
> the fallout sometimes.

Easy, keep the working package any time

-- 
Kristian Fiskerstrand
OpenPGP keyblock reachable at hkp://pool.sks-keyservers.net
fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] Re: taking a break from arches stabilization

[Thomas Deutschmann]

>>> Anecdotal evidence against, currently gnupg 2.1.21 scdaemon bug will
>>> happily sign a third party public keyblock's UID using signature subkey
>>> on smartcard, which results in useless signature that doesn't have any
>>> effect, but the application builds fine.
>>>
>>> This means gnupg 2.1.21 is not a candidate for stabilization, but it
>>> certainly builds fine.
>>>
>>
>> Stop trolling - you know perfectly well that this sort of issue would
>> never ever be caught during arch testing. Nor should it be - it's called
>> *arch* testing for a reason.

Question is what's more a problem: Having an outdated stable package
because nobody cared about stabilizing a new version (in most cases this
will end with a rushed stabilization once a security bug was fixed in
the package) or move a package in time from ~ARCH to ARCH and deal with
the fallout sometimes.

Having a real AT doing real arch testing work would be ideal. But face
it: We don't have the required man power. Let's try Debian's testing
approach and move packages to ARCH in time and don't wait for some
magical appearing bug reports because someone really tested a package in
~ARCH. Severe problems will be reported anyways...


-- 
Regards,
Thomas



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] Auto adding packages to world was -> Sets vs Meta ebuilds

[Daniel Campbell]

On 07/11/2017 01:57 PM, William L. Thomson Jr. wrote:
> On Tue, 11 Jul 2017 13:27:57 -0700
> Daniel Campbell  wrote:
> 
>> On 07/10/2017 04:37 PM, William L. Thomson Jr. wrote:
>>> On Mon, 10 Jul 2017 19:22:47 -0400
>>
>>> A rule for portage could be;
>>>
>>>  - If the package is not in world and already installed. Do not add
>>> the package to world. If you are re-emerging a package already
>>>installed. You do not have to use the -1 option. 
>>>
>>> I have polluted so many world files with system packages and/or
>>> dependencies I re-emerged directly without -1. Those IMHO should
>>> never have been recorded to that file. They were brought in by
>>> other things. Only things in my world should be packages merged
>>> directly, not from profile, set, or a dep.
> 
>> Portage's fault. If you don't want a package added to a set or world,
>> you'll need to use the -1 (--oneshot) option.
> 
> Did you even read above? I clearly state WITHOUT -1 option
> 
>> I added it to my default
>> emerge options in make.conf for exactly that reason (clean world);
> 
> The point is people should not have to do such. Or remember to always
> use -1 when re-emerging a dep, system, world, or set package.
> 
>> though, I have to be careful and make sure packages I care about are
>> in a set somewhere or --depclean will wipe'em out. In short, Portage
>> won't stop you from shooting yourself in the foot.
> 
> If those package are in your world file portage will not remove on
> depclean.
> 
>> If you decide you want to add a package to world without re-merging
>> it, -n (--noreplace) will do the job.
> 
> Or you can add it to the world file, or your profile/packages
> in /etc/portage, etc. There are other places, one does not have to
> emerge every package then want in world. Just the same it should not
> add stuff just the same from system, world,  sets, or deps of any of
> those 3.
> 
>> That said, having helpful messages is a good addition, but needs to be
>> done in a way that is unambiguous and gives the user a clear solution.
> 
> So now it must be clear to the user? That is the entire point I am
> making. The output now is not clear... But in improving such now there
> is concern over something no one cares about now Funny stuff!!!
> 

I understand where you're coming from, I just thought to give a few tips
to make life a bit easier for you since it works out pretty well for
myself. I think your idea has merit, just unsure of where the
functionality goes, since I'm not a Portage developer.

I think the hard part of implementing this will be detecting whether a
command-line-given package is (a) in a set/profile/world and (b) part of
a dependency tree (i.e. shouldn't be removed).

-c already traverses the dependency tree. This one message may mean
iterating through the list of candidate packages and comparing them
against a set/profile/world *per package*, unless the vdb/cache makes
this lookup cheap in terms of cycles. The message would be helpful,
though again, eix-test-obsolete might be the right tool to build that
feature into. I'd be interested in following the bug discussion, if
you've already filed it.

If you're really interested in the message from -C, it could be done by
checking the argument list for packages in sets or profiles. But it's
going to incur similar overhead that -c has because it necessarily has
to check some sort of list before producing the message.

I do not think this message belongs in -C output, however; -C is
intentionally meant for operations where you don't care what happens to
the dependency tree. Sometimes that's good (resolving a blocker the hard
way), sometimes it's bad (removing gcc on a whim). Warnings won't stop a
user doing that, because --unmerge is already documented with the
caveat. There must be a certain point where we as developers have to say
"You're on your own," or there will be so many rails and training wheels
that it loses value for the more experienced users, or a bunch of bugs
filed over failing to heed documentation, i.e. PEBKAC. I think -C is a
great place to draw that line.

The best way to get the ball rolling is file a feature request against
Portage and see what 'upstream' has to say. In the meantime, maybe try
your hand at hacking it. I've found people are a lot more receptive to
suggestions when you've already attempted to provide it. Even if the
solution is crap, people seem willing to help those who have the
gumption to help themselves.

Please don't interpret this e-mail as aggressive or dismissive. I'm
simply trying to offer explanation and guidance to help you make this
happen. It's clear that you care about it, so I'm sure there's a way for
this to go forward.
-- 
Daniel Campbell - Gentoo Developer
OpenPGP Key: 0x1EA055D6 @ hkp://keys.gnupg.net
fpr: AE03 9064 AE00 053C 270C  1DE4 6F7A 9091 1EA0 55D6



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] Re: taking a break from arches stabilization

[Kristian Fiskerstrand]

On 07/11/2017 04:21 PM, Michael Palimaka wrote:
> On 07/12/2017 12:15 AM, Kristian Fiskerstrand wrote:
>> On 07/11/2017 04:13 PM, Kristian Fiskerstrand wrote:
>>> On 07/11/2017 03:47 PM, Michael Palimaka wrote:
 The main risk of breakage of a package moving from testing to
 stable is always at build time anyway.
>>>
>>> citation needed
>>>
>>
>> Anecdotal evidence against, currently gnupg 2.1.21 scdaemon bug will
>> happily sign a third party public keyblock's UID using signature subkey
>> on smartcard, which results in useless signature that doesn't have any
>> effect, but the application builds fine.
>>
>> This means gnupg 2.1.21 is not a candidate for stabilization, but it
>> certainly builds fine.
>>
> 
> Stop trolling - you know perfectly well that this sort of issue would
> never ever be caught during arch testing. Nor should it be - it's called
> *arch* testing for a reason.

That presumes that the maintainer is the one calling for the
stabilization, and it is not an automated procedure simply due to 30
days in ~arch. In this particular case, look for the number of bug
reports filed in Gentoo for the issue.

But the main risk is certainly not built testing, it is breaking
operational live stable systems. Nowhere was it claimed that the arch
testers are responsible for it, but it certainly doesn't coincide, at
any point, with "The main risk of breakage of a package moving from
testing to stable is always at build time anyway."

-- 
Kristian Fiskerstrand
OpenPGP keyblock reachable at hkp://pool.sks-keyservers.net
fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] Auto adding packages to world was -> Sets vs Meta ebuilds

[William L. Thomson Jr.]

On Tue, 11 Jul 2017 13:27:57 -0700
Daniel Campbell  wrote:

> On 07/10/2017 04:37 PM, William L. Thomson Jr. wrote:
> > On Mon, 10 Jul 2017 19:22:47 -0400
>
> > A rule for portage could be;
> > 
> >  - If the package is not in world and already installed. Do not add
> > the package to world. If you are re-emerging a package already
> >installed. You do not have to use the -1 option. 
> > 
> > I have polluted so many world files with system packages and/or
> > dependencies I re-emerged directly without -1. Those IMHO should
> > never have been recorded to that file. They were brought in by
> > other things. Only things in my world should be packages merged
> > directly, not from profile, set, or a dep.

> Portage's fault. If you don't want a package added to a set or world,
> you'll need to use the -1 (--oneshot) option.

Did you even read above? I clearly state WITHOUT -1 option

> I added it to my default
> emerge options in make.conf for exactly that reason (clean world);

The point is people should not have to do such. Or remember to always
use -1 when re-emerging a dep, system, world, or set package.

> though, I have to be careful and make sure packages I care about are
> in a set somewhere or --depclean will wipe'em out. In short, Portage
> won't stop you from shooting yourself in the foot.

If those package are in your world file portage will not remove on
depclean.

> If you decide you want to add a package to world without re-merging
> it, -n (--noreplace) will do the job.

Or you can add it to the world file, or your profile/packages
in /etc/portage, etc. There are other places, one does not have to
emerge every package then want in world. Just the same it should not
add stuff just the same from system, world,  sets, or deps of any of
those 3.

> That said, having helpful messages is a good addition, but needs to be
> done in a way that is unambiguous and gives the user a clear solution.

So now it must be clear to the user? That is the entire point I am
making. The output now is not clear... But in improving such now there
is concern over something no one cares about now Funny stuff!!!

-- 
William L. Thomson Jr.


pgpAbQgpYXXiY.pgp
Description: OpenPGP digital signature

Re: [gentoo-dev] Auto adding packages to world was -> Sets vs Meta ebuilds

[Daniel Campbell]

On 07/11/2017 01:27 PM, Daniel Campbell wrote:
> On 07/10/2017 04:37 PM, William L. Thomson Jr. wrote:
>> On Mon, 10 Jul 2017 19:22:47 -0400
>> "William L. Thomson Jr."  wrote:
>>>
>>> That part does not require it to resolve deps. Just check world file,
>>> assuming its correct. Though could be thrown off if say gcc, or
>>> another was in the world file. I think the profile or set would catch
>>> that as it does now and generate a warning, regardless.
>>
>> Speaking of gcc in the world file. I think portage should STOP adding
>> packages that are in the profile or a dep to world. If you merge a
>> package as part of a set, I am pretty sure it does not get recorded to
>> world, need to confirm, could be wrong.
>>
>> A rule for portage could be;
>>
>>  - If the package is not in world and already installed. Do not add the
>>package to world. If you are re-emerging a package already
>>installed. You do not have to use the -1 option. 
>>
>> I have polluted so many world files with system packages and/or
>> dependencies I re-emerged directly without -1. Those IMHO should never
>> have been recorded to that file. They were brought in by other things.
>> Only things in my world should be packages merged directly, not from
>> profile, set, or a dep.
>>
>> I will file a bug on that as well.
>>
> Whether Portage adds a package to a set or world file is dependent on
> how you invoke emerge. Some people (like me) include sets as part of
> their world, via /var/lib/portage/world_sets . At that point, sets added
> to that file are basically treated as the world package list.
> 
> If gcc or other @system packages end up in your world, it's not
> Portage's fault. If you don't want a package added to a set or world,
> you'll need to use the -1 (--oneshot) option. I added it to my default
> emerge options in make.conf for exactly that reason (clean world);
> though, I have to be careful and make sure packages I care about are in
> a set somewhere or --depclean will wipe'em out. In short, Portage won't
> stop you from shooting yourself in the foot.
> 
> If you decide you want to add a package to world without re-merging it,
> -n (--noreplace) will do the job.
> 
> I'm not sure if eix-test-obsolete (from app-portage/gentoolkit) will
> catch a @system atom inside a set/world file, but that's where I'd
> expect such a notification to come from. The tool can help clean up
> unneeded entries in /etc/portage files, and would be a good fit for this
> particular issue.
> 
> That said, having helpful messages is a good addition, but needs to be
> done in a way that is unambiguous and gives the user a clear solution.
> 
> Hope this helps,
> 
> zlg
> 
Whoops.

s/gentoolkit/eix/

Sorry for the spam.

-- 
Daniel Campbell - Gentoo Developer
OpenPGP Key: 0x1EA055D6 @ hkp://keys.gnupg.net
fpr: AE03 9064 AE00 053C 270C  1DE4 6F7A 9091 1EA0 55D6



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] Auto adding packages to world was -> Sets vs Meta ebuilds

[Daniel Campbell]

On 07/10/2017 04:37 PM, William L. Thomson Jr. wrote:
> On Mon, 10 Jul 2017 19:22:47 -0400
> "William L. Thomson Jr."  wrote:
>>
>> That part does not require it to resolve deps. Just check world file,
>> assuming its correct. Though could be thrown off if say gcc, or
>> another was in the world file. I think the profile or set would catch
>> that as it does now and generate a warning, regardless.
> 
> Speaking of gcc in the world file. I think portage should STOP adding
> packages that are in the profile or a dep to world. If you merge a
> package as part of a set, I am pretty sure it does not get recorded to
> world, need to confirm, could be wrong.
> 
> A rule for portage could be;
> 
>  - If the package is not in world and already installed. Do not add the
>package to world. If you are re-emerging a package already
>installed. You do not have to use the -1 option. 
> 
> I have polluted so many world files with system packages and/or
> dependencies I re-emerged directly without -1. Those IMHO should never
> have been recorded to that file. They were brought in by other things.
> Only things in my world should be packages merged directly, not from
> profile, set, or a dep.
> 
> I will file a bug on that as well.
> 
Whether Portage adds a package to a set or world file is dependent on
how you invoke emerge. Some people (like me) include sets as part of
their world, via /var/lib/portage/world_sets . At that point, sets added
to that file are basically treated as the world package list.

If gcc or other @system packages end up in your world, it's not
Portage's fault. If you don't want a package added to a set or world,
you'll need to use the -1 (--oneshot) option. I added it to my default
emerge options in make.conf for exactly that reason (clean world);
though, I have to be careful and make sure packages I care about are in
a set somewhere or --depclean will wipe'em out. In short, Portage won't
stop you from shooting yourself in the foot.

If you decide you want to add a package to world without re-merging it,
-n (--noreplace) will do the job.

I'm not sure if eix-test-obsolete (from app-portage/gentoolkit) will
catch a @system atom inside a set/world file, but that's where I'd
expect such a notification to come from. The tool can help clean up
unneeded entries in /etc/portage files, and would be a good fit for this
particular issue.

That said, having helpful messages is a good addition, but needs to be
done in a way that is unambiguous and gives the user a clear solution.

Hope this helps,

zlg
-- 
Daniel Campbell - Gentoo Developer
OpenPGP Key: 0x1EA055D6 @ hkp://keys.gnupg.net
fpr: AE03 9064 AE00 053C 270C  1DE4 6F7A 9091 1EA0 55D6



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] taking a break from arches stabilization

[Daniel Campbell]

On 07/10/2017 10:22 AM, Agostino Sarubbo wrote:
> Hi all.
> 
> every time that I attach my tmux session to see what happens on irc, I always 
> see the same discussion about the 'minor' arches status.
> Since I joined gentoo(2011) I worked on all arches except hppa, I put more 
> effort in amd64 and less where I saw other people work on it (arm,alpha)
> But every time the magic phrase is that those arches are unmaintained.
> 
> Now, since I work on these arches just to help, i.e. I don't have any 
> business 
> and I do non have any installation of those arches and the work I'm doing is 
> not appreciated at all I decided to stop for now.
> If your dream is to put sparc and ia64 to ~arch, go ahead, I have no 
> objections.
> I will take a break also from amd64 and x86...let's see how things will 
> change.
> 
Thanks for your efforts in stabilization. I try to make it a point to
thank people like you and Toralf since stabilization and arch testing
are both time-consuming, and probably frustrating to get the tooling
correct.

Take some time off! I'm sure Gentoo won't implode. :)

-- 
Daniel Campbell - Gentoo Developer
OpenPGP Key: 0x1EA055D6 @ hkp://keys.gnupg.net
fpr: AE03 9064 AE00 053C 270C  1DE4 6F7A 9091 1EA0 55D6



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] Re: taking a break from arches stabilization

[William Hubbs]

On Tue, Jul 11, 2017 at 11:47:32PM +1000, Michael Palimaka wrote:
> On 07/11/2017 11:06 PM, Rich Freeman wrote:
> > On Tue, Jul 11, 2017 at 8:59 AM, Michael Palimaka  
> > wrote:
> >> On 07/11/2017 09:29 AM, Andrew Savchenko wrote:
> >>>
> >>> Even if such stabilization is allowed, there are unanswered
> >>> questions here:
> >>> - is following seciton 4.1 from wg recommendations is sufficient?
> >>> - should developer test each stabilization candidate on an
> >>> up-to-date stable setup?
> >>
> >> The guidelines from that document are ripped straight out of the
> >> devmanual and are a good starting point but rather generic. You can find
> >> some more detailed suggestions on things to consider while testing on
> >> the wiki: https://wiki.gentoo.org/wiki/Package_testing
> >>
> > 
> > I think that in practice arch teams don't do have the stuff on that
> > wiki page.  Maybe some people do, but back when I was an amd64 AT I
> > don't think anybody went testing multiple USE combinations for a
> > typical package.
> 
> Everything on that page is deliberately a suggestion only, and not
> necessarily specific to stabilisation testing.
> 
> In the end, we've never been able to reach any consensus on what exactly
> an arch tester should do. Personally, I think we should just switch to
> fully-automated, build-only testing for stabilistions unless the
> maintainer opts otherwise (something that largely happens in practice
> already). The main risk of breakage of a package moving from testing to
> stable is always at build time anyway.

I would not be opposed to this. As a maintainer, I am as guilty as the
next guy of not filing stable requests or not stabilizing packages.



signature.asc
Description: Digital signature

Re: [gentoo-portage-dev] Leader election

[Zac Medico]

On 07/11/2017 09:22 AM, Brian Dolbec wrote:
> On Sun, 2 Jul 2017 11:28:09 -0700
> Brian Dolbec  wrote:
> 
>> -BEGIN PGP SIGNED MESSAGE-
>> Hash: SHA512
> 
>> On Sat, 24 Jun 2017 20:20:16 -0700
>> Brian Dolbec  wrote:
> 
>>> On Sun, 18 Jun 2017 15:48:44 +0200
>>> Alexander Berntsen  wrote:
>>>   
 Friends,

 It's that time of year. We're having a leader election again, as
 well as a general development meeting. The agenda will be updated
 in more detail at:
 https://wiki.gentoo.org/wiki/Project:Portage/Meetings

 Please schedule a time at: http://whenisgood.net/portage

 Thanks.
 - -- 
 Alexander
 berna...@gentoo.org
 https://secure.plaimi.net/~alexander

>>>
>>> I've picked Sunday, July 2 at 4:00 PM UTC
>>>   
> 
>> We've decided (the members in attendance), to do the lead election via
>> email.
> 
>> So, nominations are open from now to July 5, 2017.
> 
>> Voting will be closed July 10, 2017, results posted here again.
> 
> 
> OK, looks the all cast votes were for Zac.
> 
> So, by unanimous decision, I hereby declare Zac as our new project lead.
> 
> Congrats Zac.

Thank you everyone!
-- 
Thanks,
Zac



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-portage-dev] Leader election

[Brian Dolbec]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

On Sun, 2 Jul 2017 11:28:09 -0700
Brian Dolbec  wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA512
> 
> On Sat, 24 Jun 2017 20:20:16 -0700
> Brian Dolbec  wrote:
> 
> > On Sun, 18 Jun 2017 15:48:44 +0200
> > Alexander Berntsen  wrote:
> >   
> > > Friends,
> > > 
> > > It's that time of year. We're having a leader election again, as
> > > well as a general development meeting. The agenda will be updated
> > > in more detail at:
> > > https://wiki.gentoo.org/wiki/Project:Portage/Meetings
> > > 
> > > Please schedule a time at: http://whenisgood.net/portage
> > > 
> > > Thanks.
> > > - -- 
> > > Alexander
> > > berna...@gentoo.org
> > > https://secure.plaimi.net/~alexander
> > >
> > 
> > I've picked Sunday, July 2 at 4:00 PM UTC
> >   
> 
> We've decided (the members in attendance), to do the lead election via
> email.
> 
> So, nominations are open from now to July 5, 2017.
> 
> Voting will be closed July 10, 2017, results posted here again.
> 

OK, looks the all cast votes were for Zac.

So, by unanimous decision, I hereby declare Zac as our new project lead.

Congrats Zac.

- -- 
Brian Dolbec 

-BEGIN PGP SIGNATURE-
Version: GnuPG v2

iQKTBAEBCgB9FiEEpdfHTggcxw20pKr1+70IcnWCDtgFAllk+z1fFIAALgAo
aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldEE1
RDdDNzRFMDgxQ0M3MERCNEE0QUFGNUZCQkQwODcyNzU4MjBFRDgACgkQ+70IcnWC
Dtj0Ew//Xl+GzPKM3+0EKKMJn+Q6G2/7z5byPFl6bPevYFTPlmmN7a+AxULw+NfB
7+EwqYdVzLk51GLzBk4Y3Pgt7gFEPbigBB/w2Fubb24QvNzwUiZ3O6N8F4o7BbVh
8T0yuRYpFn79mWIOACudEgCt9HfSSAYcCREjIWyqrXSxuqCs4SKGRhReKx8S2XGt
eUjD67HCCUKGmnVIZJrOFiQomDegLxl/gUqB2kHvzZWnpRVfYGHJbLXGCQtR8fro
Qm2qvnnIbyxtYf3xaCKLPTCMqYwOtQuEL2xwuJ/KVvIcRErxJIYgutksjJIodEWj
vE18hE7RzVsIyiIB/wvlJk2+ViTRSwlkRY2af6Sq0QakPlI+OtrKgqxPNQE041hl
kMWogpawZm3z5hCwh9TuGW9ZCbI3ego6woTYp9DE9B953UiYwg0MbjBnh4AFeE0O
Qy23zybjrD6ANzc1328W3hZpU5jjWnWEAvOlaH/pGH7vYBgdMfPcs0vyijkKE6gg
JOwrFGpIAqcFBkIfa6cUgc2OQX3yBKZJgrqfFHniRo0KGXt3Jl9e4+eG0EK6CcqZ
eVSAjflo+5mCVjaVGugXrcTd3O6/SVzsUH+/woazyLKogc1x+4rzk0dVmfDBPuow
zFEbdcPzlJW6wxpPhCjHz1GFD/MzZsEvAxGW+q+GF71GLlJXeKI=
=hDeZ
-END PGP SIGNATURE-

Re: [gentoo-portage-dev] Re: [gentoo-dev] taking a break from arches stabilization

[Brian Dolbec]

On Mon, 10 Jul 2017 20:20:12 +0200
Alexander Berntsen  wrote:

> On 10/07/17 20:11, Jonas Stein wrote:
> > It would be so motivating to see that many user are glad about a
> > special package. One gets rarely feedback.  
> 
> Interesting idea. We could have some (separate) portage-y helper tool
> send a standardised email that could easily be filtered based on
> sender and/or topic. 'emerge foo --thank' just thanks people, 'emerge
> foo --thank="msg"' thanks people with a msg.
> 
> It's gimmicky and could be abused. But then again perhaps it wouldn't
> be abused, and it would just be wholesome fun. Maybe someone would
> meet their future lover via the thank-parametre.
> 
> What do you think?


We had a past GSOC project that ended up with a decent start to a stats
system.  I know I had hoped for (and suggested) a survey system for it.
A maintainer could put up a stabilization survey, or some other query
about a pkg/version.  The people interested could fill it out, the
maintainer could get feedback.  It would also give estimates of the
number of people using the pkg/version (dpends on the number of people
with stats enabled).

There was even a dev that recently tried to get it deployed (again).  I
volunteered to help.  But it never got the vm from it from infra.
Infra is not set up to allow root access to a system for only certain
systems/devs.  So in order to maintain a system, you primarily have to
be in the infra team.  So, as a result, progress stalls, dies out.

-- 
Brian Dolbec 



pgpVROTFPVD5Q.pgp
Description: OpenPGP digital signature

Re: [gentoo-dev] Re: taking a break from arches stabilization

[Rich Freeman]

On Tue, Jul 11, 2017 at 10:35 AM, Michael Palimaka
 wrote:
> On 07/12/2017 12:25 AM, James Le Cuirot wrote:
>> On Tue, 11 Jul 2017 16:15:51 +0200
>> Kristian Fiskerstrand  wrote:
>>
>>> On 07/11/2017 04:13 PM, Kristian Fiskerstrand wrote:
 On 07/11/2017 03:47 PM, Michael Palimaka wrote:
> The main risk of breakage of a package moving from testing to
> stable is always at build time anyway.

 citation needed

>>>
>>> Anecdotal evidence against, currently gnupg 2.1.21 scdaemon bug will
>>> happily sign a third party public keyblock's UID using signature
>>> subkey on smartcard, which results in useless signature that doesn't
>>> have any effect, but the application builds fine.
>>>
>>> This means gnupg 2.1.21 is not a candidate for stabilization, but it
>>> certainly builds fine.
>>
>> This is a good opportunity to remind ourselves what stable means. Are
>> we referring exclusively to our packaging or are upstream issues taken
>> into account too? 30 days seems like a reasonable time for any upstream
>> issues to be reported. Unfortunately security issues mean that new
>> releases sometimes get stabilised immediately. Ideally these releases
>> would carry just the security fixes but that isn't always the case.
>>
>
> I think we should consider both our packaging as well as upstream
> issues, and I agree that for most packages 30 days in ~arch is enough
> time to smoke out upstream issues.
>

Agree.  Arch testing is really more of a sanity test.  I think there
is some value in runtime testing, but perhaps it is a bit of a luxury
compared to build testing.

It isn't going to detect obscure bugs.  The only way to do something
like that is to have an extensive testing protocol for each package,
and almost nobody can afford to do that let alone Gentoo.  Upstream
might be able to do it in some cases, though unfortunately not in our
environment.  To the extent that upstream supplies working automated
tests we can try to leverage those.

-- 
Rich

[gentoo-dev] Re: taking a break from arches stabilization

[Michael Palimaka]

On 07/12/2017 12:25 AM, James Le Cuirot wrote:
> On Tue, 11 Jul 2017 16:15:51 +0200
> Kristian Fiskerstrand  wrote:
> 
>> On 07/11/2017 04:13 PM, Kristian Fiskerstrand wrote:
>>> On 07/11/2017 03:47 PM, Michael Palimaka wrote:  
 The main risk of breakage of a package moving from testing to
 stable is always at build time anyway.  
>>>
>>> citation needed
>>>   
>>
>> Anecdotal evidence against, currently gnupg 2.1.21 scdaemon bug will
>> happily sign a third party public keyblock's UID using signature
>> subkey on smartcard, which results in useless signature that doesn't
>> have any effect, but the application builds fine.
>>
>> This means gnupg 2.1.21 is not a candidate for stabilization, but it
>> certainly builds fine.
> 
> This is a good opportunity to remind ourselves what stable means. Are
> we referring exclusively to our packaging or are upstream issues taken
> into account too? 30 days seems like a reasonable time for any upstream
> issues to be reported. Unfortunately security issues mean that new
> releases sometimes get stabilised immediately. Ideally these releases
> would carry just the security fixes but that isn't always the case.
> 

I think we should consider both our packaging as well as upstream
issues, and I agree that for most packages 30 days in ~arch is enough
time to smoke out upstream issues.

Re: [gentoo-dev] Re: taking a break from arches stabilization

[James Le Cuirot]

On Tue, 11 Jul 2017 16:15:51 +0200
Kristian Fiskerstrand  wrote:

> On 07/11/2017 04:13 PM, Kristian Fiskerstrand wrote:
> > On 07/11/2017 03:47 PM, Michael Palimaka wrote:  
> >> The main risk of breakage of a package moving from testing to
> >> stable is always at build time anyway.  
> > 
> > citation needed
> >   
> 
> Anecdotal evidence against, currently gnupg 2.1.21 scdaemon bug will
> happily sign a third party public keyblock's UID using signature
> subkey on smartcard, which results in useless signature that doesn't
> have any effect, but the application builds fine.
> 
> This means gnupg 2.1.21 is not a candidate for stabilization, but it
> certainly builds fine.

This is a good opportunity to remind ourselves what stable means. Are
we referring exclusively to our packaging or are upstream issues taken
into account too? 30 days seems like a reasonable time for any upstream
issues to be reported. Unfortunately security issues mean that new
releases sometimes get stabilised immediately. Ideally these releases
would carry just the security fixes but that isn't always the case.

-- 
James Le Cuirot (chewi)
Gentoo Linux Developer

[gentoo-dev] Re: taking a break from arches stabilization

[Michael Palimaka]

On 07/12/2017 12:15 AM, Kristian Fiskerstrand wrote:
> On 07/11/2017 04:13 PM, Kristian Fiskerstrand wrote:
>> On 07/11/2017 03:47 PM, Michael Palimaka wrote:
>>> The main risk of breakage of a package moving from testing to
>>> stable is always at build time anyway.
>>
>> citation needed
>>
> 
> Anecdotal evidence against, currently gnupg 2.1.21 scdaemon bug will
> happily sign a third party public keyblock's UID using signature subkey
> on smartcard, which results in useless signature that doesn't have any
> effect, but the application builds fine.
> 
> This means gnupg 2.1.21 is not a candidate for stabilization, but it
> certainly builds fine.
> 

Stop trolling - you know perfectly well that this sort of issue would
never ever be caught during arch testing. Nor should it be - it's called
*arch* testing for a reason.

Ensuring that a package's functionality is of merchantable quality is
the maintainer's responsibility (that's you!).

[gentoo-dev] Re: taking a break from arches stabilization

[Michael Palimaka]

On 07/12/2017 12:13 AM, Kristian Fiskerstrand wrote:
> On 07/11/2017 03:47 PM, Michael Palimaka wrote:
>> The main risk of breakage of a package moving from testing to
>> stable is always at build time anyway.
> 
> citation needed
> 

Based on my experience doing package testing in stabilisation work. Feel
free to provide citations (or complete sentences) to the contrary.

Re: [gentoo-dev] Re: taking a break from arches stabilization

[Kristian Fiskerstrand]

On 07/11/2017 04:13 PM, Kristian Fiskerstrand wrote:
> On 07/11/2017 03:47 PM, Michael Palimaka wrote:
>> The main risk of breakage of a package moving from testing to
>> stable is always at build time anyway.
> 
> citation needed
> 

Anecdotal evidence against, currently gnupg 2.1.21 scdaemon bug will
happily sign a third party public keyblock's UID using signature subkey
on smartcard, which results in useless signature that doesn't have any
effect, but the application builds fine.

This means gnupg 2.1.21 is not a candidate for stabilization, but it
certainly builds fine.

-- 
Kristian Fiskerstrand
OpenPGP keyblock reachable at hkp://pool.sks-keyservers.net
fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-dev] Re: taking a break from arches stabilization

[Kristian Fiskerstrand]

On 07/11/2017 03:47 PM, Michael Palimaka wrote:
> The main risk of breakage of a package moving from testing to
> stable is always at build time anyway.

citation needed

-- 
Kristian Fiskerstrand
OpenPGP keyblock reachable at hkp://pool.sks-keyservers.net
fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3



signature.asc
Description: OpenPGP digital signature

[gentoo-dev] Re: taking a break from arches stabilization

[Michael Palimaka]

On 07/11/2017 11:06 PM, Rich Freeman wrote:
> On Tue, Jul 11, 2017 at 8:59 AM, Michael Palimaka  
> wrote:
>> On 07/11/2017 09:29 AM, Andrew Savchenko wrote:
>>>
>>> Even if such stabilization is allowed, there are unanswered
>>> questions here:
>>> - is following seciton 4.1 from wg recommendations is sufficient?
>>> - should developer test each stabilization candidate on an
>>> up-to-date stable setup?
>>
>> The guidelines from that document are ripped straight out of the
>> devmanual and are a good starting point but rather generic. You can find
>> some more detailed suggestions on things to consider while testing on
>> the wiki: https://wiki.gentoo.org/wiki/Package_testing
>>
> 
> I think that in practice arch teams don't do have the stuff on that
> wiki page.  Maybe some people do, but back when I was an amd64 AT I
> don't think anybody went testing multiple USE combinations for a
> typical package.

Everything on that page is deliberately a suggestion only, and not
necessarily specific to stabilisation testing.

In the end, we've never been able to reach any consensus on what exactly
an arch tester should do. Personally, I think we should just switch to
fully-automated, build-only testing for stabilistions unless the
maintainer opts otherwise (something that largely happens in practice
already). The main risk of breakage of a package moving from testing to
stable is always at build time anyway.

Re: [gentoo-dev] package up for grabs: mail-filter/spambayes

[Thomas - LordVan - Raschbacher]

Am 11.07.2017 um 15:21 schrieb Igor Savlook:
> On Tue, 2017-07-11 at 07:28 +0200, Thomas - LordVan - Raschbacher
> wrote:
>> Hi.
>>
>> I decided to either give up maintainership of spambayes (not that
>> there
>> were any releases in the last 5+ years anyway).
>>
>> Anyone interested? if not I'll change it to maintainer-needed (or
>> maybe
>> just treeclean it .. not sure yet if anyone at all still cares - as
>> far
>> as I could see no packages depend on it in the tree)
>>
>> Regards
>> P.S.: if you're interested at least CC me directly please
>>
> 
> Seems project dead. Not developed from 2008 year.
> 
yes, but it seems to still merge fine and the page is still up so I am
not sure if anyone (who still uses it) might want to take it..

-- 
Thomas Raschbacher
Gentoo Developer
https://keybase.io/lordvan

Re: [gentoo-dev] package up for grabs: mail-filter/spambayes

[Igor Savlook]

On Tue, 2017-07-11 at 07:28 +0200, Thomas - LordVan - Raschbacher
wrote:
> Hi.
> 
> I decided to either give up maintainership of spambayes (not that
> there
> were any releases in the last 5+ years anyway).
> 
> Anyone interested? if not I'll change it to maintainer-needed (or
> maybe
> just treeclean it .. not sure yet if anyone at all still cares - as
> far
> as I could see no packages depend on it in the tree)
> 
> Regards
> P.S.: if you're interested at least CC me directly please
> 

Seems project dead. Not developed from 2008 year.

Re: [gentoo-dev] Re: taking a break from arches stabilization

[Rich Freeman]

On Tue, Jul 11, 2017 at 8:59 AM, Michael Palimaka  wrote:
> On 07/11/2017 09:29 AM, Andrew Savchenko wrote:
>>
>> Even if such stabilization is allowed, there are unanswered
>> questions here:
>> - is following seciton 4.1 from wg recommendations is sufficient?
>> - should developer test each stabilization candidate on an
>> up-to-date stable setup?
>
> The guidelines from that document are ripped straight out of the
> devmanual and are a good starting point but rather generic. You can find
> some more detailed suggestions on things to consider while testing on
> the wiki: https://wiki.gentoo.org/wiki/Package_testing
>

I think that in practice arch teams don't do have the stuff on that
wiki page.  Maybe some people do, but back when I was an amd64 AT I
don't think anybody went testing multiple USE combinations for a
typical package.

However, to directly answer one of Andrew's questions, yes, you
definitely should test on a stable system/chroot/container.  I've seen
a few package updates over the years that wouldn't even build and it
was probably because whoever keyworded it never even tried building it
with stable dependencies.  That is pretty rare though.

-- 
Rich

[gentoo-dev] Re: taking a break from arches stabilization

[Michael Palimaka]

On 07/11/2017 09:29 AM, Andrew Savchenko wrote:
> On Mon, 10 Jul 2017 22:17:34 +0200 Kristian Fiskerstrand wrote:
>> On 07/10/2017 10:02 PM, Rich Freeman wrote:
>>> On Mon, Jul 10, 2017 at 3:57 PM, Andrew Savchenko  
>>> wrote:

 On Mon, 10 Jul 2017 13:49:40 -0400 Rich Freeman wrote:

> In the case of amd64 we already
> encourage individual package maintainers to stabilize their own
> packages

 Huh? Have our rules changed? As per devmanual[1] and GLEP 40[2]
 stabilization must be carried out by arch teams, unless a special
 arrangement is done between a developer and a team.

>>>
>>> The docs are probably out of date - I'm not sure if the policy is
>>> documented anywhere.  However it has been a fairly longstanding policy
>>> at this point that amd64 allows individual maintainers to stabilize
>>> their own packages.
>>>
>>
>> We looked after it for wg-stable (which died out as a result of rather
>> low participation, maybe it should be rebooted if people feel like
>> discussing this again), there isn't any authoritative policy allowing
>> it, GLEP:40 explicitly removes the possibility to do it for x86. That
>> said, for a number of packages maintainer stabilization can likely make
>> sense, the opposite view is four-eyes principle, it is always good to
>> have someone else build-test etc, but this is greatly helped by
>> tinderboxing efforts (thanks toralf) etc. So one likely output if
>> wg-stable is to come up with something would be a replacement GLEP for
>> 40 that matches the current state, and also kernel auto-stabiliation (as
>> discussed in [section 3.2 (Kernel)]
> 
> So, am I understanding this correctly that right now a package
> stabilization by maintainer without explicit permit from an arch
> team will be the violation of active and approved policies?

As Rich pointed out, amd64 team has long allowed maintainers to perform
their own stabilisations. I've asked x86 team about this in the past,
and they too were OK with maintainer stabilisations.

It would be nice to improve documentation of this, but it is certainly
not a policy violation just because some ancient document was never updated.

> Despite the maintainer-driven stabilization seems to be "a fairly
> longstanding policy" I'm reluctant to do such stabilization myself,
> because anyone may point out later that such action is a violation
> of the written policies and I will have nothing to defend me.
> 
> Even if such stabilization is allowed, there are unanswered
> questions here:
> - is following seciton 4.1 from wg recommendations is sufficient?
> - should developer test each stabilization candidate on an
> up-to-date stable setup?

The guidelines from that document are ripped straight out of the
devmanual and are a good starting point but rather generic. You can find
some more detailed suggestions on things to consider while testing on
the wiki: https://wiki.gentoo.org/wiki/Package_testing

Re: [gentoo-dev] taking a break from arches stabilization

[Rich Freeman]

On Mon, Jul 10, 2017 at 7:54 PM, Andrew Savchenko  wrote:
> On Mon, 10 Jul 2017 16:27:54 -0400 Rich Freeman wrote:
>> On Mon, Jul 10, 2017 at 4:05 PM, M. J. Everitt  wrote:
>> > This is why stabilisation, if not for individual package maintainers on
>> > amd64, has become a joke, save for Ago's efforts, and recent efforts by
>> > kensington to streamline the effort for the likes of ago with his bot,
>> > and one or two other arch stabilisers (who I know exist, but not by name
>> > or nick).
>>
>> Sure.  If nobody is maintaining stable keywords on an arch, then there
>> shouldn't be stable keywords on that arch, unless the stable keywords
>> are used for a different purpose and maintainers are free to downgrade
>> them at any time.
>
> I'm confused, again. I can't find any official policy regarding
> dekeywording packages from stable to testing.

I tried to find the best documented answers I could to your questions,
but as has been mentioned our documentation around what really goes on
with arch keywording today is poor.  This was one of the drivers for
the stable-wg.

>
> Can developers remove packages from stable on whim or only on
> certain conditions? Under what conditions exactly? Should arch
> teams be notified on such actions? Or even requested permissions
> from?

"If a maintainer has an open STABLEREQ, or a KEYWORDREQ blocking a
pending STABLEREQ, for 90 days with archs CCed and otherwise ready
to be stabilized, the maintainer can remove older versions of
the package at their discretion. A package is considered ready to be
stabilized if it has been in the tree for 30 days, and has no known
major flaws on arches that upstream considers supported."  [1]

Note that this only pertains to stable arches.  For a non-stable arch
there are no restrictions on the removal of the last stable version as
far as I'm aware.

>
> IMO a valid reason to remove from stable is arch team failure to
> stabilize this package for a long time. But how long? Month, two,
> half a year?

90 days, per the above.

>
> What to do with reverse dependencies? Should they be dropped to
> ~arch altogether with the package in question? Or should their
> maintainers be given a warning before? How long to wait after?
> One more month or two, another half a year?

They need to be dropped to ~arch as well, at the same time the stable
version is removed.  In practice this is painful and this was one of
the drivers for the stable-wg.  The 90 day relief the Council provided
did not completely solve this problem.

>
> A well established arch -> ~arch policy should help to keep number
> of stable packages sane and manageable for arch teams. A well
> established policy of doing ~arch -> arch by devs themselves will
> help to lower load on arch teams as well. So for everyone be happy
> (arch teams by keeping them stable and manageable, devs by solving
> stabilization requests in a sane time) we need good policies!
>

A big problem historically is that it is much easier to go from
~arch->arch than it is to go from arch->~arch.  I think well-meaning
users of minor arches enthusiastically deal with STABLEREQ bugs so
that there are more stable packages available on the arch.  However,
once whatever itch they had was scratched they don't necessarily keep
up with future requests on the same package, causing the package to
become stale.

The intent of the 90 day policy was to make it easier to drop keywords
back.  However, without scripts/etc to make this simple for
maintainers most don't actually take advantage of this opportunity.

It has also been rightly pointed out that this is a bit of a chaotic
solution to the problem.  If the minor arch doesn't keep up with
stable keywords on a core dependency somebody running one of these
scripts might remove almost all the stable keywords for that arch in
the entire tree.  The counter argument is that these core dependencies
are the ones the arch team should be paying the most attention to, and
if they spent more time stabilizing boring libraries and less time
stabilizing applications maybe the problem wouldn't exist.

GLEP 40 has been mentioned and this is an interesting policy and the
way it is being mentioned makes it moreso.  At the time it was written
everything contained within was already being done on all the arches
EXCEPT x86, where maintainers were doing their own stabilizations.
Since the amd64 team was viewed as a model of best practice at the
time the purpose of the GLEP was to make x86 the same as the other
arches.  Since then manpower and interests have changed, and amd64
initially just gave individual maintainers a go-ahead to do their own
stabilizations and then eventually a blanket authorization for anybody
to do so.  Ironically this makes x86 the only arch with a documented
arch testing policy in a GLEP.

I can't find any documentation of the amd64 exception.  I think it
goes all the way back to kingtaco.


1 - 

Re: [gentoo-dev] taking a break from arches stabilization

[Agostino Sarubbo]

Thanks all for the 'appreciation'.

I'd like to remember that I'm not going away nor I'm retiring, I will just 
avoid to touch stabilizations, unless the stable package is part of my 
interest.

I'd like also to reminder that in the past I monitored the bugs via the 
bugzilla UI, and in the recent past I used the getatoms script to monitor the 
load for each arch:

for ARCH in alpha amd64 arm ia64 ppc ppc64 sparc x86
do
echo "${ARCH}"

python \
/root/getatoms.py \
-a "${ARCH}" \
--stablereq \
--no-depends \
--all-bugs > /dev/null 2>&1

grep "=" /etc/portage/package.keywords/test | wc -l
echo -ne "\n\n"
done


Actually we have a result like this:

alpha   
 
82  
 



  
amd64   
 
99  
 



  
arm 
 
154 
 



  
ia64
 
5   
 



  
ppc 
 
64  
 



  
ppc64   
 
64  
 



  
sparc   
 
20  
 



  
x86 

Re: [gentoo-dev] About adding a *warning* to remind maintainers to check for new PYTHON_COMPAT values

[Pacho Ramos]

El lun, 10-07-2017 a las 11:55 -0500, William Hubbs escribió:
> On Mon, Jul 10, 2017 at 01:04:10PM +0200, Pacho Ramos wrote:
> > Hello
> > 
> > Looking to the list of packages still not supporting python 3.5:
> > https://qa-reports.gentoo.org/output/gpyutils/34-to-35.txt
> > 
> > and considering that we should even start testing python 3.6, I think it
> > would
> > be nice if we could make portage to warn when PYTHON_COMPAT value is not
> > updated. It's really frustrating to still see new ebuilds being added with
> > obsolete values for PYTHON_COMPAT and relying on a few people looking to
> > update
> > this. This is also causing huge delays to migrate to newer python versions
> > and I
> > think it's responsibility of the maintainer to ensure his/her package is
> > supported on newer versions or, at least, have a bug and ping upstream for
> > the
> > cases they need further fixing.
> > 
> > Of course, this wouldn't be a fatal check preventing you from committing a
> > package with outdated PYTHON_COMPAT, it would be a warning to remind you to
> > update it as soon as possible.
> > 
> > Any issues on trying to go further into implementing this warning? 
> 
> What about the situation where a package is not compatible with newer
> versions of python so does not need a PYTHON_COMPAT change?
> 
> I don't think you can assume PYTHON_COMPAT is outdated for a package
> just because it doesn't have the latest versions of python
> listed. The only time you can know for sure that it is outdated is if it
> lists a version of python that no longer exists in the tree.
> 
> William
> 

Maybe we could handle it like other warnings: ignore it but ensure we have a bug
report to track the issue (I have seen cases of a comment being added showing
that a package wasn't ready for python 3.5 but no bug reference listed :/). 

It looks to me like the same we do for parallel build warnings

Re: [gentoo-dev] Sets vs Meta ebuilds

[Ulrich Mueller]

> On Mon, 10 Jul 2017, William L Thomson wrote:

> Stop getting lost in the weeds
> You all are making this about -c vs -C. I am not talking about that!

> LET ME CLARIFY

> [...] SHOULD [...] PERIOD. NOTHING [...]

> So PLEASE stop with that!

Right. Please stop shouting in the gentoo-dev mailing list. Thank you.

Ulrich


pgpasY5wKD6yJ.pgp
Description: PGP signature

Re: [gentoo-dev] taking a break from arches stabilization

[Lars Wendler]

On Mon, 10 Jul 2017 19:22:20 +0200 Agostino Sarubbo wrote:

>Hi all.
>
>every time that I attach my tmux session to see what happens on irc, I
>always see the same discussion about the 'minor' arches status.
>Since I joined gentoo(2011) I worked on all arches except hppa, I put
>more effort in amd64 and less where I saw other people work on it
>(arm,alpha) But every time the magic phrase is that those arches are
>unmaintained.
>
>Now, since I work on these arches just to help, i.e. I don't have any
>business and I do non have any installation of those arches and the
>work I'm doing is not appreciated at all I decided to stop for now.
>If your dream is to put sparc and ia64 to ~arch, go ahead, I have no 
>objections.
>I will take a break also from amd64 and x86...let's see how things
>will change.
>

Hi Agostino,

this is very sad news but I'm not really surprised about your
decision. It seems like every arch except amd64 and to some point
arm(64) seems to be the target of massive vilification.

On behalf of base-system team let me thank you for your awesome
stabilization work. I hope you will return at some point when your
frustration level has sufficiently decreased.

Kind regards
Lars

-- 
Lars Wendler
Gentoo package maintainer
GPG: 21CC CF02 4586 0A07 ED93  9F68 498F E765 960E 9B39


pgpatNBwwwG4t.pgp
Description: Digitale Signatur von OpenPGP