Re: [gentoo-dev] Current unavoidable use of xz utils in Gentoo

[Rich Freeman]

On Sat, Mar 30, 2024 at 10:57 AM Eddie Chapman  wrote:
>
> No, this is the the bad actor *themselves* being a
> principal author of the software, working stealthily and in very
> sophisticated ways for years, to manoeuvrer themselves and their software
> into a position of trust in the ecosystem whereby they were almost able to
> pull off the mother of all security nightmares for the world.

This is entirely speculative at this point.  It isn't certain that the
author is the one behind the exploit, and if they were, it is not
known for how long their intentions were malicious, or even what their
motivations were.  It is also unclear what pseudonymous accounts with
what projects are associated with the attacker.

You could end up being right, but it probably makes sense to at least
give things a few days for more facts to become available, before
making decisions to retool the entire distro.

I think the bigger challenge is what could have been done to prevent
this sort of problem in the first place.  There are so many projects
that end up with code running as root that have one or two people
taking care of them, and if somebody does the work to become one of
those maintainers, there aren't many people looking out for problems.

I think one thing that would help here is for distros to have better
ways to ensure that the code in the scm matches the code in the
tarball.  It is pretty common for releases to be manipulated in some
way (even if only to gpg sign them, but often to switch from commit
IDs to version numbers and so on), and that can be a place where stuff
gets added.  That still says nothing about obfuscated code, which this
also involved.

-- 
Rich

Re: [gentoo-dev] Current unavoidable use of xz utils in Gentoo

[Rich Freeman]

On Sat, Mar 30, 2024 at 3:06 AM Dale  wrote:
>
> when I got to the part about it not likely to affect Gentoo, my level of 
> concern dropped significantly.  If this is still true, there's no need to be 
> concerned.

"not likely" is the best way to characterize this.  The exploit has
not been fully analyzed, and it could have additional malicious
behavior, either designed by its author, or perhaps even unintended by
its author.

I just wanted to toss in that caveat, but agree that the defaults
deployed in Gentoo seem the most sensible for general use.  There is
nothing magical about xz - ANY widely-used library could have
something like this embedded in it, and the attacker exploited what
they had access to in order to go after a configuration that was going
to be widely deployed and reachable (xz+deb/rpm+systemd+openssh).  If
the attacker had an intended target that used gentoo+openrc and access
to something in our supply chain, this could have been a vulnerability
that only impacted Gentoo.

I think the big lesson here is that FOSS continues to suffer from core
dependencies that are challenged for resources, and that efforts to
fix this have to constantly keep up with the changing landscape.  xz
is going on 15 years old, but I don't think it was nearly as commonly
used until fairly recently.

libz has been a pretty well-known source of security flaws for ages
(granted, usually not intentional like this).  It isn't too surprising
that in both cases compression libraries were targeted, as these are
so widely depended on.

This is getting tangential, but part of me wonders if there is a
better way to do authentication.  Programs like ssh tend to run as
root so that they can authenticate users and then fork and suid to the
appropriate user.  Could some OS-level facility be created to allow
unprivileged processes to run the daemons and then as part of the
authentication process they can have the OS accept a controlled and
minimal set of data to create the process as the new user and hand
over the connection?  PAM already has a large amount of control over
the authentication process, so it seems like we just need to change
the security context that this runs in.  That's just
brainstorming-level thinking though - there could be obvious issues
with this that just haven't occurred to me.

-- 
Rich

Re: [gentoo-dev] RFC: banning "AI"-backed (LLM/GPT/whatever) contributions to Gentoo

[Rich Freeman]

On Wed, Feb 28, 2024 at 1:50 PM Arthur Zamarin  wrote:
>
> I know that GitHub Copilot can be limited to licenses, and even to just
> the current repository. Even though, I'm not sure that the copyright can
> be attributed to "me" and not the "AI" - so still gray area.

So, AI copyright is a bit of a poorly defined area simply due to a
lack of case law.  I'm not all that confident that courts won't make
an even bigger mess of it.

There are half a dozen different directions I think a court might rule
on the matter of authorship and derived works, but I think it is VERY
unlikely that a court will rule that the copyright will be attributed
to the AI itself, or that the AI itself ever was an author or held any
legal rights to the work at any point in time.  An AI is not a legal
entity. The company that provides the service, its
employees/developers, the end user, and the authors and copyright
holders of works used to train the AI are all entities a court is
likely to consider as having some kind of a role.

That said, we live in a world where it isn't even clear if APIs can be
copyrighted, though in practice enforcing such a copyright might be
impossible.  It could be a while before AI copyright concerns are
firmly settled.  When they are, I suspect it will be done in a way
that frustrates just about everybody on every side...

IMO the main risk to an organization (especially a transparent one
like ours) from AI code isn't even whether it is copyrightable or not,
but rather getting pulled into arguments and debates and possibly
litigation over what is likely to be boilerplate code that needs a lot
of cleanup anyway.  Even if you "win" in court or the court of public
opinion, the victory can be pyrrhic.

--
Rich

Re: [gentoo-dev] RFC: banning "AI"-backed (LLM/GPT/whatever) contributions to Gentoo

[Rich Freeman]

On Tue, Feb 27, 2024 at 9:45 AM Michał Górny  wrote:
>
> Given the recent spread of the "AI" bubble, I think we really need to
> look into formally addressing the related concerns.

> 1. Copyright concerns.

I do think it makes sense to consider some of this.

However, I feel like the proposal is redundant with the existing
requirement to signoff on the DCO, which says:

>>> By making a contribution to this project, I certify that:

>>> 1. The contribution was created in whole or in part by me, and
>>> I have the right to submit it under the free software license
>>> indicated in the file; or

>>> 2. The contribution is based upon previous work that, to the best of
>>> my knowledge, is covered under an appropriate free software license,
>>> and I have the right under that license to submit that work with
>>> modifications, whether created in whole or in part by me, under the
>>> same free software license (unless I am permitted to submit under a
>>> different license), as indicated in the file; or

>>> 3. The contribution is a license text (or a file of similar nature),
>>> and verbatim distribution is allowed; or

>>> 4. The contribution was provided directly to me by some other person
>>> who certified 1., 2., 3., or 4., and I have not modified it.

Perhaps we ought to just re-advertise the policy that already exists?

> 2. Quality concerns.

As far as quality is concerned, I again share the concerns you raise,
and I think we should just re-emphasize what many other industries are
already making clear - that individuals are responsible for the
quality of their contributions.  Copy/pasting it blindly from an AI is
no different from copy/pasting it from some other random website, even
if it is otherwise legal.

> 3. Ethical concerns.

I think it is best to just avoid taking a stand on this.  Our ethics
are already documented in the Social Contract.

I think everybody agrees that what is right and wrong is obvious and
clear and universal.  Then we're all shocked to find that large
numbers of people have a universal perspective different from our own.
Even if 90% of contributors agree with a particular position, if we
start lopping off parts of our community 10% at a time we'll probably
find ourselves alone in a room sooner or later.  We can't make every
hill the one to die on.

> I think adding "made by real people" to the list of our advantages
> would be a good thing

Somehow I doubt this is going to help us steal market share from the
numerous other popular source-based Linux distros.  :)

To be clear, I don't think it is a bad idea to just reiterate that we
aren't looking for help from people who want to create scripts that
pipe things into some GPT API and pipe the output into a forum, bug,
issue, PR, or commit.  I've seen other FOSS projects struggling with
people trying to be "helpful" in this way.  I just don't think any of
this actually requires new policy.  If we find our policy to be
inadequate I think it is better to go back to the core principles and
better articulate what we're trying to achieve, rather than adjust it
to fit the latest fashions.

-- 
Rich

Re: [gentoo-dev] last rites: sys-fs/eudev

[Rich Freeman]

On Thu, Sep 14, 2023 at 1:39 PM Eddie Chapman  wrote:
>
> If you don't like it, then just go and roll your own. Of course
> I know I (and anyone else) can do that. So then what's the point of
> discussing anything then?

This is a fair question, but I think you're missing how most FOSS work
gets done in practice. Look at this list.  95% of it are FYIs from
devs talking about the work they've already done.  "I've introduced
this new feature - here is how you need to adjust what you're doing to
take advantage of it" - and so on.

If there is discussion about hypothetical future changes, it is
typically because somebody already plans to do the work and they're
soliciting advice, or working towards council approval for a breaking
change (ie a change that impacts other packages in the repo).

In any case, most of the reactions on this list were probably
anticipated before eudev was masked, so the discussion isn't really
informing any decisions.

As I said before the thing that would be most likely to change the
course of events is somebody stepping up to maintain things, and if
that happens it probably won't involve much discussion on the list
anyway.  They'd just fix things.

> What's the point of having a big tree with
> hundreds of packages? Why not have a very minimal tree instead and let
> everyone go and run multiple independent repos so we can all do what we
> want?

Actually, I would kind-of prefer if Gentoo were organized in just such
a way, but to make it practical the package manager would need a bit
of enhancement (such as letting users prioritize repositories at the
individual package level, a reasonable system of cross-repo
dependencies, and better tools for tagging repos and communicating the
QA standards for them and taking this into consideration when
syncing).

You won't see me badgering the portage team to make it possible
though, because that would be a lot of work, and if I cared about it
that much I'd be just asking for a few guidelines and making my own
PRs.

-- 
Rich

Re: [gentoo-dev] last rites: sys-fs/eudev

[Rich Freeman]

On Thu, Sep 14, 2023 at 12:50 PM Eddie Chapman  wrote:
>
>  if people want to run the damn thing just let them be!

If you keep using eudev, and you don't tell anybody about it, then
they won't even know.  Nobody is keeping anybody from using eudev.
They're just not actively doing work to keep it working with changes
in the repo.  If you stop syncing the repo, or fix those issues
yourself, or just avoid the use-cases that have issues, then you can
use eudev forever.

You could even publish an overlay and accept contributions from others
who want to use eudev, so as to share the effort required to do so.
You don't need anybody's permission to do so - all you need is a free
git repo somewhere to sync from.  Being source-based, Gentoo is
probably one of the easiest/most-practical distros out there to fork
system-level packages on.

-- 
Rich

Re: [gentoo-dev] last rites: sys-fs/eudev

[Rich Freeman]

On Tue, Sep 12, 2023 at 9:36 AM Eddie Chapman  wrote:
> in Gentoo. Have any of these 4 maintainers publicly said (anywhere) that
> they are not interested in being maintainers anymore (which is fine if
> that is the case)?  We're not talking here about a lone maintainer of some
> peripheral package that's disappeared leaving an orphaned package.

It isn't like somebody is censoring the lists or waging commit wars on
the metadata.xml/mask file.  If somebody was eager to maintain it I'm
sure they'd have spoken up.

> I'm an outsider to Gentoo development (just a heavy user for over a decade
> both personally and professionally) so I might have missed something. I
> just find it puzzling.

I'm not puzzled by what is going on, or by your email, because it
happens basically anytime a high-profile package is treecleaned.  Yes,
Gentoo is about choice, but somebody has to actually do work to make
the choices viable.  There are always more people interested in using
software than maintaining it.  The frustration is completely
understandable, but also kinda unavoidable.

Repo QA standards don't mean that it has to barely work for your
specific use case.  The package has to deal with compatibility issues
with stuff you don't use as well, which is why maintaining a system
package can be hard work.  It is usually less of an issue for more
ordinary applications, which tend to have fewer interactions.  If it
is "good enough" for you as it is, then just move it to a private
overlay and keep using it.  You probably would need to override a
virtual or two as well.  Or publish your work somewhere others can use
it.

--
Rich

Re: [gentoo-dev] last rites: sys-fs/eudev

[Rich Freeman]

On Mon, Sep 11, 2023 at 10:34 PM orbea  wrote:
>
> Regardless the disappointment is a valid concern when Gentoo is willing
> to pull the rug up from under users feet under erroneous claims of the
> project being dead.
>

As a complete outsider, I think this conversation is focusing on the
wrong issue.

IMO the main reason it is getting treecleaned is the lack of a
maintainer.  Everything about this entire back-and-forth screams
lack-of-maintainer.

You're essentially arguing that the Gentoo devs are out of touch with
the real status of upstream.  To a point I'd agree - that's because
normally it is a maintainer who stays on top of that stuff.

This is a fairly critical system package for anybody who has it
installed.  You don't want something like that not getting attention
when it has a problem, whether upstream or packaging-related.

Blueness kinda summed it up in his original news item.  This isn't
something that can be handled by a drive-by commit or two, or a
loosely involved proxy-maintainer.  Somebody needs to really step up
and take ownership of eudev for it to be viable.  Even if upstream
were great I wouldn't want to be using a stale package that is barely
maintained.

What is really needed is somebody stepping up and saying I will
maintain this.  If they were a Gentoo dev then the mask would probably
be gone already (IMO never would have happened if they stepped up
before now).  If it were a proxy I don't want to speculate too much
but they'd probably need to have a good history with such things.

Without anybody saying "I will personally fix this stuff" I just don't
see the treecleaning process stopping.  Arguing about the state of
upstream isn't going to change much, and if there were a maintainer
they'd just be dealing with it.

Sorry for this being a bit of a ramble.  I do feel for your situation,
but I don't want to see you fighting the wrong battle.  Disclaimer:
this is just my outside observation having seen many a treecleaning
frustration in the past.  I don't speak for any authority in Gentoo
here.

-- 
Rich

Re: [gentoo-dev] adding sec-keys/openpgp-keys-gentoo-release to @system

[Rich Freeman]

On Sat, Aug 12, 2023 at 10:34 AM Andreas K. Huettel
 wrote:
>
> I'd like to add
>
>   sec-keys/openpgp-keys-gentoo-release
>
> to @system - any objections?
>
> This is more of a formal request since portage already depends on it anyway, 
> and
> the package is present in every stage3. However, it in my opinion makes sense
> to explicitly state that it needs to be present.

This seems like the opposite of making the dependency explicit.  It is
already explicit, because everything that actually needs it depends on
it.  If you stick it in @system then the packages that need it will
remove the dependency, and it will become less clear what actually
uses it.

IMO @system is a design flaw.  The most obvious impact of it is a
large number of packages that cannot be built in parallel, because
portage can't know what packages depend on packages in @system,
because we don't specify these dependencies.

Obviously getting rid of @system would take a fair bit of effort to
capture all the dependencies, and have the stage3 builds pull from a
meta-package or something similar.  However, at the very least it
would be best to avoid adding more to it.

@system is also one of the reasons why some configuration changes can
be accomplished via USE flags and virtual packages, and other
configuration changes must be accomplished via profiles.  For example,
supporting multiple sets of signing keys would be simpler with these
just being pulled in by virtual packages and USE flags, and harder if
these all required profiles or something like Funtoo mix-ins.

Obviously this package is one of the less impactful ones to add to
@system, but I figured I'd at least toss that out as food for
thought...

-- 
Rich

Re: [gentoo-dev] Massive Github PR Queue

[Rich Freeman]

On Fri, Aug 11, 2023 at 12:54 PM Sam James  wrote:
> juippis' whole email nails the issue, but I'd just like to add that
> there's kind of a baseline at ~400/~450 or so where everything below
> that is PRs for new packages or long-obsolete stuff nobody closed yet,
> or where we're waiting on the submitter. It's just that going ahead
> and going over those is time-consuming and means we're spending less
> time on active PRs which need looking at.

++

In general the Gentoo workflow tends to be a bit different from a lot
of other projects as well, with quite a bit more individual
ownership/interest.  Many distros just stick to core packages in a
main repo, and consign everything else to user-maintained
repos/overlays/etc, or tiers with lower expectations for quality.

Then you have all the build-related issues that can be hard to
reproduce/test, that other distros simply don't have to worry about
because they only support one or two build configurations.

Another thing I've seen in fairly large projects is auto-closing of
issues/bugs/PRs/etc.  If your submission sits around for more than a
few weeks without getting merged, a bot comes along and helpfully
marks it closed.  That of course results in a nice low open-issue
count, but of course that doesn't mean those issues are actually
fixed.

-- 
Rich

Re: [gentoo-dev] EGO_SUM (was: [gentoo-project] Gentoo Council Election 202306 ... Nominations Open in Just Over 24 Hours.)

[Rich Freeman]

On Wed, Jul 5, 2023 at 2:40 PM Gerion Entrup  wrote:
>
> Am Mittwoch, 5. Juli 2023, 01:09:30 CEST schrieb Oskari Pirhonen:
> > On Tue, Jul 04, 2023 at 21:56:26 +, Robin H. Johnson wrote:
> > >
> > > Developing it requires PMS work in addition to package manager
> > > development, because it introduces phases.
> > >
> > > - primary fetch of $SRC_URI per ebuild, including indirect Manifest
> > > - primary validation of distfiles
> > > - secondary fetch of $SRC_URI per indirect Manifest
> > > - secondary validation of additional distfiles
> > >
> > > A significantly impacted use case is "emerge -f", it now needs to run
> > > downloads twice.
> >
> > I'm not sure double downloading is required. Consider a flow similar to
> > this:
> >
> > 1. distfiles are fetched as per the ebuild
> > 2. distfiles are hashed into a temporary Manifest
> > 3. temporary Manifest is hashed and compared with the hashes stored in
> >the in-tree Manifest for the direct Manifest
>
> This is exactly, what I meant. A webstorage is not needed. A second
> download process is also not needed. Just an additional Manifest format
> is needed for ebuilds with more than n distfiles.
>

I suspect that Robin was proposing indirect manfests AND src uris, and
not just indirect manifests.  In any case, if he wasn't, then I'd
suggest it would make sense to have that so that we don't need giant
lists of src_uris or go sums or whatever in ebuilds.  Sure, the
manifests are even larger than the original file references, but those
will still be long.  Plus if a file is used by 5 versions of an ebuild
it will be present in the manifests once per hash function, but in the
ebuilds 5 times.

I agree though that if only the manifests are moved to a fetched file
then you could fetch that on the first pass, though you'd still need
the extra logic to parse it.  I'm not sure it really is much of a
difference to the effort involved.

Aren't go sums already content hashes?  It might make even more sense
to create some kind of modular manifest verification logic in portage
so that the same eclass that handles EGO_SUM could tell the package
manager how to check the integrity of the files that are fetched.
Well, assuming we trust whatever hash function they're using (I'm
afraid to check - maybe this isn't such a great idea...).

-- 
Rich

Re: [gentoo-dev] QA PG 0305 (manpages must always be installed) discussion

[Rich Freeman]

On Fri, Jan 20, 2023 at 9:13 AM orbea  wrote:
>
> Protecting users from themselves can be a misfeature. Its better to
> educate and then let them freely choose than to play as their nanny.
>

TL;DR - it makes sense to unconditionally install prebuilt manpages,
but not to require pulling in dependencies to build them.  The latter
should be up to maintainer discretion.

I suspect the original policy was more about the fact that install
masks are already available to cover use cases like avoiding manpages.
I think the dependency issue is the much bigger one.  If the
dependencies to generate manpages are trivial or can be expected to be
otherwise installed I can see the argument for just installing them
anyway.  However, when they're large it doesn't make much sense.

If you're going to go on an expedition to the middle of the wilderness
where you can't use your phone to google a manpage, then I'd think
that manpages are probably just one of MANY things you'll be wanting
to pack.  I get that once upon a time people only owned a single
computer and were concerned about having that one computer down.
Access to specialized tools could still be something you have to plan
for today.  However, the text of most manpages is easily searched
online today so missing them is at worst an inconvenience unless you
literally have no access to a browser on another device.

-- 
Rich

Re: [gentoo-dev] Gentoo LTS or: proper backward compatibility?

[Rich Freeman]

On Mon, Jan 2, 2023 at 7:48 AM m1027  wrote:
>
> When we create apps on Gentoo they become easily incompatible for
> older Gentoo systems in production where unattended remote world
> updates are risky. This is due to new glibc, openssl-3 etc.

So, unless you're proposing some improvement this might be
better-suited for the -user list.  Officially Gentoo doesn't really
"support" a stable/LTS/release-based environment.  That isn't to say
that it couldn't be made to do this, however, a more release-based
process is one of the core competencies of most other distros, so I'm
not sure how much Gentoo would add here vs just running something
else.

> (1) Keeping outdated developer boxes around and compile there. We
> would freeze portage against accidental emerge sync by creating a
> git branch in /var/db/repos/gentoo. This feels hacky and requires a
> increating number of develper VMs. And sometimes we are hit by a
> silent incompatibility we were not aware of.

If you're running a large production environment and want a stable
platform to target, I'd think you'd want to minimize the number of
those you have.  You wouldn't want every server running a different
base OS, and then have a branch to target development at it.

The idea of basing your own releases on Gentoo is probably something
many companies do.  All you need is to just host one repository and
distfile mirror for it, and point all your production hosts at it.
Then you could have staging/development environments, and promote the
core OS as appropriate.  Then your migration path is the same for all
hosts and you can account for major vs minor changes.

You do need to mirror distfiles as one of the weaknesses of Gentoo for
a release-based strategy is that we do not have a solid solution for
archiving things like patches/services/etc that are distributed
outside of the repo.  If a package is removed from the current repo no
QA process ensures that the SRC_URIs for anything they used remain
stable.

Of course somebody would need to do the work but it shouldn't be THAT
difficult to have a Gentoo Reference Release project that basically
just snapshots the repository/distfiles at points in time and provides
a guarantee of clean updates between milestones.  Of course if you
want backported security updates on top of that this would be quite a
bit more effort.  It would mainly be useful for those who aren't
updating regularly, but completely ignoring all updates isn't a good
practice regardless which is probably part of why this hasn't
happened.  You could greatly minimize the cost of backporting security
updates if you only targeted packages of interest to you, or where the
updates are low-risk to you, but that depends quite a bit on your own
needs.

> (3) Distributing apps as VMs or docker: Even those tools advance and
> become incompatible, right? And not suitable when for smaller Arm
> devices.

I don't see why containers would be unsuitable for ARM.  VMs certainly
are memory hogs but well-designed containers don't really consume much
more memory than the service they are hosting.  Of course if you dump
a whole OS in a container that will consume a fair bit of RAM.

An advantage of containers is that they make the host OS less
relevant.  You could run a release-based distro with security-only
updates on the host, and then application containers are low-risk to
update remotely and can be built in whatever environment they are most
suited to.

There is a reason k8s is popular.

-- 
Rich

Re: [gentoo-dev] Disturbing state of arch testing in Gentoo

[Rich Freeman]

On Mon, Nov 7, 2022 at 7:34 PM John Helmert III  wrote:
>
> On Mon, Nov 07, 2022 at 07:23:33PM -0500, Rich Freeman wrote:
> > Proprietary tools do contribute to this since they can
> > generate results that are harder to reproduce, but if they are clear
> > and accurate and actionable it could still be a net-positive.
>
> In some cases, yes, this is exactly the problem. This was one of the
> bugs reported in the now-deleted issue tracking repository on Github.
>

This was hinted at earlier in the thread.  My goal wasn't to say
whether it was or was not an issue, but bring the focus more on the
tangible impact, as I think that will probably help to make this less
about philosophy and more about impact.  I think that is more likely
to create action (whether that is a policy change or improvement to
the tooling).

-- 
Rich

Re: [gentoo-dev] Disturbing state of arch testing in Gentoo

[Rich Freeman]

On Mon, Nov 7, 2022 at 6:16 PM Sam James  wrote:
>
> > On 7 Nov 2022, at 06:07, Oskari Pirhonen  wrote:
> >
> > On Sun, Nov 06, 2022 at 11:37:24 +0100, Piotr Karbowski wrote:
> >> I would be in favour of stepping up the social contract and actually
> >> prohibiting this kind of things, we had that before too, the nattka you
> >> mgorny wrote is replacement for old bugzilla bot that was ...
> >> closedsource and perished, though nattka now have way more features than
> >> the old thing ever had.
> >
> > As a user, I think it would be really cool if there was a requirement
> > that all infra and infra-adjacent stuff was free software.
> >
> > I feel like I've read that Debian already has something like this. While
> > doing some quick searches I didn't find a full-on requirement, but all
> > their infra bits I did find were powered by free software. The most
> > relevant ones being buildd [1] and debci [2]. Additionally, the debci
> > docs has inctructions on reproducing tests yourself [3] which is a nice
> > extra IMO.
>
> Gentoo has https://www.gentoo.org/get-started/philosophy/social-contract.html.

I feel like something like a dev-run tinderbox is a bit out of the
scope of that.

Suppose I file a bug against a package, pointing out some issue in it.
How do you know I didn't use some proprietary static code analysis
tool to discover that error?  Does it even really matter?  The bug
speaks for itself.  It is like worrying about whether somebody who
filed a bug was running Windows or another proprietary OS or browser
on their desktop.

Well, a tinderbox is just an automated process for doing just that.
We don't require any dev to use a proprietary tinderbox before
committing.  It is something that individual devs choose to use for
themselves, automating the testing workflow and possibly the
submission of bugs.

I think the key is something that was brought up earlier in the
thread: is this causing problems?  If somebody is running some tool
against the repository and automatically filing bugs, and those bugs
are not useful/actionable and waste the time of volunteers, then that
is a problem.  Proprietary tools do contribute to this since they can
generate results that are harder to reproduce, but if they are clear
and accurate and actionable it could still be a net-positive.

Of course if somebody wants to contribute to 100% FOSS tinderbox
efforts that would be even better.  Perhaps if our 100% FOSS tinderbox
efforts addressed our needs very well, then nobody would want to
bother with the proprietary reports, or generating them.  IMO it would
be better to create the FOSS solution before abandoning the
proprietary one.  Doing otherwise is basically burning bridges - it
can be motivating in a sense but not really ideal.  I'd love to have a
100% FOSS solution around all of this, but I appreciate what has been
created and can hardly criticize volunteers for failing to make it
happen, especially since I haven't contributed to that myself.

-- 
Rich

Re: [gentoo-dev] [RFC] Encouraging using hardening options in systemd units

[Rich Freeman]

On Fri, Aug 26, 2022 at 4:57 AM Florian Schmaus  wrote:
>
> While then can not be modified, settings made in /usr/lib/systemd/system
> can be overridden by the sysadmin by placing a file in /etc/systemd/system.
>
> I am not aware of a reason why a package manger should install systemd
> configuration files under /etc.
>

I'd say the biggest use case would be settings that typically require
modification that are needed at service launch time.  Think of
packages that stick stuff in /etc/conf.d for non-systemd settings.

An example of this can be found in sys-devel/distcc/files/distccd.service.conf

Sure, you could stick that in /usr, and then the user could copy it to
/etc (or use systemctl edit).  However, since it is something that is
basically intended to be edited you can argue that it makes more sense
to just stick it in /etc.  This also means that if a new setting gets
added the user will be made aware via config protection.  For a
drop-in that changes in /usr the user would receive no notice of this,
and the new settings would get merged with theirs or ignored depending
on how it was done.

For a distro override that wouldn't typically be modified, like
integrating one service with another that only service the local host,
then maybe /usr would make more sense.

Using systemctl edit is also a little awkward due to the way it
presents you with the original config at the bottom and you have to
copy the stuff you want to modify to the top.  I assume it lets you
copy comments as well, but it is a bit like opening up two editors and
copying from a skeleton file to a new file, vs just editing one in
place.  Well, except you aren't working between two files but editing
at two places in one file, so if your editor doesn't handle split
screen well you will scroll around a lot if the file is large.

I don't have very strong feelings about this either way, but I can
definitely see why things were done the way they're done.  I'm not
sure if there are any examples of fairly large drop-ins in /etc that
we install but I'd probably want to look at one before changing the
approach to see if it makes sense.

-- 
Rich

Re: [gentoo-dev] Switching default password hashes from sha512 to yescrypt

[Rich Freeman]

On Mon, Jul 25, 2022 at 11:11 AM Marek Szuba  wrote:
>
> On 2022-07-25 15:35, Peter Stuge wrote:
>
> > Please only do that based on proven merit and nothing else.
>
> https://pthree.org/2018/05/23/do-not-use-sha256crypt-sha512crypt-theyre-dangerous/
> , https://www.password-hashing.net/ , the fact we still us the default
> number of rounds (i.e. 5000) with SHA512 which is *ridiculously* weak
> for modern hardware, lack of Argon2 support in libxcrypt for the time
> being due to upstream having decided to wait for an official RFC. You
> can probably find more yourself if you look.

The fedora link in the original email details why they changed it.  I
don't think regurgitating the argument will add to it.  By all means
point out if there is a concern with their reasoning though.

My initial question was whether this was some vanity hash change but
the changes are intended to greatly increase the cost of cracking
attacks.  I'm in no position to evaluate their merit but their
proposal contains various citations to people who presumably are.

-- 
Rich

Re: [gentoo-dev] proposal

[Rich Freeman]

On Wed, Jul 6, 2022 at 8:42 AM Florian Schmaus  wrote:
>
>
> It appears that we have at least two options here:
>
> A) Establish that the default is non-maintainer-commits-welcome, and
> introduce a  metadata element.
>
> B) Declare the default to be unspecified and introduce two metadata
> elements:  and
> .
>
> I think you are proposing A) here, but please correct me if I am wrong.
>
> Personally I would tend to B). But I have no strong opinion on this, as
> long as some kind of signalling is established.
>
> How do others feel about this?

What about ?  I
guess that is what we're calling "disallowed" but that seems to have a
connotation that devs don't want contributions, when they just want to
be aware of what is going into their packages before it happens.

Deferring maintenance to bumps is the one use case that came up in the
thread that seems likely to be pretty common.

-- 
Rich

Re: [gentoo-dev] proposal

[Rich Freeman]

On Mon, Jul 4, 2022 at 7:21 PM Robin H. Johnson  wrote:
>
> It had 3 states however:
> a) go ahead and touch it, no additional approvals needed
> b) please get a maintainer to approve it
> c) do not touch it
>

++

Though to be fair b is really no different from what just about
anybody can do via a pull request.  I don't think most maintainers are
going to be hovering between a vs c.  I suspect most are going to be
divided between a vs b.  I guess I could see an argument for c if some
package is really finicky and tends to get a lot of repetitive
requests for changes that won't work for reasons that might not be
obvious, but I'm not sure if that is really a concern.

-- 
Rich

Re: [gentoo-dev] packages touching files in /dev

[Rich Freeman]

On Tue, May 24, 2022 at 6:49 AM  wrote:
>
> Is there some hook to emerge I can use where I can attach some code to
> run tests after each individual package when doing emerge @world ?
>

So, Portage has hooks, and that would work for any file being
installed normally (so would config protection and that would be a
much easier solution).

There are a couple of problems though:
1. The only package I'm aware of that directly touches /dev is
static-dev (which I hadn't even heard of until you mentioned it).  It
uses a post-install hook to create device nodes, so there is no
opportunity to inspect anything before /dev is modified.  This isn't
the normal way to install files, but of course it isn't installing
normal files.
2. I think it is very unlikely that a package is directly modifying
/dev.  It seems more likely that a package is installing some daemon
that gets run as root and then it modifies /dev, maybe on your next
boot.  Obviously if you install something like udev you'd expect to
end up with /dev getting modified when it runs.  Again, there is
nothing for a hook to detect.

Having a backup (it is static after all), and something like a
read-only mount might be your better solutions, if you really want a
static dev, or maybe marking files as immutable or something.  (You
might want to test that - I am assuming you could still write to a
device node on a read-only filesystem but it isn't like I've tried.  I
don't think there is anything special about /dev so you could just
create a device node in some other read-only filesystem and test it
out.)

If you do find a random package touching /dev I think most here would
be pretty interested, as that seems rather bizarre.

-- 
Rich

Re: [gentoo-portage-dev] Normaliser function for distfiles

[Rich Freeman]

On Tue, May 17, 2022 at 8:32 AM Markus Walter  wrote:
>
> On Tue, 17 May 2022 14:14:57 +0200,
> Rich Freeman wrote:
> >
> > On Mon, May 16, 2022 at 1:37 PM Markus Walter  
> > wrote:
> > >
> > > My use case is the following: I would like to improve the gs-elpa program
> > > and provide a precomputed overlay for melpa. However the melpa distfiles 
> > > are
> > > rebuilt everyday and cause checksum failures. However the only thing
> > > changing are the timestamps. Hence if a normaliser program could simply 
> > > set
> > > all timestamps to some predefined value (say 1.1.1970) then this problem
> > > should vanish.
> > >
> >
> > Wouldn't a simpler solution be to just have an ebuild setting that
> > tells the package manager to not check the timestamp?
>
> The timestamps are inside archive files thus changing the overall file
> hash. This happens during distfile download, where some more sophisticated
> replace all timestamps function would be necessary than just ignoring one
> timestamp.

Ah, apologies.  Totally missed that.  Yeah, obviously if the
timestamps INSIDE the archive are changing your only solution is to
edit the contents.  I thought it was just the external timestamp
changing (I'm not sure if portage even checks that, or if wget changes
the mtime on files it downloads).

-- 
Rich

Re: [gentoo-portage-dev] Normaliser function for distfiles

[Rich Freeman]

On Mon, May 16, 2022 at 1:37 PM Markus Walter  wrote:
>
> My use case is the following: I would like to improve the gs-elpa program
> and provide a precomputed overlay for melpa. However the melpa distfiles are
> rebuilt everyday and cause checksum failures. However the only thing
> changing are the timestamps. Hence if a normaliser program could simply set
> all timestamps to some predefined value (say 1.1.1970) then this problem
> should vanish.
>

Wouldn't a simpler solution be to just have an ebuild setting that
tells the package manager to not check the timestamp?

-- 
Rich

Re: [gentoo-dev] proposal: use only one hash function in manifest files

[Rich Freeman]

On Wed, Apr 6, 2022 at 1:29 PM Jason A. Donenfeld  wrote:
>
> Sort of. The security between infra and users relies on SHA2-512. The
> security between devs and infra relies on SHA-1. I guess the "full
> system" depends on both, but I've been focused on the more likely
> issue of a community-run mirror serving bogus files.

Well, that depends on how you're syncing the tree.  If you're using
rsync then there is a signed manifest in the root, so I agree in that
case it is just SHA2-512.  If you're syncing using git then the
manifests only reference distfiles, and the only link between the
commit and the tree/objects are their SHA-1 hashes until git adopts a
different hash function.

> Yea I see this argument, but I don't quite buy it. Maintaining two
> sets of hashes for the unlikely event that one gets broken AND we
> absolutely cannot incrementally transition gradually to an unbroken
> one seems rather overblown.

It is very much a hand-waving judgement call.  This is one of those
low cost, low risk, high reward situations IMO.  The cost of
calculating hashes is fairly low (especially if done in a more sane
way).  The odds it will ever have a benefit are low.  If it does have
a benefit, it will be in a situation where the world is on fire and
we'll be very happy to not have to go verify a gazillion distfiles on
top of everything else we have to fix.  I'll defer to those wiser than
me to make the call.  :)

-- 
Rich

Re: [gentoo-dev] proposal: use only one hash function in manifest files

[Rich Freeman]

On Tue, Apr 5, 2022 at 8:05 PM Sam James  wrote:
> > On 5 Apr 2022, at 22:13, Jonas Stein  wrote:
> >
> >> In other words, what are we actually getting by having _both_ SHA2-512
> >> and BLAKE2b for every file in every Manifest?
> >
> > Implementations are often broken and we have to expect zero day attacks on 
> > hashes and on signatures. Hence it does not hurt to have a second hash.
>
> I don't think this is the case. They're not broken often, it's a very very 
> big deal when they do, and we'd also have far bigger problems in such a case 
> (as already pointed out, TLS would be an issue, but also GPG signatures, git 
> commit hashes, ...).

Our security fails currently if EITHER SHA2-512 or a hardened version
of SHA-1 are defeated.  Our top gpg signature is bound to a git commit
record by SHA2-512, and the git commit record is bound to everything
else in the repository (including the manifest objects) by SHA-1,
because git hasn't transitioned away from that (as far as I'm aware it
is still a work in progress - the SHA-1 algorithm it uses is hardened
against known attacks).

That said, I think there is still an argument for having two hashes in
the manifests.  If we have two independent manifests, then if either
SHA-1 or SHA2-512 are defeated all we need to do is update git+gpg to
the patched version (which no doubt would be rushed into a release
quickly), and then do a commit to the repo and sign it with the Gentoo
key.  The new commit would have a full set of new hashes using a
secure hash function, and then a back-reference to the previous commit
using SHA-1 (assuming we didn't rebase the entire tree and lose all
our historical gpg signatures - we might consider creating a new repo
and saving a historical one).   That would have new hashes all the way
from the top commit down to all the objects it references, so the top
commit would now be secure.  When signed with an updated gpg the
signature would be attached with a secure hash.  So now we're secure
again.  If we're concerned about old signatures getting recycled in
preimage attacks we could of course revoke the key and issue a new
one.

What we don't need to do is redo all the manifests, and that is
important because we don't actually have the ability to redo those
centrally.  Anybody can add a commit to the repo and re-sign it, but
we'd need all the maintainers to go through and generate new manifests
for anything that is fetch-restricted, or aggressively treeclean.

So it isn't that having two hashes can't fail, but rather that if it
does fail it is easier to recover.

>
> >
> > It is very likely that we can not trust in X for a while in the next years, 
> > but it is very unlikely that two different implementations are affected.
> >
>
> I don't think it is likely that e.g. SHA512 will be broken in the next few 
> years, no, but if it is going to be, we have far bigger issues and we'd need 
> to have double algorithms in our whole stack, which we don't have.

I agree that this is an unlikely scenario, so it is a judgement call
as to whether the ease of recovery in the event of a failure is worth
the cost to maintain the second hash.  I agree that we'd need double
algorithms in the whole stack to prevent a failure, but in the current
state we do have advantages for recovering from a failure after the
fact.

It seems that the likely scenario is that we get advance warning of
weaknesses in a hash function, but without a practical exploit being
readily available.  In that case we could do a  more orderly
transition.  We'd still save time with the double hashed manifests,
and whether this makes a difference is hard to say.

>
> > Additionally calculating a second hash does not cost anything.
>
> It does have a cost at both Manifest-generation time and emerge-time.

This is certainly true, though if the current algorithm is reading the
files twice we could at least fix that.

I don't really have a strong opinion here.  I just wanted to point out
the recovery benefit of having two hashes on just the manifests, given
that it isn't easy to access all the distfiles.  I also wanted to
point out that we have SHA-1 exposure today, at least in git.

-- 
Rich

Re: [gentoo-dev] Deprecating repoman

[Rich Freeman]

On Wed, Mar 9, 2022 at 4:00 PM Matt Turner  wrote:
>
> Are there any useful checks or behaviors of repoman that are missing
> from pkgcheck and pkgcommit?

Would it make sense to package pkgcommit or otherwise stick it
somewhere official?  I know there is a copy on mgorny's github
repo/blog, but if this is going to become the recommended way to do
commits we might want to stick it someplace a bit more canonical, and
having it in a package would ensure that if there are any changes they
get distributed.

Obviously there are other ways to set it, but it lacks obtaining the
gpg key to use from make.conf.  I'm not sure that is really the best
place to put it in any case.  Just figured I'd point it out.

-- 
Rich

Re: [gentoo-dev] Unmask >=net-p2p/bitcoin*-0.21.1

[Rich Freeman]

On Wed, Feb 2, 2022 at 10:31 AM Joonas Niilola  wrote:
>
> Maybe I'm overthinking it due to all the attention bitcoin has received
> lately in Gentoo. But yeah, we haven't received any comments or bugs
> about the mask so I guess it's fine to remove it finally. I still kind
> of do think a news item wouldn't be the "wrong thing to do" either, but
> don't wish to prolong this process any further.
>

My two cents: bitcoin should work the same as any other package with
updates.  Any package update could do something you don't want it to
do.  That's why you have --pretend, and the ability to impose your own
masks.

Any package can of course issue news but it isn't very typical for
packages to issue "release notes" in advance for every single update.
Upstream of course often has them, and people who want them can just
subscribe to the upstream lists.

If somebody isn't interested in following upstream they probably
aren't interested in vetting every single release, and I'm guessing
that being 2 versions behind on everything is going to cause them and
the network more harm than good.  If you actually want to participate
in some fork of the network I fully support your right to do that, but
it shouldn't really be the default behavior of a package (and as I
understand it that is basically what happens if you aren't running a
version that supports some feature when that feature becomes
mandatory/etc).

-- 
Rich

Re: [gentoo-dev] [RFC] making rust-bin ordered first in virtual/rust

[Rich Freeman]

On Thu, Jan 20, 2022 at 4:10 PM Piotr Karbowski  wrote:
>
> Ideally we'd have some way to mark binary packages with new EAPI and
> have FEATURES flag like 'prefer-binary' and go with -bin in case there's
> || ( ) dependencies list, regardless of the original order in virtual.
> This way everyone could be happy and not choose one workflow over another.

Ideally we'd just have a repository of binary builds for everything
with default USE flags for a few profiles, and users could choose to
configure portage to just download the binary package if the flags
match, and of course this could be overridden per-package.  Then there
would be no need for -bin anything.  We have to maintain half of that
for the stage builds anyway.

I don't have a strong preference but I doubt that many of our users
receive any benefit whatsoever from building rust from source, and so
I'd suggest -bin ought to be the default.  However, having a
pre-installation notice that the -bin is available with a link/etc to
how to use it seems reasonable.  I bet a lot of users spend more time
getting the source build to work than it would take to just
reconfigure things so that they get the -bin.  If users are building
in a tmpfs or with numerous parallel jobs odds are that they'll end up
with a build failure for rust - CPU cores are way cheaper than RAM.

-- 
Rich

Re: [gentoo-dev] Don't use UIDs and GIDs below 100 without QA approval

[Rich Freeman]

On Thu, Nov 11, 2021 at 6:34 AM Florian Schmaus  wrote:
>
> On 11/11/2021 11.59, Ulrich Mueller wrote:
> > We could:
> >
> > - Open some part of the range between 500 and 1000. For example,
> >500..799, which would leave 200 IDs for dynamic allocation.
>
> +1, since I am not aware of any significant downsides doing so.
>

I will confess that 90% of the time that when I run into headaches due
to mismatching GID/UIDs it involves two different distros anyway.  I
definitely see the value in standardization, and there is some value
in doing it at the distro level, but really most of the value would
come from doing it cross-distro.

Ultimately I think a big part of the problem is that the whole UID/GID
model in Unix is a bit broken.  Of course that isn't going to change
anytime soon.

Probably still worth band-aid fixes for the time being.

-- 
Rich

Re: [gentoo-dev] You currently cannot smoothly upgrade a 4 months old Gentoo system

[Rich Freeman]

On Wed, Nov 3, 2021 at 1:14 PM Ulrich Mueller  wrote:
>
> > On Wed, 03 Nov 2021, John Helmert wrote:
>
> >> | https://projects.gentoo.org/council/meeting-logs/20091109-summary.txt
> >> |
> >> | Upgrade path for old systems
> >> | 
> >> | Vote (unanimous): The ebuild tree must provide an upgrade path to a
> >> | stable system that hasn't been updated for one year.
>
> > Does "upgrade path" imply a simple world upgrade is all that should be
> > necessary to upgrade the system? I wouldn't interpret it this way.
>
> That a "simple world upgrade" was meant is very clear from the full
> meeting log, and from the log of the previous 2009-10-12 meeting (open
> floor section).
>

It probably would be good to get these policies documented someplace
OTHER than the council decision logs.  I do remember the discussion
from the distant past but it is worth having it in a more prominent
place, especially since a one year stable update path is not going to
happen by accident.

I was thinking that it matters way more that @system is updatable than
world in general, since @system can be used to bootstrap everything
else.  However, I think this distinction really doesn't make much of a
difference.  If your system can't be smoothly updated, it is unlikely
to be due to some leaf package like a browser/MUA/etc.  Most likely it
is due to a widely-used dependency.

So, by all means require @world to be updatable, but I think that if
you focus on the packages in @system you'll basically get the rest for
free.

EAPI 8 came up in a later email and to consolidate replies I'll just
say that the issue isn't so much adopting EAPIs in newer packages, but
the rush to tidy up old ones that creates the problems.  If you have
v1/2/3 of a package stable and in the repo, and v3 uses an unsupported
EAPI, and v1 is installed, I think portage will (or at least ought to)
offer an upgrade from v1 to v2 - it should just not see v3 or consider
it in the depgraph.  Of course keeping around old versions might
require dealing with security issues/etc that impact them.  An
alternative would be of course to just avoid EAPI updates on @system
for a little while, or at least the parts of @system that portage
needs to upgrade.

Having QA tools detect this would be ideal, but right now they could
only reliably detect things like newer EAPIs in @system.  Since we
don't require putting @system dependencies in packages there is no way
for a QA tool to tell what is or isn't needed to update portage.  Then
you have more complex situations like PYTHON_TARGETS, and probably
others as well.  I had one host that struggled a bit with the xcrypt
update for some reason (some weird preserved libs issue or something -
libcrypt was still showing up as owned by glibc after updating it and
I had to override collisions to get xcrypt to install).  When
upgrading a system that is completely up to date requires careful
following of news items it is going to be painful to execute a whole
bunch of updates at once.

Maybe another path would be to mark milestone repositories for the
last year that are easy to update in sequence.  So if you have an
update that requires manual care, you could mark a milestone just
before that update which is easy to get to, and then another one after
the update (ideally right before the next difficult update).  This
would just be a snapshot of portage or a git commit reference, and
along with that a commitment to maintain distfiles/etc if they aren't
hosted in reliable places (ie patches/etc in devspace).

Another option might be to have collections of binary packages at key
milestones.  Sure, they won't be built to a user's specification, but
if you have a year old system you could use them to quickly bootstrap
it up to something more recent, and then an emerge will rebuild
anything that has the wrong USE flags/etc and to bring the system
completely up to date.  You could even just use those binary packages
as a sort of release-based version of the distro.

Just some things to consider.

-- 
Rich

Re: [gentoo-dev] You currently cannot smoothly upgrade a 4 months old Gentoo system

[Rich Freeman]

On Wed, Nov 3, 2021 at 11:03 AM Thomas Deutschmann  wrote:
>
> This is not about finding solution to upgrade the system (in this case
> it was enough to force PYTHON_TARGETS=python3_8 for portage). This is
> about raising awareness that Gentoo is a rolling distribution and that
> we guarantee users to be able to upgrade their system when they do world
> upgrades just once a year (remember: in my case the last world upgrade
> is just 4 months old!). If they cannot upgrade their system without
> manual intervention, we failed to do our job.

Do we have this "guarantee" documented somewhere?  I thought I've
heard six months tossed around.  You say one year.  It seems
reasonable to have some sort of guideline like this and try to stick
with it, at least for @system.

(I had a painful update on a container that was about six months old a
little while ago - I just did updates from git checkouts (which isn't
guaranteed to work due to distfiles issues).  Obviously
troubleshooting a container where a rollback is a one-liner is a lot
easier, but progressive updates also tend to require a lot of
semi-redundant updates.)

-- 
Rich

Re: [gentoo-dev] [RFC] Portage einfo, elog... output format change

[Rich Freeman]

On Mon, Oct 4, 2021 at 2:48 AM Michał Górny  wrote:
>
> On Tue, 2021-09-28 at 17:36 +0200, Michał Górny wrote:
> > I know I'm going to regret asking this... but I've prepared a change to
> > the Portage output format and I think it asks for a wider discussion
> > than internally in Portage team.
>
> As I suspected, I truly regret sending this mail.  I'm dangerously close
> to burning out, so I'm going to retract these patches.  When you decide
> what you want and make patches for it, feel free to send them to
> Portage.

Keep in mind that while distributing patches and soliciting comments
is a good practice, I don't believe any of our policies REQUIRE that
you:

1.  Reply to anybody who comments.
2.  Address any comments.
3.  Wait until anybody (let alone everybody) agrees before proceeding.

I think that we sometimes let the requirement to communicate somehow
stifle the desire to get things done.  While I don't recommend it, you
can technically satisfy any communications requirements by posting
your patch and literally never checking your email before committing
it.  Obviously if you mess something up in doing so you'll look dumb,
but it isn't intended to be a bureaucratic requirement.

I suggest just skimming the comments to see if there is anything that
seems like a good idea, then implementing those ideas (which you'll
probably want to do anyway), and ignore the rest without comment.  If
somebody has a problem with what you're doing, the onus is on them to
go find somebody to complain to in order to stop you.  If you're the
maintainer then you don't need permission to commit.

In my experience the Council is fairly resistant to requests to meddle
for things that aren't super-critical, so I'd be shocked if they
didn't just ack and dismiss any request to bikeshed exactly what
prefix your elog output uses.

Open to contrary opinions, but IMO I think maintainers perceive that
the distro is more consensus-driven than it actually is.  You don't
have to "win" the email battle.

-- 
Rich

Re: [gentoo-dev] Guidance on distributed patented software

[Rich Freeman]

(This is all tangential to the main issue of this thread and just
discussing internet history - skip as you wish...)

On Mon, Sep 27, 2021 at 2:14 PM Marek Szuba  wrote:
>
> I am no expert on US law but from what I have read (in many different
> sources, with me having begun using PGP in either late 1996 or early
> 1997 i.e. when it was still very much subject to US export restrictions)
> about this case, both the publishing of the source-code book and it
> having subsequently been taken out of the country has been legal - the
> former due to publishing the first amendment

Well, based on the little I know of US export controls, I doubt that
being in book form vs some other form really has any bearing in
principle.

HOWEVER, I think it was probably done specifically as a challenge to
the constitutionality of the law.  Ie, the argument would be that it
ought to be legal to take the source code out in any form.  By doing
it via a formally published book though they take away all the "exotic
internetness" out of the equation and this way all the 60 year old
judges (in the 90s) who might get involved are forced to confront it
as suppression of book distribution.  In principle though I think most
of us would agree that there is no difference in sharing information
no matter the way in which it is conveyed.  It was probably their hope
that if it did go to court any ruling that secured the right to
distribute via book could then be leveraged against other modes.

I'm guessing that it was never challenged in court precisely for this
reason.  US export controls cover the communication of information via
any mode:
https://www.bis.doc.gov/index.php/policy-guidance/deemed-exports

If they had fought the export of this book it is quite possible that
there would have been a ruling that just finds all export controls to
be illegal.  Really when you think about it any sort of restriction on
communication of classified information or whatever is going to run
into the 1st amendment.  Courts are going to tend to make their
rulings on what can be restricted narrow as a result.  The government
probably prefers to maintain some FUD around where those boundaries
lie, to get companies in particular to follow policies that in the end
might not be enforceable.  (Note I'm not arguing that dissemination of
classified info ought to be legal, but as you move away from things
like locations of troops or blueprints of specific aircraft or
whatever, into more generic topics like entire classes of technology,
I think you're going down a slippery slope...)

The main reason that all of this went away though was that I think it
was Clinton that specifically granted an exception to cryptography
software from ITAR.  The concerns were that the cat was basically out
of the bag anyway, and consumers were potentially going to be harmed
by things like web browsers getting distributed with 40-bit key
limitations.  Sure, secure browsers were also probably available to
people who knew the right links to click, but do we really want
somebody reputable like Google to end up having to have a link on
their website for a non-secure version of their software, and where
the non-secure link just gives you a download, while the secure link
makes you jump through hoops to verify your location/etc?  The popular
use of SSL for entering credit card info on e-commerce sites really
was what drove it IMO.

-- 
Rich

Re: [gentoo-dev] Guidance on adding kernel config checks to ebuilds

[Rich Freeman]

On Mon, Sep 27, 2021 at 1:44 PM Robin H. Johnson  wrote:
>
> I think we need to strip out a lot of the crap about trying to detect
> things in the stuff being built, and reduce the check to the simplest
> possible form:
> $ time zgrep -w CONFIG_PACKET /proc/config.gz
> CONFIG_PACKET=y
>

Sure, just please don't strip out the part that actually does what you
just did - check the running kernel.  I don't think we should assume
that the user has the configured+prepared sources for a kernel just
lying around all the time.  I don't build in /usr/src (which is
apparently discouraged by upstream anyway), so /proc/config.gz is
really the only reliable indication of how things are setup.  Well,
that or a config file in /boot which I'm sure others don't use (but
make install puts it there).

-- 
Rich

Re: [gentoo-dev] Guidance on distributed patented software

[Rich Freeman]

On Sun, Sep 26, 2021 at 1:09 PM Peter Stuge  wrote:
>
> There's not neccessarily a conflict between a patented idea and a
> public domain implementation of that idea.

It kind of depends on what you mean by "conflict."

>
> Take a fictional example:
>
> You and I independently invent the same thing. You patent it, then
> write and publish an LGPL implementation. I, ignorant of your
> accomplishment, later write and publish a different implementation,
> placing it in the public domain.
>
> You having a patent doesn't neccessarily matter to me publishing my
> implementation.

Sure it does.  Depending on the claims of the patent, your publication
of the completely independent implementation is basically a public
confession that you violated the original patent.  Now, if you didn't
obtain any commercial value from it and were ignorant it is pretty
unlikely that somebody would actually go after you for it or get
damages, but you can infringe on a patent while being completely
ignorant of it and doing all your own independent work.  The patent
covers the idea, no matter how many times it is independently
invented.

> What users have to do is determined by the terms set forth in the
> patent. E.g.: the QR-code patent has (I believe) not expired yet but
> has always permitted using the idea without any explicit license under
> the condition that all use is actually spec conformant.

Generally patents, like copyright registrations, do not have "terms"
for how you are allowed to use the patented technology.  The patent
just makes claims as to what is covered.  For something completely new
those claims often are pretty broad covering anything somebody could
imagine the technology being used for.  For something repurposed the
claims tend to be narrow, as the inventor didn't actually come up with
the original idea but claims the new application of it is novel in
some way, and if you use it for anything else you're more likely to be
free and clear.

What you are talking about is a patent license, which is similar to a
software license.  You don't "copyright software under the GPL" - you
just copyright the software full stop.  Then, as a separate act, you
issue however many licenses you wish to anybody you wish allowing them
to do things that would otherwise infringe on the copyright.  In the
same way all a patent does is stake a claim on an idea, and then any
licenses are a completely separate matter.  If a patent holder wants
to sue somebody they would point to what they're doing and then to the
patent, and the court would determine if the action infringed, but the
defense would be free to provide any license they believe they
legitimately have from the patent holder that makes the action legal.

> You seem to expect some due diligence from libtomcrypt authors before
> having decided to publish their work and I don't find that reasonable.
> I hope I've managed to explain why?

So, I can't speak for libcrypt at all specifically, but I wouldn't
assume that somebody has actually done due diligence before publishing
anything.  For all you know the authors of some random piece of
software live someplace the patent is unenforceable and don't even
care if it infringes, or they're protected by anonymity online.  Back
in the PGP ITAR days I believe somebody went through some loopholes to
publish the software outside the US, and it is probably debatable
whether that was legal under US law, but presumably the people who did
it didn't care, and enforcement was unlikely at all, and especially
unlikely if you didn't have plans to visit the US after bragging about
distributing it.

Really the same is true of software patents in general.  For random
individuals other than your own conscience it is pretty unlikely for
there to be any kind of enforcement.  For a business or concern of any
size though it obviously is a risk that is probably not worth taking.
This is arguably one of the value-adds a commercial distro could add:
providing an assurance that anything in the distro is systematically
and competently screened.  Whether commercial distros actually due
this in practice is something I don't know, though if they make
promises and have the appearance of a decent program it might
accomplish the CYA needs of due diligence for most companies...

-- 
Rich

Re: [gentoo-dev] Guidance on distributed patented software

[Rich Freeman]

On Mon, Sep 20, 2021 at 12:46 PM Alec Warner  wrote:
>
> Could we add some text to the license concepts covering patents? It
> seems to have been omitted?
> Is my understanding of how we manage patented software correct?

I think you have the gist of it.  Is there actually anything in the
repo these days which is patent-encumbered?  I realize this is a
little tangential, but I think this is probably why we don't have a
well-thought policy: it just doesn't come up much.

The situation comes up less often since everything is copyrighted by
default, but software patents in FOSS are relatively rare.  (Partially
because they're such a minefield that it discourages even creating
FOSS in the first place.  Partially because they're such a minefield
that people tend to favor non-encumbered algorithms for things that
are commonplace now.)

Things that used to be patent-encumbered that were prevalent in FOSS
in the past include:
1. The GIF file format.
2. FAT-based filesystems.
3. MPEG-related codecs (codecs might be a space where patents are
still relevant).
4. RSA

I'm sure there are others I'm not thinking of offhand.  All of these
helped drive adoption of more open standards, which is why we don't
run into this stuff as often.

Another topic like this are encryption keys like for DVDCSS and so on.
Those fall outside both copyright and patent law, but are legally
troublesome.  Then there are export controls like ITAR/etc - less of
an issue today but might still apply to some things.

-- 
Rich

Re: [gentoo-dev] [GLEP78] Updating specification

[Rich Freeman]

On Mon, Sep 13, 2021 at 5:02 PM Michał Górny  wrote:
>
> On Mon, 2021-09-13 at 12:08 +0200, Ulrich Mueller wrote:
> >
> > Also, IIRC one of the goals of the format was to allow partial
> > download
> > of metadata. That will only work if the Manifest file will be the
> > first
> > file in the archive (or at least appear before the image archive).
>
> I disagree.  This is solved by having detached metadata signature -- you
> can do a partial fetch and verify the metadata directly.
>

Another option I've tossed out there in the past is having a content
hash of the metadata and putting that in the filename.  That obviously
won't tell you anything about the contents of the file without reading
it, but if you're looking for a file with specific metadata you could
predict its filename.  This was intended to work with having multiple
hashes for the same file using subsets of the metadata, using symbolic
links.

The thinking here is that you'd just hash a subset of metadata useful
for identifying what file you'd want to download, such as CHOST,
linked dependency versions, use flags, etc.  You'd probably hash it
with/without stuff like use flags so that you could either take a shot
at getting the file exactly configured how you want, or accepting a
version with any set of flags.

Of course, this idea goes in direct opposition to your statement about
not wanting to specify the filename.  I get that argument.  The intent
here was to allow portage to go hunting through trusted repositories
to find packages it can use without having to sync a lot of data - if
you know the exact filename then a simple GET tells you if it is there
or not.

-- 
Rich

Re: [gentoo-dev] News item for eudev deprecation

[Rich Freeman]

On Mon, Aug 23, 2021 at 10:46 AM David Seifert  wrote:
>
> Let assume the counterfactual for a moment here: We retained the
> USE=systemd flag for all unit files and what not, so people can cleanse
> themselves of the systemd units etc. without resorting to INSTALL_MASK.
>
> How would USE=-systemd have prevented this situation? USE=-systemd would
> randomly mv and sed random files so the "systemd-" prefix doesn't show
> up?

So, I think using USE=systemd to control installing units is a bad
idea, for the same reason that it is a bad idea for controlling init.d
scripts.  It results in users having to rebuild half their system just
to get those files installed if they later need them.

However, the argument would be that if we had used USE=systemd to
control installing units, then users wouldn't set an INSTALL_MASK, and
thus when udev comes along it would still install everything just
fine.  I doubt we'd have it rename anything - the systemd- prefix
would still apply, but since there are no INSTALL_MASKs then it
wouldn't cause any issues.  The issue isn't systemd in the
filenames/paths, but users attempts to keep things from being
installed with those names/paths.

I'm not sure what exactly udev installs, but obviously install masks
might or might not cause harm depending on how specific they are.  If
they are just for "*systemd*" then that would be pretty likely to
clobber important stuff.  If it is just targeting systemd units in
/etc/systemd/system then that seems pretty unlikely to harm anything
if you aren't running systemd as your service manager.

I notice systemd installs udev to /lib/systemd/systemd-udevd and that
would be probably the path most likely to cause issues.  Most of the
rules are under /lib/udev and so on, and there are things in the
generic lib directories.  However, I'm using systemd and not the
standalone udev ebuild - it might do some things differently.

IMO if users really just want to get rid of unit files they're
probably better off masking /etc/systemd/system and
/lib/systemd/system.  Most of the other stuff in that path is
installed by systemd itself, which users won't have if they're not
using it.  I get that there are a lot of strong opinions in this area,
but the issues that can arise probably aren't worth the fuss except in
very extreme situations where every inode counts/etc.

-- 
Rich

Re: [gentoo-dev] News item for eudev deprecation

[Rich Freeman]

On Mon, Aug 23, 2021 at 10:36 AM Ulrich Mueller  wrote:
>
> > On Mon, 23 Aug 2021, Anthony G Basile wrote:
>
> >>> **WARNING**
> >>>
> >>> If you happen to have an INSTALL_MASK with a blanket "*systemd*"
> >>> glob, you will inevitably break your system. sys-fs/udev contains
> >>> "systemd" in some of its filenames, hence a blanket filter rule will
> >>> likely lead to a non-functional udev installation.
> >>
> >> Will an INSTALL_MASK of "/usr/lib/systemd /etc/systemd" cause any
> >> issues?
>
> > I have not tested, but I think so since "systemd-" is used as a prefix
> > for files installed by sys-fs/udev.
>
> So, we've abandoned the systemd USE flag, and I remember that one of
> the arguments was that users could use INSTALL_MASK for precisely the
> above mentioned directories.

Well, the argument is that we don't use USE flags to prevent packages
from installing small text files.  It is the same reason we don't have
an openrc USE flag to control installing init.d scripts.  We're now
talking about pretty far back in history but I think this was a
general guideline before systemd even came along.

> Now the message is that users' systems will be broken if they had
> followed our previous advice? Seriously?

Did we ever officially advise people to use INSTALL_MASK at all?  I
thought that was mostly a "you can keep the pieces if you break
things" option we provide.  IMO the risks of people misusing it are
far greater than the possible harm of having a few hundred small text
files installed on their system, but it is there if people really want
to use it.

However, having used the option in the past shouldn't hurt anybody.
It only impacts people if they use it when they install udev, hence
the news item.

-- 
Rich

Re: [gentoo-dev] About the 'eapi' file in profile directories

[Rich Freeman]

On Sun, Aug 1, 2021 at 5:54 PM Ulrich Mueller  wrote:
>
> Quoting ciaranm [1]:
>
> | > "EAPIs whose value starts with the string paludis- are reserved for
> | > experimental use by the Paludis package manager."
> |
> | Don't tell anyone, but that's mostly just in there because some people
> | insisted that EAPIs were numbers (and thus comparable), so I wanted an
> | explicit mention of one that wasn't.
>

The other aspect of it is that they cannot be assumed to be ordered.
You could at some point in the future have a branching hierarchy of
sorts where some EAPIs contain some features and others contain
others, with neither being a superset of the other even accounting for
deprecation.

I get that we haven't generally done that historically, but you
shouldn't be doing ordered/range/etc comparisons with EAPIs (ie "if
EAPI < 5 do this, else do that").  Such things should always be based
on explicit values ("if EAPI in (1,2,3,4) do this, else do that").

-- 
Rich

Re: [gentoo-dev] [RFC] Removing SHA512 hash from Manifests

[Rich Freeman]

On Sun, Jul 25, 2021 at 11:23 AM Ulrich Mueller  wrote:
>
> We can reiterate when there are indications that SHA512 would be broken.
> (Then again, the same applies to BLAKE2B.)

Unless both are broken at the same time you'd also have the advantage
of not having to try to scramble to figure out whether anything was
compromised.  I get that typically hash functions are first broken in
a way that makes them very difficult to exploit, but that isn't some
sort of guarantee.  In the very unlikely event that somebody comes up
with a preimage attack against one of the functions, it would be even
more unlikely that an attack would be devised against both.

Sure, we're talking about low risks here, but we're also talking about
low cost and security.  When security is this cheap, why not have it?
I mean, if people didn't care about this stuff they wouldn't bother
migrating off of md5, and you'd have critical software like source
code control using broken hashes like sha1.

-- 
Rich

Re: [gentoo-dev] Declare the type of source

[Rich Freeman]

On Mon, Jun 28, 2021 at 11:58 AM Agostino Sarubbo  wrote:
>
> On lunedì 28 giugno 2021 17:07:57 CEST Michael Orlitzky wrote:
> > If the package declares a dependency on e.g. virtual/c-compiler, ago
> > would want to re-test it whenever a new version of any compiler is
> > released that satisfies virtual/c-compiler. Conversely, if the package
> > doesn't require virtual/c-compiler, we may assume that it doesn't
> > compile C code, and is not affected by the new version.
>
> I need to admit that your solution is more simplest because there is nothing
> to implement.
>
> We can create a new category (like virtual) called tinderbox, then for example
> we could have:
> tinderbox/c
> tinderbox/c++
> tinderbox/go
>
> and so on.
>
> Those tinderbox 'packages' added as DEPEND must not pull a default compiler or
> so, instead they will not pull anything.

This seems unnecessarily complex.  Why not make them REAL virtuals.
If your package depends on any C compiler then have it pull in
virtual/c-compiler (or whatever we want to call it).  If your package
depends on gcc explicitly, then just depend on gcc.  With the system
of empty virtuals you provide you then need to have logic for what the
virtuals "really" mean since they don't actually pull in anything, and
they're also useless for actual dependency-resolution as a bonus.

> They are there with the purpose of show the output of something like:
> equery depends tinderbox/c

The problem with this is that if something works with gcc and not with
clang, and clang changes, you test it anyway, because you have these
hard-coded definitions of what "c" is.  If you use real virtuals, then
you just find all the reverse deps of clang and that is what you need
to test, because they're just normal dependencies.

You don't actually have to implement the full removal of the @system
special logic to do this.  You can specify things that are in @system
as dependencies even if our policies don't currently require it.
Listing two paths to the same dependency doesn't hurt anything as far
as I'm aware.

-- 
Rich

Re: [gentoo-dev] Declare the type of source

[Rich Freeman]

On Mon, Jun 28, 2021 at 9:46 AM Michael Orlitzky  wrote:
>
> On Mon, 2021-06-28 at 15:00 +0200, Agostino Sarubbo wrote:
> >
> > Instead, imagine that each ebuild declares a variable called SOURCETYPE ( or
> > similar, or in metadata.xml if you prefer ) and with a tool like equery/eix 
> > we
> > are able to get the list of all packages that compiles C code.
> >
>
> I think all you are really asking for is that we stop omitting a random
> subset of @system from *DEPEND.
>
> This is long overdue, for many reasons, but in particular it would
> force us to declare a dependency on a C compiler if one is needed and
> allow you to re-test only those packages that use a C compiler.

++ - this would also support parallel building of @system.

Obviously we'll still need a core set of packages needed for
bootstrapping/etc, but there is no reason @system couldn't just be
another virtual.

You could also have convenience virtuals for things like the C
toolchain and so on.  This will both support alternate implementations
and avoid having to have laundry lists of deps in every ebuild.

A simple way to transition would be to create a system virtual and add
it to all ebuilds, but ask that this be removed in future updates in
favor of more specific dependencies.  Over time then the tree would
move to specified true deps.  Catalyst could still use a virtual as a
target for bootstrapping stages.

Another tool that would be useful is what some other distros do - use
mount namespaces/etc to allow build systems to only see parts of the
filesystem (down to the file level) that are specified in
dependencies.  This would basically eliminate unspecified or automagic
dependencies, since anything not specified basically doesn't exist at
build time.  If you didn't want to use mount namespaces then our
sandbox already allows limiting read access to only specified files -
we just configure it to allow read-only to everything for every
package.

-- 
Rich

Re: [gentoo-dev] [RFC] Should we switch IRC client defaults off Freenode?

[Rich Freeman]

On Wed, Jun 16, 2021 at 11:42 AM Michał Górny  wrote:
>
> What is the point you're making?  If GF decides to kick you from Gentoo,
> will you also claim that it was their legal right?
>

IMO the lesson to be learned here is that we ought to design Gentoo in
such a way that if the GF kicks us all from Gentoo, it isn't a big
deal at all.  The reason everybody is all upset is that the Freenode
change WAS a big deal and has been difficult to work through.
Obviously it wasn't THAT big of a problem, but ideally we should be
more resilient and just take things like this in stride.

People get deplatformed all the time these days, often from multiple
platforms at the same time.  Anybody sane doing anything online should
just expect it to happen to them at some point.

-- 
Rich

Re: [gentoo-dev] [RFC] Should we switch IRC client defaults off Freenode?

[Rich Freeman]

On Wed, Jun 16, 2021 at 7:28 AM Michal Prívozník  wrote:
>
> On 6/16/21 11:21 AM, Ulrich Mueller wrote:
> >> On Wed, 16 Jun 2021, Michał Górny wrote:
> >
> >> We've moved our official support channels from Freenode to Libera.chat.
> >> All that's happened afterwards pretty much proves that this was
> >> the right decision.  Maybe even to the point of saying that staying
> >> on Freenode would be dangerous to our users (e.g. because of the late
> >> NickServ impersonations, ops with bad reputation etc.).
> >
> >> However, there are still IRC clients in Gentoo that default to Freenode.
> >> I think the next questions we need to answer are:
> >
> >> 1. Should we be proactively changing the default network in IRC clients
> >> (provided they have one) from Freenode to Libera.chat
> >
> > Yes. IMHO Freenode is no longer a reasonable default.
>
> Why should we mangle with packages this way? I mean, to me Gentoo was
> one of the few distros that allowed real choice for users (systemd vs
> openrc, selinux or !selinux, etc.). We shouldn't be making choices on
> behalf of users. The best we can do is to open issues for whatever IRC
> client we have in portage to switch from freenode to something different.

++

This makes about as much sense as changing the default homepage on
browsers or the default address book on email clients.  All of these
things are used to interact with Gentoo.

At this point I think it should be pretty obvious to anybody
connecting to Freenode that something has changed.  If our concern is
users getting bad advice there, perhaps we ought to have more devs
participate on the gentoo-user mailing list where we seem to hand out
plenty of bad advice daily?  :)

I think that some point we need to just call this done.  I was a
proponent of switching networks, but at this point it seems like every
FOSS org I'm in has some kind of daily Freenode drama going on.  Just
about everybody has switched - just call it a win already...

I'm sure there will be people on Freenode 10 years from now, probably
in many of these same channels.

FWIW many IRC clients are already taking these steps.  Connecting to
Freenode on my phone IRC client is a bit like connecting to my UniFI
controller at home using a browser...

--
Rich

Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?

[Rich Freeman]

On Sat, Mar 27, 2021 at 5:43 PM Joshua Kinard  wrote:
>
> I kinda wish the Linux kernel had an ability to partially boot, init the
> networking subsystem, then fetch an initramfs image over TFTP like it can do
> with NFS Root.  That would solve the problem on my MIPS system(s) (and make
> install netboots better).  I've dug around, but this does not seem to be a
> capability currently in the kernel, unless I have over looked something.

Support exists, in the form of an initramfs.  Bear with me for a second.

An initramfs does not need to be large.  It is just software.  Sure,
they're often implemented in bash with all sorts of userspace tools in
the image, and that uses lots of space.  However, an initramfs could
be nothing but a small compiled binary and nothing else - a few
kilobytes even.

I'm not aware if anything like this has already been created, but you
could do something reminiscent of coreboot, and it would be
distro-agnostic.  Just build a minimal kernel with only support for
whatever you need to read a kernel and initramfs (from disk, network,
whatever).  Then build an initramfs that minimally fetches that kernel
- it could just be a tiny binary, or it could use tools like curl/etc.
Then you'd kexec the new kernel/initramfs which would do the full
boot, and this wouldn't have any size limitation.

An initramfs is just a way to automate the kernel boot process using
userspace code, vs needing to do a lot of fancy stuff in kernel space.
They can get a bit bloated when you make them full-featured/generic,
but if you just want a secondary bootloader you can shrink both the
kernel and the initramfs considerably, and use that to boot another
kernel.

If I were to look anywhere for something I could use out-of-the-box it
would be with coreboot.  That is a linux-based kernel/initramfs
designed to be a bootloader really.

-- 
Rich

Re: [gentoo-dev] rfc: usrmerge script

[Rich Freeman]

On Wed, Mar 24, 2021 at 11:09 AM William Hubbs  wrote:
>
> On Wed, Mar 24, 2021 at 08:48:41AM +0100, Michał Górny wrote:
> >
> > What really can help is reflinking on filesystems supporting that.
>
> What really can help is more info instead of being terse like this.
> Which filesystems support it?
>

According to Google right now: Btrfs, CIFS, NFS 4.2, OCFS2, overlayfs, and XFS

Lizardfs ought to, but doesn't currently.  zfs does not because clones
only are supported at the dataset level.

In any case, if you're using coreutils cp to do the copy, just pass
--reflink=auto.  Honestly, I have no idea why this isn't the default
behavior.  Who wouldn't want instant copy operations that consume zero
space (aside from metdata)?  If you're doing this in C or some other
language you would need to see if they have a library call to do it
easily - see man ioctl_ficlone.

-- 
Rich

Re: [gentoo-dev] timezone configuration - why copying, not symlinking /etc/localtime ?

[Rich Freeman]

On Mon, Mar 22, 2021 at 6:54 PM Andreas K. Huettel  wrote:
>
> > > Council decided years ago that we don't support separate /usr without
> > > an initramfs, but we haven't completed that transition yet.
> >
> > Which doesn't imply that we deliberately break things.
>
> That's right. Though we should at some point start thinking about an end of 
> support for separate usr without initramfs.
>

Just to clarify - it is already unsupported at a distro level.  It is
just that some individual packages still work with it.

The current Council decisions on the issue are (just providing for
general reference):

- "Since that particular setup may already be subtly broken today
  depending on the installed software, Council recommends using an
  early boot mount mechanism, e.g. initramfs, to mount /usr if /usr
  is on a separate partition."
  Accepted unanimously. [1]

- "The intention is to eventually not require maintainers to support
  a separate /usr without an early boot mechanism once the Council
  agrees that the necessary docs/migration path is in place."
  Accepted with 4 yes votes, 1 no vote, 2 abstentions. [1]

- "The Council agrees that all preparations for dropping support for
  separate /usr without an initramfs or similar boot mechanism are
  complete. A news item will be prepared, and users will be given one
  month to switch after the news item has been sent."
  Accepted with 5 yes votes, 1 no vote, 1 abstention. [2]

Current policy documentation:
Developers are not required to support using separate /usr filesystem
without an initramfs. [3]

1 - https://projects.gentoo.org/council/meeting-logs/20130813-summary.txt
2 - https://projects.gentoo.org/council/meeting-logs/20130924-summary.txt
3 - https://projects.gentoo.org/qa/policy-guide/filesystem.html#pg0202

-- 
Rich

Re: [gentoo-dev] New project: binhost

[Rich Freeman]

On Sat, Feb 13, 2021 at 8:51 PM Zac Medico  wrote:
>
> > 2.  Generate a hash of the file contents - this can go in the filename
> > so that the file can co-exist with other files, and be located
> > assuming you have a full matching set of metadata.
>
> For FEATURES=binpkg-multi-instance we currently use an integer BUILD_ID
> ensure that file names are unique.
>
> > 3.  Start dropping attributes from the file based on a list of
> > priorities and generate additional hashes.  Create symlinked files to
> > the original file using these hashes (overwriting or not existing
> > symlinks based on policy).  This allows the binary package to be found
> > using either an exact set of attributes or a subset of higher-priority
> > attributes.  This is analogous to shared object symlinking.
> > 4.  The package manager will look for a binary package first using the
> > user's full config, and then by dropping optional elements of the
> > config (so maybe it does the search without CFLAGs, then without USE
> > flags).  Eventually it aborts based on user prefs (maybe the user only
> > wants an exact match, or is willing to accept alternate CFLAGs but not
> > USE flags, or maybe anything for the arch is selected> 5.  As always the 
> > final selected binary package still gets evaluated
> > like any other binary package to ensure it is usable.
> >
> > Such a system can identify whether a potentially usable file exists
> > using only filename, cutting down on fetching.  In the interests of
> > avoiding useless fetches we would only carry step 3 reasonably far -
> > packages would have to match based on architecture and any dynamic
> > linking requirements.  So we wouldn't generate hashes that didn't
> > include at least those minimums, and the package manager wouldn't
> > search for them.
> >
> > Obviously you could do more (if you have 5 combinations of use flags,
> > look for the set that matches most closely).  That couldn't be done
> > using hashes alone in an efficient way.  You could have a small
> > manifest file alongside the binary package that could be fetched
> > separately if the package manager wants to narrow things down and
> > fetch a few of those to narrow it down further.
>
> All of the above is oriented toward multi-profile binhosts, so we'll
> have to do a cost/benefit analysis to determine whether it's worth the
> effort to introduce the complexity that multi-profile binhosts add.

The hash label on the filenames was also considered around
multi-profiles.  I figured that if you're going to be building
variants of packages you'd want to parallelize and hashes work better
for that.  Plus at least in concept you could potentially identify and
fetch files by hash using info already in the local repo without
having to sync additional metadata from the binhost.  User-contributed
binaries would also work better in such a world though for obvious
security issues that might just take the form of local user-generated
repos (allowing users to supplement the upstream repo with local
builds for a cluster, without having to mirror/reporoduce the entire
upstream.

I do get that multi-profiles aren't entirely an essential feature, but
when you consider stuff like X11 support or stable/unstable it seems
like we're probably going to have to provide at least a few variants
on packages for this to be practical.  You could just put each profile
in a separate repo, but then anything that doesn't actually change
across profiles gets built multiple times.  The hash-based solution is
also a form of deduping.

But, hey, it is great to see anything like this being done at all.
Walking before running isn't a bad thing!

-- 
Rich

Re: [gentoo-dev] New project: binhost

[Rich Freeman]

On Wed, Feb 10, 2021 at 12:57 PM Andreas K. Hüttel  wrote:
>
> * what portage features are still needed or need improvements (e.g. binpkg
> signing and verification)
> * how should hosting look like

Some ideas for portage enhancements:

1.  Ability to fetch binary packages from some kind of repo.
2.  Ability to have multiple binary packages co-exist in a repo (local
or remote) with different build attributes (arch, USE, CFLAGS,
DEPENDS, whatever).
3.  Ability to pick the most appropriate binary packages to use based
on user preferences (with a mix of hard and soft preferences).

One idea I've had around how #2-3 might be implemented is:
1.  Binary packages already contain data on how they were built (USE
flags, dependencies, etc).  Place this in a file using a deterministic
sorting/etc order so that two builds with the same settings will have
the same results.
2.  Generate a hash of the file contents - this can go in the filename
so that the file can co-exist with other files, and be located
assuming you have a full matching set of metadata.
3.  Start dropping attributes from the file based on a list of
priorities and generate additional hashes.  Create symlinked files to
the original file using these hashes (overwriting or not existing
symlinks based on policy).  This allows the binary package to be found
using either an exact set of attributes or a subset of higher-priority
attributes.  This is analogous to shared object symlinking.
4.  The package manager will look for a binary package first using the
user's full config, and then by dropping optional elements of the
config (so maybe it does the search without CFLAGs, then without USE
flags).  Eventually it aborts based on user prefs (maybe the user only
wants an exact match, or is willing to accept alternate CFLAGs but not
USE flags, or maybe anything for the arch is selected.
5.  As always the final selected binary package still gets evaluated
like any other binary package to ensure it is usable.

Such a system can identify whether a potentially usable file exists
using only filename, cutting down on fetching.  In the interests of
avoiding useless fetches we would only carry step 3 reasonably far -
packages would have to match based on architecture and any dynamic
linking requirements.  So we wouldn't generate hashes that didn't
include at least those minimums, and the package manager wouldn't
search for them.

Obviously you could do more (if you have 5 combinations of use flags,
look for the set that matches most closely).  That couldn't be done
using hashes alone in an efficient way.  You could have a small
manifest file alongside the binary package that could be fetched
separately if the package manager wants to narrow things down and
fetch a few of those to narrow it down further.

Or you could skip the hash searching and just fetch all the manifests
for a particular package/arch and just search all of those, but that
is more data to transfer just to do a query.  A metadata cache of some
kind of might be another solution.  Content hashes would probably
still be useful just to allow co-existence of alternate builds.

-- 
Rich

Re: [gentoo-dev] [News item review] Python preference to follow PYTHON_TARGETS

[Rich Freeman]

On Sun, Jan 24, 2021 at 2:09 PM Michał Górny  wrote:
>
> On Sun, 2021-01-24 at 13:53 -0500, Rich Freeman wrote:
> > On Sun, Jan 24, 2021 at 7:21 AM Michał Górny  wrote:
> > >
> > > For this reason, we have decided to change the default python-exec
> > > configuration to match PYTHON_TARGETS by default, in the eclass
> > > preference order, that is from the newest CPython version to oldest,
> > > with alternative Python implementations coming afterwards.  This change
> > > will be propagated via the configuration protection mechanism whenever
> > > dev-lang/python-exec-conf is installed or rebuilt due to PYTHON_TARGETS
> > > changes.  This will permit the users to interactively confirm
> > > the updates.
> > >
> > > If the new default is not correct for you, please use your preferred
> > > configuration update tool to discard or edit the new configuration file.
> >
> > Could we just spell out what the actual setting is?  That way if a
> > user accepts or rejects the change accidentally it is trivial to fix,
> > vs making them hunt through the installed files to do a diff...
> >
> > Nothing wrong with the instructions - I'd just add one line about what
> > setting controls this.
> >
>
> The exact paths are provided in the second paragraph.  Am I missing
> something?
>

No - the way this works makes sense now.  For some reason I missed it
on the first two reads, which makes me suspect others will as well.
It wasn't the location of the config file I missed, but the fact that
the eclass will just do what eselect python used to do, and thus
trigger config protection (which is at the end of paragraph 4).

For some reason when I read the section about discarding the changes I
was thinking that there was some config toggle to change this behavior
vs the old way things worked.  Instead the new behavior is
unconditional, but the updates it makes to the python-exec config can
be rejected.

-- 
Rich

Re: [gentoo-dev] [News item review] Python preference to follow PYTHON_TARGETS

[Rich Freeman]

On Sun, Jan 24, 2021 at 7:21 AM Michał Górny  wrote:
>
> For this reason, we have decided to change the default python-exec
> configuration to match PYTHON_TARGETS by default, in the eclass
> preference order, that is from the newest CPython version to oldest,
> with alternative Python implementations coming afterwards.  This change
> will be propagated via the configuration protection mechanism whenever
> dev-lang/python-exec-conf is installed or rebuilt due to PYTHON_TARGETS
> changes.  This will permit the users to interactively confirm
> the updates.
>
> If the new default is not correct for you, please use your preferred
> configuration update tool to discard or edit the new configuration file.

Could we just spell out what the actual setting is?  That way if a
user accepts or rejects the change accidentally it is trivial to fix,
vs making them hunt through the installed files to do a diff...

Nothing wrong with the instructions - I'd just add one line about what
setting controls this.

-- 
Rich

Re: [gentoo-dev] Re: [gentoo-dev-announce] We are finally shutting down CVS

[Rich Freeman]

On Sun, Dec 27, 2020 at 1:31 PM Alec Warner  wrote:
>
> On Sun, Dec 27, 2020 at 6:39 AM Ulrich Mueller  wrote:
> >
> > > On Sun, 27 Dec 2020, Max Magorsch wrote:
> >
> > > To access the old repositories you can use gitweb.gentoo.org instead.
> > > We have migrated all old cvs repositories to git. All of them are
> > > available read-only now at [0].
> >
> > I've just looked at
> > https://sources.gentoo.org/archive/cvs/gentoo.git/
> > and its commit history ends in 2004.
> >
> > Can you please reinstate CVS until a more accurate conversion is
> > available?
>
> I'm happy to make tarballs available (as discussed in a bunch of
> places on irc.) Is that sufficient or is there some particular
> requirement for the CVS protocol specifically?
>

The conversion tools all just work on a bare repository (RCS
files/etc) and don't use any CVS network protocols - you don't need
any daemons running.

-- 
Rich

Re: [gentoo-dev] Pushing to distfiles?

[Rich Freeman]

On Wed, Nov 11, 2020 at 7:23 PM Joshua Kinard  wrote:
>
> Forgive me for being a dunce, but what is the current procedure to push
> files to distfiles for distribution to the mirrors?  The devmanual is blank
> on this topic, and GLEP75 just talks about the motivations behind the change
> away from the flat directory format.  I am not easily finding anything that
> details how to get new distfiles onto the mirrors.
>

I thought that the whole mirror:// thing was discouraged.  For
anything else you just stick SRC_URI in the ebuild and the mirrors
should fetch it when they see it in the repo.

I just host stuff like that on my dev webspace, or better yet on
github or something else that will auto-tarball stuff.

-- 
Rich

Re: [gentoo-dev] New QA policy suggestion: Disallow "live-only" packages

[Rich Freeman]

On Wed, Nov 4, 2020 at 3:57 PM Joonas Niilola  wrote:
>
> On 11/4/20 10:43 PM, Rich Freeman wrote:
> >
> > Do you really think that users who just blindly run "emerge
> > --autounmask-write" are going to be both masking and unmasking
> > packages by hand (per your other email)?
> Just by following wiki...
>
> >
> > And how are they any better off if they do? They just end up in the
> > exact same state, except now we have zero control over their
> > experience instead of only a little control.
> Exactly, we should try to prevent this situation! Glad we agree.
>

Great, then no need to remove working packages that only have live
versions.  Just add snapshots where appropriate.

> >
> > Then why not do that, instead of removing things?
> Did you bother reading my reply?

Did you consider that somebody could read your email and not actually
agree with you?

> There's work being done towards fixing
> packages which seem to have hope. Now for some of these packages there's
> been last upstream activity 8 years ago. Is having a - ebuild
> justified there?

Does it work?  If so, then there is no harm.  If not, then just remove
it - you don't need a new policy to treeclean stuff that doesn't work.

> Also for some, upstream is dead, gone, making the
> package totally un-emergeable.

Then treeclean it.  Again, no need for a new policy here.  Stuff that
doesn't build is already grounds for removal if it isn't fixed in a
timely manner.

> Now imagine if we had a snapshot tarball
> in our mirrors, maybe it wouldn't need to be removed, if it still could
> be built.

Stuff that has no working SRC_URI still should be removed.  By all
means create your own upstream for it if you wish.

Removing that package years ago wouldn't have done anything to make a
snapshot available.

You're saying that live-only packages should be removed because they
could have snapshots.  I'm saying that if you want to maintain a
snapshot just add it and co-maintain - you don't have to remove
packages that don't have them.

-- 
Rich

Re: [gentoo-dev] New QA policy suggestion: Disallow "live-only" packages

[Rich Freeman]

On Wed, Nov 4, 2020 at 2:46 PM Joonas Niilola  wrote:
>
> On 11/4/20 8:52 PM, Rich Freeman wrote:
> >
> > 4.  If somebody finds one they probably have to add some random
> > overlay to their config, which causes this package to become
> > available, probably along with 47 other packages that can potentially
> > conflict with other things in the tree, all with zero QA.
> (It's common practice to mask the entire overlay then unmask the
> package(s) you need from it)
>

Do you really think that users who just blindly run "emerge
--autounmask-write" are going to be both masking and unmasking
packages by hand (per your other email)?

And how are they any better off if they do? They just end up in the
exact same state, except now we have zero control over their
experience instead of only a little control.

> > Certainly it is good to have snapshotted versions that get more QA
> > where appropriate, and that should probably be communicated.  When it
> > is done with an "or else" attached the result is likely to be that a
> > lot of stuff just gets removed, with little benefit...
> >
> Well my intention is to get up-to-date packages KEYWORDED properly, for
> better visibility and support.
>

Then why not do that, instead of removing things?

We should be careful with QA policies to avoid the concept that we
want somebody to do A, but we can't make them do A, so we tell them
they're not allowed to do B unless they also do A, and then we're
shocked when nobody wants to do B now.

If A is absolutely essential then maybe it is better to not have B to
ensure that A happens.  However, often A is a nice-to-have, and we end
up pruning stuff that really isn't that bad because we were just
trying to force devs to do something else.

If we want snapshots, then we should just add snapshots.  Removing the
package doesn't accomplish adding a snapshot.

-- 
Rich

Re: [gentoo-dev] New QA policy suggestion: Disallow "live-only" packages

[Rich Freeman]

On Wed, Nov 4, 2020 at 1:39 PM Marty E. Plummer  wrote:
>
> On Wed, Nov 04, 2020 at 06:24:39PM +, Alexey Sokolov wrote:
> >
> > What you're describing is live ebuilds, and I agree they are useful.
> > Joonas was talking about packages which have *only* live ebuilds, and no
> > other versions, and not even snapshots.
> >
> I must have misread, then, as I assumed a kill of all ${PN}-**.ebuild
> I suppose no live-only is reasonable for the most part, if there are
> non-live releases (not all software is at that point in their release
> cycles).

This still seems like a solution searching for a problem.  What is the
downside to having these in the tree?

If you remove them from the tree:

1.  They move from having minimal QA to zero QA.
2.  They don't get updated when larger Gentoo-wide things happen.
3.  People have to do searches to realize they even exist.
4.  If somebody finds one they probably have to add some random
overlay to their config, which causes this package to become
available, probably along with 47 other packages that can potentially
conflict with other things in the tree, all with zero QA.

Obviously if somebody notices that such an ebuild is broken it should
get treecleaned if the maintainer doesn't respond to a bug.  However,
it sounds like we're talking about a handful of packages after almost
two decades of allowing them.  Since they're masked, they don't do
anything unless somebody actually goes poking at them, and if they do
they probably file a bug and they'll go away.

Certainly it is good to have snapshotted versions that get more QA
where appropriate, and that should probably be communicated.  When it
is done with an "or else" attached the result is likely to be that a
lot of stuff just gets removed, with little benefit...

-- 
Rich

Re: [gentoo-dev] [RFC] Refactor display manager openrc init scripts to independent package

[Rich Freeman]

On Sun, Oct 18, 2020 at 4:17 AM Andreas Sturmlechner  wrote:
>
> I'm sure there is a way for the display-manager ebuild to migrate from old xdm
> configs on users' systems. How much do config and init scripts differ at all?
>

Couldn't you just use a symlink so that either script name works?
Then the news item can be about deprecating the old name.  There need
not be any rush to stop supporting it.

-- 
Rich

Re: [gentoo-dev] How to stabilize packages with frequent release cycles?

[Rich Freeman]

On Wed, Sep 16, 2020 at 4:08 PM Jonas Stein  wrote:
>
> Hi,
>
> > When the latest release remains 'latest ~arch' for less than 3 days,
> > stabilizing it after 30 days makes little sense.  After all, people with
> > frequent upgrade cycle will test it for no more than that, and people
> > with infrequent upgrade cycle may miss the version entirely.
>
> > Do you have any suggestions how we could improve this?
>
> At first we need a strict definition of "stable" and "testing", then we
> can discuss how to stabilize.
>

Not sure it is a definition issue so much that the concept doesn't fit
with these sorts of packages.  Normally the idea of stable is that
you're willing to trade speed for quality.

The problem is that in these sorts of packages you're often getting
neither.  For example, you're not going to have a more-bug-free
experience with youtube-dl if you run a two month old version, because
the APIs are all changing and you're just losing the cat and mouse
game.

IMO these sorts of packages probably shouldn't have stable versions at
all.  Then users will accept ~arch, and both know what they're getting
into, and also not get stuck with old versions that give them
suboptimal results.

Now, if somebody can come up with a better interface for that which is
cleaner than having to stick foo/bar in accept_keywords that would be
nice.  But that almost suggests another class of keyword entirely.
These packages aren't really "stable" - so much as stable not being an
option.

-- 
Rich

Re: [gentoo-dev] [RFC] Services and software which is critical for Gentoo should be developed/run in Gentoo namespace

[Rich Freeman]

On Wed, Sep 16, 2020 at 11:44 AM Alec Warner  wrote:
>
>  - repomirror-ci and all the CI stuff is on infra because mgorny is also on 
> infra! It's not like we set his stuff up for him; instead we gave him access 
> to all the infra repos and he had to write his own puppet configs and 
> whatnot. The benefit of course is that anyone on infra can bump the stuff and 
> login to the machines and debug...but its not exactly a low bar.
> ..
> I think traditionally it's been a slog for non-infra people to get infra to 
> host much of anything and due to difficulties with the all-or-nothing 
> approach we take with infra credenteials; can really set a high bar to host 
> much of anything these days.

Seems like a way to improve this would be better documentation and a
DIY infra testing platform.

First, document how to prepare a service for infra hosting.  Maybe
provide an example service.

Second, publish a tarball of a container/chroot that basically
simulates an infra host for testing purposes.  Provide instructions on
how to configure/run it.  Make it easy - edit this config file to
point to your repo, put the name of your service/host here, etc.

Anybody developing a service could then just follow the instructions
and then test out their service in a similar environment to what infra
uses.  Nobody needs to be trusted with any credentials.  Nobody needs
access to any special repos.  They can run the container on their own
host, and point it at their favorite git repo hosting service.

Once it is working all they need to do is give the link to their
repo/etc to infra.  Infra can then fork it and host it.  The
maintainer can submit pull requests as needed to infra.

-- 
Rich

Re: [gentoo-dev] [RFC] Services and software which is critical for Gentoo should be developed/run in Gentoo namespace

[Rich Freeman]

On Wed, Sep 16, 2020 at 4:17 AM Kent Fredric  wrote:
>
> On Mon, 14 Sep 2020 10:15:31 -0400
> Rich Freeman  wrote:
>
> > It might be easier to take smaller steps, such as having a policy that
> > "any call for devs to use/test a new tool/service, or any service that
> > automatically performs transactions on bugzilla, must be FOSS, and the
> > link to the source must be included in the initial communication, and
> > it must be clear what version of the code is operating at any time."
> > That is a pretty low barrier to those creating tools, though it
> > doesn't address the infra concern.  However, it does mean that infra
> > is now free to fork the service at any time, and reduces the bus
> > factor greatly.
>
> For the situation of things that take life before being part of infra,
> I think the least we can do is recognize their utility and importance,
> and at least, have infra *offer* some sort of shared location to run a
> deployment.
>
> That I think helps everyone, gives people a place to remove their own
> bus factor, but without mandatory strongarming.

This might be one option.  Another might be to try to have a more
"open infra" approach where core services like authentication are
provided by Gentoo, but available to 3rd parties.  This could allow
more services to be externally hosted, ideally with redundancy (not
just at the host level, but at the maintainer/software level as well).
The obvious downside to this would be chaos - we might have 3 list
servers, 5 bug trackers, 10 git repos, and so on.  However, if
anything did go down we'd have half a dozen potential replacements so
all anybody needs to do is migrate their own stuff to their chosen
copy.

The value add of Gentoo would be in central services like
identity/reputation and curation.  There might be 14 git repos out
there but Gentoo would control which ones end up on the master rsync
server and which rsync mirrors get advertised as being genuine, and
which list servers are official.

I realize this is a bit more tangential.  I just think that infra is
already a huge failure point, so having more stuff on infra actually
makes that failure point more critical.  A Gentoo where little is
hosted on stuff we own is much more resilient in the face of
legal/money/etc issues.  If Gentoo just becomes some blessed config
files, a website, and SAML then anybody could host the core from their
basement.

Maybe it is a good thing that core services aren't always hosted by infra?

-- 
Rich

Re: [gentoo-dev] [RFC] Services and software which is critical for Gentoo should be developed/run in Gentoo namespace

[Rich Freeman]

On Sun, Sep 13, 2020 at 11:52 PM Kent Fredric  wrote:
>
> But when you file a bug, you rely on bugzilla being maintained by
> Gentoo Infra, not some 3rd party.
>

I think the Council will need to consider where it wants to draw the
lines on something like this.  Here is my sense of how these sorts of
things come about:

1.  Somebody sees an opportunity for improvement and writes some code.
They interface it on their own with git/bugzilla/whatever and host it
on their own systems.  They use it for a while and improve it.

2.  They start to advertise it and call for testers.  This is nothing
more than a list post at the start.  It is completely optional.  A few
people start using it and find that it is helpful.

3.  It is still optional, but since it is helpful the 10% of the devs
who do 90% of the work in the relevant area (like arch teams/etc)
adopt it, which means that 90% of the work is using the new tool,
still self-hosted by the dev.  It might or might not have any source
published.

4.  The devs who are using the tool are also the ones maintaining all
the documentation for the official workflows, and they update it to
reflect what they're actually doing.  It might still be optional.  (In
fact, as far as I can tell from reading the docs nattka is still
optional - you could still just CC arch teams and so on yourself -
heck, arch teams can stabilize things even if you don't file a bug
though this is unlikely to happen much.)

5.  At some point somebody notices that 80% of the problems come from
the 10% of the work that isn't doing things the new way, and the new
way stops being optional.

Maybe somebody closer to these tools might want to correct something
above.  However, as an observer this is how these things seem to
evolve - it is a very bazaar-like methodology.

Keep in mind that rules don't make things happen - they prevent things
from happening.  The hope behind a rule is that if you dam off
something suboptimal the enthusiasm travels down some other path and
doesn't just die off.  So, where do you build the dam above?  Do you
let steps 1-4 happen and draw the line at step 5, which might just
mean that we accept the 80% of the problems that come from it being
optional until infra hosts it?  Do we draw the line all the way up at
1 and block any use of APIs in ways that are not explicitly approved?
Do we block it at step 4, so the arch team is using nattka for 90% of
the cases, and they just trade notes via email and nobody else knows
what they're doing because the wiki reflects a process nobody actually
follows?

I realize that I'm mostly pointing out things that can go wrong.

I don't think anybody would say that it is better not having infra
maintaining critical infra.  The problem is that the infra team
probably isn't going to officially host stuff way back at step 1.  A
random dev can't write a script and ask infra to start running it and
bug them 3 times a day to do a pull from their git repo.  Infra is
probably going to wait until something closer to step 3-4 before they
get involved, which means the tool is already being used for a
substantial amount of work.  I'm not sure if we even have a defined
process for getting new tools like these onto infra, or how we do
config/change management in these cases.

The council can say "don't use non-infra-hosted services as part of
essential processes" but what does that actually mean?  Does that mean
going up to step 3, so 90% of your arch testing bugs are going through
nattka, but it just isn't documented on the wiki?  Does it mean going
up to step 2, so some portion of them are - if so how do you prevent
it from going from 10% to 90% if the new tool works better?  Does it
mean not interfering at all with 1-5 but imploring infra and the
service maintainers to figure something out?  If the service isn't
expensive to run, those maintaining the service might not see much
benefit in moving it over, and infra is of course always
manpower-constrained.

It might be easier to take smaller steps, such as having a policy that
"any call for devs to use/test a new tool/service, or any service that
automatically performs transactions on bugzilla, must be FOSS, and the
link to the source must be included in the initial communication, and
it must be clear what version of the code is operating at any time."
That is a pretty low barrier to those creating tools, though it
doesn't address the infra concern.  However, it does mean that infra
is now free to fork the service at any time, and reduces the bus
factor greatly.

-- 
Rich

Re: [gentoo-dev] Please port your packages to Python 3.8

[Rich Freeman]

On Fri, Sep 4, 2020 at 9:06 AM Michael Orlitzky  wrote:
>
> It's easy to say "well this is not an issue because it can be solved by
> ..."
>
> If it's easy, get it added to the PMS and I'll agree with you.
>

Current Gentoo policy:

"Maintainers must not assume that dynamic dependencies will be applied
by the package manager. When changing runtime dependencies the
maintainer should revision the ebuild if the changes are likely to
cause problems for end users." [1]

Certainly having a discussion about whether this could change down the
road is reasonable, but keep in mind this would require package
managers to actually be changed, which requires code.

Out of the box portage has issues with dynamic deps[2] so it isn't a
solved problem on any package manager, let alone all of them.

In the interim, devs MUST revbump in these situations.  The Council
left some room for discretion, and as a result I end up having portage
rebuild everything on changed deps because frankly I don't trust
people to get it right, since if people can't see for themselves the
reason for a rule it seems to be a reason to ignore it.

The rule is also documented in the devmanual[3].

1 - https://projects.gentoo.org/council/meeting-logs/20151011-summary.txt
2 - 
https://wiki.gentoo.org/wiki/Project:Portage/Changed_Deps#Ebuild_Revision_Bumps
3 - https://devmanual.gentoo.org/general-concepts/ebuild-revisions/index.html

-- 
Rich

Re: [gentoo-dev] Please port your packages to Python 3.8

[Rich Freeman]

On Tue, Sep 1, 2020 at 7:02 AM Michał Górny  wrote:
>
> QA reports provide a list [2] and a graph [3] of packages needing
> porting.

These lists would be far more useful if they actually listed the
maintainer(s) of each package.  Then devs could just grep to discover
if any of their packages need fixing.

Otherwise I fear that you're just going to run into the same issue as
last time - most devs will just wait until you take the time to do
this and file bugs.

-- 
Rich

Re: [gentoo-dev] rfc: switching default udev provider for new systems to udev

[Rich Freeman]

On Mon, Aug 10, 2020 at 9:55 PM Joshua Kinard  wrote:
>
> On 8/10/2020 11:22, William Hubbs wrote:
> > On Mon, Aug 10, 2020 at 12:00:44AM -0400, Joshua Kinard wrote:
> >>
> >> If eudev is not broken, then why your proposed fix?
> >
> > bitrot and bus factor.
>
> Examples?

The sole maintainer of eudev is going to suddenly disappear before
getting a chance to tell anybody about the horrible security issue
they discovered earlier that day.

> You meant to say "has yet to come true".
>
> Elsewise, as long as that door remains open, then future tense is
> the correct tense.

Note that the disappearance of the sole maintainer of eudev has yet to
happen, but we absolutely need to be taking steps today because
everybody knows it will happen.  After all, it COULD happen, and so as
long as that door remains open the future tense is the correct tense.
:)

I find it amusing that everybody is still trembling in fear that
Lennart is going to take their shell scripts away from them in the
middle of the night.  But it isn't like anybody needs to touch that
cruft if they don't want to just because they're working on Gentoo, so
whatever rocks your boat.

Really though I'd just stick with "ain't broke don't fix it" as there
really is no reason to get into paranoid FUD.

-- 
Rich

Re: [gentoo-dev] rfc: switching default udev provider for new systems to udev

[Rich Freeman]

On Mon, Aug 10, 2020 at 8:16 AM Thomas Deutschmann  wrote:
>
> On 2020-08-10 14:07, Michał Górny wrote:
> > ...or a revert of a change made for change's sake.
>
> That's a bold statement for an unambiguous 7-0 decision as seen in
> https://bugs.gentoo.org/575718.

As one who voted yes, my rationale is already in the bug comments, and
I voted yes because it seemed to be the preference of most non-systemd
users on the mailing list.  I can't say whether this is still the case
but I'm guessing it is.  I don't think it is really a well-founded
preference but I don't really see a point in fighting it when people
can use whichever they prefer.

If the eudev bus factor drops from 1 to 0 and people get tired of
dealing with it, I suspect switching back will become more popular.
If that never happens that is fine too.  If people have unusual
configs not addressed by eudev, or just plain old good taste,  they
can always use udev or systemd.

If eudev were causing serious problems or holding back other projects
for some reason I'd feel differently.  Otherwise I tend to agree with
the sense that if you're going to make a change there should be a
reason.  The reason for the previous change was that a strong majority
had a strong preference.  Based on the tone of discussion I'm not sure
that has changed - there isn't as much vehemence in the discussion,
but I suspect that is mostly because most don't think this is likely
to happen so they don't bother to reply.

-- 
Rich

Re: [gentoo-dev] rfc: switching default udev provider for new systems to udev

[Rich Freeman]

On Sat, Aug 8, 2020 at 6:48 PM Roy Bamford  wrote:
>
> On 2020.08.08 23:22, Rich Freeman wrote:
> > On Sat, Aug 8, 2020 at 4:17 PM Roy Bamford 
> > wrote:
> > >
> > > With the declared aim from upstream of making udev inseparable from
> > > systemd, its not something to be done lightly.
> > > That's the entire reason that eudev was necessary.
> > >
> > > I would want some convincing that it was not another step on the
> > road
> > > to Gentoo being assimilated by systemd.
> >
> > So, I really could care less what the default is since it won't impact
> > any of my Gentoo hosts either way, ..
>
> I don't have a dog in this fight. Being old and cynical, I have static /dev,
> so I use neither.
>
> I'm interested in what's changed since the Council decision [1] to make
> eudev the default.
>

And you'll note that this is the one line in your post I didn't quote,
because it was about the only thing that you said which made sense.  I
wasn't in any way criticizing that point, and basically asked the same
question myself.

-- 
Rich

Re: [gentoo-dev] rfc: switching default udev provider for new systems to udev

[Rich Freeman]

On Sat, Aug 8, 2020 at 4:17 PM Roy Bamford  wrote:
>
> With the declared aim from upstream of making udev inseparable from
> systemd, its not something to be done lightly.
> That's the entire reason that eudev was necessary.
>
> I would want some convincing that it was not another step on the road
> to Gentoo being assimilated by systemd.

So, I really could care less what the default is since it won't impact
any of my Gentoo hosts either way, but this seems like a silly reason
to base the decision on.  IMO it was paranoid years ago when people
first brought it up.  Now it is even moreso considering that years
have elapsed without any grand systemd conspiracy being revealed.  If
their goal was to make it impossible to use udev on its own just to
mess with the 0.01% of Linux users who don't use systemd but do use
(e)udev, I'd think they'd have gotten around to it by now, or at least
they would still be talking about it.

William - can you actually elaborate on WHY you want to change things?
 Is there some problem with eudev?  Is it actively maintained and
generally tracking upstream udev commits (minus whatever they
intentionally don't want to accept)?

I'd be curious as to a list of the practical differences between the
two at this point.  For the longest time the only ones I was aware of
were the de-bundled build system, and the change in the default
persistent ethernet device name rule which was made in udev but not
made (by default) in eudev.  Perhaps at this point there are other
differences.

-- 
Rich

Re: [gentoo-dev] News item: Multiple root kernel command-line arguments

[Rich Freeman]

On Thu, Aug 6, 2020 at 4:19 PM Michał Górny  wrote:
>
> On Thu, 2020-08-06 at 23:03 +0300, Joonas Niilola wrote:
> > On 8/6/20 10:58 PM, Thomas Deutschmann wrote:
> > > Well, the purpose of this is to educate and avoid problems for
> > > headless/server users. But if so many devs seem to care about pushing
> > > maybe unrelated information and believe that avoiding that has much more
> > > value than avoid a problem like an unbootable system for just a few
> > > people (and for headless/servers this is a major problem in case you
> > > cannot trigger remote reboot)... ¯\_(ツ)_/¯
> > >
> > Yeah let's break some setups and make people change distributions instead.
> >
> > I'd support showing it. Weren't we all taught that too much
> > communication is better than no communication?
> >
>
> That's actually bullshit.  Too much noise leads to people stopping to
> read stuff, and losing important info as a result.  Compare: our mailing
> lists.

Well, we could solve that problem and the Foundation funding
challenges by switching the mailing list to a paid-superchat system.

The only thing that might bother some would be having to give me a
slot on the sponsor's page, but then again I would be paying your
salary at that point.  :)

More seriously, I think the simplest compromise is to just display the
news item to those with the relevant genkernel versions installed.
While the general principle of using UUIDs and such in boot lines is
important, this is better placed in the handbook, wiki, and so on.
The news should focus on what is actually changing, IMO.

-- 
Rich

Re: [gentoo-dev] News item: Multiple root kernel command-line arguments

[Rich Freeman]

On Thu, Aug 6, 2020 at 1:41 PM Mike Gilbert  wrote:
>
> On Thu, Aug 6, 2020 at 11:59 AM Thomas Deutschmann  wrote:
> >
> > On 2020-08-06 17:44, Michał Górny wrote:
> > > I'm not sure if you've noticed but there are people actively working
> > > towards removing stale news items and trying not to dump everything
> > > on once on a user freshly installing the system.  Don't you consider
> > > this a worthwhile goal?
> >
> > I don't see how this is conflicting.
> >
> > This news item can probably go away after 1-2 years.
> >
> > But for now, people who were just lucky will probably trigger this when
> > upgrading to genkernel-4.1 on their first reboot due to switched device
> > manager.
> >
> > But again: It's not a genkernel issue, so displaying that only for
> > people who have genkernel installed would miss a bunch of users.
>
> I would guess that most users do not utilize kexec at all, and this
> news item is irrelevant for them.
>
> Personally, I agree that this is not worth spamming every Gentoo user.
>

Has anything even changed with kexec?  Or is this an issue that has
been an issue for many years in kexec, that will suddenly become an
issue in genkernel?  In that case it is news from a genkernel
perspective, and something anybody with a correctly-booting system
fixed a long time ago if they're using kexec.

-- 
Rich

Re: [gentoo-dev] IPython 7.17 drops Python 3.6 support AKA upgrade reminder

[Rich Freeman]

On Sat, Aug 1, 2020 at 11:36 AM Aaron Bauman  wrote:
>
> On August 1, 2020 6:25:09 AM EDT, Lars Wendler  
> wrote:
> >
> >Honestly... seeing such replies from you or knowing that you do not
> >hesitate to hit other devs with your full QA deputy power once they
> >dare to touch python packages is not motivating in any way to even
> >consider helping you.
> >
> Lars, do you not recall the previous threads on this? The very same questions 
> were answered about tooling.

I'm sure everybody is tired of reading these threads over and over.
Simply saying that you answered these questions doesn't mean that
people will be satisfied with your answers.

> I see plenty of other devs and contributors touch Python packages with no 
> problems... Is it just you maybe?

You probably aren't being driven up the wall by these 50-reply threads
because only one dev has a problem with the approach that has been
taken in the past.

> Provide tooling? Not good enough.

Well, not if you don't advertise the tooling, and the tools don't
output maintainer info so that maintainers can quickly determine if
they're impacted.

> Provide a reasonable timeline? Not good enough.

Nobody has complained about timelines as far as I'm aware.

> Open bugs? We ignore them.

I'm not aware of ANYBODY who has complained about action being taken
after a bug was assigned to them.

Yes, some people ignore bugs.  They don't get much sympathy.  If you
file a bug and somebody ignores it for months, and then you depclean
their package, nobody is going to take their side.

The complaints you are getting are from devs who find out about a
problem with their package for the first time from a package mask,
perhaps due to a dependency/etc.

In any case, it sounds like we're now filing bugs, so hopefully we'll
see fewer problems like this the next time around.  Really, if you're
filing bugs I'd suggest leading with that as it will get you a LOT
more support than just pointing out the previous threads that nobody
seems to think were resolved but you.

-- 
Rich

Re: [gentoo-dev] IPython 7.17 drops Python 3.6 support AKA upgrade reminder

[Rich Freeman]

On Sat, Aug 1, 2020 at 7:09 AM Andreas Sturmlechner  wrote:
>
> On Samstag, 1. August 2020 12:15:18 CEST Rich Freeman wrote:
> > Just based on what is already happening, it seems like most devs don't
> > really care what versions of python are supported by their packages,
> > let alone the dependencies of their packages.

So, to start, I'll apologize as my original reply was worded a bit strongly.

I'm happy to hear that bugs were filed this time.  Obviously a lot of
fairly active devs were taken unaware by a bunch of package masks only
a few days ago, so that isn't being done consistently, but if we're
doing it going forward that is great.

>
> That's the definition of an unmaintained package to me.

I didn't say they were ignoring bugs.  I said they didn't care about
python.  It is ok not to care about python, or C, or glibc, or
whatever.  They're a means to an end for most devs.

Some devs like to focus on a tool, and some devs focus on the software
that uses those tools.  There is nothing wrong with either.  The key
is communication, which didn't happen enough (IMO) the last time
around.  Communication is what lets two people who have different
interests pool their resources.  Yes, some will ignore
well-intentioned efforts to communicate, but most won't, so it is
usually worth the effort.

> In case anyone still didn't know that list:
> https://qa-reports.gentoo.org/output/gpyutils/36-to-37.txt

So, if there are bugs filed then this list isn't all that important,
since maintainers will find out via bugs.  However, if you really want
lists like this to be directly useful to maintainers then you really
need to include maintainer names in them, because otherwise they're
very difficult to grep.  I doubt most devs know off the top of their
head the list of packages they maintain.

Somebody will no doubt link (again) repology or whatever.  Great, so
now we have two tables and we're asking humans to do a join on them.
Much better to just have the tools do this for us, and rather than
asking every dev to do it independently it makes more sense for the
first person that does it to just post the combined list.

In any case, this is moot if bugs were filed.

-- 
Rich

Re: [gentoo-dev] IPython 7.17 drops Python 3.6 support AKA upgrade reminder

[Rich Freeman]

On Sat, Aug 1, 2020 at 3:29 AM Michał Górny  wrote:
>
> I would like to take this as an opportunity to remind you to port your
> packages to Python 3.7 and 3.8.  According to our timeline [1], packages
> that are not ported by the end of the year are going to be last rited.
>  We would also like to switch to 3.8 in December.
>
> [1] 
> https://wiki.gentoo.org/wiki/Project:Python/Implementations#Implementation_support_timeline

So, has anybody given thought to publishing a list of packages that
still need to be updated, including their maintainers?

Or perhaps filing bugs?

Or is the plan to go ahead and watching nothing happen for the next
few months, then start masking hundreds of packages, and then watch
devs scramble to fix problems they didn't realize existed?

Just based on what is already happening, it seems like most devs don't
really care what versions of python are supported by their packages,
let alone the dependencies of their packages.  Expecting that to
change is just going to lead to a lot of frustration.

I don't think it is productive to just keep doing the same thing until
either the python team ragequits, or until we no longer have anything
that uses python left in the tree.

My guess is that a bit more communication will end up turning your
"enemies" into your allies, and ideally cut down on the amount of
masking/etc you have to do in the first place.  It certainly will be
less intrusive for users than having masks pop up and disappear.

-- 
Rich

Re: [gentoo-dev] Last rites: */*: More Py2 stuff

[Rich Freeman]

On Wed, Jul 29, 2020 at 4:09 AM Ulrich Mueller  wrote:
>
> > On Wed, 29 Jul 2020, Aaron Bauman wrote:
>
> > # Aaron Bauman  (2020-07-28)
> > # More Py2 only stuff. Plz see -dev ML for discussions
> > # Remove bindings, port to Py3, etc
> > # Removal in 30 days
> > [...]
> > app-office/lyx
>
> I have unmasked this one again:
>
> "All python scripts distributed with LyX should now be compatible with
> both python 2.x and python 3.x."
> https://www.lyx.org/announce/2_3_1.txt

Using package.mask in this way creates a TERRIBLE user experience.  It
causes users to look for alternatives and go through a lot of churn
expecting the package to be removed, when it turns out there is
nothing wrong and the package doesn't need to be removed.

Bugs are a much more appropriate way to handle these situations.  To
start with, they ensure the maintainer even gets notified at all.  A
package mask doesn't actually notify the maintainer at all - it
notifies anybody who has the package installed, and only when the host
it is installed on is updated.  It is possible (albeit less likely)
that the maintainer might not have it installed, and more likely that
they could have it installed in some container that they don't update
daily.

When the maintainer is able to fix the problem then users don't get
the churn of a package mask.

Obviously we're going to have packages that can't be migrated or which
aren't maintained, and these should be treecleaned as with any other
issue.

If for some reason bugs are just THAT difficult to create, why not at
least post the list on -dev-announce a few days before actually
implementing the mask?  You obviously have the list of packages if
you're masking them, and you're even posting it on the list.  So just
post it on the list ahead of time so that people can react before they
actually get masked.

It seems like we're creating a terrible user experience simply because
we can.  This seems to be going back to how things were 10+ years ago
when stuff broke for users often simply because nobody cared to even
bother with communication and QA.

-- 
Rich

Re: [gentoo-dev] Bug #733802, USE 'scp' now defaults to off in net-misc/openssh

[Rich Freeman]

On Sat, Jul 25, 2020 at 7:40 PM Joshua Kinard  wrote:
>
> This seems like something that needs a news entry, or
> at least a "heads up" on the mailing list?

Definitely not a "heads up" on the mailing list - that is not an
appropriate way to communicate anything to users - not even devs are
required to read this list.

The two appropriate ways to communicate something like this are
einfo/ewarn/etc or news.  Never hurts to use news.  Ideally I'd point
to a substitute, and I'd suggest one myself if I were aware of one...

-- 
Rich

Re: [gentoo-dev] RFC: Standard build environment variables

[Rich Freeman]

On Wed, Jul 1, 2020 at 9:36 AM Michael Orlitzky  wrote:
>
> On 2020-06-30 12:22, Matthew Thode wrote:
> >
> > I'd like to suggest allowing only approved variables in the build
> > environment, having portage unset all variables and setting only what is
> > needed (or configured).
>
> I think this is orthogonal to the problem I'm trying to solve. Even if
> all environment variables had to be whitelisted, ebuilds would still
> need to know how to use them when they happen to be defined.
>

Agree.  I'm not actually certain what that proposal was intended to
convey.  Are we talking about:

1.  Blocking anything that happens to be in the environment when
emerge is run?  (Ie 'CFLAGS="-O2" emerge -1 foo'?)
2.  Blocking any variable at all that isn't whitelisted by an ebuild
or eclass?  (ie CFLAGS in make.conf is ignored unless the ebuild
whitelists it)

I get how environment pollution can cause issues, but #1 is something
we've generally supported for a long time, and it is useful for
troubleshooting/etc or just trying out different things.  Maybe a
FEATURE flag could be used to control it to keep newbs out of trouble,
and you can just as easily pass that in the environment too.

I'm not sure that #2 adds a lot of value.  The default phase functions
probably already don't work well for exotic build systems, and
eclasses can of course take care of remapping for most of the popular
ones.  For one-offs some flag-o-matic or other eclass functions to aid
in remapping variables might be helpful in some cases if there isn't
already something there.

But in any case it isn't essential to what you're proposing.  It does
go along with it to a degree and is worth at least thinking about
(imo)...

-- 
Rich

Re: [gentoo-dev] */*: Mask Py2 only packages

[Rich Freeman]

On Fri, Jun 26, 2020 at 10:36 AM Aaron Bauman  wrote:
>
> On June 26, 2020 7:13:07 AM EDT, Rich Freeman  wrote:
> >> Of all the methods listed in the previous posts, the QA reports, etc.
> >> there is no excuse individuals can't find out if their package is py2
> >> only.
> >
> >None of those methods were posted until a day or two ago, and the
> >python team has done nothing to actually ensure all the impacted
> >maintainers are aware of them.  Perhaps a communication to
> >-dev-announce with the preferred approach would be better?
> >
>
> You should also look at qa-reports. Do we really need to *teach* others "how 
> to fish" here? Why can't folks just ask for assistance?

Just looked at it:

1.  I had no idea that a list of py2-only packages was listed there.
I don't think I've ever actually looked at that page.

2.  The report does not list maintainers, which means nobody is likely
to know they have a package on the list.

>
> See above. Qa-reports will output a very nice list (even a graphic!) of such 
> things. Anyway, yes, I do expect devs to understand their packages state if 
> they maintain it. Don't be so myopic.

Well, you can expect whatever you want, and then you can be frustrated
out of your mind when 95% of devs fail to meet your expectations.

Or you could just work with them where they're at and maybe get your
project completed more quickly and with less effort...

If you want people to look at a qa-report, maybe post on -dev-announce
and ask everybody to do it?  Most people aren't going to be following
all the tools used by the python team if they aren't python devs.

> >At least some devs here seemed surprised about the masks.  Did you try
> >filing a bug?
>
> Have you looked for said bugs?

Of course.  Do you think I'd invite such an obvious reply without
actually checking.

I just went to the first complaint on this list about there not being
a bug, and verified that there wasn't a bug.

As far as I can tell there is no bug for app-misc/golly.  If I missed
one feel free to cite it.

>
> >
> >Masking something for all users is basically like torturing a kitten
> >to get the attention of its owner.  It is a necessary step if the
> >package is actually to be removed.  I don't think it is even allowable
> >under our policies if no bug was filed.
> >
>
> Do tell where said policy is?

https://devmanual.gentoo.org/general-concepts/package-maintainers/index.html

Granted, a mask isn't a package commit, but I think the spirit of the
policy covers it.

In any case, there is no reason not to communicate with a maintainer
before touching a package.  That should involve something more than a
generic notice that everybody should become a detective to figure out
if they are covered by an upcoming change.  If you have a list of
impacted packages, then just file bugs against them.

>
> Nothing is really hard about masking packages for removal... honestly.

The complaint isn't that masks are hard on you.  The complaint is that
it bombards users with unnecessary warnings.

> The work comes in defending the position here for the few that complain.

And how are you enjoying doing all that extra work?  Would you prefer
if devs started opening up QA/Comrel bugs that you then have to
formally respond to?

Or maybe you could try notifying devs before masking their packages?

> If I filed a bug... they would complain or not respond... If I sent out a 
> dev-announce they would complain or not respond.

Sometimes, sure.  But at least you would have gone through due
process, and you're unlikely to get much push back.

And I suspect a number of those packages would actually get fixed.

>
> You see the fun here? Which method is effective? Mask a 100 packages for 
> removal... Someone complains... A few packages get saved and 90 get 
> removed... Life goes on.

Would you want a dev to just mask one of your packages if they saw a
bug in it, without bothering to open a bug?

-- 
Rich

Re: [gentoo-dev] */*: Mask Py2 only packages

[Rich Freeman]

On Thu, Jun 25, 2020 at 11:07 PM Aaron Bauman  wrote:
>
> On Wed, Jun 24, 2020 at 04:21:14PM -0400, Rich Freeman wrote:
> >
> > We're removing python2 around .  You can help us out by updating
> > any packages you have that use python2.  If you want to easily
> > identify these packages just do .
> >
> > I think the problem here is that we're basically telling maintainers
> > that the beatings will continue until morale improves.  Then we're
> > wondering why nothing is getting done.
> >
>
> I am thoroughly confused here. Some how you have completely changed your
> opinion from previous posts.

Perhaps we failed to communicate then.  My opinion has always been this:

I support letting the python team manage the versions of python
available - if people want legacy versions to stick around they need
to do something to make it happen.

HOWEVER, the python team would also find its job much easier if they
partnered with the myriad of package maintainers to accomplish their
goals, instead of just throwing them over the fence and then breaking
things for users to try to get everybody's attention periodically.

>
> Of all the methods listed in the previous posts, the QA reports, etc.
> there is no excuse individuals can't find out if their package is py2
> only.

None of those methods were posted until a day or two ago, and the
python team has done nothing to actually ensure all the impacted
maintainers are aware of them.  Perhaps a communication to
-dev-announce with the preferred approach would be better?

You can't expect every Gentoo dev to independently cobble together a
bunch of scripts to go hunting for py2 reverse deps.

> Ironically, it would be a very sad state if an individual doesn't know
> what Python interpreter their package is compatible with. This is the
> essence of "maintainer" status, correct?

Maintainers generally care about what the package does, and how it
does it is a means to an end.  Sure, some care more about the build
system and dependencies than others, and when working on a package you
need to pay more attention to such things.  However, I suspect most
package maintainers do not know off the top of their head the
dependency list of all their packages.

> Obviously, the myriad of tools, ML threads, and all the other "avenues"
> individual developers have taken to alert others simply doesn't work...
> until something is p.masked... people don't budge.

At least some devs here seemed surprised about the masks.  Did you try
filing a bug?

Masking something for all users is basically like torturing a kitten
to get the attention of its owner.  It is a necessary step if the
package is actually to be removed.  I don't think it is even allowable
under our policies if no bug was filed.

But if filing bugs is painful at least make things easier on
maintainers.  Post a list of packages and owners, for example.

It just seems like you're making things harder on yourself.  Gentoo
has done countless migrations like this and for whatever reason in the
past creating a tracker and blocker bugs hasn't been a problem.

I don't think the community will be served by having the python team
work itself into a frenzy until they ragequit.  Just give in, send out
a -announce post, and maybe cut your workload in half at least.  I get
that you seem to want to stand on some kind of principle that
everybody in Gentoo should care about the py2 migration as much as you
do, but it probably isn't going to happen.  Help everybody else help
you...

-- 
Rich

Re: [gentoo-dev] */*: Mask Py2 only packages

[Rich Freeman]

On Thu, Jun 25, 2020 at 2:45 PM John Helmert III  wrote:
>
> On Thu, Jun 25, 2020 at 07:32:04AM -0400, Michael Orlitzky wrote:
> > On 2020-06-24 16:08, Michał Górny wrote:
> > >
> > > $ git grep -l mgo...@gentoo.org '**/metadata.xml' | cut -d/ -f1-2 |
> > > xargs gpy-py2 2>/dev/null
> > find -L "${REPO}" \
> maint=$(pquery ${pkg} --one-attr maintainers | tail -1)

Great, so now we have 4 ways (and counting) to get 4 answers to this
question that hopefully will be mostly the same.

My point is more that it makes more sense for one person to just file
the bugs or send out the list so that maintainers can go fix their
packages, as opposed to playing a game where every developer in Gentoo
independently engineers a solution to the same problem.  If some
maintainers decide not to play the game, or play it and make a
mistake, then it ends up being the python team or the users who lose.

If a maintainer ignores a blocker bug for too long nobody is going to
shed a tear over some treecleaning.  Spending a bit more time on
communication might save a lot more time in cleanup.

-- 
Rich

Re: [gentoo-dev] */*: Mask Py2 only packages

[Rich Freeman]

On Wed, Jun 24, 2020 at 4:08 PM Michał Górny  wrote:
>
> $ git grep -l mgo...@gentoo.org '**/metadata.xml' | cut -d/ -f1-2 |
> xargs gpy-py2 2>/dev/null
>

I have no idea what gpy-py2 is, but I'll take your word for it.

In any case, the solution in this case is to send a nice email to
-dev-announce saying:

We're removing python2 around .  You can help us out by updating
any packages you have that use python2.  If you want to easily
identify these packages just do .

I think the problem here is that we're basically telling maintainers
that the beatings will continue until morale improves.  Then we're
wondering why nothing is getting done.

I'm not saying anybody has to do it a particular way - it just seems
obvious that the way we're doing it is more successful at getting
people upset than actually getting results.

Ideally you would just open a tracker bug and then per-package bugs
for every impacted package.  That would be the cleanest solution.  If
that is too painful then by all means do some email announcements, but
make it easy for devs to realize when they're missing something.

Having a package mask be the first time a maintainer finds out that
they have a problem isn't good.  Now, you can blame that on the
maintainer, or you can blame that on the python team, but either way
the users end up getting exposed to breakage unnecessarily.

-- 
Rich

Re: [gentoo-dev] */*: Mask Py2 only packages

[Rich Freeman]

On Wed, Jun 24, 2020 at 3:04 PM Andreas Sturmlechner  wrote:
>
> > On Wed, Jun 24, 2020 at 11:29 AM Rich Freeman  wrote:
> > > Sure, you can use the portage API to find this info.  However, that is
> > > as easy to do for a list of all impacted packages in the tree with
> > > their maintainers as for any individual maintainer to obtain this info
> > > for their own packages.
>
> I'm appealing to a more proactive maintenance, not in search for excuses why
> it is not like that. And ftr I don't mean trying to be "first!" on every
> upstream version bump; it is just that the python topic has come up often
> enough that it should have sparked individual head scratching at one point or
> another.
>
> > On Wednesday, 24 June 2020 20:40:58 CEST Alec Warner wrote:
> > You say there is not a straightforward way, but then you say there is an
> > api? :p
>
> grep all the things! But hey, there's even external tools to help you get an
> overview:
>
> https://repology.org/maintainer/rich0%40gentoo.org
>
> You're welcome.

I'm well aware of that.  That will get you a list of what you
maintain, but not which of those things use python2.  It is also
completely external.  (I do love that tool though - great for finding
bumps.)

grep is really not a reliable tool for parsing ebuilds.  The API is
really the right way to do it.

What I was getting at though is that just posting a big list up-front
will probably get more results than just telling everybody to try to
figure out if they're impacted.

(Also, I noticed that the list I sent out earlier contained some
overlay-only packages - probably best to re-run it on a clean
repository.  Since it uses the API it sees everything portage sees,
including overlays.)

-- 
Rich

Re: [gentoo-dev] */*: Mask Py2 only packages

[Rich Freeman]

On Wed, Jun 24, 2020 at 2:40 PM Alec Warner  wrote:
>
> On Wed, Jun 24, 2020 at 11:29 AM Rich Freeman  wrote:
>>
>> Sure, you can use the portage API to find this info.  However, that is
>> as easy to do for a list of all impacted packages in the tree with
>> their maintainers as for any individual maintainer to obtain this info
>> for their own packages.
>
>
> You say there is not a straightforward way, but then you say there is an api? 
> :p
>

Yeah - the number of people around here who have used it is pretty
small.  My point though is that if you've done that work it is easiest
to just do it once for everybody.

> Extend the existing QA report?
>
> https://qa-reports.gentoo.org/output/gpyutils/py2.txt
>
> There is a list of py2 only packages. We just need to add the maintainer 
> metadata?

Exactly.

I decided to get off my rear end and try to contribute a bit.  See the
attachments.  Script adapted from my ancient (ironically v2-only)
script to find missing slot op deps.  Contents are meant to inform,
and not to shame (mostly)...

-- 
Rich
app-accessibility/SphinxTrain-1.0.8 - +python_single_target_python2_7 - 
accessibil...@gentoo.org, so...@gentoo.org
app-accessibility/sphinx3-0.8-r1 - python_targets_python2_7 - 
accessibil...@gentoo.org
app-accessibility/sphinxbase-0.8 - python_targets_python2_7 - 
accessibil...@gentoo.org
app-admin/conkyforecast-2.24-r1 - +python_single_target_python2_7 - 
app-admin/denyhosts-2.9 - python_targets_python2_7 - proxy-ma...@gentoo.org
app-admin/denyhosts-3.0 - python_targets_python2_7 - proxy-ma...@gentoo.org
app-admin/syslog-summary-1.14-r1 - +python_single_target_python2_7 - 
lmip...@gmail.com, proxy-ma...@gentoo.org
app-admin/syslog-summary-1.14-r2 - +python_single_target_python2_7 - 
lmip...@gmail.com, proxy-ma...@gentoo.org
app-admin/syslog-summary-1.14-r3 - +python_single_target_python2_7 - 
lmip...@gmail.com, proxy-ma...@gentoo.org
app-antivirus/clamtk-6.03 - +python_single_target_python2_7 - 
conik...@gentoo.org
app-arch/cfv-1.18.3-r1 - +python_single_target_python2_7 - 
app-arch/deltarpm-3.6 - +python_single_target_python2_7 - 
app-arch/ipkg-utils-1.7.050831-r3 - python_targets_python2_7 - 
jnr...@gmail.com, proxy-ma...@gentoo.org
app-backup/bareos-17.2.9 - +python_single_target_python2_7 - msch...@gentoo.org
app-backup/bareos-18.2.8 - +python_single_target_python2_7 - msch...@gentoo.org
app-backup/genbackupdata-1.9 - python_targets_python2_7 - robb...@gentoo.org
app-backup/holland-1.0.10 - python_targets_python2_7 - 
app-backup/holland-backup-example-1.0.10 - python_targets_python2_7 - 
app-backup/holland-backup-pgdump-1.0.10 - python_targets_python2_7 - 
app-backup/holland-backup-random-1.0.10 - python_targets_python2_7 - 
app-backup/holland-backup-sqlite-1.0.10 - python_targets_python2_7 - 
app-backup/holland-lib-common-1.0.10 - python_targets_python2_7 - 
app-backup/holland-lib-lvm-1.0.10 - python_targets_python2_7 - 
app-cdr/burn-cd-1.8.0-r1 - +python_single_target_python2_7 - 
canutethegr...@gmail.com, proxy-ma...@gentoo.org
app-cdr/burn-cd-1.8.1 - python_targets_python2_7 - canutethegr...@gmail.com, 
proxy-ma...@gentoo.org
app-cdr/cdcover-0.7.4-r1 - +python_single_target_python2_7 - 
app-crypt/openssl-blacklist-0.5.3 - +python_single_target_python2_7 - 
ha...@gentoo.org
app-crypt/openvpn-blacklist-0.4-r1 - +python_single_target_python2_7 - 
ha...@gentoo.org
app-crypt/openvpn-blacklist-0.5 - +python_single_target_python2_7 - 
ha...@gentoo.org
app-crypt/pius-2.2.4 - python_targets_python2_7 - nickaristocra...@gmail.com, 
proxy-ma...@gentoo.org
app-crypt/ssh-multiadd-1.3.2-r1 - +python_single_target_python2_7 - 
k...@gentoo.org
app-crypt/virtualsmartcard-0.7 - +python_single_target_python2_7 - 
mgo...@gentoo.org
app-dicts/opendict-0.6.7-r1 - +python_single_target_python2_7 - 
wxwidg...@gentoo.org
app-editors/bluefish-2.2.10 - +python_single_target_python2_7 - 
app-editors/editra-0.7.20-r2 - python_targets_python2_7 - wxwidg...@gentoo.org
app-editors/leo-5.6 - python_targets_python2_7 - 
app-editors/pluma-1.22.2 - +python_single_target_python2_7 - m...@gentoo.org
app-emacs/pymacs-0.26-r1 - python_targets_python2_7 - gnu-em...@gentoo.org, 
pyt...@gentoo.org
app-emulation/crossover-bin-12.5.0-r2 - +python_single_target_python2_7 - 
r...@gentoo.org
app-emulation/crossover-bin-12.5.1-r2 - +python_single_target_python2_7 - 
r...@gentoo.org
app-emulation/crossover-bin-13.0.0-r2 - +python_single_target_python2_7 - 
r...@gentoo.org
app-emulation/crossover-bin-13.0.1-r2 - +python_single_target_python2_7 - 
r...@gentoo.org
app-emulation/crossover-bin-13.1.0-r2 - +python_single_target_python2_7 - 
r...@gentoo.org
app-emulation/crossover-bin-13.1.2-r2 - +python_single_target_python2_7 - 
r...@gentoo.org
app-emulation/crossover-bin-13.1.3-r2 - +python_single_target_python2_7 - 
r...@gentoo.org
app-emulation/crossover-bin-13.2.0-r2 - +python_single_target_python2_7 - 
r...@gentoo.org
app-emu

Re: [gentoo-dev] */*: Mask Py2 only packages

[Rich Freeman]

On Wed, Jun 24, 2020 at 2:18 PM Andreas Sturmlechner  wrote:
>
> The lack of curiosity for one's own packages' python compatibility is not just
> a py27 isolated issue, it was a big problem with py36 -> py37 with so many
> devs simply not filing that necessary stabilisation.

That suggests that if you keep doing what you're doing, you're going
to keep hitting your head against the wall.

Right now in Gentoo there isn't really even a straightforward way for
a maintainer to cleanly obtain a list of all the packages they
maintain, let alone whether they use python v2.

Sure, you can use the portage API to find this info.  However, that is
as easy to do for a list of all impacted packages in the tree with
their maintainers as for any individual maintainer to obtain this info
for their own packages.

I think that if you give the maintainers a bit more info, you'll find
them being more proactive about helping you out.  Basically you would
be helping them help you.

Otherwise you're going to mask a bunch of packages and run into a
bunch of upset devs, and as a byproduct we create a bunch of upset
users.

There is no reason to mask a package only to unmask it a few days
later.  Masks are a mechanism for deprecating packages so that users
take action.  They're not a substitute for devs talking to each other.

-- 
Rich

Re: [gentoo-dev] */*: Mask Py2 only packages

[Rich Freeman]

On Sat, Jun 20, 2020 at 1:36 PM Aaron Bauman  wrote:
>
> On Sat, Jun 20, 2020 at 10:32:28AM +0200, Ulrich Mueller wrote:
> > > On Sat, 20 Jun 2020, Aaron Bauman wrote:
> >
> > >> # Aaron Bauman  (2020-06-20)
> > >> # Py2 only
> > >> # Removal in 14 days
> >
> > I see these short deadlines quite often recently. Any reason why this
> > can't be the usual 30 days?
> >
>
> Hi, Ulrich. Yes, the deadlines are meant to speed up the process as we
> have *roughly* 1000+ pkgs which must be converted to py3 or removed
> before we can drop the interpreter.
>

Wouldn't it make more sense to just file bugs on ALL the impacted
packages, wait a few weeks, and then makes ALL of them at once, with
the regular 30d deadline?

Or if filing bugs is administratively difficult then just post a list
with packages and maintainers on -dev - this has been done for changes
in the past.

Right now it seems like some maintainers are finding out that their
packages are impacted for the first time by having their packages
masked.  That means that end-users get a package mask notice and start
taking action.  Then a few days later the package is unmasked.  Of
course, by then half the users have probably started migrating to
other packages - perhaps ones that are less suitable for them.

You seem to think that maintainers should know if they're maintaining
a v2-only package.  I suspect that most maintainers don't pay that
close attention to what versions of python are supported by their
various packages, and neither do most users.  If it runs then it runs,
and most don't care which interpreter is being used.  I get that it
impacts the python team but we need to make these issues more visible
to maintainers.

Maintainers often have an assortment of packages, and probably don't
realize when any particular one has a particular python compatibility
issue.

If it is difficult for you to identify all the impacted packages, why
would it be any easier for a maintainer who has probably spent less
time than you hunting down v2-only packages?

It seems like we really need a better solution here.  And just masking
a package without filing any bug beforehand doesn't really seem
in-line with policy.  We don't even do that for serious security
vulnerabilities.

-- 
Rich

Re: [gentoo-dev] Re: [gentoo-dev-announce] */*: Mask Py2 only packages

[Rich Freeman]

On Sat, Jun 20, 2020 at 4:36 AM Sergei Trofimovich  wrote:
>
> On Sat, 20 Jun 2020 00:43:03 -0400
> Aaron Bauman  wrote:
>
> > # Aaron Bauman  (2020-06-20)
> > # Py2 only
> > # Removal in 14 days
> ...
> > app-misc/golly
>
> If you decided to delete a maintained package you should file a bug against
> the maintainer. Otherwise they won't see the effect until mask hits the users.

So, I get that the python mess has been a ton of work for those
involved, and mostly thankless work.  You get complaints from people
who are interested in a particular package, but not with maintaining
the vast number of packages in an old python implementation that
probably isn't as interesting to people interested in python for the
sake of python.  As such I don't really question the removal of v2
from the tree.  I also don't question the phased approach of
progressively fixing, removing packages, etc.  This sort of thing
should be done at the convenience of the python team, who do a ton of
work behind the scenes that keeps the whole distro working.

I would suggest that process-wise there are some things that could be
done to make this sort of thing smoother in the future (and please
understand that this is intended as a lessons-learned and not so much
as criticism):

1.  Just follow the standard policies like ulm suggested and pick 30
days.  You're already doing something that is going to get complaints.
Waiting two weeks at this point won't make much difference.  Of course
exceptions can be made for pressing security issues/etc, and if these
exist just mention them up-front to deflect criticism.

2.  Help maintainers help you.  Ideally that means opening a ton of
bugs on day 1 of this whole mess against all the impacted packages.  I
get that this isn't easy.  An alternative might be to post lists of
impacted packages on -dev periodically, and when you do this stick the
maintainers on the list.  A lot of maintainers maintain a lot of
packages, and they probably won't notice if a package they maintain is
on the list, but if you stick their name on the list they can just
grep it.  I bet a lot of maintainers would pitch in if they just
realized they needed to.  I've seen lots of bugs that say "fix this,
oh and fix anything else you might happen to have with the same
problem."  The problem with this is that the people cleaning up python
probably have scripts to go detecting impacted packages, but everybody
else in the distro doesn't, and it doesn't make sense to have 100 devs
all trying to figure out if they have a package with a particular
issue, especially if the package has a quiet upstream that doesn't do
a lot of bumps.

I get that this won't fix the entire problem.  You'll get that
stubborn dev that just refuses to fix a bug.  When that happens don't
waste your time fighting WW3 - just point out that packages depending
on v2 will get masked on $DATE and move on, and ideally get the
Council to bless your decision.  If the Council doesn't bless the
decision or a compromise you can live with then just remove v2 from
the python project and make it maintainer-needed, and thus somebody
else's problem.  Don't put the weight of the world on your shoulders -
when it comes to actual work focus on the stuff that is directly
python and make the rest of us do the rest, but you need to spend a
bit of time around engagement.  A bit of up-front bug-filing or list
posts might save you a ton of harassing later.

The rest of us do need to appreciate that this is fairly thankless
work.  Maybe python v2 COULD be supported longer. But are YOU stepping
up to do that?  The existing members of the python team aren't
obligated to keep v2 in the tree.  Indeed, they aren't obligated to
maintain python at all.  Of course if some are willing to do the work
to keep v2 around then we should find a way to allow them to do so,
but I don't really see that happening.

Now I'm sure I didn't appreciate some things that happened behind the
scenes.  Accept what makes sense and reject that which does not.  And
by all means feel free to share lessons-learned that others might
benefit from.  I don't want to turn this into a criticism thread/etc,
so I'd encourage others to refrain from doing so...

-- 
Rich

Re: [gentoo-dev] [RFC] Codec project

[Rich Freeman]

On Fri, Jun 12, 2020 at 10:33 AM Alexis Ballier  wrote:
>
> What about /j #gentoo-media, discuss, join the current projects, get a
> few things done (there is a lot of choice there ;) ), maybe orphan
> unmaintained players/viewers, or check if they are maintained and hand
> them to a specific maintainer, and then see about merging or splitting
> all those projects *after* gaining experience and knowledge about the
> peculiarities of some of those packages ?
>

Probably best to leave the details up to those doing the work, but I
would suggest this as a guideline:

Keep the scope reasonable.  If you expand the scope to a point where
90% of the project members don't care about 50% of the project scope,
then you're setting yourself up for a repeat of the past.  You want
80% of the project members to be interested in 80% of the packages
being maintained ideally.  Sure, there will be little niches that a
subset are more interested in, but you want to keep it focused around
a core where coordination makes sense.  You can have different roles
in the project but it should still be one project.

If most of the project members aren't talking to each other about most
of the things they're doing in the project, then it isn't really a
project - it is just a category tag.  The point of a project is to
coordinate things that actually NEED to be coordinated or at least
benefit from it in some way.

-- 
Rich

Re: [gentoo-dev] Graphics Project disbanded [pkgs up for grabs]

[Rich Freeman]

On Sun, Jun 7, 2020 at 7:22 PM Philip Webb  wrote:
>
> 200607 Pacho Ramos wrote:
> > I think this is the list of completely unmaintained packages now,
> > indeed most of them, around 100.
>
> -- extract from list --
>
> > media-gfx/imagemagick : 200516
> > media-libs/giflib : 200312
> > media-libs/libjpeg-turbo : 200328
> > media-libs/openjpeg : 200328
> > virtual/jpeg : 200606
>
> There have been upgrades of all these in recent months :
> dates when I upgraded on my desktop system are added (the last yesterday).
> Surely, that means someone is maintaining them.
> Perhaps the culprits could own up (smile).
>
> As a long-time user, I find it disturbing
> that a huge list of packages should suddenly be declared unmaintained,
> esp as some of them -- eg above -- are likely needed by most users.

They were not maintained by identified developers before - that is the
point.  The only thing that is changing is that metadata is being
updated to reflect reality.  Now these packages will get more notice
and developers can set up and maintain them as needed.

The packages aren't being removed - just the project.

If any of the packages assigned to the graphics projects already had
individual maintainers then those would still remain after the project
is removed.

Put it this way - suppose we created a project called "dummy" with no
developers in it, and we assigned that project to all the packages
that are maintaner-needed.  Would that actually change anything?  XML
tags in metadata files don't maintain packages - people do.

This sort of thing has happened many times in the past.  Sometimes it
does result in packages getting treecleaned, but mainly when they have
other serious issues.  Popular packages aren't likely to get removed
this way - certainly not something like libjpeg or imagemagick and so
on.

-- 
Rich

Re: [gentoo-dev] [RFC] Concept of Projects - How to proceed?

[Rich Freeman]

On Sun, Jun 7, 2020 at 2:14 PM Jonas Stein  wrote:
>
> On 07/06/2020 03.43, Aaron Bauman wrote:
> > On Sun, Jun 07, 2020 at 01:49:28AM +0200, Jonas Stein wrote:
>
> > I will happily revert my change on the graphics project Wiki [..]
>
> Glad to read your offer. Yes, please do so.
>
> I think it would hurt the Gentoo project if single developers delete
> projects
>
> - without without informing the project members
> - without prior discussion (on gentoo-dev for example),
> - without vote/consent
> - without an organized shutdown (reassign bugs, archive things...).
>
> However we should continue to find a general solution for the problems
> discussed in this thread and find a general consent.

While I get what you're saying, I think it would also be helpful if we
just let people who feel they are actually impacted by changes like
this speak up for themselves, instead of assuming that they must exist
and that it is our duty to speak up for them.

Are you, directly, impacted in any negative way by this change?  If so
it would probably be helpful if you just explained the issue.

This really seems like a fairly uneventful change.  I do think it is
better to pre-announce changes.  However, I suspect that most of the
fuss is because a lot of people assume that a change like this must
have some kind of big impact, and for whatever reason all the people
who are being harmed by it are just afraid to speak up so we must do
so on their behalf.

I say this as somebody who used to raise a lot more hypothetical
objections to changes in the past.  I've since learned that it is easy
to over-react, and that when others are actually impacted by a change
they will tend to speak up.

I'm pretty sure in this case there was an organized shutdown - I doubt
they just removed the project without reassigning packages or bugs.
They were effectively already assigned to nobody as it was, since the
project was inactive.

I guess my point is that while this probably could be done in a better
way, I think it is likely to end up happening either way, so all
undoing it is going to do is send a lot of people two more rounds of
bugspam at best.  Or, it will result in one more round of bugspam and
then these packages continue to be unmaintained because nobody is
going to bother doing all the steps you're suggesting to get rid of it
in the future.  Easier to just leave the dead project around and let
users wonder why nobody pays attention to the bugs they open.

-- 
Rich

Re: [gentoo-dev] [RFC] Concept of Projects - How to proceed?

[Rich Freeman]

On Sat, Jun 6, 2020 at 7:49 PM Jonas Stein  wrote:
>
> our concept of "Projects" (Herds in the past) maintaining packages has
> several problems.

You might want to search the list archives as many of the issues
you've raised have been discussed extensively.  There was never a
complete push to fix things, but most project removals/etc have been
along the lines of what has been discussed.

While tangential, I'll point out that there have been similar
discussions around appropriate uses of eclasses and there are some
loose parallels for when eclasses make sense.

> * Many projects are too heterogeneous
>   Projects should only maintain either
>   a) many similar packages such as libraries (like Perl, Python) or
>   b) very few strong correlated packages (like KDE, Kernel, Xfce)
>
>   It makes no sense to group packages by usage as in
>   Science, Games, Theology, Sound, Netmon, Video, Electronics...

...graphics...

This is the gist of the outcome of previous discussions.  Projects
make sense when developers actually work TOGETHER to maintain things.
Nobody who works on qt is going to just upgrade one component of qt
without talking to the other maintainers.  It makes sense for those
packages to be managed by a project.

Historically a lot of projects worked more like "tags" as an
alternative way of grouping packages other than categories.  While
tags are a great idea projects are a terrible way to implement them.

> I think we should first find a consent about the following questions
> before someone deletes projects.

In general I'm a fan of announcing things like this BEFORE doing them.
However, I don't think they need pre-approval when they make sense.
If anything we haven't been doing it often enough.

I don't see any point in bringing back graphics though - if somebody
who actually intends to lead such a project wants to speak up on that
they should certainly do so, but it sounds like it is just a vestige
of an old herd that probably never worked as an actual team.

People reading the thread shouldn't think that this has anything to do
with the importance of the individual packages.  This is purely about
how devs are organized around them.

Some of what you wrote was more about our larger meta-structure and
how devs maintain packages in general.  That has also been discussed
quite a bit, like having a core group of devs who don't maintain any
individual packages and all they do is QA pull requests from a much
larger pool of individuals who do care about packages.  There are a
lot of pros and cons to that and I won't rehash the previous
discussions here.  That isn't to discourage anybody from doing so - I
mainly just want to point out that this stuff is in the archives.

-- 
Rich

Re: [gentoo-dev] Gentoo Council Election 2020 - Call For Election Officials

[Rich Freeman]

On Sat, May 30, 2020 at 6:09 AM Roy Bamford  wrote:
>
> We sill need more volunteers.
>

Not going to be running, so I'm happy to pitch in.

-- 
Rich

Re: [gentoo-dev] x11-base/xorg-server: No longer enabling suid by default.

[Rich Freeman]

On Tue, May 26, 2020 at 4:12 AM Haelwenn (lanodan) Monnier
 wrote:
>
> [2020-05-25 23:41:23+0200] Piotr Karbowski:
> > There are 3 common ways the xorg-server is started:
> >
> > - via XDM of some sort, usually forked as root, does not require suid,
> > systemd or elogind.
>
> Launching X as root and having it be suid is quite the same thing…
>

Sort-of.  An SUID X binary is a potential source of vulnerabilities
even if you never run it, since it is still sitting there and ready to
be exploited by somebody else.  It also gives a user more control over
how X is launched as root (command lines/control over stdin/out, etc).
When X is launched as root by something the user doesn't control it
reduces the attack surface somewhat.  And if you never launch X11 at
all it is just another unprivileged binary that can't do anything the
user can't already do with system calls.

In any case, setting suid on any binary is something that should only
be done if there is no other practical solution.  It certainly seems
like this shouldn't be the default, especially if it is available for
users to toggle if they wish.  We can always put out a news item when
this changes.  If elogind is already enabled by default on a profile,
then it doesn't make sense to ship X11 suid with that same profile
when it isn't necessary.  If a user wants to depart from the default
config to not use elogind then they can just change the USE flag on
xorg as well.

-- 
Rich

Re: [gentoo-dev] Re: [PR] ivy, mvn, sbt, gradle builders improvement for ebuild development

[Rich Freeman]

On Mon, Apr 20, 2020 at 2:07 PM Michael Orlitzky  wrote:
>
> On 4/20/20 1:31 PM, Patrick McLean wrote:
> >> Simply having things in ::gentoo does not affect anyone who does not
> use them.
> >
>
> You keep saying that, but the fact that dev-go/* is filled with trash
> that has your name on it prevents anyone else from doing a better job.
>

As far as I'm aware Gentoo policy does not prevent somebody from
re-implementing a package that is already in Gentoo.  Obviously it
could get pretty confusing if that happened.

IMO it isn't really worth worrying about, because right now the main
limitation seems to be a lack of people working on projects, not 25
devs who each want to re-implement go their own way...

-- 
Rich

Re: [gentoo-dev] keywording workflow

[Rich Freeman]

On Mon, Apr 13, 2020 at 4:12 AM Michał Górny  wrote:
>
> An example workflow is to:
>

Just picking this to reply to though this is more of a general comment
on the two recent keywords threads.

I get that this is Gentoo and we don't want to dictate how people do
things.  However, I feel like this is an area where we'd actually
benefit from more direction.

It seems like we're trying to support more different workflows for
doing keywording/etc than we even have developers doing keywording in
the first place.  It seems like we probably have 5 people at any time
doing actual arch testing but we're maintaining a lot of legacy
code/etc to support 487 ways of going about arch testing because we
don't want to upset somebody who probably doesn't actually do any arch
testing.  At the same time, we have no idea how the 5 people doing the
actual work are actually doing their work, so we can't reliably ensure
that their workflows don't break other than by making sure that we
don't accidentally break any legacy behavior in any way.

What I don't want to do is start a conversation where 27 devs
(including myself) try to tell the people doing a lot of keywording
work how to do their work.  What I would love to see is the people who
actually do keywording share how they actually go about it in
practice, and then maybe try to document some kind of best practice
around this and put it in the devmanual or in a GLEP or something.
Then we can give that workflow more of a first-class support in our
tooling and maybe worry less about others.

I know I was completely taken by surprise by the idea that most
keywording is done using tools these days, and that STABLEREQ isn't
supposed to be a thing now.  Not that these are bad things, but it
seems to have been organic and not really formally transitioned.  The
devmanual no longer mentions the bugzilla keywords, but it also
doesn't mention the bugzilla components and I didn't realize that you
couldn't just turn an existing bug into a stable request just by
adding STABLEREQ and copying arches.  Obviously now I know but my
point is more that it seems like this whole area would benefit from
some kind of suggested workflow.

Heck, this thread is also the first time I think I've seen "pkgcheck
scan --commit" mentioned as a thing.  I see that it was blogged about
a few months ago, but I've stopped following planet Gentoo since
Google reader died.  Maybe we need a planet Gentoo mailing list or
something.

I guess my point is that there seem to be a lot of improved workflows
out there, and we'd probably benefit from them being pointed out a bit
more and mainstreamed.  Those maintaining these tools would probably
benefit as well if more people were using them as intended and they
didn't have to maintain as much legacy compatibility simply because
many don't realize there is a better way...

-- 
Rich

Re: [gentoo-dev] zoom concerns

[Rich Freeman]

On Wed, Apr 1, 2020 at 8:18 PM Alessandro Barbieri
 wrote:
>
> I have concerns about the inclusion of zoom in ::gentoo. For me it's more 
> like a malware.
> From the hacker news feed you'll find out that:

I guess we could stick an einfo in the post-install messages, but if
you're joining a zoom meeting are you going to be any more secure if
you manually install the files instead?  I can't imagine that people
are going to stop attending meetings just because they picked the
wrong software to host them.  Plus a few of those concerns apply to
MANY packages - such as a lack of end-to-end encryption, or ever
having had a zero day.

I'm not intending to endorse Zoom here, but Gentoo isn't really
intended as a purist distro that will never include a package if it is
associated with a service that might collect user data and so on.  In
fact, we have many packages with these associations.  Ultimately users
can decide what they want to run, and we're just providing the files
in the most convenient and secure manner possible.  For example, when
the zero day is fixed if you're using Gentoo you'll benefit from our
security policy, while you would not if you had just manually
installed some files/etc...

-- 
Rich

Re: [gentoo-dev] network sandbox challenge

[Rich Freeman]

On Fri, Mar 27, 2020 at 7:33 AM Michał Górny  wrote:
>
> On Fri, 2020-03-27 at 11:29 +, Samuel Bernardo wrote:
>
> > Same question for unpack context when using directly the source
> > repository with vcs functions.
>
> VCS ebuilds generally suck, for multiple reasons.  We allow users to use
> them but with minimal support.  However, e.g. git-r3 supports local
> mirrors to resolve some problems.
>

Any guides on using that from an ebuild maintenance standpoint?  The
eclass docs make it appear more for local sysadmin use vs as a way to
distribute things, but I could be reading into it.

Usually my solution to VCS ebuilds when that is all upstream supports
is to just create my own github mirror of the repo, tag an appropriate
commit, and then fetch the resulting tarball directly as a non-VCS
ebuild.  Essentially I end up running my own private fork of upstream.
At least, this is what I do for actual releases that upstream can't be
bothered to release properly - for a live - VCS ebuild I just
point to upstream.

I don't believe Gentoo has any kind of recommended/standardized
solution for this, but having ebuilds point to my own private fork of
things obviously is non-ideal.  However, I think it is still more
transparent than just bundling up a tarball manually and sticking it
in my devspace since at least with my forked repo everybody can see
where it came from and what state it is in, and of course it is easier
to maintain.

If there is a more preferred way of doing this I'd be interested to
hear about it.

-- 
Rich

Re: [gentoo-dev] Fwd: [gentoo-commits] repo/gentoo:master commit in: app-office/calcurse/

[Rich Freeman]

On Fri, Mar 13, 2020 at 1:29 AM Chí-Thanh Christopher Nguyễn
 wrote:
>
> Alec Warner schrieb:
> > On Thu, Mar 12, 2020 at 9:54 AM Andreas K. Huettel  > > wrote:
> >
> > Someone needs to grow up here.
> >
> >
> > Meh, to me (someone who can't commit to ::gentoo) I have a few concerns 
> > here.
> > First, I don't see a lot of QA reverts on the gentoo-dev list. Is it common
> > practice to post reverts publicly? Second, I'm not aware that we would 
> > revert
> > for things like this. Most of the items you mention look fairly minor (maybe
> > the python comment looks impactful?) Why can't we fix these items in a 
> > future
> > commit, rather than revert? What did Patrice's commit break?
>
> If the issues are so serious that we have to prevent any breakage/regressions
> from reaching users, I guess an alternative response would have been to
> p.mask the offending new ebuild. Unless the commit caused some tree-wide
> breakage which I can't see here however.

Don't really want to comment on where the line should have been drawn
on this particular case, but the idea of reverting commits doesn't
seem particularly abhorrent, and certainly commits that don't create a
new ebuild couldn't be addressed with masking unless we want to impact
end users.

It seems like the drama here is mostly about how this ended up on the
lists vs just being a discussion between QA and the committer/etc.
Reading between the lines I'm not sure if it was ever intended to be
on the list at least initially.

If this was intended for public consumption it probably wouldn't hurt
to note why (hey, we're singling out this commit because it has this
error we've been seeing a lot of lately and you can see how this sort
of thing could sneak in...).  Otherwise it just seems like it causes
drama without actually achieving the desired impact.

-- 
Rich

Re: [gentoo-dev] Should we allow "GPL, v2 or later" for ebuilds?

[Rich Freeman]

On Tue, Feb 11, 2020 at 10:05 AM Haelwenn (lanodan) Monnier
 wrote:
>
> Maybe it could for now be a simple agreement on putting your code to
> the Gentoo Foundation under the GPL-2+ but it would be published under
> the GPL-{2,3,…}?
>

Well, if we were going to get people to start signing things I suggest
just sticking to the FLA since it actually was written by lawyers.

I attached a copy, but along these lines the key section is:
We agree to (sub)license the Contribution or any Materials containing,
based on or derived from your Contribution under the terms of any
licenses the Free Software Foundation classifies as Free Software
License and which are approved by the Open Source Initiative as Open
Source licenses.

That is, Gentoo would control the licenses, but they would have to be
FSF/OSI approved.  That doesn't mean that anybody could choose any
FSF-approved license - Gentoo would still have to do the licensing.
This is just a limitation on the grant of power from the original
author to Gentoo on WHAT licenses GENTOO can choose.

There is also a variant of the FLA that can further narrow down the
licenses that Gentoo gets to choose from, but IMO if you're going to
go down this path it makes sense to keep things flexible.  We could of
course just limit Gentoo to GPL v2+, and initially Gentoo does v2/3
and later Gentoo could revise to any later version of the GPL.  But if
for whatever reason the GPL falls out of favor then we can't adapt
futher.

Ultimately though anything like this involves giving up control.

For those interested in the FLA there is a license generator at:
http://contributoragreements.org/ca-cla-chooser/

You pick the terms (I used the defaults - which IMO are most
appropriate but not the only valid option).  It spits out an agreement
for you.


-- 
Rich


fiduciary-license-license-agreement-2.0-2020-02-11-15_47_12.pdf
Description: Adobe PDF document

Re: [gentoo-dev] Should we allow "GPL, v2 or later" for ebuilds?

[Rich Freeman]

On Thu, Jan 30, 2020 at 8:39 AM Hanno Böck  wrote:
>
> *If* Gentoo decides to go this relicensing way I'd recommend to only do
> that if it's coordinated with organizations that have deep legal
> knowledge of these issues (e.g. like software freedom conservancy) and
> if some lawyers that know this stuff well approve the plan.
>

IMO no organization has "deep legal knowledge" of these issues,
because as far as I'm aware something like this has never been done
and tested in court.  Really there are only a handful of legal cases
at all that deal with copyleft and FOSS relicensing.

There is no end of lawyers who will hand-wave on the issue.  I think
the bottom line is that doing something like this is legally risky,
because until something like this has been done successfully many
times it is novel.  You're never going to find a lawyer who will sign
off saying "this is safe and definitely legal."  The only way you
could make something like this risk-free would be to get governments
around the world to pass laws setting up requirements for
FOSS-relicensing without the consent of all contributors.

The best we can do is mitigate risks, if we elect to do something like
this.  That can include being transparent, giving notice, having a way
to opt out, and so on.  Then when somebody sends us a cease and desist
notice we just tell them no problem, their contributions will be
treated as v2-only.  That doesn't completely prevent them from suing
us, but it would mitigate the impact, and probably make it unlikely
that most would sue in the first place.  Really, with something like
this that is the best you're ever going to be able to hope for.

If you don't want to do something unless a lawyer can guarantee that
it can't be found to be a tort by a court, then you definitely don't
want to pursue this change, unless we only make it forward-going for
new contributions and carefully track existing code, and I doubt that
will ever be very practical, so you might as well just give up and say
we'll be v2 forever because that's how things were set up 20 years
ago.

-- 
Rich

Re: [gentoo-dev] Should we allow "GPL, v2 or later" for ebuilds?

[Rich Freeman]

On Thu, Jan 30, 2020 at 6:20 AM Haelwenn (lanodan) Monnier
 wrote:
>
> [2020-01-27 12:41:26+0100] Ulrich Mueller:
> > So, the question is, should we allow ebuilds
> > # Distributed under the terms of the GNU General Public License, v2 or later
> > in the repository, or should we even encourage it for new ebuilds?
> >
> > I have somewhat mixed feelings about this. One the one hand, I think
> > that GPL-2+ should generally be preferred because it offers better
> > compatibility. For example, the compatibility clause in CC-BY-SA-4.0
> > won't work with GPL-2.
>
> Is there another reason for GPL-2+ than just compatibility?
> Because I quite find the "or later" thing to be quite a scary one as
> whatever will come up next as a GPL will become applicable and it feels
> quite weird to me to have a license that can evolve to whatever
> license over time.

Well, there are two sides to this particular issue.

GPL 2+ means that anybody can choose to redistribute the code under
the terms of any version of the GPL that is >=2.  So, if they add
terms to GPL v4 that you really don't like, you can still redistribute
it under the terms of GPL v2-3 if you prefer.

The other side to this is that you can't stop others from
redistributing it under v4.  They could also incorporate it into other
code that is v4+ which you could only redistribute under v4 or
greater.  Of course, the original code can still be redistributed
under v2 - it is just the parts that are comingled with other v4 code
that is at issue.

Really the main threat (IMO) is that the code could be de-copylefted.
They could make GPL v4 a copy of the BSD license, and now anything
that was v2+ is effectively BSD and can be used in non-FOSS software
without issue.  I guess that isn't any worse than the previous case of
it instead being merged into some other v4 variant that you can access
the source for but prefer to avoid because of something else in the
license, except now you might not see the code at all.

The advantage of 2+ is of course flexibility:

For one it reduces license proliferation.  Code that is v2-only is
effectively orphaned with regard to v3, v4, v5, and so on projects in
the future.  GPLv2 is fairly restrictive by design around
compatibility with other licenses and accepting future versions helps
mitigate this insofar as you trust the FSF.

And of course if at some point some fatal flaw is found in the GPL in
a court case, it is possible that a future version could mitigate that
flaw.  Of course, if that flaw lets anybody ignore the copyleft bits
you can't prevent people from using it under the old flawed v2, but at
least you can still use the code in your own v4 or whatever.  Of
course, if the flaw effectively made the v2 code public domain you can
do that anyway, but if the flaw were of a different nature it might
cause problems having code being locked up as v2-only.

>
> I think I would personally slightly prefer to have it be properly
> dual-licensed GPL-{2,3} or GPL-2 & CC-BY-SA-4.0 instead.
>

The problem like this is that this is basically just kicking the can
down the road.  It is of course equivalent for the moment, but when
GPLv4 comes along we have to go through this again.  Right now most of
the Gentoo authors are alive and might be willing to explicitly sign
off on a relicense (maybe).  However, maybe in another 10 years when
GPLv4 comes out it is going to be much harder to track everybody down.

On the flip side the fact is that none of us know what the FSF will
look like in 10 years, or 40 years.  There are plenty of large
non-profits today that bear little resemblance to what they looked
like 100 years ago, for good or ill.  The GPL v2 (or v3) are known
quantities that we can debate on in a concrete manner, but unknown
future versions can only be speculated on.

Another solution to this problem is the FLA - which is something we've
talked about but shelved until we've sorted out some of our other
copyright issues which were thorny enough.  Perhaps we could consider
taking that up again.  Without getting into the details it is a bit
like a copyleft-style copyright assignment, which isn't actually an
assignment.  We envisoned it being voluntary and would allow any
contributor to give the Foundation the authority to relicense their
contributions, with a number of restrictions, like the new license
being FOSS.  I'd have to dig up the latest version and take a look at
it again.  Basically instead of trusting the FSF you'd be trusting the
Foundation instead, but there are some limitations on what they'd be
allowed to do, and if they violate those limitations the agreement
would be canceled and the rights would revert back to whatever was on
the original contribution, which would probably be whatever the author
originally wanted.  That said, I'm not sure it really provides a whole
lot more protection over what happens except for the fact that
Foundation members have more say in how the Foundation operations than
the FSF, if only because 

Re: [gentoo-dev] Should we allow "GPL, v2 or later" for ebuilds?

[Rich Freeman]

On Mon, Jan 27, 2020 at 6:41 AM Ulrich Mueller  wrote:
>
> Historically, all ebuilds in the Gentoo repository were licensed under
> GPL-2+. At a later point they were relicensed [1] to GPL-2. See [2] for
> a rationale (or absence of it, YMMV).

I think the historical policy made sense in its context, which was a
world where all copyrights were to be assigned.  In that case you can
already relicense at will, so you still have flexibility, but by
keeping it pinned at one version you don't get pulled into something
by somebody else that you didn't intend.

Now, over time the whole assignment thing became fuzzier and I don't
really want to get into a largely-moot debate at this point over how
effective those assignments were at various points in time.

Today we are in a world where our intent isn't for the default to
involve assignment, and so the v2-only licenses create (IMO) more
problems than they prevent.

> On the other hand, we would presumably never achieve a complete
> transition to GPL-2+, so we would have ebuilds with either GPL variant
> in the tree. Not sure how big an issue that would be. Updating ebuilds
> wouldn't be a problem (as the old header would stay), but devs would
> have to spend attention to the header when copying code from one ebuild
> to another.

Devs already have to be careful about copying code into ebuilds that
go into our repo.  Somebody could attach an ebuild to a bug and stick
"Copyright Joe Smith all rights reserved" at the top of it.

I think it would make sense to have a call for Devs to voluntarily
report in and give permission for their contributions to be licensed
v2+ with no change in copyright ownership and see what happens.  I
wouldn't be surprised if we could relicense 80-90% of the tree
quickly.  If that happens then we could just require it for new
contributions (if we wanted to), and then over time the problem would
just go away, just like an old EAPI.

We could also stick warnings in ebuild comments like "# Warning
v2-only ebuild - do not copy !" and maybe copy it
every 20 lines if we wanted to be super-paranoid.

I do agree with the general argument that much of this code isn't
really subject to copyright.  We could just do both an opt-in and
opt-out approach to this.  Have the opt-in so that we get as much
explicit approval as we can.  Also do an opt-out with a prominent
announcement like, "hey, we're about to adopt GPL v2+ for all our
ebuilds so if you think you have contributions that are non-trivial
and want to object to those contributions being relicensed please let
us know."  It isn't an airtight defense, but it isn't entirely
unreasonable either.

Or we could just see how many fish we catch with a very conservative
opt-in approach and go from there.  We might not need to even consider
the risk of an opt-out approach.

-- 
Rich

Re: [gentoo-dev] GLEP81 and /home

[Rich Freeman]

On Sun, Jan 19, 2020 at 10:16 PM Michael Orlitzky  wrote:
>
> This is retarded, stop wasting my time.
>

There is nothing retarded about shared /home directories.  They're
pretty common in the real world.

> >> I've already got responses from two QA members. This thread is pretty
> >> hard to miss.
> >
> > Well, then why go posting stuff like "guess we'll be triggering a
> > warning after all?"
>
> If these two things are logically connected, I don't see it.

If you're working with QA to change the QA checks, then you won't be
triggering warnings.

> >> I'm working on a patch for the install-qa-check.d check
> >> and I'm sure I'll get more when I post it.
> >
> > Are you just allowing it to not create the directory, or are we
> > considering patching it to allow creating stuff under /home?  It would
> > seem that the policy would also need updating in that case, but
> > probably not the former.
>
> The patch will make an exception for acct-user packages only; for /home,
> /home/${PN}, and /home/${PN}/.keep*. In other words, it makes things
> work exactly how they did before the GLEP81 eclass started keepdir'ing
> the home directory.

IMO this isn't the right direction to go in, but we can always put it
on the council agenda.  Maintaining the status quo (pre-QA-check) in
the interim isn't unreasonable, nor is keeping your package behavior
as it is for now.  Obviously this issue has been around for some time.
I realize that you didn't invent it.

I guess this is the sort of thing that people will tend to disagree
on.  At least Gentoo doesn't force this nonsense down my throat.  :)

-- 
Rich