Re: [gentoo-dev] [QA] New policy: 'files' directory must not be larger than 32 KiB

[Patrick Lauer]

On 12/17/17 19:39, Mike Gilbert wrote:
> On Sun, Dec 17, 2017 at 8:21 AM, Michał Górny  wrote:
>> Hello, everyone.
>>
>> It's my pleasure to announce that with a majority vote the QA team has
>> accepted a new policy. The accepted wording is:
>>
>>   Total size of 'files' subdirectory of a package should not be larger
>>   than 32 KiB. If the package needs more auxiliary files, they should
>>   be put into SRC_URI e.g. via tarballs.
>>
>> (the total size being computed as a sum of apparent file sizes)
>>
>> The relevant policy vote is finishing at bug #633758 [1]. The CI reports
>>  [2] were updated to report packages whose 'files' directories exceed
>> 64 KiB, to avoid adding many new warnings at once. The limit will
>> be lowered down to 32 KiB as packages are fixed to comply with the new
>> policy.
>>
>> At the same time, I would like to explicitly remind developers that
>> the spirit of the policy is 'do not let "files" grow large', not 'make
>> sure you're one byte less than 32769.' Do not argue that your package
>> exceeds the limit only by few bytes -- even if it gets close to the
>> limit, then it means it's way too large.
> 
> I just want to voice my opinion on this: as a developer, this policy
> is a royal pain in the ass.
> 
> I would ask the council to please increase this limit to at least 100
> KiB, preferably more.
> 
As a user I would like to ask everyone involved to stick to the 32kB
limit so that we (as in everyone) don't have to fetch megabytes of
patches we'll never use, just because someone was lazy.

Re: [gentoo-dev] [RFC] Restricted version of gentoo-dev mailing list

[Patrick Lauer]

On 05/24/2017 12:24 PM, Michał Górny wrote:



Yes, I *do not want feedback* on *how to do Gentoo* from people who do
*not help me do Gentoo* but instead only complain and demand.



But you do gentoo wrong, so as a user I'd like you to reconsider what 
you wrote there and maybe take a long vacation.


(Now of course this is exactly the kind of complaint you don't want to 
hear, so feel free to ignore it ...)

Re: [gentoo-dev] Fixing elasticsearch maintainer

[Patrick Lauer]

On 05/22/2017 03:52 AM, Sam Jorna wrote:

On 18/05/17 02:38, Patrick Lauer wrote:

Since proxy-maint refuses to be removed from packages (especially since they
were unconditionally added to all packages with a non-gentoo-dev maintainer in
metadata) they are the de facto maintainers, and overrule everything else.


Just to clarify, the Proxy Maintainers project is not required to be
added to all packages maintained by non-Gentoo maintainers. If a Gentoo
developer is willing to work with and proxy commits for maintainer(s)
without commit access, Proxy Maintainers are happy to be removed. There
are several metadata.xml's in the tree with examples, including a few
for which you are one of the maintainers.


That's nice.

Could proxy-maint as a team maybe try to agree on such things so that 
everyone is on the same page? It's a tiny bit annoying when the actions 
of some contradict the suggested rules of others, while appearing as a 
single team to the outside ...

Re: [gentoo-dev] Fixing elasticsearch maintainer

[Patrick Lauer]

On 05/20/2017 10:51 PM, Kristian Fiskerstrand wrote:

On 05/20/2017 10:46 PM, Michał Górny wrote:

Tomas, please don't go this road. We all know Patrick does a shitty job
as Gentoo developer, both technically and socially but you do not have
to try to match him.


Was this comment really necessary?

Yes it was, because now I point out how mgorny fails at stuff, then we 
throw ~30 mails around, someone invokes ComRel, and after a week the 
whole thing dies down.


Then in a month or so the same cycle begins.


... Now that we know how it goes we can skip the rest and just glare at 
Michal for not being a reasonable person once more, and just continue 
doing useful stuff.

Re: [gentoo-dev] Fixing elasticsearch maintainer

[Patrick Lauer]

On 05/19/2017 03:10 PM, Michał Górny wrote:

On Wed, 2017-05-17 at 18:38 +0200, Patrick Lauer wrote:

Bonus mention:
bbdc5412061adf598ed935697441a7d6b05f7614
 app-admin/logstash-bin: drop old

 Signed-off-by: Andrew Savchenko <birc...@gentoo.org>

That removed the versions I was using, so I better maintain the versions
I use in an overlay. Well ok then.



I'm sorry that the situation turned out badly for you. However, I would
like to point out that problems like this are rarely unilateral,
and usually involve issues on both ends.

I'd like to ask you a very simple question: what did you do to ensure
that the versions you are using are not accidentally removed?

I could have a few ideas, such as:

a. slotting the package to indicate that multiple versions might be
meaningful,

b. opening a bug requesting the old version to be kept,

c. leaving a comment in the ebuild (unlikely to help but still),

d. just mailing proxy-maint@ to let us know of the issue.



I tried removing proxy-maint from metadata after multiple discussions 
failed. Extra happiness towards monsieurp "but the GH PR is over 3 days 
old, I have to commit" and gokturk "Yes I understand. I commit anyway"


This has been an uphill struggle since about October, around New Year I 
stopped actively caring, and since these two commits:


12c3eacda7c4d23686eaf10eab21d810cc95ea49
f42d6679c038c3efcc257d38547267d01823aea9

I see no way to fix this situation that doesn't involve a review board 
in front of all proxy-maint commits. Because we discussed this in IRC, 
and still ... "but is open bug"



However, as far as I'm aware none of this happened. Note that I might
have missed the mail, or it might have been sent before I joined --
correct me if that is the case.


There were multiple discussions in IRC, which the involved people 
usually forgot within about 20 minutes and then resumed doing stuff.


I tried removing proxy-maint from metadata, which was reverted (sooo how 
does one *not* have constant interference?)



As Alec pointed out, it is a normal procedure in Gentoo to remove old
versions of software if there is no explicit indication that they need
to be kept. Therefore, I don't see anything wrong with the proxied
maintainer wishing to clean the old versions up and/or not requesting
your explicit permission for that. If you needed the old versions, you
should have made that clear.


One could ask, maybe. I guess I can (mis)understand this to mean that I 
can do with packages with you in metadata what I want because ... err... 
shiny!



I should also point out that the steps you've taken (and listed in this
mail) are not really relevant. They make you look like a sloppy
maintainer, and a bad Gentoo developer at the best -- and I doubt anyone
would connect removing proxy-maint team with a necessity of keeping
an old version.


The cooperation that I had with ferki was pretty good (mostly because we 
sat next to each other in the office). The contributions from Tomas were 
on average pretty ok, just needed some minor cleanups here and there.


The blind "but PR is open for 3 days" commits from proxy-maint made it 
extremely hard to review what changed in a timely manner, so that I 
basically didn't want to care for this pile of stupid for the last, 
ahem, 6 months or so. Especially since whenever I wanted to review 
things some joker made some new changes which made me go "eh whut how 
you? banana banana!" so I pushed reviewing a week into the future and ...


I have no idea how I could have fixed this without the QA+Comrel 
banhammer combo, which is a totally insane "fix" to a problem that 
shouldn't even exist. But I see no other options how to make people 
understand that "No means no".


Is this the new normal?

Re: [gentoo-dev] Fixing elasticsearch maintainer

[Patrick Lauer]

On 05/18/2017 07:17 PM, Alec Warner wrote:



On Wed, May 17, 2017 at 12:38 PM, Patrick Lauer <patr...@gentoo.org
<mailto:patr...@gentoo.org>> wrote:

Ohey,

as you might have noticed I've just corrected the metadata.xml of
all elasticsearch-related packages.
For some strange reason I was listed there as maintainer, but since
no one wanted to listen to my ideas I guess I wasn't. So now last
person who touched it gets stuck with it.

Since proxy-maint refuses to be removed from packages (especially
since they were unconditionally added to all packages with a
non-gentoo-dev maintainer in metadata) they are the de facto
maintainers, and overrule everything else.
I've tried multiple strategies including removing them from
metadata, but ... see app-admin/elasticsearch, proxy-maint is like
the toe fungus that always comes back (e.g. commit
f0925c10834464e62ce7209f2afa7797b594d350 )

Sometimes it's almost absurdly funny, especially when you commit
RESTRICT="test" because tests fail reliably just to have that reverted.
(See dev-python/elasticsearch-py )

Bonus mention:
bbdc5412061adf598ed935697441a7d6b05f7614
app-admin/logstash-bin: drop old

Signed-off-by: Andrew Savchenko <birc...@gentoo.org
<mailto:birc...@gentoo.org>>

That removed the versions I was using, so I better maintain the
versions I use in an overlay. Well ok then.


I don't quite get this gripe. Gentoo is a rolling distro. Versions of
things "you are using" get removed and replaced with newer versions all
the time. Why is this a big deal now?



I "was" package maintainer and relied on these versions.

If I as maintainer have no control over such things, why am I 
maintainer, and why do I need an overlay?



... that sounds exquisitely confused, I have no idea why this discussion 
even exists.

[gentoo-dev] Fixing elasticsearch maintainer

[Patrick Lauer]

Ohey,

as you might have noticed I've just corrected the metadata.xml of all 
elasticsearch-related packages.
For some strange reason I was listed there as maintainer, but since no 
one wanted to listen to my ideas I guess I wasn't. So now last person 
who touched it gets stuck with it.


Since proxy-maint refuses to be removed from packages (especially since 
they were unconditionally added to all packages with a non-gentoo-dev 
maintainer in metadata) they are the de facto maintainers, and overrule 
everything else.
I've tried multiple strategies including removing them from metadata, 
but ... see app-admin/elasticsearch, proxy-maint is like the toe fungus 
that always comes back (e.g. commit 
f0925c10834464e62ce7209f2afa7797b594d350 )


Sometimes it's almost absurdly funny, especially when you commit 
RESTRICT="test" because tests fail reliably just to have that reverted.

(See dev-python/elasticsearch-py )

Bonus mention:
bbdc5412061adf598ed935697441a7d6b05f7614
app-admin/logstash-bin: drop old

Signed-off-by: Andrew Savchenko 

That removed the versions I was using, so I better maintain the versions 
I use in an overlay. Well ok then.


Since I, as maintainer, can't ... anything, well [CENSORED] this, now 
they are your packages. Don't try to reassign or drop them: You've 
demanded, insisted, to be maintainers ... wish granted.


So, err, well, is like ... wtf? I'm not sure how this all makes sense, 
but it's Not My Problem now. Take care now, bye bye then.


Oh, and Erki Ferenc was in metadata too, he's been inactive but told me 
that he wants to continue maintaining these packages in the near future. 
If he asks I'd recommend adding him back.


Patrick

P.S. If this sounds a bit incoherent, well ... the whole situation is, I 
have no idea what's going on or why I was in metadata ;)

[gentoo-dev] Re: Local workarounds with no reported bugs

[Patrick Lauer]

On 10/17/16 09:23, Michał Górny wrote:
> Hello, everyone.
> 
> I'd like to point out a major problem in Gentoo: there's a fair number
> of developers who add various local workarounds to problems they meet
> and don't bother to report a bug. Worst than that, this applies not
> only for upstream problems but also to Gentoo eclass/ebuild-related
> issues.
> 
> 
> Example: udev people had problems with MULTILIB_WRAPPED_HEADERS
> in the past. Instead of contacting me
... bus factor of 1, that's not a good strategy in general.

> (which would result in helpful
> explanation how to do things properly), they abused bash to disable
> the check function implicitly in the ebuilds.


> Nobody bothered to inform me of the issue there.
Well, you didn't ask ;)

 Instead, I had to
> notice it looking at the udev ebuilds accidentally. Furthermore, in
> most of the ebuilds the workaround was no longer necessary but nobody
> bothered to check that.
> 
> 
> Example 2: Coacher had problem with git-r3 not trying fallback URIs
> when earlier URI was https and https wasn't supported in git. So he
> reordered URIs to have https last. With tiny explanation in some random
> commit message.
> 
> So we have a problem that affects around a half of git-r3 packages
> (using quick grep, results inaccurate), however minor it is. Worse, it
> affects the policy of preferring https and causes some people to reject
> the policy silently. And nobody gives a damn to report it!
> 
> 
> Therefore, I'd like to request establishing an official policy against
> workarounds with no associated bug reports.
That's too fuzzy to make a policy. Feels good, everyone agrees with the
idea, but where's the limit? When is it a required fix, when it is just
a workaround? What needs to be upstreamed, and what can't be upstreamed?

> 
> Your thoughts?
> 
I like the general idea of ... like ... improving ... things, but you
should be more precise and try to avoid creating situations with a bus
factor of one.

Re: [gentoo-dev] rfc: /etc/hostname on gentoo

[Patrick Lauer]

On 08/28/2016 04:21 PM, Michał Górny wrote:
> On Sun, 28 Aug 2016 14:34:20 +0200
> Patrick Lauer <patr...@gentoo.org> wrote:
> 
>> On 08/28/2016 08:30 AM, Daniel Campbell wrote:
>>> On 08/24/2016 09:42 AM, Zac Medico wrote:  
>>>> On 08/24/2016 09:33 AM, Michael Orlitzky wrote:  
>>>>>   * no benefit put forth so far, other than that it's the same file that
>>>>> systemd uses, which is true but not beneficial as far as I can tell  
>>>>
>>>> It's a de facto standard. Being different for the sake of being
>>>> different is not a virtue in cases like this.
>>>>  
>>>
>>> And doing things because "everyone else does it" is dumb, because it
>>> precludes our ability to choose and makes us subject to the decisions
>>> made outside of our distribution. Of course, as a distro we're subject
>>> to outside decisions often, but what's the point of being a distro if
>>> you're doing things the same way everyone else does?  
>>
>>
>> At this point I feel the need to point at /etc/mtab and how it doesn't
>> work anymore. Or rather:
>>
>> In the old days it did *not* carry all mountpoints, so you could hide
>> things like /dev and /run so that "umount -a" would not screw you sideways.
>>
>> Then tools forgot to properly update mtab because hurr why u no symlink
>> to /proc/mounts (oh wait, /proc/self/mounts )
>>
>> So everyone migrated to /etc/mtab as a symlink (even OpenRC, because
>> everyone does it)
>>
>> ... and now if you still instincively use umount -a you unmount /run and
>> other bits, breaking lots of stuff (can't shutdown if OpenRC strongly
>> considers not having booted!)
>>
>>
>> That's why some of us are very resistant to change.
> 
> Which could be pretty much summarized as 'I'm unhappy because I was
> abusing the existing system to make "umount -a" not do what it was
> supposed to do, and I'm unhappy because now it started to work
> correctly'.
> 

"Just because it worked for 30 years doesn't mean it worked" ?

I like how you claim to be the authority on how umount is supposed to
behave ...

(and what abuse? it did exactly what it was supposed to do quite nicely,
until it stopped doing that. Now you need to track state and hope you
don't have race conditions ... )

Re: [gentoo-dev] rfc: /etc/hostname on gentoo

[Patrick Lauer]

On 08/28/2016 08:30 AM, Daniel Campbell wrote:
> On 08/24/2016 09:42 AM, Zac Medico wrote:
>> On 08/24/2016 09:33 AM, Michael Orlitzky wrote:
>>>   * no benefit put forth so far, other than that it's the same file that
>>> systemd uses, which is true but not beneficial as far as I can tell
>>
>> It's a de facto standard. Being different for the sake of being
>> different is not a virtue in cases like this.
>>
> 
> And doing things because "everyone else does it" is dumb, because it
> precludes our ability to choose and makes us subject to the decisions
> made outside of our distribution. Of course, as a distro we're subject
> to outside decisions often, but what's the point of being a distro if
> you're doing things the same way everyone else does?


At this point I feel the need to point at /etc/mtab and how it doesn't
work anymore. Or rather:

In the old days it did *not* carry all mountpoints, so you could hide
things like /dev and /run so that "umount -a" would not screw you sideways.

Then tools forgot to properly update mtab because hurr why u no symlink
to /proc/mounts (oh wait, /proc/self/mounts )

So everyone migrated to /etc/mtab as a symlink (even OpenRC, because
everyone does it)

... and now if you still instincively use umount -a you unmount /run and
other bits, breaking lots of stuff (can't shutdown if OpenRC strongly
considers not having booted!)


That's why some of us are very resistant to change.
> 
> mjo made a good point. What if the meaning of /etc/hostname changes? Or
> rather, what if the file gets moved altogether? All this effort to
> "follow the flock" will lead to higher maintenance burden. Symlinking it
> in pkg-postinst or some other mostly-automatic behavior makes sense
> because then a package "owns" the file. Should an update happen where
> the decision to follow the flock is rescinded, a revbump with the
> symlinking line removed would cleanly get rid of the symlink without any
> user intervention and next to zero maintenance burden.
> 
> /etc/conf.d/hostname sits alongside multiple other files, including
> hwclock, consolefont, localmount, fsck, modules, sshd, udev, etc. By
> glancing at it, it's clear that /etc/conf.d/ relates to system (or
> rather, package) configuration.
> 
> Considering that OpenRC puts package configuration there, and OpenRC (by
> default) looks for the hostname file in that directory, it's a
> non-issue. Why should OpenRC look elsewhere for configuration when
> there's already a place for it?
> 
> If systemd or other inits need it, then they should install the file and
> guess the initial value by sourcing /etc/conf.d/hostname. It's none of
> OpenRC's concern what other inits need.
> 

Re: [gentoo-dev] Re: Packages up for grabs

[Patrick Lauer]

On 08/07/2016 10:04 PM, Alan McKinnon wrote:
> On 07/08/2016 19:36, james wrote:
>>> The interesting apps out there are mostly running python, go and
>>> (sometimes) lua. And that's what I observe in my day job -
>>> business/mobile ISP.
>>
>>
>> Look at the job listing on stackoverflow and elsewhere (java) is very
>> popular when they list several programming languages to meet the
>> requirements. I'm not promoting java, at all, but just stating that it
>> is very popular, on new projects (but not all) and it is a large and
>> frequent requirement, dictating by employers. Kids coming out of college
>> want a job, more than anything, and most are having java crammed down
>> their throats. So we should  find a way to robustly
>> support those that need java. Nothing is precluding other languages
>> in my message. Personally I avoid java, unless it is critical to
>> a code or family of codes I need to run.
> 
> 
> I recommend Java as a teaching language at university level.

I've seen the fallout from trying to do that.

It's a horribly bad idea ...

> You get all the benefits of a C-like syntax without the overhead of
> learning to deal with C and/or C++. You don't have to deal with the
> toolchain (much), you can easily show correct implementations of OOP
> style without getting into generics (or, you can avoid Java generics
> altogether at this level and pretend they don't exist).

Java and OOP ? If you want to do things right, best to use something
proper like Eiffel or Oberon. And Java will be most excellent at
teaching about pointers (but there are no pointers!) to maximize the
learning curve gradient ...

On the upside your students will learn useless incantations along the
lines of "publicstaticvoidmain!" that they can't explain and copypasta :)

I've found these two a long time ago, and they still amuse me:

http://gentooexperimental.org/~patrick/keywords.java.txt
http://gentooexperimental.org/~patrick/helloworld.java.txt


> In short, what's not to like for teaching? All win not much lose.
> 
> Well OK some kids come away thinking Java is the one and only, but they
> will have that too if Python is the teaching language. Realizing there
> are other things out there is part of the learning process.
> 
> But, despite all that, Java is not special. It should run on Gentoo for
> anyone who wants it, just like things starting with P.
> 
> You volunteering to do the grunt work?
> 

Java works pretty well on Gentoo, I'm not quite sure what needs to be
fixed ... I mean, apart from our insane idea to "build from source"
which doesn't fit with the existing structures in the java ecosystem

Re: [gentoo-dev] Packages up for grabs

[Patrick Lauer]

On 08/07/2016 10:12 AM, Dirkjan Ochtman wrote:
> On Sun, Aug 7, 2016 at 9:51 AM, Pacho Ramos  wrote:
>> This packages are now up for grabs:
>> app-portage/euscan
> 
> Patrick,
> 
> Are you still keeping euscan running?
> 

Yes. It's not in the best shape, but for now it works well enough to
keep it around.

Re: [gentoo-dev] The future of the Sunrise project

[Patrick Lauer]

On 06/07/2016 10:31 PM, Andreas K. Huettel wrote:
>
> > Your thoughts?
>
> > [1]:https://wiki.gentoo.org/wiki/Project:Sunrise
>
> Sunrise was a great way to learn packaging for Gentoo. Reviews were
> *very*
> strict in the past, resulting in better QA standards than the Gentoo
> main tree
But here's the funny thing:

While there was a syntactical review that made these ebuilds wonderfully
strict no one seems to have compile-tested them. So when I ran a
test-build over all of sunrise about half the packages had invalid
SRC_URI, didn't compile, patches didn't apply, etc.

So from my perspective it was *useless* enforcement of arbitrary rules
with little to show for it.

> - and a definite frustration threshold that one had to overcome. With
> a couple
> of packages in Sunrise, doing the quizzes was a piece of cake though.
>
> That said...
>
> If there's no activity anymore, we definitely should remove the
> overlay from
> Layman and (important!) remove the mentions of Sunrise from our web pages
> (e.g., "contributing to Gentoo").
>
> We now have functioning and active alternatives, see proxy-maintainers.
>
>

Re: [gentoo-dev] [RFC] Global USE=gui

[Patrick Lauer]

On 06/06/2016 04:53 PM, Ian Stakenvicius wrote:
>
> This -can- be simplified using a REQUIRED_USE to force just-one-of
> gtk3,qt4,qt5 , but you can technically do the same with USE=gui too --
> all you'd need to do is add dependencies for the no-specific-flag case.
>
> RDEPEND="...
>   qt5? ( dev-qt/qtcore:5 )
>   qt4? ( dev-qt/qtcore:4 )
>   gtk3? ( x11-libs/gtk+:3 )
>   gui? ( !qt5? ( !qt4? ( !gtk3? ( dev-qt/qtcore:5 ) ) )"
Wow, that is wonderfully horrible, and making me a little bit angry ...

So USE="-qt5 gui" enables qt5 in this scenario. How is that reasonable?


(And as usual I'm now at the stage where I am not sure what problem that
we had before we are actually solving ...)

Re: [gentoo-dev] [RFC] Global USE=gui

[Patrick Lauer]

On 06/03/2016 10:52 AM, Michał Górny wrote:
> Dnia 2 czerwca 2016 21:36:10 CEST, waltd...@waltdnes.org napisał(a):
>>
>>  Is it broken right now?  What improvement will we see from having to
>> add a "GUI" flag?
> TL;DR: it's broken as hell, missing GUI, flag conflicts, implicit flags, full 
> package.use and inability to sanely replace old toolkits with new ones.
I see it the other way around: Adding Yet Another Flag will just make
things more, uhm, exciting.

For example since gtk3 is totally useless on HiDPI and only mostly
useless on normal screens I want to actively avoid it. Having a 'gui'
useflag adds another indirection that badly emulates what the profile
already does ...

> So, let's say we're considering only GTK+ and Qt, for simplicity's sake. Wrt 
> QA policy, we only are supposed to use the following flags: gtk2, gtk3, qt4, 
> qt5...
>
> Now, we have the major types of packages (with some GUI):
>
> 1. Packages with a single obligatory GUI -- having no flags,
>
> 2. packages with a single optional GUI -- having one flag controlling whether 
> it is enabled,
But is it gtk2 or gtk3? If 2 then I'm ok, if 3 then not, now I can't
easily see what is pulling in the wrong deps
>
> 3. packages with multiple GUIs -- having multiple flags.
>
> Now, the third type could be further split depending on:
So now dependencies become funny ...

RDEPEND="gui? ( X? ( gtk2? ( ...) gtk3? ( ...)))"
REQUIRED_USE="gui? ( ^^ ( gtk2 gtk3 ))"

See how gui just adds a middle layer that doesn't help most cases? Sigh. :\

[snip]
> Let's say he underwent the effort and enabled gtk2 on some packages, qt5 on 
> some other.
>
> Now, when packages gain gtk3 support, he either gets new conflicts or two 
> flags being enabled cause app to prefer one of them. When packages gain qt6 
> support, he is stuck with 5 until support for 5 is removed. Even though some 
> packages start using 6 implicitly anyway.
Which is good / expected: If I deviate from defaults, then don't change
the defaults for me. If I wanted you to decide for me I'd Ubuntu all day
long ;)
>
>
> B. */* gtk3 qt5
>
> This covers packages not supporting GTK+ better but leaves multiple GTK+ 
> problem unsolved. But now, some packages will either request him to disable 
> one of the two, or implicitly choose one of them, possibly the one less 
> preferred.
>
>
> C. */* gtk2 gtk3 qt5
>
> Because he is tired with a lot of packages not supporting GTK+3. Now, he's 
> either in a world full of conflicts, or implicit choices. For the latter, 
> he's unhappy that some developers make him prefer old GTK+2.
Implicit is usually wrong, so the packages should have an explicit
one-of-enabled check ... even if it's not the most aesthetical solution.
Otherwise, why have the useflags if you decide for the user?
>
>
> Of course, that are just the closest user facing problems. Additionally, 
> every time developer refuses to use REQUIRED_USE, the user is left with some 
> implicit choice whose result can't be predicted, possibly slow and unreadable 
> pkg_pretend that actually tells him about the choice (too late), and multiple 
> meaningless flag combinations that make it impossible to reuse the same 
> binary package even though the same GUI has been actually used.
>
>
> I'm not saying that 'gui' solves all the problems. But it's a step in the 
> right direction where USE flags actually start meaning something and not 
> resembling the Ubuntu 'apt-get install libgtk+3 libfoo ...'

I kinda disagree that it solves any problems we have, and makes my life
more exciting. And exciting can be bad for my mood ;)

Re: [gentoo-dev] [RFC] gtk/gtk2/gtk3 USE flag situation

[Patrick Lauer]

On 05/27/2016 04:21 PM, Mart Raudsepp wrote:
> Hello,
>
> Despite it being 2016 and gtk2 pretty much dead, buried and forgotten
> upstream, many applications still support only gtk2, have subtle issues
> with their gtk3 port, or support both, with some of our userbase
> clinging to gtk2 for dubious political or aesthetical reasons.
>
> For the latter cases, despite GNOME teams policy and strong preference
> on not providing a choice and just choosing gtk2 or gtk3 (gtk3 if it's
> working as good as gtk2), some cases exist where the maintainers want
> to provide such choice. In some cases it is understandable for a short
> while during transition, e.g firefox. In other cases, it is purely for
> the sake of providing the choice of working with a deprecated toolkit,
> apparently.
>
> My highly biased essay aside, we need to finally globally agree on what
> we do in this situation. If we allow this choice at all, only for
> special cases, or widespread. And if this choice is provided, how do we
> name the USE flag.
Looking at the gtk-apps I use ...


Firefox with gtk3 is TOFU. UI elements are invisible, Scrollbars are
broken, etc.
The font rendering is hilariously wrong, even more so than gtk2. I'd
call this "late alpha" if I'm in a good mood.
("wrong" ? well, everything else at fontsize ~8-9 is the same as gtk2 at
fontsize 24, and gtk3 at ~32. Which means that the defaults are
literally unreadable because text ends up 3 pixels high. I have no idea
why everything else understands config, and special snowflake has to
guess instead...)

Thunderbird ... oy vey. The new new theme in 45.1 after a new theme in
45 repeats all the problems with font rendering, and it's "flat" because
who needs to know where UI elements are, where they end, or if they are
active/usable. Also fun that *now* it is 'catching up' with the UI
stupidity of Android-4, which is abandoned. Change to have change ... :\

I think Chromium uses their own layer of madness on top of gtk.
Chromium-51 has a tiny URL bar which is not resizable (sizing only
affects the html viewport). It is proportionally smaller than
Chromium-50, so I guess they switched to gtk3 too.


All in all: GTK+-3 looks substantially worse, has wrong font rendering
(which gtk2 already got wrong), and I consider it a strict downgrade
from gtk2. So where I can, when I can, I will aim to keep gtk2 until I
can switch to something that isn't dead upstream or just braindead.

So while I understand that you don't want to support a toolkit that has
no or little support (like qt4, btw, which is abandoned but only about
half the things have been ported to qt5) ... as long as it's not at
feature-parity some users like me will fight for the option to have
usable software. And that means, for now, requiring gtk2 / gtk3 useflags
to allow us to choose the correct version where possible. I don't care
much how it is exposed, I don't consider "force-gtk2" worse than "gtk2"
or "partial-sanity". But it'll take a lot of improvement before you can
consider deprecating gtk2, and crude methods like the suggested
package.use.mask will motivate me to fix the mistake by reverting it.

(I remember fighting with ssuominen over guvcview, he always tried to
remove the last gtk2 version of it, I readded it because otherwise
there's a whole pile of new bad dependencies, he'd remove it again, I
readded it, lots of fun. Let's not play that game too much please ...)


Thanks,

Patrick

Re: [gentoo-dev] Re: Transitioning from #!/sbin/runscript to,#!/sbin/openrc-run

[Patrick Lauer]

On 05/05/2016 08:59 PM, Andreas K. Huettel wrote:
> Am Donnerstag, 5. Mai 2016, 09:53:10 schrieb Patrick Lauer:
> > On 05/05/2016 09:44 AM, M. J. Everitt wrote:
> >> On 05/05/16 08:32, Patrick Lauer wrote:
> >>> To summarize: Lots of churn, no visible benefit, except that some OCD
> >>> people could feel better: except that we can't actually fix the core
> >>> 'issue' without making lots of other people very sad.
> >>>
> >>>
> >>> Y'all have too much free time ... ;)
> >>
> >> I'm inclined to say, that provided there *is* someone doing it .. let
> >> them be. Whatever the motives.
>
> > This ignores the externalized cost for potentially thousands of users
> > that have to fix stuff because it was actively broken.
>
>
> So how many custom init scripts have you deployed that you can't fix
> with a
> single Rex command?
>
I like the naive assumption that I only have one central deployment :)

To be honest, I don't know, because I shouldn't have to care ... but
what's a few hours of changing stuff between friends, especially when it
doesn't add any features.

Re: [gentoo-dev] Re: Transitioning from #!/sbin/runscript to,#!/sbin/openrc-run

[Patrick Lauer]

On 05/05/2016 09:44 AM, M. J. Everitt wrote:
> On 05/05/16 08:32, Patrick Lauer wrote:
>> To summarize: Lots of churn, no visible benefit, except that some OCD
>> people could feel better: except that we can't actually fix the core
>> 'issue' without making lots of other people very sad.
>>
>>
>> Y'all have too much free time ... ;)
>>
> I'm inclined to say, that provided there *is* someone doing it .. let
> them be. Whatever the motives.
This ignores the externalized cost for potentially thousands of users
that have to fix stuff because it was actively broken.


How much is your time worth?
>
> We have enough problems with painting the bike shed not to stifle
> activity and contribution ...

I dislike repeating myself, but ... "Change is not Progress"
>
> Just my 2c ... ;]
>
> MJE
>

Re: [gentoo-dev] Re: Transitioning from #!/sbin/runscript to,#!/sbin/openrc-run

[Patrick Lauer]

On 05/05/2016 09:17 AM, Duncan wrote:
> Patrick Lauer posted on Thu, 05 May 2016 07:13:00 +0200 as excerpted:
>
>> So again, because I feel like either I'm too stupid to understand this,
>> or too smart to let such an obviously bad idea continue:
>>
>> What problem is being solved here?
> For one thing, the namespace issue of runscript being generic, while 
> openrc-run is properly namespaced and thus much less likely to conflict 
> with anything else.
... which wasn't a problem for the first decade. The first time a name
collision was noticed was when debian packaging was attempted.
>
> That would be why openrc's upstream maintainer is changing the name, with 
> appropriate deprecation notice for the old one.  Given that, what gentoo 
> has to decide is how it's going to respond to that.  Sure, we /could/ 
> rename the executable to runscript here and be done with it, but that 
> would violate gentoo's policy of defaulting to consistency with upstream 
> unless there's a very good reason not to.
>
> The fact that the packages upstream maintainer happens to be a gentoo dev 
> and that gentoo happens to host the project and be its primary testing 
> ground and user base shouldn't change that.
>
> Of course if upstream policy is thought by devs willing to do the work to 
> be irrational, they can of course fork the package.  There's certainly 
> precedent for that.  But someone's gotta be willing to do the work 
> necessary to create and maintain that fork, so...
>
So you're saying that a Gentoo-specific change in Gentoo happens because
the Gentoo maintainer doesn't care about Gentoo? ;)

Somehow I still don't see a *problem* being solved, and the runscript
binary/symlink pretty much has to stay there indefinitely unless you
want to make life exciting for people that have their own or adapted
init scripts.

To summarize: Lots of churn, no visible benefit, except that some OCD
people could feel better: except that we can't actually fix the core
'issue' without making lots of other people very sad.


Y'all have too much free time ... ;)

Re: [gentoo-dev] Transitioning from #!/sbin/runscript to,#!/sbin/openrc-run

[Patrick Lauer]

On 05/05/2016 01:12 AM, William Hubbs wrote:
> On Wed, May 04, 2016 at 07:41:39PM +1000, Sam Jorna wrote:
>> On Wed, May 04, 2016 at 10:57:44AM +0200, Kristian Fiskerstrand wrote:
>>> On 05/04/2016 10:52 AM, Sam Jorna wrote:
 On Wed, May 04, 2016 at 10:00:05AM +0200, Ulrich Mueller wrote:
>> On Wed, 4 May 2016, Austin English wrote:
>>> Your list of affected packages obtained with "git grep" in the
>>> Portage tree will not be complete, since the command won't catch
>>> any init scripts installed from elsewhere. You should look for the
>>> set of installed files instead.
>> How is that relevant here at all? I'm cleaning up portage installed
>> init scripts, [...]
> You are cleaning up only those init scripts that are installed from
> FILESDIR, but you will miss the ones that are installed from a file
> in SRC_URI.
 Perhaps an alternate way to do it would be to have a QA check look at
 any files installed to ${D}etc/init.d/ and throw a warning if their
 shebang is "#!/sbin/runscript"

>>> A repoman check is a much saner approach, I'm not convinced there is
>>> sufficient need for this change to begin with, in particular to start
>>> touching a wide range of packages. Breaking backwards compatibility in
>>> any way should have a darn good reason, and I haven't seen one yet
>> I'm not arguing for or against it in general, just in terms of technical
>> implementation.
>>
>> That being said, a repoman check would only catch those distributed in
>> ${FILESDIR} as well. My thinking with the above was to also identify
>> those installed from distfiles to be handled accordingly.
> Actually, you won't need to worry about any qa checks in portage,
> because I am going to put a deprecation warning in OpenRC upstream which
> will be displayed when a service script invokes runscript instructing
> you to convert to openrc-run.
>
> OpenRC will keep runscript, with this warning, for a while.
>
>
So again, because I feel like either I'm too stupid to understand this,
or too smart to let such an obviously bad idea continue:

What problem is being solved here?

Re: [gentoo-dev] Transitioning from #!/sbin/runscript to,#!/sbin/openrc-run

[Patrick Lauer]

On 05/04/2016 06:27 AM, Austin English wrote:
> Hi there,
>
> I've been working on the transition from #!/sbin/runscript to
> #!/sbin/openrc-run [1],
... and once more I have to ask:

Is there any reason that Stuff Needs To Change because of a packaging
conflict in *debian* where it really doesn't matter at all for us?

I mean, ok, it's your time, do what you want, but I'm still confused
about the benefits of this change ...

>  by starting on the maintainer-needed packages.
> That's done (aside from some stabilizations needed, but I'll deal with that
> latter). The trouble is that there are roughly 700 packages that need to
> be updated, and that's an insane number of bugs to file.
>
> So, instead, I'm going to give developers to two weeks to update their
> initscripts or ask me not to touch it. On/after 2016/05/18, I'll update
> initscripts/copyrights, but will leave revbumping to maintainer's discretion.
>
> Thanks,
> Austin
>
> [1] https://bugs.gentoo.org/show_bug.cgi?id=573846
>
>

Re: [gentoo-dev] glibc 2.23 and willfully breaking stuff

[Patrick Lauer]

On 04/16/2016 09:23 AM, Patrick Lauer wrote:
> I very strongly suggest bumping the glibc ebuild, removing the patch in
> the bump, and masking the broken version. Then asking people to test the
> patched version to smoke out failures, and in a few months we can
> consider re-enabling this tomfoolery.
>
> (And with very strongly suggest I mean QA might just do it because
> breaking random shit is not cool, yo. So please fix it soon)
After discussion on IRC reverting that patch has been ack'ed by zlogene
and mgorny.

sys-libs/glibc-2.23-r2 has been committed with this patch removed.

Re: [gentoo-dev] glibc 2.23 and willfully breaking stuff

[Patrick Lauer]

On 04/16/2016 09:23 AM, Patrick Lauer wrote:
>I very strongly suggest bumping the glibc ebuild, removing the patch in
the bump, and masking the broken version. Then asking people to test the
patched version to smoke out failures, and in a few months we can
consider re-enabling this tomfoolery.

And this is the required patch:

--- glibc-2.23-r1.ebuild2016-04-14 09:46:35.204239147 +0200
+++ glibc-2.23-r2.ebuild2016-04-16 09:30:44.653753397 +0200
@@ -166,6 +166,7 @@
# Bug 558636 we don't apply the pie works around for 2.22. It
shoud have the support. #558636
GLIBC_PATCH_EXCLUDE+="
00_all_0002-workaround-crash-when-handling-signals-in-static-PIE.patch"
GLIBC_PATCH_EXCLUDE+="
00_all_0012-disable-PIE-when-checking-for-PIC-default.patch"
+   GLIBC_PATCH_EXCLUDE+="
00_all_0009-sys-types.h-drop-sys-sysmacros.h-include.patch"
 }
 
 eblit-src_prepare-post() {

[gentoo-dev] glibc 2.23 and willfully breaking stuff

[Patrick Lauer]

As of commit ed047cf2c607277629c20bf1a88d727a7f9bb79e we have
sys-libs/glibc-2.23 in ~arch.


This breaks *lots* of stuff. For example coreutils was broken [1].

According to the tracker bug [2] most of the breakage was introduced in
a gentoo-specific patch.

On the upstream mailinglist [3] people seem to be concerned about the
change:
"
It's risky enough that I think it's worth doing a distro rebuild with
that change to find out what, if anything, breaks - who do I talk to to
make that happen?
"


So why on earth are we applying a random patch that upstream is not
using, *and* unleashing it on ~arch without any of the usual precautions
like masking the package until some of the issues have been smoked out?

I very strongly suggest bumping the glibc ebuild, removing the patch in
the bump, and masking the broken version. Then asking people to test the
patched version to smoke out failures, and in a few months we can
consider re-enabling this tomfoolery.

(And with very strongly suggest I mean QA might just do it because
breaking random shit is not cool, yo. So please fix it soon)

At this point I'm a little bit confused why Gentoo users are used as
guinea pigs and stuff gets broken on purpose. It's not how one should do
things, and excuses like "it finds bugs" are just lame excuses.

So anyway. Please not to break things. For prosperous happy of users!



[1] https://bugs.gentoo.org/580014
[2] https://bugs.gentoo.org/show_bug.cgi?id=575232#c7
[3] https://sourceware.org/ml/libc-alpha/2015-11/msg00253.html

[gentoo-dev] Re: BROKEN: repository became broken!

[Patrick Lauer]

On 03/30/2016 04:32 PM, mgo...@bonedaddy.net wrote:
> Looks like someone just broke Gentoo!
>
> New issues:
> https://qa-reports.gentoo.org/output/gentoo-ci/33f062a/output.html#dev-db/aerospike-server-community
>
>
> Introduced by commits:
> https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=54033c5
>
>
> Changes since last check:
> https://gitweb.gentoo.org/repo/gentoo.git/log/?qt=range=0d2b3ad..54033c5
>
> --
> Gentoo repository CI
... can you please either use repoman like everyone else, or fix repoman
to match the behaviour you like?

'cause it's annoying to have repoman say all is well and then get such a
nice response. Hard to comply with rules if there's no good way to check
the rules.

Re: [gentoo-dev] Re: [gentoo-project] Portage repo usage survey and change evaluation

[Patrick Lauer]

On 03/02/2016 08:48 PM, malc wrote:
> I still fail to understand the bikeshedding here - you really don't
> need a git checkout to get something akin to a changelog. Use the
> github API directly...
>
> The following 1-liner could be trivially productised (maybe even parse
> $PWD to set the path argument...)
>
> curl https://api.github.com/repos/gentoo/gentoo/commits?path=app-admin/eselect
> | perl -MJSON -e 'foreach $i (@{decode_json(join("",@lines=))})
> { print "$i->{commit}->{author}->{name} -
> $i->{commit}->{author}->{date}\n\n $i->{commit}->{message}\n"; }'
Requires you to be online, can't grep over multiple packages.

This version relies on an unreliable thirdparty service and is thus more
of an intellectual curiosity.
> Yeah - it's not quite as pretty as our current Changelog, but date,
> author/committer, commit-msg etc. are all there and you can filter by
> path just the same as you would with native git log...
> You could parse the local $PORTDIR/metadata/timestamp* and add an
> 'until' param to the URL to filter commits beyond where a user has
> rsync'd up to...
>
It is almost, but not completely unlike it. A simple ChangeLog is a lot
easier ...


(Why are people now trying to add middleware layers to indirect the
problem to become invisible in a huge machinery? This is wonderfully
insane ...)

Re: [gentoo-dev] Re: [gentoo-project] Portage repo usage survey and change evaluation

[Patrick Lauer]

On 03/02/2016 02:32 AM, Robin H. Johnson wrote:
> On Mon, Feb 29, 2016 at 09:01:19AM +0100, Ulrich Mueller wrote:
>> Have I missed your posting the results of this? Especially, what is
>> the preferred ordering of ChangeLog entries?
> I just hadn't finished putting the results into a long-term format quite
> yet, but did so this afternoon:
> http://dev.gentoo.org/~robbat2/201602-portage-survey/
>
> I have included a CSV of the public answers, excludes only the last
> question about contact info.
>
> Some remarks about question #2 and #3:
>
> Q2: Reduce local disk usage by excluding ChangeLogs?
> 
> It was unfortunately pointed out to me very late that my question #2 had
> some confusing text:
> - "No, but only if were optional (I do NOT want it, but others might)"
> - "Yes, but only if it were optional (I want it, but others might NOT)"
>
> The bracket portion of each answer was interpreted as meaning the
> opposite as the start of each answer :-(.
>
> Either way, ~60% are in favour of getting rid of changelogs.
Well, with those confusing answers I'd interpret it differently:

~15% are in favour of removal (see Q3)
~45% are in favour of available-but-not-default ("No but optional")
~40% are in favour of available and default

That'd be, like, 85% in favour of keeping changelogs, and about half the
people would want an option to remove them.
>
> IMO this is a BETTER goal than continuing to generate them for rsync,
> and bike-shedding about what the order should be; and it provides a huge
> benefit by reducing the size of rsync by 155MiB.
There's no bikeshedding about order (see below), and if most people are
in favour of keeping or providing optionally I don't see how removal is
in the interest of the majority - which was the reason you did this survey.
>
> Q3: What order should ChangeLog entries be in?
> --
> - 85.3% of responses either preferred newest first OR didn't care (incl
>   so as long as the tools work).
> - 2.9% wanted oldest first.
> - NOBODY selected "I'd prefer oldest entries first, but do what is best
>   for distribution"
> - 11.8% said get rid of changelogs.
>
So people want ChangeLogs in ChangeLog order. An important, but
unexpected result :)

The obvious thing to do is to continue providing ChangeLogs, in the
obvious order, and possibly document a way for users to exclude them
without breaking Manifest. I'm not sure if there's a simple/clean way to
do that except maybe providing two rsync trees ...

Re: [gentoo-dev] Changing order of default virtual/udev provider

[Patrick Lauer]

On 02/08/2016 10:08 AM, Patrick Lauer wrote:
> Ohey,
>
> I've opened a bug at:
> https://bugs.gentoo.org/show_bug.cgi?id=573922
>
> The idea here is to change the order of the providers of virtual/udev.
> For existing installs this has zero impact.
> For stage3 this would mean that eudev is pulled in instead of udev.
>
>
Council vote on it was 7:0 in favour:
https://bugs.gentoo.org/show_bug.cgi?id=575718

The change has been committed.

Re: [gentoo-dev] Re: Bug #565566: Why is it still not fixed?

[Patrick Lauer]

On 02/27/2016 11:50 PM, Robin H. Johnson wrote:
> On Sat, Feb 27, 2016 at 02:14:12PM +0100, Luca Barbato wrote:
>> On 24/02/16 01:33, Duncan wrote:
>>> That option is there, and indeed, a patch providing it was specifically 
>>> added to portage for infra to use, because appending entries to existing 
>>> files is vastly easier and more performant than trying to prepend entries 
>>> and having to rewrite the entire file as a result.
>> This sounds wrong in many different ways. The changelog files are tiny
>> and makes next to no difference truncate+write or append.
> Prior to seperating ChangeLog files into years, this was way worse:
> a kernel bump present in any of gentoo-sources, hardened-sources,
> vanilla-sources meant another 100k of data to sent. It's not a lot
> overall, but here's some quick stats from one of our rsync servers, on
> bytes sent.
[snip]
>
> So, now the question:
> If we use appending changelogs, the large changelogs only differ by a
> few hundred bytes. If we instead have to rewrite them, it's 50k+ per
> changelog.
from /usr/share/portage/config/make.globals:

PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times
--omit-dir-times --compress --force --whole-file --delete --stats
--human-readable --timeout=180 --exclude=/distfiles --exclude=/local
--exclude=/packages --exclude=/.git"

Notice the --whole-file part there.

>
> For each 50k changelog, the median transfer would get 0.25% larger.
>
Well, we could just have less changes ;)

160GB/day per server is about 2MB/s, ~16Mbit, or about 5TB/month. That's
still included in the 'free' bandwidth that el cheapo hosters like
Hetzner provide with their smallest servers ...

Re: [gentoo-dev] Bug #565566: Why is it still not fixed?

[Patrick Lauer]

On 02/23/2016 07:07 PM, Alec Warner wrote:
> On Tue, Feb 23, 2016 at 9:14 AM, Patrick Lauer <patr...@gentoo.org
> <mailto:patr...@gentoo.org>> wrote:
>
> See https://bugs.gentoo.org/show_bug.cgi?id=565566
>
> Since we have ChangeLogs again (November) they've been in backwards
> order. Which is not really good - it breaks tools (like emerge
> --changelog) and makes it harder to read for humans.
>
> As a bonus it's inconsistent because the old Changelog-2015 files
> are in
> normal order, and the new ones are reversed. Which sense no makes.
>
> The suggestions in the bug are of great entertainment value, but they
> all avoid the simple idea of generating ChangeLogs in changelog
> (reverse
> chronological) order. Which would fix all tools and make almost every
> consumer of changelogs happy.
>
> So, can we please, after over 4 months of stalling, just fix this
> embarassment?
>
>
> I don't see any attached patches...so it looks like there is room for
> you to contribute.
>
> -A
>
>
from gitweb.gentoo.org -
infra/mastermirror-scripts.git

~line 167:

|

case $HOURS in
3|9|15|21) EGENCACHE_CHANGELOG="--update-changelogs --changelog-reversed
--changelog-output ChangeLog" ;;
esac


remove "--changelog-reversed"

Enjoy cookie.
(Of course this may require removing the existing changelogs first etc. ...)

|

[gentoo-dev] Bug #565566: Why is it still not fixed?

[Patrick Lauer]

See https://bugs.gentoo.org/show_bug.cgi?id=565566

Since we have ChangeLogs again (November) they've been in backwards
order. Which is not really good - it breaks tools (like emerge
--changelog) and makes it harder to read for humans.

As a bonus it's inconsistent because the old Changelog-2015 files are in
normal order, and the new ones are reversed. Which sense no makes.

The suggestions in the bug are of great entertainment value, but they
all avoid the simple idea of generating ChangeLogs in changelog (reverse
chronological) order. Which would fix all tools and make almost every
consumer of changelogs happy.

So, can we please, after over 4 months of stalling, just fix this
embarassment?

[gentoo-dev] Last rites: dev-lang/niecza{,bin}

[Patrick Lauer]

# Patrick Lauer <patr...@gentoo.org> (22 Feb 2016)
# Inactive upstream, build failures, obsoleted by rakudo/perl6
dev-lang/niecza
dev-lang/niecza-bin

[gentoo-dev] bidi / fribidi useflag harmonization

[Patrick Lauer]

We have two useflags expressing the same thing: bidi and fribidi.

I suggest collapsing it into one useflag, and to make it a global useflag.

Affected packages:

dev-libs/efl/metadata.xml:  Enable
bidirectional text support
games-action/supertuxkart/metadata.xml: Support for right-to-left languages
games-strategy/megaglest/metadata.xml:  Enable FriBIDi support
games-strategy/wesnoth/metadata.xml:Support for
right-to-left languages
media-libs/avidemux-plugins/metadata.xml:Enable
unicode bidirectional algorithm support via
dev-libs/fribidi.
media-video/ffmpeg/metadata.xml:Enables
fribidi support in the drawtext filter.
media-video/vdr/metadata.xml:   fribidi
support, for languages, written from right to left
x11-wm/fluxbox/metadata.xml:

Re: [gentoo-dev] Changing order of default virtual/udev provider

[Patrick Lauer]

On 02/17/2016 07:37 AM, Michał Górny wrote:
>
 * Both udev and eudev have pretty much feature parity, so there won't be
 any user-visible changes

 * udev upstream strongly discourages standalone udev (without systemd)
 since at least 2012

 (see for example:
 https://lists.freedesktop.org/archives/systemd-devel/2012-June/005516.html
 https://lkml.org/lkml/2012/10/3/618
 )

 So it'd be (1) following upstreams recommendations and (2) dogfooding
 our own tools. I don't see any downsides to this :)  
>>> I'm strongly against this, because:
>>>
>>> 1. It is a conflict-induced fork. As such, it will never be merged
>>> upstream and it will never be supported upstream. In fact, it is
>>> continually forces to follow upstream changes and adapt to them. eudev
>>> is more likely to break because of the Gentoo developer(s) working hard
>>> to merge upstream changes to their incompatible code.  
>> That was the entire point of the project. Upstream rejected any attempts
>> to do things that we actually needed and broke things claiming the
>> distributions were responsible for handling the breakage, so eudev was
>> started on the basis that we needed a project that would ensure that
>> changes in udev occur in a way that makes sense.
> What things? Things like 'let's try to spawn this script third time in
> case someone mounted something so that it happens to work this time'?
> Yes, that old behavior really made sense.
It did. Not having it made booting a lot more exciting, and exciting is bad.

Now you need to boot before you boot (mount all relevant filesystems
before starting PID1?), which is fragile and suddenly makes the
initramfs complex enough to require, err, a dhcp client, different fsck
implementations, and pretty much all that would have been in /bin or
/sbin before the /usr movearounding started. And firmware and kernel
modules, like this it's easy to go >150MB for a compressed initramfs if
you need firmware and a dhcp client to bring up the NFS share with /boot
(hey, with ceph it's even more exciting ...)

"initramfs is the new rootfs" - yeah, that sounds like a good idea,
until you figure out that the initramfs doesn't fit in /boot anymore
(kernels are ~3-4MB each, so 25MB is plenty, eh?) and you need to either
repartition or boot from USB.

And since there was no visible failure mode for us before, of course
some of us are very confused why a reasonable and effective feature got
pruned without replacement. Just because somewhere else it didn't work
100% - that's no reason to remove features for everyone!
>
>>> 2. Many of Gentoo users are programmers who appreciate the 'vanilla'
>>> API experience Gentoo often provides. Switching the defaults to a fork
>>> that is known to intentionally diverge from upstream goes against that
>>> principle. Programs written against eudev may not work correctly with
>>> upstream udev.  
>> If upstream udev were stable, that would be one thing, but it
>> intentionally diverges from itself continuously. The only experience
>> that could be reliably provided with upstream udev is one of continual
>> breakage.
> This is FUD, without any proof.
See your reply to (3) - you disagree with yourself!
>>> 3. eudev has fallen behind systemd/udev more than once in the past,
>>> and caused visible breakage to users this way.  
>> When?
> Whenever we installed new systemd and udev versions, and needed to bump
> the dependency in virtual, and eudev maintainers were nowhere to be
> found.
Which was only needed due to API changes and/or feature removal. Which
is exactly what you claimed didn't happen, so go FUD yourself :)
>
>> Can we also consider all of the times udev broke the boot process
>> because upstream just didn't care about doing changes in a sane way and
>> the people interested in providing the upstream experience delivered on
>> that goal?
> Proof needed.
"Persistent network naming" caused me at least three independent boot
failures -

(1) network device got renamed away from eth0 by default. That's
horrible, why would you change the default like this?!
(If it were an option that I had to consciously turn on it'd be great)

(2) persistent naming only triggers when net link status is up. Thus if
the switch takes more time than the server (which it did) the renaming
did *not* happen, init script looks for en3p7 or whatever but now it's
eth0 again. This means I can't automatically power on systems after
power failure ...

(3) race condition in persistent naming renames on average 2 out of 4 of
the interfaces on bnx2 / bnx2x quad ethernet cards. So you may have
en3p0, eth1, eth2, en3p4 ... or eth0, en3p1, eth3, eth2.

The only available fixes for this are *not* using the persistently buggy
renamer thingy and instead use either MAC-based naming or pass
'net.ifnames=0' on the kernel command line to keep the stable kernel names.

At my current workplace we're stapling it to the MAC address - that way
whenever a NIC fails we need 

Re: [gentoo-dev] Changing order of default virtual/udev provider

[Patrick Lauer]

On 02/16/2016 08:33 PM, Michał Górny wrote:
> On Tue, 16 Feb 2016 20:14:03 +0100
> Chí-Thanh Christopher Nguyễn  wrote:
>
>> Alexis Ballier schrieb:
>>> I also fail to see how udev using a new linux ipc would make it require
>>> systemd. Quoting Lennart:
>>> "You need the userspace code to set up the bus and its policy and handle
>>> activation. That's not a trivial task. For us, that's what sytemd does
>>> in PID 1. You'd need to come up with an alternative for that."
>>>
>>> If it's just that, it's not limited to udev, but anything using
>>> kdbus/bus1, and would mean openrc/${favorite init system} will have to
>>> do the same thing anyway. But again, almost 2 years is extremely
>>> old considering all the flux that has been around kbus.  
>> OpenRC itself can for now just ignore kdbus, bus1, or whatever kernel 
>> IPC system comes next. But if upstream udev makes use of the systemd 
>> userspace interface to the kernel IPC system, then OpenRC would have to 
>> implement the same interface in order to have working udev.
>>
>> Also given the close relationship between systemd and udev, there is no 
>> guarantee that supporting other users of kdbus/bus1 will make udev 
>> automagically work. As these two are released together, there is no 
>> reason to have a stable, public API between them.
> This all is going into some bickering nonsense and noise made by
> systemd haters just to feed their troll, FUD and whatever else they
> made around here.
You call it hate, I call it having a choice.
If I didn't want a choice I'd be using MacOS anyway, so please don't
claim to understand my motivations (and why they are obviously wrong,
since you know The Truth)

>
> So, yes, we should definitely switch to semi-maintained,
> semi-documented fork made plainly of systemd hate.
You mean sys-fs/udev ? I'd say unsupported instead of semi-maintained ;)

>  Because certainly
> project that is created plainly for political reasons is better.
> Because it will certainly be technically better if people have to focus
> on copying regular udev maintainers and reworking their changes to keep
> them working on forked codebase.
>
> And after all, as someone said, this will give eudev proper testing.
(1) It's already used by lots of people
(2) 'proper' testing? As opposed to be the default in more than a dozen
distros that have usecases you and I rarely think of?
> Because why default to something tested and working when you can throw
> your random fork on our users. After all, if we force them to use it,
> they will eventually start co-maintaining it, and it will no longer be
> semi-maintained!
>
I have no idea why you think eudev is so horrible, but you're entitled
to having opinions.

And we don't throw it on our users more than we do now: If you don't
like it just remove it. emerge -C is easy!
You make it sound like we're removing choice, which is just ... how the
... what are you?!?

The whole discussion, which seems to turn everyone into a raging
squirrel, is about changing the default provider of a virtual. All other
providers will continue being listed and available. The change affects
none of the current userbase (who seem to have a preference for eudev if
forums polls have any meaning).

The change will only affect the default selected when no udev provider
is already installed. This would finally allow releng to have eudev on
stage3, which so far they were unable to do without manually overriding
defaults.

^^ That is pretty much all that changes. Seriously.


I have no idea why this topic that doesn't even affect the most vocal
neigh-sayers in this discussion seems to be so insanely difficult ...

Re: [gentoo-dev] rfc: Does OpenRC really need mount-ro

[Patrick Lauer]

On 02/16/2016 07:05 PM, William Hubbs wrote:
> All,
>
> I have a bug that points out a significant issue with
> /etc/init.d/mount-ro in OpenRC.
>
> Apparently, there are issues that cause it to not work properly for file
> systems which happen to be pre-mounted from an initramfs [1].
I don't understand how this fails, how does mounting from the initramfs
cause issues?

The failure message comes from rc-mount.sh when the list of PIDs using a
mountpoint includes "$$" which is shell shorthand for self. How can the
current shell claim to be using /usr when it is a shell that only has
dependencies in $LIBDIR ?
As far as I can tell the code at this point calls fuser -k ${list of
pids}, and fuser outputs all PIDs that still use it. I don't see how $$
can end up in there ...

>
> This service only exists in the Linux world; there is no equivalent in
> OpenRC for any other operating systems we support.
>
> The reason it exists is very vague to me; I think it has something to do
> with claims of data loss in the past.
Yes, if you just shut down without unmounting file systems -
(1) you may throw away data in the FS cache that hasn't ended up on disk yet
(2) the filesystem has no chance to mark itself cleanly unmounted, so
you will trigger journal replay or fsck or equivalent on boot

That's why sysvinit had a random "sleep(1)" in the halt and "sleep(2)"
in the reboot function, to give computers more of a chance to shutdown
and reboot sanely.

The changes in sysvinit-2.88-r8 and later add the "-n" option:
   -n Don't sync before reboot or halt. Note that the kernel and
storage drivers may still sync.

This was added *because* we can guarantee that filesystems are
consistent enough with mount-ro. If you wish to remove it you need to
reconsider all these little details ...
>
> I'm asking for more specific information, and if there is none, due to
> the bug I lincluded in this message, I am considering removing this
> service in 0.21 since I can't find an equivalent anywhere else.
Please don't just remove things you don't understand.
>
> Thanks,
>
> William
>
> [1] https://bugs.gentoo.org/show_bug.cgi?id=573760

Looking at the init script as of openrc-0.20.5:

~line32:
# Bug 381783
local rc_svcdir=$(echo $RC_SVCDIR | sed
's:/lib\(32\|64\)\?/:/lib(32|64)?/:g')
This looks relatively useless with everything migrated to /run and can
most likely be removed

~line35:
  local
m="/dev|/dev/.*|/proc|/proc.*|/sys|/sys/.*|/run|${rc_svcdir}" x= fs=
Since this is a regexp it can be cut down to something more simple - why
/dev and /dev/* when the second one is already excluded by the first
one. Also rc_svcdir is most likely a subdir of /run ...

Re: [gentoo-dev] Changing order of default virtual/udev provider

[Patrick Lauer]

On 02/14/2016 09:17 PM, Rich Freeman wrote:
> On Sun, Feb 14, 2016 at 2:41 PM, Brian Dolbec <dol...@gentoo.org> wrote:
>> On Sun, 14 Feb 2016 11:00:30 -0500
>> Rich Freeman <ri...@gentoo.org> wrote:
>>
>>> On Sun, Feb 14, 2016 at 10:14 AM, Patrick Lauer <patr...@gentoo.org>
>>> wrote:
>>>> If, for any reason, eudev should be abandoned - we can just change
>>>> the virtual back. One-line change.
>>> Which is precisely the corresponding argument for not switching the
>>> default to eudev in the first place.
>>>
>> OH, my, this is looking more like you are being paid by systemd peeps...
> Nobody has ever paid me to do anything involving open-source software,
> systemd or otherwise.
>
> My point is just that there is no need to change today, because:
> 1.  udev works just fine today
> 2.  If udev doesn't work just fine in the future, we can just change
> the virtual.  One-line change.
>
> That's all.  I'm not saying that there might not be other reasons to
> change the virtual.
>
> I'm just saying that the possibility that udev might break in the
> future isn't any more a reason to change the virtual than the
> possibility that eudev might be abandoned in the future.
>
> I love it when Patrick violently agrees with me.  :)
>
Eh yes. If we can avoid a problem we better wait until there is visible
breakage so we can heroically run around like headless chickens and
people see that we do something.

You're definitely of the "Fireman Sysadmin" type that doesn't want to do
preventative maintenance :)


Why are you so insistent on controlling something that doesn't even
affect you?  I mean ... the only difference for you would be that from a
default stage3 you do "emerge -C eudev" instead of "emerge -C udev" and
then you're on your way.

As far as users are concerned, most don't care and won't see a
difference, and those that care seem to be strongly in support of having
eudev ...

Re: [gentoo-dev] Changing order of default virtual/udev provider

[Patrick Lauer]

On 02/14/2016 09:23 PM, Mike Frysinger wrote:
> On 14 Feb 2016 11:41, Brian Dolbec wrote:
>> On Sun, 14 Feb 2016 11:00:30 -0500 Rich Freeman wrote:
>>> On Sun, Feb 14, 2016 at 10:14 AM, Patrick Lauer wrote:
>>>> If, for any reason, eudev should be abandoned - we can just change
>>>> the virtual back. One-line change.  
>>> Which is precisely the corresponding argument for not switching the
>>> default to eudev in the first place.
>> OH, my, this is looking more like you are being paid by systemd peeps...
> honestly ?  cut the crap man.
>
>> You are just refusing to acknowledge these simple facts.
>>
>> systemd.:  irrelevant to this decision
>>
>> standalone systemd-udev.:  Vehemently unsupported, support for its
>>capability to exist is planned to be punted
>>in the future.
>>
>> eudev...:  fully functional, actively developed,
>>and fully supported, mature project, been
>>around for years.
> udev: it's the default in every major distro that everyone tests and
> develops against.
Not the standalone config we're using, so if you remove all
systemd-using distros which are irrelevant to this discussion you end up
with gentoo, and ~15 distros that use eudev. And of course other
irrelevant weirdos that use mdev, vdev etc.
>
> eudev: no one of any relevance outside of Gentoo runs it.
No one runs udev either. So that's a non-argument


So given the context of this discussion, and your ignorant contribution
... maybe you should cut the crap, man. Being a bit more polite wouldn't
be wrong either.

Re: [gentoo-dev] Uncoordinated changes

[Patrick Lauer]

On 02/12/2016 11:02 PM, Michał Górny wrote:
> On Fri, 12 Feb 2016 10:07:10 +0100
> Patrick Lauer <patr...@gentoo.org> wrote:
>
>> On 02/12/2016 08:48 AM, Michał Górny wrote:
>>> Dear Ignorant Patrick,  
>> Hello human! Your politeness module seems to have crashed.
> Please do not expect politeness when you insult someone.
>
>
>> So now I know that in the future I will
>>
>> (1) categorically deny any change requests coming from you and
>> (2) block/revert any changes that I don't like or understand.
>>
>> Nothing personal, just basic sanity.
> If you want to leave Gentoo, please do that without unnecessary drama.
> That will reduce the load on ComRel and/or QA resulting from your CoC
> violations and childish behavior, and reduce the discouragement you're
> causing to other developers.
>
Now now. That's a severe case of projection, and I should take offense
at it. But that's silly, so please stop being such a drama queen and
maybe don't start yet another project this month until you've finised at
least one of the half dozen unfinished things that you started.


If you feel insulted because I pointed out some breakage caused by you
(which now you claimed you did against your wishes, which is a strange
thing for voluntary work) then I suggest you hide breakage (oh wait,
"differently behaving") better so I don't notice it. And if I
discouraged you from working on more things then that's good too ...

Re: [gentoo-dev] Uncoordinated changes

[Patrick Lauer]

On 02/14/2016 07:44 PM, Andreas K. Hüttel wrote:
> On Sunday 14 February 2016 13:18:30 Rich Freeman wrote:
>> Feel free to peruse the various list discussions and council logs.
>> Most of what you're bringing up has come up before.
> Thanks rich0, you seem to be reading my mind.
>
> This is turning into a severe case of "I didn't bother to speak up earlier or 
> volunteer to improve something, and now I'm unhappy with what was decided and 
> implemented."
>
Silly me for assuming that changes to metadata would not affect me.
Or that tools would be adapted to the changes.

Or that the changes would make sense.

I don't have time to follow every little change, so it's quite annoying
to reverse-engineer this change that apparently is a compromise that no
one actually wanted, and hasn't really been finished yet. Sigh.

Re: [gentoo-dev] Changing order of default virtual/udev provider

[Patrick Lauer]

On 02/09/2016 01:17 PM, Rich Freeman wrote:
> On Tue, Feb 9, 2016 at 3:43 AM, Kent Fredric  wrote:
>
>> And a lot of Gentoo is surprisingly simple: Like our use of bash
>> scripts for recipies to build things, like using rsync to deploy/relay
>> not just those recipies, but security notices and  news items, which
>> are themselves reasonably simple formats.
> Well, one thing about Gentoo that certainly isn't simple is our init.d 
> scripts.
>
> Compare this:
> http://pastebin.com/sSDtpF4t
More stable link:
https://gitweb.gentoo.org/proj/apache.git/tree/2.4/init/apache2.initd
>
> With this:
> http://pastebin.com/Lfn8r7qP
More stable link:
https://gitweb.gentoo.org/repo/gentoo.git/tree/www-servers/apache/files/apache2.2.service
> Systemd does the job in 10% of the code (and half of it is a comment),
> and doesn't implement its own service polling and killer script during
> shutdown independently for every service (not that every init.d script
> even does this - most of them will just leave orphans behind, and
> systemd will catch orphans that even the lengthy init.d script for
> apache misses).
>
Right, that's a bad comparison.

The equivalent OpenRC init script is:

```
#!/sbin/runscript
command="/usr/sbin/apache2"
command_args="${APACHE2_OPTS}"
description_reload="A graceful restart advises the children to exit
after the current request and reloads the configuration."

stop() {
$command $APACHE2_OPTS -k graceful-stop
}
reload() {
$command $APACHE2_OPTS -k graceful
}
```
So that's almost exactly the same (modulo braces and newlines). There's
no equivalent for PrivateTmp, and we ignore the extra data in
/etc/tmpfiles.d (for creating runtime dirs). Which is bad, but that's
another rant ;)

Just that the current initscript does a lot more, and ... uhm ... why is
the systemd unit sourcing /etc/conf.d/apache2 ?
(Oh, and dependencies, but those just slow down startup )

So if you compile the equivalent naive init script there's not much
difference, and the initial argument falls on its face and disappears.

I'm getting tired of having this argument :)

Re: [gentoo-dev] Uncoordinated changes

[Patrick Lauer]

On 02/14/2016 02:16 AM, Rich Freeman wrote:
> On Sat, Feb 13, 2016 at 7:58 PM, Raymond Jennings  wrote:
>> So what do you guys think of leaving behind empty stubs for compatibility
>> and then simply filing a tracking bug blocked by any packages that removing
>> herds broke?
> It isn't entirely clear that anything is actually broken at the
> moment, but if distributing an empty herds.xml file makes somebody's
> life easier I have no objections.  I don't think it would have
> alleviated Patrick's original concerns in this case.
>
Nope :)

But it would have made debugging easier.

Re: [gentoo-dev] Changing order of default virtual/udev provider

[Patrick Lauer]

On 02/09/2016 10:03 AM, Kent Fredric wrote:
> On 9 February 2016 at 18:27, Anthony G. Basile  wrote:
>> all the vitriolic attacks i get about eudev come from the gentoo
>> community.  outside of this community i get praise.
>
> In case my  earlier messages stating a desire to exercise much caution
> gave the wrong impression, I just want to state for the record I think
> its awesome eudev exists, and I think its awesome other distro's are
> using it.
>
> Just when it comes to "Change the defaults", I want to be certain
> about the path we're setting ourselves up for on this very important
> component, because here, a change of defaults paves a broader path for
> eudev to be a potential leading competition to systemd.
If, for any reason, eudev should be abandoned - we can just change the
virtual back. One-line change.
>
> Because if we do that, I feel we must be so sure of ourselves that
> eudev can be a profitable choice for at least a moderately long term,
> even in the event we get no more support from the systemd codebase.
>
> Having it in tree and having users who know what they're doing being
> able to choose their own risk factors and say "yeah, eudev is the
> right choice for what I'm doing" is one thing.
>
> But stating implicitly that "Hey, this is the default", this needs to
> be the *best* recommendation we can make based on all the other
> factors and other defaults Gentoo uses.
Given the options of {systemd, systemd-udevd standalone, eudev} -
Those that choose systemd are not in this discussion.
Systemd-udevd standalone is unsupported, with upstream suggesting that
they want to remove support for it.
Leaves eudev as the only 'sane' option, since we even have an upstream.

And it's the 'mainstream' choice.

And it wins the popular vote:
https://forums.gentoo.org/viewtopic-t-1038696-postdays-0-postorder-asc-highlight-eudev-start-0.html

>
> Because new users *will* be inclined to pick the default, and
> *existing* users of the *current* default are likely to see the
> default change, and reason "well, the default has changed, so the new
> default is the new best thing", and will be inclined to switch.
Existing installs won't have a visible change. Since a provider of
virtual/udev is installed nothing changes, even if we shuffle the
virtual providers around.
>
> And the worst thing we could have is a combination of bad defaults
> that leads new users to have a poor first experience because they used
> all the default recommendations, and their system made magic smoke
> instead of booting.
>
> In short, to change *this* default, it seems pertinent that we *know*
> that the change we're making *is* better and a more reliable long-term
> choice than the *current* default, and if there is *any reasonable
> doubt*, we should delay changing until that reasonable doubt is
> expunged.
>
Since eudev is a drop-in replacement that uses a good part of the
original udev codebase - if you notice any deviation it's either a bug
(which we should fix) or udev misbehaves (e.g. persistent network
naming, which is a wonderful way to make people with more than zero
network cards angry)

And again: It would only affect what gets installed if you do "emerge -C
udev eudev; emerge virtual/udev"
(and, as a positive side-effect, changes the udev implementation of
stage3, which again only affects the default on *new* installs)

I don't see any serious doubts, apart from "we're deviating from
upstream" - but that's exactly what I want to fix! Yes, we deviate
*now*, we should not do that. And there's reasonable doubt that we can
keep sys-fs/udev going.


I like it when people violently agree with me :)

Re: [gentoo-dev] Changing order of default virtual/udev provider

[Patrick Lauer]

On 02/14/2016 05:00 PM, Rich Freeman wrote:
> On Sun, Feb 14, 2016 at 10:14 AM, Patrick Lauer <patr...@gentoo.org> wrote:
>> If, for any reason, eudev should be abandoned - we can just change the
>> virtual back. One-line change.
> Which is precisely the corresponding argument for not switching the
> default to eudev in the first place.
>
That no sense :)

Since we're, right now, already using an unsupported version ...
Can't get worse.

And since you argued for mainstream earlier you'd have to agree that
eudev is the mainstream solution and Gentoo is lagging behind the others ...

Re: [gentoo-dev] Uncoordinated changes

[Patrick Lauer]

On 02/11/2016 09:15 PM, Patrick Lauer wrote:
> Now instead of looking up [metadata.xml] -> (herd name) -> [herds.xml]
> -> email it goes backwards:
> [metadata.xml] -> (maintainer type=project) -> email -> [projects.xml]
> -> Project name
>
> Since this involves XML and python's ElementTree library it's a
> nontrivial change that also removes a few now useless helpers
> (_get_herd_email has no reason to be, but we'd need a _get_herd_name
> helper instead. Err, get_proj ... ah well, whatever name works)
>
> And all that just so (1) gentoolkit output works and (2) euscan updates
> properly. Both of which I don't really care about much, but now that
> I've invested ~4h into debugging and trying to fix it I'm a tiny bit
> IRRITATED.
>
So this turns out to be more fun than expected.

Having spent a little bit of time staring at XML, DTDs and wondering why
we do things the most difficult way ...

Previously the herd tag was defined as:


So we end up with, for example:
kde

The new schema collapses herd (err, project!) into maintainers (err,
sustainers ... staff ... linchpin?)
And maintainer is defined as:


Which means that only email is mandatory. So instead of search by name
you are now required to search by email.
And it leads to inconsistent (partial) duplication: Some metadata.xml
entries carry Name, some Description, and some are Email only.

For example for gentoolkit this means that instead of search by name now
it needs to be search by email, and the previous search by name
functionality requires herds.xml, err, projects.xml to figure out the
name of a project. Which might not match the one in metadata.xml!
(And you may need to filter out maintainers-that-are-not-projects, and
what about maintainers that are undefined? So much extra code complexity!)

And this is why I avoided the topic and hoped that the 'migration' would
make sense:
(1) Using XML is mildly insane. Neither machine- nor human-readable
(2) The DTD is even more insane, and few people have the patience to
figure it out
(3) The recent changes to the DTD change the data model in subtle ways
so that there's even *more* denormalization possible
(4) The tooling is, due to XML, wonderfully horrible and requires things
like XPATH to get the required data (because query by attribute is
harder than query by tag)

There's fundamental questions that should be handled before doing more
modifications - for example, should the data be more normalized (e.g.
name only in projects.xml / maintainers.xml and only email in
metadata.xml)? If we allow denormalization, do we have tools to check
and autocorrect (e.g. a maintainer changing name)?

Once we decide to abstract it away so that people should use tools and
not mangle it manually (have you looked at herds.xml ?! omg ...) there's
the question ... why XML? It's about the worst format for this job, INI
format is sufficient and easier to parse. Or JSON, or YAML, or whatever
is trendy now. Or do we autogenerate from templates?

Another funny thing: projects.xml is not in the same repository, so
synchronizing changes gets more tricky. And the metadata.dtd is in yet
another place. Wouldn't it make sense to have this organized in a less
confusing way?

You see where this is going - and why I didn't object loud enough to the
changes: I want to not care about this whole cluster of topics and do
things that are more rewarding. But that choice got taken away when
things broke (oh, they didn't break, they Function Differently now) and
I had to spend some time investigating why things deviate.

Sigh.


Am I grumpy?

Re: [gentoo-dev] Uncoordinated changes

[Patrick Lauer]

On 02/12/2016 08:48 AM, Michał Górny wrote:
> Dear Ignorant Patrick,
Hello human! Your politeness module seems to have crashed.

And thanks for making me do a quintuple facepalm with backflip. I think
that's a new record. So anyway ...
> On Thu, 11 Feb 2016 21:15:34 +0100
> Patrick Lauer <patr...@gentoo.org> wrote:
>
>> ... or why just changing stuff is not enough:
>>
>> A few days ago I was told that
>> http://euscan.gentooexperimental.org/herds/ was displaying an empty
>> list. Which is annoying because people sometimes want to see what
>> upstream updates are available for their herd.
>>
>> Well, we renamed herd to project. Because reasons.
> No, we didn't. Herd was collection a packages. Project is a collection
> of developers. Project coexisted with herds for a long time. As it was
> explained already in length. Multiple times.
So you just pivoted the organization from A->B to B->A.

I still don't see the advantage in that. Maybe I should have expressed
my concerns more vocally, but in general I don't have time to worry
about all the little things.

>
>> I don't care how it is named, but this change broke euscan in a
>> user-visible way. Now I could just try to rename things there too, but
>> that won't work:
>>
>> euscan uses gentoolkit for parsing metadata.xml and herds.xml
>> (Since herds.xml is basically unmaintained cruft at this point this will
>> break soon anyway ... but ...)
>> Changing gentoolkit to use projects.xml instead of herds.xml won't be a
>> simple migration since the data organization changed.
>>
>> Now instead of looking up [metadata.xml] -> (herd name) -> [herds.xml]
>> -> email it goes backwards:  
>> [metadata.xml] -> (maintainer type=project) -> email -> [projects.xml]
>> -> Project name  
>>
>> Since this involves XML and python's ElementTree library it's a
>> nontrivial change that also removes a few now useless helpers
>> (_get_herd_email has no reason to be, but we'd need a _get_herd_name
>> helper instead. Err, get_proj ... ah well, whatever name works)
>>
>> And all that just so (1) gentoolkit output works and (2) euscan updates
>> properly. Both of which I don't really care about much, but now that
>> I've invested ~4h into debugging and trying to fix it I'm a tiny bit
>> IRRITATED.
> You are completely incorrect, as you have been told already multiple
> times. People would really appreciate if you spent at least a little
> part of the time you spend complaining, inventing issues and insulting
> others listening to what they're telling you.
>
> So let me repeat, again. euscan works. Want packages from Python
> project? Then select the appropriate maintainer from the 'maintainers'
> section:
So you're saying I have no way to search by herd, err, project now.
And I can't even find projects in the soup of maintainers.

... and metadata is now partially broken.

*ahem* This not of good idea sounding.
>
> http://euscan.gentooexperimental.org/maintainers/pyt...@gentoo.org/
>
> Done. Was it that hard? Now the big surprise: you didn't have to create
> some convoluted logic to get that! You don't need projects.xml to get
> that! Of course, you'd know that if you would listen for a single
> minute instead of throwing insults at others.
If you had actually understood my criticism you would understand why I
might be a tiny bit irritated.

Some functionality is now actively *gone*, and that's not a feature.
>
>> Please, next time someone has the brilliant idea of changing stuff just
>> to change it (I still don't see a reason why we had to change
>> metadata.xml?), it should be required that support tools are fixed
>> *before* the change, and working versions released. This avoids grumpy
>> people and makes it harder for those that change things to head-in-sand
>> and claim everything works as expected when it obviously doesn't.
> The fact is: things *work as expected*. If you have problem accepting
> reality as it is, then it's your fault, not ours. Herds no longer
> exist. Everything is based on *maintainers* now. Tools are not supposed
> to magically turn project information back into herd-oriented design.
Right, so gentoolkit returning bad info is a good thing. I find that
hard to integrate into my understanding of the world ...


Please don't redefine what 'expected' means to suit your limited
usecases. It just causes friction and unhappy response from people that
now have to spend lots of time figuring out how things diverge from
their 'expected', which usually ends in *facepalm* omg how did that happen.

Plus the usual sequence of strongly-worded letters to the UN ;)
>
> As I said before, please direct any further complaints direc

[gentoo-dev] Uncoordinated changes

[Patrick Lauer]

... or why just changing stuff is not enough:

A few days ago I was told that
http://euscan.gentooexperimental.org/herds/ was displaying an empty
list. Which is annoying because people sometimes want to see what
upstream updates are available for their herd.

Well, we renamed herd to project. Because reasons.

I don't care how it is named, but this change broke euscan in a
user-visible way. Now I could just try to rename things there too, but
that won't work:

euscan uses gentoolkit for parsing metadata.xml and herds.xml
(Since herds.xml is basically unmaintained cruft at this point this will
break soon anyway ... but ...)
Changing gentoolkit to use projects.xml instead of herds.xml won't be a
simple migration since the data organization changed.

Now instead of looking up [metadata.xml] -> (herd name) -> [herds.xml]
-> email it goes backwards:
[metadata.xml] -> (maintainer type=project) -> email -> [projects.xml]
-> Project name

Since this involves XML and python's ElementTree library it's a
nontrivial change that also removes a few now useless helpers
(_get_herd_email has no reason to be, but we'd need a _get_herd_name
helper instead. Err, get_proj ... ah well, whatever name works)

And all that just so (1) gentoolkit output works and (2) euscan updates
properly. Both of which I don't really care about much, but now that
I've invested ~4h into debugging and trying to fix it I'm a tiny bit
IRRITATED.



Please, next time someone has the brilliant idea of changing stuff just
to change it (I still don't see a reason why we had to change
metadata.xml?), it should be required that support tools are fixed
*before* the change, and working versions released. This avoids grumpy
people and makes it harder for those that change things to head-in-sand
and claim everything works as expected when it obviously doesn't.


And this, again, supports my default phrase: "Change is not Progress",
because now we have regressions and user-visible breakage, and it's hard
to qualify how the change actually improved anything because we haven't
actually *finished* the change. (Like the git migration that is still
ongoing ...)

Do the needfull! Be of excellent moral person!

[gentoo-dev] Changing order of default virtual/udev provider

[Patrick Lauer]

Ohey,

I've opened a bug at:
https://bugs.gentoo.org/show_bug.cgi?id=573922

The idea here is to change the order of the providers of virtual/udev.
For existing installs this has zero impact.
For stage3 this would mean that eudev is pulled in instead of udev.

The rationale behind this is:

* eudev is an in-house fork, and there's more than a dozen distros
already using it by default that are not us. Which is a little bit weird ...

* Both udev and eudev have pretty much feature parity, so there won't be
any user-visible changes

* udev upstream strongly discourages standalone udev (without systemd)
since at least 2012

(see for example:
https://lists.freedesktop.org/archives/systemd-devel/2012-June/005516.html
https://lkml.org/lkml/2012/10/3/618
)

So it'd be (1) following upstreams recommendations and (2) dogfooding
our own tools. I don't see any downsides to this :)

Re: [gentoo-dev] Changing order of default virtual/udev provider

[Patrick Lauer]

On 02/08/2016 01:30 PM, Daniel Campbell wrote:
>
> Out of curiosity, which distros are shipping with eudev by default?
>
>From [1]:

"""
1. AUSTRUMI switched to eudev in March 2013 (see package list for the
2.6.8 release).

2. Parted Magic switched to eudev in August 2013.

3. Quirky (experimental version of Puppy Linux) switched to eudev in
December 2013.

4. 0linux switched to eudev in February 2014 (see base packages for the
eta release).

5. Linux From Scratch (standard version) switched to eudev in March 2014
(see this commit).

6. Vine Linux switched to eudev in June 2014.

7. Funtoo Linux switched to eudev in June 2014.

8. CRUX switched to eudev in July 2014.

9. Void Linux switched to eudev in July 2014 (see this commit).

10. Guix System Distribution switched to eudev in September 2014.

11. NuTyX switched to eudev in October 2014 (see system packages for the
Saravane release).

12. Puppy Linux (standard version) switched to eudev in October 2014
(see package list for the 6.0 tahrpup release).

13. Manjaro Linux (OpenRC edition) has used eudev since the initial
release in December 2014.

14. Calculate Linux switched to eudev in April 2015.

15. Alpine Linux (desktop edition) switched to eudev in July 2015.
"""

[1] https://forums.gentoo.org/viewtopic-t-1003230.html

Re: [gentoo-dev] New USE_EXPAND NGINX_MODULES_STREAM

[Patrick Lauer]

On 02/08/2016 11:59 PM, Luis Ressel wrote:
> On Tue, 9 Feb 2016 11:34:12 +1300
> Kent Fredric  wrote:
>
>> nginx_modules_http_geoip? ( dev-libs/geoip )
>> nginx_modules_http_gunzip? ( sys-libs/zlib )
>> nginx_modules_http_gzip? ( sys-libs/zlib )
>> nginx_modules_http_gzip_static? ( sys-libs/zlib )
>> nginx_modules_http_image_filter? ( media-libs/gd[jpeg,png] )
>> nginx_modules_http_perl? ( >=dev-lang/perl-5.8 )
>> nginx_modules_http_rewrite? ( >=dev-libs/libpcre-4.2 )
>> nginx_modules_http_secure_link? (
>> userland_GNU? (
>> !libressl? ( dev-libs/openssl:0= )
>> libressl? ( dev-libs/libressl:= )
>> )
>> )
>> nginx_modules_http_xslt? ( dev-libs/libxml2 dev-libs/libxslt )
>> nginx_modules_http_lua? ( !luajit? ( dev-lang/lua:0= ) luajit?
>> ( dev-lang/luajit:2= ) )
>> nginx_modules_http_auth_pam? ( virtual/pam )
>> nginx_modules_http_metrics? ( dev-libs/yajl )
>> nginx_modules_http_dav_ext? ( dev-libs/expat )
>> nginx_modules_http_security? ( >=dev-libs/libxml2-2.7.8
>> dev-libs/apr-util www-servers/apache )
>> nginx_modules_http_auth_ldap? ( net-nds/openldap[ssl?] )
>>
> Thanks for citing this, I think it demonstrates mgorny's point rather
> nicely; we have global USE flags for many of those modules:
>
> * nginx_modules_http_perl -> perl
> * nginx_modules_http_auth_pam -> pam
> * nginx_modules_http_auth_ldap -> ldap
> * nginx_modules_http_geoip -> geoip
> * nginx_modules_http_g(un)zip -> zlib
> * nginx_modules_http_secure_link -> ssl
>
> The following two ones aren't quite as obvious, but could also be
> changed:
> * nginx_modules_http_rewrite -> pcre
> * nginx_modules_http_image_filter -> gd
>
> Introduce new USE flags for the remaining few modules -- voilà, there
> you go, no need for a new USE_EXPAND and the users will even get a
> useful set of default modules enabled based on their global USE flags.
>
And now I can't figure out what I need to enable to have "rewrite" work.
Good job!

The names match the internal module names, which is what I care about.
Figuring out if I need USE="zlib" or USE="compress" or even a combo is a
lot more effort and frustrating than having to enable the useflag that
has the name of the module.

It might not be 'pure' or very aesthetical, but we try to get stuff done
here.

Re: [gentoo-dev] Automatic Bug Assignment

[Patrick Lauer]

On 01/30/2016 06:45 PM, Alex Brandt wrote:
> Hey Guys,
>
> I've oft wondered why we don't automatically assign bugs to the 
> ebuild maintainer (if a CPV is in the subject).  Would there be an 
> issue with adding a bug modification hook to bugzilla or a daily 
> job to re-assign bugs to ebuild owners (if a CPV is in the 
> subject)?
>
> Just curious not trying to incite anything.
>
> Regards,
>
Maybe we could add a "Assign to maintainer(s)" button visible only to
certain groups of users, so that a bugwrangler who decides this bug is
valid just has to hit one button instead of figuring out the details of
assignment?

There seems to be valid criticism about fully automating the workflow,
but partial automation can still save huge amounts of time ...

Re: [gentoo-dev] Packages Up For Grabs

[Patrick Lauer]

On 01/24/2016 11:49 AM, M. J. Everitt wrote:
> On 24/01/16 10:39, Amadeusz Żołnowski wrote:
>> Alex Brandt  writes:
>>> * app-backup/rdiff-backup
>> Wasn't it meant for removal?
>>
>> -- Amadeusz Żołnowski
> Looks fine on p.g.o  stable too. Upstream site is present.
>
Inactive upstream, known data loss bugs, should be nuked until dead.

Re: [gentoo-dev] Re: repo/gentoo.git, or how committing is challenging

[Patrick Lauer]

On 12/22/2015 03:04 PM, Rich Freeman wrote:
> On Tue, Dec 22, 2015 at 8:31 AM, Patrick Lauer <patr...@gentoo.org> wrote:
>>> Do you want to see this fixed?
>>> Are you willing to do the fixing yourself?
>> I don't have infinite time, and wasting a day documenting things that
>> should have been documented a year ago is not a good way of spending time.
> So, it sounds like the answer to the first question is yes, and the
> second is no...
I shouldn't have to clean up other people's mess. I mean, cleaning up
after toddlers is ok, but, well ... sigh.
Are any of you toddlers? :D

>
>>> If the answer to the first question is yes, and the second is no, then
>>> you've just volunteered for the job of either community motivator, or
>>> frustrated user.  The goal then is to make people care more about
>>> going out of their way to fix things than going out of their way to
>>> find ways to make you even more frustrated.  Which do you think is
>>> going to be more emotionally satisfying to those who read this thread?
>>>
>> Things working.
>>
>> So, the trick is not to have user-visible breakage.
>>
>> Now you know the great trick too and can apply it to your daily life.
>>
> Yeah, but if I don't lift a finger to help fix this bug, I know it
> will drive you crazy for a few more days, or even months.  That's
> basically my point.  You're basically begging everybody who would
> otherwise want to fix this issue to just troll you instead.  And that
> isn't helpful to anybody.
>
> You can't just wave your hands and have no user-visible breakage.  You
> either need to fix things yourself or help motivate others to do it.
> The approach you're taking is about as helpful as telling your
> significant other that they're fat.  After they're finished stabbing
> you to death with their spoon they're going to stick it in some Ben
> and Jerry's.
>
> Ugh, gotta take a break.  Happy holidays!
>
So here's the magic:

Create a file "keyspec.txt" containing something like:

"""
%echo Generating a basic OpenPGP key
Key-Type: RSA
Key-Length: 4096
Key-Usage: sign
Expire-Date: 1y
Subkey-Type: RSA
Subkey-Length: 4096
Subkey-Usage: sign
Name-Real: Patrick Lauer
Name-Email: changet...@email.xyz
Passphrase: ThisIsBadPassphrayse
%commit
%echo done
"""
Not that Name-* and Passphrase should be personalized (or Passphrase
removed!)

Now make a backup of .gnupg because this will be destructive.
Do not run this command as a normal user unless you are sure you want to
overwrite the default keyring.

So now that we are sure that you don't accidentally all your keys (do
not run this for fun! This is a destructive command)

"""
gpg --batch --gen-key keyspec.txt
"""

Tadaah! (well, it'll take a few minutes, since gpg wants really sexy
entropy and takes its time to get there)

The part I haven't figured out yet is how to non-interactively set key
validity, so you will need to run a second command:

"""
gpg --edit-key expire
"""
and set validity to, say, 3 years, confirm, save, done.


(This mostly obsoletes the 500 lines of eyebleed that were gkeys-gen,
and it actually works. Do you understand why I'm feeling very confused
that something this trivial took over a year?!)

Time from start of RTFM to email: 1h.

Re: [gentoo-dev] Re: repo/gentoo.git, or how committing is challenging

[Patrick Lauer]

On 12/21/2015 04:21 AM, Ryan Hill wrote:
> On Sun, 13 Dec 2015 19:00:45 -0800
> Brian Dolbec  wrote:
>
>
>> But, one of the biggest things keeping me from doing more work on it
>> when I do have some time, is the fact that barely any of the devs seem
>> to care (other than the OP, who just seems to bitch about everything
>> not working for him).  Since the GLEP 63 spec has been approved.
>> Barely any of the gentoo developers have even tried to update their gpg
>> key or generate a new one that does meet the spec.  For that reason, I
>> have not endeavored to get more done in it.  I've been trying to
>> keep the gentoo-devs seed file reasonably up to date, but since there
>> are few devs actually fixing or generating new keys, it is not needed
>> that often.  In fact weeks go by before there is a change in LDAP in
>> regards to gpg keys.
>>
>> As Andrew pointed out in another reply, there is a fairly decent
>> document about generating new gpg keys either directly using gpg or
>> using gkeys-gen (gkeys-gen-) has the most troublesome bugs fixed in
>> it btw).
> It's a little difficult for people to generate new keys with gkeys-gen when
> the version of gkeys-gen in the tree is completely and utterly broken, and has
> been for almost a year now.
Wiki says:

"In this guide we are going to show you how to create a GLEP 63
 based OpenPGP Key using
app-crypt/gkeys-gen
 tool which is
the official way of managing OpenPGP keys in the Gentoo Infrastructure."

So either the documentation is wrong, or we're supposed to use a broken
tool.

Interesting challenge!
>  The last time I tried to make a new key it spit
> out a bunch of errors and tried to put data in $HOME/~/gkeys-user/gpghome.
> Like it didn't expand the tilde, but made a directory literally named '~'.  
> I'm
> supposed to use this for security sensitive data?  You want me to use a
> potentially unstable live ebuild instead?  Well, no, that's not gonna happen.
It gets even better when you try to read the code. But, not to worry -
it's actually pretty easy. Took me only about 4h to combine the
fragments together ...


So, first part of the puzzle:
https://wiki.gentoo.org/wiki/GLEP:63

Build a gpg.conf with the suggestions there.

Now read http://www.gnupg.org/gph/en/manual.html ... well, the
interesting part is:

"""
$ gpg --full-gen-key

Your selection? 4
What keysize do you want? (2048) 4096
Key is valid for? (0) 36m
"""
Those are the required base parameters, all other questions are
identifier (name/email). It'll take a minute or five to collect enough
entropy.

Now you want to generate a subkey (where ${keyid} is the keyid of the
main key):

"""
$ gpg --edit-key ${keyid}
gpg> addkey
Your selection? 4
What keysize do you want? (2048) 4096
Key is valid for? (0) 12m
"""
and maybe a revocation certificate:
"""
$ gpg --output revoke.asc --gen-revoke ${keyid}
"""

What I did then was to export the subkey, and keep the main key
somewhere safe. Then import the subkey on the victim machine(s) used for
gentoo committery.

Now you need to read the gpg docs again and figure out that you need
"gpg --send-keys" to upload the key to the public keyservers.

Then you wait a few minutes for it to become visible, you can check that
on http://pool.sks-keyservers.net.

Now your wiki skills are needed, if you don't know the magic invocation
you won't find it.
Hint: https://wiki.gentoo.org/wiki/Project:Infrastructure/LDAP_Guide
||

The magic line|||is: "perl_ldap -b user -C gentooGPGfingerprint
"" $USER".

So now log in to dev.gentoo.org and add your key's fingerprint there.
Wait 15 minutes.

Use that time to read https://wiki.gentoo.org/wiki/Gentoo_git_workflow

especially the repository settings chapter.


... and now you can clone the repo, and do (signed) commits. Easy!



So, our onboarding experience sucks, this information is spread out in a
way that makes it hard to find even if you know what you want.

It took me literally hours, which means every new dev trying to do this
will spend hours. It's a colossal waste of time, drains motivation, and
especially the conflicting/wrong docs are not really a good idea.

The complaints are mostly that no one seems to have thought about how a
new user will find things, so there's no combined doc. The wiki is hard
to search, making it extra challenging to figure out what to do.



How to improve? Take my email, cut out the parts that state the obvious,
turn it into a wiki page referencing the other wiki pages (if wiki is
your thing - I refuse to touch MediaWiki outside of paid work, because I
got paid too long to work with it and understand the deeply ingrained
confusion its authors had about the universe)

Or just point people at a random email, because that's about as good as
documentation.
I've wasted enough time documenting the missing pieces, instead of
running gkeys-gen and doing this whole process in 

Re: [gentoo-dev] Re: repo/gentoo.git, or how committing is challenging

[Patrick Lauer]

On 12/22/2015 02:14 PM, Rich Freeman wrote:
> On Tue, Dec 22, 2015 at 7:53 AM, Patrick Lauer <patr...@gentoo.org> wrote:
>> I'd replace gkeys-gen with a ~10-line shell script ... if I had some
>> motivation to dig through some old experiments of mine where I managed
>> to set all parameters for pgp from CLI. Which is all that gkeys-gen
>> would do!
> Sounds great.
>
>> I guess we fundamentally disagree - if you do shoddy work, it is shoddy.
>> I won't praise you for it.
> Nobody is looking for your praise.
>
>> That's my time, spent to work around deficiencies I shouldn't even see -
>> if other people had done their job. And that's just frustrating if it
>> happens again and again, and instead of doing something interesting I
>> spend most of my time just being janitor and cleaning up stuff.
> I hope you're not the one looking for praise.  I can't imagine that
> your pep talk has done a great deal to motivate people to spend time
> improving the Gentoo Keys experience.
Well, it's abandoned anyway (bugs open for >1 year means there's
literally no one working on it, for a year)
Like the git 'migration' it's half-finished work with little thought
about workflow or user experience, "Change is Progress"

If we didn't have it at all I would not have had to file bugs, spend
time being very confused, etc. So all in all it has negative value in
its current state. And it wastes the time of everyone who tries to use
it, which is a few man-weeks of work ground away with inefficiency and
carelessness. Imagine how much progress we could have had if someone had
spent one afternoon on writing coherent docs!

>
> Do you want to see this fixed?
> Are you willing to do the fixing yourself?
I don't have infinite time, and wasting a day documenting things that
should have been documented a year ago is not a good way of spending time.

>
> If the answer to the first question is yes, and the second is no, then
> you've just volunteered for the job of either community motivator, or
> frustrated user.  The goal then is to make people care more about
> going out of their way to fix things than going out of their way to
> find ways to make you even more frustrated.  Which do you think is
> going to be more emotionally satisfying to those who read this thread?
>
Things working.

So, the trick is not to have user-visible breakage.

Now you know the great trick too and can apply it to your daily life.

Re: [gentoo-dev] Re: repo/gentoo.git, or how committing is challenging

[Patrick Lauer]

On 12/22/2015 01:08 PM, Rich Freeman wrote:
>> Or just point people at a random email, because that's about as good as
>> documentation.
> Thank you for writing up a guide/outline.
>
> You appear to hate mediawiki, but you do realize that you could
> probably copy/paste that email into the box and call it half-done,
> right?  Somebody else can always come along and improve it, and that
> is kind of the whole point of a wiki, and of FOSS in general.
I've worked with Semantic Mediawiki long enough to understand that it is
a pile of buggy hacks, on top of a horribly bad codebase, on top of a
horribly broken language. Upstream developers don't understand concepts
like data truncation, and debugging this pile of code is going to make
you cry.

(Just as an example: I found a 'pathological' pageview that cost ~4
SQL connections (yes!) and 90 CPU-seconds render time, server side, on a
4Ghz machine. Moving the database from dedicated hardware to the MW
server sped up page render time because the network latency of ethernet
becomes painful ...)

>From the beginning I've suggested to use something sane, but people Know
what needs to be done, so there's no way to avoid such badness to
spread.  And thus I just refuse to interact with it now, because I know
enough details about SMW templates to not want to stare at that buggy
ad-hoc mess of random again.

>
>> Please, stop wasting people's time, if you write code or documentation
>> write it once properly, don't release untested things and claim they are
>> an official tool, and don't ignore complaints (because they mean, as a
>> first approximation, that you screwed up and need to fix stuff)
>>
> Gentoo devs and volunteers are more than welcome to ignore complaints.
> I'll take half-implemented code over no code any day of the week.
Broken code is worse than no code: Now you spend lots of time on
debugging, instead of doing something more useful.

I'd replace gkeys-gen with a ~10-line shell script ... if I had some
motivation to dig through some old experiments of mine where I managed
to set all parameters for pgp from CLI. Which is all that gkeys-gen
would do!
> Maybe somebody isn't good at writing documentation, and we benefit
> from getting their contributions all the same which somebody can later
> follow-up on (perhaps somebody who is better at writing documentation
> than code).  You're going to make more progress with evolutionary
> steps.
>
> BTW, bugs aren't complaints, and I don't really consider "complaints"
> nearly as useful.  If you want to point out an error by all means do
> so.  You can do it without implying that somebody somehow owed you
> something better. They don't.
>
I guess we fundamentally disagree - if you do shoddy work, it is shoddy.
I won't praise you for it.

Look, *I* spent about a working day all in all on just figuring out why
things don't work. Multiply by number of contributors, and it starts
looking really sad. Time and motivation are not free resources!

That's my time, spent to work around deficiencies I shouldn't even see -
if other people had done their job. And that's just frustrating if it
happens again and again, and instead of doing something interesting I
spend most of my time just being janitor and cleaning up stuff.

Re: [gentoo-dev] repo/gentoo.git, or how committing is challenging

[Patrick Lauer]

On 12/14/2015 04:00 AM, Brian Dolbec wrote:
> On Sun, 13 Dec 2015 22:20:01 +0300
> Andrew Savchenko <birc...@gentoo.org> wrote:
>
>> Hi,
>>
>> On Sun, 13 Dec 2015 18:36:51 +0100 Patrick Lauer wrote:
>>> Oh hey. We're in the future. Let's try to commit something to
>>> repo/gentoo.git!
>>>
>>> So apparently we're signing things with gpg now, so let's read the
>>> official documentation.
>>> The [1] wiki seems to be the canonical location for such things.
> ...
>
>>> Since signing is mandatory since the git migration, ahem, this means
>>> that no one in the last 5 months(!) actually followed the
>>> documentation (because that does NOT work!). I'm almost impressed,
>>> but, wow, this is enterprisey.  
>> It is absolutely possible to create correct gpg key, put it into
>> LDAP according to GLEP and to sign commits and pushes properly.
>> What is not currently possible is to verify all tree automatically.
>>
>> I agree that gkeys needs more work. But we are all volunteers here.
>> You may help them if you are that interested into this
>> functionality.
>>
>> What worries me more that we still have no way for rsync users to
>> verify the portage tree (or Gentoo tree in the newspeak someone
>> prefers here). And most users use rsync.
>>
>>> So, what can we do to make this whole story of 'commit (and push) to
>>> repo/gentoo.git' make sense? And why do I appear to be the only one
>>> to notice this chain of breakage?!  
>> We need to complete gkeys project, right? That's not all of the
>> story, but a start. So send patches :) As for the full story, we
>> still need to somehow verify rsync tree. For now only snapshots are
>> verified.
>>  
>> Best regards,
>> Andrew Savchenko
> Thank you Andrew.
>
> Let me first say, that while I am leading the gentoo-keys project.  I
> have not been doing much with making progress to get another release
> out.  My time is mostly taken up by full time work, add some health
> issues (not serious), plus with coding full time now (it was just a
> hobby before).  I am finding it difficult to switch codebases to keep
> development going in a non work project.  There is a certain amount of
> time needed to get back into the code to make any significant progress.
>
> But, one of the biggest things keeping me from doing more work on it
> when I do have some time, is the fact that barely any of the devs seem
> to care (other than the OP, who just seems to bitch about everything
> not working for him).  Since the GLEP 63 spec has been approved.
> Barely any of the gentoo developers have even tried to update their gpg
> key or generate a new one that does meet the spec.  For that reason, I
> have not endeavored to get more done in it.  I've been trying to
> keep the gentoo-devs seed file reasonably up to date, but since there
> are few devs actually fixing or generating new keys, it is not needed
> that often.  In fact weeks go by before there is a change in LDAP in
> regards to gpg keys.
>
> As Andrew pointed out in another reply, there is a fairly decent
> document about generating new gpg keys either directly using gpg or
> using gkeys-gen (gkeys-gen-) has the most troublesome bugs fixed in
> it btw).
>
> But lets get back to the OP's gpg keys.
>
> dolsen@vulture /var/lib/gkeys $ gkeys spec-check -C gentoo-devs -n patrick
>
>  Checking keys...
>
>
>   patrick, Patrick Lauer: 0x10F17899A28441CC, 0xA8447784E52864CE
>   ==
>
> --
> Fingerprint..: 0A0901B8F018D5D6A4A31A4410F17899A28441CC
> Key type : PUBCapabilities.: sc  
> Algorithm: Pass   Bit Length...: Fail
> Create Date..: Pass   Expire Date..: Pass
> Key Version..: Pass   Validity.: e, Expired
> Days till expiry.: 0  
> Capability...: Pass   
> Qualified ID.: Fail   <== '@gentoo.org' user id not found
> This primary key.: Fail
>
> --
> Fingerprint..: 043F1AB5B35382E666BDBEA4058F9332F0BAE0B1
> Key type : SUBCapabilities.: e  
> Algorithm:    Bit Length...: 
> Create Date..: Pass   Expire Date..: Pass
> Key Version..: Pass   Validity.: e, Expired
> Days till expiry.: 0  
> Capability...: Pass   
> Qualified ID.: Fail   <== '@gentoo.org' user id not found
> This subkey..: Fail
>
> Key summary
> primary..: Fail signing subkey: Fail
> encryption subkey: Noauthentication subk

Re: [gentoo-dev] Breakage and frustration

[Patrick Lauer]

On 12/14/2015 02:58 PM, Rich Freeman wrote:
> On Mon, Dec 14, 2015 at 5:48 AM, Peter Stuge  wrote:
>> The key point to remember is that it is NOT neccessary to be part of
>> the team in order to contribute solutions. You *first* contribute
>> solutions and only *then* have a chance of becoming part of the team.
>>
>> I for one am working in my non-existant spare time on a fast
>> ChangeLog generator.
>>
> ++, and thanks.
>
> I was actually trying to think beyond this point though.  Right now we
> have lots of infra stuff that the infra team would probably love to
> publish except there is no easy way to do it without creating security
> issues (intermixing of credentials and code, etc).  We have a few devs
> who have access to all of it, but their time needs to be split between
> keeping the current state working, and trying to build some future
> state which is easier to contribute to.  For the rest of our part, we
> do need to put up and start building things.
>
> So, how do we get from that to a future state where our infra servers
> have public backups or whatever minus /etc/credentials.include and
> /var/spool/private-email and so on?  That is, a future state where the
> default is open with necessary exceptions?  And how do we do it with
> what we have, or are able to get?
>
> I just want to be constructive.  Clearly the first step is to
> acknowledge that the onus is on those who really want to see change to
> chip in to make it happen.  We have to be here to serve - this is a
> volunteer FOSS project.
>
Y'know ...

If I had access to things I could help. But I don't, so I can't.

The rather obvious solutions seem to be politically impossible, and the
politically possible solutions are not satisfactory.

Re: [gentoo-dev] repo/gentoo.git, or how committing is challenging

[Patrick Lauer]

On 12/13/2015 06:36 PM, Patrick Lauer wrote:
>  So apparently we're signing things with gpg now

And a related question:

How would I actually verify the signatures in a meaningful way?

... and why is that not default then.

[gentoo-dev] Breakage and frustration

[Patrick Lauer]

Broken breakage


tl;dr: Stuff is broken, and no one seems to care



In August the git "migration" happened, moving our main repository from
old stupid cvs to modern shiny git.

Well, migration is not the word I'd use, because this was an untested
forced migration that is now, months later, still suffering from
regressions and failures. But, hey, no more cvs! That's good because,
reasons.

So at first there was [1] the lack of proper Manifests. Which broke
things for all rsync users for  a few hours.

Once that was 'fixed' there was the fun of [2], which made emerge --sync
very expensive because it refetched lots of files. Every time.

The 'fix' to the fix of the fix for that is still in progress ...


And a few little things like [3] happened. Oopsie.
Also there seems to be confusion how git works, leading to hilarity like
[4].

Now [5] was reported. Who needs ChangeLogs, this is git! Except for all
users, who don't get ChangeLogs. Well, let's add them and [6] not test
what happens. Guess what? Stuff breaks more.
And they are added backwards so that emerge --changelog fails in a
different way. No, I didn't want to read all changlog except the part I
cared about.

And fixing that introduces [7] some more regressions that broke updating
@system for about 3.5 days.

The fix to that fix (notice a pattern here?) broke rsync for *all* users
[8]. Almost as if no one ever tests things in a test environment ... but
hey, we're agile, let's fix stuff in production!

And the manifest issues are still [9] making life exciting.

So to summarize, in about 5 months there was user-visible breakage:

- ~1 day downtime for git migration (no updates)
- 8h no Manifests (no updates possible)
- a few days of emerge --sync being stupidly slow
- a few days of emerge-webrsync not updating
- about 3 months of emerge --changelog being broken, just to be broken
in a different way
- 3.5 days of emerge @system being broken
- about a day of emerge --sync needing manual interaction to be able to
update again
- a few days of grub being uninstallable (iow, making installing
impossible for many users)

So all in all emerge --sync && emerge -uND @system being down for >10%
of the time.

Now, I don't know if you use Gentoo, but I do, and I use it at work, so
having this level of randomization happen is not really useful.

Tell me then, please - what can I/we do so that this kind of breakage
stops, and we can actually aim at having a most excellent distro? In the
long run I am considering just creating my own clone of all
infrastructure bits so I can fix things, but it's an option that is
needlessly braindead, wasting effort, and not really useful to users
that are not me.


[1] https://bugs.gentoo.org/show_bug.cgi?id=557184
[2] https://bugs.gentoo.org/show_bug.cgi?id=557192
[3] https://bugs.gentoo.org/show_bug.cgi?id=557344
[4] https://bugs.gentoo.org/show_bug.cgi?id=557400
[5] https://bugs.gentoo.org/show_bug.cgi?id=557826
[6] https://bugs.gentoo.org/show_bug.cgi?id=565574
[7] https://bugs.gentoo.org/show_bug.cgi?id=565694
[8] https://bugs.gentoo.org/show_bug.cgi?id=567074
[9] https://bugs.gentoo.org/show_bug.cgi?id=567830

[gentoo-dev] repo/gentoo.git, or how committing is challenging

[Patrick Lauer]

Oh hey. We're in the future. Let's try to commit something to
repo/gentoo.git!

So apparently we're signing things with gpg now, so let's read the
official documentation.
The [1] wiki seems to be the canonical location for such things.

Oh dear. The layout is VERY broken. See [2]. Which redirects to [3],
which is a duplicate of [4], which has been closed because apparently
the persons responsible don't understand how to internet.
Since this bug is only about a year old I don't expect any progress soon
- but fetching random crap from untrusted hosts is not a sane option.
Especially since there is already a webserver, which is also trusted, so
I'm confused why we're still having this conversation.

But hey, let's blindly fetch CSS from unknown, just to notice that this
'theme' needs JavaScript to display properly. Because reasons.

Why would I want to blindly execute code when reading the text of a
wiki? Because, reasons. Because, future!
Sigh. I'll just live with the breakage then.

But anyway, we find [5] the right document, and ... hit [6]. Can't
install, bug is over half a year old, so I have to consider upstream
dead. But we can easily patch the ebuild and somehow install
app-crypt/gkeys.

Well, we can install it, but won't be able to use it because [7][8] it's
TOFU. Totally Fine and Usable!
Nothing some random stabbing won't fix, eh, but now we're an hour in
just trying to get dependencies of dependencies installed.

Sigh.

Now that gkeys is out of the way, let's try to use gkeys-gen!
[9][10][11] Nope. Nope nope, you don't get to play!

So there's no way to actually *use* this software in the default config
(how was this ever released?!), and upstream has not fixed any of these
issues in almost a year. This parrot is an ex-parrot!


Let's capitulate, err, repudiate. Wait, wrong word. Recapitulate! That's
it. Let's recapitulate:

The official docs are running on an unmaintained broken platform. If you
manage to read them they are wrong. And the software to use has been
abandoned a year ago, but is still suggested as default in the docs.

Since signing is mandatory since the git migration, ahem, this means
that no one in the last 5 months(!) actually followed the documentation
(because that does NOT work!). I'm almost impressed, but, wow, this is
enterprisey.

So, what can we do to make this whole story of 'commit (and push) to
repo/gentoo.git' make sense? And why do I appear to be the only one to
notice this chain of breakage?!


[1] http://wiki.gentoo.org
[2] https://bugs.gentoo.org/show_bug.cgi?id=559530
[3] https://bugs.gentoo.org/show_bug.cgi?id=547536
[4] https://bugs.gentoo.org/show_bug.cgi?id=536744
[5]
https://wiki.gentoo.org/wiki/Project:Gentoo-keys/Generating_GLEP_63_based_OpenPGP_keys
[6] https://bugs.gentoo.org/show_bug.cgi?id=550848
[7] https://bugs.gentoo.org/show_bug.cgi?id=536338
[8] https://bugs.gentoo.org/show_bug.cgi?id=557090
[9] https://bugs.gentoo.org/show_bug.cgi?id=567768
[10] https://bugs.gentoo.org/show_bug.cgi?id=566782
[11] https://bugs.gentoo.org/show_bug.cgi?id=536316

Re: [gentoo-dev] repo/gentoo.git, or how committing is challenging

[Patrick Lauer]

On 12/13/2015 07:50 PM, Andrew Savchenko wrote:
> Hi,
>
> On Sun, 13 Dec 2015 18:38:55 +0100 Patrick Lauer wrote:
>> On 12/13/2015 06:36 PM, Patrick Lauer wrote:
>>>  So apparently we're signing things with gpg now
>> And a related question:
>>
>> How would I actually verify the signatures in a meaningful way?
> git log --show-signature does this using GnuPG.
That's not very automated or effective.
I'd assume 'emerge' has such functionality included ...?
>
> Of course, in order to gpg to work one have to mark dev keys as
> trusted, they can be verified using ldap or several public
> keyservers. LDAP is more reliable, of course, but this method works
> only for devs (and probably some stuff members) having an access
> here.
That's what the app-crypt/gkeys thing is for, as far as I can tell.

Re: [gentoo-dev] rfc: OpenRC public API definition

[Patrick Lauer]

On 12/03/2015 06:36 PM, William Hubbs wrote:
> All,
>
> I would like opinions on what is considered the public api of librc.
>
> 1) All definitions in rc.h, even though they are not formally documented
> in man pages.
Yes.
The header *is* the public API.
>
> 2) the definitions in rc.h which are documented in section 3 man pages.
>
> I'm bringing this up, because I am looking at redesigning one of the
> undocumented functions soon, and the redesign will definitely break
> things if people are using the function outside of OpenRC.
>
> Thoughts?
>
> William
>

Re: [gentoo-dev] EAPI 6 portage is out!

[Patrick Lauer]

On 11/18/2015 01:01 PM, Rich Freeman wrote:
> On Wed, Nov 18, 2015 at 6:12 AM, Alexander Berntsen  
> wrote:
>> When I do QA in projects I'm involved with (at least outside of
>> Gentoo), we don't do it live on end-user systems. I'll leave the
>> details as an exercise for the Gentoo developer.
>>
> People who run ~arch are not really end-users - they're contributors
> who have volunteered to test packages.
Or people that use Gentoo because it allows them to satisfy requirements.

At work I don't 'want' to use ~arch packages, but external constraints
very strongly suggest that. Otherwise we'd just be on CentOS 5 and not
worry about things working properly.

And still we try to run updates in a sandbox first so we catch breakage
before it becomes a problem.

Re: [gentoo-dev] [RFC] ban use of base-4 casemods in ebuilds due to locale collation instability

[Patrick Lauer]

On 11/11/2015 03:51 AM, Mike Frysinger wrote:
> On 10 Nov 2015 18:53, Mike Frysinger wrote:
>> i randomly stumbled across an ebuild that was using ^^ to make a variable
>> uppercase.  this is new to bash-4.0 and thus invalid for EAPI=[0-5].  only
>> the fresh EAPI=6 permits it since we bumped the min ver to bash-4.2.
> Arfrever highlights these are not even safe to use.  bash is locale aware,
> so it'll apply LC_COLLATE rules when processing the ^/, casemods.  while
> you can fix this with external programs ala:
>   LC_COLLATE=C tr ...
>
> you can't do it with inline code like:
>   LC_COLLATE=C SRC_URI=".../${PN^^}/..."
>
> you can if you do something like:
>   SRC_URI=".../$(LC_COLLATE=C; echo "${PN^^}")/..."
>
This points out a class of problems we've hit in the past: locale-aware
things in ebuilds.

Wouldn't it be 'easier' (fsov easy) to have portage use sane-default
locale settings, so that estonian or turkish users don't get hit by
weirdness in the [a-z] character class etc.?

(And as a side-effect the build logs are always readable ;) )

Re: [gentoo-dev] Re: ChangeLog

[Patrick Lauer]

On 11/03/2015 02:52 AM, Matt Turner wrote:
>
> So you are telling me that people using github and the switch that took
> place has absolutely nothing to do with the changelogs going dead?
> You keep saying GitHub. Github is not relevant to this discussion.
>
> Yes, the ChangeLogs stopped being updated because of git.
>
> The git transition had been 9 years in the making and has massively
> improved Gentoo development. Look at the graph of contributions per
> month: https://www.openhub.net/p/gentoo
Yes, the rate of git commits has gone up since the git switcheroo.
But that's not what you wanted to say I think ;)

> It was decided that the missing infrastructure for ChangeLogs was not
> a sufficient reason to block a hugely important change.
>
> People care about finishing this, as evidence by this thread. Please
> stop complaining.
>
Apparently my complaining finally re-triggered some action, so sadly
this looks like the currently best strategy.

Sigh,

Re: [gentoo-dev] ChangeLog

[Patrick Lauer]

On 11/01/2015 01:53 PM, Rich Freeman wrote:
> On Sun, Nov 1, 2015 at 7:33 AM, Мисбах-Соловьёв Вадим  wrote:
>> And why don't just only generate them on rsync mirrors, but remove them from 
>> git repo (like was planned initially, AFAIRC)?
>>
> That is in fact how it works.  Or, at least how it is supposed to
> work.  I don't use the rsync mirror, so I can't vouch for whether
> they're producing ChangeLogs.
Supposed to, but it doesn't
>
> Personally I'd just as soon see them go away entirely, but if somebody
> wants to make them work I won't stop them.
>
I'd really not prefer to fly blind, ChangeLogs are awesome for users.

I am a user too. I really would like to know why something changed, and
maybe who did it.

[gentoo-dev] ChangeLog

[Patrick Lauer]

Ahoi,

I'm getting mildly very irritated with the lack of easily accessible
ChangeLogs for our packages.

Apparently updating them stopped some time in August, so now there are
some outdated ChangeLogs that don't really serve any purpose, and the
easiest way for users to figure out why something changed is to yell at
the clumsy gitweb.g.o interface. So instead of grep we now need lots of
patience.

This does not look reasonable to me.

Can we please either properly remove ChangeLogs and tell people to not
be curious about changes, or make them useful again?

Thanks,

A Gentoo User.

Re: [gentoo-dev] ChangeLog

[Patrick Lauer]

On 11/01/2015 02:24 PM, hasufell wrote:
> On 11/01/2015 01:16 PM, Patrick Lauer wrote:
>> Ahoi,
>>
>> I'm getting mildly very irritated with the lack of easily accessible
>> ChangeLogs for our packages.
>>
>> Apparently updating them stopped some time in August, so now there are
>> some outdated ChangeLogs that don't really serve any purpose, and the
>> easiest way for users to figure out why something changed is to yell at
>> the clumsy gitweb.g.o interface. So instead of grep we now need lots of
>> patience.
>>
>> This does not look reasonable to me.
>>
>> Can we please either properly remove ChangeLogs and tell people to not
>> be curious about changes, or make them useful again?
>>
>
> ChangeLogs are a deprecated and unreliable method of the times we were
> still on CVS. E.g. some people didn't find it useful to add ChangeLog
> entries when they did large eclass changes. This problem is gone now.
... ?!??!#??$>@%%*%**%@!!!

>From the point of view of a user ChangeLogs are VERY VERY USEFUL.

Why would you claim they are deprecated?
>
> git log -- app-misc/foo
> or
> git log -- eclass/autotools.eclass
>
> will give you _any_ commit that has touched that file/directory, even if
> it was part of a huge mass commit.

 $ cd /usr/portage/app-admin/rex/; git log
fatal: Not a git repository (or any of the parent directories): .git

sooo ... ???

>
> There's really not much ChangeLogs add for you here, except duplicating
> git functionality. It's more useful to familiarize yourself with git
> log. There's no reason to depend on the gitweb interface.
>
> If you want the history from before the migration to work with that as
> well, you can use this method:
> https://wiki.gentoo.org/wiki/Gentoo_git_workflow#Grafting_Gentoo_History_Onto_the_Active_Repo
>
> Also see
> https://www.atlassian.com/git/tutorials/git-log/filtering-the-commit-history
>
I have no idea what you are trying to say, but for most users this
advice is not even useless but actively wrong.

With that out of the way,

can we please return to the original discussion?

Re: [gentoo-dev] Re: ChangeLog

[Patrick Lauer]

On 11/02/2015 02:56 AM, Rich Freeman wrote:
> On Sun, Nov 1, 2015 at 8:22 PM, Duncan <1i5t5.dun...@cox.net> wrote:
>>  I know if I were still on rsync (or webrsync), I'd be raising hell about 
>> the lack of
>> changelogs well before now
> Perhaps rather than raising hell you'd do better to raise money to
> hire an infra team to fix the bug or something.
Hire?
I'm still willing to fix things, if I were given access. And I would
presume that I'm not the only one.

But since I don't have access, and can only affect things by motivating
or upsetting people, we have a large mail thread that is mostly about
smug people being smug. Which confuses me since it's not Tuesday, but I
digress ...
>
> I get the frustration, but we only have a few people who have the
> necessary access to fix the problem.
So fix that.
>   Infra is also a difficult
> project to deal with in general because it is fairly closed due to the
> implications of having random people messing with it.  I don't really
> see anybody stepping up to try to change anything fundamental about it
> either.  This isn't the sort of thing that will get better if the
> council votes on something.
>
Yes, voting is not going to fix anything directly. So I didn't even
suggest it.

But one of the conditions for tolerating the git migration was that we
have no regressions.
Now it's about 3 months later and *basic* functionality is still
"Oopsiedaisy, I must have missed that"

Which is confusing me because ... uhm... didn't anyone ... test things?
Document? How can something be deployed that is obviously missing
features like this?

(And as a consequence, why doesn't it then get fixed in a reasonable time?)


But at least now we get some good information what is broken how, and
maybe someone can fix it. And then I won't have to be the stone in
people's shoe anymore ;)

Re: [gentoo-dev] Re: ChangeLog

[Patrick Lauer]

On 11/02/2015 03:04 AM, Michael Orlitzky wrote:
> On 11/01/2015 08:22 PM, Duncan wrote:
>> Personally, I'd love the primary sync method to be git, and the primary 
>> change logs conveyed via native git log methods, but in ordered to do 
>> that, all those rsync mirrors need to switch to git mirrors.
>>
> I wonder what would happen if we just... dropped a full clone of
> gentoo.git onto the rsync mirrors?
>
>
You find out how many users have limited bandwidth (again)

Why on earth would you do the most wasteful hybrid strategy when people
are in general trying to optimize for efficiency?

And why am I still sober?

Re: [gentoo-dev] Re: [gentoo-commits] repo/gentoo:master commit in: dev-libs/libgit2/

[Patrick Lauer]

On 10/10/15 14:25, hasufell wrote:
> On 10/10/2015 02:24 PM, Fabian Groffen wrote:
>> On 10-10-2015 14:19:44 +0200, hasufell wrote:
 +RDEPEND="
 +  !libressl? ( dev-libs/openssl:0 )
 +  libressl? ( dev-libs/libressl )
 +  sys-libs/zlib
 +  net-libs/http-parser
>>> Please order deps alphabetically (I know I added libressl without
>>> reordering, but that was just to keep the diff as small as possible).
>> Is this a policy 
> It is undocumented policy.
>
>
No it's not.

Re: [gentoo-dev] Re: [gentoo-commits] repo/gentoo:master commit in: dev-ruby/metasm/

[Patrick Lauer]

On 10/09/15 17:42, Davide Pesavento wrote:
> On Fri, Oct 9, 2015 at 5:35 PM, hasufell  wrote:
>> On 10/08/2015 11:04 PM, Richard Farina wrote:
>>
>> +all_ruby_prepare() {
>> + [ -f Gemfile.lock ] && rm Gemfile.lock
>> missing "|| die" afais, should probably be
>>
>> [ -f Gemfile.lock ] && { rm Gemfile.lock || die ; }
>>
> Or simply:
>
> rm -f Gemfile.lock || die
>
With -f it always succeeds, so the ||die is redundant  ...

Re: [gentoo-dev] Re: useflag policies

[Patrick Lauer]

On 08/12/15 22:38, William Hubbs wrote:

 I always wondered why pkg_pretend never caught on.

Because, in a way, it triggers at the wrong point of the merge.

emerge -pv fnurk = dependencies look ok

emerge fnurk = pkg_pretend bails out ... eh?!

(This would be a little bit confusing, if not actively hostile, and
useflags + required_use are a lot more 'natural' to the emerge workflow)

 I to can see the advantage of it over REQUIRED_USE; it would allow the
 package maintainer to give specific error messages about why use flag
 combinations are invalid for a package.

And now someone will say annotations. Sigh.


have fun,

Patrick

Re: [gentoo-dev] Git workflow, commit messages

[Patrick Lauer]

On Sunday 09 August 2015 08:06:53 Anthony G. Basile wrote:
 Hi everyone,
 
 I hate to be a nag, but please read
 https://wiki.gentoo.org/wiki/Gentoo_git_workflow.  In particular, commit
 messages.  I'm seeing lots of cvs style messages going in and without
 history it will be hard to figure out what happened just from the
 message.  Particularly, we should prepend CATEGORY/PN:  to the first
 line so we can easily search git log for what happened to a package.

Lazy me says:

Can't we make repoman do that?

Re: [gentoo-dev] useflag policies

[Patrick Lauer]

On Sunday 02 August 2015 22:22:28 Ciaran McCreesh wrote:
 On Sun, 02 Aug 2015 17:14:47 -0400
 
 NP-Hardass np-hard...@gentoo.org wrote:
  ^^ has the pleasant side effect of being easier to read, as a user.
  The user receives a message saying at-most-one-of instead of some
  convoluted other expression that they don't understand.
  
  I am all for the use of ^^ add the default for this reason.
  
  Additionally, ?? has the same effect of being easy to understand as
  the description in the error message is in plain English.
 
 If you cared about readability, you'd use pkg_pretend.

No.

Re: [gentoo-dev] useflag policies

[Patrick Lauer]

On Monday 03 August 2015 00:34:51 Ben de Groot wrote:
 Recently some team members of the Qt project have adopted these ebuild
 policies: https://wiki.gentoo.org/wiki/Project:Qt/Policies
 
 I have an issue with the policy adopted under Requires one of two Qt
 versions. In my opinion, in the case where a package offers a choice
 between qt4 or qt5, we should express this in explicit useflags and a
 REQUIRED_USE=^^ ( qt4 qt5 ). This offers the user the clearest choice.

Since qt4 and qt5 are both relatively 'heavy' dependencies and quite different 
in many ways (including differences in default styles) many users will want to 
stick with only one of those.

The gtk 'solution' forced some ugly things like masking gtk+:3, gconf:3, ... 
and then selecting packages based on specific -r200 / -r300 revisions. So much 
work to avoid regressing into gtk3!

(Which is especially frustrating because *dbus* has wrong dependencies just so 
that gtk/gnome apps using dconf can save config ... )
 
 Other developers state that users are not interested in such implementation
 details, or that forced choice through REQUIRED_USE is too much of a
 hassle. This results in current ebuilds such as quassel to not make it
 clear that qt4 is an option.

I find setting USE=qt4 -qt5 a lot more obvious than having USE=qt (why not 
USE=X ?) which then does different things based on another useflag, 
sometimes. Maybe. It's horribly inconsistent and even might change result over 
time, which is not very user-friendly.
 
 This goes against the principle of least surprise, as well as against QA
 recommendations. I would like to hear specifically from QA about how we
 should proceed, but comments from the wider developer community are also
 welcome.

I would prefer having qt4 and qt5 useflags independent, and no generic qt 
useflag.

Re: [gentoo-dev] [RFC] Rebooting the Installer Project

[Patrick Lauer]

On Sunday 19 July 2015 02:05:21 Patrice Clement wrote:
 Saturday 18 Jul 2015 15:36:01, NP-Hardass wrote :
 I fancy your idea a lot. We ought to do it for complete newbies who are new
 to Gentoo but would like to give the distribution a shot nonetheless. I
 fondly remember my first attempt who took me ages to complete.. Good times.
 :)
 Following up on NP-Hardass' email in which he's raising good points:
  Who is your target audience?  New users?  Experienced users?  Not so
  much a comment about your installer, but installers in general, I feel
  the handbook is critically important to helping new users to understand
  Gentoo. Whether the handbook would remain a resource that users would
  look at if they had an installer remains to be seen.
 
 IMHO, users should be guided and walked through the installation process.
 They should understand what's going on so the Handbook must be equally
 important as when a normal chap installs Gentoo without using the
 installer.

Then the Handbook should be actually correct and useful first:

https://bugs.gentoo.org/show_bug.cgi?id=550646

Re: Code Review Systems Was: [gentoo-dev] Git Migration: launch plan schedule

[Patrick Lauer]

On 07/07/15 01:27, Ciaran McCreesh wrote:
 On Mon, 06 Jul 2015 13:04:35 -0400
 Michael Orlitzky m...@gentoo.org wrote:
 I would love my commits to be reviewed, but I usually don't feel like
 reviewing anyone else's. That's... not uncommon.
 
 Well, you could at least get your commits reviewed by an automated build
 system that starts from a clean stage each time. That's better than
 nothing.
 
I'll laugh about it next time I update OpenFOAM:

Fri Jun 27 12:52:19 2014  sci-libs/openfoam-2.3.0
   merge time: 1 hour, 5 minutes and 8 seconds.
or

Sun Jun 29 20:36:09 2014  sci-mathematics/nusmv-2.5.4
   merge time: 2 hours, 58 minutes.


That's without dependencies, so from a clean minimal starting point
(once you figure out what useflags are needed) you're looking at 12+
hours of walltime to get that checked. (On a reasonably modern Xeon with
SSDs ...)

So thanks for your intentional comedy, but let's be serious here.

Have fun,

Patrick

Re: Code Review Systems Was: [gentoo-dev] Git Migration: launch plan schedule

[Patrick Lauer]

On Sunday 05 July 2015 13:46:10 William Hubbs wrote:
 On Sun, Jul 05, 2015 at 09:05:59AM +0300, Andrew Savchenko wrote:
  On Sat, 4 Jul 2015 20:20:23 +0200 Peter Stuge wrote:
   It's important that the review flow is well-understood and efficient.
  
  This is impossible in our case due to the lack of manpower.
  We already have a lot of bugs, patches, stabilization requests
  hanging over there for months and even years. Stabilization request
  will require at least two developers to participate in each commit.
  This will double manpower required at least. Such approach can kill
  the whole project.
 
 Agreed. Forcing all commits from developers to go through a code review
 from another developer before they hit the tree would potentially
 kill the entire project. I would strongly veto something like this,
 because we flat out don't have the manpower to keep up with it.
 
... or you have some pranksters just ok-ing all commits during their morning 
coffee, independent of content, which would keep things working at the cost of 
quality ...

Re: [gentoo-dev] RFC: Indention in metadata.xml

[Patrick Lauer]

On 06/06/15 09:26, Justin Lecher (jlec) wrote:
 Hi everyone,
 
 Can we get an agreement on how we are indenting metadata.xml?
 
 I like to properly format and indent metadata.xml, but without having
 an agreement or policy on the indention, I make unhappy by choosing
 the wrong.
 
 The two options which are already suggested are
 
 * 2 spaces
 * single tab
 
 So what should it be?

Obviously a tab, as tabs were made for indentation.

Re: [gentoo-dev] more packages up for grabs

[Patrick Lauer]

On 05/19/15 05:13, Tim Harder wrote:
 Here are some packages that I've dropped (or will drop) myself as
 primary maintainer from. Many of them (e.g. protobuf*) could really use
 some more collaborative non-maintainer update method but no one has
 gotten around to finalizing, documenting, and implementing the required
 metadata.xml changes yet as far as I know.
 
 Anyway, here's the package list (somewhat incomplete because some were
 dropped months ago and might also have new maintainers already but I
 doubt they'd mind extra help):
 
 * app-admin/lsyncd

I'll grab this one if no one else does

Re: [gentoo-dev] Becoming a Gentoo developer?

[Patrick Lauer]

On Saturday 18 April 2015 11:15:56 hasufell wrote:
 On 04/17/2015 07:15 PM, Rich Freeman wrote:
  On Fri, Apr 17, 2015 at 10:41 AM, Alexander Berntsen
  
  berna...@gentoo.org wrote:
  On 17/04/15 16:33, Andrew Savchenko wrote:
  The problem is double effort: previously one developer effort was
  needed, now effort is doubled at least
  
  You have correctly identified the problem; in order to do things
  properly one must do things properly, which is more difficult than not
  doing things properly.
  
  Properly is just a matter of requirements.  Gentoo has 18k packages
  right now.  In my general experience, they install fine maybe 95% of
  the time.
 
 Can you back up your general experience with a tinderbox log? In
 addition, you are decreasing QA to compiles. That's not the definition.


Out of all the packages that are visible (i.e. not masked, hidden by 
uninstallable dependencies, or hidden by useflag requiremenst)

For amd64 I see a build failure rate of ~2%, iow. out of ~10k packages that 
are buildable with a standard profile about 200 fail. (Last run done about half 
a year ago, it's slowly improved since I started in 2006)

So the 95% buildable sounds like a very pessimistic estimate to me.

For x86 the data looks pretty much the same, I think marginally worse.

Re: [gentoo-dev] Re: [gentoo-commits] gentoo-x86 commit in eclass: fcaps.eclass

[Patrick Lauer]

On Thursday 19 February 2015 12:31:27 Alexis Ballier wrote:
 On Wed, 18 Feb 2015 22:48:27 +0100
 
 Michał Górny mgo...@gentoo.org wrote:
  Dnia 2015-02-18, o godz. 16:11:53
  
  Mike Frysinger (vapier) vap...@gentoo.org napisał(a):
   vapier  15/02/18 16:11:53
   
 Modified: fcaps.eclass
 Log:
 clarify USE=filecaps intention #540430
   
   Revision  ChangesPath
   1.11 eclass/fcaps.eclass
  
  Please commit the missing ChangeLog update and remember to update
  the ChangeLog after changing any eclass in any way. This is an
  official policy for any commits to the Gentoo repository [1] and a
  lack of consistency in entries to the ChangeLog is confusing to our
  developers and users.
  
  [1]:http://devmanual.gentoo.org/ebuild-writing/misc-files/changelog/index.
  html
 this policy is about packages; cvs log is *mch* better than any
 global changelog for eclasses: who will dig into a thousand changelog
 entries to find what changed in fcaps.eclass ?
 
 if you want changelogs in eclass/,  make it per-eclass, like it is
 already per-package.
 

We've had this discussion before ... so ...

if you want to change the rules, follow the usual path of bureaucracy. Civil 
disobedience does not apply here.

Re: [gentoo-dev] Re: [gentoo-commits] gentoo-x86 commit in dev-python/watchdog: watchdog-0.8.3.ebuild ChangeLog

[Patrick Lauer]

On Thursday 19 February 2015 10:07:30 Kristian Fiskerstrand wrote:
 On 02/19/2015 09:57 AM, Markos Chandras wrote:
  On 02/19/15 06:10, Mike Gilbert wrote:
  What saddens me the most is that these pointless threads are
  becoming sort of a habit not because the reporter is really
  offended by the original action, but because he/she uses that to
  prove that QA is dying, Gentoo is a zombie, cvs sucks and
  elephants could fly.
 
 I'm not sure I agree the original thread was pointless. it was a
 relatively succinct statement that reported a breach of procedure, for
 which I'd consider the -dev list to be the appropriate place.

There's a pattern where a few people hit the big red button for any breach of 
protocol, which in itself is a breach of protocol.

The only way to contain their behaviour is to assume that common sense doesn't 
exist and then add so many rules to every little action that no ambiguity 
exists, thus killing most actions in bureaucratic red tape.

I don't see that path as a viable option, it'd be easier if such people got 
beaten with some common sense until they learned.
 
 What is more relevant is how that is being followed up by others
 afterwards. Ok, I did a mistake, will remember to check maintainer
 more closely before bumping a package that is new to me and I'm not
 familiar enough with to know it is in Herd X would be a perfectly
 acceptable answer.
 

Trivial python package gets bumped. OMG TEH END OF TAH WORLD.

Is that really a problem we should worry about? And do we need to be this 
territorial?
My packages are pretty much Do what you want, unless you are the sci herd 
and repeatedly breaking it for no apparent reason, in which case I must doubt 
your sanity ... but apart from that ... just do it.
I'd appreciate it if we could get a more community thingy going (as Senor 
Hasufell always complains), which also would imply being less territorial lone 
wolf. The obvious self-contradictions are amusing!


So anyway.

We don't need more rules, we need less complaining, and more fixing. I still 
have over 500 open bugs, a lot of them simple dependency missing bugs, and 
maintainers usually try to sit those out. That's something worthy of a rant, 
not some random python package ... bumping of which didn't cause any bugs ... 
etc.etc.

Re: [gentoo-dev] Re: [gentoo-commits] gentoo-x86 commit in dev-python/watchdog: watchdog-0.8.3.ebuild ChangeLog

[Patrick Lauer]

On Wednesday 18 February 2015 18:43:59 hasufell wrote:
 Is there a communication problem?
 
 I don't remember getting either:
 * a bug report
 * a ping
 * a review request
 
 Did I miss something?

Yes.

Why is this package metadata missing the python herd for no reason?


Also
Why are you pretending to own packages when we're supposed to be working 
together as a community / team / ... ?!


 
 Justin Lecher (jlec):
  jlec15/02/17 13:39:15
  
Modified: ChangeLog
Added:watchdog-0.8.3.ebuild
Log:
Version Bump

(Portage version: 2.2.17/cvs/Linux x86_64, signed Manifest commit with
key B9D4F231BD1558AB!)

Re: [gentoo-dev] Re: [gentoo-commits] gentoo-x86 commit in dev-python/watchdog: watchdog-0.8.3.ebuild ChangeLog

[Patrick Lauer]

On Thursday 19 February 2015 00:48:18 hasufell wrote:
 Patrick Lauer:
  Why is this package metadata missing the python herd for no reason?
 
 Because the python herd doesn't currently maintain the package, nor did
 they ask me to be co-maintainers.
 

So you put a python package in the dev-python category (which is almost 
completely managed by the python herd).

Then you don't ask the python herd if they want to co-maintain it ... 


I'm not quite sure how your reality works, but it seems to not share much with 
mine. But at least you found something new to be offended about and life 
doesn't get too boring ...

[gentoo-dev] Making more repoman checks fatal

[Patrick Lauer]

Right now repoman is relatively permissive - it whines about many things, but 
treats many issues as warning. 
The result is that many ebuilds get committed with 'minor' cosmetic issues 
which then someone more OCD than the original committer cleans up, making 
pretty much everyone involved more unhappy.

There's no reason to not error out on, for example, an invalid RESTRICT. Just 
printing a message is relatively useless.


Thus I suggest making the following warnings proper errors:

(Taken from current repoman 'qawarnings' set)

changelog.missing,
changelog.notadded,
digest.assumed,
digest.unused,
ebuild.notadded,
ebuild.nesteddie,
DESCRIPTION.toolong,
RESTRICT.invalid,
ebuild.minorsyn,
ebuild.badheader,
metadata.warning,
LIVEVCS.stable,
LIVEVCS.unmasked,

Most of these have few or no occurrences in the current tree, so changing the 
default from warn to error won't get in the way of the normal workflow.

(A few of them, like DESCRIPTION.toolong, still have about a dozen leftovers, 
but that should be easy to fix)

Have fun,

Patrick

Re: [gentoo-dev] Re: [gentoo-commits] gentoo-x86 commit in dev-libs/libusbhp: ChangeLog Manifest libusbhp-1.0.2.ebuild metadata.xml

[Patrick Lauer]

On Monday 16 February 2015 06:13:10 Mike Frysinger wrote:
 On Mon, Feb 16, 2015 at 1:16 AM, Alec Warner anta...@gentoo.org wrote:
  On Sun, Feb 15, 2015 at 8:05 PM, Mike Frysinger vap...@gentoo.org wrote:
  On Wed, Dec 31, 2014 at 12:21 AM, Patrick Lauer (patrick)
  
  patr...@gentoo.org wrote:
   patrick 14/12/31 05:21:11
   
 Removed:  ChangeLog Manifest libusbhp-1.0.2.ebuild
 
   metadata.xml
 
 Log:
 QA: Remove package with invalid copyright
  
  you do not go reverting code without actually talking to people.  if
  you feel like a revert is necessary, then file a bug.  putting a QA
  tag at the start of the commit message doesn't give you a pass.
  
  Normally I'd side with you on this...but I'm fairly sure repoman doesn't
  let you commit packages to the tree missing these headers. This leads me
  to believe you didn't use repoman, or ignored it?
 
 feel free to grab the code i originally committed and run `repoman
 full` yourself.  no fatal errors.  in fact you can see the generated
 tags in my commit message.

Well, AutoRepoman triggered on it.

Testing for fun on a random ebuild:

RepoMan scours the neighborhood...
  ebuild.badheader  1
   dev-db/hyperdex/hyperdex-1.6.0-r1.ebuild: Invalid Gentoo Copyright on line: 
1


Which again leads me to the question:

Why are these checks not properly fatal?

(And I really do not like having to repeat myself ...)

 
 even then, deleting an ebuild purely due to different copyright is
 complete bs.  anyone who understands copyright knows the situation in
 Gentoo is completely unenforceable.  we have no CLA.  this was
 patrick/QA wasting people's time to check a meaningless box.
 -mike

As others have pointed out, policy is policy. Don't shoot the massager.

Since I can't just fix the copyright (that would be more wrong) I opted for the 
easy way out - remove offending bits.


Have fun,

Patrick

Re: [gentoo-dev] Mozilla products in Gentoo

[Patrick Lauer]

On Friday 06 February 2015 06:28:27 Jory A. Pratt wrote:
 As many of you are aware we are using esr ( extended service releases
 ) for stable. We will remove all versions prior to 31.x in 7 days. If
 you are needing an older version please let us know and we will try to
 accommodate all users as much as possible. It is time to remove all
 security effected builds that are no longer supported upstream.
 
 Thanks
 Jory

If you don't mind, I'd like TB-24 to stay around a little while longer.

Upgrades break IMAP on ~1/3rd of all machines I tested on, and restarting from 
a clean profile takes a few hours to ingest emails during which TB is pretty 
much completely unresponsive. So until upstream figures out upgrades some of us 
are stuck with old versions ...


Thanks,

Patrick

Re: [gentoo-dev] Things one could be upset about

[Patrick Lauer]

On 01/19/15 17:47, Jeroen Roovers wrote:
 On Sat, 17 Jan 2015 19:35:09 +0800
 Patrick Lauer patr...@gentoo.org wrote:
 
 * AutoRepoman catches on average maybe 2 user-visible breakages.
 Mostly removing stable on HPPA ;)
 Fix: Make repoman faster (tree-wide scans take ~2 CPU-hours)
 Fix: Remind people that using repoman is not optional
 
 repoman doesn't check reverse dependencies for the package you're
 working on.

If it were fast enough we could do that.

pcheck from pkgcore was at one point down to under 3 minutes for a full
tree scan, that's pretty much fast enough so that people could run it
on almost every ebuild removal. repoman takes around 30 minutes when
parallelized, on the fastest hardware I currently have access to. Or
about 2 CPU-hours with a single thread ... that's prohibitively slow.

Re: [gentoo-dev] Review: desc/cpu_flags_x86.desc

[Patrick Lauer]

On Sunday 18 January 2015 21:44:05 Michał Górny wrote:
 Hello,
 
 I would like to commit the following flags as cpu_flags_x86_desc.
 The list combines global USE flags with some local USE flags I've been
 able to find.
 
 
 3dnow - Use the 3DNow! instruction set
 3dnowext - Use the Enhanced 3DNow! instruction set

Those are kinda mostly dead (no new CPUs have them anymore)

 aes-ni - Enable support for Intel's AES instruction set (aes in cpuinfo)
 avx - Adds support for Advanced Vector Extensions instructions
 avx2 - Adds support for Advanced Vector Extensions 2 instructions
 fma - Use the Fused Multiply Add instruction set

 mmx - Use the MMX instruction set
Not sure if this one needs to be specified - amd64 always has it, and on x86 
your code should do cpu feature detection anyway

 mmxext - Use the Extended MMX instruction set (intersection of Enhanced
 3DNow! and SSE instruction sets) (3dnowext or sse in cpuinfo) padlock - Use
 VIA padlock instructions
Kinda very dead, even more than 3dnow

 popcnt - Enable popcnt instruction support
Why?!

 sse - Use the SSE instruction set
Always exists on amd64, so this would be x86 special

 sse2 - Use the SSE2 instruction set
 sse3 - Use the SSE3 instruction set (pni in cpuinfo)
 sse4 - Enable SSE4 instruction support
 sse4_1 - Enable SSE4.1 instruction support
 sse4_2 - Enable SSE4.2 instruction support
 sse4a - Enable SSE4a instruction support
 ssse3 - Use the SSSE3 instruction set
Wow, such a wonderful mess :)


So half of those are obsolete/dead, and the other half you need to do proper 
feature detection - why do we want that as useflags again?

Re: [gentoo-dev] Things one could be upset about

[Patrick Lauer]

On Saturday 17 January 2015 15:03:05 Ciaran McCreesh wrote:
 On Sat, 17 Jan 2015 22:58:33 +0800
 
 Patrick Lauer patr...@gentoo.org wrote:
  On Saturday 17 January 2015 14:32:03 Ciaran McCreesh wrote:
   On Sat, 17 Jan 2015 21:03:30 +0800
   
   Patrick Lauer patr...@gentoo.org wrote:
Last time I tested paludis it was slower
   
   You've yet to do a like-for-like comparison...
  
  Hello hostile upstream.
  
  It was as like for like as possible
 
 Well that's the point...

A rare case of violent agreement?

Excellent. And/or you're terminally bored - either way you're not adding 
anything relevant to this conversation.

Re: [gentoo-dev] Things one could be upset about

[Patrick Lauer]

On Saturday 17 January 2015 14:00:34 William Hubbs wrote:
 On Sat, Jan 17, 2015 at 01:44:21PM +0100, Dirkjan Ochtman wrote:
  On Sat, Jan 17, 2015 at 12:35 PM, Patrick Lauer patr...@gentoo.org 
wrote:
   * Stage3 archives are too fat
   
   See https://bugs.gentoo.org/show_bug.cgi?id=531632
   We're now shipping three python versions and glib for extra fun!
   Fix: Motivate releng to stop being silly
  
  Why the heck do we ship both 3.3 and 3.4? I forget the exact situation
  with 2.x and 3.x, but I don't think setting PYTHON_TARGETS to 2.7-only
  is a great option if that remains the default after installation
  (although it would be fine for just the initial stages).
 
 I'm going to be very blunt. I am sick of the finger being pointed
 only at releng for this.
 
 The issue is package dependencies.
 
 If even one package in the tree has a dumb dependency on python, e.g.
 dev-lang/python, it will pull in all stable versionf of python.

Only if you absolutely insist that releng can never deviate from tree.

Which is a silly idea, see bindist useflag, which is locally enabled for stage 
building and then removed. Oh wait, it's not removed because we can't deviate 
while deviating. So users regularly find ssl 'broken' ...

It took me about 2h of fiddling around to find a few spots where stage3 has 
useless bloat:
- pkgconfig pulling in 30MB of glib
- python installing tests (that's 3x 25MB now ...)
- having more than one python installed (which is not really absolutely 
necessary, and could easily be reduced to one)

Out of 700MB on-disk I could prune about 150MB - about 20%, without affecting 
functionality

It's not about pointing a finger, it's about fixing issues when they are easy 
to 
fix and not hiding behind a fake complexity argument.
(I would fix things, if I had access and/or certainty that patches provided 
would be integrated ...)

Have fun,

Patrick

Re: [gentoo-dev] Things one could be upset about

[Patrick Lauer]

On Saturday 17 January 2015 13:12:56 Zac Medico wrote:
 On 01/17/2015 03:35 AM, Patrick Lauer wrote:
  * Portage is too slow
  
  On 'small' hardware emerge -upNDv @world can take enough time
  to make updates prohibitive - on an 800Mhz machine it took me
  about 3 days to figure out a solution to some silly blockers due to
  the
  very slow change test cycle
  Fix: Do some profiling and un-refactoring to remove useless code
  Fix: Set up a reference system (CI) to catch future regressions
 
 I'm certainly in favor of improving portage performance. However, for
 small hardware (which includes many ARM and MIPS systems), we really
 need to focus on binary package support. Note that dependency resolution
 for installing binary packages tends to be much simpler than for
 building ebuilds from source, and this translates into much shorter
 dependency resolution time.
 
That's an orthogonal problem. And that's coming from someone who extensively 
uses binpkgs already ...

The problem with (dependency resolution) performance is that in some 
scenarios, even on fast machines, it takes too long - especially if there's 
some silly useflag mismatch and two packages that have ~arch keywords, and now 
you need 12 attempts to figure out a solution that works for you ...
At 30 seconds for each resolution cycle that's 6 minutes waiting for portage.

If it were faster it'd almost be interactive ...

Also - just for comparison:
On my currently fastest box emerge -ep @world takes about 3 seconds since 
there's almost no packages installed.
On my currently slowest box same takes about 15 minutes, because (1) lots of 
packages installed and (2) slow CPU and (3) brutally slow IO

Binpkgs only help so much - if any dependencies change I need to sync the 
configuration between client and server, and to see if it resolves I need to do 
a dryrun on the client (or be very certain that the configuration really 
matches) - or risk that it's going to fail because of mismatches.


I haven't quite figured out why portage needs such humongous amounts of memory 
(300MB ?!) and why it needs to spend a minute to figure out how to install a 
simple almost-no-dependencies tool like htop or iotop. There's some obvious 
badness, and just saying well let's use binpkgs won't fix it (and, well, on 
the binpkg server if you have 3k packages installed you get the same 
performance issues, so ignoring the issue is kinda not a good idea)

There's been some good progress, I remember you reducing memory usage already 
(500MB - 350MB or so for merging kernel sources) and there's some speedups 
from the last round of performance work. Still I think we can do a lot better 
:)

Thanks for your efforts,

Patrick

Re: [gentoo-dev] Re: [gentoo-commits] gentoo-x86 commit in dev-libs/libuv: libuv-1.2.1.ebuild ChangeLog

[Patrick Lauer]

On Saturday 17 January 2015 17:25:15 hasufell wrote:
 Patrick Lauer:
  On Friday 16 January 2015 18:29:08 hasufell wrote:
  Patrick Lauer:
  On 01/16/15 23:26, hasufell wrote:
  Patrick Lauer (patrick):
  patrick 15/01/16 04:16:55
  
Modified: ChangeLog
Added:libuv-1.2.1.ebuild
Log:
Bump
  
  I expect people to ask me for review if they bump any of my packages.
  That includes QA team members.
  
  Are you always in such a bad mood?
  
  Do you, as QA team member, think that a review workflow improves quality?
  
  No.
  
  Bureaucracy does not improve quality by itself.
 
 Patrick, I am really sorry, but this answer made me laugh and cry.
 
 A review workflow is a fundamental concept in programming methodology
 and we have decades of experience that taught us how important it is.

But just add review or add agile doesn't fix quality. Same way Security is 
not just a checkbox, but a process. (Plus there's the whole manpower thing 
we'll ignore for now, etc. etc. ...)

I'm unwilling to entertain your attempts at baiting me into statements you 
hope to use against me, and I'm unwilling to play your passive-agressive 
whining game.

 
 I have no words to describe how disappointed I am right now.

It'll pass, don't worry ...

[gentoo-dev] Things one could be upset about

[Patrick Lauer]

Here's a random unsorted list of things that it would make sense to be upset 
about. Some issues that people have successfully ignored for a few years ...

In no way exhaustive list, feel free to remember a dozen things I forgot ;)
(If you suggest other things please try to offer constructive criticism,
i.e. possible strategies to fix issues ... whining by itself is not very 
useful) 

* stable genkernel still doesn't enable all useful kernel features
   e.g. accounting statistics are absent, so iotop doesn't work ootb
   See for example #442936 (from 2012 ?!)
   Fix: Stable newer versions

* stage3 still enables bindist in make.conf
   See https://bugs.gentoo.org/show_bug.cgi?id=473332
   Causes random breakage of tools like wget/curl, etc.
   Fix: poke releng until they stop being silly 

* AutoRepoman catches on average maybe 2 user-visible breakages.
Mostly removing stable on HPPA ;)
Fix: Make repoman faster (tree-wide scans take ~2 CPU-hours)
Fix: Remind people that using repoman is not optional

* Portage is too slow
On 'small' hardware emerge -upNDv @world can take enough time 
to make updates prohibitive - on an 800Mhz machine it took me 
about 3 days to figure out a solution to some silly blockers due to the
very slow change test cycle
Fix: Do some profiling and un-refactoring to remove useless code
Fix: Set up a reference system (CI) to catch future regressions

* Stage3 archives are too fat
See https://bugs.gentoo.org/show_bug.cgi?id=531632
We're now shipping three python versions and glib for extra fun!
Fix: Motivate releng to stop being silly

* AutoRepoman catches issues, but no one but me seems to care
Fix: Remind people of 
http://packages.gentooexperimental.org/repoman-current-issues.txt
Fix: Make it yell louder? It currently reports on IRC to #gentoo-{bugs,qa}

* AutoRepoman still runs on my hardware
Fix: Remind infra of https://bugs.gentoo.org/show_bug.cgi?id=484064

* mail archives have been broken since 2012
Fix: get infra to either fix it, or provide enough information that it can 
be fixed. See https://bugs.gentoo.org/show_bug.cgi?id=424647#c27

* git.overlays.gentoo.org only partially functional (web interface / cgit 
down)

* euscan doesn't run on infra hardware
Fix: https://bugs.gentoo.org/show_bug.cgi?id=453886

* libreoffice-bin isn't built on infra hardware
Fix: Remind infra to set up a build environment for that

* Some stable bugs are left alone for months
   See e.g. https://bugs.gentoo.org/show_bug.cgi?id=485632
   Fix: Have more people work on stable bugs
   Fix: Motivate people to file more stable bugs (continuous updates)

* Updates from very old installs are still difficult
   Fix: Collect stage3 archives and tree snapshots maybe every 3 months apart
 then update from one snapshot to the next. Possibly fix upgrade bugs 
 retroactively in the snapshots and automate the process


It'd be nice to have such things fixed, but alas, many rely on someone else to 
respond. And for some there's no 'easy' solution.

But they can all be fixed.

Let's not tolerate mediocrity.

Take care,

Patrick

Re: [gentoo-dev] Things one could be upset about

[Patrick Lauer]

On Saturday 17 January 2015 13:44:21 Dirkjan Ochtman wrote:
 On Sat, Jan 17, 2015 at 12:35 PM, Patrick Lauer patr...@gentoo.org wrote:

  * AutoRepoman catches issues, but no one but me seems to care
  
  Fix: Remind people of
  http://packages.gentooexperimental.org/repoman-current-issues.txt
  Fix: Make it yell louder? It currently reports on IRC to
  #gentoo-{bugs,qa}
 I'll happily fix my repoman issues when I notice them. I try to always
 run repoman full on a package, but like you say, a tree-wide scan
 isn't really viable on a per-commit basis. Can we have AutoRepoman
 report open issues to gentoo-dev on a weekly basis? That seems like a
 fine idea to me. Per-maintainer and per-team/project/herd RSS/Atom
 feeds would also be pretty nice to have.

Most issues are transient, often fixed with either a keywording bug, or careful 
masking of useflags / pruning of old versions. Per-maintainer doesn't really 
make sense as most issues are indirect - things like removing package.mask 
entry for new version causes dependency breakage on ia64 to become visible, 
or dependency of dependency changed. In both cases it's often hard enough to 
figure out what caused the issue.
 
 Also, I hate something like
 ['dev-python/restkit[python_targets_python2_7(-)?,-python_single_target_pyt
 hon2_7(-)]']. What the hell kind of warning is that? I guess maybe these
 are the results of USE_EXPAND trickery and what not, but it would sure be
 nice to have something more readable.

Indeed, portage output is quite complex and often hard to read. I'm not sure 
what's the best way to improve it - I've filed a few small bugs for output 
prettyfication, but I don't even know what I'd want to be displayed for these 
useflag / blocker issues.

 
  * Portage is too slow
  
  On 'small' hardware emerge -upNDv @world can take enough time
  to make updates prohibitive - on an 800Mhz machine it took me
  about 3 days to figure out a solution to some silly blockers due to
  the
  very slow change test cycle
  Fix: Do some profiling and un-refactoring to remove useless code
  Fix: Set up a reference system (CI) to catch future regressions
 
 Why not use paludis, or another package manager?

Last time I tested paludis it was slower (and building it OOMed on that 
machine with default settings, so that's quite amusing)

Also it suffers from a hostile upstream, makes bugreports very tricky to handle 
etc.etc.

Pkgcore is in hibernation, it needs a bit more work to become really useful 
again. Possibly some ideas from pkgcore can be migrated to portage again, but 
that'd need someone with lots of free time.


Thanks for your feedback,

Patrick

Re: [gentoo-dev] Things one could be upset about

[Patrick Lauer]

On Saturday 17 January 2015 14:32:03 Ciaran McCreesh wrote:
 On Sat, 17 Jan 2015 21:03:30 +0800
 
 Patrick Lauer patr...@gentoo.org wrote:
  Last time I tested paludis it was slower
 
 You've yet to do a like-for-like comparison...

Hello hostile upstream.

It was as like for like as possible, even when there had to be workaround to 
make paludis build (and wait multiple hours as it is kinda horribly C++) as it 
tried to OOM very nicely.

You can continue whining about it as much as you want, my results were 
duplicated by multiple people and you never offered a tweaked comparison that 
is to your liking. So stop whining (which is amusingly the trigger for my 
initial email - people whining about irrelevant things instead of doing 
something useful).

Re: [gentoo-dev] Things one could be upset about

[Patrick Lauer]

On Saturday 17 January 2015 14:45:51 Ciaran McCreesh wrote:
 On Sat, 17 Jan 2015 17:39:29 +0300
 
 Andrew Savchenko birc...@gentoo.org wrote:
  There is some progress here. In portage-2.2.15 profile based
  optimizations are included (see bugs 529660, 530010). On my
  hardware (Athlon-XP, 2200 MHz and Intel Atom N270, 1600 MHz) they
  speed up dependency resolution by ~ factor 2.
 
 The problem isn't the constants, though. The problem is the resolution
 algorithm. There's not much point tweaking performance until the
 resolver is fixed to produce a correct answer...

Patches welcome :)