Re: [gentoo-portage-dev] [PATCH] portage.output: Replace darkblue colors with teal

[Michael Orlitzky]

On Sat, 2021-12-04 at 10:47 +0100, Fabian Groffen wrote:
> 
> Now, if you would make a supported claim that all terminals we install
> use a black background by default, your change becomes more valid.
> 
> 

For one, when you're working through the handbook to install Gentoo,
you're doing it on a black background.

Or if you have any physical servers. The dark-blue-on-black is even
more fun to read from three feet away on the CRT monitor from 1995 in
the back of a poorly lit server room.

Re: [gentoo-portage-dev] [RFC] LTS branch of Portage

[Michael Orlitzky]

On Tue, 2021-10-05 at 17:13 +0200, Michał Górny wrote:
> 
> >   2. What happens to the LTS branch when the next EAPI is released?
> > 
> 
> I haven't really thought about it.  Are you suggesting that we could
> bump 'master' Portage to newer EAPI earlier or...?
> 

I just mean that, a priori, the LTS branch won't be able to install
ebuilds that use the next EAPI. Backporting an entire EAPI doesn't seem
appropriate for a stable series; but maintaining an increasingly-
obsolete branch at that point doesn't make much sense either.

Re: [gentoo-portage-dev] [RFC] LTS branch of Portage

[Michael Orlitzky]

On Tue, 2021-10-05 at 10:31 +0200, Michał Górny wrote:
> Hi, everyone.
> 
> I've been thinking about this for some time already, and the recent
> FILESDIR mess seems to confirm it: I'd like to start a more stable LTS
> branch of Portage.
> 

I think this is healthy for most software projects, but two things
stand out to me for portage specifically:

  1. Most portage users can fake this already by telling portage to 
 accept only stable keywords for sys-apps/portage. I know it's not 
 exactly the same, but it provides many of the same benefits.

  2. What happens to the LTS branch when the next EAPI is released?

[gentoo-portage-dev] [PATCH 1/1] Revert "repoman: deprecate netsurf.eclass."

[Michael Orlitzky]

This reverts commit a73024729860f9224b8d1660d24c450080b67d9f. This
eclass was successfully purged from the tree, so the deprecation is no
longer needed. And eventually, to address an eblit infestation,
another eclass with the same name will return. The new one will not be
deprecated.

Signed-off-by: Michael Orlitzky 
---
 repoman/lib/repoman/modules/linechecks/deprecated/inherit.py | 1 -
 1 file changed, 1 deletion(-)

diff --git a/repoman/lib/repoman/modules/linechecks/deprecated/inherit.py 
b/repoman/lib/repoman/modules/linechecks/deprecated/inherit.py
index 60410347b..5848a0c37 100644
--- a/repoman/lib/repoman/modules/linechecks/deprecated/inherit.py
+++ b/repoman/lib/repoman/modules/linechecks/deprecated/inherit.py
@@ -33,7 +33,6 @@ class InheritDeprecated(LineCheck):
"gst-plugins10": "gstreamer",
"ltprune": False,
"mono": "mono-env",
-   "netsurf": False,
"python": "python-r1 / python-single-r1 / python-any-r1",
"ruby": "ruby-ng",
"user": "GLEP 81",
-- 
2.26.2

Re: [gentoo-portage-dev] erroneous behavior in 2-style USE dependencies?

[Michael Lienhardt]

Le 17/06/2020 à 08:15, Michał Górny a écrit :

On Tue, 2020-06-16 at 23:07 +, Michael Lienhardt wrote:

Dear all,

My bad for not noticing it sooner, but when there is a dependency like 
">=sys-fs/udev-208-r1:0/0[static-libs?]" (that occurs in 
virtual/libgudev-215-r3),
  since 'static-libs' is not a use flags of sys-fs/udev-242, that cpv is 
silently not considered during dependency solving by emerge.
However, the PMS states:
  - it is an error for a use dependency to be applied to an ebuild which does 
not have the flag in question in IUSE_REFERENCEABLE



This is a bit like undefined behavior.  PMS says such a thing shouldn't
happen (i.e. the ebuild is wrong) but does not force specific error
handling.  You could reject the ebuild entirely or reject dependencies
that don't have the flag in question (even if it's disabled).  You could
also pretend it's 'static-libs(-)?' but that would be bad if the default
was supposed to be '(+)'.


Indeed. It's true that when I read "error" I understand "something that 
never occur in a distributed gentoo repository".





This is related to the tool I'm working on: should my tool allow this behavior, 
or fail like it is currently doing (I guess the former)?


That depends on what the tool is doing.  I suppose you probably don't
need very strict behavior there.


Indeed, I don't need a strict behavior, but since I saw an ambiguity 
between the PMS and emerge, I went to check with you if this ambiguity 
was intentional, and found out along the way how to deal with this 
situation in my tool. My tool is still the SAT-based dependency analysis 
you don't really believe in :p. It's going forward slowly but (among 
other things) thanks to the help of you all, I got a paper accepted to a 
top software engineering conference. So, thanks! Of course, my final 
goal is to have the tool fully functional (it will the subject of two 
other papers -- there's a lot to say on how to deal with gentoo).


Since the current gentoo repo includes "erroneous" 2-style USE 
dependency, I will change my tool's current behavior (raising an error) 
to the "no match with warning" that Zac proposed (which seems the safest 
approach to take in the current situation).


Best,
Michael

Re: [gentoo-portage-dev] erroneous behavior in 2-style USE dependencies?

[Michael Lienhardt]

Le 17/06/2020 à 10:35, Ulrich Mueller a écrit :

On Wed, 17 Jun 2020, Michael Lienhardt wrote:



But maybe, "error" here in the PMS mean "the cpvs without the use flag
does not match that dependency and a warning should be raised to
improve compatibility in the future". In that case, it would be
clearer for me to change 'error' in the PMS into something like
"results in a no match,


IMHO we cannot assume that. If the flag is not in the dependency's
IUSE_EFFECTIVE then behaviour is undefined.


Fair enough.
However, currently the PMS says it is an error, not an undefined behavior.


but should be avoided". That way, it is explicitly stated that missing
use flags for a 2-style USE dependency is accepted (which is the
current behavior of emerge) but frown upon, without forcing any
specific error handling, like Michał accurately pointed out.


The real problem is that we don't have a good procedure for removing
flags from ebuilds with reverse (2-style) use dependencies. (And even
with 4-style use dependencies the problem remains that one cannot know
in advance whether removal of the flag implies that the feature is now
unconditionally enabled, or that it is disabled.)


Indeed.
This is outside the scope of my original question, but intuitively, I 
would imagine that the devs should know why they remove a use flag. It's 
just an idea, but I see two possibilities.
1. either the change is temporary: in that case, they could keep the use 
flag and set its value in REQUIRED_USE during that period
2. either the change is durable: in that case, it is still possible to 
keep the use flag (while still setting its value in REQUIRED_USE) during 
a period of time during which it is possible to update the dependencies 
toward that package (the use flag would become deprecated before being 
removed).
That way, enforcing the error described in the PMS would be telling to 
the devs that they didn't update their dependencies during the 
transition period.


Best,
Michael

Re: [gentoo-portage-dev] erroneous behavior in 2-style USE dependencies?

[Michael Lienhardt]

On 6/17/20 1:25 AM, Zac Medico wrote:
> On 6/16/20 7:47 PM, Michael Lienhardt wrote:
>>
>>
>> On 6/16/20 11:59 PM, Zac Medico wrote:
>>> On 6/16/20 6:38 PM, Michael Lienhardt wrote:
>>>> With the first version of DEPEND, emerge succeed:
>>>> # emerge -pv app-misc/dummy-master
>>>>
>>>> These are the packages that would be merged, in order:
>>>>
>>>> Calculating dependencies... done!
>>>> [ebuild  N ] app-misc/dummy-slave-2::gentoo  USE="-static-libs" 0 KiB
>>>> [ebuild  N ] app-misc/dummy-master-1::gentoo  USE="-static-libs" 0 KiB
>>>
>>> This success is expected, yes? Do you suggest to change the behavior
>>> somehow?
>>
>> The way I interpret the PMS, this success is not expected:
>>  the atom ">=app-misc/dummy-slave-1" matches the cpv 
>> "app-misc/dummy-slave-1" which does not contains the use flag 'static-libs',
>>  and thus I expected a 'missing use flag' error.
> 
> For this calculation, it would be a waste of time to read the IUSE
> metadata for app-misc/dummy-slave-1, since app-misc/dummy-slave-2 is the
> highest available version.

Good point.
I changed the version of app-misc/dummy-slave-1 into app-misc/dummy-slave-3 (so 
now the higher version is the one without the 'static-libs' use flag), and 
still no error/warning message.

> I hope that PMS does not specify that package managers must read IUSE
> metadata for irrelevant package versions!

I think there is indeed an ambiguity there:
 Section 8.3 of the PMS states when a cpv matches an atom, but does not say 
which cpvs should be tested against an atom during dependency analysis.
This is where my interpretation was maybe wrong: when I see "error" in Section 
8.3.4 I understand
 "all cpv matching an atom with this 2-style USE dependency *must* have the use 
flag declared, otherwise
 the .ebuild should be considered erroneous" (I have a strong notion of error).
I thus thought that all .ebuilds in the distributed repos were checked (before 
distribution -- not by emerge) against that error.

But maybe, "error" here in the PMS mean "the cpvs without the use flag does not 
match that dependency and a warning should be raised to improve compatibility 
in the future".
In that case, it would be clearer for me to change 'error' in the PMS into 
something like "results in a no match, but should be avoided".
That way, it is explicitly stated that missing use flags for a 2-style USE 
dependency is accepted (which is the current behavior of emerge) but frown upon,
 without forcing any specific error handling, like Michał accurately pointed 
out.


>> I'm not suggesting to change the behavior of emerge, I'm saying that:
>>  - the way I read the PMS, I expect behavior A, but in practice, I see 
>> behavior B.
>>  - what does the portage devs / PMS gurus think about that?
>> - is my understanding of the PMS wrong, and it actually says "behavior B 
>> is expected"?
>> - if yes, where did I fail in my understanding?
>> - if no, should emerge or the PMS be updated so they both describe the 
>> same behavior?
>>  - I will implement your ruling in my tool, which I try to match as closely 
>> as possible to the PMS
> 
> In this context I think the spirit of what PMS says is that the package
> manager should emit some kind of warning message if it finds missing
> IUSE. Now, in the example above, if the package manager has no reason to
> examine the IUSE metadata of app-misc/dummy-slave-1 because
> app-misc/dummy-slave-2 is the highest available version, then there's no
> opportunity for it to emit a warning message.

>From what I've seen now, emerge considers a 2-style USE dependency error as a 
>"no match without warning message", and fails with my second version of DEPEND 
>because there are no .ebuild matching the dependency: the error message it 
>issues only describes why there is no solution, it is not a warning about an 
>erroneous dependency.

Best,
Michael

Re: [gentoo-portage-dev] erroneous behavior in 2-style USE dependencies?

[Michael Lienhardt]

On 6/16/20 11:59 PM, Zac Medico wrote:
> On 6/16/20 6:38 PM, Michael Lienhardt wrote:
>> With the first version of DEPEND, emerge succeed:
>> # emerge -pv app-misc/dummy-master
>>
>> These are the packages that would be merged, in order:
>>
>> Calculating dependencies... done!
>> [ebuild  N ] app-misc/dummy-slave-2::gentoo  USE="-static-libs" 0 KiB
>> [ebuild  N ] app-misc/dummy-master-1::gentoo  USE="-static-libs" 0 KiB
> 
> This success is expected, yes? Do you suggest to change the behavior
> somehow?

The way I interpret the PMS, this success is not expected:
 the atom ">=app-misc/dummy-slave-1" matches the cpv "app-misc/dummy-slave-1" 
which does not contains the use flag 'static-libs',
 and thus I expected a 'missing use flag' error.
I'm not suggesting to change the behavior of emerge, I'm saying that:
 - the way I read the PMS, I expect behavior A, but in practice, I see behavior 
B.
 - what does the portage devs / PMS gurus think about that?
- is my understanding of the PMS wrong, and it actually says "behavior B is 
expected"?
- if yes, where did I fail in my understanding?
- if no, should emerge or the PMS be updated so they both describe the same 
behavior?
 - I will implement your ruling in my tool, which I try to match as closely as 
possible to the PMS

>> With the second version of DEPEND, emerge fails:
>> # emerge -pv app-misc/dummy-master
>>
>> These are the packages that would be merged, in order:
>>
>> Calculating dependencies... done!
>>
>> emerge: there are no ebuilds built with USE flags to satisfy 
>> "=app-misc/dummy-slave-1[static-libs?]".
>> !!! One of the following packages is required to complete your request:
>> - app-misc/dummy-slave-1::gentoo (Missing IUSE: static-libs)
>> (dependency required by "app-misc/dummy-master-1::gentoo" [ebuild])
>> (dependency required by "app-misc/dummy-master" [argument])
> 
> This failure is expected, yes? Do you suggest to change the behavior
> somehow?

The way I interpret the PMS, this failure is expected.

I'm sorry if I'm not always clear, I try to be, and many thanks to take the 
time to answer my (unexpected and strange) questions.

Best,
Michael

Re: [gentoo-portage-dev] erroneous behavior in 2-style USE dependencies?

[Michael Lienhardt]

On 6/16/20 9:31 PM, Zac Medico wrote:
> On 6/16/20 11:07 PM, Michael Lienhardt wrote:
>> I'm sorry, my client didn't allow to send plain text email anymore...
>> 
>> So, here is my original email.
>> 
>> Dear all,
>> 
>> My bad for not noticing it sooner, but when there is a dependency like 
>> ">=sys-fs/udev-208-r1:0/0[static-libs?]" (that occurs in 
>> virtual/libgudev-215-r3),
>>  since 'static-libs' is not a use flags of sys-fs/udev-242, that cpv is 
>> silently not considered during dependency solving by emerge.
>> However, the PMS states:
>>  - it is an error for a use dependency to be applied to an ebuild which does 
>> not have the flag in question in IUSE_REFERENCEABLE
>>  - For EAPIs listed in table 5.4 as not supporting profile defined IUSE 
>> injection, IUSE_REFERENCEABLE is equal to the calculated IUSE value. For 
>> EAPIs where profile defined IUSE injection is supported, IUSE_REFERENCEABLE 
>> is equal to IUSE_EFFECTIVE
>> And 'static-libs' is not in the IUSE_EFFECTIVE of sys-fs/udev-242 (that 
>> ebuild has EAPI=6).
>> So it seems to me that this current behavior of emerge should be considered 
>> an error, no? Or the PMS should be updated?
>> 
>> This is related to the tool I'm working on: should my tool allow this 
>> behavior, or fail like it is currently doing (I guess the former)?

> It's valid as a 4-style dependency with use-dep-defaults.

I know. Except that in virtual/libgudev-215-r3.ebuild we have in the DEPEND:
 >=sys-fs/udev-208-r1:0/0[${MULTILIB_USEDEP},gudev(-),introspection(-)?,static-libs?]
It is a 2-style dependency (following the PMS).

I checked with 3 dummy packages

1. app-misc/dummy-master-1
EAPI=6
SLOT=0
KEYWORDS="amd64"
IUSE="static-libs"
DEPEND=">=app-misc/dummy-slave-1[static-libs?]"
#DEPEND="=app-misc/dummy-slave-1[static-libs?]"

2. app-misc/dummy-slave-1
EAPI=6
SLOT=0
KEYWORDS="amd64"
IUSE=""
DEPEND=""

3. app-misc/dummy-slave-2
EAPI=6
SLOT=0
KEYWORDS="amd64"
IUSE="static-libs"
DEPEND=""

With the first version of DEPEND, emerge succeed:
# emerge -pv app-misc/dummy-master

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild  N ] app-misc/dummy-slave-2::gentoo  USE="-static-libs" 0 KiB
[ebuild  N ] app-misc/dummy-master-1::gentoo  USE="-static-libs" 0 KiB


With the second version of DEPEND, emerge fails:
# emerge -pv app-misc/dummy-master

These are the packages that would be merged, in order:

Calculating dependencies... done!

emerge: there are no ebuilds built with USE flags to satisfy 
"=app-misc/dummy-slave-1[static-libs?]".
!!! One of the following packages is required to complete your request:
- app-misc/dummy-slave-1::gentoo (Missing IUSE: static-libs)
(dependency required by "app-misc/dummy-master-1::gentoo" [ebuild])
(dependency required by "app-misc/dummy-master" [argument])



Michael

Re: [gentoo-portage-dev] erroneous behavior in 2-style USE dependencies?

[Michael Lienhardt]

I'm sorry, my client didn't allow to send plain text email anymore...

So, here is my original email.

Dear all,

My bad for not noticing it sooner, but when there is a dependency like 
">=sys-fs/udev-208-r1:0/0[static-libs?]" (that occurs in 
virtual/libgudev-215-r3),
 since 'static-libs' is not a use flags of sys-fs/udev-242, that cpv is 
silently not considered during dependency solving by emerge.
However, the PMS states:
 - it is an error for a use dependency to be applied to an ebuild which does 
not have the flag in question in IUSE_REFERENCEABLE
 - For EAPIs listed in table 5.4 as not supporting profile defined IUSE 
injection, IUSE_REFERENCEABLE is equal to the calculated IUSE value. For EAPIs 
where profile defined IUSE injection is supported, IUSE_REFERENCEABLE is equal 
to IUSE_EFFECTIVE
And 'static-libs' is not in the IUSE_EFFECTIVE of sys-fs/udev-242 (that ebuild 
has EAPI=6).
So it seems to me that this current behavior of emerge should be considered an 
error, no? Or the PMS should be updated?

This is related to the tool I'm working on: should my tool allow this behavior, 
or fail like it is currently doing (I guess the former)?

Best,
Michael


On 6/16/20 7:42 PM, Brian Dolbec wrote:
> 
> Please do NOT send html emails.  text only please
> 

[gentoo-portage-dev] [PATCH 1/1] repoman: deprecate netsurf.eclass.

[Michael Orlitzky]

While investigating bug 489542, it became clear that the netsurf
eclass is deprecated in spirit if not in fact. All of the newer
netsurf ebuilds use a custom function loaded from a script that is
shipped in netsurf-buildsystem's $FILESDIR to do (some of) what the
eclass used to do. That function probably does belong in an eclass,
but at this point, we should throw this thing out and start from
scratch.

By deprecating the eclass, we make sure that no one else inherits it
during the time it takes to purge the two remaining consumers. Then,
once those are gone, the build system script magic can be put back
into an eclass, and its consumers updated one-at-a-time.

Bug: https://bugs.gentoo.org/489542
Bug: https://bugs.gentoo.org/637824
Signed-off-by: Michael Orlitzky 
---
 repoman/lib/repoman/modules/linechecks/deprecated/inherit.py | 1 +
 1 file changed, 1 insertion(+)

diff --git a/repoman/lib/repoman/modules/linechecks/deprecated/inherit.py 
b/repoman/lib/repoman/modules/linechecks/deprecated/inherit.py
index 5848a0c37..60410347b 100644
--- a/repoman/lib/repoman/modules/linechecks/deprecated/inherit.py
+++ b/repoman/lib/repoman/modules/linechecks/deprecated/inherit.py
@@ -33,6 +33,7 @@ class InheritDeprecated(LineCheck):
"gst-plugins10": "gstreamer",
"ltprune": False,
"mono": "mono-env",
+   "netsurf": False,
"python": "python-r1 / python-single-r1 / python-any-r1",
"ruby": "ruby-ng",
"user": "GLEP 81",
-- 
2.26.2

Re: [gentoo-portage-dev] [PATCH] Change BINPKG_COMPRESS default from bzip2 to xz

[Michael Orlitzky]

On 4/26/20 3:25 PM, Ulrich Mueller wrote:
>>>>>> On Sun, 26 Apr 2020, Michael Orlitzky wrote:
> 
>> Fuel for the fire:
> 
>>  * https://www.nongnu.org/lzip/lzip_benchmark.html
>>  * https://www.nongnu.org/lzip/xz_inadequate.html
> 
> Yep. That's why lzip is the dominant compression format now, and nobody
> is using xz any more.
> 

If you believed that argument, you would be using Windows =P

Dude has GRAPHS, it must be true.

Re: [gentoo-portage-dev] [PATCH] Change BINPKG_COMPRESS default from bzip2 to xz

[Michael Orlitzky]

On 4/26/20 12:55 PM, Matt Turner wrote:
> Bug: https://bugs.gentoo.org/715108
> Signed-off-by: Matt Turner 
> ---
> Strawman patch. Bikeshed away.
> 

Fuel for the fire:

 * https://www.nongnu.org/lzip/lzip_benchmark.html
 * https://www.nongnu.org/lzip/xz_inadequate.html

Re: [gentoo-portage-dev] precisions on installed packages' dependencies

[michael . lienhardt]

- Alec Warner  a écrit :

> Sorry I'm being overly academic. My concern earlier is that you mentioned a
> goal of "never breaking installed packages' which I found to be a fairly
> audacious goal. The idea is that we should build tools that achieve this
> practically (e.g. via heuristics such as := slot operators) while
> understanding that the complexities of application deploys are legion and
> the tool will never handle them all. So the goal of never breaking them is
> more an idealistic goal rather than a practical reality.

I agree.
I started this discussion because I thought that the content of the 
/var/db/pkg/* folder was not enough to keep the dependencies.
Then Zack and you showed me that I only saw the tip of the iceberg (in my 
defense, I first wanted to only keep the package's abstract dependencies, not 
the ones of the actual code. And then the discussion was really interesting).
My experience in dependency management is limited to an extended model of 
debian packages, so my question were (and will be) naive.

I understand that with the current status of Portage:
 - we can consider that the dependencies specified in a package ensure that it 
can be installed and run
 - however, package update and package removal is not guaranteed to work. Slot 
operators is an integrated way to capture some update behaviors, but not all. 
In general, a dedicated method (like for perl) can be needed.

I do believe that never breaking dependencies (at the code level) is a nice 
idealistic goal.
It might not always be possible to achieve it, but you did talk of much work 
done to do it where it is possible.
And, to come back to my previous question, I imagine that the slot operator is 
an ad-hoc but very useful to avoid dependency breakage when possible.
However, this operator looks strange to me: my (naive) intuition to express a 
trigger for package recompilation would be the other way around (i.e., it is 
the package that is updated that says what changes, and so, which other 
packages must be recompiled); like you illustrated with perl, an external tool 
(possibly different for each package) that gives which packages must be 
recompiled due to a specific package update.
This is why I asked why is the slot operator better as a recompilation trigger 
compared to other approaches? Is it because it only requires for the developer 
to add a "=" sign (simplicity is very important)? Or for another reason?

Many thanks!
Michael

Re: [gentoo-portage-dev] precisions on installed packages' dependencies

[michael . lienhardt]

- Alec Warner  a écrit :
> On Tue, Mar 24, 2020 at 11:31 AM  wrote:
> > However, I still doubt that only storing the soname dependencies is enough.
> > Consider package A (that cannot be recompiled) that depends on package B
> > which provides lib L.so.
> > B is recompiled with different use flags, which put different
> > functionalities in L.so.
> > The dependencies of A are still satisfied (B is installed, L.so is
> > available), but since the content of L.so changed, A cannot execute anymore.
> > Hypothetically, can this scenario occur?
> > Can this scenario occur in practice?
> > Is there a way in emerge/portage to avoid it?


> > You have far more experience than me on this, and it would be nice for me
> > to know what I'm up against.
> 
> A lot of this has to do with the specifics of how package managers manage
> system state, as well as various quirks of subsets of the tree. For
> example, a perl upgrade (X->Y) will often break perl modules who expect
> perl-X, but get perl-Y. So one fix is to try to keep perl-X installed (so
> we SLOT perl and have N perls installed.) Then you need to decide which
> version of perl to build things against (X or Y, or both?) We took this
> tactic in the python ecosystem; but perl is not slotted in Gentoo, and so
> upgrading perl breaks all perl modules. There is a tool
> (gentoo-perl-cleaner) that will walk the deptree and fix all of these
> broken packages that you run after an upgrade.
> 
> I'm not sure it's strictly avoidable. You could build perl-Y, then rebuild
> all perl-modules against perl-Y, then merge the entire result to the
> livefs. This will reduce the breakage time but likely not eliminate it;
> plus it seems hard to implement in practice without modern filesystem tools
> (overlayfs, btrfs, zfs or similar tech to make it atomic.) It also doesn't
> account for executing code. What happens to perl-X code that is executing
> when you unmerge perl-X? The short answer is that code might break and
> 'proper' management means you should restart services after an upgrade
> (something Gentoo doesn't typically do; but is common in Debian for
> example.)

Many thanks for this answer.
To sum up what I understood, the problem is not really the dependencies, but 
which recompilation (and service restart) are triggered with an update.
In gentoo, there is the ":=" slot operator (and others similar) in dependencies 
that trigger the recompilation when the dependency's slot change, but this is 
the only existing mechanism.
And this is why every time perl changes, the compilation of its modules is not 
triggered and they are most probably broken.
Correct?
And then, in this context, keeping the installed packages' dependencies 
consistent is up to debate: packages will get broken in any case...
It is clearly impossible to have a tool that automatically detect all 
implementation dependency breakage.

Again, there's something I probably don't see: why was slot operators chosen 
(among other possibilities) as a mechanism to trigger recompilation?
I'm very grateful to you all for the time you take to read and answer my 
questions.

Best,
Michael

Re: [gentoo-portage-dev] precisions on installed packages' dependencies

[michael . lienhardt]

- Zac Medico  a écrit :

> > The goal of my tool is to have correct manipulation of package 
> > dependencies, and in particular here, I focus on the packages that are 
> > installed but not in the portage tree/a local overlay anymore (the problem 
> > does not occur for other packages).
> > It seems that installed packages do not store which are the actual cpv they 
> > depend on. Correct?
> 
> Right, because unfortunately that's something that changes over time.
> 
> Also, we may not be able to pin it down at any given moment if we have
> inconsistent || preferences as described here:
> 
> https://archives.gentoo.org/gentoo-dev/message/550d3859dea6d0fb0b39064628992634

Hmm, I think I see what you mean.
Storing the cpvs that was used during solving the package's dependencies would 
be too restrictive, since two different packages could provide the exact same 
functionalities/libraries.
And so, during a system update, only looking at the cpv dependencies would 
trigger useless recompilation of the packages that depend on the updated 
packages.
Correct?

Btw, my tool's solver does not have the problem discussed in the thread you're 
mentioning: atom order in lists has no influence in my solver.
Would fixing the inconsistent || preferences make storing cpvs for installed 
packages more realistic?


> > Also, I wanted to use the ebuild tool to install/uninstall package, which 
> > is not possible with this solution apparently.
> 
> Why not? Would the preserve-libs feature solve your problem?

... I'm sorry, I wasn't aware of this feature.
It would definitively solve the issue (except, as described in the bug 459038, 
when external tools remove libs).

This discussion is very interesting!
If I take this double layer of dependencies, I have to check how this 
influences the theory underlying my tool.

However, I still doubt that only storing the soname dependencies is enough.
Consider package A (that cannot be recompiled) that depends on package B which 
provides lib L.so.
B is recompiled with different use flags, which put different functionalities 
in L.so.
The dependencies of A are still satisfied (B is installed, L.so is available), 
but since the content of L.so changed, A cannot execute anymore.
Hypothetically, can this scenario occur?
Can this scenario occur in practice?
Is there a way in emerge/portage to avoid it?


> Well, there are a lot of upgrades that can't be performed without
> temporarily breaking something, so the "forbid broken dependencies" idea
> doesn't sound feasible to me.

Could you tell me about several instances of such needed dependency breakage?
You have far more experience than me on this, and it would be nice for me to 
know what I'm up against.

Many thanks!
Michael

Re : Re: [gentoo-portage-dev] precisions on installed packages' dependencies

[michael . lienhardt]

- Zac Medico  a écrit :

> >  3. before removing a library, "ebuild unmerge" always checks if it is used 
> > by another package: this means that installed packages' dependencies are 
> > never broken.
> 
> That's true if the package is removed via emerge --depclean, but emerge
> --unmerge does not account for dependencies.
> 
> Also, it's possible for dependencies of installed packages to be
> temporarily broken by upgrades. In cases like this, the breakage will
> eventually be resolved by a rebuild (which occurs automatically for slot
> operator := deps), upgraded, or by emerge --depclean (which removes
> unneeded packages).

Many thanks for your answers.
They made me realize that the problem I'm facing is a bit more tricky than I 
first quickly though.

I'll try to explain the problem, tell me if I'm not clear somewhere.

The goal of my tool is to have correct manipulation of package dependencies, 
and in particular here, I focus on the packages that are installed but not in 
the portage tree/a local overlay anymore (the problem does not occur for other 
packages).
It seems that installed packages do not store which are the actual cpv they 
depend on. Correct?
Hence, when an installed package cannot be updated/recompiled because it is not 
in the tree anymore, like you said, its dependencies can be broken (due to the 
package it depends on being updated).
Currently, this issue is circumvented (only using depclean) by keeping the 
libs: the package's dependencies are broken, but it's ok because it can still 
run (which, in the end of the day, is what we want).
However, from your answer, it seems that this fix is not entirely integrated in 
the emerge/portage toolchain (like you said, emerge --unmerge removes 
everything, and emerge -u removes the old libs).

To sum up, the problem I'm facing is that with the current way installed 
packages are managed, we can break dependencies (and the only way to fix them 
is to remove the installed package with the broken dependencies, that can never 
be installed again).

Hence, for my tool, I have two solutions for that problem: either I forbid for 
dependencies to ever be broken, or I allow it.

Solution 1: forbid broken dependencies.
This requires to extend the information stored on installed package with the 
list of the actual cpvs they depend (or at least the cp+slot, which is enough 
to get back the cpvs).
That way, I can say in the solver "if you want to keep that package, you need 
to keep these packages as they currently are".
However, I have no idea on how to do that, and doing this only for my tool 
would mean that one cannot switch between emerge (quick) and my tool (correct), 
which is a feature I think is essential.
Do you think adding this new information to installed packages could be 
integrated into emerge/portage itself? I could work on it (expect question ^^), 
test it on my prototype, and do a pull request when everything's working.

Solution 2: allow broken dependencies.
Here, the idea is to use the same fix as is currently done with depclean, but 
in my tool's planner (i.e., the part that install/unistall the packages) 
directly.
That way, I say in the solver "that installed package has no dependency", but 
when I upgrade/remove packages, I say "Oh but wait, that other package still 
need these libs, let's keep them".
This solution may not require any change in portage/emerge, but I have no idea 
on how to know which libs are needed by a package, and how to track these libs 
owners without looking at every installed package's files (which are stored in 
the CONTENT file, if I'm not mistaken).
Also, I wanted to use the ebuild tool to install/uninstall package, which is 
not possible with this solution apparently.
In case I need to implement this, could you give me some clue on how to achieve 
it?


Among these two solutions, I prefer the first one: we stay at the level of 
package dependencies (and it looks simpler to implement).
However, it is maybe easier/better to use the second approach, I don't know.
Do you have some suggestions?

Thanks!
Michael

[gentoo-portage-dev] precisions on installed packages' dependencies

[michael . lienhardt]

Dear all,

Still in the process of improving my solver (and make it a usable tool), I need 
to have a better idea on how installed packages should be managed.
I didn't find anything on that topic in the PMS (if I've missed it, I'm sorry).
Could you confirm/correct my following understanding:
 1. installed packages that are still in the portage tree can be 
unmerged/updated without any restriction (as specified in their .ebuild)
 2. installed packages that are not in the portage tree can only be kept as is 
or unmerged
 3. before removing a library, "ebuild unmerge" always checks if it is used by 
another package: this means that installed packages' dependencies are never 
broken.

Many thanks!
Michael

Re: [gentoo-portage-dev] [PATCH gentoolkit 1/2] eclean: Rewrite findPackages()

[Michael 'veremitz' Everitt]

On 21/02/20 05:29, Matt Turner wrote:
> I found the original code to be nearly incomprehensible. Instead of
> populating a dict of potential binpkgs to remove and then removing from
> the to-be-removed list, just selectively add to-be-removed packages.
>
> Signed-off-by: Matt Turner 
> ---
> I switched from tabs to spaces in the process. I can revert back if
> desired.
>
Probably best to stick to tabs for consistency with the other portage code,
although naturally Zac probably better to ACK/NACK that.

Otherwise I think this is a good refresh. +1.



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-portage-dev] [PATCH] Add QA check for unresolved soname dependencies (bug 704320)

[Michael 'veremitz' Everitt]

On 06/01/20 06:38, Zac Medico wrote:
> Example output for maven-bin-3.6.2 (bug 704618):
>
>  * QA Notice: Unresolved soname dependencies:
>  *
>  */usr/share/maven-bin-3.6/lib/jansi-native/freebsd32/libjansi.so: 
> libc.so.7 libutil.so.9
>  */usr/share/maven-bin-3.6/lib/jansi-native/freebsd64/libjansi.so: 
> libc.so.7 libutil.so.9
>  *
>
> This warning comes up when a library or executable has one or
> more soname dependencies (found in its NEEDED.ELF.2 metadata)
> that could not be resolved by usual means. If you run ldd
> on files like these then it will report a "not found" error
> for each unresolved soname dependency. In order to correct
> problems with soname dependency resolution, use one or more
> of the approaches described in the following sections.
>
> Content of the NEEDED.ELF.2 metadata file may be useful
> for debugging purposes. Find the NEEDED.ELF.2 file in the
> ${D}/../build-info/ directory after the ebuild src_install
> phase completes, or in the /var/db/pkg/*/*/ directory for an
> installed package. Each line of the NEEDED.ELF.2 file contains
> semicolon separated values for a single ELF file. The soname
> dependencies are found in the DT_NEEDED column:
>
>  E_MACHINE;path;DT_SONAME;DT_RUNPATH;DT_NEEDED;multilib category
>
> External dependencies
>
> For packages that install pre-built binaries, it may be possible
> to resolve soname dependencies simply by adding dependencies
> for one or more other packages that are known to provide the
> needed sonames.
>
> Removal of unecessary files
>
> For packages that install pre-built binaries, it may be possible
> to resolve soname dependencies simply by removing unnecessary
> files which have unresolved soname dependencies. For example,
> some pre-built binary packages include binaries intended for
> irrelevant architectures or operating systems, and these files
> can simply be removed because they are unnecessary.
>
> Addition of DT_RUNPATH entries
>
> If the relevant dependencies are installed in a location that
> is not included in the dynamic linker search path, then it's
> necessary for files to include a DT_RUNPATH entry which refers
> to the appropriate directory. The special $ORIGIN value can
> be used to create a relative path reference in DT_RUNPATH,
> where $ORIGIN is a placeholder for the directory where the
> file having the DT_RUNPATH entry is located.
>
> For pre-built binaries, it may be necessary to fix up
> DT_RUNPATH using patchelf --set-rpath. For example, use
> patchelf --set-rpath '$ORIGIN' if a given binary should link
> to libraries found in the same directory as the binary itself,
> or use patchelf --set-rpath '$ORIGIN/libs' if a given binary
> should link to libraries found in a subdirectory named libs
> found in the same directory as the binary itself.
>
> For binaries built from source, a flag like
> -Wl,-rpath,/path/of/directory/containing/libs will create
> binaries with the desired DT_RUNPATH entry.
>
> Bug: https://bugs.gentoo.org/704320
> Signed-off-by: Zac Medico 
> ---


Looks awesome!

One comment: would it be helpful for developers/maintainers to have some
indication of what's going on here, and what attempts portage is making in
this regard? Something like a 'verbose' option that would spit out some
debugging info so that /if/ something goes wrong, or the outcome wasn't
what was intended, then this can be picked up and corrected somehow else?

I think automation is great for picking these issues up, and shining a
light on where/why something goes wrong, but sometimes it needs a few
minutes of human intervention to achieve the sane outcome in many
circumstances.



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-portage-dev] [PATCH gentoolkit 1/2] eclean: Fix typos

[Michael 'veremitz' Everitt]

On 02/01/20 18:57, Matt Turner wrote:
> Signed-off-by: Matt Turner 
> ---
>  pym/gentoolkit/eclean/cli.py| 4 ++--
>  pym/gentoolkit/eclean/search.py | 2 +-
>  2 files changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/pym/gentoolkit/eclean/cli.py b/pym/gentoolkit/eclean/cli.py
> index 1d2f52b..1a99b3e 100644
> --- a/pym/gentoolkit/eclean/cli.py
> +++ b/pym/gentoolkit/eclean/cli.py
> @@ -304,7 +304,7 @@ def parseArgs(options={}):
>   options['size-limit'] = 0
>   options['verbose'] = False
>   options['ignore-failure'] = False
> - # if called by a well-named symlink, set the acction accordingly:
> + # if called by a well-named symlink, set the action accordingly:
>   action = None
>   # temp print line to ensure it is the svn/branch code running, etc..
>   #print(  "## svn/branch/gentoolkit_eclean ### ==> ", 
> os.path.basename(sys.argv[0]))
> @@ -400,7 +400,7 @@ def doAction(action,options,exclude={}, output=None):
>   )
>  
>   # initialize our cleaner
> - cleaner = CleanUp( output.progress_controller)
> + cleaner = CleanUp(output.progress_controller)
>  
>   # actually clean files if something was found
>   if clean_me:
> diff --git a/pym/gentoolkit/eclean/search.py b/pym/gentoolkit/eclean/search.py
> index ce455a3..58bd97e 100644
> --- a/pym/gentoolkit/eclean/search.py
> +++ b/pym/gentoolkit/eclean/search.py
> @@ -574,7 +574,7 @@ def findPackages(
>   del clean_me[cpv]
>   continue
>   if portage.cpv_getkey(cpv) in cp_all and 
> port_dbapi.cpv_exists(cpv):
> - # exlusion because of --package-names
> + # exclusion because of --package-names
>   del clean_me[cpv]
>  
>   # the getname method correctly supports FEATURES=binpkg-multi-instance,
LGTM fwiw.



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-portage-dev] Build path abi missing in qemu-user

[Michael 'veremitz' Everitt]

On 18/12/19 17:36, Joakim Tjernlund wrote:
> On Wed, 2019-12-18 at 11:22 -0500, Mike Gilbert wrote:
>> CAUTION: This email originated from outside of the organization. Do not 
>> click links or open attachments unless you recognize the sender and know the 
>> content is safe.
>>
>>
>> On Thu, Dec 12, 2019 at 10:18 AM Joakim Tjernlund
>>  wrote:
>>> When build in a qemu-user chroot I get odd build paths:
>>>   /var/tmp/portage/dev-libs/libxml2-2.9.9-r1/work/libxml2-2.9.9-.ppc
>>>
>>> This path is missing the abi_ppc_32 like so:
>>>   
>>> /var/tmp/portage/dev-libs/libxml2-2.9.9-r1/work/libxml2-2.9.9-abi_ppc_32.ppc
>>>
>>> I struggle to find out why this happens, any ideas?
>> I would guess you are using a profile that doesn't define the ABI_PPC
>> USE_EXPAND variable properly.
>>
> # portageq envvar USE_EXPAND
> ABI_MIPS ABI_PPC ABI_S390 ABI_X86 ALSA_CARDS APACHE2_MODULES APACHE2_MPMS 
> CALLIGRA_FEATURES CAMERAS COLLECTD_PLUGINS CPU_FLAGS_X86 CROSSCOMPILE_OPTS 
> CURL_SSL DRACUT_MODULES
> DVB_CARDS ELIBC ENLIGHTENMENT_MODULES FCDSL_CARDS FFTOOLS FOO2ZJS_DEVICES 
> FRITZCAPI_CARDS GPSD_PROTOCOLS GRUB_PLATFORMS INPUT_DEVICES KERNEL 
> LCD_DEVICES LIBREOFFICE_EXTENSIONS
> LINGUAS LIRC_DEVICES MONKEYD_PLUGINS NETBEANS_MODULES NGINX_MODULES_HTTP 
> NGINX_MODULES_MAIL OFED_DRIVERS OFFICE_IMPLEMENTATION OPENMPI_FABRICS 
> OPENMPI_OFED_FEATURES OPENMPI_RM
> PHP_TARGETS PYTHON_SINGLE_TARGET PYTHON_TARGETS QEMU_SOFTMMU_TARGETS 
> QEMU_USER_TARGETS ROS_MESSAGES RUBY_TARGETS SANE_BACKENDS USERLAND 
> UWSGI_PLUGINS VIDEO_CARDS
> VOICEMAIL_STORAGE XFCE_PLUGINS XTABLES_ADDONS
> cu3-jocke fs # portageq envvar ABI_PPC
> 32
> cu3-jocke fs # eselect profile list
> Available profile symlink targets:
>   [1]   default/linux/powerpc/ppc32/13.0
>   [2]   default/linux/powerpc/ppc32/13.0/desktop
>   [3]   default/linux/powerpc/ppc32/13.0/desktop/gnome
>   [4]   default/linux/powerpc/ppc32/13.0/desktop/gnome/systemd
>   [5]   default/linux/powerpc/ppc32/13.0/desktop/kde
>   [6]   default/linux/powerpc/ppc32/13.0/desktop/kde/systemd
>   [7]   default/linux/powerpc/ppc32/13.0/developer
>   [8]   default/linux/powerpc/ppc64/13.0/32bit-userland
>   [9]   default/linux/powerpc/ppc64/13.0/32bit-userland/desktop
>   [10]  default/linux/powerpc/ppc64/13.0/32bit-userland/desktop/gnome
>   [11]  default/linux/powerpc/ppc64/13.0/32bit-userland/desktop/gnome/systemd
>   [12]  default/linux/powerpc/ppc64/13.0/32bit-userland/desktop/kde
>   [13]  default/linux/powerpc/ppc64/13.0/32bit-userland/desktop/kde/systemd
>   [14]  default/linux/powerpc/ppc64/13.0/32bit-userland/developer
>   [15]  hardened/linux/powerpc/ppc32
>   [16]  hardened/linux/powerpc/ppc64/32bit-userland
>   [17]  hardened/linux/musl/ppc
>   [18]  default/linux/uclibc/ppc
>   [19]  hardened/linux/uclibc/ppc
>   [20]  tmv3-target-overlay:cusfpv3 *
>
> cusfpv3 inherits default/linux/powerpc/ppc32/13.0
>
>  Jocke
>
Haven't 13.0 profiles been deprecated? (and/or deleted by now ...)



signature.asc
Description: OpenPGP digital signature

[gentoo-portage-dev] RFC: [QA] notice with 'failed' seds [was PATCH: eapply drop -s option]

[Michael 'veremitz' Everitt]

On 13/12/19 20:36, Michał Górny wrote [excerpted]:
>
> Is this really an argument for or *against* it?  Developers are entirely
> capable of keeping seds that do nothing for years, as well as patches
> that -- while apparently applying correctly -- are entirely meaningless.


I think there is some merit in some kind of feedback when sed's are doing
nothing, although how feasible it is to generate any useful feedback I
can't say. I wouldn't say it needs to explicitly fail or make lots of
noise, just an info message that could prompt some further investigation.



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-portage-dev] [PATCH] eapply: Drop -s option for patch.

[Michael 'veremitz' Everitt]

On 13/12/19 21:59, Michał Górny wrote:
> On Fri, 2019-12-13 at 21:56 +0000, Michael 'veremitz' Everitt wrote:
>> On 13/12/19 21:42, Michał Górny wrote:
>>> On Fri, 2019-12-13 at 16:37 -0500, Mike Gilbert wrote:
>>>> On Fri, Dec 13, 2019 at 3:36 PM Michał Górny  wrote:
>>>>> Just like 'many of the proposals lately', developers are going to be
>>>>> the ones disabling it (because they don't care), and users will be the
>>>>> ones enabling it (because they do care), just to learn that developers
>>>>> don't care and go complaining to the mailing lists that users dare
>>>>> report issues they don't care about.
>>>> I care if the patch is actually broken, which the warning doesn't
>>>> really tell me. It's just not a very reliable indicator, and will
>>>> produce false-positives frequently.
>>>>
>>> You can also take less context into the patch and use -F0.  Then you'll
>>> have the same effect, no warnings to bother you and no pretending that
>>> the patch applies when it doesn't.
>>>
>> Is there any mileage in having a similar scheme to which we already apply
>> '-p' increments to the -F variable?
>> eg.
>> 1) attempt patch with -F0
>> 2) if (1) fails, attempt with -F2/3 & display 'yellow' warning & QA notice
> That is precisely what my patch does and what people are complaining
> about.
Ah, apologies for the failure to grok.

Tangentially, but also brought up on the thread, I'm actually even
moderately concerned about the ghost seds that may never apply. Topic for
another thread though I feel.
>> 3) if (2) fails, attempt with, say, -F10 & display big fat 'red' warning
>> and QA notice
> That makes no sense as it exceeds context provided in most patches.
>
Fair .. hadn't thought of that - depends very much if you're using unified
diffs, which I believe we largely are.



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-portage-dev] [PATCH] eapply: Drop -s option for patch.

[Michael 'veremitz' Everitt]

On 13/12/19 21:42, Michał Górny wrote:
> On Fri, 2019-12-13 at 16:37 -0500, Mike Gilbert wrote:
>> On Fri, Dec 13, 2019 at 3:36 PM Michał Górny  wrote:
>>> Just like 'many of the proposals lately', developers are going to be
>>> the ones disabling it (because they don't care), and users will be the
>>> ones enabling it (because they do care), just to learn that developers
>>> don't care and go complaining to the mailing lists that users dare
>>> report issues they don't care about.
>> I care if the patch is actually broken, which the warning doesn't
>> really tell me. It's just not a very reliable indicator, and will
>> produce false-positives frequently.
>>
> You can also take less context into the patch and use -F0.  Then you'll
> have the same effect, no warnings to bother you and no pretending that
> the patch applies when it doesn't.
>
Is there any mileage in having a similar scheme to which we already apply
'-p' increments to the -F variable?
eg.
1) attempt patch with -F0
2) if (1) fails, attempt with -F2/3 & display 'yellow' warning & QA notice
3) if (2) fails, attempt with, say, -F10 & display big fat 'red' warning
and QA notice
4) Fail and abort

Regards,
veremitz/Michael.



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-portage-dev] [PATCH] eapply: Drop -s option for patch.

[Michael Orlitzky]

On 12/13/19 9:28 AM, Fabian Groffen wrote:
> 
> We are providing those patches, maybe.  In reality very often the
> patches originate from somewhere else though.  And you don't want to
> have to respin all of those just because.  At least that's what I feel.
> 

Just because... the context changed? A new "!" in a line of context can
be the difference between letting someone log in with the right password
and letting them log in with the wrong password. You should at least
have to stop and verify that the patch does what you think it does when
it "gains" fuzz. And at that point, git diff will give you a clean
version of it.

Re: [gentoo-portage-dev] [PATCH] eapply: Drop -s option for patch.

[Michael Orlitzky]

On 12/12/19 3:15 PM, Ulrich Mueller wrote:
> 
> It was also suggested that we add -F0 in EAPI 8, but that would break
> the build in those cases that are producing extra output now. I don't
> think that would be preferable.

It would only break the build for the maintainer, who would then
presumably fix the patch.

Re: [gentoo-portage-dev] [PATCH v2] eapply: Output verbosely only if patch fails to apply with -F0

[Michael Orlitzky]

On 11/27/19 2:17 PM, Michał Górny wrote:
> 
> To achieve that, attempt to apply each patch with -F0 --dry-run first.
> If this succeeds, just silently apply the patch for real.  If it
> doesn't, output an explicit eqawarn that the patch does not apply
> cleanly and retry with the default fuzz factor and verbose output.

This now disagrees with the PMS algorithm, doesn't it? Not that the
change isn't sensible otherwise.

Re: [gentoo-portage-dev] Re: seed emerge with old /var/db/pkg ?

[Michael 'veremitz' Everitt]

On 13/11/19 19:07, Zac Medico wrote:
> On 11/8/19 9:09 AM, Joakim Tjernlund wrote:
>> On Fri, 2019-11-08 at 01:57 +0100, Joakim Tjernlund wrote:
>>> I am looking for a way to seed emerge with an older pkg db so emerge can 
>>> calculate
>>> which packages needs to be rebuild/upgraded in order to get to the same 
>>> state as the system pkg db,
>>> Is that possible?
>>>
>>> Also, is there some tool that allows med to copy just files needed for a 
>>> profile?
>>> Something like "cp" /etc/portage/make.profile /my/destination
>>>
>> portage-utils has variables Q_VDB and Q_EDB which allows qmerge to use 
>> different VDBs/EDBs
>> Does portage have something similar?
> No, nothing like either of those things.
>
> However, if you want to operate on an old system then the usual
> recommendation is to unpack a stage3 and bind mount the old system root
> into the stage3 so that you can chroot into the stage3 and run something
> like this:
>
> emerge --root /mnt/old-system portage
In the interests of clarity, what does this achieve, and what would the
next couple of steps look like?
(actual commands not necessary, pseudo-code WFM)



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-portage-dev] [PATCH] install-qa-check.d: remove check that bans libtool files and static libs from /

[Michael 'veremitz' Everitt]

On 03/11/19 21:37, Michał Górny wrote:
> On Sun, 2019-11-03 at 15:26 -0600, William Hubbs wrote:
>>
>> You being a qa member doesn't have a lot to do with this mgorny. you
>> know there was no official policy when I posted this, and as far as I
>> know there is not one now.
>>
> That is a really poor argument.  Something that's respected for 10+
> years and reported as QA violation is a standing policy as far as I'm
> concerned.  Just because it isn't backed by a formally stamped policy
> (at least as far as we know -- maybe it was actually stamped somewhere
> in the past?) doesn't mean you it's fine for one person to change it ad-
> hoc because it stands in his way.
>
> I should point that I'm very concerned that you're pushing this forward
> even though:
>
> 1) I've objected to the change itself,
>
> 2) I've pointed out that it's been sent to the wrong mailing list,
> and that this explicitly prevents a number of developers from even
> knowing that this is happening,
>
> 3) removing it provides a way for regressions that can have major impact
> on users and that involve much effort in reverting that.
>
> So if I send a revert patch afterwards, and you object, should the patch
> be accepted because only one person objected?
>
Children, please take this off-list ...



signature.asc
Description: OpenPGP digital signature

[gentoo-portage-dev] Re: [gentoo-dev] [RFC] Disable --autounmask auto-unmasking keywords / masks by default

[Michael Everitt]

On 10/10/19 03:57, Kent Fredric wrote:
> On Wed, 9 Oct 2019 19:45:34 -0700
> Zac Medico  wrote:
>
>> I'd prefer to disable --autounmask by default and include warnings about
>> harmful behavior in the documentation.
> I think autounmasks behaviour with regards to USE flags is useful,
> (heh), its just everything else it does tends to violate stability
> assumptions.
>
> For instance, I can see why automatically escalating "arch" to "~arch"
> may be useful in some cases, but its a very dangerous default,
> *especially* for perl.
Is there any mileage in something as ridiculous as AUTOUNMASK_EXCLUDES ?



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-portage-dev] Constraint-Based Dependency Solver: initial results

[Michael Orlitzky]

On 8/30/19 10:34 AM, michael.lienha...@laposte.net wrote:
> 
> All comments/suggestions are welcomed.
> 

Since no one else has said it yet (?), I think this approach is really
cool and I'm glad someone is working on it.

Modeling difficult computations as abstract optimization problems is a
"hobby" of mine, and I think it will pay off if we can abstract away a
big ugly part of the PM and try to swap it out for something more
principled.

[gentoo-portage-dev] Constraint-Based Dependency Solver: initial results

[michael . lienhardt]

Dear all,

It's possible that the goal of my current work and its possible benefit for the 
gentoo community are not very clear.
In my experience, emerge is a very good tool to install new packages whose use 
flags have already been configured.
However, when the packages are not correctly configured, emerge can guess, 
without guarantee, some use flags to select/unselect; and unmerging a package 
seems to be the user's entire responsability.
My work has several goals:
 - provide a correct and complete implementation of a dependency solver for 
portage that can help debug emerge
 - since the solver is correct and complete, it would provide a "safe" 
implementation of unmerge, where missing dependencies would be satisfied by the 
installation of new packages
 - provide a rather small code base that is easy to debug and on which it is 
easy to prototype some tools, like REQUIRED_USE checks, or interactive use flag 
configuration
 - be usable, both in term of memory usage and computation time.

I finished implementing the solver and did some preliminary benchs (1000 tests 
where I checked if a random set of packages -- different for each test -- could 
be installed), including comparison with emerge in "search" modality, i.e., 
with the options --autounmask y --autounmask-continue y --autounmask-backtrack y
Since this is preliminary, I wanted to talk to you about it but I don't have 
every bugs identified yet.
In average, my solver is a bit less than 10 times slower than emerge (not very 
good, but not bad as it is still far better than a brute force approach, which 
is more than 100 times slower and takes 3Gb of memory).
It is important to note that my solver is not suited for end user usage yet 
(the answer it gives is correct, but random -- it includes useless packages, 
useless package removal and old versions), but it is the first tool that I know 
of that can correctly and completely (modulo bugs ^^) check emerge's behavior.

A first result: in more than 25% of the tests that can be installed, emerge 
fails.
Most of these failures come from the fact that even in "search" modality, 
emerge stops when it sees a REQUIRED_USE that is not correctly configured (I'll 
post a bug report about it and about the SLOT heuristics in a few days).
I still need to look at the other failures to see what caused them.

Additionally, it seems that when I do "emerge package1 package2", emerge first 
solves the dependencies of package1, and then of package2.
It seems that when resolving the depenendencies of package2, emerge can end up 
with a conflict between the choices it made for solving the dependencies of 
package1 and the requirements of package2.
I say "seems" because my tests were automatically done with a rather long list 
of packages (so other reasons for the observed failures are possible).
I'll try to pin down the actual emerge behavior and possibly file a bug report.

I am currently porting the tests on docker (to have a safe testing environment) 
and once that's done, I will be able to give actual bug reports.
In the future, I have the following plan:
 - cleanup the output of the solver, so it wouldn't be random and be usable 
instead (this is "just" a technical and boring algorithm to implement, but time 
consuming)
 - install the configuration computed by my solver (I am still unsure that 
emerge --nodeps would be flexible enough, and maybe I will have to implement my 
own planner)
 - find a correct abstraction of packages, similar to the SLOT heuristics ( 
https://gitweb.gentoo.org/proj/portage.git/commit/?id=a9064d08ef4c92a5d0d1bfb3dc8a01b7850812b0
 ), to improve efficiency
 - design an interactive package configuration algorithm + UI that would happen 
during the dependency solving process and really help the user configuring what 
he wants and let the solver do the rest
 - start reading portage's bug tracker and continue reading its code
 - extend pdepa with other kind of relevant analysis

All comments/suggestions are welcomed.

Best,
Michael

Re: [gentoo-portage-dev] [RFC] Adding extra vars to md5-cache, for QA purposes

[Michael Orlitzky]

On 7/25/19 4:29 PM, Michał Górny wrote:
>>
>> * In the md5-cache entry, maybe use a common prefix like EXT_ for the
>> extra keys in order to distinguish them from normal keys.
> 
> Yeah, I was thinking of something like '__ext_foo', or '__ext[foo]'.
> 

What are the pros/cons of this? The names refer to global variables, so
they should already be safely namespaced, right?.

There is a possibility that an eclass variable name (e.g. PATCHES) could
become standardized at a later date. If that happens, we could wind up
with both FOO and __ext_FOO in the cache, and tools would have to figure
out what to do with zero, one, or both present. (This has happened in
email/web protocols when an X-Foo header was standardized.) It's not the
end of the world, but someone would have to stop and think about it.

Finally, just having the name be predictable so that I can grep '^FOO='
without having to care where it came from is nice.

OTOH for testing, and for figuring out why these weird variables are
showing up in my cache, the prefix would help.

Re : Re: [gentoo-portage-dev] Constraint-Based Dependency Solver for Portage: again

[michael . lienhardt]

- Zac Medico  a écrit :
> It's capable of considering older versions, but maybe there's some
> deficiency in the algorithm. We should analyze a specific example in
> order to understand the behavior.
> 
> Maybe you're referring to this code which forces the highest version in
> the event of a conflict:
> 
> https://gitweb.gentoo.org/proj/portage.git/commit/?id=a9064d08ef4c92a5d0d1bfb3dc8a01b7850812b0
> 

Yes, this seems to be the cause of the problem, thank you.

For testing, I created two ebuilds (and tested with "emerge -pv 
--autounmask-backtrack y net-misc/pdepa"):

## net-misc/pdepa-1.0
EAPI=6
KEYWORDS="amd64"
SLOT="1"
IUSE="feature"
REQUIRED_USE=""
DEPEND=""

## net-misc/pdepa-2.0
EAPI=6
KEYWORDS="amd64"
SLOT="1"
IUSE="feature"
REQUIRED_USE="^^ ( feature )" # feature is not set => not installable
DEPEND=""

Like you said, due to SLOT conflict, only net-misc/pdepa-2.0 is tried, and 
emerge fails.
Changing the SLOT of net-misc/pdepa-2.0 to 2 "solves the issue": emerge 
succeeds and propose to install net-misc/pdepa-1.0

However, this solution is only local: if in net-misc/pdepa-2.0 I replace
REQUIRED_USE="" # now the package has no configuration problem
DEPEND="!virtual/libc" # but it conflicts with an important library

then emerge fails again, saying that virtual/libc blocks net-misc/pdepa-2.0


I don't know how many actual packages cannot be installed due to this 
optimization.
I noticed this behavior in a previous version of the portage tree, when trying 
to install sys-auth/polkit without configuring it:
 - old version: REQUIRED_USE="??( systemd consolekit )"
 - more recent version REQUIRED_USE="^^ ( consolekit, elogind systemd )"
In practice however, this was not a problem, as some dependencies of polkit 
deep in the tree did require one of ( systemd consolekit ) to be set.

If you want, I can implement a test that check if this optimization is a 
problem in practice.
Checking the issue shouldn't be too difficult (I think I just need to check 
that all REQUIRED_USE and *DEPEND are equivalent for a SLOT).

Best,
Michael

[gentoo-portage-dev] Constraint-Based Dependency Solver for Portage: again

[michael . lienhardt]

Dear all,

A few months ago, I got back to my constraint-based dependency solver for 
portage, that I had to leave for a while.
Thanks to Zac Medico, it is now based on portage itself to query the portage 
tree, and so the code is far simpler (and far less buggy).
It is on github: https://github.com/gzoumix/pdepa

I still have some work to do on the implementation, and with some colleagues, 
we are planning to publish it in a conference, with the related theory.
However, to have relevant information to publish, I need your help, if you 
could answer some questions that will come up during testing.
For instance, in all my tests, emerge (during its dependency resolution) always 
replaces atoms with the latest version of the pc that matches it, even with all 
possible backtracking options being selected
 (I noticed this behavior because emerge failed installing a package such that 
the latest corresponding cpv could be installed, while the previous version 
could be).
Is it really the default behavior of emerge, and if yes, is there a way to make 
emerge consider all matching cpv for an atom?

Thank you!
Michael

Re: [gentoo-portage-dev] [PATCH] f{owners,perms}: Warn when using relative path

[Michael Orlitzky]

On 09/17/2018 02:52 AM, Michał Górny wrote:
> 
> --- a/bin/ebuild-helpers/fowners
> +++ b/bin/ebuild-helpers/fowners
> ...
> + eqawarn "This is unsupported. Please use 'chmod' when you need 
> to work on files"

This one should be 'chown' instead of 'chmod'.

(Calling chown on the live filesystem is often also a dangerous mistake,
but this probably isn't the place to fuss about it.)

Re: [gentoo-portage-dev] [PATCH v2] install-qa-checks.d: Add a check for Gentoo path policies (FHS-y)

[Michael Orlitzky]

On 09/04/2018 01:53 PM, Michał Górny wrote:
> + # TODO: do we need it? gconf installs empty dir there but that's
> + # all
> + root

FWIW, we should not allow this.

Re: [gentoo-portage-dev] [PATCH v3 1/2] bin/install-qa-check.d: add new 90bad-bin-owner QA check.

[Michael Orlitzky]

On 08/07/2018 01:34 PM, Zac Medico wrote:
> 
> Why not use ${ED%/} instead of ${D%/} here, so that the output is the
> same regardless of ${EPREFIX}?
> 

We want to show where the executable was actually installed, and
generally that includes EPREFIX. For example, I'd want to see

  /var/tmp/whatever/.../root/prefix/usr/bin/foo

reported as

  /root/prefix/usr/bin/foo

rather than

  /usr/bin/foo

Of course, these checks are now skipped on prefix systems anyway, so
it's a bit of a moot point. But I think it's more future-proof to strip
only the $D.

[gentoo-portage-dev] [PATCH v3 2/2] bin/install-qa-check.d: add new 90bad-bin-group-write QA check.

[Michael Orlitzky]

System executables that are writable by a non-root user pose a
security risk. Anyone who can write to an executable can change its
behavior. If that executable is later run with elevated privileges
(say, by root, when the machine starts), then the non-root user can
escalate his own privileges to those of the person running the
modified executable.

The 90bad-bin-owner check already addresses one cause for a non-root
user to be able to modify an executable: because he owns it. This
commit adds another check, to ensure that no non-root *groups* have
write access to any system executables. On a "normal" system, all
system executables should be writable only by the super-user's group,
if any. To avoid false-positives, non-"normal" systems (like prefix)
are skipped.

Closes: https://bugs.gentoo.org/629398
---
 bin/install-qa-check.d/90bad-bin-group-write | 55 
 1 file changed, 55 insertions(+)
 create mode 100644 bin/install-qa-check.d/90bad-bin-group-write

diff --git a/bin/install-qa-check.d/90bad-bin-group-write 
b/bin/install-qa-check.d/90bad-bin-group-write
new file mode 100644
index 0..786dde712
--- /dev/null
+++ b/bin/install-qa-check.d/90bad-bin-group-write
@@ -0,0 +1,55 @@
+# Copyright 1999-2018 Gentoo Foundation
+# Distributed under the terms of the GNU General Public License v2
+
+bad_bin_group_write_check() {
+   # Warn about globally-installed executables (in /bin, /usr/bin, /sbin,
+   # /usr/sbin, or /opt/bin) that are group-writable by a nonzero GID.
+
+   # This check doesn't work on non-root prefix installations at
+   # the moment, because every executable therein is owned by a
+   # nonzero GID.
+   [[ "${EUID}" -ne "0" || "${PORTAGE_INST_UID}" -ne "0" ]] && return
+
+   local d f found=()
+
+   for d in "${ED%/}/opt/bin" "${ED%/}/bin"  "${ED%/}/usr/bin" \
+  "${ED%/}/sbin" 
"${ED%/}/usr/sbin"; do
+   [[ -d "${d}" ]] || continue
+
+   # Read the results of the "find" command into the "found" array.
+   #
+   # Use -L to catch symlinks whose targets are vulnerable,
+   # even though it won't catch ABSOLUTE symlinks until the package
+   # is RE-installed (the first time around, the target won't 
exist).
+   #
+   # We match the GID and not the name "root" here because (for
+   # example) on FreeBSD, the superuser group is "wheel".
+   #
+   # We don't make an exception for setguid executables here, 
because
+   # a group-writable setguid executable is likely a mistake. By
+   # altering the contents of the executable, a member of the group
+   # can allow everyone (i.e. the people running it) to obtain the
+   # full privileges available to that group. While only existing
+   # group members can make that choice, it's a decision usually
+   # limited to the system administrator.
+   while read -r -d '' f; do
+   found+=( "${f}" )
+   done < <(find -L "${d}"   \
+   -maxdepth 1   \
+   -type f   \
+   -perm /g+w\
+   ! -gid 0  \
+   -print0)
+   done
+
+   if [[ ${found[@]} ]]; then
+   eqawarn "system executables group-writable by nonzero gid:"
+   for f in "${found[@]}"; do
+   # Strip off the leading destdir before outputting the 
path.
+   eqawarn "  ${f#${D%/}}"
+   done
+   fi
+}
+
+bad_bin_group_write_check
+:
-- 
2.16.4

[gentoo-portage-dev] [PATCH v3 1/2] bin/install-qa-check.d: add new 90bad-bin-owner QA check.

[Michael Orlitzky]

System executables that are not owned by root pose a security
risk. The owner of the executable is free to modify it at any time;
so, for example, he can change a daemon's behavior to make it
malicious before the next time the service is started (usually by
root).

On a "normal" system, the superuser should own every system executable
(even setuid ones, for security reasons). This commit adds a new
install-time check that reports any such binaries with a QA
warning. To avoid false positives, non-"normal" systems (like prefix)
are skipped at the moment.

Bug: https://bugs.gentoo.org/629398
---
 bin/install-qa-check.d/90bad-bin-owner | 48 ++
 1 file changed, 48 insertions(+)
 create mode 100644 bin/install-qa-check.d/90bad-bin-owner

diff --git a/bin/install-qa-check.d/90bad-bin-owner 
b/bin/install-qa-check.d/90bad-bin-owner
new file mode 100644
index 0..c3ee30746
--- /dev/null
+++ b/bin/install-qa-check.d/90bad-bin-owner
@@ -0,0 +1,48 @@
+# Copyright 1999-2018 Gentoo Foundation
+# Distributed under the terms of the GNU General Public License v2
+
+bad_bin_owner_check() {
+   # Warn about globally-installed executables (in /bin, /usr/bin, /sbin,
+   # /usr/sbin, or /opt/bin) that are owned by a nonzero UID.
+
+   # This check doesn't work on non-root prefix installations at
+   # the moment, because every executable therein is owned by a
+   # nonzero UID.
+   [[ "${EUID}" -ne "0" || "${PORTAGE_INST_UID}" -ne "0" ]] && return
+
+   local d f found=()
+
+   for d in "${ED%/}/opt/bin" "${ED%/}/bin"  "${ED%/}/usr/bin" \
+  "${ED%/}/sbin" 
"${ED%/}/usr/sbin"; do
+   [[ -d "${d}" ]] || continue
+
+   # Read the results of the "find" command into the "found" bash 
array.
+   #
+   # Use -L to catch symlinks whose targets are owned by a 
non-root user,
+   # even though it won't catch ABSOLUTE symlinks until the package
+   # is RE-installed (the first time around, the target won't 
exist).
+   #
+   # We do want to list non-superuser setuid executables, because
+   # they can be exploited. The owner can simply wipe the setuid
+   # bit, and then alter the contents of the file. The superuser
+   # will then have a time bomb in his $PATH.
+   while read -r -d '' f; do
+   found+=( "${f}" )
+   done < <(find -L "${d}"   \
+   -maxdepth 1   \
+   -type f   \
+   ! -uid 0  \
+   -print0)
+   done
+
+   if [[ ${found[@]} ]]; then
+   eqawarn "system executables owned by nonzero uid:"
+   for f in "${found[@]}"; do
+   # Strip off the leading destdir before outputting the 
path.
+   eqawarn "  ${f#${D%/}}"
+   done
+   fi
+}
+
+bad_bin_owner_check
+:
-- 
2.16.4

[gentoo-portage-dev] [PATCH v3 0/2] Two insecure ownership and group-writability QA checks.​

[Michael Orlitzky]

Changes in v3:

  * Undo the setguid exception from v2, and add a comment explaining why.
  * Add line breaks for readability in two comments.
  * Try to put back the leading "/" in the output list.
  * Remove a superfluous comment mentioning the "prefix."

Michael Orlitzky (2):
  bin/install-qa-check.d: add new 90bad-bin-owner QA check.
  bin/install-qa-check.d: add new 90bad-bin-group-write QA check.

 bin/install-qa-check.d/90bad-bin-group-write | 55 
 bin/install-qa-check.d/90bad-bin-owner   | 48 
 2 files changed, 103 insertions(+)
 create mode 100644 bin/install-qa-check.d/90bad-bin-group-write
 create mode 100644 bin/install-qa-check.d/90bad-bin-owner

-- 
2.16.4

Re: [gentoo-portage-dev] [PATCH 1/2] bin/install-qa-check.d: add new 90bad-bin-owner QA check.

[Michael Orlitzky]

On 07/29/2018 09:16 PM, Ulrich Mueller wrote:
> 
> Staying with the man:man example, how would anybody become the "man"
> user, in the first place? That user has /bin/false as a shell and no
> valid password.

One way would be to exploit a process that's running as the "man" user.
Ostensibly such a thing is possible; otherwise we wouldn't be dropping
privileges in the first place.

The shell/password settings for that user are also in no way guaranteed.

But on principle: who cares, non-root users shouldn't be able to gain
root =)


> Setgid executables shouldn't be group writable

Why not? I'm happy to pull that part back out.

[gentoo-portage-dev] [PATCH v2 0/2] Two insecure ownership and group-writability QA checks.

[Michael Orlitzky]

Changes in v2:

  * Also check executables in /opt/bin (mgorny).
  * Don't report group-writable executables that are setgid (ulm).
  * Add a comment on why we don't do the same for setuid.
  * Wrap long lines with backslashes.
  * Fix nesting of output loop; output should happen after all checks.

Michael Orlitzky (2):
  bin/install-qa-check.d: add new 90bad-bin-owner QA check.
  bin/install-qa-check.d: add new 90bad-bin-group-write QA check.

 bin/install-qa-check.d/90bad-bin-group-write | 49 
 bin/install-qa-check.d/90bad-bin-owner   | 47 ++
 2 files changed, 96 insertions(+)
 create mode 100644 bin/install-qa-check.d/90bad-bin-group-write
 create mode 100644 bin/install-qa-check.d/90bad-bin-owner

-- 
2.16.4

[gentoo-portage-dev] [PATCH 1/2] bin/install-qa-check.d: add new 90bad-bin-owner QA check.

[Michael Orlitzky]

System executables that are not owned by root pose a security
risk. The owner of the executable is free to modify it at any time;
so, for example, he can change a daemon's behavior to make it
malicious before the next time the service is started (usually by
root).

On a "normal" system, there is no good reason why the superuser should
not own every system executable. This commit adds a new install-time
check that reports any such binaries with a QA warning. To avoid false
positives, non-"normal" systems (like prefix) are skipped at the moment.

Bug: https://bugs.gentoo.org/629398
---
 bin/install-qa-check.d/90bad-bin-owner | 47 ++
 1 file changed, 47 insertions(+)
 create mode 100644 bin/install-qa-check.d/90bad-bin-owner

diff --git a/bin/install-qa-check.d/90bad-bin-owner 
b/bin/install-qa-check.d/90bad-bin-owner
new file mode 100644
index 0..748e1dc99
--- /dev/null
+++ b/bin/install-qa-check.d/90bad-bin-owner
@@ -0,0 +1,47 @@
+# Copyright 1999-2018 Gentoo Foundation
+# Distributed under the terms of the GNU General Public License v2
+
+bad_bin_owner_check() {
+   # Warn about globally-installed executables (in /bin, /usr/bin, /sbin,
+   # /usr/sbin, or /opt/bin) that are owned by a nonzero UID.
+
+   # This check doesn't work on non-root prefix installations at
+   # the moment, because every executable therein is owned by a
+   # nonzero UID.
+   [[ "${EUID}" -ne "0" || "${PORTAGE_INST_UID}" -ne "0" ]] && return
+
+   local d f found=()
+
+   for d in "${ED%/}/opt/bin" "${ED%/}/bin"  "${ED%/}/usr/bin" \
+  "${ED%/}/sbin" 
"${ED%/}/usr/sbin"; do
+   [[ -d "${d}" ]] || continue
+
+   # Read the results of the "find" command into the "found" bash 
array.
+   # Use -L to catch symlinks whose targets are owned by a 
non-root user,
+   # even though it won't catch ABSOLUTE symlinks until the package
+   # is RE-installed (the first time around, the target won't 
exist).
+   # We do want to list non-superuser setuid executables, because
+   # they can be exploited. The owner can simply wipe the setuid
+   # bit, and then alter the contents of the file. The superuser
+   # will then have a time bomb in his $PATH.
+   while read -r -d '' f; do
+   found+=( "${f}" )
+   done < <(find -L "${d}"   \
+   -maxdepth 1   \
+   -type f   \
+   ! -uid 0  \
+   -print0)
+   done
+
+   if [[ ${found[@]} ]]; then
+   eqawarn "system executables owned by nonzero uid:"
+   for f in "${found[@]}"; do
+   # Strip off the leading destdir before outputting the 
path,
+   # but leave the prefix if there is one.
+   eqawarn "  ${f#${D%/}/}"
+   done
+   fi
+}
+
+bad_bin_owner_check
+:
-- 
2.16.4

[gentoo-portage-dev] [PATCH 2/2] bin/install-qa-check.d: add new 90bad-bin-group-write QA check.

[Michael Orlitzky]

System executables that are writable by a non-root user pose a
security risk. Anyone who can write to an executable can change its
behavior. If that executable is later run with elevated privileges
(say, by root, when the machine starts), then the non-root user can
escalate his own privileges to those of the person running the
modified executable.

The 90bad-bin-owner check already addresses one cause for a non-root
user to be able to modify an executable: because he owns it. This
commit adds another check, to ensure that no non-root *groups* have
write access to any system executables. On a "normal" system, all
system executables should belong to the super-user's group. To avoid
false-positives, non-"normal" systems (like prefix) are skipped.

Closes: https://bugs.gentoo.org/629398
---
 bin/install-qa-check.d/90bad-bin-group-write | 49 
 1 file changed, 49 insertions(+)
 create mode 100644 bin/install-qa-check.d/90bad-bin-group-write

diff --git a/bin/install-qa-check.d/90bad-bin-group-write 
b/bin/install-qa-check.d/90bad-bin-group-write
new file mode 100644
index 0..3c5021e0d
--- /dev/null
+++ b/bin/install-qa-check.d/90bad-bin-group-write
@@ -0,0 +1,49 @@
+# Copyright 1999-2018 Gentoo Foundation
+# Distributed under the terms of the GNU General Public License v2
+
+bad_bin_group_write_check() {
+   # Warn about globally-installed executables (in /bin, /usr/bin, /sbin,
+   # /usr/sbin, or /opt/bin) that are group-writable by a nonzero GID.
+
+   # This check doesn't work on non-root prefix installations at
+   # the moment, because every executable therein is owned by a
+   # nonzero GID.
+   [[ "${EUID}" -ne "0" || "${PORTAGE_INST_UID}" -ne "0" ]] && return
+
+   local d f found=()
+
+   for d in "${ED%/}/opt/bin" "${ED%/}/bin"  "${ED%/}/usr/bin" \
+  "${ED%/}/sbin" 
"${ED%/}/usr/sbin"; do
+   [[ -d "${d}" ]] || continue
+
+   # Read the results of the "find" command into the "found" bash
+   # array. Use -L to catch symlinks whose targets are vulnerable,
+   # even though it won't catch ABSOLUTE symlinks until the package
+   # is RE-installed (the first time around, the target won't 
exist).
+   # We match the GID and not the name "root" here because (for
+   # example) on FreeBSD, the superuser group is "wheel".
+   # We avoid listing setgid executables because -- even though 
they're
+   # super sketchy -- their non-root group is intentional.
+   while read -r -d '' f; do
+   found+=( "${f}" )
+   done < <(find -L "${d}"   \
+   -maxdepth 1   \
+   -type f   \
+   -perm /g+w\
+   ! -gid 0  \
+   ! -perm -2000 \
+   -print0)
+   done
+
+   if [[ ${found[@]} ]]; then
+   eqawarn "system executables group-writable by nonzero gid:"
+   for f in "${found[@]}"; do
+   # Strip off the leading destdir before outputting the 
path,
+   # but leave the prefix if there is one.
+   eqawarn "  ${f#${D%/}/}"
+   done
+   fi
+}
+
+bad_bin_group_write_check
+:
-- 
2.16.4

Re: [gentoo-portage-dev] [PATCH 1/2] bin/install-qa-check.d: add new 90bad-bin-owner QA check.

[Michael Orlitzky]

On 07/29/2018 03:43 PM, Ulrich Mueller wrote:
> 
>> On a "normal" system, there is no good reason why the superuser should
>> not own every system executable. This commit adds a new install-time
>> check that reports any such binaries with a QA warning. To avoid false
>> positives, non-"normal" systems (like prefix) are skipped at the moment.
> 
> Shouldn't this check for setuid binaries like /usr/bin/mandb (which is
> owned by man:man)? I think these are legitimate usage case.
> 

After thinking about this for a while, I think we should ignore setgid
but not setuid executables. The problem with setuid and a non-root owner
is that the owner can always exploit the situation:

Suppose /bin/foo is owned by "foo" and setuid. If root (or any other
privileged user) is about to run /bin/foo, then the "foo" user can
simply strip away the setuid bit and fill /bin/foo with malicious code.

The same situation with setgid is safe because (as far as I know)
members of the group can't strip off the setgid bit.

Re: [gentoo-portage-dev] [PATCH 1/2] bin/install-qa-check.d: add new 90bad-bin-owner QA check.

[Michael Orlitzky]

On 07/29/2018 03:43 PM, Ulrich Mueller wrote:
> 
> Shouldn't this check for setuid binaries like /usr/bin/mandb (which is
> owned by man:man)? I think these are legitimate usage case.
> 

In general, yeah. I think we should be skeptical of setuid/gid
executables, but this isn't the right place to make that stand.

In this specific case, though, I don't see why that program is setuid.
In fact, I'm pretty sure it lets the "man" user gain root privileges.

[gentoo-portage-dev] [PATCH 2/2] bin/install-qa-check.d: add new 90bad-bin-group-write QA check.

[Michael Orlitzky]

System executables that are writable by a non-root user pose a
security risk. Anyone who can write to an executable can change its
behavior. If that executable is later run with elevated privileges
(say, by root, when the machine starts), then the non-root user can
escalate his own privileges to those of the person running the
modified executable.

The 90bad-bin-owner check already addresses one cause for a non-root
user to be able to modify an executable: because he owns it. This
commit adds another check, to ensure that no non-root *groups* have
write access to any system executables. On a "normal" system, all
system executables should belong to the super-user's group. To avoid
false-positives, non-"normal" systems (like prefix) are skipped.

Closes: https://bugs.gentoo.org/629398
---
 bin/install-qa-check.d/90bad-bin-group-write | 40 
 1 file changed, 40 insertions(+)
 create mode 100644 bin/install-qa-check.d/90bad-bin-group-write

diff --git a/bin/install-qa-check.d/90bad-bin-group-write 
b/bin/install-qa-check.d/90bad-bin-group-write
new file mode 100644
index 0..f8a0259e5
--- /dev/null
+++ b/bin/install-qa-check.d/90bad-bin-group-write
@@ -0,0 +1,40 @@
+# Copyright 1999-2018 Gentoo Foundation
+# Distributed under the terms of the GNU General Public License v2
+
+bad_bin_group_write_check() {
+   # Warn about globally-installed executables (in /bin, /usr/bin, /sbin,
+   # or /usr/sbin) that are group-writable by a nonzero GID.
+
+   # This check doesn't work on non-root prefix installations at
+   # the moment, because every executable therein is owned by a
+   # nonzero GID.
+   [[ "${EUID}" -ne "0" || "${PORTAGE_INST_UID}" -ne "0" ]] && return
+
+   local d f found=()
+
+   for d in "${ED%/}/bin" "${ED%/}/usr/bin" "${ED%/}/sbin" 
"${ED%/}/usr/sbin"; do
+   test -d "${d}" || continue
+
+   # Read the results of the "find" command into the "found" bash
+   # array. Use -L to catch symlinks whose targets are vulnerable,
+   # even though it won't catch ABSOLUTE symlinks until the package
+   # is RE-installed (the first time around, the target won't 
exist).
+   # We match the GID and not the name "root" here because (for
+   # example) on FreeBSD, the superuser group is "wheel".
+   while read -r -d '' f; do
+   found+=( "${f}" )
+   done < <(find -L "${d}" -maxdepth 1 -type f -perm /g+w ! -gid 0 
-print0)
+
+   if [[ ${found[@]} ]]; then
+   eqawarn "system executables group-writable by nonzero 
gid:"
+   for f in "${found[@]}"; do
+   # Strip off the leading destdir before 
outputting the path,
+   # but leave the prefix if there is one.
+   eqawarn "  ${f#${D%/}/}"
+   done
+   fi
+   done
+}
+
+bad_bin_group_write_check
+:
-- 
2.16.4

[gentoo-portage-dev] [PATCH 0/2] Two insecure ownership and group-writability QA checks.

[Michael Orlitzky]

Discussed and implemented in,

  https://bugs.gentoo.org/629398

Michael Orlitzky (2):
  bin/install-qa-check.d: add new 90bad-bin-owner QA check.
  bin/install-qa-check.d: add new 90bad-bin-group-write QA check.

 bin/install-qa-check.d/90bad-bin-group-write | 40 
 bin/install-qa-check.d/90bad-bin-owner   | 38 ++
 2 files changed, 78 insertions(+)
 create mode 100644 bin/install-qa-check.d/90bad-bin-group-write
 create mode 100644 bin/install-qa-check.d/90bad-bin-owner

-- 
2.16.4

[gentoo-portage-dev] [PATCH 1/2] bin/install-qa-check.d: add new 90bad-bin-owner QA check.

[Michael Orlitzky]

System executables that are not owned by root pose a security
risk. The owner of the executable is free to modify it at any time;
so, for example, he can change a daemon's behavior to make it
malicious before the next time the service is started (usually by
root).

On a "normal" system, there is no good reason why the superuser should
not own every system executable. This commit adds a new install-time
check that reports any such binaries with a QA warning. To avoid false
positives, non-"normal" systems (like prefix) are skipped at the moment.

Bug: https://bugs.gentoo.org/629398
---
 bin/install-qa-check.d/90bad-bin-owner | 38 ++
 1 file changed, 38 insertions(+)
 create mode 100644 bin/install-qa-check.d/90bad-bin-owner

diff --git a/bin/install-qa-check.d/90bad-bin-owner 
b/bin/install-qa-check.d/90bad-bin-owner
new file mode 100644
index 0..188d67a51
--- /dev/null
+++ b/bin/install-qa-check.d/90bad-bin-owner
@@ -0,0 +1,38 @@
+# Copyright 1999-2018 Gentoo Foundation
+# Distributed under the terms of the GNU General Public License v2
+
+bad_bin_owner_check() {
+   # Warn about globally-installed executables (in /bin, /usr/bin, /sbin,
+   # or /usr/sbin) that are owned by a nonzero UID.
+
+   # This check doesn't work on non-root prefix installations at
+   # the moment, because every executable therein is owned by a
+   # nonzero UID.
+   [[ "${EUID}" -ne "0" || "${PORTAGE_INST_UID}" -ne "0" ]] && return
+
+   local d f found=()
+
+   for d in "${ED%/}/bin" "${ED%/}/usr/bin" "${ED%/}/sbin" 
"${ED%/}/usr/sbin"; do
+   [[ -d "${d}" ]] || continue
+
+   # Read the results of the "find" command into the "found" bash 
array.
+   # Use -L to catch symlinks whose targets are owned by a 
non-root user,
+   # even though it won't catch ABSOLUTE symlinks until the package
+   # is RE-installed (the first time around, the target won't 
exist).
+   while read -r -d '' f; do
+   found+=( "${f}" )
+   done < <(find -L "${d}" -maxdepth 1 -type f ! -uid 0 -print0)
+
+   if [[ ${found[@]} ]]; then
+   eqawarn "system executables owned by nonzero uid:"
+   for f in "${found[@]}"; do
+   # Strip off the leading destdir before 
outputting the path,
+   # but leave the prefix if there is one.
+   eqawarn "  ${f#${D%/}/}"
+   done
+   fi
+   done
+}
+
+bad_bin_owner_check
+:
-- 
2.16.4

Re: [gentoo-portage-dev] [PATCH 1/2] portage.package.ebuild.config: Rename iuse_implicit -> iuse_effective

[Michael Lienhardt]

Many thanks.

I should definitively read this document, that is far more precise that 
anything I have found on the wiki or on devmanual.

Il 06/02/2018 00:05, Zac Medico ha scritto:

On 02/05/2018 02:46 PM, Michael Lienhardt wrote:

Is the IUSE_IMPLICIT variable in the make.defaults also changed into
IUSE_EFFECTIVE?

I'm sorry if this question was already discussed/answered somewhere else.


The IUSE_EFFECTIVE variable is generated from IUSE_IMPLICIT and some
other variables. It's documented in the "USE and IUSE handling" section
here:

https://dev.gentoo.org/~ulm/pms/head/pms.html#x1-11900011.1.1

Re: [gentoo-portage-dev] [PATCH 1/2] portage.package.ebuild.config: Rename iuse_implicit -> iuse_effective

[Michael Lienhardt]

Is the IUSE_IMPLICIT variable in the make.defaults also changed into 
IUSE_EFFECTIVE?

I'm sorry if this question was already discussed/answered somewhere else.

Michael

Il 04/02/2018 14:40, Michał Górny ha scritto:

Rename the iuse_implicit variable used in USE_EXPAND handling to
iuse_effective, since that is what is actually passed there. Correct
naming makes figuring out what the function does much easier.
---
  pym/portage/package/ebuild/config.py | 6 +++---
  1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/pym/portage/package/ebuild/config.py 
b/pym/portage/package/ebuild/config.py
index 5624e86d3..35cf4f614 100644
--- a/pym/portage/package/ebuild/config.py
+++ b/pym/portage/package/ebuild/config.py
@@ -1307,13 +1307,13 @@ class config(object):
"""
  
  		def __init__(self, settings, unfiltered_use,

-   use, usemask, iuse_implicit,
+   use, usemask, iuse_effective,
use_expand_split, use_expand_dict):
self._settings = settings
self._unfiltered_use = unfiltered_use
self._use = use
self._usemask = usemask
-   self._iuse_implicit = iuse_implicit
+   self._iuse_effective = iuse_effective
self._use_expand_split = use_expand_split
self._use_expand_dict = use_expand_dict
  
@@ -1331,7 +1331,7 @@ class config(object):

if has_wildcard:
var_split = [ x for x in var_split if x != "*" ]
has_iuse = set()
-   for x in self._iuse_implicit:
+   for x in self._iuse_effective:
if x[:prefix_len] == prefix:
has_iuse.add(x[prefix_len:])
if has_wildcard:

Re: [gentoo-portage-dev] [PATCH v3] install-qa-check: New QA check/cleanup for empty directories

[Michael Orlitzky]

On 01/30/2018 02:23 AM, Michał Górny wrote:
> Warn about empty directories installed to /var

Why only warn about /var, considering that FEATURES=strict-keepdir will
delete the others? People will probably assume that if their package
throws no warnings, it's strict-keepdir-safe.

[gentoo-portage-dev] Re: [PATCH v2 3/3] _emerge.Ebuild*: delay creating DISTDIR shadow until src_unpack

[Michael Haubenwallner]

On 01/25/2018 10:11 AM, Michał Górny wrote:
> W dniu czw, 25.01.2018 o godzinie 10∶07 +0100, użytkownik Michael
> Haubenwallner napisał:
>> Hi,
>>
>> ${Subject} ringing a bell here:
>>
>> dev-db/oracle-instantclient is fetch restricted. As a binary package with
>> multiple USE options there's a bunch of files to download - even for
>> multiple archs when multilib is active.
>>
>> So in pkg_nofetch() I'm telling the user whether a file to download is
>> "already here" or "still absent", by testing if $A exists in $DISTDIR.
>>
>> With ${Subject}, I'm wondering if DISTDIR is created for pkg_nofetch too.
>>
>
> You're doing the wrong thing then. DISTDIR is not allowed
> in pkg_nofetch().

Is there a supported way to tell the user exactly which files are still missing?

> Furthermore, you're touching files whose hashes have
> not been verified which is twice wrong.

Well - does portage actually provide unverified files in the shadow DISTDIR?

Thanks!
/haubi/

Re: [gentoo-portage-dev] [PATCH] emerge: add --changed-deps-report option (bug 645780)

[Michael Orlitzky]

Since ::gentoo is the only repository governed by the PMS, can't we make
repoman do this? The problem with requiring "repoman --force" for an
in-place dependency change is that repoman generally won't have access
to the unedited ebuild; but for ::gentoo, we can probably hack something
together in git. Have it source the old and new ebuilds, and compare
their dependency lists.

It won't work outside of a git repo, but it will work in the one place
that really matters.

[gentoo-portage-dev] Re: [PATCH v2 3/3] _emerge.Ebuild*: delay creating DISTDIR shadow until src_unpack

[Michael Haubenwallner]

Hi,

${Subject} ringing a bell here:

dev-db/oracle-instantclient is fetch restricted. As a binary package with
multiple USE options there's a bunch of files to download - even for
multiple archs when multilib is active.

So in pkg_nofetch() I'm telling the user whether a file to download is
"already here" or "still absent", by testing if $A exists in $DISTDIR.

With ${Subject}, I'm wondering if DISTDIR is created for pkg_nofetch too.

/haubi/

On 01/25/2018 09:50 AM, Michał Górny wrote:
> ---
>  pym/_emerge/EbuildExecuter.py | 4 
>  pym/_emerge/EbuildPhase.py| 6 --
>  2 files changed, 4 insertions(+), 6 deletions(-)
> 
> diff --git a/pym/_emerge/EbuildExecuter.py b/pym/_emerge/EbuildExecuter.py
> index ab79ce901..d387b42be 100644
> --- a/pym/_emerge/EbuildExecuter.py
> +++ b/pym/_emerge/EbuildExecuter.py
> @@ -8,7 +8,6 @@ import portage
>  from portage import os
>  from portage.eapi import eapi_has_src_prepare_and_src_configure, \
>   eapi_exports_replace_vars
> -from portage.package.ebuild.prepare_build_dirs import _prepare_fake_distdir
>  
>  class EbuildExecuter(CompositeTask):
>  
> @@ -25,9 +24,6 @@ class EbuildExecuter(CompositeTask):
>   cleanup = 0
>   portage.prepare_build_dirs(pkg.root, settings, cleanup)
>  
> - alist = settings.configdict["pkg"].get("A", "").split()
> - _prepare_fake_distdir(settings, alist)
> -
>   if eapi_exports_replace_vars(settings['EAPI']):
>   vardb = pkg.root_config.trees['vartree'].dbapi
>   settings["REPLACING_VERSIONS"] = " ".join(
> diff --git a/pym/_emerge/EbuildPhase.py b/pym/_emerge/EbuildPhase.py
> index aa3a66831..d3fada622 100644
> --- a/pym/_emerge/EbuildPhase.py
> +++ b/pym/_emerge/EbuildPhase.py
> @@ -1,4 +1,4 @@
> -# Copyright 1999-2013 Gentoo Foundation
> +# Copyright 1999-2018 Gentoo Foundation
>  # Distributed under the terms of the GNU General Public License v2
>  
>  import gzip
> @@ -12,7 +12,7 @@ from _emerge.MiscFunctionsProcess import 
> MiscFunctionsProcess
>  from _emerge.EbuildProcess import EbuildProcess
>  from _emerge.CompositeTask import CompositeTask
>  from portage.package.ebuild.prepare_build_dirs import (_prepare_workdir,
> - _prepare_fake_filesdir)
> + _prepare_fake_distdir, _prepare_fake_filesdir)
>  from portage.util import writemsg
>  
>  try:
> @@ -171,6 +171,8 @@ class EbuildPhase(CompositeTask):
>   def _start_ebuild(self):
>  
>   if self.phase == "unpack":
> + alist = self.settings.configdict["pkg"].get("A", 
> "").split()
> + _prepare_fake_distdir(self.settings, alist)
>   _prepare_fake_filesdir(self.settings)
>  
>   fd_pipes = self.fd_pipes
> 

[gentoo-portage-dev] [PATCH v2 1/2] man/ebuild.5: document that dodir is for nonempty directories.

[Michael Orlitzky]

---
 man/ebuild.5 | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/man/ebuild.5 b/man/ebuild.5
index 42a0599fe..28e9582d1 100644
--- a/man/ebuild.5
+++ b/man/ebuild.5
@@ -1271,7 +1271,8 @@ Creates directories inside of ${ED}.
 .br
 .BR 'dodir\ /usr/lib/apache'
 creates ${ED}/usr/lib/apache.  Note that the do* functions will run
-\fBdodir\fR for you.
+\fBdodir\fR for you. If this directory will be empty when it is merged,
+then please use \fBkeepdir\fR instead.
 .TP
 .B diropts\fR \fI[options for install(1)]
 Can be used to define options for the install function used in
-- 
2.13.6

[gentoo-portage-dev] [PATCH v2 0/2] man page updates to promote keepdir usage

[Michael Orlitzky]

The main barrier to proper keepdir usage is that nobody knows what
it's for. These two commits update the ebuild (5) man page with an
explanation, namely that empty directories are undefined by the PMS.

v2 uses different wording for dodir, and tries to be more precise
about what we mean by "is (non)empty."

Michael Orlitzky (2):
  man/ebuild.5: document that dodir is for nonempty directories.
  man/ebuild.5: document the rationale for using keepdir over dodir.

 man/ebuild.5 | 8 +---
 1 file changed, 5 insertions(+), 3 deletions(-)

-- 
2.13.6

[gentoo-portage-dev] [PATCH v2 2/2] man/ebuild.5: document the rationale for using keepdir over dodir.

[Michael Orlitzky]

---
 man/ebuild.5 | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/man/ebuild.5 b/man/ebuild.5
index 28e9582d1..8784a14ee 100644
--- a/man/ebuild.5
+++ b/man/ebuild.5
@@ -1285,8 +1285,9 @@ Sets the root (\fIDESTTREE\fR) for other functions like 
\fBdobin\fR,
 The default root is /usr.
 .TP
 .B keepdir\fR \fI [more paths]
-Tells portage to leave directories behind even if they're empty.  Functions
-the same as \fBdodir\fR.
+Similar to \fBdodir\fR, but used to create directories that would otherwise
+be empty. The treatment of completely-empty directories is undefined by the
+PMS, and using \fBkeepdir\fR ensures that they are tracked.
 .TP
 .B dobin\fR \fI [list of more binaries]
 Installs a \fIbinary\fR or a list of binaries into \fIDESTTREE\fR/bin.
-- 
2.13.6

[gentoo-portage-dev] [PATCH 0/2] man page updates to promote keepdir usage

[Michael Orlitzky]

The main barrier to proper keepdir usage is that nobody knows what
it's for. These two commits update the ebuild (5) man page with an
explanation, namely that empty directories are undefined by the PMS.

Michael Orlitzky (2):
  man/ebuild.5: document that dodir is for nonempty directories.
  man/ebuild.5: document the rationale for using keepdir over dodir.

 man/ebuild.5 | 10 ++
 1 file changed, 6 insertions(+), 4 deletions(-)

-- 
2.13.6

[gentoo-portage-dev] [PATCH 2/2] man/ebuild.5: document the rationale for using keepdir over dodir.

[Michael Orlitzky]

---
 man/ebuild.5 | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/man/ebuild.5 b/man/ebuild.5
index 9dd969b03..5e2fdbcf5 100644
--- a/man/ebuild.5
+++ b/man/ebuild.5
@@ -1285,8 +1285,9 @@ Sets the root (\fIDESTTREE\fR) for other functions like 
\fBdobin\fR,
 The default root is /usr.
 .TP
 .B keepdir\fR \fI [more paths]
-Tells portage to leave directories behind even if they're empty.  Functions
-the same as \fBdodir\fR.
+Similar to \fBdodir\fR, but used to create directories that would otherwise
+be empty. The treatment of completely-empty directories is undefined by the
+PMS, and using \fBkeepdir\fR ensures that they are tracked.
 .TP
 .B dobin\fR \fI [list of more binaries]
 Installs a \fIbinary\fR or a list of binaries into \fIDESTTREE\fR/bin.
-- 
2.13.6

[gentoo-portage-dev] [PATCH 1/2] man/ebuild.5: document that dodir is for nonempty directories.

[Michael Orlitzky]

---
 man/ebuild.5 | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/man/ebuild.5 b/man/ebuild.5
index 42a0599fe..9dd969b03 100644
--- a/man/ebuild.5
+++ b/man/ebuild.5
@@ -1267,11 +1267,12 @@ that this expression does \fBNOT\fR use the offset 
prefix.
 runs sed on ${ED}/usr/bin/some\-script
 .TP
 .B dodir\fR \fI [more paths]
-Creates directories inside of ${ED}.
+Creates (nonempty) directories inside of ${ED}.
 .br
 .BR 'dodir\ /usr/lib/apache'
 creates ${ED}/usr/lib/apache.  Note that the do* functions will run
-\fBdodir\fR for you.
+\fBdodir\fR for you. Empty directories must be created with \fBkeepdir\fR
+instead.
 .TP
 .B diropts\fR \fI[options for install(1)]
 Can be used to define options for the install function used in
-- 
2.13.6

Re: [gentoo-portage-dev] [PATCH v2] install-qa-check: Do not install empty directories

[Michael Orlitzky]

On 01/10/2018 03:55 PM, Zac Medico wrote:
>>
>> This is going to break a lot of packages whose build systems create e.g.
>> /var/lib/foo and do nothing with it immediately. The ebuild should be
>> calling keepdir on those paths, but how would anyone know that?
> 
> If we consider that portage already removes these directories if they
> are still empty when the package is re-merged or upgraded, then maybe
> the risk is low enough?
> 

Does it? (Are we talking about the same thing?)

For example, the nagios-core build system creates a bunch of empty
directories under /var/nagios:

  $ find /var/nagios
  /var/nagios
  /var/nagios/rw
  /var/nagios/home
  /var/nagios/spool
  /var/nagios/spool/checkresults
  /var/nagios/archives

Re-emerging the same version of nagios-core doesn't kill them off.

Re: [gentoo-portage-dev] [PATCH v2] install-qa-check: Do not install empty directories

[Michael Orlitzky]

On 01/10/2018 03:13 PM, Michał Górny wrote:
> Remove empty directories in install-qa-check phase in order to prevent
> Portage from installing them, and therefore from developers relying
> on them being installed.

This is going to break a lot of packages whose build systems create e.g.
/var/lib/foo and do nothing with it immediately. The ebuild should be
calling keepdir on those paths, but how would anyone know that?

Re: Re : Re: [gentoo-portage-dev] Constraint-Based Dependency Solver for Portage: a prototype

[Michael Lienhardt]

Dear Alexander,

Many thanks for the advices.
I started few discussions on the forum and will go to FOSDEM.
I'll see where it will go.

Best,
Michael

Il 16/12/2017 14:39, Alexander Berntsen ha scritto:

On 13/12/17 02:52, michael.lienha...@laposte.net wrote:

But maybe there are things we can do to help start a dialog, like:
  - reaching in other mailing lists

I don't think a post to gentoo-dev would be remiss in this case.


  - posting on a Gentoo forum

Always useful, I'm told, though I don't venture there. But that way
you're far more likely to engage *users*.


  - participating in a workshop/conference/other where we could directly meet 
and discuss with the community

FOSDEM and Linux Days are probably the best choices.


  - or simply starting an informal discussion by email where instead of having 
to look into the Github repository, you could directly ask me

If someone has the time, that'll probably naturally happen through the
MLs. Christmas time tends to be peak bikeshedding hours at Gentoo, so
maybe cross-post to -dev closer to the holidays?

Re : Re: [gentoo-portage-dev] Constraint-Based Dependency Solver for Portage: a prototype

[michael . lienhardt]

Dear Alexander,

Many thanks for your reply and your encouragements.
The point that you raised is very interesting and was partially done in Debian 
(they defined a wrapper around apt-get instead of refactoring it): 
http://manpages.ubuntu.com/manpages/zesty/man8/apt-cudf-get.8.html
Part of their work was formalized in coq and implemented in OCaml.
In our case, we don't have any mechanized formalization of our model (maybe in 
the future).

I too (and my colleagues) hope that someone on the team could have some time to 
look into our project.
But maybe there are things we can do to help start a dialog, like:
 - reaching in other mailing lists
 - posting on a Gentoo forum
 - participating in a workshop/conference/other where we could directly meet 
and discuss with the community
 - or simply starting an informal discussion by email where instead of having 
to look into the Github repository, you could directly ask me

Does anyone have suggestions on that topic?

Again, many thanks.
I really hope that with everyone's feedback, suggestions, and help, we could 
make something useful from this prototype.


Michael Lienhardt

PS: I forgot in my previous mail to talk about the other persons involved in 
this project:
 - Jacopo Mauro, Post-doc in UiO (Norway), developer of the solver backend
 - Simone Donetti, Engineer in Unito (Italy), he helped me perform some tests
 - Ferruccio Damiani (Unito), Einar Broch Johnsen and Ingrid Chieh Yu (UiO), 
our supervisors

[gentoo-portage-dev] Constraint-Based Dependency Solver for Portage: a prototype

[Michael Lienhardt]

Dear Portage developers,

I am a Post-doc in formal methods and software engineering. With my colleagues, 
we are working on a formal model for software composition, and were looking for 
a concrete example of such model to motivate and guide our work. I knew portage 
from using gentoo since 2007, and knew that it is the perfect use case for us.

The first result of our work is a prototype for a constraint-based dependency 
solver for Portage:
 like the emerge tool, it takes in parameter a list of atoms to install, and 
computes a full list of packages to install to satisfy the package dependency 
relation.
Up to bugs, this tool is correct and complete: it will always find a solution 
if it exists, and always tell if there are none.
For instance, it successfully computed that gnome-base/gnome cannot be 
installed by default (on a udev system), but found a solution that replaces 
sys-fs/eudev by sys-apps/systemd when we allow the tool to change the USE flag 
selection of the packages.

With this prototype, we also compiled (90% of) a documentation on how portage 
manages package configuration (USE flags declaration, selection, masking, 
keywording, ...).

Link to the prototype: https://github.com/HyVar/gentoo_to_mspl
Link to the documentation: 
https://github.com/HyVar/gentoo_to_mspl/blob/master/PORTAGE.md


We would really like to know your opinions, impressions and suggestions about 
this work.
We would also like to know how useful this tool could be for the community:
 as for now, it is a prototype of a dependency solver (that would definitively 
need some work to be usable in production), but it also offers the possibility 
of any kind of formal analysis on the REQUIRED_USE and dependencies in 
packages, like the one described in https://bugs.gentoo.org/417753
For instance, our tool already checks for obvious reasons (inconsistent 
REQUIRED_USE or unmet dependencies) causing a package not to be installable. In 
particular, on the Portage version available in http://www.osboxes.org/gentoo/ 
, our tool identified 14 packages that could not be installed for these reasons 
(the full list in in post-scriptum).


Additionally, our implementation is based on what I understood of the portage's 
documentation, which I compiled in the PORTAGE.md document: it would be very 
helpful if you could point error that I made or subtleties that I didn't 
understand or missed.

Best Regards,
Michael Lienhardt


PS: list of uninstallable packages:
 dev-java/jruby-1.7.12
 media-video/nvidia-settings-340.58
 dev-ruby/bitescript-0.0.9
 dev-java/spring-core-3.2.4
 app-i18n/ibus-table-code-1.2.0.20100305
 dev-ruby/weakling-0.0.4
 sci-libs/ogdi-3.1.5-r1
 dev-java/jcs-2.0
 net-misc/asterisk-rate_engine-0.5.4
 games-fps/doom3-mitm-20070129
 app-office/impressive-0.10.5
 dev-java/spring-aop-3.2.4
 dev-ruby/duby-0.0.2-r1
 dev-db/mycli-

Re: [gentoo-portage-dev] [PATCH v2] emerge: auto-enable --with-bdeps if --usepkg is not enabled (bug 598444)

[Michael Orlitzky]

On 03/05/2017 02:12 PM, Zac Medico wrote:
> 
> Incorrect.
> 
> ...
> 
> Incorrect.
> 

I see my mistakes, but maintain that this is confusing =)


> 
> The --with-bdeps-auto option results in desirable behavior by default,
> and it's also backward compatible with existing --with-bdeps and
> --usepkg usage. It just does the right thing, minimizing the impact to
> existing emerge usage.

I was hoping that since nothing breaks with --update-bdeps=y and
--clean-bdeps=n (the new defaults), we could just disable --with-bdeps
immediately and emit a warning when it's given.


> 
> There some problems with --update-bdeps/--clean-bdeps:
> 
> * The program logic will be more complicated, since --with-bdeps will
> have to override both options for backward-compatibility with existing
> --with-bdeps usage.

Not applicable if --with-bdeps goes away.


> * The meaning of --clean-bdeps is confusing. Does --clean-bdeps=y mean
> to clean build time deps, or does it mean to pull build time deps into
> the dependency graph for removal operations?

--clean-bdeps means clean the bdeps.


I totally agree that the worst option of all is to keep --with-bdeps AND
introduce the other two.

Re: [gentoo-portage-dev] [PATCH v2] emerge: auto-enable --with-bdeps if --usepkg is not enabled (bug 598444)

[Michael Orlitzky]

On 03/05/2017 03:40 AM, Zac Medico wrote:
> 
> A new --with-bdeps-auto= option is provided, making it possible to
> enable or disable the program logic that causes --with-bdeps to be
> automatically enabled. Use --with-bdeps-auto=n to prevent --with-bdeps
> from being automatically enabled for installation actions. This is useful
> for some rare cases in which --with-bdeps triggers unsolvable dependency
> conflicts (and putting --with-bdeps=n in EMERGE_DEFAULT_OPTS would cause
> undesirable --depclean behavior).
> 

If I understand correctly, the end result of this is two --flags that
combine in a (rather complicated) way to let the user control the
default bdeps behavior of both "emerge --update" and "emerge --depclean"
separately. I'll try to summarize my understanding:

   I. emerge --update

 I.a. Some people want to --update the build dependencies they have
  installed for e.g. security purposes.

 I.b. Others don't want build deps pulled in by "emerge --update",
  because they're using binary packages or because it causes
  conflicts in the resolver (llvm).

  II. emerge --depclean

II.a. The default behavior is to not remove build-only dependencies
  because, generally, they will just have to rebuilt again.

II.b. However, some people want to remove the build-only deps that
  aren't strictly in use -- particularly if those build-only
  deps are not being updated by emerge --update.


To choose between those four options...

  i.   --with-bdeps=n and --with-bdeps-auto=y gives you I.a. + II.b.

  ii.  --with-bdeps=n and --with-bdeps-auto=n gives you I.b. + II.b.

  iii. --with-bdeps=y and --with-bdeps-auto=y gives you I.a. + II.a.

  iv.  --with-bdeps=y and --with-bdeps-auto=n gives you I.a. + II.a.

That only gets you three out of the four options. You have to read the
fine print to get the other:

  v.   passing no flags explicitly gives you I.b. and II.a.

If there's going to be two flags, can't we do better? Why not just name
the flags after what we want them to do:

  --update-bdeps= (default: y)

  --clean-bdeps=  (default: n)

With those two options, it's easy to tell portage exactly what you want.
If I don't like the defaults, I can turn one of them off without
affecting the other.

But what about the --usepkg magic? I think the workaround can be left
as-is. When --usepkg is enabled, switch the default for --update-bdeps
to "n" unless it is explicitly set.

Food for thought. Anyway, thanks for working on this.

Re: [gentoo-portage-dev] [PATCH] sys-apps/portage: add native-extensions USE flag (bug 571444)

[Michael Orlitzky]

On 02/01/2017 04:03 PM, Michał Górny wrote:
>>  SLOT="0"
>> -IUSE="build doc epydoc +ipc linguas_ru selinux xattr"
>> +IUSE="build doc epydoc +ipc linguas_ru native-extensions selinux
>> xattr"
> 
> Wouldn't it be better to enable it by default?
> 

Please don't enshrine personal preferences into IUSE defaults unless
they're critical to the package.

Re: [gentoo-portage-dev] [PATCH 1/1] Add a test case for PMS-compliant usage of the ROOT variable.

[Michael Orlitzky]

On 05/11/2016 12:33 PM, Brian Dolbec wrote:
> 
> I'll work on adding this check to repoman after I finish getting some
> in progress changes done so the new repoman code can be released.
> 

I took a look at the new code and it was really easy to get something
working. I added a new QA category, "variable.illegal", and left it an
error. People can --force it through if they want to, so I don't think
maintaining backwards-compatibility (against what the PMS says) is crucial.

Adding a new check in ebuild/checks.py was simple: add a PhaseCheck with
a whitelist of phases and perform the check in any other scope.

The only tricky part is the regular expression. The dumbest thing that
could possibly work is,

  rootvar_re = re.compile(r'\$(ROOT|{ROOT})')

That will throw false positives in all sorts of situations, like
app-eselect/eselect-rails-0.21:

  src_prepare() {
# Fix/Add Prefix support
sed -i -e 's/\${ROOT}/${EROOT}/' *.eselect || die
  }

The only check I saw that tried to be clever about string parsing is the
variable quoting check. Both seem to share a common need, to perform a
check only when something really is a variable and not part of a string
or otherwise cleverly-disguised.

Should we try to factor something out, or do you want me to send the
naive version (based on the repoman branch) and fix it later?

[gentoo-portage-dev] [PATCH 0/1] Test PMS-compliant usage of the ROOT variable.

[Michael Orlitzky]

Per the discussion over on -dev, this adds a test case for usage of
the ROOT variable outside of the PMS-defined pkg_* functions (which is
not allowed).

There should probably be a warning for this in repoman, since most
usages I've seen are really intended to be EPREFIX and not ROOT. Any
cases that are not so easy to fix will stand out once the trivial ones
are. Perhaps in EAPI-$next, the warning can become an error.

Michael Orlitzky (1):
  Add a test case for PMS-compliant usage of the ROOT variable.

 ebuild-test/root-var-usage/metadata.xml| 24 ++
 ebuild-test/root-var-usage/root-var-usage-0.ebuild | 90 ++
 2 files changed, 114 insertions(+)
 create mode 100644 ebuild-test/root-var-usage/metadata.xml
 create mode 100644 ebuild-test/root-var-usage/root-var-usage-0.ebuild

-- 
2.7.3

[gentoo-portage-dev] Old news items

[Michael Palimaka]

On a fresh amd64 stage3 a user is presented with fourteen news items to
read, and three[1][2][3] of these are directly related to Portage.

In order to improve the out-of-box experience, please review them for
relevance and consider removing or restricting any that are not.

1:
https://gitweb.gentoo.org/data/gentoo-news.git/tree/2012-05-21-portage-config-protect-if-modified/2012-05-21-portage-config-protect-if-modified.en.txt
2:
https://gitweb.gentoo.org/data/gentoo-news.git/tree/2013-06-07-portage-preserve-libs-default/2013-06-07-portage-preserve-libs-default.en.txt
3:
https://gitweb.gentoo.org/data/gentoo-news.git/tree/2015-02-04-portage-sync-changes/2015-02-04-portage-sync-changes.en.txt

[gentoo-portage-dev] [PATCH 0/2] Document globbing for INSTALL_MASK.

[Michael Orlitzky]

This documents shell globs in INSTALL_MASK and warns about filenames
containing spaces. I also added a few more comments to the code after
I figured out what it was going.

Michael Orlitzky (2):
  bin/misc-functions.sh: Elaborate on some comments in install_mask().
  man/make.conf.5: Document globbing for INSTALL_MASK.

 bin/misc-functions.sh | 16 +---
 man/make.conf.5   | 44 
 2 files changed, 49 insertions(+), 11 deletions(-)

-- 
2.3.6

[gentoo-portage-dev] [PATCH 1/2] bin/misc-functions.sh: Elaborate on some comments in install_mask().

[Michael Orlitzky]

---
 bin/misc-functions.sh | 16 +---
 1 file changed, 13 insertions(+), 3 deletions(-)

diff --git a/bin/misc-functions.sh b/bin/misc-functions.sh
index 9b79351..c2ff70a 100755
--- a/bin/misc-functions.sh
+++ b/bin/misc-functions.sh
@@ -261,20 +261,30 @@ install_mask() {
shift
local install_mask=$*
 
-   # we don't want globbing for initial expansion, but afterwards, we do
+   # We think of $install_mask as a space-separated list of
+   # globs. We don't want globbing in the for loop; that is, we
+   # want to keep the asterisks in the indivual entries.
local shopts=$-
set -o noglob
local no_inst
for no_inst in ${install_mask}; do
+   # Here, $no_inst is a single entry potentially
+   # containing a glob. From now on, we *do* want to
+   # expand it.
set +o noglob
 
-   # normal stuff
+   # The standard case where $no_inst is something that
+   # the shell could expand on its own.
if [[ -e ${root}/${no_inst} || ${root}/${no_inst} != $(echo 
${root}/${no_inst}) ]] ; then
__quiet_mode || einfo Removing ${no_inst}
rm -Rf ${root}/${no_inst} /dev/null
fi
 
-   # we also need to handle globs (*.a, *.h, etc)
+   # We also want to allow the user to specify a bare
+   # glob. For example, $no_inst=*.a should prevent
+   # ALL files ending in .a from being installed,
+   # regardless of their location/depth. We achieve this
+   # by passing the pattern to `find`.
find ${root} \( -path ${no_inst} -or -name ${no_inst} \) \
-print0 2 /dev/null \
| LC_ALL=C sort -z \
-- 
2.3.6

[gentoo-portage-dev] [PATCH] unpack: avoid useless chmods to improve speed

[Michael Haubenwallner]

X-Gentoo-Bug: 554084
X-Gentoo-Bug-url: https://bugs.gentoo.org/show_bug.cgi?id=554084
---
 bin/phase-helpers.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/bin/phase-helpers.sh b/bin/phase-helpers.sh
index efd2cfa..b446060 100644
--- a/bin/phase-helpers.sh
+++ b/bin/phase-helpers.sh
@@ -531,8 +531,8 @@ unpack() {
done
# Do not chmod '.' since it's probably ${WORKDIR} and 
PORTAGE_WORKDIR_MODE
# should be preserved.
-   find . -mindepth 1 -maxdepth 1 ! -type l -print0 | \
-   ${XARGS} -0 chmod -fR a+rX,u+w,g-w,o-w
+   find . -mindepth 1 '!' -type l '!' -perm /a+rX,u+w,g-w,o-w \
+   -exec chmod -f a+rX,u+w,g-w,o-w '{}' +
 }
 
 econf() {
-- 
2.0.5

[gentoo-portage-dev] [PATCH] unpack: avoid useless chmods to improve speed

[Michael Haubenwallner]

X-Gentoo-Bug: 554084
X-Gentoo-Bug-url: https://bugs.gentoo.org/show_bug.cgi?id=554084
---
 bin/phase-helpers.sh | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/bin/phase-helpers.sh b/bin/phase-helpers.sh
index efd2cfa..bf3ae1f 100644
--- a/bin/phase-helpers.sh
+++ b/bin/phase-helpers.sh
@@ -531,8 +531,8 @@ unpack() {
done
# Do not chmod '.' since it's probably ${WORKDIR} and 
PORTAGE_WORKDIR_MODE
# should be preserved.
-   find . -mindepth 1 -maxdepth 1 ! -type l -print0 | \
-   ${XARGS} -0 chmod -fR a+rX,u+w,g-w,o-w
+   find . -mindepth 1 '!' -type l '!' -perm /a+rX,u+w,g-w,o-w -print0 | \
+   ${XARGS} -0 chmod -f a+rX,u+w,g-w,o-w
 }
 
 econf() {
-- 
2.0.5

[gentoo-portage-dev] Re: [PATCH] Make the USE variable readonly (bug 325009)

[Michael Haubenwallner]

On 04/26/2015 11:14 PM, Zac Medico wrote:
 This requires the EBUILD_FORCE_TEST code from dyn_test to execute
 before USE is declared readonly.
 
 X-Gentoo-Bug: 325009
 X-Gentoo-Bug-URL: https://bugs.gentoo.org/show_bug.cgi?id=325009
 ---

 --- a/bin/ebuild.sh
 +++ b/bin/ebuild.sh
 @@ -1,5 +1,5 @@
 + declare -r USE

This breaks the 'ebuildshell' feature patch,
https://bugs.gentoo.org/show_bug.cgi?id=155161

Shouldn't USE show up in PORTAGE_READONLY_VARS as well?

/haubi/

Re: [gentoo-portage-dev] Re: [PATCHv3 1/2] MEDIUM: misc-functions: Be more quiet when removing non existing INSTALL_MASK

[Michael Orlitzky]

On 04/21/2015 01:28 PM, Zac Medico wrote:

 The docs for INSTALL_MASK (man 5 make.conf) don't mention that globs
 will work. It's expecting a space delimited list of file names. Does
 it really take a space-delimited list of globs instead? If so, how does
 that reconcile with the fact that * could match spaces?
 
 How does it conflict?
 

I guess it's more of a filenames-with-spaces question. Would this work?

  INSTALL_MASK=Boyd\ -\ Convex Optimization.pdf

If you can escape the spaces, then the space-separated globs aren't
ambiguous. I was thinking of something like this:

  $ /bin/ls B*\ Convex*
  Boyd - Convex Optimization.pdf

Normally if you stick something like that in a quoted variable, its
spaces can be unescaped:

  INSTALL_MASK=B* Convex*

But now it's two globs instead of one. The whole thing makes sense if
you can leave the space escaped though.

Re: [gentoo-portage-dev] Re: [PATCHv3 1/2] MEDIUM: misc-functions: Be more quiet when removing non existing INSTALL_MASK

[Michael Orlitzky]

On 04/21/2015 05:48 AM, Duncan wrote:

 Well, I don't use INSTALL_MASK myself, so I don't have a real-world
 use-case for you. However, it's clear that the code will expand shell
 globs, so I preserved that behavior for compatibility.
 
 I do, with shell globs, tho I didn't bother checking the above to see if 
 they'd have been affected.
 

The docs for INSTALL_MASK (man 5 make.conf) don't mention that globs
will work. It's expecting a space delimited list of file names. Does
it really take a space-delimited list of globs instead? If so, how does
that reconcile with the fact that * could match spaces?

Re: [gentoo-portage-dev] [RFC] Add 'emerge --sync-glsa' action and 'emaint sync-glsa' command

[Michael Orlitzky]

On 12/17/2014 05:32 PM, Brian Dolbec wrote:
 
 Only code changes I see to portage, pkgcore (I know nothing about
 paludis) are to look for the glsa's in the 2 possible locations.  The
 standalone glsa repo, failing that, backup to the gentoo tree.
 

Could we ship a GLSA overlay enabled by default, regardless of what we
do with the main repo? Then for simplicity, we update the tools to check
metadata/glsa in overlays as well. No need to special-case a fallback
location.

glsa-check would of course want to sync the GLSA repo before running.

[gentoo-portage-dev] [PATCH] install-qa-check.d/90world-writable: fix usage of missing function

[Michael Palimaka]

Fixes: 6dafdc288976 (Remove __eqalog  __eqawarnlog)
---
 bin/install-qa-check.d/90world-writable | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/bin/install-qa-check.d/90world-writable 
b/bin/install-qa-check.d/90world-writable
index 2b435ac..1fb2a8f 100644
--- a/bin/install-qa-check.d/90world-writable
+++ b/bin/install-qa-check.d/90world-writable
@@ -23,9 +23,8 @@ world_writable_check() {
if [[ -n ${unsafe_files} ]] ; then
eqawarn QA Notice: Unsafe files detected (set*id and world 
writable)
 
-   for x in $unsafe_files ; do
-   __eqawarnlog world-writable-setid $x
-   done
+   eqatag -v world-writable-setid $unsafe_files
+
die Unsafe files found in \${D}.  Portage will not install 
them.
fi
 
-- 
2.0.4

[gentoo-portage-dev] Re: [PATCH 1/3] bin/misc-functions.sh: Introduce eqalog and eqawarnlog functions.

[Michael Palimaka]

On 27/10/14 07:05, Zac Medico wrote:
 On 10/26/2014 12:31 PM, Michael Palimaka wrote:
 On 26/10/14 07:57, Zac Medico wrote:
 On 10/25/2014 01:32 PM, Zac Medico wrote:
 On 10/25/2014 01:26 PM, Michał Górny wrote:
 Dnia 2014-10-25, o godz. 12:53:15
 Zac Medico zmed...@gentoo.org napisał(a):

 These functions are internals, so they need to be prefixed with __ like
 __eqalog and __eqawarnlog.

 eqawarnlog shouldn't be internal since we support adding QA checks
 in repositories. In fact, I am planning to move some Gentoo-specific QA
 checks out of portage code.

 It's a PMS thing. If it's not in PMS and the package manager provides
 it, it's supposed to be prefixed.

 Note that we could have unprefixed aliases inside misc-functions.sh,
 since misc-functions.sh env is never saved.


 I've sent updated patches based on the last feedback. Should I send a
 new one with the aliases, and if so, should the portage checks use the
 alias or real function?
 
 Considering Michał's plan to expose these functions to QA checks in
 repositories, it would make sense to go ahead and add expose the aliases
 in misc-functions.sh now. On the other hand, it makes sense to use the
 prefixed versions in all internal portage code, for consistency. So, I'd
 probably just wait until later to add the unprefixed versions. I don't
 have a strong opinion though. The new patch set that you posted looks
 good to me.

That's fine, we can just add the alias when Michał's GLEP is finalised then.

[gentoo-portage-dev] [PATCH 2/3] install-qa-check.d/05double-D: Write to log and improve consistency.

[Michael Palimaka]

Present the list of offending files newline-delimitered for consistency
with other checks.
---
 bin/install-qa-check.d/05double-D | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/bin/install-qa-check.d/05double-D 
b/bin/install-qa-check.d/05double-D
index 1634afd..7d958f1 100644
--- a/bin/install-qa-check.d/05double-D
+++ b/bin/install-qa-check.d/05double-D
@@ -2,9 +2,10 @@
 
 DD_check() {
if [[ -d ${D%/}${D} ]] ; then
+   eqawarn QA Notice: files installed in \${D}/\${D}:
local -i INSTALLTOD=0
while read -r -d $'\0' i ; do
-   eqawarn QA Notice: /${i##${D%/}${D}} installed in 
\${D}/\${D}
+   __eqawarnlog double-d /${i##${D%/}${D}}
((INSTALLTOD++))
done  (find ${D%/}${D} -print0)
die Aborting due to QA concerns: ${INSTALLTOD} files installed 
in ${D%/}${D}
-- 
2.0.4

[gentoo-portage-dev] [PATCH 3/3] install-qa-check.d/90world-writable: Write log and general cleanup.

[Michael Palimaka]

Use eqawarn instead of __vecho for visibility.

Present the list of offending files newline-delimitered for consistency
with other checks.
---
 bin/install-qa-check.d/90world-writable | 28 +---
 1 file changed, 21 insertions(+), 7 deletions(-)

diff --git a/bin/install-qa-check.d/90world-writable 
b/bin/install-qa-check.d/90world-writable
index 771027e..4d5f4ab 100644
--- a/bin/install-qa-check.d/90world-writable
+++ b/bin/install-qa-check.d/90world-writable
@@ -2,21 +2,35 @@
 
 world_writable_check() {
# Now we look for all world writable files.
-   local unsafe_files=$(find ${ED} -type f -perm -2 | sed -e s:^${ED}:- 
:)
+   local unsafe_files=$(find ${ED} -type f -perm -2 | sed -e 
s:^${ED}:/:)
+   local OLDIFS x
+
+   OLDIFS=$IFS
+   IFS=$'\n'
+
if [[ -n ${unsafe_files} ]] ; then
-   __vecho QA Security Notice: world writable file(s):
-   __vecho ${unsafe_files}
-   __vecho - This may or may not be a security problem, most of 
the time it is one.
-   __vecho - Please double check that $PF really needs a world 
writeable bit and file bugs accordingly.
-   sleep 1
+   eqawarn QA Security Notice: world writable file(s):
+
+   for x in $unsafe_files ; do
+   __eqawarnlog world-writable $x
+   done
+
+   eqawarn This may or may not be a security problem, most of the 
time it is one.
+   eqawarn Please double check that $PF really needs a world 
writeable bit and file bugs accordingly.
+   eqawarn
fi
 
local unsafe_files=$(find ${ED} -type f '(' -perm -2002 -o -perm 
-4002 ')' | sed -e s:^${ED}:/:)
if [[ -n ${unsafe_files} ]] ; then
eqawarn QA Notice: Unsafe files detected (set*id and world 
writable)
-   eqawarn ${unsafe_files}
+
+   for x in $unsafe_files ; do
+   __eqawarnlog world-writable-setid $x
+   done
die Unsafe files found in \${D}.  Portage will not install 
them.
fi
+
+   IFS=OLDIFS
 }
 
 world_writable_check
-- 
2.0.4

[gentoo-portage-dev] Re: [PATCH 1/3] bin/misc-functions.sh: Introduce eqalog and eqawarnlog functions.

[Michael Palimaka]

On 26/10/14 07:57, Zac Medico wrote:
 On 10/25/2014 01:32 PM, Zac Medico wrote:
 On 10/25/2014 01:26 PM, Michał Górny wrote:
 Dnia 2014-10-25, o godz. 12:53:15
 Zac Medico zmed...@gentoo.org napisał(a):

 On 10/25/2014 09:15 AM, Michael Palimaka wrote:
 +eqalog() {
 + local tag=$1 x
 + shift
 + for x in $@ ; do
 + echo ${tag} ${x}  ${T}/qa.log
 + done
 +}
 +
 +eqawarnlog() {
 + eqalog $@
 + shift
 + for x in $@ ; do
 + eqawarn   $x
 + done
 +}
 +

 These functions are internals, so they need to be prefixed with __ like
 __eqalog and __eqawarnlog.

 eqawarnlog shouldn't be internal since we support adding QA checks
 in repositories. In fact, I am planning to move some Gentoo-specific QA
 checks out of portage code.

 It's a PMS thing. If it's not in PMS and the package manager provides
 it, it's supposed to be prefixed.
 
 Note that we could have unprefixed aliases inside misc-functions.sh,
 since misc-functions.sh env is never saved.
 

I've sent updated patches based on the last feedback. Should I send a
new one with the aliases, and if so, should the portage checks use the
alias or real function?

Just let me know what to change. I have no opinion what goes where. :-)

[gentoo-portage-dev] [PATCH 2/3] install-qa-check.d/05double-D: Write to log and improve consistency.

[Michael Palimaka]

Present the list of offending files newline-delimitered for consistency
with other checks.
---
 bin/install-qa-check.d/05double-D | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/bin/install-qa-check.d/05double-D 
b/bin/install-qa-check.d/05double-D
index 1634afd..84fcc89 100644
--- a/bin/install-qa-check.d/05double-D
+++ b/bin/install-qa-check.d/05double-D
@@ -2,9 +2,10 @@
 
 DD_check() {
if [[ -d ${D%/}${D} ]] ; then
+   eqawarn QA Notice: files installed in \${D}/\${D}:
local -i INSTALLTOD=0
while read -r -d $'\0' i ; do
-   eqawarn QA Notice: /${i##${D%/}${D}} installed in 
\${D}/\${D}
+   eqawarnlog double-d /${i##${D%/}${D}}
((INSTALLTOD++))
done  (find ${D%/}${D} -print0)
die Aborting due to QA concerns: ${INSTALLTOD} files installed 
in ${D%/}${D}
-- 
2.0.4

[gentoo-portage-dev] [PATCH 1/3] bin/misc-functions.sh: Introduce eqalog and eqawarnlog functions.

[Michael Palimaka]

These functions are to be used for creating a log of QA violations in a
machine-readable format.

Stored in ${T]/qa.log, the log consists of one entry per line in the
following format: violation-tag data

For example:
deprecated-directory /usr/man
deprecated-directory /usr/info
world-writable /var/db/foo/bar
---
 bin/misc-functions.sh | 16 
 1 file changed, 16 insertions(+)

diff --git a/bin/misc-functions.sh b/bin/misc-functions.sh
index cc652a9..78da589 100755
--- a/bin/misc-functions.sh
+++ b/bin/misc-functions.sh
@@ -162,6 +162,22 @@ prepcompress() {
return 0
 }
 
+eqalog() {
+   local tag=$1 x
+   shift
+   for x in $@ ; do
+   echo ${tag} ${x}  ${T}/qa.log
+   done
+}
+
+eqawarnlog() {
+   eqalog $@
+   shift
+   for x in $@ ; do
+   eqawarn   $x
+   done
+}
+
 install_qa_check() {
local f i qa_var x
if ! ___eapi_has_prefix_variables; then
-- 
2.0.4

[gentoo-portage-dev] [PATCH 3/3] install-qa-check.d/90world-writable: Write log and general cleanup.

[Michael Palimaka]

Use eqawarn instead of __vecho for visibility.

Present the list of offending files newline-delimitered for consistency
with other checks.
---
 bin/install-qa-check.d/90world-writable | 28 +---
 1 file changed, 21 insertions(+), 7 deletions(-)

diff --git a/bin/install-qa-check.d/90world-writable 
b/bin/install-qa-check.d/90world-writable
index 771027e..ff186c5 100644
--- a/bin/install-qa-check.d/90world-writable
+++ b/bin/install-qa-check.d/90world-writable
@@ -2,21 +2,35 @@
 
 world_writable_check() {
# Now we look for all world writable files.
-   local unsafe_files=$(find ${ED} -type f -perm -2 | sed -e s:^${ED}:- 
:)
+   local unsafe_files=$(find ${ED} -type f -perm -2 | sed -e 
s:^${ED}:/:)
+   local OLDIFS x
+
+   OLDIFS=$IFS
+   IFS=$'\n'
+
if [[ -n ${unsafe_files} ]] ; then
-   __vecho QA Security Notice: world writable file(s):
-   __vecho ${unsafe_files}
-   __vecho - This may or may not be a security problem, most of 
the time it is one.
-   __vecho - Please double check that $PF really needs a world 
writeable bit and file bugs accordingly.
-   sleep 1
+   eqawarn QA Security Notice: world writable file(s):
+
+   for x in $unsafe_files ; do
+   eqawarnlog world-writable $x
+   done
+
+   eqawarn This may or may not be a security problem, most of the 
time it is one.
+   eqawarn Please double check that $PF really needs a world 
writeable bit and file bugs accordingly.
+   eqawarn
fi
 
local unsafe_files=$(find ${ED} -type f '(' -perm -2002 -o -perm 
-4002 ')' | sed -e s:^${ED}:/:)
if [[ -n ${unsafe_files} ]] ; then
eqawarn QA Notice: Unsafe files detected (set*id and world 
writable)
-   eqawarn ${unsafe_files}
+
+   for x in $unsafe_files ; do
+   eqawarnlog world-writable-setid $x
+   done
die Unsafe files found in \${D}.  Portage will not install 
them.
fi
+
+   IFS=OLDIFS
 }
 
 world_writable_check
-- 
2.0.4

[gentoo-portage-dev] Re: [PATCH 1/3] bin/misc-functions.sh: Introduce eqalog and eqawarnlog functions.

[Michael Palimaka]

On 26/10/14 07:00, Zac Medico wrote:
 On 10/25/2014 12:54 PM, Zac Medico wrote:
 On 10/25/2014 12:53 PM, Zac Medico wrote:
 On 10/25/2014 09:15 AM, Michael Palimaka wrote:
 +eqalog() {
 +  local tag=$1 x
 +  shift
 +  for x in $@ ; do
 +  echo ${tag} ${x}  ${T}/qa.log
 +  done
 +}
 +
 +eqawarnlog() {
 +  eqalog $@
 +  shift
 +  for x in $@ ; do
 +  eqawarn   $x
 +  done
 +}
 +

 These functions are internals, so they need to be prefixed with __ like
 __eqalog and __eqawarnlog.


 Also, please unset them inside bin/save-ebuild-env.sh.

 
 Actually, these suggestions are optional, since the environment from
 misc-functions.sh is never saved. However, if you wanted to move them to
 isolated-functions.sh, then these suggestions are mandatory.
 

Is it better to move them to isolated-functions and implement the above
changes then? I only put it in misc-functions since it's near
install_qa_check and I'm not too familiar with the file layout. :-)

Re: [gentoo-portage-dev] support multiple eclass variants (aka unstable eclasses)

[Michael Haubenwallner]

On 07/08/2014 07:11 PM, Sebastian Luther wrote:
 Am 08.07.2014 18:58, schrieb Michael Haubenwallner:
 Hello fellow Portage developers,

 attached portage patch draft aims to allow for easy distributing eclasses to 
 be tested by
 multiple tinderboxes on various architectures, without being active for 
 normal installs.
 
 What does the patch allow you to do, that you can't do right now? (i.e.
 put an eclass with the same name in an repository and use
 eclass-overrides to force its use in another repo?)

Is there an easy way to use the eclasses from the overriding tree, but not the 
(usually newer)
ebuilds from there - such as 'configure but disable repo (except for use in 
eclass-override)'?

Thanks!
/haubi/

[gentoo-portage-dev] support multiple eclass variants (aka unstable eclasses)

[Michael Haubenwallner]

Hello fellow Portage developers,

attached portage patch draft aims to allow for easy distributing eclasses to be 
tested by
multiple tinderboxes on various architectures, without being active for normal 
installs.

Usage is to have in make.conf (or profile): PORTAGE_ECLASS_VARIANTS=testing
and for the eclass: to live in tree-or-overlay/eclass/testing/

Thoughts? (even for var-naming, manpage yes/no/wording, ...)

Thank you!
/haubi/
From 2ddc1db0f1c15873e2476fbf5ba0352464c8725a Mon Sep 17 00:00:00 2001
From: Michael Haubenwallner ha...@gentoo.org
Date: Tue, 8 Jul 2014 17:48:54 +0200
Subject: [PATCH] support PORTAGE_ECLASS_VARIANTS

---
 bin/ebuild.sh | 6 --
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/bin/ebuild.sh b/bin/ebuild.sh
index be044e0..366cab6 100755
--- a/bin/ebuild.sh
+++ b/bin/ebuild.sh
@@ -246,12 +246,14 @@ inherit() {
 		fi
 
 		for repo_location in ${PORTAGE_ECLASS_LOCATIONS[@]}; do
-			potential_location=${repo_location}/eclass/${1}.eclass
+		  for x in ${PORTAGE_ECLASS_VARIANTS:-} ''; do
+			potential_location=${repo_location}/eclass${x:+/}${x}/${1}.eclass
 			if [[ -f ${potential_location} ]]; then
 location=${potential_location}
 debug-print   eclass exists: ${location}
-break
+break 2
 			fi
+		  done
 		done
 		debug-print inherit: $1 - $location
 		[[ -z ${location} ]]  die ${1}.eclass could not be found by inherit()
-- 
1.8.3.2

Re: [gentoo-portage-dev] support multiple eclass variants (aka unstable eclasses)

[Michael Haubenwallner]

Am 2014-07-08 19:11, schrieb Sebastian Luther:
 Am 08.07.2014 18:58, schrieb Michael Haubenwallner:
 Hello fellow Portage developers,

 attached portage patch draft aims to allow for easy distributing eclasses to 
 be tested by
 multiple tinderboxes on various architectures, without being active for 
 normal installs.
 
 What does the patch allow you to do, that you can't do right now? (i.e.
 put an eclass with the same name in an repository and use
 eclass-overrides to force its use in another repo?)

Ohw, haven't been aware of that - will try first.

Thanks!
/haubi/

[gentoo-portage-dev] [PATCH] repoman: remove deprecation checks for eclasses no longer in the tree

[Michael Palimaka]

---
 pym/repoman/checks.py | 10 --
 1 file changed, 10 deletions(-)

diff --git a/pym/repoman/checks.py b/pym/repoman/checks.py
index 8032b28..abb9545 100644
--- a/pym/repoman/checks.py
+++ b/pym/repoman/checks.py
@@ -385,19 +385,9 @@ class InheritDeprecated(LineCheck):
boost-utils: False,
distutils: distutils-r1,
gems: ruby-fakegem,
-   git: git-2,
mono: mono-env,
-   mozconfig-2: mozconfig-3,
-   mozcoreconf: mozcoreconf-2,
-   php-ext-pecl-r1: php-ext-pecl-r2,
-   php-ext-source-r1: php-ext-source-r2,
-   php-pear: php-pear-r1,
python: python-r1 / python-single-r1 / python-any-r1,
-   python-distutils-ng: python-r1 + distutils-r1,
-   qt3: False,
-   qt4: qt4-r2,
ruby: ruby-ng,
-   ruby-gnome2: ruby-ng-gnome2,
x-modular: xorg-2,
}
 
-- 
1.8.3.2

[gentoo-portage-dev] [PATCH] Add the has function to the ebuild(5) man page.

[Michael Orlitzky]

I WTF'ed on this for a long time before I noticed that the docs for
has were sort-of contained in hasv. Might as well give has its own.
From 423123cc2ea429c06914ef858a6fdb86c0c6d30b Mon Sep 17 00:00:00 2001
From: Michael Orlitzky m...@gentoo.org
Date: Wed, 22 Jan 2014 16:18:23 -0500
Subject: [PATCH] Add the has function to the ebuild(5) man page.

---
 man/ebuild.5 | 8 
 1 file changed, 8 insertions(+)

diff --git a/man/ebuild.5 b/man/ebuild.5
index 524006a..36836b3 100644
--- a/man/ebuild.5
+++ b/man/ebuild.5
@@ -1049,6 +1049,14 @@ of \fI\-\-without\-\fR. Beginning with \fBEAPI 4\fR, an empty \fIconfigure
 opt\fR argument is recognized. In \fBEAPI 3\fR and earlier, an empty
 \fIconfigure opt\fR argument is treated as if it weren't provided.
 .TP
+.B has\fR \fIitem\fR \fIitem list
+If \fIitem\fR is in \fIitem list\fR, then \fBhas\fR returns
+0. Otherwise, 1 is returned. There is another version, \fBhasv\fR, that
+will conditionally echo \fIitem\fR.
+.br
+The \fIitem list\fR is delimited by the \fIIFS\fR variable.  This variable
+has a default value of ' ', or a space.  It is a \fBbash\fR(1) setting.
+.TP
 .B hasv\fR \fIitem\fR \fIitem list
 If \fIitem\fR is in \fIitem list\fR, then \fIitem\fR is echoed and \fBhasv\fR
 returns 0.  Otherwise, nothing is echoed and 1 is returned. As indicated with
-- 
1.8.3.2

Re: [gentoo-portage-dev] [RFC/PATCH] prepstrip/ecompressdir: parallelize operations

[Michael Haubenwallner]

On 05/11/2012 06:39 PM, Mike Frysinger wrote:
 +multijob_child_init() {
 + trap 'echo ${BASHPID} $? '${mj_control_fd} EXIT
 + trap 'exit 1' INT TERM
 +}

Just wondering why $! in parent isn't used anywhere, even not for some
integrity check if the child's BASHPID actually was forked by parent.

 +multijob_post_fork() {
 + : $(( ++mj_num_jobs ))
 + if [[ ${mj_num_jobs} -ge ${mj_max_jobs} ]] ; then
 + multijob_finish_one

Feels like ignoring this child's exitstatus isn't intentional here.

 + fi
 + return 0
 +}

/haubi/

Re: [gentoo-portage-dev] EbuildProcess logs poll-error to already removed $T (on AIX)

[Michael Haubenwallner]

On 03/25/2011 05:23 PM, Zac Medico wrote:
  * EbuildProcess received strange poll event: 16384

 You can compare 16384 to the values of POLLERR and POLLNVAL in order to
 see what type of event it is. Apparently the values on AIX are different
 from those on Linux, because here's what I see on Linux:

On AIX 5.3 this is:

Python 2.7.1 (r271:86832, Feb 28 2011, 17:51:02) 
[GCC 4.2.4 (Gentoo 4.2.4-r01.2 p1.1)] on aix5
Type help, copyright, credits or license for more information.
 import select
 dir(select)
['PIPE_BUF', 'POLLERR', 'POLLHUP', 'POLLIN', 'POLLMSG', 'POLLNVAL', 'POLLOUT',
'POLLPRI', 'POLLRDBAND', 'POLLRDNORM', 'POLLWRBAND', 'POLLWRNORM', '__doc__',
'__file__', '__name__', '__package__', 'error', 'poll', 'select']
 select.POLLNVAL
32768
 select.POLLERR
16384

On AIX 6.1 it looks similar except for missing 'PIPE_BUF'.

 This will handle the IOError:
 http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=0a64f784003c11e151405b7f708d0de0ed57

Yes, that makes it work, thank you!

 It might be risky to skip logging of the POLLNVAL / POLLERR events, so
 hopefully we can determine their cause and handle them somehow. Do they
 seem to cause any problems? It might be something specific about pty
 devices on AIX.

There doesn't seem to go anything wrong so far.

I've no idea about programming with pty devices at all.
However, one relevant (IMHO) thing I can see is:
portage/util/_pty.py:_can_test_pty_eof() returns True for Linux only.

Anything I can try out?

Thank you!
/haubi/
-- 
Michael Haubenwallner
Gentoo on a different level

[gentoo-portage-dev] EbuildProcess logs poll-error to already removed $T (on AIX)

[Michael Haubenwallner]

Hi Zac (et al),

while this problem occurs on AIX only (for now?), I doubt this problem is
introduced in prefix-portage.

With recent prefix-portage-2.2.01.18125 (Fabian, how do you calculate the
version numbers since moving to git?), the EbuildProcess spits this
every now and then during emerge mime-types fex:

 * EbuildProcess received strange poll event: 16384

While I don't understand (yet) why this is there on AIX at all, it does
trigger an IOError when trying to log this message to $T/build.log after
$WORKDIR has been cleaned up. When I avoid the logging of this message,
everything (seems to) work fine.

For the attached logfile, I've added these two lines to 
usr/lib/portage/bin/ebuild.sh:
@@ -1,3 +1,5 @@
 #!/big5tk/local/gprefix/bin/bash
 # Copyright 1999-2011 Gentoo Foundation
 # Distributed under the terms of the GNU General Public License v2
+echo ebuild.sh: $0 $@ 2
+echo ebuild.sh: WORKDIR: ${WORKDIR} 2
@@

Any idea?

Thank you!
/haubi/
-- 
Michael Haubenwallner
Gentoo on a different level
WARNING: One or more repositories have missing repo_name entries:

/big5tk/local/prefix-overlay/profiles/repo_name
/big5tk/local/gentoo-x86/profiles/repo_name

NOTE: Each repo_name entry should be a plain text file containing a
unique name for the repository on the first line.



 * IMPORTANT: 1 news items need reading for repository 'gentoo_prefix'.
 * Use eselect news to read news items.

Calculating dependencies  ... done!

 Verifying ebuild manifests

 Emerging (1 of 1) app-misc/mime-types-8
ebuild.sh: /big5tk/local/gprefix/usr/lib/portage/bin/ebuild.sh clean
ebuild.sh: WORKDIR: /big5tk/tmp/gprefix/portage/app-misc/mime-types-8/work
 * EbuildProcess received strange poll event: 16384

 * mime-types-8.tar.bz2 RMD160 SHA1 SHA256 size ;-) ...  [ ok ]
 * Package:app-misc/mime-types-8
 * Repository: gentoo_prefix
 * Maintainer: d...@gentoo.org net-m...@gentoo.org
 * USE:elibc_AIX kernel_AIX ppc-aix prefix userland_GNU
 * FEATURES:   nostrip preserve-libs
ebuild.sh: /big5tk/local/gprefix/usr/lib/portage/bin/ebuild.sh setup
ebuild.sh: WORKDIR: /big5tk/tmp/gprefix/portage/app-misc/mime-types-8/work
 * EbuildProcess received strange poll event: 16384

ebuild.sh: /big5tk/local/gprefix/usr/lib/portage/bin/ebuild.sh unpack
ebuild.sh: WORKDIR: /big5tk/tmp/gprefix/portage/app-misc/mime-types-8/work
 Unpacking source...
 Unpacking mime-types-8.tar.bz2 to 
 /big5tk/tmp/gprefix/portage/app-misc/mime-types-8/work
 Source unpacked in /big5tk/tmp/gprefix/portage/app-misc/mime-types-8/work
 * EbuildProcess received strange poll event: 16384

ebuild.sh: /big5tk/local/gprefix/usr/lib/portage/bin/ebuild.sh compile
ebuild.sh: WORKDIR: /big5tk/tmp/gprefix/portage/app-misc/mime-types-8/work
 Compiling source in 
 /big5tk/tmp/gprefix/portage/app-misc/mime-types-8/work/mime-types-8 ...
 Source compiled.
 * EbuildProcess received strange poll event: 16384

ebuild.sh: /big5tk/local/gprefix/usr/lib/portage/bin/ebuild.sh test
ebuild.sh: WORKDIR: /big5tk/tmp/gprefix/portage/app-misc/mime-types-8/work
 Test phase [not enabled]: app-misc/mime-types-8
 * EbuildProcess received strange poll event: 16384

ebuild.sh: /big5tk/local/gprefix/usr/lib/portage/bin/ebuild.sh install
ebuild.sh: WORKDIR: /big5tk/tmp/gprefix/portage/app-misc/mime-types-8/work

 Install mime-types-8 into 
 /big5tk/tmp/gprefix/portage/app-misc/mime-types-8/image/big5tk/local/gprefix/
  category app-misc
 Completed installing mime-types-8 into 
 /big5tk/tmp/gprefix/portage/app-misc/mime-types-8/image/big5tk/local/gprefix/

 * EbuildProcess received strange poll event: 16384

ebuild.sh: /big5tk/local/gprefix/usr/lib/portage/bin/misc-functions.sh 
ebuild.sh: WORKDIR: /big5tk/tmp/gprefix/portage/app-misc/mime-types-8/work
 * MiscFunctionsProcess received strange poll event: 16384


 Installing (1 of 1) app-misc/mime-types-8
ebuild.sh: /big5tk/local/gprefix/usr/lib/portage/bin/ebuild.sh preinst
ebuild.sh: WORKDIR: /big5tk/tmp/gprefix/portage/app-misc/mime-types-8/work
 * EbuildProcess received strange poll event: 16384

ebuild.sh: /big5tk/local/gprefix/usr/lib/portage/bin/misc-functions.sh 
ebuild.sh: WORKDIR: /big5tk/tmp/gprefix/portage/app-misc/mime-types-8/work
 * MiscFunctionsProcess received strange poll event: 16384

ebuild.sh: /big5tk/local/gprefix/usr/lib/portage/bin/ebuild.sh prerm
ebuild.sh: WORKDIR: 
/big5tk/tmp/gprefix/portage/._unmerge_/app-misc/mime-types-8/work
 * EbuildProcess received strange poll event: 16384

ebuild.sh: /big5tk/local/gprefix/usr/lib/portage/bin/ebuild.sh postrm
ebuild.sh: WORKDIR: 
/big5tk/tmp/gprefix/portage/._unmerge_/app-misc/mime-types-8/work
 * EbuildProcess received strange poll event: 16384

ebuild.sh: /big5tk/local/gprefix/usr/lib/portage/bin/ebuild.sh clean
ebuild.sh: WORKDIR: 
/big5tk/tmp/gprefix/portage/._unmerge_/app-misc/mime-types-8/work
 * EbuildProcess received strange poll event: 16384


 * Messages for package app-misc/mime-types-8:
 * EbuildProcess received strange poll event

[gentoo-portage-dev] Re: Re: Patch problem

[Michael]

Zac Medico wrote:
 Inside depgraph._resolve(), it returns False if
 self._dynamic_config._needed_use_config_changes is non-empty. You want
 to add an option that causes it to return True instead. Also, you'll
 need to propagate these changes to the config.setcpv() method somehow,
 so that the changes will be applied by the Scheduler at build time.
 Normally, the config.setcpv() method calculates USE based on config
 files, but you want it override the config files with whatever values
 the depgraph's _needed_use_config_changes contains.

I've been able to collect the list of flags required to satisfy the 
dependency and apply them, but I'm having difficulty with ensuring that they 
do not override the user's settings.

eg. need +foo to satisfy the dependency, but do not do it it USE=-foo.

As suggested by yourself, portage.settings.configdict[conf][USE] indeed 
does contain explicit enable/disable information from make.conf, however the 
other keys, such as those representing the env or package.use do not.

How can I get this information?


Thanks,
Michael