Re: Re : Re: [gentoo-portage-dev] precisions on installed packages' dependencies

2020-03-23 Thread Zac Medico
On 3/23/20 3:21 PM, michael.lienha...@laposte.net wrote:
> - Zac Medico  a écrit :
> 
>>>  3. before removing a library, "ebuild unmerge" always checks if it is used 
>>> by another package: this means that installed packages' dependencies are 
>>> never broken.
>>
>> That's true if the package is removed via emerge --depclean, but emerge
>> --unmerge does not account for dependencies.
>>
>> Also, it's possible for dependencies of installed packages to be
>> temporarily broken by upgrades. In cases like this, the breakage will
>> eventually be resolved by a rebuild (which occurs automatically for slot
>> operator := deps), upgraded, or by emerge --depclean (which removes
>> unneeded packages).
> 
> Many thanks for your answers.
> They made me realize that the problem I'm facing is a bit more tricky than I 
> first quickly though.
> 
> I'll try to explain the problem, tell me if I'm not clear somewhere.
> 
> The goal of my tool is to have correct manipulation of package dependencies, 
> and in particular here, I focus on the packages that are installed but not in 
> the portage tree/a local overlay anymore (the problem does not occur for 
> other packages).
> It seems that installed packages do not store which are the actual cpv they 
> depend on. Correct?

Right, because unfortunately that's something that changes over time.

Also, we may not be able to pin it down at any given moment if we have
inconsistent || preferences as described here:

https://archives.gentoo.org/gentoo-dev/message/550d3859dea6d0fb0b39064628992634

> Hence, when an installed package cannot be updated/recompiled because it is 
> not in the tree anymore, like you said, its dependencies can be broken (due 
> to the package it depends on being updated).
> Currently, this issue is circumvented (only using depclean) by keeping the 
> libs: the package's dependencies are broken, but it's ok because it can still 
> run (which, in the end of the day, is what we want).
> However, from your answer, it seems that this fix is not entirely integrated 
> in the emerge/portage toolchain (like you said, emerge --unmerge removes 
> everything, and emerge -u removes the old libs).
> 
> To sum up, the problem I'm facing is that with the current way installed 
> packages are managed, we can break dependencies (and the only way to fix them 
> is to remove the installed package with the broken dependencies, that can 
> never be installed again).
> 
> Hence, for my tool, I have two solutions for that problem: either I forbid 
> for dependencies to ever be broken, or I allow it.
> 
> Solution 1: forbid broken dependencies.
> This requires to extend the information stored on installed package with the 
> list of the actual cpvs they depend (or at least the cp+slot, which is enough 
> to get back the cpvs).
> That way, I can say in the solver "if you want to keep that package, you need 
> to keep these packages as they currently are".
> However, I have no idea on how to do that, and doing this only for my tool 
> would mean that one cannot switch between emerge (quick) and my tool 
> (correct), which is a feature I think is essential.
> Do you think adding this new information to installed packages could be 
> integrated into emerge/portage itself? I could work on it (expect question 
> ^^), test it on my prototype, and do a pull request when everything's working.

It's possible to store this information in a cache of the most recently
calculated dependency graph.

> Solution 2: allow broken dependencies.
> Here, the idea is to use the same fix as is currently done with depclean, but 
> in my tool's planner (i.e., the part that install/unistall the packages) 
> directly.
> That way, I say in the solver "that installed package has no dependency", but 
> when I upgrade/remove packages, I say "Oh but wait, that other package still 
> need these libs, let's keep them".
> This solution may not require any change in portage/emerge, but I have no 
> idea on how to know which libs are needed by a package, and how to track 
> these libs owners without looking at every installed package's files (which 
> are stored in the CONTENT file, if I'm not mistaken).

For soname dependencies, we've got PROVIDES and REQUIRES metadata.

> Also, I wanted to use the ebuild tool to install/uninstall package, which is 
> not possible with this solution apparently.

Why not? Would the preserve-libs feature solve your problem?

> In case I need to implement this, could you give me some clue on how to 
> achieve it?
> 
> 
> Among these two solutions, I prefer the first one: we stay at the level of 
> package dependencies (and it looks simpler to implement).
> However, it is maybe easier/better to use the second approach, I don't know.
> Do you have some suggestions?

Well, there are a lot of upgrades that can't be performed without
temporarily breaking something, so the "forbid broken dependencies" idea
doesn't sound feasible to me.

> Thanks!
> Michael
-- 
Thanks,
Zac




Re : Re: [gentoo-portage-dev] precisions on installed packages' dependencies

2020-03-23 Thread michael . lienhardt
- Zac Medico  a écrit :

> >  3. before removing a library, "ebuild unmerge" always checks if it is used 
> > by another package: this means that installed packages' dependencies are 
> > never broken.
> 
> That's true if the package is removed via emerge --depclean, but emerge
> --unmerge does not account for dependencies.
> 
> Also, it's possible for dependencies of installed packages to be
> temporarily broken by upgrades. In cases like this, the breakage will
> eventually be resolved by a rebuild (which occurs automatically for slot
> operator := deps), upgraded, or by emerge --depclean (which removes
> unneeded packages).

Many thanks for your answers.
They made me realize that the problem I'm facing is a bit more tricky than I 
first quickly though.

I'll try to explain the problem, tell me if I'm not clear somewhere.

The goal of my tool is to have correct manipulation of package dependencies, 
and in particular here, I focus on the packages that are installed but not in 
the portage tree/a local overlay anymore (the problem does not occur for other 
packages).
It seems that installed packages do not store which are the actual cpv they 
depend on. Correct?
Hence, when an installed package cannot be updated/recompiled because it is not 
in the tree anymore, like you said, its dependencies can be broken (due to the 
package it depends on being updated).
Currently, this issue is circumvented (only using depclean) by keeping the 
libs: the package's dependencies are broken, but it's ok because it can still 
run (which, in the end of the day, is what we want).
However, from your answer, it seems that this fix is not entirely integrated in 
the emerge/portage toolchain (like you said, emerge --unmerge removes 
everything, and emerge -u removes the old libs).

To sum up, the problem I'm facing is that with the current way installed 
packages are managed, we can break dependencies (and the only way to fix them 
is to remove the installed package with the broken dependencies, that can never 
be installed again).

Hence, for my tool, I have two solutions for that problem: either I forbid for 
dependencies to ever be broken, or I allow it.

Solution 1: forbid broken dependencies.
This requires to extend the information stored on installed package with the 
list of the actual cpvs they depend (or at least the cp+slot, which is enough 
to get back the cpvs).
That way, I can say in the solver "if you want to keep that package, you need 
to keep these packages as they currently are".
However, I have no idea on how to do that, and doing this only for my tool 
would mean that one cannot switch between emerge (quick) and my tool (correct), 
which is a feature I think is essential.
Do you think adding this new information to installed packages could be 
integrated into emerge/portage itself? I could work on it (expect question ^^), 
test it on my prototype, and do a pull request when everything's working.

Solution 2: allow broken dependencies.
Here, the idea is to use the same fix as is currently done with depclean, but 
in my tool's planner (i.e., the part that install/unistall the packages) 
directly.
That way, I say in the solver "that installed package has no dependency", but 
when I upgrade/remove packages, I say "Oh but wait, that other package still 
need these libs, let's keep them".
This solution may not require any change in portage/emerge, but I have no idea 
on how to know which libs are needed by a package, and how to track these libs 
owners without looking at every installed package's files (which are stored in 
the CONTENT file, if I'm not mistaken).
Also, I wanted to use the ebuild tool to install/uninstall package, which is 
not possible with this solution apparently.
In case I need to implement this, could you give me some clue on how to achieve 
it?


Among these two solutions, I prefer the first one: we stay at the level of 
package dependencies (and it looks simpler to implement).
However, it is maybe easier/better to use the second approach, I don't know.
Do you have some suggestions?

Thanks!
Michael