Re: [gentoo-dev] Repository stacking and complementary overlays
On Mon, 2009-03-02 at 16:48 +, Ciaran McCreesh wrote: On Fri, 27 Feb 2009 04:41:23 +0200 Mart Raudsepp l...@gentoo.org wrote: So here the reverting of a masking in gentoo-x86 is quite intentional and currently desired. This is fundamentally broken as a concept. Adding an overlay should not have any impact upon other repositories. Adding an overlay conceptually and fundamentally and per dictionary means laying some packages over something else, PORTDIR in this case - the main repository and therefore adding an overlay impacts other repositories - at least the main one. This is how it has always worked and I do not understand why the behaviour should suddenly change to mean something illogical to the term. It should be possible for a user to add an overlay, and make limited use of that repository, without having to worry that the mere act of adding that overlay will make massive changes to what's visible in other repositories. Perhaps the package manager should add such a support then with PORTDIR_PREVIEW_OVERLAY or some such if you want your users to be able to have the overlay VCS checkout and addition to PORDIR_OVERLAY or the like to be one operation. Overlays shouldn't be altering the visibility of things outside of that overlay without explicit user action. Can you repeat the technically sound reasoning to that again please or point to exact archived posts? This discussion has been going on in other mediums as well, I might have missed the core points. By this snippet we could simply move the current relevant maskings from profiles/package.mask to profiles/base/package.mask and call it a day (and screw over the few profiles that don't end up parenting base/), as QA forced us to do in case of per-arch mask negations in gentoo-x86 a while back. But it doesn't seem to be as simple as that. Well no, because profiles/base/ in your overlay is entirely unrelated to profiles/base/ in the master. Only reason it flies for portage is because it collapses it all into one stack; for managers designed to support multiple standalone repos that assumption no longer applies, thus that behaviour (outside of PMS) breaks. Last I knew the official council approved PMS was meant to describe portage behaviour at the time, which appears to have been the same along the way - treating all overlays in the same stack as PORTDIR, perhaps as there is no means to declare a different stack. PMS does not attempt to document Portage behaviour in the cases where Portage behaviour is dumb. That's the reason there's as little as possible mentioned regarding overlays there -- Portage's overlay model is a horrible hack, and forcing package managers to implement it rather than offering a true multiple repository model would be a serious hit on usability. The way forward here is to identify what you're trying to achieve, whilst ignoring how things are currently defined or what is or is not possible. Then we can look at that and work out whether it can be mapped to an existing solution or some easily-implementable new solution. Starting with implementation is the wrong approach. I'm trying to achieve that merely adding an overlay on top of gentoo-x86 repository actually adds that overlay and things work as desired in regards to visibility. In related reasons, we need to do the unmasking of things masked in main repository because the masks there affect the visibility of packages in the overlay by masking those as well. Perhaps that's the actual core problem for us, if playing along with the notion of current Portage behaviour being dumb (Where do I read how it is dumb and how it'd be better and what a true multiple repository model would be like?) Regards, Mart Raudsepp
Re: [gentoo-dev] Repository stacking and complementary overlays
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
The problem of ebuilds in one overlay not seeing ebuilds in another overlay,
would also be solved by the package manager NOT failing to see/notice/use/allow
ebuilds from all installed overlays. Then there would be no need for a hierarchy
among overlays.
Marijn
- --
Sarcasm puts the iron in irony, cynicism the steel.
Marijn Schouten (hkBst), Gentoo Lisp project, Gentoo ML
http://www.gentoo.org/proj/en/lisp/, #gentoo-{lisp,ml} on FreeNode
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iEYEARECAAYFAkmvp1IACgkQp/VmCx0OL2wFqACfSTjfdtY7abG/yc06gW4Q+YlT
Nd0AoIYIwFBD4BEKoA/PVg35K4Sia+C4
=IWat
-END PGP SIGNATURE-
Re: [gentoo-dev] Repository stacking and complementary overlays
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Marijn Schouten (hkBst) wrote:
The problem of ebuilds in one overlay not seeing ebuilds in another overlay,
would also be solved by the package manager NOT failing to
see/notice/use/allow
ebuilds from all installed overlays. Then there would be no need for a
hierarchy
among overlays.
Marijn
It was pointed out to me by zlin that this does not help with installing
`missing' `master' overlays. This is a good point. However it is possible that
even though an overlay is missing, ebuilds from another installed overlay or
main tree or a local overlay can satisfy dependencies. Further it is possible
that two overlays have a mutual dependency on eachother.
Maybe these possibilities are only theoretical.
Marijn
- --
Sarcasm puts the iron in irony, cynicism the steel.
Marijn Schouten (hkBst), Gentoo Lisp project, Gentoo ML
http://www.gentoo.org/proj/en/lisp/, #gentoo-{lisp,ml} on FreeNode
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iEYEARECAAYFAkmvxGMACgkQp/VmCx0OL2x2+ACgh6ESoDDWhW1DCFnc2bz1VItw
xFIAoLuSQt1G42MR5umWrCoZwGD3thbG
=XnsF
-END PGP SIGNATURE-
Re: [gentoo-dev] Repository stacking and complementary overlays
2009/3/5 Marijn Schouten (hkBst) hk...@gentoo.org: The problem of ebuilds in one overlay not seeing ebuilds in another overlay, would also be solved by the package manager NOT failing to see/notice/use/allow ebuilds from all installed overlays. Then there would be no need for a hierarchy among overlays. Not sure if this has been mentioned already, but why not making the profile only affect the corresponding repository and allow repository aware configuration (make.conf and /etc/portage/* in case of portage being the package manager)? That way repository based masking and unmasking would be possible. -- Regards, Daniel
Re: [gentoo-dev] Repository stacking and complementary overlays
Le lundi 02 mars 2009 à 23:59 +, Ciaran McCreesh a écrit : On Tue, 03 Mar 2009 00:55:38 +0100 Gilles Dartiguelongue e...@gentoo.org wrote: We didn't implement anything but let's just talk about what we wanted to see. We simply wanted overlay users to keep testing gnome 2.24 components that were masked or using masked packages in base/package.mask so we just made sure those packages had the proper keyword visibility. So if you could mask 'testing'ish things that're in the overlay (already possible), well there was nothing masked in the overlay really, for 2.24, because the overlay is for testing stuff already. Let me try to rephrase and see if I can make it clearer. Although we moved some packages to the tree, some packages didn't ended up moving because they needed more testing or huge regression fixing patches but they were masked by gentoo-x86 masks because of their dependencies. Hence the mask reverting in overlay. and provide an easy, consistent way for users who choose to to unmask and keyword a particular group of packages, would that solve the problem? wrt to previous point, it would probably help a great deal even though I'm not sure it would completely solve this case. -- Gilles Dartiguelongue e...@gentoo.org Gentoo
Re: [gentoo-dev] Repository stacking and complementary overlays
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Jorge Manuel B. S. Vicetto wrote: This seems desirable and reasonable. As I replied to this subject earlier regarding KDE, let me complement that information. In the case of the KDE team, we keep work on a release all in the same place, so we don't need to unmask some KDE packages in tree for those using the overlay. However, we some times have deps on packages from other teams and or in other overlays, so I hope the repo deps would help here (not to unmask those packages, if they're masked, but to add a dep on a particular repo and allowing the PM explain to the user that he/she needs to unmask a particular version in the tree / overlay). The problem with repo deps is that they're too restrictive since they assume that only a specific repo can satisfy the dep. Suppose that you migrate some of the packages from the overlay to the main tree? Now you've got installed packages that are trying to pull in deps from the wrong repo. Or suppose that somebody else has an overlay with a compatible package? I think a better way to reference another repo is with the layout.conf approach suggested in the QA Overlay Layout support thread [1]. For example, if packages from the java-experimental repo depend on some ebuilds or eclasses from the java-overlay repo, it's specified via a masters entry in layout.conf. If any of those ebuilds/eclasses happen to migrate to the main tree then the migration is seamless. [1] http://archives.gentoo.org/gentoo-dev/msg_33c61550b4ed2b7b25dd5a4110e1ec81.xml - -- Thanks, Zac -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.10 (GNU/Linux) iEYEARECAAYFAkmvMcIACgkQ/ejvha5XGaNmkwCePw/XjWCqZtIXq5yXQ4gpHALL fXUAoMqkmJ30Go2SJaqS2lzs+8axyLwn =ju66 -END PGP SIGNATURE-
Re: [gentoo-dev] Repository stacking and complementary overlays
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Zac Medico wrote: Jorge Manuel B. S. Vicetto wrote: This seems desirable and reasonable. As I replied to this subject earlier regarding KDE, let me complement that information. In the case of the KDE team, we keep work on a release all in the same place, so we don't need to unmask some KDE packages in tree for those using the overlay. However, we some times have deps on packages from other teams and or in other overlays, so I hope the repo deps would help here (not to unmask those packages, if they're masked, but to add a dep on a particular repo and allowing the PM explain to the user that he/she needs to unmask a particular version in the tree / overlay). The problem with repo deps is that they're too restrictive since they assume that only a specific repo can satisfy the dep. Suppose that you migrate some of the packages from the overlay to the main tree? Now you've got installed packages that are trying to pull in deps from the wrong repo. Or suppose that somebody else has an overlay with a compatible package? I agree repo deps might be restrictive, but in some cases we might really want to be restrictive. For instance, I might want to have a dep on a package in the GNOME official overlay - I might not want to get the version in some other overlay listed in layman. When the package moves to the tree or if work moves to another overlay, then sure, it will mean more work. But sometimes that might be preferred. I think a better way to reference another repo is with the layout.conf approach suggested in the QA Overlay Layout support thread [1]. For example, if packages from the java-experimental repo depend on some ebuilds or eclasses from the java-overlay repo, it's specified via a masters entry in layout.conf. If any of those ebuilds/eclasses happen to migrate to the main tree then the migration is seamless. I like this proposal as well. I think repo deps and master layouts are complementary and not alternatives. Getting back to KDE (sorry for being so self-centered), we would make the kde-experimental overlay have kde-testing as the master overlay. So we could do some work in eclasses in kde-testing without needing to copy them to kde-experimental. [1] http://archives.gentoo.org/gentoo-dev/msg_33c61550b4ed2b7b25dd5a4110e1ec81.xml - -- Regards, Jorge Vicetto (jmbsvicetto) - jmbsvicetto at gentoo dot org Gentoo- forums / Userrel / Devrel / SPARC / KDE -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkmvNhUACgkQcAWygvVEyAJTDwCfeFwqVMtUY4r+BR5mX6kyGhuk tTAAoJG0kQksJnVM6Fd/WwPHxJmBJZoi =eSaw -END PGP SIGNATURE-
Re: [gentoo-dev] Repository stacking and complementary overlays
On Fri, 27 Feb 2009 04:41:23 +0200 Mart Raudsepp l...@gentoo.org wrote: So here the reverting of a masking in gentoo-x86 is quite intentional and currently desired. This is fundamentally broken as a concept. Adding an overlay should not have any impact upon other repositories. It should be possible for a user to add an overlay, and make limited use of that repository, without having to worry that the mere act of adding that overlay will make massive changes to what's visible in other repositories. Overlays shouldn't be altering the visibility of things outside of that overlay without explicit user action. By this snippet we could simply move the current relevant maskings from profiles/package.mask to profiles/base/package.mask and call it a day (and screw over the few profiles that don't end up parenting base/), as QA forced us to do in case of per-arch mask negations in gentoo-x86 a while back. But it doesn't seem to be as simple as that. Well no, because profiles/base/ in your overlay is entirely unrelated to profiles/base/ in the master. Only reason it flies for portage is because it collapses it all into one stack; for managers designed to support multiple standalone repos that assumption no longer applies, thus that behaviour (outside of PMS) breaks. Last I knew the official council approved PMS was meant to describe portage behaviour at the time, which appears to have been the same along the way - treating all overlays in the same stack as PORTDIR, perhaps as there is no means to declare a different stack. PMS does not attempt to document Portage behaviour in the cases where Portage behaviour is dumb. That's the reason there's as little as possible mentioned regarding overlays there -- Portage's overlay model is a horrible hack, and forcing package managers to implement it rather than offering a true multiple repository model would be a serious hit on usability. The way forward here is to identify what you're trying to achieve, whilst ignoring how things are currently defined or what is or is not possible. Then we can look at that and work out whether it can be mapped to an existing solution or some easily-implementable new solution. Starting with implementation is the wrong approach. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Repository stacking and complementary overlays
Le lundi 02 mars 2009 à 16:48 +, Ciaran McCreesh a écrit : On Fri, 27 Feb 2009 04:41:23 +0200 Mart Raudsepp l...@gentoo.org wrote: So here the reverting of a masking in gentoo-x86 is quite intentional and currently desired. This is fundamentally broken as a concept. Adding an overlay should not have any impact upon other repositories. It should be possible for a user to add an overlay, and make limited use of that repository, without having to worry that the mere act of adding that overlay will make massive changes to what's visible in other repositories. Overlays shouldn't be altering the visibility of things outside of that overlay without explicit user action. well that's unfortunate that it doesn't fit that view but that's still what was desired. See next block for details. The way forward here is to identify what you're trying to achieve, whilst ignoring how things are currently defined or what is or is not possible. Then we can look at that and work out whether it can be mapped to an existing solution or some easily-implementable new solution. Starting with implementation is the wrong approach. We didn't implement anything but let's just talk about what we wanted to see. We simply wanted overlay users to keep testing gnome 2.24 components that were masked or using masked packages in base/package.mask so we just made sure those packages had the proper keyword visibility. -- Gilles Dartiguelongue e...@gentoo.org Gentoo signature.asc Description: Ceci est une partie de message numériquement signée
Re: [gentoo-dev] Repository stacking and complementary overlays
On Tue, 03 Mar 2009 00:55:38 +0100 Gilles Dartiguelongue e...@gentoo.org wrote: We didn't implement anything but let's just talk about what we wanted to see. We simply wanted overlay users to keep testing gnome 2.24 components that were masked or using masked packages in base/package.mask so we just made sure those packages had the proper keyword visibility. So if you could mask 'testing'ish things that're in the overlay (already possible), and provide an easy, consistent way for users who choose to to unmask and keyword a particular group of packages, would that solve the problem? -- Ciaran McCreesh signature.asc Description: PGP signature
[gentoo-dev] Repository stacking and complementary overlays
A. On K, 2009-02-25 at 04:56 -0800, Brian Harring wrote:
On Wed, Feb 25, 2009 at 01:42:38PM +0100, Gilles Dartiguelongue wrote:
Le mardi 24 février 2009 à 09:47 -0800, Brian Harring a écrit :
On Sun, Feb 22, 2009 at 11:26:48PM -0800, Donnie Berkholz wrote:
This is your friendly reminder! Same bat time (typically the 2nd 4th
Thursdays at 2000 UTC / 1600 EST), same bat channel (#gentoo-council @
irc.freenode.net) !
Informal request, but it would be useful to get an idea of the
councils views on portage overlay compatibility issues.
Specifically, when it comes to gentoo repositories, there is one, and
only one definition of what that is- pms's repo spec. The problem
here is that the only repository truly conformant to that spec is
gentoo-x86, for the rest of the repositories (overlays realistically)
whatever portage supports seems to be the eventual standard they grow
towards.
Problem with this is that there is *zero* way to spot these non-pms
repositories as it stands. Simplest example, under portage overlays
can unmask pkgs globally (gnome overlay reverting masks in
gentoo-x86),
I reply here as part of the gnome herd and partly responsible for the
mask reverting in the overlay. I didn't know something used in
gentoo-x86 couldn't be used in an overlay.
Suspect I wasn't clear; you *can* use things from the parent (although
that whole relationship is outside of PMS); the problem here is that
y'all are reverting something in the *master*.
Literally, bug-buddy was masked in gentoo-x86; enabling your overlay
reverts that masking in *gentoo-x86*. Only reason this even works is
due to portage internals being limited (everything is stacked
together, no true standalones possible).
So here the reverting of a masking in gentoo-x86 is quite intentional
and currently desired. The problem is that
a) most importantly - no way to explicitly express a complementary
overlay to a given repository
b) less importantly - gentoo-x86 has no true base profile where to mask
things, as base/ isn't really the base of everything, and the global
package.mask is for some reason defined as something different from a
really base profile (global mask negation disallowed even in arch
profiles of the same repository, etc)
This mail thread is about a)
Could you point me to the PMS section that treat this ?
Flip through the package.mask section (snagging from profiles.tex
directly)-
Note that the \t{-spec} syntax can be used to remove a mask in a
parent profile, but not necessarily a global mask (from
\t{profiles/package.mask}, section~\ref{profiles-package.mask}).
\note Portage currently treats \t{profiles/package.mask} as being on
the leftmost branch of the inherit tree when it comes to \t{-lines}.
This behaviour may not be relied upon.
By this snippet we could simply move the current relevant maskings from
profiles/package.mask to profiles/base/package.mask and call it a day
(and screw over the few profiles that don't end up parenting base/), as
QA forced us to do in case of per-arch mask negations in gentoo-x86 a
while back.
But it doesn't seem to be as simple as that.
The wording but not _necessarily_ a global mask is quite inconclusive
as well.
Note the 'parent profile'. Why they're claiming repo level masking
can't be reversed for that repo, not sure (reasonably sure several
profiles rely on it). Either way, your overlay is trying to revert
entries it doesn't have in that stack.
We'd like the stack to be the same as gentoo-x86. It is a complementary
overlay to gentoo-x86 and definitely not a separate stand-alone
repository.
Only reason it flies for portage is because it collapses it all into
one stack; for managers designed to support multiple standalone repos
that assumption no longer applies, thus that behaviour (outside of
PMS) breaks.
Last I knew the official council approved PMS was meant to describe
portage behaviour at the time, which appears to have been the same along
the way - treating all overlays in the same stack as PORTDIR, perhaps
as there is no means to declare a different stack.
There are claims that this behaviour (everything in the same stack) is
simply a bug - I dare to disagree and claim instead that this behaviour
is simply about defaulting PORTDIR_OVERLAYs to be complementary to
PORTDIR (as the relation in the variable names quite clearly and
logically states!) and therefore in the same stack and that we need a
different way to declare a different stack and standalone repository.
I also claim other PMs than portage should default to extending the
PORTDIR repo_name stack as well with overlay entries in
PORTDIR_OVERLAY instead of making a new stack as they do now. An
PORTDIR_OVERLAY overlay is an overlay on top of PORTDIR and therefore in
the same stack, not a new repository that gets a new stack.
With a very quick thought I see some possible ways to make
