Re: [gentoo-dev] new profile layout with flavors and mix-ins
On Thu, Jul 3, 2014 at 7:09 PM, Tom Wijsman wrote: > Did the Funtoo devs tell you that they don't run repoman because of the > explosive set of possible combinations that flavors & mix-ins introduce? > > Having it run over them should be easy; but having it run within a > reasonable time when scaling up is going to be quite painful, ... I don't think we need to check all the combinations. To keep things manageable I think we'd want to define some structure around the profiles and what different kinds of profiles should and shouldn't do. If a profile just pulls in some packages and sets some application-oriented USE flags we shouldn't really need to worry about conflicts - if there are any users will either learn not to run them in combination or at least they won't end up with a completely cripped system if some non-core applications have issues.Messing with ABIs or system-wide configuration settings should be reserved for specific types of profiles which wouldn't be run in combinations unless carefully controlled. We should definitely avoid the mess of inheritance that leads to settings being toggled back and forth. Even if we still end up being stuck with a kernel/arch/subarch hierarchy we'll at least be able to ditch layering on things like kde/gnome further down and open the door to having more variety in our profiles at that level. Something like x32 coming along is fairly rare, even if it is really interesting (and this stuff is one of Gentoo's strong points, actually). There is probably a lot more interest in having more application-level mix-ins in the main tree, or especially in overlays. The reason we avoid them is that today they'd lead to 300 more profiles on top of what is already a big mess. If that part of the tree gets flattened there is no reason somebody could have a minimal profile that doesn't stick openssh in @system, or a bunch of server profiles for various use cases, or some profiles for clusters, config-managed boxes, etc. So, if we come up with a decent strategy, deciding what repoman does and doesn't have to worry about would be part of it. We obviously can't check all the combos, but that doesn't mean that we can't make sure that linux/x86/foo doesn't break. Rich
Re: [gentoo-dev] new profile layout with flavors and mix-ins
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Wed, 02 Jul 2014 14:41:03 -0400 "Rick \"Zero_Chaos\" Farina" wrote: > I've talked to the funtoo devs a few times about stealing their > profile idea. A few things need to be done to make this happen, > mainly eselect, catalyst, and repoman support. I can do the eselect > and catalyst changes myself, however, I'll require some help for the > repoman support most likely. I'll prototype this locally, see what I > can make work, and then see about making it usable for others to test. Did the Funtoo devs tell you that they don't run repoman because of the explosive set of possible combinations that flavors & mix-ins introduce? Having it run over them should be easy; but having it run within a reasonable time when scaling up is going to be quite painful, ... It sounds like a trade-off between better profiles and better repoman. - -- With kind regards, Tom Wijsman (TomWij) Gentoo Developer E-mail address : tom...@gentoo.org GPG Public Key : 6D34E57D GPG Fingerprint : C165 AF18 AB4C 400B C3D2 ABF0 95B2 1FCD 6D34 E57D -BEGIN PGP SIGNATURE- Version: GnuPG v2 iQEcBAEBAgAGBQJTteLDAAoJEPWZc8roOL/QXbIIAJpM1QZhfu51deklcxIi+6PJ 6yxNc1CA6eJfhQouGYR2QgEHUI4YASEz0invFpXbv4IM/NYY/JSX/e+EOoykqcub eN9ZIFcUOdnuP7GnG+tVOe0oMJ0jhj7yEB/+Iifz63qPlL04gMqZDYnMpz2EIrUB 2SmiFKaxWaJBv/nFoVF2ErWuNjVTMwZE8cP7Eob2HO+G2N2tLbRVMf0cJPOYHGGt 00Y+4c8EPnWvdcnHN0ei14FyTYKb+eRnvGsN5xqTnMw6oG/bGhEiHtIkdMGfrzfq UeG5cPGfxll7u8m9isA22c+lKdtnLZ9EIQtPs7Jxg09hSW3HqiNgkA5b/o/m+1A= =ROh0 -END PGP SIGNATURE-
Re: [gentoo-dev] new profile layout with flavors and mix-ins
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 03/07/14 04:47 AM, Joshua Kinard wrote: > On 07/03/2014 03:00, Michael Haubenwallner wrote: >> >> On 07/03/2014 08:18 AM, Joshua Kinard wrote: >>> On 07/02/2014 13:54, Michał Górny wrote: Dnia 2014-07-02, o godz. 10:44:16 >>> [snip] I don't feel like we ought to vote on something like this without understanding most of the current profiles. And I'm afraid there are only few people who have any idea about the current profile structure... >>> >>> I am going to throw this out there and see what people think. >>> Maybe it's insane, maybe it's not, maybe it's a mix of insane >>> and not-insane. >>> >>> Years ago, before we had the current stacking profile design >>> (we were discussing the current design, actually), I kinda >>> conjured up this "building blocks" like approach for a profile >>> design. >> >>> The idea being that, in /etc/make.conf (or wherever that file >>> is now), you'd define $PROFILE like this: >>> >>> linux-mips o32 uclibc server: >>> PROFILE="base:kernel/linux:arch/mips:subarch/mips-o32:libc/uclibc:roles/server:releases/13.0" >> >> >>> What about making /etc/portage/make.profile a directory rather than a symlink, >> having /etc/portage/make.profile/parent to reference all the >> flavours? >> >> Tools that need to respect the /current/ profile should work >> without any change, and tools that need to respect the >> /available/ profiles (repoman) already do have a list of profiles >> to respect (profiles/profiles.desc). >> >> So the only missing thing would be the eselect profile module to >> manage entries of /etc/portage/make.profile/parent, maybe using >> /usr/portage/profiles/profiles.desc as the source for available >> flavours. >> >> my 2 cents /haubi/ > > That's the thing, make.profile technically *is* a directory -- a > symlink to one. The original design of make.profile was to specify > generic, base settings for a given profile and keep that in the > tree. Things like default CHOST, default ARCH, default , > etc. make.conf then overrides in-tree settings with settings > specific to your system. So making /etc/make.profile an actual > directory disconnects it from the tree, which I don't think will > work very well for Portage, since it won't know what your > currently-chosen profile is. > I did a test where I dropped my make.profile symlink, made a directory of the same name, and populated a 'parent' file in that directory with paths to various /usr/portage/profiles/ bits which made up my previous main profile. Everything continued to work as expected. I expect for portage proper, we could accomodate mixins or whatever new structure method we want simply by adjusting 'eselect profile' to list the various possible combinations and adjust an /etc/make.profile/parent file accordingly. (We probably also need to ensure portage warns or errors appropriately when one of these inherit lines in the parent file no longer points to a directory, the same as it does now with the symlink points to nothing; I didn't test but I expect that error would not be pretty or easily understood right now) -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.22 (GNU/Linux) iF4EAREIAAYFAlO1f4IACgkQ2ugaI38ACPB1wgEApznnssegz2ImYIq6fAeR2oGa RX0TNT6bThDcypkn0skA/j/w33cWekomOQanCKIspuOWxXgOAeMxMWlnhsORZfj7 =Qwj0 -END PGP SIGNATURE-
Re: Re: [gentoo-dev] new profile layout with flavors and mix-ins
Am Mittwoch 02 Juli 2014, 15:07:12 schrieb Anthony G. Basile:
>
> I don't know how to get from here to there. The problem isn't just
> constructing an alternative profile tree. We could even have
> /usr/portage/profiles-r2 and switch between the two on demand. The
> problem is there's a lot of memory with flags and masks and these only
> make sense in the context of the current stacking profiles.
> Disentangling this information and bringing it over to profiles-r2 is
> going to be work.
Crazy idea:
* introduce a change that makes portage look at a new filename for
inheritance, i.e. existing "parent" files are disregarded and a new filename
is introduced ("inherits" ?)
* in most dirs that file will not exist -> no inheritance
* new profile specs will have new main directories that pull in the resulting
flat structure piece by piece
This means that existing files can (carefully) be re-used in the transition.
--
Andreas K. Huettel
Gentoo Linux developer
kde, council
Re: [gentoo-dev] new profile layout with flavors and mix-ins
On 07/03/2014 03:00, Michael Haubenwallner wrote: > > On 07/03/2014 08:18 AM, Joshua Kinard wrote: >> On 07/02/2014 13:54, Michał Górny wrote: >>> Dnia 2014-07-02, o godz. 10:44:16 >> [snip] >>> >>> I don't feel like we ought to vote on something like this without >>> understanding most of the current profiles. And I'm afraid there are >>> only few people who have any idea about the current profile >>> structure... >>> >> >> I am going to throw this out there and see what people think. Maybe it's >> insane, maybe it's not, maybe it's a mix of insane and not-insane. >> >> Years ago, before we had the current stacking profile design (we were >> discussing the current design, actually), I kinda conjured up this "building >> blocks" like approach for a profile design. > >> The idea being that, in /etc/make.conf (or wherever that file is now), you'd >> define $PROFILE like this: >> >> linux-mips o32 uclibc server: >> PROFILE="base:kernel/linux:arch/mips:subarch/mips-o32:libc/uclibc:roles/server:releases/13.0" > > What about making /etc/portage/make.profile a directory rather than a symlink, > having /etc/portage/make.profile/parent to reference all the flavours? > > Tools that need to respect the /current/ profile should work without any > change, and > tools that need to respect the /available/ profiles (repoman) already do have > a list > of profiles to respect (profiles/profiles.desc). > > So the only missing thing would be the eselect profile module to manage > entries of > /etc/portage/make.profile/parent, maybe using > /usr/portage/profiles/profiles.desc > as the source for available flavours. > > my 2 cents > /haubi/ That's the thing, make.profile technically *is* a directory -- a symlink to one. The original design of make.profile was to specify generic, base settings for a given profile and keep that in the tree. Things like default CHOST, default ARCH, default , etc. make.conf then overrides in-tree settings with settings specific to your system. So making /etc/make.profile an actual directory disconnects it from the tree, which I don't think will work very well for Portage, since it won't know what your currently-chosen profile is. -- Joshua Kinard Gentoo/MIPS ku...@gentoo.org 4096R/D25D95E3 2011-03-28 "The past tempts us, the present confuses us, the future frightens us. And our lives slip away, moment by moment, lost in that vast, terrible in-between." --Emperor Turhan, Centauri Republic
Re: [gentoo-dev] new profile layout with flavors and mix-ins
On 07/03/2014 03:32, Michał Górny wrote: > Dnia 2014-07-03, o godz. 02:18:23 > Joshua Kinard napisał(a): > >> [...] >> >> And so on. The goal was to have profiles/ be extremely flat, with limited >> nesting only for categorization purposes. Each component would contain all >> of the information specific to that component, and rely on OOP-like >> inheritance to override parent-level variables only within that component >> (although, and would have to be a little bit more broad in >> scope). > > Another sub-thread, the same question: how do you handle information > that needs to be applied to the intersection of the profiles? CHOST for > amd64 FreeBSD, for example. Well, as I stated, I thought the bulk of this up *years* ago (at least 6-7 years ago, probably longer). The issues you're raising now weren't issues back then. I only updated it slightly based on what I know now about how we do things and what things are supported in Gentoo. I am only proposing it as a base from which others to think about and either build upon or enhance, or strike it down. >> PROFILE also had a set order for the first few pieces, if only to make >> parsing and building the profile stack saner for the Portage developers: >>PROFILE="base::[:]::" >> >> base - Core Gentoo/Portage/whatever information/vars/foobar/kittens > > I'm against plain 'base' since it quickly becomes a mess. I would > prefer having flavor-specific bases, like for example arch/base to mask > stuff available only in 1/2 arches and revert it in another arch/. The only reason I put a 'base' folder in was so that where was some kind of anchor point for Portage to build off of. I didn't put any thought into what it might actually contain. For all intents and purposes, it might just contain empty variable declarations, kinda like an abstract base class in an OOP language. By itself, it's unusable and must be overridden, but it's the point from which everything derives. Maybe that's not needed now in current Portage. Dunno. >> kernel - Specifies the OS kernel. At the time, only Linux existed, but >> I *think* Debian was eyeballing kFreeBSD at the time. So I *think* >> I accounted for it back then. > > What about kernel-ABI intersection? What about Prefix? I know very little about Prefix. Any new profile system we create should focus only what is in-tree *first*, then once we work the bugs and kinks out, we can figure out how Prefixes will be incorporated. But that said, what little I do know about Prefix is that it's Portage ported to other OSes, like Solaris, MacOS, Interix, IRIX, etc. I guess an out-of-tree Prefix overlay would just define missing bits in its profiles/* folder. I.e., for Solaris, you'd have a kernel/solaris, libc/solaris-libc, etc. I'm not the best at OOP-designs, but if we set the in-tree stuff up right, it should be trivial for out-of-tree overlays to extend and override to support things, without going down the endlessly-nested directory snafu that the current profile system is. In-tree should be as flat as possible, Prefix and other external overlays can extend it out however far they want. As for kernel-ABI intersection, can you expand on that a little more? A given kernel can support multiple ABIs simultaneously. Using MIPS as the classic example, a 32-bit MIPS kernel only does o32, but a 64-bit kernel does n64 natively, and you can enable o32 and n32 support individually within the configuration. And MIPS has even more obscure ABIs that aren't well-known, either. >> arch - Machine arch-specific generic information -- doesn't handle >>lower-level things like ISAs/ABIs/Endian, etc. >> >> subarch - Defines items specific given to given subarch of a main arch. >> Items under this directory would carry their parent arch's >> name for clarity only. Again, at the time, MIPS would have been >> the only real user of this, and, back then, it would've dealt >> with SGI-machine-specific packages and USE flags, such as >> differentiating an ip32 machine from an ip27 machine or a >> cobalt. Now that amd64/x86_64 is considering its own form of >> n32 (x32), that would go here, and I imagine arm would >> make heavy use of it as well. > > This is a no-go for multilib support. We need to work on a consistent > arch profile tree, not more splitting. Well, it'll have to be nested somehow. A given arch can handle multiple ABIs, so there should at least be a top-level arch/ folder that defines very high-level, very generic things specific to a given arch. How we fold the multilib and multiple ABI stuff in probably needs a lot of thought. It may not be possible with this proposed idea, so, if not, we'll need to think of something else. Also keep in mind that with MIPS, we up the fun fun with the ISA (Instruction Set Architecture) mess (mips1 to mips4, mips32r2, mips64r2, etc), which are also mostly independent o
Re: [gentoo-dev] new profile layout with flavors and mix-ins
Dnia 2014-07-03, o godz. 02:18:23 Joshua Kinard napisał(a): > [...] > > And so on. The goal was to have profiles/ be extremely flat, with limited > nesting only for categorization purposes. Each component would contain all > of the information specific to that component, and rely on OOP-like > inheritance to override parent-level variables only within that component > (although, and would have to be a little bit more broad in > scope). Another sub-thread, the same question: how do you handle information that needs to be applied to the intersection of the profiles? CHOST for amd64 FreeBSD, for example. > PROFILE also had a set order for the first few pieces, if only to make > parsing and building the profile stack saner for the Portage developers: >PROFILE="base::[:]::" > > base - Core Gentoo/Portage/whatever information/vars/foobar/kittens I'm against plain 'base' since it quickly becomes a mess. I would prefer having flavor-specific bases, like for example arch/base to mask stuff available only in 1/2 arches and revert it in another arch/. > kernel - Specifies the OS kernel. At the time, only Linux existed, but > I *think* Debian was eyeballing kFreeBSD at the time. So I *think* > I accounted for it back then. What about kernel-ABI intersection? What about Prefix? > arch - Machine arch-specific generic information -- doesn't handle >lower-level things like ISAs/ABIs/Endian, etc. > > subarch - Defines items specific given to given subarch of a main arch. > Items under this directory would carry their parent arch's > name for clarity only. Again, at the time, MIPS would have been > the only real user of this, and, back then, it would've dealt > with SGI-machine-specific packages and USE flags, such as > differentiating an ip32 machine from an ip27 machine or a > cobalt. Now that amd64/x86_64 is considering its own form of > n32 (x32), that would go here, and I imagine arm would > make heavy use of it as well. This is a no-go for multilib support. We need to work on a consistent arch profile tree, not more splitting. > libc - Defines the main C library used. A bit redundant for *BSD, because >they really only have one, but might help if we ever add k*BSD >support (such as freebsd/glibc-based or such). For Linux...could be >tricky, since there are so many libc possibilities there. I feel it >fits better getting defined after , especially in the case >of MIPS. I think you need to handle kernel & libc in the same group. > eapi - EAPI didn't exist back when this topic was visited. Basically, >everything was EAPI=0 and there was only Portage (Paludis nor >pkgcore had been invented yet). And what is this supposed to do? > releases - I don't think this existed back then. How it'd operate >nowadays, I have no idea. Probably would contain >version-specific maskings for specific @system packages >to facilitate upgrading, and help us if we ever tried to cut >actual, fixed release points again (like what FreeBSD does >w/ -RELEASE-x.y) I don't think releases would make any sense without heavy intersection with other profiles. -- Best regards, Michał Górny signature.asc Description: PGP signature
Re: [gentoo-dev] new profile layout with flavors and mix-ins
On 07/03/2014 08:18 AM, Joshua Kinard wrote: > On 07/02/2014 13:54, Michał Górny wrote: >> Dnia 2014-07-02, o godz. 10:44:16 > [snip] >> >> I don't feel like we ought to vote on something like this without >> understanding most of the current profiles. And I'm afraid there are >> only few people who have any idea about the current profile >> structure... >> > > I am going to throw this out there and see what people think. Maybe it's > insane, maybe it's not, maybe it's a mix of insane and not-insane. > > Years ago, before we had the current stacking profile design (we were > discussing the current design, actually), I kinda conjured up this "building > blocks" like approach for a profile design. > The idea being that, in /etc/make.conf (or wherever that file is now), you'd > define $PROFILE like this: > > linux-mips o32 uclibc server: > PROFILE="base:kernel/linux:arch/mips:subarch/mips-o32:libc/uclibc:roles/server:releases/13.0" What about making /etc/portage/make.profile a directory rather than a symlink, having /etc/portage/make.profile/parent to reference all the flavours? Tools that need to respect the /current/ profile should work without any change, and tools that need to respect the /available/ profiles (repoman) already do have a list of profiles to respect (profiles/profiles.desc). So the only missing thing would be the eselect profile module to manage entries of /etc/portage/make.profile/parent, maybe using /usr/portage/profiles/profiles.desc as the source for available flavours. my 2 cents /haubi/
Re: [gentoo-dev] new profile layout with flavors and mix-ins
On 07/02/2014 13:54, Michał Górny wrote: > Dnia 2014-07-02, o godz. 10:44:16 [snip] > > I don't feel like we ought to vote on something like this without > understanding most of the current profiles. And I'm afraid there are > only few people who have any idea about the current profile > structure... > I am going to throw this out there and see what people think. Maybe it's insane, maybe it's not, maybe it's a mix of insane and not-insane. Years ago, before we had the current stacking profile design (we were discussing the current design, actually), I kinda conjured up this "building blocks" like approach for a profile design. Basically building on how most of a Gentoo system is pieced together in /etc/make.conf (or wherever that file lives now). A new variable, $PROFILE, would be defined that contains a colon-separated list of profile "blocks" that specifies the nature of the system. Kinda like how $PATH is built. This idea never got any traction, though, and may still not. Quoting from an e-mail I sent months ago to blueness after reading one of his blog posts, here's what I worked out years ago with some updated thinking a few months ago: -- An idea I had at the time, but which never gained any traction, was to use a similar profile layout as we have now, but not treat it like a sequence of directory structures. Instead, the idea was to have it modeled more like how $PATH is built up from discrete components. So instead of something like this: /usr/portage/profiles/base /usr/portage/profiles/default/linux /usr/portage/profiles/arch/base /usr/portage/profiles/features/multilib /usr/portage/profiles/features/multilib/lib32 /usr/portage/profiles/arch/amd64 /usr/portage/profiles/releases /usr/portage/profiles/eapi-5-files /usr/portage/profiles/releases/13.0 /usr/portage/profiles/hardened/linux /usr/portage/profiles/hardened/linux/amd64 /usr/portage/profiles/targets/desktop /usr/portage/profiles/hardened/linux/amd64/destkop You would have a layout in /usr/portage/profiles look like this instead (btw, this is from long-term memory, so some things might be off from my original idea way back when): arch/ base/desktops/ |-amd64/ |-e17/ |-arm/ |-kde/ |-mips/ |-gnome/ |-sparc/ |-xfce/ |-x86/ eapi/ kernel/libc/ |-3/ |-freebsd/ |-glibc |-4/ |-linux/ |-uclibc |-5/ |-musl |-6/ |-freebsd options/ releases/ roles/ |-hardened/ |-11.0/|-desktop/ |-multilib/ |-12.0/|-firewall/ |-13.0/|-ids/ |-freebsd-9.2/ |-router/ |-server/ servers/ subarch/ |-dns/|-mips-o32 |-file/ |-mips-n32 |-mail/ |-sparc-v8 |-www/|-amd64-x32 The idea being that, in /etc/make.conf (or wherever that file is now), you'd define $PROFILE like this: linux-mips o32 uclibc server: PROFILE="base:kernel/linux:arch/mips:subarch/mips-o32:libc/uclibc:roles/server:releases/13.0" linux-amd64/x86_64/x64 glibc hardened 13.0 KDE desktop: PROFILE="base:kernel/linux:arch/amd64:libc/glibc:roles/desktop:options/hardened:desktops/kde:releases/13.0" freebsd-amd64 9.2 firewall: PROFILE="base:kernel/freebsd:arch/amd64:libc/freebsd:roles/firewall:releases/freebsd-9.2" And so on. The goal was to have profiles/ be extremely flat, with limited nesting only for categorization purposes. Each component would contain all of the information specific to that component, and rely on OOP-like inheritance to override parent-level variables only within that component (although, and would have to be a little bit more broad in scope). PROFILE also had a set order for the first few pieces, if only to make parsing and building the profile stack saner for the Portage developers: PROFILE="base::[:]::" base - Core Gentoo/Portage/whatever information/vars/foobar/kittens kernel - Specifies the OS kernel. At the time, only Linux existed, but I *think* Debian was eyeballing kFreeBSD at the time. So I *think* I accounted for it back then. arch - Machine arch-specific generic information -- doesn't handle lower-level things like ISAs/ABIs/Endian, etc. subarch - Defines items specific given to given subarch of a main arch. Items under this directory would carry their parent arch's name for clarity only. Again, at the time, MIPS would have been the only real user of this, and, back then, it would've dealt with SGI-machine-specific packages and USE flags, such as
Re: [gentoo-dev] new profile layout with flavors and mix-ins
On Wed, Jul 2, 2014 at 3:19 PM, Rick "Zero_Chaos" Farina wrote: > > On 07/02/2014 03:07 PM, Anthony G. Basile wrote: >> >> I don't know how to get from here to there. The problem isn't just >> constructing an alternative profile tree. We could even have >> /usr/portage/profiles-r2 and switch between the two on demand. The >> problem is there's a lot of memory with flags and masks and these only >> make sense in the context of the current stacking profiles. >> Disentangling this information and bringing it over to profiles-r2 is >> going to be work. >> > I agree entirely. Right now the mixin style profiles are not supported > in gentoo _at_all_, and if no one ever works on that, then we will never > support it. I will work on the first step in a long road, that's all > I'm talking about here. I agree. I think getting support in all the tools for multiple profiles is a good start. Volunteers can then begin working on profiles that dis-entangle things. A first step might be to just rip out the desktop/etc categories, leaving the mess of arch/hardened/kernel/etc. Then we try to keep peeling back the layers. We don't have to get there in one step, nor do we have to release anything to the general public before it is finished. We might not end up with completely flat profiles - the goal is to end up in a better place than we are now, and perhaps we can make more progress later. Rich
Re: [gentoo-dev] new profile layout with flavors and mix-ins
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/02/2014 03:07 PM, Anthony G. Basile wrote: > On 07/02/14 14:41, Rick "Zero_Chaos" Farina wrote: >> -BEGIN PGP SIGNED MESSAGE- >> Hash: SHA1 >> >> On 07/02/2014 02:10 PM, Rich Freeman wrote: >>> On Wed, Jul 2, 2014 at 1:54 PM, Michał Górny wrote: I don't feel like we ought to vote on something like this without understanding most of the current profiles. And I'm afraid there are only few people who have any idea about the current profile structure... >>> No argument there. >>> >>> We may very well still end up with something hierarchical, but we can >>> at least limit that to the parts of the profile where it matters. >>> Maybe x86/BSD and amd64/Linux and amd64/Linux-hardened need to be >>> interdependent. However, that still gets rid of need to deal with >>> desktop environments, init systems, arguments over what belongs in >>> @system, and so on. We could have a blocker mechanism to keep people >>> from mixing systemd with BSD, or we could just let people shoot >>> themselves in the foot. >>> >>> Sounds like a good time to start reverse engineering the profiles... >>> >> I've talked to the funtoo devs a few times about stealing their profile >> idea. A few things need to be done to make this happen, mainly eselect, >> catalyst, and repoman support. I can do the eselect and catalyst >> changes myself, however, I'll require some help for the repoman support >> most likely. I'll prototype this locally, see what I can make work, and >> then see about making it usable for others to test. >> >> - -Zero >> -BEGIN PGP SIGNATURE- >> > > I don't know how to get from here to there. The problem isn't just > constructing an alternative profile tree. We could even have > /usr/portage/profiles-r2 and switch between the two on demand. The > problem is there's a lot of memory with flags and masks and these only > make sense in the context of the current stacking profiles. > Disentangling this information and bringing it over to profiles-r2 is > going to be work. > I agree entirely. Right now the mixin style profiles are not supported in gentoo _at_all_, and if no one ever works on that, then we will never support it. I will work on the first step in a long road, that's all I'm talking about here. Thanks, Zero -BEGIN PGP SIGNATURE- Version: GnuPG v2 Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJTtFtTAAoJEKXdFCfdEflKDk0P/j6F/7ivY2HkbRV3Y60YiahE IbzwrZ7wQCap5sAELOtQaXTyTHuyTLkTrAqqM/r+/8a0/Ebm67yEALbzE4IOMAFL DoiS7mg0FZeq2Bm8AGKtYwmBHAiIYbGWX+XgxuFbwpHz4uilsVhQH9+e+rFlu6SN 22BDmW6tQSDL5a3C2Coqoq/KVH/heHu43w6/TuZmVVByLcABHrEG/KOnLRHSGfwF ps+R3UaMmFA9w6VGqGxlOXuIbF+cp//xOTfqmetYfe5kkOsls/ZAKou1MMCrCSI7 j6oJbH5q5Gn5W6vxctf8PJPI8R5NWZBOFwvDgkeJ9Zcsqhsinb19QQFVkG6HOrt1 2nSTOP9U/5d0wYn9P7cNgQsXjg8P2KwHfPXG8qZ8jx1CSM0T2bY4csjALgqAmvpz ATa9XbpGmcRMoldqiVZEVN2EqL1NWVXo0i490Y51noiLpNSiSO2XT4lbIj1fae+i yxt6RuJ04QltsQKKzH3swCqgFBUZmNZe2h0p0dt+GFOl9Pg0oa3QX3eGR0msgmEy Qti0JRZwQJrHkDCSzZP0OAI2RZDj5XxmhS1brTTvb7LnxFecWR760LfQ/bzJgqno M800gZKrJ9AjzC8lvzqWQrd39W4VYhcDn19S5zHtBPb3KK/4jC9J1TVwwgzbW7jb z6ytQ2biVKhkBPrec5tf =2WzO -END PGP SIGNATURE-
Re: [gentoo-dev] new profile layout with flavors and mix-ins
On 07/02/14 14:41, Rick "Zero_Chaos" Farina wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/02/2014 02:10 PM, Rich Freeman wrote: On Wed, Jul 2, 2014 at 1:54 PM, Michał Górny wrote: I don't feel like we ought to vote on something like this without understanding most of the current profiles. And I'm afraid there are only few people who have any idea about the current profile structure... No argument there. We may very well still end up with something hierarchical, but we can at least limit that to the parts of the profile where it matters. Maybe x86/BSD and amd64/Linux and amd64/Linux-hardened need to be interdependent. However, that still gets rid of need to deal with desktop environments, init systems, arguments over what belongs in @system, and so on. We could have a blocker mechanism to keep people from mixing systemd with BSD, or we could just let people shoot themselves in the foot. Sounds like a good time to start reverse engineering the profiles... I've talked to the funtoo devs a few times about stealing their profile idea. A few things need to be done to make this happen, mainly eselect, catalyst, and repoman support. I can do the eselect and catalyst changes myself, however, I'll require some help for the repoman support most likely. I'll prototype this locally, see what I can make work, and then see about making it usable for others to test. - -Zero -BEGIN PGP SIGNATURE- I don't know how to get from here to there. The problem isn't just constructing an alternative profile tree. We could even have /usr/portage/profiles-r2 and switch between the two on demand. The problem is there's a lot of memory with flags and masks and these only make sense in the context of the current stacking profiles. Disentangling this information and bringing it over to profiles-r2 is going to be work. -- Anthony G. Basile, Ph.D. Gentoo Linux Developer [Hardened] E-Mail: bluen...@gentoo.org GnuPG FP : 1FED FAD9 D82C 52A5 3BAB DC79 9384 FA6E F52D 4BBA GnuPG ID : F52D4BBA
Re: [gentoo-dev] new profile layout with flavors and mix-ins
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 07/02/2014 02:10 PM, Rich Freeman wrote: > On Wed, Jul 2, 2014 at 1:54 PM, Michał Górny wrote: >> I don't feel like we ought to vote on something like this without >> understanding most of the current profiles. And I'm afraid there are >> only few people who have any idea about the current profile >> structure... > > No argument there. > > We may very well still end up with something hierarchical, but we can > at least limit that to the parts of the profile where it matters. > Maybe x86/BSD and amd64/Linux and amd64/Linux-hardened need to be > interdependent. However, that still gets rid of need to deal with > desktop environments, init systems, arguments over what belongs in > @system, and so on. We could have a blocker mechanism to keep people > from mixing systemd with BSD, or we could just let people shoot > themselves in the foot. > > Sounds like a good time to start reverse engineering the profiles... > I've talked to the funtoo devs a few times about stealing their profile idea. A few things need to be done to make this happen, mainly eselect, catalyst, and repoman support. I can do the eselect and catalyst changes myself, however, I'll require some help for the repoman support most likely. I'll prototype this locally, see what I can make work, and then see about making it usable for others to test. - -Zero -BEGIN PGP SIGNATURE- Version: GnuPG v2 Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJTtFI/AAoJEKXdFCfdEflK7GYP/0GS+Sh4odpHNMUfNfyU+q7F HpNuJ/OSnnNBRol4dDjcvhE3RFxIqU1DkC1atsl19K14vJXk+JBSKw8fgV4J6ous 0uOYqN472Noy7NxzmrbfrlR1PePG1b878ZpOMbGce1dgMfamoDwv48IjfcSwakcv 6HSy09xfr5O8xG+cSBqyVmaiDxsPHPQwv1s+/JUpcadBr4AmD7qc7yTZ5QjKJoIg o1eL/j//KiYxQywoQ1udqXwx/beBgYMQB4R5Vu4d1BBtIttnJdTHCZdAv6Uoi686 x4VWuvBPDaILojyhIYqpoMbKAu/c/AFRFkpFZXFVl5kWyolw/YuBg4RL21qecDvp 9qyoQFCzIWCSzXl747O4iFYjS1ACbMSUXOSDEecvnYuCsjvI+bpJT2pWVlAxIMUH 79IgKTu0+1LETl/lEeMN8ccrG8R2hhrYCxxyTw1nZfevN9SPC88RwEIcuBwVFHC3 NC7fpxKzDFy2+0ixBnLCQA7ICnvV5crF2Tt4/bg9Hbp2UP/QMmhxIwUbceRSDXc5 3c8nqsUGavLX8K9IE3IlI5ZYutmYJgJwfqX8yA6wdOeZgPvH59gypnBYFKJjO2DQ JZE+lVZ4vXLmgw1WTAdRUYFhLEqesioCAVH7Bm7gUnccCeWE4PFHr15BGhdh/Mw9 L88DkCnfAbLcSHRJS0Cx =iBrl -END PGP SIGNATURE-
Re: [gentoo-dev] new profile layout with flavors and mix-ins
On Wed, Jul 2, 2014 at 2:32 PM, Anthony G. Basile wrote: > Shallow profiles avoid this. Also "features" avoid this (the > closest thing we have to mix-ins) provided they operate on a set of > flags/packages orthogonal to the rest of the stack. You then have shallow > base and you can add as many features as you like in, in any order, > confident that one will not clobber stuff from another since each feature is > well separated. That was my thinking as well. If we had categories of profiles and set up the rules to try to keep things orthogonal then we're going to be better off. Also, things like feature profiles should try to be additive in nature. Rich
Re: [gentoo-dev] new profile layout with flavors and mix-ins
On 07/02/14 14:10, Rich Freeman wrote: On Wed, Jul 2, 2014 at 1:54 PM, Michał Górny wrote: I don't feel like we ought to vote on something like this without understanding most of the current profiles. And I'm afraid there are only few people who have any idea about the current profile structure... No argument there. We may very well still end up with something hierarchical, but we can at least limit that to the parts of the profile where it matters. Maybe x86/BSD and amd64/Linux and amd64/Linux-hardened need to be interdependent. However, that still gets rid of need to deal with desktop environments, init systems, arguments over what belongs in @system, and so on. We could have a blocker mechanism to keep people from mixing systemd with BSD, or we could just let people shoot themselves in the foot. Sounds like a good time to start reverse engineering the profiles... Rich The way the profiles stack via the parent file makes them difficult to work with if they get to any significant depth. Here, for example, is the stacking for default/linux/mips/13.0/mipsel/multilib/n32 /usr/portage/profiles/base /usr/portage/profiles/default/linux */usr/portage/profiles/arch/base */usr/portage/profiles/arch/mips */usr/portage/profiles/default/linux/mips /usr/portage/profiles/releases /usr/portage/profiles/releases/13.0 /usr/portage/profiles/default/linux/mips/13.0 */usr/portage/profiles/arch/base */usr/portage/profiles/arch/mips */usr/portage/profiles/arch/mips/mipsel /usr/portage/profiles/default/linux/mips/13.0/mipsel */usr/portage/profiles/arch/base */usr/portage/profiles/arch/mips */usr/portage/profiles/arch/mips/mipsel /usr/portage/profiles/arch/mips/mipsel/mips64el /usr/portage/profiles/features/multilib /usr/portage/profiles/arch/mips/mipsel/mips64el/multilib /usr/portage/profiles/arch/mips/mipsel/mips64el/multilib/n32 /usr/portage/profiles/default/linux/mips/13.0/mipsel/multilib/n32 I put asterisks there to point out a pattern that gets pulled in repeatedly. Sometimes this isn't a problem but sometimes this leads to asserting something, then reverting it, then asserting it again. In the ancient selinux profiles (circa 2009) this meant you couldn't have a no-mutlilib amd64 system. Shallow profiles avoid this. Also "features" avoid this (the closest thing we have to mix-ins) provided they operate on a set of flags/packages orthogonal to the rest of the stack. You then have shallow base and you can add as many features as you like in, in any order, confident that one will not clobber stuff from another since each feature is well separated. -- Anthony G. Basile, Ph.D. Gentoo Linux Developer [Hardened] E-Mail: bluen...@gentoo.org GnuPG FP : 1FED FAD9 D82C 52A5 3BAB DC79 9384 FA6E F52D 4BBA GnuPG ID : F52D4BBA
Re: [gentoo-dev] new profile layout with flavors and mix-ins
On Wed, Jul 2, 2014 at 1:54 PM, Michał Górny wrote: > I don't feel like we ought to vote on something like this without > understanding most of the current profiles. And I'm afraid there are > only few people who have any idea about the current profile > structure... No argument there. We may very well still end up with something hierarchical, but we can at least limit that to the parts of the profile where it matters. Maybe x86/BSD and amd64/Linux and amd64/Linux-hardened need to be interdependent. However, that still gets rid of need to deal with desktop environments, init systems, arguments over what belongs in @system, and so on. We could have a blocker mechanism to keep people from mixing systemd with BSD, or we could just let people shoot themselves in the foot. Sounds like a good time to start reverse engineering the profiles... Rich
Re: [gentoo-dev] new profile layout with flavors and mix-ins
Dnia 2014-07-02, o godz. 10:44:16
William Hubbs napisał(a):
> All,
>
> I'm moving to a new thread since the discussion has moved away from just
> a sub profile for no-multilib.
>
> On Wed, Jul 02, 2014 at 09:30:50AM -0400, Rich Freeman wrote:
> > On Wed, Jun 25, 2014 at 4:01 PM, Andreas K. Huettel
> > wrote:
> > > Am Mittwoch 25 Juni 2014, 15:11:40 schrieb Rich Freeman:
> > >> On Wed, Jun 25, 2014 at 2:44 PM, Michał Górny wrote:
> > >> > Long story short, doing anything to Gentoo profiles is utter pain
> > >> > and comes with random breakage guarantee. Therefore, I'm asking -- nuke
> > >> > those damn profiles, and start over! The current situation is
> > >> > completely unmaintainable.
> > >>
> > >> ++
> > >>
> > >> But, would it make sense to just go the Funtoo route with "mix-ins."
> > > ++
> > >
> > > this is what we've been just discussing on the irc channel
> >
> > So, not wanting this to die on the vine.
> >
> > If we did the mix-in approach, would we just follow the example of Funtoo?
> >
> > They use an arch profile, a stability profile (~arch vs arch), a
> > "flavor" profile (core, minimal, desktop), and then users can layer as
> > much other stuff on top of that as they want (gnome, kde, multimedia,
> > etc).
>
> I think this could work for us as well, or something similar anyway.
>
> For those who are curious, I am including the link to the flavors and
> mix-ins descriptions from the funtoo site. [1]
It's not that easy. As you can see on that site, they're supporting
much less variants than Gentoo does. In particular, they don't seem to
support non-GNU/Linux at all. No Prefix, no Hardened, no FreeBSD.
I was thinking about modularization a bit and the main issue is
handling intersecting profiles. As you can see in Funtoo, it already
starts with the 'build' flavor -- they're pretty much applying a cheap
hack (ACCEPT_KEYWORDS="${ARCH}") but such hacks can't cover all our
needs.
A simple example is CHOST. The value of CHOST depends on the arch,
often ABI, kernel, libc. Of course, we could hack this around by
creating some intermediate variables and merging them afterwards.
But not everything can be hacked around like this.
I don't feel like we ought to vote on something like this without
understanding most of the current profiles. And I'm afraid there are
only few people who have any idea about the current profile
structure...
> > Do we want to do things the same way?
> >
> > Some things to think about include multilib (just another arch?),
> > systemd, and usr-merge. I'm not saying that we need to implement any
> > of that stuff completely - but when planning the profile layout we
> > should at least consider whether it will handle things like this in
> > the future. Should some types of profiles be only additive? Etc...
>
> I see systemd and multilib as mix-ins, like the ones you mentioned
> above.
Multilib won't work as Funtoo-style mix-in for the simple reason it
relies on heavily on the architecture in use.
--
Best regards,
Michał Górny
signature.asc
Description: PGP signature
[gentoo-dev] new profile layout with flavors and mix-ins
All, I'm moving to a new thread since the discussion has moved away from just a sub profile for no-multilib. On Wed, Jul 02, 2014 at 09:30:50AM -0400, Rich Freeman wrote: > On Wed, Jun 25, 2014 at 4:01 PM, Andreas K. Huettel > wrote: > > Am Mittwoch 25 Juni 2014, 15:11:40 schrieb Rich Freeman: > >> On Wed, Jun 25, 2014 at 2:44 PM, Michał Górny wrote: > >> > Long story short, doing anything to Gentoo profiles is utter pain > >> > and comes with random breakage guarantee. Therefore, I'm asking -- nuke > >> > those damn profiles, and start over! The current situation is > >> > completely unmaintainable. > >> > >> ++ > >> > >> But, would it make sense to just go the Funtoo route with "mix-ins." > > ++ > > > > this is what we've been just discussing on the irc channel > > So, not wanting this to die on the vine. > > If we did the mix-in approach, would we just follow the example of Funtoo? > > They use an arch profile, a stability profile (~arch vs arch), a > "flavor" profile (core, minimal, desktop), and then users can layer as > much other stuff on top of that as they want (gnome, kde, multimedia, > etc). I think this could work for us as well, or something similar anyway. For those who are curious, I am including the link to the flavors and mix-ins descriptions from the funtoo site. [1] > > Do we want to do things the same way? > > Some things to think about include multilib (just another arch?), > systemd, and usr-merge. I'm not saying that we need to implement any > of that stuff completely - but when planning the profile layout we > should at least consider whether it will handle things like this in > the future. Should some types of profiles be only additive? Etc... I see systemd and multilib as mix-ins, like the ones you mentioned above. It is funny that you mention usr-merge. I don't want to get into a big debate on it on this thread, so for now I'll just say that I don't see that as affecting the profile structure at all. William [1] http://www.funtoo.org/Flavors_and_Mix-ins signature.asc Description: Digital signature
