Re: [gentoo-dev] Re: Eclass vs EAPI For Utility Functions (Patching/etc)
On Thu, Jun 19, 2014 at 6:05 PM, Steven J. Long wrote: > So which way do you actually prefer? > > You appear to be arguing for, and implementing, common code by EAPI, > in the rest of your mail. Since that's always been the point of > them, based on developer consensus about what is truly essential, > and what is only needed for a subset of the tree, that's fine by me. This is all becoming moot since the Council just voted on this earlier this week, nearly unanimously bringing in user patches. However, I believe he was advocating going with an EAPI in this case, but offering alternatives. While brainstorming the options I was thinking that you could almost have an EAPI-like eclass. That is, instead of declaring an EAPI every ebuild could just source an EAPI-foo eclass which basically does the same thing. Of course, this would be somewhat less flexible than what we do today - if we went as far as being able to determine EAPI without sourcing an ebuild (no, I'm not bringing back that debate now) then having EAPI in the PM lets you change the ebuild format in almost entirely arbitrary ways over time. There is nothing wrong with playing devil's advocate in a thread like this. Hopefully the Council members can weigh the arguments on their own. :) Rich
[gentoo-dev] Re: Eclass vs EAPI For Utility Functions (Patching/etc)
On Sun, Jun 15, 2014 at 03:30:10PM +0200, Michał Górny wrote: > Dnia 2014-06-15, o godz. 07:00:15 > Rich Freeman napisał(a): > > > The Eclass argument goes like this: > > Eclasses already work in every PM. Half of what we're debating is > > already in eutils. Why move this code into the PM, where it has to be > > re-implemented everywhere? If anything we should be moving more PM > > functionality out and into eclasses where we can have competing > > implementations and more flexibility. > > I think this point is a bit tangential. While eclasses are the obvious > way of having code common between package managers, I don't think > sharing code should be an argument for putting something in an eclass. Er, it's the most fundamental argument for using an eclass. Thought not between package managers, since they don't factor into the equation, so long as they provide EAPI conformance (theoretically.) > Please remember that you can create your own base repository without > having 'gentoo' as master. Then, putting code in eclasses actually > requires you to duplicate it -- unlike code in PM which is shared > between all repos. Well that's an argument for certain base functionality being in the PM although I'd note that "in the PM" in most cases, and this one, means "in ebuild.sh or one of its associated BASH modules." It's not an argument for sharing script across ebuilds in general, by putting it in the the context for every ebuild. As for different master repos, that's an appropriate discussion at at the package-manager code level, but not at the distro level, imo. > Now, for the overall point. I think there are a few cases when you want > something in the PM: > > 1. when it's a common thing to do that doesn't require repository- > specific details like relying on some out-of-@system packages, > > 2. when it requires specific interaction with the package manager, > > 3. when it is required by some other PM-specific code. That's enforced by the requirement. > Keeping it > 'internal' just means having duplicate code between PM and eclasses. No: it also means you don't expose it until it's needed. The worst thing in the world is having to rely on variant behaviour according to the phase of the moon, which is about what you get when internal APIs are exposed without cause and consensus. Since you're speaking in general terms. > I think the only debatable thing here is (1). If we ever decide that > having common stuff in an eclass is feasible, we ought to move all > the common stuff that's possible -- including econf, emake, possibly > some install helpers. I'm not sure what that's got to with out-of-@system packages, but it sounds like a bad idea, and counter to your argument about sharing code via the PM, and not eclasses. > This may even have more benefits than that. For example, you would fit > the implementation to specific system -- like 'gentoo' repository > eclass would be fit for Gentoo-specific tools. Instead of putting > requirements for a system in PMS and trying to fit PM and profiles > together, we'd just use whatever the repository provides. That sounds insane to me. What happened to reproducible builds? Let alone a robust user experience. Perhaps at the package-manager code level, but not at the distro level. And it's the Gentoo PMS, so I don't see any conflict. Though this appears to be the same "tangent" Rich brought up. > Of course this brings another argument -- every single ebuild would > have to source a specific eclass. For EAPI 6, I've focused on going > the opposite way So which way do you actually prefer? You appear to be arguing for, and implementing, common code by EAPI, in the rest of your mail. Since that's always been the point of them, based on developer consensus about what is truly essential, and what is only needed for a subset of the tree, that's fine by me. Just so long as they are essential, and you do have consensus on that. > As far as both eapply & eapply_user are concerned Why can't we just have epatch and upatch? = I'm on record as wanting this in the PM, btw, back when we were discussing how it should deal with patches not being applied. -- #friendly-coders -- We're friendly, but we're not /that/ friendly ;-)
[gentoo-dev] Re: Re: Re: Re: crossdev and multilib interference
On Tue, Jun 17, 2014 at 10:56 -0400, Alexandre Rostovtsev wrote:
> All multilib packages that use pkgconfig, for one thing. (Which means almost
> all multilib packages.) Because current crossdev versions blindly install
> their
> /usr/bin/i686-pc-linux-gnu-pkg-config wrapper script, overwriting the binary
> belonging to pkgconfig[abi_x86_32].
Well I've spent far too long at crossdev code, only to see this and realise
you can simply hard-mask:
cross-i686-pc-linux-gnu/{binutils,gcc,glibc,pkg-config}
in the amd64 multilib profile, unless I'm missing something. You'd be
hard-pushed to install a clashing crossdev with such a mask, afaict.
If you do want to change crossdev[1], afaict you're looking at interaction
between toolchain.eclass (and toolchain-binutils, and likely -funcs),
crossdev and gcc-config. I could well be wrong, as ever. This is just my
preliminary understanding, and maybe it'll provoke a more thorough
explanation.
Crossdev set_links() links the overlay to the tree. toolchain.eclass installs
the versioned links so the internal gcc-bin path is exposed in /usr/bin, in
toolchain_src_install(). For cross-toolchains, it only installs prefixed ones:
dosym ${BINPATH}/${CTARGET}-${x} /usr/bin/${CTARGET}-${x}-${GCC_CONFIG_VER}
For a native compiler it first, in: "${D}"${BINPATH} does:
ln -sf ${CTARGET}-${x} ${x} # unversioned private link
dosym ${BINPATH}/${CTARGET}-${x} /usr/bin/${x}-${GCC_CONFIG_VER}
..then the above. Note that the prefixed link is effectively a race as to
which was installed last, but only in the case of a clash.
gcc-config update_wrappers() makes links in the user's path, though all of
them are run via wrapper.c[2], aka /usr/lib/misc/gcc-config. This is a
multi-switch binary based on argv[0]/$0. However it specifically notes
that it's intended to be used with a PATH-based setup [3]:
/* Find the first file with suitable name in PATH. The idea here is
* that we do not want to bind ourselfs to something static like the
* default profile, or some odd environment variable, but want to be
* able to build something with a non default gcc by just tweaking
* the PATH ... */
crossdev dirs are added to path after system ones in env.d; that's where
gcc-config gets the paths to use from.
crossdev uninstall() removes CTARGET-based links. Note that your native
machine CBUILD == CHOST, also has CTARGET = CHOST, so this would also
be a reason to block/mask according to arch.
I haven't reviewed wrapper.c as well as I'd like: some of it seems a bit
odd but I'd need more time. I did wonder why it doesn't just use
readlink(3P) til I saw that comment.
Anyhow, that all seems a bit pointless when you can just hardmask the
specific crossdev configuration that causes the problem on multilib
profiles, and the same mechanism can be applied elsewhere, as decided
by the arch-team (eg for: o32/n32/n64)
Although canadian-cross and ROOT-based toolchains are another matter.
Regards,
steveL
[1] ie stop installing symlinks in /usr/bin for CTARGET-gcc, as well as
env.d files, and just provide a sh wrapper in each PORTAGE_CONFIGROOT
(= cross-overlay) that can be sourced from /etc/profile or anywhere
else, to add the same settings instead as and when the user chooses.
The most you'd need is the ability to choose whether it takes precedence
over current PATH or not, and that's probably easiest with a variant
'source' ('.' in shell) so cross-builds do, and the profile one doesn't.
[2]
http://git.overlays.gentoo.org/gitweb/?p=proj/gcc-config.git;a=blob;f=wrapper.c;hb=HEAD
[3] ll 125-129 h=b00e8187a6063e329049ab9a57023fe9113c598d;hb=HEAD
--
#friendly-coders -- We're friendly, but we're not /that/ friendly ;-)
Re: [gentoo-dev] Multilib dependencies need to have >= on least version having EAPI=5 or multilib support
On Thu, Jun 19, 2014 at 1:02 PM, Ciaran McCreesh
wrote:
>> "|| (
>> >=foo/bar-3.5:3[${MULTILIB_USEDEP}]
>> >=foo/bar-4.5:4[${MULTILIB_USEDEP}]
>> >=foo/bar-5.2:5[${MULTILIB_USEDEP}]
>> )"
>>
>> would solve such a problem correctly.
>
> No it wouldn't... Best version first in a || block.
Yes, good point, didn't even think of it. Man, that is some pretty
subtle business. Especially if "best" might change over time...
perhaps in such circumstances it would be better to create a virtual?
Re: [gentoo-dev] Multilib dependencies need to have >= on least version having EAPI=5 or multilib support
On Thu, 19 Jun 2014 12:21:35 -0700
Greg Turner wrote:
> One thing people should be aware of is how this policy will interact
> with slotted ebuilds. >=foo/bar-3.5[${MULTILIB_USEDEP}] could be met
> by the ancient >=foo/bar-4.0 EAPI0 ebuild, but a dep like:
>
> "|| (
> >=foo/bar-3.5:3[${MULTILIB_USEDEP}]
> >=foo/bar-4.5:4[${MULTILIB_USEDEP}]
> >=foo/bar-5.2:5[${MULTILIB_USEDEP}]
> )"
>
> would solve such a problem correctly.
No it wouldn't... Best version first in a || block.
--
Ciaran McCreesh
signature.asc
Description: PGP signature
Re: [gentoo-dev] Multilib dependencies need to have >= on least version having EAPI=5 or multilib support
Ouch!
I suspect a nice side-effect of this policy is that portage will be
faster as it will be less prone to backtracking.
One thing people should be aware of is how this policy will interact
with slotted ebuilds. >=foo/bar-3.5[${MULTILIB_USEDEP}] could be met
by the ancient >=foo/bar-4.0 EAPI0 ebuild, but a dep like:
"|| (
>=foo/bar-3.5:3[${MULTILIB_USEDEP}]
>=foo/bar-4.5:4[${MULTILIB_USEDEP}]
>=foo/bar-5.2:5[${MULTILIB_USEDEP}]
)"
would solve such a problem correctly.
Also, what Sven said! Thanks for your Herculean stick-to-it-iveness,
Michał, and thanks to the others on the multilib team, and the many
who have contributed to the multilib-utilization effort via bugzilla
and IRC! It seems that we are nearing the day when the emul-linux-x86
ebuilds will be empty shells or even a thing of the past.
On Wed, Jun 18, 2014 at 3:13 PM, Michał Górny wrote:
> Hi, developers and users.
>
> TLDR: if you do [${MULTILIB_USEDEP}] or [abi_x86_32], always use >= dep
> on the oldest package version that is EAPI=5 or multilib. In other
> words, the >= dependency must not match a single non-multilib, EAPI<5
> ebuild. And please do not commit new EAPI<5 ebuilds following EAPI=5
> ebuilds.
>
> Long version:
>
> First of all, I'd like to thank all supporters of our little multilib
> effort. We're getting pretty close to finishing stage 1 of our roadmap
> [1], with most of important packages being multilib-ready already [2].
> Most of our effort right now is focused on fixing 32-bit package
> dependencies [3].
>
> However, it was lately brought to my attention that the very complex
> issue regarding portage behavior wrt USE_EXPAND flags and EAPI<5. Since
> PMS is awfully incomprehensible in this area, I won't get into the fine
> details. The same issue was discussed wrt Python in bug #446720 [4].
>
> The important part is that a dependency of the following form:
>
> dev-foo/bar[abi_x86_64(-),abi_x86_32(-)...]
>
> can be satisfied not only by multilib ebuilds. It is also satisfied by
> EAPI<5 ebuilds that don't have abi_* flags in IUSE if the user has
> enabled respective abi_* flags globally. In other words, if the user
> has:
>
> ABI_X86="32 64"
>
> in his make.conf, portage thinks that all EAPI<5 packages (both ebuilds
> and installed ones) instantly started supporting multilib.
>
>
> This has a few important implications:
>
> 1. when trying to install (upgrade) a multilib package, portage may not
> upgrade its dependencies to proper multilib versions, therefore
> triggering seemingly random failures or limitations,
>
> 2. when trying to stabilize a multilib package without stabilizing
> multilib versions of its dependencies, repoman may not complain if
> there is at least a single EAPI<5 version available,
>
> 3. when such a stabilization happens, the upgrade of this package on
> stable arch will cause emerge to downgrade the dependency to EAPI<5
> version.
>
> While we were roughly aware of the first implication, we were not of
> the latter two. This was specifically brought to us by _AxS_ who
> keyworded multilib ffmpeg on stable amd64 (without keywording libsdl)
> and noticed that this causes portage to downgrade EAPI=5, non-multilib
> libsdl-1.2.15-r4 to EAPI=2 libsdl-1.2.15-r2. A similar issue was also
> reported on the Forums [5].
>
>
> We've discussed this with the PMS team and the Portage team, and it
> seems that the only safe and portable way of fixing this is through
> using proper >= dependencies. More specifically, making sure that all
> dependencies on multilib packages require proper versions so that no
> EAPI<5, non-multilib package can satisfy it.
>
> We've decided to satisfy this requirement through using dependencies of
> the following form:
>
> >=dev-foo/bar-${MIN_PV}[${MULTILIB_USEDEP}]
>
> where ${MIN_PV} corresponds to a version that is at least the lower of
> the following two versions:
>
> a) the lowest version of the package that is EAPI=5 and
> is not followed by any EAPI<5 ebuild,
>
> b) the lowest version of the package that supports multilib and is not
> followed by any non-multilib ebuild.
>
> While a) may seem unnecessary considering b), it decreases the number
> of stabilizations necessary. It also makes it possible to backport
> the multilib support without adjusting deps. However, it relies on
> developers not committing new EAPI<5 ebuilds following EAPI=5 ebuilds,
> or new non-multilib ebuilds following multilib ebuilds.
>
>
> Since adding proper versions for every single dependency would be
> tedious, we provide multilib-dep-fixor [6] to help with that. The usage
> is pretty simple:
>
> multilib-dep-fixor *.ebuild
>
> and it tries to rewrite all multilib dependencies into proper >=-deps.
> However, it has a few limitations. More specifically:
>
> a) it doesn't handle variable substitution (${PV}, ${SLOT} in versions),
>
> b) it doesn't handle <, <=, = and ~ deps,
>
> c) it doesn't handle the cases when the newest ebuild is non-multilib
> and EAPI<5 (for examp
Re: [gentoo-dev] Eclass vs EAPI For Utility Functions (Patching/etc)
On Thu, Jun 19, 2014 at 1:03 PM, William Hubbs wrote: > Hi all: > > I am strongly in favor of the eapi-based approach as well, for all of > the reasons mentioned in the thread so far. Good thing your proxy got it right then! :) http://www.gentoo.org/proj/en/council/meeting-logs/20140617-summary.txt Rich
Re: [gentoo-dev] back up and running
On 06/19/14 12:14, William Hubbs wrote: All, after having several hardware and software issues that I finally got some assistance with, I am back up and running. :-) So, now, I should be able to grab OpenRC and gentoo-base-functions by the horns again. :-) I intend to get a new OpenRC out the door, hopefully before next Thursday. Thanks for your patience, William Hi William, please note I pushed out a release of gentoo-functions that i needed to stop my stage builds from breaking. It involved adding a missing include file. See the last commit message at github for the explanation. -- Anthony G. Basile, Ph.D. Gentoo Linux Developer [Hardened] E-Mail: bluen...@gentoo.org GnuPG FP : 1FED FAD9 D82C 52A5 3BAB DC79 9384 FA6E F52D 4BBA GnuPG ID : F52D4BBA
Re: [gentoo-dev] Eclass vs EAPI For Utility Functions (Patching/etc)
Hi all: On Sun, Jun 15, 2014 at 07:00:15AM -0400, Rich Freeman wrote: > During the council meeting there was a bit of a philosophical debate > over the proper role of EAPI vs implementing functions in eclasses. I > felt that it was important enough to at least get more community input > before we continue voting on features like user patching/etc which > tend to favor an EAPI-based approach. I am strongly in favor of the eapi-based approach as well, for all of the reasons mentioned in the thread so far. Eclasses can and should be used for functions, imo, that are used by some ebuilds, but once it is determined that functionality in an eclass has potential for very wide use, that functionality should be moved into an eapi. The eutils functions are a prime example of this. These are general purpose functions, so there is no reason for them to be kept in an eclass. William signature.asc Description: Digital signature
[gentoo-dev] back up and running
All, after having several hardware and software issues that I finally got some assistance with, I am back up and running. :-) So, now, I should be able to grab OpenRC and gentoo-base-functions by the horns again. :-) I intend to get a new OpenRC out the door, hopefully before next Thursday. Thanks for your patience, William signature.asc Description: Digital signature
Re: [gentoo-dev] Multilib dependencies need to have >= on least version having EAPI=5 or multilib support
On Thu, Jun 19, 2014 at 12:13:12AM +0200, Michał Górny wrote: > Hi, developers and users. [...] > Thank you for your cooperation. If you have any questions, please do > not hesitate to ask. Hi Michał Thank you for your endless effort to get Gentoo to this stage (and further). Wkr, Sven Vermeulen PS I'm in a positive mood
Re: [gentoo-dev] [PATCH] qt4-r2.eclass: simplify doc handling
19.06.2014 05:25, hasufell пишет: > Sergey Popov: >> As we should not do anything crazy with DOCS and HTML_DOCS, let's >> simplify our eclass >> > > Just deprecate the whole eclass. I don't see much useful stuff there > except running base.eclass phases and that is already discouraged wrt > #497050. If you remove those parts, not much is left anyway. > We are planning to get rid of base eclass, but we are not planning to change all of our ebuilds to utilize qmake-utils eclass manually, thanks, but would not happen. -- Best regards, Sergey Popov Gentoo developer Gentoo Desktop Effects project lead Gentoo Qt project lead Gentoo Proxy maintainers project lead signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Re: Re: Re: crossdev and multilib interference
Dnia 2014-06-18, o godz. 17:24:03
Peter Stuge napisał(a):
> Alexandre Rostovtsev wrote:
> > current crossdev versions blindly install their
> > /usr/bin/i686-pc-linux-gnu-pkg-config wrapper script, overwriting
> > the binary belonging to pkgconfig[abi_x86_32].
>
> Thanks for getting to the point.
>
> It seems silly for two toolchains (abi_x86_32 and a crossdev i686 toolchain)
> to compete for one and the same name.
>
> Is that really desirable?
No. But the user specifies prefix for crossdev. As far as I'm
concerned, crossdev could refuse prefixes that are already used
in Gentoo profiles.
> Both toolchains integrate with emerge; either one can be used to emerge
> packages, right? (Where the packages are emerged to doesn't matter.)
No, 'integrating with emerge' is a bad word. Crossdev wraps emerge
and runs on top of it. Multilib runs 'below' emerge as run by eclass.
> If Gentoo would like to support different toolchains for one CHOST
> then there obviously needs to be some abstraction, and creating it
> should take into account that things like autoconf, cmake, waf and
> friends may not support any such abstraction for a long time to come.
>
> An obvious low-finesse solution is to only allow one toolchain per CHOST.
Obviously.
> That would formalize the current situation where only one toolchain or
> the other works correctly, and would be fine as a first step.
>
> If long-term Gentoo does indeed want to support both an abi_x86_32
> and a crossdev-built i686-pc-linux-gnu toolchain, then obviously
> someone interested in that will have to come up with how it will
> work.
I'd rather see multilib gcc installing 'i686-pc-linux-gnu' wrappers
that call the native gcc with proper '-m' option. As far as I know,
there's no real difference between the code emitted by native compiler
with -m32 and cross-compiler for i686. This would remove some need for
crossdev (and therefore some breakages) and allow our users to avoid
rebuilding the same thing twice.
> > The initial multilib code didn't use prefixed pkg-config binaries
> > but simple PKG_CONFIG_PATH override.
>
> Do you mean PKG_CONFIG_LIBDIR?
Both.
> > However, crossdev installing i686-pc-linux-gnu-pkg-config caused
> > configure scripts to find and use it instead of plain 'pkg-config',
>
> That sounds to me like autoconf's pkg-config support might also be
> involved in the problem?
The autoconf's AC_PROG_TOOL macros are proper and very useful. What's
the real problem is that crossdev installs some custom wrapper that
clobbers PKG_CONFIG_* where real pkg-config is expected.
> > and since the wrapper dumbly overrides PKG_CONFIG_PATH it broke our
> > original solution.
> >
> > We specifically made pkg-config packages install the prefixed binaries
> > to trigger collisions with crossdev
>
> That strikes me as really unhelpful. You spent time on creating something
> you were sure would cause a problem, instead of on something to *avoid*
> the problem. Oh well.
This was hitting our users and crossdev team didn't care. I did what I
could from our side to fix this, though this probably wasn't good
enough.
Do you prefer if I add some logic to detect i686 crossdev and bail out
completely, telling users to remove it? That wouldn't be very friendly
but at least they wouldn't hit random build failures anymore.
> > -- so that people who have systems broken with crossdev's
> > i686-pc-linux-gnu-pkg-config would be more likely informed there's
> > something wrong with their systems.
>
> You seem to have made an arbitrary decision that crossdev's
> pkg-config is at fault. I think we need to look at this in a larger
> perspective as outlined above; take a step back.
Because it is at fault. Build systems expect *-pkg-config to be
a binary compatible with pkg-config. When crossdev installs something
that does not respect PKG_CONFIG_{LIBDIR,PATH} and instead uses some
fancy directories, it is inevitable that it will break something.
And just to be clear, we didn't invent anything new. The breakage was
there for years, we were just first ones to mix all the ingredients.
The CHOSTs used by our stuff come from profiles, we didn't invent them.
The multilib_toolchain_setup that sets the build environment comes from
multilib.eclass and was already used for multilib in the past. The code
that causes tc-getBUILD_CC to return incorrect (crossdev) compiler
comes from toolchain-funcs.eclass.
If you look at it all, you'd notice that all code that results in those
failures is maintained by toolchain team, and so is crossdev. So what
can small gx86-multilib team to do to fix it?
--
Best regards,
Michał Górny
signature.asc
Description: PGP signature
