Re: GLEP draf for cross-compile support in multilib profiles (was: Re: [gentoo-dev] Re: spec draft for cross-compile support in future EAPI (EAPI-5))
On Fri, Jun 29, 2012 at 10:30 AM, Thomas Sachau to...@gentoo.org wrote: I'm interested in this because I'm regularly annoyed with the emul- packages and also because multilib is pretty important for mips. If a package has dependencies, then those dependencies are required to have at least the same targets enabled as the package That seems like the obvious (but perhaps naive) choice. What about depending on packages that don't install libraries, like x11-proto/ packages or generators like dev-util/indent? Maybe I just don't understand. Would these packages even have ABI flags? It's clear to me that libraries would be installed in different lib* directories dependent on their ABI, but how would typical executables be handled? For mips, we'd like to have gcc built as an n64 binary, which would require its run-time dependencies (mpfr, mpc, gmp, etc.) to be available as n64 as well, but the rest of the system to be n32 binaries. Does this fit with your understanding (and proposed solution) of the problem? Thanks, Matt
GLEP draf for cross-compile support in multilib profiles (was: Re: [gentoo-dev] Re: spec draft for cross-compile support in future EAPI (EAPI-5))
Brian Harring schrieb: You need a glep here frankly; per the norm, if you want things to move faster, then put in time- aka, generate a patch against PMS, write a patch for portage, etc, you get the idea. The bit re: a PMS patch is mostly that in looking at your proposal... well, I personally don't want to write that patch (nor do I suspect ulm/ciaran do either). As written earlier, a patch for portage exists already (just the diff between multilib and master branch of portage). Now, in addition, a GLEP draft for cross-compile support in multilib-portage. One thing to note; this has been posted for all of 2-3 days; that's not exactly much time for 1) people to comment, 2) people to frankly comprehend the quite dense description you wrote. This may be true, but as you may have seen afterwards, beside the responses to my mail about planned council asking, there have not been any further responses, so the relatively short time since my mail doesnt seem like the reason for missing responses. Please write a glep covering details of the implementation, background, preferablly why this route over others. Bluntly... clue everyone else in rather than hoping they'll just sign off on a fairly opaque list of things. :) It'll be useful for dev education also- which is a bit of a requirement for stuff of this sort considering it's not going to be a magic deploy/shit works everywhere situation I suspect. See attached GLEP draft. Would also be useful getting commentary from crossdev folk considering your solution is intended to be (best I can tell) full cross compilation support, and they've been leading that front for many, many years. There is an important difference between my cross-compile support for multilib profiles and crossdev: My suggestion works with the default toolchain, which itself is able to cross-compile for other targets (like the amd64/multilib gcc is able to create 32bit libs), while crossdev is using a different toolchain for the targets. Of course, if someone from crossdev maintainers wants to comment, he is free to do so. -- Thomas Sachau Gentoo Linux Developer GLEP: XXX Title: Crosscompile support for multilib profiles Version: $Revision: 1.4 $ Last-Modified: $Date: 2003/07/19 12:09:20 $ Author: Thomas Sachau to...@gentoo.org Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 24 Jun 2012 Post-History: 2-Jun-2003 Abstract Currently, multilib profiles like amd64 do support other targets, but there is no simple way to build packages for those seperate targets. A workaround for this issue are emul packages, which contain precompiled 32bit libs. This GLEP extends the package manager to allow the user to build packages from source for targets supported by the profile and toolchain. Motivation == There is no way to build packages for other then the default target also the profile and toolchain do support them. For amd64, there are precompiled emul packages, which only support a predefined subset of packages, are not compiled with the respect for the user USE or comiler flags and tend to become outdated due to the amount of packages they include. Specification = This GLEP will extend the functionality of package managers with a future EAPI. With this EAPI, users will be able to select additional or different targets per package by adjusting the additional USE flags. Ebuilds themselves dont have to be modified to support this. If a dependency does support the future EAPI, depending packages may require a specific target to be enabled via use dependencies. Exmaple: profile: amd64/multilib package dev-libs/a uses the future EAPI packge app-misc/b requires 32bit libs from dev-libs/a It can now define this in the dependency section as following: DEPEND=dev-libs/a[multilib_abi_x86] Rationale = The proposal does add USE flags for each supported target ABI. This allows the user to select per package, if he wants to compile it for the default target, additionally for another target or just for another target. An alternative would have been a global variable defining this support for all packages at once. This was not done, because it would force the user to either build all (even unneeded packages) or nothing with additional or different target support without the ability to choose on a per package level. The proposel does add a USE flag called abiwrapper to control the behaviour for binaries. If this flag is enabled and there is a none-default target enabled, the binaries for each target are preserved and installed with their target as suffix. A symlink is created for the binary name to a wrapper, which does execute the real binary based on a set environment variable. Following alternative implementations have not been selected: * no wrapper: This would mean, that the installed binaries change, depending on the selected targets. Since some binaries have target
Re: [gentoo-dev] Re: spec draft for cross-compile support in future EAPI (EAPI-5)
On Tue, 19 Jun 2012 23:07:01 +0200 Thomas Sachau to...@gentoo.org wrote: Do you prefer having everything hardcoded in PMS or can you accept outsourcing bigger code pieces into some sort of eclass (i am thinking about some external code base, which can be duplicated by the package manager with internal code, but has to be used, if the external eclass has a newer version/revision then the duplicated internal code)? The package manager mustn't require any particular eclass to be present, and there mustn't be duplication between eclasses and the package manager. I am especially thinking about the setup of the environment and the code details for the wrappers for binaries and headers, hardcoding those details into PMS makes it hard to change/fix issues later on. Sounds like you haven't really got a clean design then. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Re: spec draft for cross-compile support in future EAPI (EAPI-5)
Thomas Sachau schrieb: Duncan schrieb: Thomas Sachau posted on Sat, 16 Jun 2012 12:31:40 +0200 as excerpted: Since i am not that sure about my ability to write formal specs, i am presenting my first draft for further review and suggestions for improvement. Just a format suggestion. Call it nitpicky if you want, and yes, my client isn't perfect, but I'm sure people with a bit of experience writing such specs will tell you I'm not alone... Several of your points ended up as very long single lines. My client can wrap, but that wraps the points as well (so for example 2.1 starts in the middle of a line). So I was left with the choice to either massively horizontally scroll, or of trying to figure out where one point ended and another began, since wrapping it... /wrapped/ it, so points appeared in the middle of a line. Please: * If you use long lines, leave a vertical space (blank line) between points so when a client wraps them, they wrap as individual paragraphs. * Alternatively, wrap at something sensible. (The traditional wrap for posting is 72 chars or so, 80 minus a few to allow a few levels of quoting without rewrap. I wouldn't complain at 90, but if you're going to bother, you might as well go the standard route and avoid further issues.) Long lines as paragraphs would probably be easier especially early in the process when you're modifying a lot, but you still risk (even more) limited clients having issues with it. YMMV. I suggest you look for a better client to handle the line wrapping better. ;-) In the meantime, the same file attached with wrapped lines. Since there is again no response at all, it seems like everyone is ok with this, so i will propose to add this to the next council agenda for EAPI-5 addition. -- Thomas Sachau Gentoo Linux Developer signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Re: spec draft for cross-compile support in future EAPI (EAPI-5)
On Tue, 19 Jun 2012 20:16:39 +0200 Thomas Sachau to...@gentoo.org wrote: Since there is again no response at all, it seems like everyone is ok with this, so i will propose to add this to the next council agenda for EAPI-5 addition. Got a diff for PMS? -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Re: spec draft for cross-compile support in future EAPI (EAPI-5)
Ciaran McCreesh schrieb: On Tue, 19 Jun 2012 20:16:39 +0200 Thomas Sachau to...@gentoo.org wrote: Since there is again no response at all, it seems like everyone is ok with this, so i will propose to add this to the next council agenda for EAPI-5 addition. Got a diff for PMS? Last time you only requested enough details for implementation, you did not require a PMS diff, so i wrote more details for the implementation. If you, Brian (for pkgcore) and zmedico accept this for EAPI-5, i might look into creating a diff against PMS but until then, i dont want to waste my time, especially since noone commented on the implementation details or the technical details and any change would require even more work to rework/adjust the PMS diff. -- Thomas Sachau Gentoo Linux Developer signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Re: spec draft for cross-compile support in future EAPI (EAPI-5)
On Tue, 19 Jun 2012 20:54:07 +0200 Thomas Sachau to...@gentoo.org wrote: Ciaran McCreesh schrieb: On Tue, 19 Jun 2012 20:16:39 +0200 Thomas Sachau to...@gentoo.org wrote: Since there is again no response at all, it seems like everyone is ok with this, so i will propose to add this to the next council agenda for EAPI-5 addition. Got a diff for PMS? Last time you only requested enough details for implementation, you did not require a PMS diff, so i wrote more details for the implementation. If you, Brian (for pkgcore) and zmedico accept this for EAPI-5, i might look into creating a diff against PMS but until then, i dont want to waste my time, especially since noone commented on the implementation details or the technical details and any change would require even more work to rework/adjust the PMS diff. Here's the thing: I doubt the lack of feedback is down to everyone being happy with the proposal. I strongly suspect it's because people look at it and go huh?. You've provided exactly the kind of information that no-one cares about (e.g. long lists of variable names, which will probably just come from a var in profiles), and none of what really matters. I think you'll get a better response if you write this up in GLEP form (i.e. motivation, examples etc) to describe to ebuild developers what's in it for them, and as a diff against PMS so that package mangler authors can work out exactly what you're saying. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Re: spec draft for cross-compile support in future EAPI (EAPI-5)
On Tue, Jun 19, 2012 at 08:54:07PM +0200, Thomas Sachau wrote: Ciaran McCreesh schrieb: On Tue, 19 Jun 2012 20:16:39 +0200 Thomas Sachau to...@gentoo.org wrote: Since there is again no response at all, it seems like everyone is ok with this, so i will propose to add this to the next council agenda for EAPI-5 addition. Got a diff for PMS? Last time you only requested enough details for implementation, you did not require a PMS diff, so i wrote more details for the implementation. If you, Brian (for pkgcore) and zmedico accept this for EAPI-5, i might look into creating a diff against PMS but until then, i dont want to waste my time, especially since noone commented on the implementation details or the technical details and any change would require even more work to rework/adjust the PMS diff. You need a glep here frankly; per the norm, if you want things to move faster, then put in time- aka, generate a patch against PMS, write a patch for portage, etc, you get the idea. The bit re: a PMS patch is mostly that in looking at your proposal... well, I personally don't want to write that patch (nor do I suspect ulm/ciaran do either). One thing to note; this has been posted for all of 2-3 days; that's not exactly much time for 1) people to comment, 2) people to frankly comprehend the quite dense description you wrote. Please write a glep covering details of the implementation, background, preferablly why this route over others. Bluntly... clue everyone else in rather than hoping they'll just sign off on a fairly opaque list of things. :) It'll be useful for dev education also- which is a bit of a requirement for stuff of this sort considering it's not going to be a magic deploy/shit works everywhere situation I suspect. Would also be useful getting commentary from crossdev folk considering your solution is intended to be (best I can tell) full cross compilation support, and they've been leading that front for many, many years. Cheers- ~brian
Re: [gentoo-dev] Re: spec draft for cross-compile support in future EAPI (EAPI-5)
Brian Harring schrieb: On Tue, Jun 19, 2012 at 08:54:07PM +0200, Thomas Sachau wrote: Ciaran McCreesh schrieb: On Tue, 19 Jun 2012 20:16:39 +0200 Thomas Sachau to...@gentoo.org wrote: Since there is again no response at all, it seems like everyone is ok with this, so i will propose to add this to the next council agenda for EAPI-5 addition. Got a diff for PMS? Last time you only requested enough details for implementation, you did not require a PMS diff, so i wrote more details for the implementation. If you, Brian (for pkgcore) and zmedico accept this for EAPI-5, i might look into creating a diff against PMS but until then, i dont want to waste my time, especially since noone commented on the implementation details or the technical details and any change would require even more work to rework/adjust the PMS diff. You need a glep here frankly; per the norm, if you want things to move faster, then put in time- aka, generate a patch against PMS, write a patch for portage, etc, you get the idea. Since multilib-portage code is just a git branch for the portage code, i effectively have a patch for portage. ;-) Before i start investing more time, a question to the PM developers: Do you prefer having everything hardcoded in PMS or can you accept outsourcing bigger code pieces into some sort of eclass (i am thinking about some external code base, which can be duplicated by the package manager with internal code, but has to be used, if the external eclass has a newer version/revision then the duplicated internal code)? I am especially thinking about the setup of the environment and the code details for the wrappers for binaries and headers, hardcoding those details into PMS makes it hard to change/fix issues later on. -- Thomas Sachau Gentoo Linux Developer signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Re: spec draft for cross-compile support in future EAPI (EAPI-5)
Duncan schrieb:
Thomas Sachau posted on Sat, 16 Jun 2012 12:31:40 +0200 as excerpted:
Since i am not that sure about my ability to write formal specs, i am
presenting my first draft for further review and suggestions for
improvement.
Just a format suggestion. Call it nitpicky if you want, and yes, my
client isn't perfect, but I'm sure people with a bit of experience
writing such specs will tell you I'm not alone...
Several of your points ended up as very long single lines. My client can
wrap, but that wraps the points as well (so for example 2.1 starts in the
middle of a line). So I was left with the choice to either massively
horizontally scroll, or of trying to figure out where one point ended and
another began, since wrapping it... /wrapped/ it, so points appeared in
the middle of a line.
Please:
* If you use long lines, leave a vertical space (blank line) between
points so when a client wraps them, they wrap as individual paragraphs.
* Alternatively, wrap at something sensible. (The traditional wrap for
posting is 72 chars or so, 80 minus a few to allow a few levels of
quoting without rewrap. I wouldn't complain at 90, but if you're going
to bother, you might as well go the standard route and avoid further
issues.)
Long lines as paragraphs would probably be easier especially early in the
process when you're modifying a lot, but you still risk (even more)
limited clients having issues with it. YMMV.
I suggest you look for a better client to handle the line wrapping
better. ;-) In the meantime, the same file attached with wrapped lines.
--
Thomas Sachau
Gentoo Linux Developer
For amd64 users, there is sometimes the issue, that they need 32bit libs for
certain packages (e.g. wine or many binary-only packages). Currently,
they only get them prepackaged in binary form with the emul-linux-x86-*
packages. But those packages have a few issues (list does not have to be
complete):
-they only contain a limited set of 32bit packages
-they are precompiled, so the user cannot define his own flags
-they have to be manually maintained and updated
So the idea was to add the ability to compile 32bit packages with support from
the package manager, so there is no need for additional packages to
maintain. After this was originally implemented, it was further extended
to allow cross-compilation for other targets, not only limited to 32bit
packages.
The basic way, how this should work:
1. Check for the current profile being multilib, this means, that the
MULTILIB_ABIS var exists and has more then 1 (space seperated) value.
Additionally, the DEFAULT_ABI var has to be defined and its content should
be part of the MULTILIB_ABIS var. Only continue with the following steps,
if this is true
2.1 Take the entries from MULTILIB_ABIS var and add a USE flag for each of them
in the form of multilib_abi_$value
2.2 Add abiwrapper as a USE flag
3. Check, if the package has USE=multilib enabled. Only continue with the
following steps, if this is false.
4. Add a use dependency for each USE flag added in step 2 for all dependencies
except those defined in a space seperated list of the NO_AUTO_FLAG var. The
dependency should then look like category/package[multilib_abi_$value?]
5. Find the first enabled USE flag from the list of step 2, start with
multilib_abi_$DEFAULT_ABI during the search. If none is enabled, use
multilib_abi_$DEFAULT_ABI. In the following, $ABI will reference the $value
part of the selected USE flag.
6. Before the pkg_setup phase is executed, setup the environment as following:
-export CHOST with $CHOST_$DEFAULT_ABI
-export $CC with $CHOST-gcc
-export $CXX with $CHOST-g++
-export $FC with $CHOST-gfortran
-export $CHOST with $CHOST_$ABI
-export $CBUILD with $CHOST_$ABI
-export $CDEFINE with $CDEFINE_$ABI
-export $CCASFLAGS with ${CCASFLAGS:-${CFLAGS}} and append $CFLAGS_$ABI
-export $CFLAGS with $CFLAGS and append $CFLAGS_$ABI
-export $CPPFLAGS with $CPPFLAGS and append $CPPFLAGS_$ABI
-export $CXXFLAGS with $CXXFLAGS and append $CFLAGS_$ABI
-export $FCFLAGS with $FCFLAGS and append $CFLAGS_$ABI
-export $FFLAGS with $FFLAGS and append $CFLAGS_$ABI
-export $ASFLAGS with $ASFLAGS and append $ASFLAGS_$ABI
-export $LDFLAGS with $LDFLAGS and append $CFLAGS_$ABI
-export $PKG_CONFIG_PATH with /usr/$LIBDIR_$ABI/pkgconfig
7. After src_install is finished:
7.1 Execute the following or equivalent code (prep_ml_binaries is a function
from multilib-portage from [1]):
prep_ml_binaries $(find ${D}usr/bin ${D}usr/sbin ${D}bin ${D}sbin -type
f -name '*-config' 2/dev/null)
7.2 If USE flag abiwrapper is enabled, execute:
local noabi=()
for i in ${MULTILIB_ABIS}; do
noabi+=( ! -name '*-'${i} )
done
if use abiwrapper ; then
for i in $(find ${D}usr/bin/ ${D}usr/sbin ${D}bin ${D}sbin
-type f ${noabi[@]} 2/dev/null); do
[gentoo-dev] Re: spec draft for cross-compile support in future EAPI (EAPI-5)
Thomas Sachau posted on Sun, 17 Jun 2012 14:02:26 +0200 as excerpted: I suggest you look for a better client to handle the line wrapping better. ;-) In the meantime, the same file attached with wrapped lines. Thanks. (And I'm actually involved upstream tho not as a coder, so maybe...) -- Duncan - List replies preferred. No HTML msgs. Every nonfree program has a lord, a master -- and if you use the program, he is your master. Richard Stallman
[gentoo-dev] Re: spec draft for cross-compile support in future EAPI (EAPI-5)
Thomas Sachau posted on Sat, 16 Jun 2012 12:31:40 +0200 as excerpted: Since i am not that sure about my ability to write formal specs, i am presenting my first draft for further review and suggestions for improvement. Just a format suggestion. Call it nitpicky if you want, and yes, my client isn't perfect, but I'm sure people with a bit of experience writing such specs will tell you I'm not alone... Several of your points ended up as very long single lines. My client can wrap, but that wraps the points as well (so for example 2.1 starts in the middle of a line). So I was left with the choice to either massively horizontally scroll, or of trying to figure out where one point ended and another began, since wrapping it... /wrapped/ it, so points appeared in the middle of a line. Please: * If you use long lines, leave a vertical space (blank line) between points so when a client wraps them, they wrap as individual paragraphs. * Alternatively, wrap at something sensible. (The traditional wrap for posting is 72 chars or so, 80 minus a few to allow a few levels of quoting without rewrap. I wouldn't complain at 90, but if you're going to bother, you might as well go the standard route and avoid further issues.) Long lines as paragraphs would probably be easier especially early in the process when you're modifying a lot, but you still risk (even more) limited clients having issues with it. YMMV. -- Duncan - List replies preferred. No HTML msgs. Every nonfree program has a lord, a master -- and if you use the program, he is your master. Richard Stallman
