[gentoo-dev] RFC: Standard build environment variables
As many of you probably know, ago@ has been expanding the scope of our CFLAGS/CC support to include some other common build variables: * CC * CXX * AR * CPP * NM * RANLIB * AS * LD Some of those are POSIX standards[0], * CC * AR Others are de-facto GNU make standards[1], * CXX * CPP * AS and a few are de-facto GNU libtool standards[2]: * NM * RANLIB * LD If we expect them all to work properly in Gentoo, we have to agree on what they mean, and thus how they should be injected into build systems. For example, we had a problem with sci-mathematics/pari, whose upstream is using the LD environment variable for something other than what GNU libtool uses it for. With LD set to something libtooly in the environment, the pari build fails. We can solve that by unsetting LD in the ebuild, but for that to be The Right Thing To Do, we should be expecting LD to contain something libtooly, and thus something inappropriate to be passed to the pari build. To avoid these issues, I suggest creating a list of "Gentoo environment variables" in the devmanual with descriptions of how they should be used and pointers to the references (for why we chose that meaning). That way a user can export LD, for example, and know that it will be used how he thinks it will be used. [0] https://pubs.opengroup.org/onlinepubs/009695399/utilities/make.html [1] https://www.gnu.org/software/make/manual/html_node/Implicit-Variables.html [2] https://www.gnu.org/software/libtool/manual/libtool.html
Re: [gentoo-dev] RFC: Standard build environment variables
On Sun, 28 Jun 2020 08:18:23 -0400 Michael Orlitzky wrote: > With LD set to something libtooly in the > environment, the pari build fails. We can solve that by unsetting LD in > the ebuild, but for that to be The Right Thing To Do, we should be > expecting LD to contain something libtooly, and thus something > inappropriate to be passed to the pari build. > > To avoid these issues, I suggest creating a list of "Gentoo environment > variables" in the devmanual with descriptions of how they should be used > and pointers to the references (for why we chose that meaning). That way > a user can export LD, for example, and know that it will be used how he > thinks it will be used. That LD problem is not unique to Pari, ftr. Its a problem in *all* Perl things that involve C parts. Because Perl code expects "LD" to be a "CCLD", and passes flags a CCLD would expect, which LD can't handle. pgpasaW8H6QqH.pgp Description: OpenPGP digital signature
Re: [gentoo-dev] RFC: Standard build environment variables
On Sun, Jun 28, 2020 at 8:18 AM Michael Orlitzky wrote: > > As many of you probably know, ago@ has been expanding the scope of our > CFLAGS/CC support to include some other common build variables: > > * CC > * CXX > * AR > * CPP > * NM > * RANLIB > * AS > * LD > > Some of those are POSIX standards[0], > > * CC > * AR > > Others are de-facto GNU make standards[1], > > * CXX > * CPP > * AS > > and a few are de-facto GNU libtool standards[2]: > > * NM > * RANLIB > * LD > > If we expect them all to work properly in Gentoo, we have to agree on > what they mean, and thus how they should be injected into build systems. > For example, we had a problem with sci-mathematics/pari, whose upstream > is using the LD environment variable for something other than what GNU > libtool uses it for. With LD set to something libtooly in the > environment, the pari build fails. We can solve that by unsetting LD in > the ebuild, but for that to be The Right Thing To Do, we should be > expecting LD to contain something libtooly, and thus something > inappropriate to be passed to the pari build. > > To avoid these issues, I suggest creating a list of "Gentoo environment > variables" in the devmanual with descriptions of how they should be used > and pointers to the references (for why we chose that meaning). That way > a user can export LD, for example, and know that it will be used how he > thinks it will be used. Makes sense to me.
Re: [gentoo-dev] RFC: Standard build environment variables
On 20-06-28 08:18:23, Michael Orlitzky wrote:
> As many of you probably know, ago@ has been expanding the scope of our
> CFLAGS/CC support to include some other common build variables:
>
> * CC
> * CXX
> * AR
> * CPP
> * NM
> * RANLIB
> * AS
> * LD
>
> Some of those are POSIX standards[0],
>
> * CC
> * AR
>
> Others are de-facto GNU make standards[1],
>
> * CXX
> * CPP
> * AS
>
> and a few are de-facto GNU libtool standards[2]:
>
> * NM
> * RANLIB
> * LD
>
> If we expect them all to work properly in Gentoo, we have to agree on
> what they mean, and thus how they should be injected into build systems.
> For example, we had a problem with sci-mathematics/pari, whose upstream
> is using the LD environment variable for something other than what GNU
> libtool uses it for. With LD set to something libtooly in the
> environment, the pari build fails. We can solve that by unsetting LD in
> the ebuild, but for that to be The Right Thing To Do, we should be
> expecting LD to contain something libtooly, and thus something
> inappropriate to be passed to the pari build.
>
> To avoid these issues, I suggest creating a list of "Gentoo environment
> variables" in the devmanual with descriptions of how they should be used
> and pointers to the references (for why we chose that meaning). That way
> a user can export LD, for example, and know that it will be used how he
> thinks it will be used.
>
>
> [0] https://pubs.opengroup.org/onlinepubs/009695399/utilities/make.html
>
> [1]
> https://www.gnu.org/software/make/manual/html_node/Implicit-Variables.html
>
> [2] https://www.gnu.org/software/libtool/manual/libtool.html
>
I'd like to suggest allowing only approved variables in the build
environment, having portage unset all variables and setting only what is
needed (or configured).
I'd like to see some variable like this used in make.conf (or maybe one
exists that I don't know about...).
# set the default bash array if GENTOO_EMERGE_ENV is not defined as an array
if ! declare -p GENTOO_EMERGE_ENV 2> /dev/null | grep -q '^declare \-a'; then
declare -a GENTOO_EMERGE_ENV
GENTOO_EMERGE_ENV+=("USE=\"-build\"")
GENTOO_EMERGE_ENV+=("FEATURES=\"binpkg-multi-instance buildpkg
parallel-fetch parallel-install\"")
GENTOO_EMERGE_ENV+=("PKGDIR=\"/tmp/portage-pkgdir\"")
GENTOO_EMERGE_ENV+=("DISTDIR=\"/tmp/portage-distdir\"")
GENTOO_EMERGE_ENV+=("PORTDIR=\"/tmp/portage-portdir\"")
export GENTOO_EMERGE_ENV
fi
# itterate over the array, exporting each 'line'
for (( i=0; i<${#GENTOO_EMERGE_ENV[@]}; i++ )); do
eval export "${GENTOO_EMERGE_ENV[i]}"
done
--
Matthew Thode (prometheanfire)
signature.asc
Description: PGP signature
Re: [gentoo-dev] RFC: Standard build environment variables
On 2020-06-30 12:22, Matthew Thode wrote: > > I'd like to suggest allowing only approved variables in the build > environment, having portage unset all variables and setting only what is > needed (or configured). I think this is orthogonal to the problem I'm trying to solve. Even if all environment variables had to be whitelisted, ebuilds would still need to know how to use them when they happen to be defined. I basically just want to write down things like "If set, CC is assumed to contain the name of a compiler driver such as /usr/bin/gcc." That way ebuilds can be written to pass $CC to the build system in places that are expecting a compiler driver. Conversely, if LD is documented to contain a dynamic linker such as /bin/ld, then ebuilds must mangle LD whenever the upstream build system (e.g. pari, perl) interprets it otherwise. These meanings are already enshrined in the tc-getFOO() functions and the various de-facto standards, but there's no user or developer documentation promising that the variables will be used in any particular way.
Re: [gentoo-dev] RFC: Standard build environment variables
On Wed, Jul 1, 2020 at 9:36 AM Michael Orlitzky wrote: > > On 2020-06-30 12:22, Matthew Thode wrote: > > > > I'd like to suggest allowing only approved variables in the build > > environment, having portage unset all variables and setting only what is > > needed (or configured). > > I think this is orthogonal to the problem I'm trying to solve. Even if > all environment variables had to be whitelisted, ebuilds would still > need to know how to use them when they happen to be defined. > Agree. I'm not actually certain what that proposal was intended to convey. Are we talking about: 1. Blocking anything that happens to be in the environment when emerge is run? (Ie 'CFLAGS="-O2" emerge -1 foo'?) 2. Blocking any variable at all that isn't whitelisted by an ebuild or eclass? (ie CFLAGS in make.conf is ignored unless the ebuild whitelists it) I get how environment pollution can cause issues, but #1 is something we've generally supported for a long time, and it is useful for troubleshooting/etc or just trying out different things. Maybe a FEATURE flag could be used to control it to keep newbs out of trouble, and you can just as easily pass that in the environment too. I'm not sure that #2 adds a lot of value. The default phase functions probably already don't work well for exotic build systems, and eclasses can of course take care of remapping for most of the popular ones. For one-offs some flag-o-matic or other eclass functions to aid in remapping variables might be helpful in some cases if there isn't already something there. But in any case it isn't essential to what you're proposing. It does go along with it to a degree and is worth at least thinking about (imo)... -- Rich
