Re: [gentoo-dev] Support for multiple ABIs (for Python modules etc.)
2009-08-01 20:10:49 Ciaran McCreesh napisał(a):
On Sat, 25 Jul 2009 12:28:44 +0200
Arfrever Frehtes Taifersar Arahesis arfre...@gentoo.org wrote:
I would like to present the plan of support for multiple ABIs. It
should be sufficient for Python modules and might be also appropriate
for some other ABI types (e.g. for Ruby modules).
How do you plan to handle the intersection of ABIs? What if you have to
build or depend upon a Python module for both 32 and 64 bit ABIs, and
for both 2.5 and 2.6? What if you have a package that provides both
Ruby and Python code, where the two ABIs are independent rather than a
product?
The proposition already handles these cases. Please describe in more details
what you don't
understand.
4.1. Implicitly specified ABI dependencies. During calculation of
dependencies of given package, Portage will verify if all
dependencies, which use given ABI type, have been built with enabled
support for these ABIs, which are enabled for given package.
How do you say I need only a single ABI for this, even though it looks
like I need every ABI I'm built with? For example, if your Python
module, being built for 2.5 and 2.6, runs (but does not use in a
library sense) a Python program that comes as part of a Python package
that is buildable with multiple ABIs?
In such case a Python package, which is a dependency of another Python package,
shouldn't
be marked as supporting multiple Python ABIs. But anyway I suggest the
following syntax:
category/package_name{python[*]}
4.2. Explicitly specified ABI dependencies. {,P,R}DEPEND variables
will support specifying ABI dependencies in explicit way.
{,P,R}DEPEND variables will also support ABI conditionals. I suggest
using {ABI_type[comma-delimited values]} syntax, but it can be
changed.
I think we're trying to avoid introducing new special characters if we
can get away with using existing ones. You can overload [] and existing
conditionals if you're careful.
The syntax suggested by me looks better for this purpose.
Having said that, you can probably do everything you described slightly
less elegantly just using things that're already in EAPI 3.
Not everything.
--
Arfrever Frehtes Taifersar Arahesis
signature.asc
Description: This is a digitally signed message part.
Re: [gentoo-dev] Support for multiple ABIs (for Python modules etc.)
On Sun, Jul 26, 2009 at 5:21 PM, Arfrever Frehtes Taifersar Arahesisarfre...@gentoo.org wrote: 2009-07-26 14:40:13 Marijn Schouten (hkBst) napisał(a): Arfrever Frehtes Taifersar Arahesis wrote: I would like to present the plan of support for multiple ABIs. It should be sufficient for Python modules and might be also appropriate for some other ABI types (e.g. for Ruby modules). In the context of which problem are you brainstorming? This proposition is intended to solve multiple problems, but Gentoo Python Project especially would like to have it available during transitions to new Python versions (e.g. Python 3.*). Python 3.1 will be added to the tree in the next week. Over 10 Python modules should work with Python 3, but the majority of packages doesn't work yet. We want to provide possibility of installing Python modules into site-packages directories of multiple Python versions (e.g. 2.6 and 3.1). In currently existing EAPIs we *will* support it, but without checking Python ABI dependencies during dependency calculation. I don't think this is easy to do, but I think the solution to this problem should be the same as the (as yet not existing) solution to the multi-ABI problem as in (x86_64 vs. ix86). The biggest issue is to handle multiple instances of the same package and how to handle overlapping (ABI independent) files. Paul -- Paul de Vrieze
Re: [gentoo-dev] Support for multiple ABIs (for Python modules etc.)
On Sat, 25 Jul 2009 12:28:44 +0200
Arfrever Frehtes Taifersar Arahesis arfre...@gentoo.org wrote:
I would like to present the plan of support for multiple ABIs. It
should be sufficient for Python modules and might be also appropriate
for some other ABI types (e.g. for Ruby modules).
How do you plan to handle the intersection of ABIs? What if you have to
build or depend upon a Python module for both 32 and 64 bit ABIs, and
for both 2.5 and 2.6? What if you have a package that provides both
Ruby and Python code, where the two ABIs are independent rather than a
product?
4.1. Implicitly specified ABI dependencies. During calculation of
dependencies of given package, Portage will verify if all
dependencies, which use given ABI type, have been built with enabled
support for these ABIs, which are enabled for given package.
How do you say I need only a single ABI for this, even though it looks
like I need every ABI I'm built with? For example, if your Python
module, being built for 2.5 and 2.6, runs (but does not use in a
library sense) a Python program that comes as part of a Python package
that is buildable with multiple ABIs?
4.2. Explicitly specified ABI dependencies. {,P,R}DEPEND variables
will support specifying ABI dependencies in explicit way.
{,P,R}DEPEND variables will also support ABI conditionals. I suggest
using {ABI_type[comma-delimited values]} syntax, but it can be
changed.
I think we're trying to avoid introducing new special characters if we
can get away with using existing ones. You can overload [] and existing
conditionals if you're careful.
Having said that, you can probably do everything you described slightly
less elegantly just using things that're already in EAPI 3.
--
Ciaran McCreesh
signature.asc
Description: PGP signature
Re: [gentoo-dev] Support for multiple ABIs (for Python modules etc.)
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Arfrever Frehtes Taifersar Arahesis wrote:
I would like to present the plan of support for multiple ABIs. It should be
sufficient for
Python modules and might be also appropriate for some other ABI types (e.g.
for Ruby modules).
In the context of which problem are you brainstorming?
Marijn
- --
If you cannot read my mind, then listen to what I say.
Marijn Schouten (hkBst), Gentoo Lisp project, Gentoo ML
http://www.gentoo.org/proj/en/lisp/, #gentoo-{lisp,ml} on FreeNode
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iEYEARECAAYFAkpsTq0ACgkQp/VmCx0OL2xqtACglJ7tpOXL4DAeSvo+sLpSc5ir
NsEAoLmyH9EbXydfk4dpCJiHxLqr6WFS
=lmdT
-END PGP SIGNATURE-
Re: [gentoo-dev] Support for multiple ABIs (for Python modules etc.)
2009-07-26 14:40:13 Marijn Schouten (hkBst) napisał(a): Arfrever Frehtes Taifersar Arahesis wrote: I would like to present the plan of support for multiple ABIs. It should be sufficient for Python modules and might be also appropriate for some other ABI types (e.g. for Ruby modules). In the context of which problem are you brainstorming? This proposition is intended to solve multiple problems, but Gentoo Python Project especially would like to have it available during transitions to new Python versions (e.g. Python 3.*). Python 3.1 will be added to the tree in the next week. Over 10 Python modules should work with Python 3, but the majority of packages doesn't work yet. We want to provide possibility of installing Python modules into site-packages directories of multiple Python versions (e.g. 2.6 and 3.1). In currently existing EAPIs we *will* support it, but without checking Python ABI dependencies during dependency calculation. -- Arfrever Frehtes Taifersar Arahesis signature.asc Description: This is a digitally signed message part.
[gentoo-dev] Support for multiple ABIs (for Python modules etc.)
I would like to present the plan of support for multiple ABIs. It should be
sufficient for
Python modules and might be also appropriate for some other ABI types (e.g. for
Ruby modules).
1. Portage will support repository-specific, profile-independent
${REPOSITORY}/profiles/settings
configuration files (filename can be changed). They will set some variables
(similarly to
make.defaults files). These variables shouldn't be overrideable by profiles,
eclasses, ebuilds
or by user's configuration files (e.g. /etc/portage/*). For future
compatibility these files
could also support lists/indexed arrays and dictionaries/associative
arrays/maps.
2. ${REPOSITORY}/profiles/settings will set ABI_TYPES and
${ABI_TYPE^^}_ABI_SUPPORTED_VALUES variables.
ABI_TYPES defines supported ABI types. ${ABI_TYPE^^}_ABI_SUPPORTED_VALUES
defines supported ABIs
for given ABI type.
Example (${REPOSITORY}/profiles/settings):
ABI_TYPES=python ruby ...
PYTHON_ABI_SUPPORTED_VALUES=2.4 2.5 2.6 3.0 3.1
RUBY_ABI_SUPPORTED_VALUES=1.8 1.9
3. make.defaults files will support
${ABI_TYPE^^}_ABI_FORCED_VALUES/${ABI_TYPE^^}_ABI_MASKED_VALUES
variables to force/mask specific ABIs in given profiles.
Example (make.defaults):
PYTHON_ABI_MASKED_VALUES=2.4 2.5
PYTHON_ABI_FORCED_VALUES=3.1
4. For convenience of developers ABI dependencies will be able to be specified
in explicit and
slightly implicit way. Ebuilds/eclasses, which support multiple ABIs, will set
USED_ABI_TYPES.
Ebuilds/eclasses will be able to set RESTRICT_${ABI_TYPE^^}_ABIS variables
which will support
'*' wildcard. USED_ABI_TYPES and RESTRICT_${ABI_TYPE^^}_ABIS variables should
be cumulative
(across eclasses and ebuild).
Example (ebuild):
USED_ABI_TYPES=python
# This package doesn't work yet with Python 2.6 and 3.*
RESTRICT_PYTHON_ABIS=2.6 3*
4.1. Implicitly specified ABI dependencies. During calculation of dependencies
of given package,
Portage will verify if all dependencies, which use given ABI type, have been
built with enabled
support for these ABIs, which are enabled for given package.
Example (ebuild):
# dev-python/bsddb3
USED_ABI_TYPES=python
RDEPEND==sys-libs/db-4.6
DEPEND=${RDEPEND}
dev-python/setuptools
doc? ( dev-python/sphinx )
Assuming that dev-python/setuptools and dev-python/sphinx set
USED_ABI_TYPES=python,
Portage will verify that, when user runs `USE_PYTHON=2.5 2.6 emerge
dev-python/bsddb3`,
dev-python/setuptools and dev-python/sphinx (when USE=doc is enabled) are
built with
enabled support for at least 2.5 and 2.6 Python ABIs. If they are
already installed
without support for e.g. 2.5 Python ABI, Portage will print appropriate
error message
(similarly to unsatisfied USE dependencies) and exit.
4.2. Explicitly specified ABI dependencies. {,P,R}DEPEND variables will support
specifying
ABI dependencies in explicit way. {,P,R}DEPEND variables will also support ABI
conditionals.
I suggest using {ABI_type[comma-delimited values]} syntax, but it can be
changed.
Example (ebuild):
DEPEND=category/package1{python[2.5,2.6],ruby[-1.8]}
RDEPEND={python[2.6]}? ( category/package2 )
Example (python.eclass):
DEPEND=
{python[2.4]}? ( dev-lang/python:2.4 )
{python[2.5]}? ( dev-lang/python:2.5 )
{python[2.6]}? ( dev-lang/python:2.6 )
{python[3.0]}? ( dev-lang/python:3.0 )
{python[3.1]}? ( dev-lang/python:3.1 )
RDEPEND=${RDEPEND}
5. 18th line of metadata cache files will contain the list of ABI types and
values supported
(i.e. not restricted by RESTRICT_${ABI_TYPE^^}_ABIS variables) by given package.
Example (18th line of metadata cache file):
python[2.5,2.6,3.0,3.1] ruby[1.8,1.9]
Every change to ${REPOSITORY}/profiles/settings should probably invalidate
whole metadata
cache of given repository.
6. Portage will support USE_${ABI_TYPE^^} variables in /etc/make.conf to
specify which ABIs
should be enabled.
Example (/etc/make.conf):
USE_PYTHON=2.5 2.6 3.1
USE_RUBY=1.8 1.9
7. Portage will set ${ABI_TYPE^^}_ABIS variables in ebuild environment.
${ABI_TYPE^^}_ABIS
should contain ABIs forced by ${ABI_TYPE^^}_ABI_FORCED_VALUES and these ABIs
enabled by user
in /etc/make.conf by USE_${ABI_TYPE^^} which aren't masked by
${ABI_TYPE^^}_ABI_MASKED_VALUES.
Example (ebuild):
src_compile() {
for abi in ${PYTHON_ABIS}; do
emake PYTHON=$(get_PYTHON ${abi})
done
}
Ebuilds/eclasses should implicitly enable support for one (default) ABI, when
user hasn't
set appropriate USE_${ABI_TYPE^^} variable. For example Python modules would
use ABI of
/usr/bin/python.
8. Portage will store the list of supported ABIs in
/var/db/pkg/${CATEGORY}/${PF}/SUPPORTED_ABIS
and the list of enabled ABIs in /var/db/pkg/${CATEGORY}/${PF}/ABIS.
Example (/var/db/pkg/${CATEGORY}/${PF}/SUPPORTED_ABIS):
python[2.4,2.5,2.6,3.0,3.1] ruby[1.8,1.9]
Example
Re: [gentoo-dev] Support for multiple ABIs (for Python modules etc.)
2009-07-25 12:28:44 Arfrever Frehtes Taifersar Arahesis napisał(a):
I would like to present the plan of support for multiple ABIs. It should be
sufficient for
Python modules and might be also appropriate for some other ABI types (e.g.
for Ruby modules).
1. Portage will support repository-specific, profile-independent
${REPOSITORY}/profiles/settings
configuration files (filename can be changed). They will set some variables
(similarly to
make.defaults files). These variables shouldn't be overrideable by profiles,
eclasses, ebuilds
or by user's configuration files (e.g. /etc/portage/*). For future
compatibility these files
could also support lists/indexed arrays and dictionaries/associative
arrays/maps.
2. ${REPOSITORY}/profiles/settings will set ABI_TYPES and
${ABI_TYPE^^}_ABI_SUPPORTED_VALUES variables.
ABI_TYPES defines supported ABI types. ${ABI_TYPE^^}_ABI_SUPPORTED_VALUES
defines supported ABIs
for given ABI type.
Example (${REPOSITORY}/profiles/settings):
ABI_TYPES=python ruby ...
PYTHON_ABI_SUPPORTED_VALUES=2.4 2.5 2.6 3.0 3.1
RUBY_ABI_SUPPORTED_VALUES=1.8 1.9
3. make.defaults files will support
${ABI_TYPE^^}_ABI_FORCED_VALUES/${ABI_TYPE^^}_ABI_MASKED_VALUES
variables to force/mask specific ABIs in given profiles.
Example (make.defaults):
PYTHON_ABI_MASKED_VALUES=2.4 2.5
PYTHON_ABI_FORCED_VALUES=3.1
4. For convenience of developers ABI dependencies will be able to be
specified in explicit and
slightly implicit way. Ebuilds/eclasses, which support multiple ABIs, will
set USED_ABI_TYPES.
Ebuilds/eclasses will be able to set RESTRICT_${ABI_TYPE^^}_ABIS variables
which will support
'*' wildcard. USED_ABI_TYPES and RESTRICT_${ABI_TYPE^^}_ABIS variables should
be cumulative
(across eclasses and ebuild).
Example (ebuild):
USED_ABI_TYPES=python
# This package doesn't work yet with Python 2.6 and 3.*
RESTRICT_PYTHON_ABIS=2.6 3*
4.1. Implicitly specified ABI dependencies. During calculation of
dependencies of given package,
Portage will verify if all dependencies, which use given ABI type, have been
built with enabled
support for these ABIs, which are enabled for given package.
Example (ebuild):
# dev-python/bsddb3
USED_ABI_TYPES=python
RDEPEND==sys-libs/db-4.6
DEPEND=${RDEPEND}
dev-python/setuptools
doc? ( dev-python/sphinx )
Assuming that dev-python/setuptools and dev-python/sphinx set
USED_ABI_TYPES=python,
Portage will verify that, when user runs `USE_PYTHON=2.5 2.6 emerge
dev-python/bsddb3`,
dev-python/setuptools and dev-python/sphinx (when USE=doc is enabled)
are built with
enabled support for at least 2.5 and 2.6 Python ABIs. If they are
already installed
without support for e.g. 2.5 Python ABI, Portage will print appropriate
error message
(similarly to unsatisfied USE dependencies) and exit.
4.2. Explicitly specified ABI dependencies. {,P,R}DEPEND variables will
support specifying
ABI dependencies in explicit way. {,P,R}DEPEND variables will also support
ABI conditionals.
I suggest using {ABI_type[comma-delimited values]} syntax, but it can be
changed.
Example (ebuild):
DEPEND=category/package1{python[2.5,2.6],ruby[-1.8]}
RDEPEND={python[2.6]}? ( category/package2 )
Example (python.eclass):
DEPEND=
{python[2.4]}? ( dev-lang/python:2.4 )
{python[2.5]}? ( dev-lang/python:2.5 )
{python[2.6]}? ( dev-lang/python:2.6 )
{python[3.0]}? ( dev-lang/python:3.0 )
{python[3.1]}? ( dev-lang/python:3.1 )
RDEPEND=${RDEPEND}
5. 18th line of metadata cache files will contain the list of ABI types and
values supported
(i.e. not restricted by RESTRICT_${ABI_TYPE^^}_ABIS variables) by given
package.
Example (18th line of metadata cache file):
python[2.5,2.6,3.0,3.1] ruby[1.8,1.9]
Every change to ${REPOSITORY}/profiles/settings should probably invalidate
whole metadata
cache of given repository.
6. Portage will support USE_${ABI_TYPE^^} variables in /etc/make.conf to
specify which ABIs
should be enabled.
Portage will also allow to set USE_${ABI_TYPE^^} variables per-package etc.
Example (/etc/make.conf):
USE_PYTHON=2.5 2.6 3.1
USE_RUBY=1.8 1.9
7. Portage will set ${ABI_TYPE^^}_ABIS variables in ebuild environment.
${ABI_TYPE^^}_ABIS
should contain ABIs forced by ${ABI_TYPE^^}_ABI_FORCED_VALUES and these ABIs
enabled by user
in /etc/make.conf by USE_${ABI_TYPE^^} which aren't masked by
${ABI_TYPE^^}_ABI_MASKED_VALUES.
${ABI_TYPE^^}_ABIS variables won't contain ABIs restricted by
RESTRICT_${ABI_TYPE^^}_ABIS
variables. Eclasses/ebuilds will be able to change the values of
${ABI_TYPE^^}_ABIS variables
in ebuild phases. For example python.eclass will support ignoring of build
failures for Python
ABIs specified by user in FAILURE_TOLERANT_PYTHON_ABIS variable. Python ABIs,
for
