Please find the patch split into three smaller patch:
* refactoring: contains the Michal's suggestions. I also removed the
whitespace changes.
* profile attributes export: same as in the previous patch
* profile-env patch: uses "has" to check if profile-env is set. I also
included the documentation in portage.5.
The Change-Id git hook was disabled for this round of patches ;)
Bertrand
On Wed, Sep 17, 2014 at 2:57 PM, Bertrand Simonnet <bsimon...@google.com>
wrote:
> Michal, not opposed to splitting the patch into three parts.
>
> I'd rather use the env/ mechanism instead of the package.env one as it is
> more flexible.
> It also feels better as ebuild.sh will walk the profiles to source the
> bashrc script so a bashrc from a
> "low priority" profile may override a package.env definition from a high
> priority profile.
>
> I'll remove the with spaces and the Change-Id (gerrit specific tag I
> believe)
>
> Bertrand
>
>
> On Wed, Sep 17, 2014 at 2:43 PM, Zac Medico <zmed...@gentoo.org> wrote:
>
>> On 09/17/2014 02:28 PM, Michał Górny wrote:
>> >> diff --git a/pym/portage/repository/config.py
>> b/pym/portage/repository/config.py
>> >> index 5e0d055..ef8054e 100644
>> >> --- a/pym/portage/repository/config.py
>> >> +++ b/pym/portage/repository/config.py
>> >> @@ -40,7 +40,7 @@ if sys.hexversion >= 0x3000000:
>> >> _invalid_path_char_re = re.compile(r'[^a-zA-Z0-9._\-+:/]')
>> >>
>> >> _valid_profile_formats = frozenset(
>> >> - ['pms', 'portage-1', 'portage-2'])
>> >> + ['pms', 'portage-1', 'portage-2', 'profile-env'])
>> >
>> > I'm not sure if dedicated profile format is the best way to go. If I
>> > understand your patch correctly, this means that 'profile-env' has only
>> > features of PMS profile but not of 'portage-*' formats. This could mean
>> > that some people would have to choose between features of one or
>> > the other. Since both are compatible, i suggest 'portage-3' instead.
>>
>> The profile-formats field is designed to allow mixing of formats. So,
>> it's legal to mix profile-env with any of the other formats.
>> --
>> Thanks,
>> Zac
>>
>>
>
From 3fb3e4923425af46c084e1f600c16c344ed5a26c Mon Sep 17 00:00:00 2001
From: Bertrand SIMONNET <bsimon...@chromium.org>
Date: Wed, 17 Sep 2014 15:10:13 -0700
Subject: [PATCH 1/3] Refactor bashrc scripts sourcing
Creates two new helper functions __try_source and __source_env_files to simplify
__source_all_bashrcs.
---
bin/ebuild.sh | 75 +++++++++++++++++++++++++++++++++-----------------
bin/save-ebuild-env.sh | 1 +
2 files changed, 51 insertions(+), 25 deletions(-)
diff --git a/bin/ebuild.sh b/bin/ebuild.sh
index be044e0..bca5b59 100755
--- a/bin/ebuild.sh
+++ b/bin/ebuild.sh
@@ -375,39 +375,64 @@ __source_all_bashrcs() {
done
fi
- if [ -r "${PORTAGE_BASHRC}" ] ; then
- if [ "$PORTAGE_DEBUG" != "1" ] || [ "${-/x/}" != "$-" ]; then
- source "${PORTAGE_BASHRC}"
- else
- set -x
- source "${PORTAGE_BASHRC}"
- set +x
- fi
- fi
+ # The user's bashrc is the ONLY non-portage bit of code
+ # that can change shopts without a QA violation.
+ __try_source --no-qa "${PORTAGE_BASHRC}"
if [[ $EBUILD_PHASE != depend ]] ; then
- # The user's bashrc is the ONLY non-portage bit of code that can
- # change shopts without a QA violation.
- for x in "${PM_EBUILD_HOOK_DIR}"/${CATEGORY}/{${PN},${PN}:${SLOT%/*},${P},${PF}}; do
- if [ -r "${x}" ]; then
- # If $- contains x, then tracing has already been enabled
- # elsewhere for some reason. We preserve it's state so as
- # not to interfere.
- if [ "$PORTAGE_DEBUG" != "1" ] || [ "${-/x/}" != "$-" ]; then
- source "${x}"
- else
- set -x
- source "${x}"
- set +x
- fi
- fi
- done
+ __source_env_files --no-qa "${PM_EBUILD_HOOK_DIR}"
fi
[ ! -z "${OCC}" ] && export CC="${OCC}"
[ ! -z "${OCXX}" ] && export CXX="${OCXX}"
}
+# @FUNCTION: __source_env_files
+# @USAGE: [--no-qa] <ENV_DIRECTORY>
+# @DESCRIPTION:
+# Source the files relevant to the current package from the given path.
+# If --no-qa is specified, use source instead of __qa_source to source the
+# files.
+__source_env_files() {
+ local argument=()
+ if [[ $1 == --no-qa ]]; then
+ argument=( --no-qa )
+ shift
+ fi
+ for x in "${1}"/${CATEGORY}/{${PN},${PN}:${SLOT%/*},${P},${PF}}; do
+ __try_source "${argument[@]}" "${x}"
+ done
+}
+
+# @FUNCTION: __try_source
+# @USAGE: [--no-qa] <FILE>
+# @DESCRIPTION:
+# If the path given as argument exists, source the file while preserving
+# $-.
+# If --no-qa is specified, source the file with source instead of __qa_source.
+__try_source() {
+ local qa=true
+ if [[ $1 == --no-qa ]]; then
+ qa=false
+ shift
+ fi
+ if [[ -r "$1" ]]; then
+ local debug_on=false
+ if [[ "$PORTAGE_DEBUG" == "1" ]] && [[ "${-/x/}" == "$-" ]]; then
+ debug_on=true
+ fi
+ [[ ${debug_on} ]] && set -x
+ # If $- contains x, then tracing has already been enabled
+ # elsewhere for some reason. We preserve it's state so as
+ # not to interfere.
+ if [[ ${qa} ]]; then
+ source "${1}"
+ else
+ __qa_source "${1}"
+ fi
+ [[ ${debug_on} ]] && set +x
+ fi
+}
# === === === === === === === === === === === === === === === === === ===
# === === === === === functions end, main part begins === === === === ===
# === === === === === === === === === === === === === === === === === ===
diff --git a/bin/save-ebuild-env.sh b/bin/save-ebuild-env.sh
index 98cff83..f114c48 100644
--- a/bin/save-ebuild-env.sh
+++ b/bin/save-ebuild-env.sh
@@ -75,6 +75,7 @@ __save_ebuild_env() {
__ebuild_main __ebuild_phase __ebuild_phase_with_hooks \
__ebuild_arg_to_phase __ebuild_phase_funcs default \
__unpack_tar __unset_colors \
+ __source_env_files __try_source \
${QA_INTERCEPTORS}
___eapi_has_usex && unset -f usex
--
2.1.0.rc2.206.gedb03e5
From 6b0693574dc2cd71f3089f01eea59ce61142dd77 Mon Sep 17 00:00:00 2001
From: Bertrand SIMONNET <bsimon...@chromium.org>
Date: Wed, 17 Sep 2014 15:12:50 -0700
Subject: [PATCH 2/3] Export profile attributes for ebuild.sh
Expose the profile-formats for all profiles listed in PROFILE_PATHS.
---
bin/save-ebuild-env.sh | 2 +-
pym/portage/package/ebuild/_config/LocationsManager.py | 6 ++++++
pym/portage/package/ebuild/_config/special_env_vars.py | 2 +-
pym/portage/package/ebuild/config.py | 2 ++
pym/portage/package/ebuild/doebuild.py | 2 ++
5 files changed, 12 insertions(+), 2 deletions(-)
diff --git a/bin/save-ebuild-env.sh b/bin/save-ebuild-env.sh
index f114c48..326410b 100644
--- a/bin/save-ebuild-env.sh
+++ b/bin/save-ebuild-env.sh
@@ -102,7 +102,7 @@ __save_ebuild_env() {
PORTAGE_DOHTML_UNWARNED_SKIPPED_EXTENSIONS \
PORTAGE_DOHTML_UNWARNED_SKIPPED_FILES \
PORTAGE_DOHTML_WARN_ON_SKIPPED_FILES \
- PORTAGE_NONFATAL PORTAGE_QUIET \
+ PORTAGE_NONFATAL PORTAGE_PROFILE_ATTRIBUTES PORTAGE_QUIET \
PORTAGE_SANDBOX_DENY PORTAGE_SANDBOX_PREDICT \
PORTAGE_SANDBOX_READ PORTAGE_SANDBOX_WRITE PREROOTPATH \
QA_INTERCEPTORS \
diff --git a/pym/portage/package/ebuild/_config/LocationsManager.py b/pym/portage/package/ebuild/_config/LocationsManager.py
index 4427f1d..345679d 100644
--- a/pym/portage/package/ebuild/_config/LocationsManager.py
+++ b/pym/portage/package/ebuild/_config/LocationsManager.py
@@ -114,6 +114,7 @@ class LocationsManager(object):
# The symlink might not exist or might not be a symlink.
self.profiles = []
self.profiles_complex = []
+ self.profile_formats = []
if self.profile_path:
try:
self._addProfile(os.path.realpath(self.profile_path),
@@ -124,6 +125,7 @@ class LocationsManager(object):
writemsg("!!! ParseError: %s\n" % str(e), noiselevel=-1)
self.profiles = []
self.profiles_complex = []
+ self.profile_formats = []
if self._user_config and self.profiles:
custom_prof = os.path.join(
@@ -137,6 +139,7 @@ class LocationsManager(object):
self.profiles = tuple(self.profiles)
self.profiles_complex = tuple(self.profiles_complex)
+ self.profile_formats = tuple(self.profile_formats)
def _check_var_directory(self, varname, var):
if not isdir_raise_eaccess(var):
@@ -151,6 +154,7 @@ class LocationsManager(object):
allow_parent_colon = True
repo_loc = None
compat_mode = False
+ current_formats = ()
eapi_file = os.path.join(currentPath, "eapi")
eapi = "0"
@@ -183,6 +187,7 @@ class LocationsManager(object):
layout_data['profile-formats'] == ('portage-1-compat',)
allow_parent_colon = any(x in _allow_parent_colon
for x in layout_data['profile-formats'])
+ current_formats = tuple(layout_data['profile-formats'])
if compat_mode:
offenders = _PORTAGE1_DIRECTORIES.intersection(os.listdir(currentPath))
@@ -234,6 +239,7 @@ class LocationsManager(object):
self.profiles.append(currentPath)
self.profiles_complex.append(
_profile_node(currentPath, allow_directories, False))
+ self.profile_formats.append(current_formats)
def _expand_parent_colon(self, parentsFile, parentPath,
repo_loc, repositories):
diff --git a/pym/portage/package/ebuild/_config/special_env_vars.py b/pym/portage/package/ebuild/_config/special_env_vars.py
index 74fedd6..b929042 100644
--- a/pym/portage/package/ebuild/_config/special_env_vars.py
+++ b/pym/portage/package/ebuild/_config/special_env_vars.py
@@ -68,7 +68,7 @@ environ_whitelist += [
"PORTAGE_INST_GID", "PORTAGE_INST_UID",
"PORTAGE_IPC_DAEMON", "PORTAGE_IUSE", "PORTAGE_ECLASS_LOCATIONS",
"PORTAGE_LOG_FILE", "PORTAGE_OVERRIDE_EPREFIX", "PORTAGE_PIPE_FD",
- "PORTAGE_PYM_PATH", "PORTAGE_PYTHON",
+ "PORTAGE_PROFILE_ATTRIBUTES", "PORTAGE_PYM_PATH", "PORTAGE_PYTHON",
"PORTAGE_PYTHONPATH", "PORTAGE_QUIET",
"PORTAGE_REPO_NAME", "PORTAGE_REPOSITORIES", "PORTAGE_RESTRICT",
"PORTAGE_SIGPIPE_STATUS",
diff --git a/pym/portage/package/ebuild/config.py b/pym/portage/package/ebuild/config.py
index f639e14..37e71f9 100644
--- a/pym/portage/package/ebuild/config.py
+++ b/pym/portage/package/ebuild/config.py
@@ -247,6 +247,7 @@ class config(object):
self.module_priority = clone.module_priority
self.profile_path = clone.profile_path
self.profiles = clone.profiles
+ self.profile_formats = clone.profile_formats
self.packages = clone.packages
self.repositories = clone.repositories
self.unpack_dependencies = clone.unpack_dependencies
@@ -554,6 +555,7 @@ class config(object):
profiles_complex = locations_manager.profiles_complex
self.profiles = locations_manager.profiles
+ self.profile_formats = locations_manager.profile_formats
self.profile_path = locations_manager.profile_path
self.user_profile_dir = locations_manager.user_profile_dir
diff --git a/pym/portage/package/ebuild/doebuild.py b/pym/portage/package/ebuild/doebuild.py
index 01707ae..0b8c3fa 100644
--- a/pym/portage/package/ebuild/doebuild.py
+++ b/pym/portage/package/ebuild/doebuild.py
@@ -336,6 +336,8 @@ def doebuild_environment(myebuild, mydo, myroot=None, settings=None,
mysettings["SANDBOX_LOG"] = mycpv.replace("/", "_-_")
mysettings["PROFILE_PATHS"] = "\n".join(mysettings.profiles)
+ mysettings["PORTAGE_PROFILE_ATTRIBUTES"] = \
+ "\n".join([" ".join(formats) for formats in mysettings.profile_formats])
mysettings["P"] = mysplit[0]+"-"+mysplit[1]
mysettings["PN"] = mysplit[0]
mysettings["PV"] = mysplit[1]
--
2.1.0.rc2.206.gedb03e5
From def62f7e1785c65b33623b1a4547738e05b901ae Mon Sep 17 00:00:00 2001
From: Bertrand SIMONNET <bsimon...@chromium.org>
Date: Wed, 17 Sep 2014 15:13:31 -0700
Subject: [PATCH 3/3] Make env/ bash scripts profile specific
This generalize the /etc/portage/env mechanism to be stackable with profiles.
ebuild.sh will walk the profiles and sources scripts in env/ following the
same matching rules as for /etc/portage/env.
---
bin/ebuild.sh | 14 ++++++++++----
man/portage.5 | 5 ++++-
pym/portage/repository/config.py | 2 +-
3 files changed, 15 insertions(+), 6 deletions(-)
diff --git a/bin/ebuild.sh b/bin/ebuild.sh
index bca5b59..7d5e15a 100755
--- a/bin/ebuild.sh
+++ b/bin/ebuild.sh
@@ -369,9 +369,15 @@ __source_all_bashrcs() {
save_IFS
IFS=$'\n'
local path_array=($PROFILE_PATHS)
+ local profile_attributes=($PORTAGE_PROFILE_ATTRIBUTES)
+ local profile_path
restore_IFS
- for x in "${path_array[@]}" ; do
- [ -f "$x/profile.bashrc" ] && __qa_source "$x/profile.bashrc"
+ for i in "${!path_array[@]}" ; do
+ profile_path="${path_array[$i]}"
+ [ -f "${profile_path}/profile.bashrc" ] && \
+ __qa_source "${profile_path}/profile.bashrc"
+ has "profile-env" ${profile_attributes[$i]} && \
+ __source_env_files "${profile_path}/env"
done
fi
@@ -421,7 +427,7 @@ __try_source() {
if [[ "$PORTAGE_DEBUG" == "1" ]] && [[ "${-/x/}" == "$-" ]]; then
debug_on=true
fi
- [[ ${debug_on} ]] && set -x
+ ${debug_on} && set -x
# If $- contains x, then tracing has already been enabled
# elsewhere for some reason. We preserve it's state so as
# not to interfere.
@@ -430,7 +436,7 @@ __try_source() {
else
__qa_source "${1}"
fi
- [[ ${debug_on} ]] && set +x
+ ${debug_on} && set +x
fi
}
# === === === === === === === === === === === === === === === === === ===
diff --git a/man/portage.5 b/man/portage.5
index e399f0f..0c1cce9 100644
--- a/man/portage.5
+++ b/man/portage.5
@@ -1047,11 +1047,14 @@ The default setting for repoman's --echangelog option.
The cache formats supported in the metadata tree. There is the old "pms" format
and the newer/faster "md5-dict" format. Default is to detect dirs.
.TP
-.BR profile\-formats " = [pms|portage-1|portage-2]"
+.BR profile\-formats " = [pms|portage-1|portage-2|profile-env]"
Control functionality available to profiles in this repo such as which files
may be dirs, or the syntax available in parent files. Use "portage-2" if you're
unsure. The default is "portage-1-compat" mode which is meant to be compatible
with old profiles, but is not allowed to be opted into directly.
+"profile-env" allows profiles to modify the environment for each any package.
+The environment modifications are defined in an "env" directory in each profile,
+following the same conventions as for /etc/portage/env.
.RE
.RE
diff --git a/pym/portage/repository/config.py b/pym/portage/repository/config.py
index 5e0d055..ef8054e 100644
--- a/pym/portage/repository/config.py
+++ b/pym/portage/repository/config.py
@@ -40,7 +40,7 @@ if sys.hexversion >= 0x3000000:
_invalid_path_char_re = re.compile(r'[^a-zA-Z0-9._\-+:/]')
_valid_profile_formats = frozenset(
- ['pms', 'portage-1', 'portage-2'])
+ ['pms', 'portage-1', 'portage-2', 'profile-env'])
_portage1_profiles_allow_directories = frozenset(
["portage-1-compat", "portage-1", 'portage-2'])
--
2.1.0.rc2.206.gedb03e5
