Re: [OE-core][PATCH] cve-check: add option to add additional patched CVEs
On 9/05/23 9:32 pm, Mikko Rapeli wrote:
On Tue, May 09, 2023 at 09:02:59AM +, Ross Burton wrote:
On 8 May 2023, at 09:57, Adrian Freihofer via
lists.openembedded.org
wrote:
Is there any defined language that we can simply adopt?
Since a lot of people talk about SPDX solving these issues would be nice
to know how that is going to work. I can't parse
https://spdx.github.io/spdx-spec/v2.3/how-to-use/#k17-linking-to-a-code-fix-for-a-security-issue
and figure out how to mark a CVE issue which has been ignored after
analysis.
Perhaps this?
https://spdx.github.io/spdx-spec/v2.3/how-to-use/#k16-linking-to-a-vulnerability-disclosure-document
To communicate that a package is not vulnerable to a specific
vulnerability it is recommended to reference a web page indicating
why given vulnerabilities are not applicable.
|"externalRefs" : [ { "referenceCategory" : "SECURITY",
"referenceLocator" :
"https://example.com/product-x/security-info.html;, "referenceType"
: "advisory" } ] |
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#181084):
https://lists.openembedded.org/g/openembedded-core/message/181084
Mute This Topic: https://lists.openembedded.org/mt/98703185/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] Overriding SDE_FILE
On 28/02/20 11:34 am, Joshua Watt wrote:
On 2/27/20 4:29 PM, Douglas Royds wrote:
On 28/02/20 10:49 am, Joshua Watt wrote:
On 2/27/20 3:22 PM, Douglas Royds wrote:
On 28/02/20 5:45 am, Joshua Watt wrote:
On 2/27/20 9:01 AM, Joshua Watt wrote:
On 2/26/20 11:46 PM, Douglas Royds wrote:
On 26/02/20 4:53 am, Jacob Kroon wrote:
On 2/24/20 8:25 AM, Jacob Kroon wrote:
Hi Douglas,
You updated a comment in reproducible_build.bbclass, commit
e7b891b76954c784f5a93bd0a1c91315673ce40d:
-# Once the value of SOURCE_DATE_EPOCH is determined, it is
stored in the recipe's ${SDE_FILE}.
+# Once the value of SOURCE_DATE_EPOCH is determined, it is
stored in the recipe's SDE_FILE.
+# If none of these mechanisms are suitable, replace the
do_deploy_source_date_epoch task
+# with recipe-specific functionality to write the
appropriate SOURCE_DATE_EPOCH into the SDE_FILE.
+#
But I can't really get this to work. What did work for me was
to replace "do_create_source_date_epoch_stamp()" in my recipe:
do_create_source_date_epoch_stamp() {
mkdir -p ${SDE_DIR}
date -d "1981-03-03" "+%s" > ${SDE_FILE}
}
What is the intended way to achieve the thing I'm trying to do
here ?
FYI, JPEW has a proposed patch here
http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=jpew/reproducible=d091d2aa53ea417f70c10f5ce89151820c3db9ce
for allowing a recipe to just set SOURCE_DATE_EPOCH directly.
But maybe that currently is at odds with SOURCE_DATE_EPOCH
being in BB_HASHBASE_WHITELIST ?
/Jacob
On the surface of it, my comment appears to be just wrong: It
does make sense to replace do_create_source_date_epoch_stamp()
as you suggest.
Joshua's proposed patch looks promising:
* Should the new function not be called first, so that it
takes priority over the git, known files, and youngest file
functions? If someone has explicitly set SOURCE_DATE_EPOCH,
then they want it to take priority.
Having that be the first option makes sense. The only case in
which that might not work, is if a recipe does something like:
SOURCE_DATE_EPOCH = "${@my_awesome_sde_calculation(d)}"
e.g. uses a function to get the SDE instead of setting to a fixed
value, but that's probably going to be extremely rare.
*
* As you observe, SOURCE_DATE_EPOCH would need to be removed
from BB_HASHBASE_WHITELIST. I'm not sure why it was in the
whitelist in the first place.
I'm not sure why exactly it is whitelisted; I didn't write the
original code that whitelisted it, but I've CC'd Juro in case he
happens to remember.
After a discussion with Richard, we figured out why
SOURCE_DATE_EPOCH has to be whitelisted. The value of the variable
*must* be calculable at parse time before any task is ran, but in
practice it's value is only available once the
__source_date_epoch.txt file is present, which is after parsing.
This causes the taskhash to be calculated differently during
parsing and task execution which causes taskhash mismatch errors.
True.
What to do? Would it work to use a different non-whitelisted
variable in the recipe, eg. SOURCE_DATE_EPOCH_FIXED?
Ya, that would work. You'd have to figure out how to get the
variable to be included in each taskhash even though it's not
directly referenced, but I'm sure that's possible.
Another option that's at lot more "magic" would be something like
this:
http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=jpew/reproducible=2b524916cf35238ff3deea34017e8a4cd73926cd
That's really weird, and I'm not sure I like it, but worth a thought.
If it works, it's good. I like the fact that the user can just set
SOURCE_DATE_EPOCH directly in their recipe.
How about ...
BB_HASHBASE_WHITELIST += "${@'SOURCE_DATE_EPOCH' if not
source_date_epoch_var(d) else ''}"
Ya, that works. The only reason I chose "is None" was in case a user
really wanted to do
SOURCE_DATE_EPOCH = "0"
True, best to allow for that.
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] Overriding SDE_FILE
On 28/02/20 10:49 am, Joshua Watt wrote:
On 2/27/20 3:22 PM, Douglas Royds wrote:
On 28/02/20 5:45 am, Joshua Watt wrote:
On 2/27/20 9:01 AM, Joshua Watt wrote:
On 2/26/20 11:46 PM, Douglas Royds wrote:
On 26/02/20 4:53 am, Jacob Kroon wrote:
On 2/24/20 8:25 AM, Jacob Kroon wrote:
Hi Douglas,
You updated a comment in reproducible_build.bbclass, commit
e7b891b76954c784f5a93bd0a1c91315673ce40d:
-# Once the value of SOURCE_DATE_EPOCH is determined, it is
stored in the recipe's ${SDE_FILE}.
+# Once the value of SOURCE_DATE_EPOCH is determined, it is
stored in the recipe's SDE_FILE.
+# If none of these mechanisms are suitable, replace the
do_deploy_source_date_epoch task
+# with recipe-specific functionality to write the appropriate
SOURCE_DATE_EPOCH into the SDE_FILE.
+#
But I can't really get this to work. What did work for me was to
replace "do_create_source_date_epoch_stamp()" in my recipe:
do_create_source_date_epoch_stamp() {
mkdir -p ${SDE_DIR}
date -d "1981-03-03" "+%s" > ${SDE_FILE}
}
What is the intended way to achieve the thing I'm trying to do
here ?
FYI, JPEW has a proposed patch here
http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=jpew/reproducible=d091d2aa53ea417f70c10f5ce89151820c3db9ce
for allowing a recipe to just set SOURCE_DATE_EPOCH directly.
But maybe that currently is at odds with SOURCE_DATE_EPOCH being
in BB_HASHBASE_WHITELIST ?
/Jacob
On the surface of it, my comment appears to be just wrong: It does
make sense to replace do_create_source_date_epoch_stamp() as you
suggest.
Joshua's proposed patch looks promising:
* Should the new function not be called first, so that it takes
priority over the git, known files, and youngest file
functions? If someone has explicitly set SOURCE_DATE_EPOCH,
then they want it to take priority.
Having that be the first option makes sense. The only case in which
that might not work, is if a recipe does something like:
SOURCE_DATE_EPOCH = "${@my_awesome_sde_calculation(d)}"
e.g. uses a function to get the SDE instead of setting to a fixed
value, but that's probably going to be extremely rare.
*
* As you observe, SOURCE_DATE_EPOCH would need to be removed
from BB_HASHBASE_WHITELIST. I'm not sure why it was in the
whitelist in the first place.
I'm not sure why exactly it is whitelisted; I didn't write the
original code that whitelisted it, but I've CC'd Juro in case he
happens to remember.
After a discussion with Richard, we figured out why
SOURCE_DATE_EPOCH has to be whitelisted. The value of the variable
*must* be calculable at parse time before any task is ran, but in
practice it's value is only available once the
__source_date_epoch.txt file is present, which is after parsing.
This causes the taskhash to be calculated differently during parsing
and task execution which causes taskhash mismatch errors.
True.
What to do? Would it work to use a different non-whitelisted variable
in the recipe, eg. SOURCE_DATE_EPOCH_FIXED?
Ya, that would work. You'd have to figure out how to get the variable
to be included in each taskhash even though it's not directly
referenced, but I'm sure that's possible.
Another option that's at lot more "magic" would be something like
this:
http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=jpew/reproducible=2b524916cf35238ff3deea34017e8a4cd73926cd
That's really weird, and I'm not sure I like it, but worth a thought.
If it works, it's good. I like the fact that the user can just set
SOURCE_DATE_EPOCH directly in their recipe.
How about ...
BB_HASHBASE_WHITELIST += "${@'SOURCE_DATE_EPOCH' if not
source_date_epoch_var(d) else ''}"
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] Overriding SDE_FILE
On 28/02/20 5:45 am, Joshua Watt wrote:
On 2/27/20 9:01 AM, Joshua Watt wrote:
On 2/26/20 11:46 PM, Douglas Royds wrote:
On 26/02/20 4:53 am, Jacob Kroon wrote:
On 2/24/20 8:25 AM, Jacob Kroon wrote:
Hi Douglas,
You updated a comment in reproducible_build.bbclass, commit
e7b891b76954c784f5a93bd0a1c91315673ce40d:
-# Once the value of SOURCE_DATE_EPOCH is determined, it is
stored in the recipe's ${SDE_FILE}.
+# Once the value of SOURCE_DATE_EPOCH is determined, it is
stored in the recipe's SDE_FILE.
+# If none of these mechanisms are suitable, replace the
do_deploy_source_date_epoch task
+# with recipe-specific functionality to write the appropriate
SOURCE_DATE_EPOCH into the SDE_FILE.
+#
But I can't really get this to work. What did work for me was to
replace "do_create_source_date_epoch_stamp()" in my recipe:
do_create_source_date_epoch_stamp() {
mkdir -p ${SDE_DIR}
date -d "1981-03-03" "+%s" > ${SDE_FILE}
}
What is the intended way to achieve the thing I'm trying to do here ?
FYI, JPEW has a proposed patch here
http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=jpew/reproducible=d091d2aa53ea417f70c10f5ce89151820c3db9ce
for allowing a recipe to just set SOURCE_DATE_EPOCH directly.
But maybe that currently is at odds with SOURCE_DATE_EPOCH being in
BB_HASHBASE_WHITELIST ?
/Jacob
On the surface of it, my comment appears to be just wrong: It does
make sense to replace do_create_source_date_epoch_stamp() as you
suggest.
Joshua's proposed patch looks promising:
* Should the new function not be called first, so that it takes
priority over the git, known files, and youngest file functions?
If someone has explicitly set SOURCE_DATE_EPOCH, then they want
it to take priority.
Having that be the first option makes sense. The only case in which
that might not work, is if a recipe does something like:
SOURCE_DATE_EPOCH = "${@my_awesome_sde_calculation(d)}"
e.g. uses a function to get the SDE instead of setting to a fixed
value, but that's probably going to be extremely rare.
*
* As you observe, SOURCE_DATE_EPOCH would need to be removed from
BB_HASHBASE_WHITELIST. I'm not sure why it was in the whitelist
in the first place.
I'm not sure why exactly it is whitelisted; I didn't write the
original code that whitelisted it, but I've CC'd Juro in case he
happens to remember.
After a discussion with Richard, we figured out why SOURCE_DATE_EPOCH
has to be whitelisted. The value of the variable *must* be calculable
at parse time before any task is ran, but in practice it's value is
only available once the __source_date_epoch.txt file is present, which
is after parsing. This causes the taskhash to be calculated
differently during parsing and task execution which causes taskhash
mismatch errors.
True.
What to do? Would it work to use a different non-whitelisted variable in
the recipe, eg. SOURCE_DATE_EPOCH_FIXED?
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] Overriding SDE_FILE
On 26/02/20 4:53 am, Jacob Kroon wrote:
On 2/24/20 8:25 AM, Jacob Kroon wrote:
Hi Douglas,
You updated a comment in reproducible_build.bbclass, commit
e7b891b76954c784f5a93bd0a1c91315673ce40d:
-# Once the value of SOURCE_DATE_EPOCH is determined, it is stored
in the recipe's ${SDE_FILE}.
+# Once the value of SOURCE_DATE_EPOCH is determined, it is stored
in the recipe's SDE_FILE.
+# If none of these mechanisms are suitable, replace the
do_deploy_source_date_epoch task
+# with recipe-specific functionality to write the appropriate
SOURCE_DATE_EPOCH into the SDE_FILE.
+#
But I can't really get this to work. What did work for me was to
replace "do_create_source_date_epoch_stamp()" in my recipe:
do_create_source_date_epoch_stamp() {
mkdir -p ${SDE_DIR}
date -d "1981-03-03" "+%s" > ${SDE_FILE}
}
What is the intended way to achieve the thing I'm trying to do here ?
FYI, JPEW has a proposed patch here
http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=jpew/reproducible=d091d2aa53ea417f70c10f5ce89151820c3db9ce
for allowing a recipe to just set SOURCE_DATE_EPOCH directly.
But maybe that currently is at odds with SOURCE_DATE_EPOCH being in
BB_HASHBASE_WHITELIST ?
/Jacob
On the surface of it, my comment appears to be just wrong: It does make
sense to replace do_create_source_date_epoch_stamp() as you suggest.
Joshua's proposed patch looks promising:
* Should the new function not be called first, so that it takes
priority over the git, known files, and youngest file functions? If
someone has explicitly set SOURCE_DATE_EPOCH, then they want it to
take priority.
* As you observe, SOURCE_DATE_EPOCH would need to be removed from
BB_HASHBASE_WHITELIST. I'm not sure why it was in the whitelist in
the first place.
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH v2] insane: file-rdeps: Readability only
On 4/11/19 11:29 PM, Richard Purdie wrote:
On Mon, 2019-11-04 at 17:52 +1300, Douglas Royds via Openembedded-core
wrote:
Mostly just longer (and hopefully more meaningful) variable names.
"Mostly" - what else is in there?
Readability changes only.
* Longer variable names
* Reworded some comments
* Some extra blank lines (as visual paragraph breaks)
I replaced this comment + hard-coded string-length:
# We already know it starts with FILERDEPENDS_
filerdepends[subkey] = key[13:]
With this one-liner:
unsatisfied_rdepends[target_dependency] =
executable_that_rdepends[len('FILERDEPENDS_'):]
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v2] insane: file-rdeps: Readability only
Mostly just longer (and hopefully more meaningful) variable names.
Signed-off-by: Douglas Royds
---
meta/classes/insane.bbclass | 81 +++--
1 file changed, 42 insertions(+), 39 deletions(-)
diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass
index 9605ac2bae..f80abebbac 100644
--- a/meta/classes/insane.bbclass
+++ b/meta/classes/insane.bbclass
@@ -707,57 +707,60 @@ def package_qa_check_rdepends(pkg, pkgdest, skip,
taskdeps, packages, d):
package_qa_handle_error("build-deps", error_msg, d)
if "file-rdeps" not in skip:
-ignored_file_rdeps = set(['/bin/sh', '/usr/bin/env',
'rtld(GNU_HASH)'])
+ignored_executable_rdeps = set(['/bin/sh', '/usr/bin/env',
'rtld(GNU_HASH)'])
if bb.data.inherits_class('nativesdk', d):
-ignored_file_rdeps |= set(['/bin/bash', '/usr/bin/perl',
'perl'])
-# For Saving the FILERDEPENDS
-filerdepends = {}
-rdep_data = oe.packagedata.read_subpkgdata(pkg, d)
-for key in rdep_data:
-if key.startswith("FILERDEPENDS_"):
-for subkey in bb.utils.explode_deps(rdep_data[key]):
-if subkey not in ignored_file_rdeps and \
-not subkey.startswith('perl('):
-# We already know it starts with FILERDEPENDS_
-filerdepends[subkey] = key[13:]
-
-if filerdepends:
-done = rdepends[:]
-# Add the rprovides of itself
-if pkg not in done:
-done.insert(0, pkg)
+ignored_executable_rdeps |= set(['/bin/bash', '/usr/bin/perl',
'perl'])
+
+# A dictionary of libs (or other providers) and one executable
that depends on each
+unsatisfied_rdepends = {}
+current_pkg_rdep_data = oe.packagedata.read_subpkgdata(pkg, d)
+for executable_that_rdepends in current_pkg_rdep_data:
+if executable_that_rdepends.startswith("FILERDEPENDS_"):
+for target_dependency in
bb.utils.explode_deps(current_pkg_rdep_data[executable_that_rdepends]):
+if target_dependency not in ignored_executable_rdeps \
+ and not target_dependency.startswith('perl('):
+unsatisfied_rdepends[target_dependency] =
executable_that_rdepends[len('FILERDEPENDS_'):]
+
+if unsatisfied_rdepends:
+rdep_packages_to_consider = rdepends[:]
+# Each package effectively rprovides itself
+if pkg not in rdep_packages_to_consider:
+rdep_packages_to_consider.insert(0, pkg)
# The python is not a package, but python-core provides it, so
# skip checking /usr/bin/python if python is in the rdeps, in
# case there is a RDEPENDS_pkg = "python" in the recipe.
-for py in [ d.getVar('MLPREFIX') + "python", "python" ]:
-if py in done:
-filerdepends.pop("/usr/bin/python",None)
-done.remove(py)
-for rdep in done:
-# The file dependencies may contain package names, e.g.,
-# perl
-filerdepends.pop(rdep,None)
-
-# For Saving the FILERPROVIDES, RPROVIDES and FILES_INFO
-rdep_data = oe.packagedata.read_subpkgdata(rdep, d)
-for key in rdep_data:
+for python_package in [ d.getVar('MLPREFIX') + "python",
"python" ]:
+if python_package in rdep_packages_to_consider:
+unsatisfied_rdepends.pop("/usr/bin/python",None)
+rdep_packages_to_consider.remove(python_package)
+
+for dependent_package in rdep_packages_to_consider:
+# Dependencies might contain package names, e.g. perl
+unsatisfied_rdepends.pop(dependent_package,None)
+
+dependent_pkg_rdep_data =
oe.packagedata.read_subpkgdata(dependent_package, d)
+for key in dependent_pkg_rdep_data:
if key.startswith("FILERPROVIDES_") or
key.startswith("RPROVIDES_"):
-for subkey in
bb.utils.explode_deps(rdep_data[key]):
-filerdepends.pop(subkey,None)
-# Add the files list to the rprovides
+for satisfied_lib_or_provider in
bb.utils.explode_deps(dependent_pkg_rdep_data[key]):
+
unsatisfied_rdepends.pop(satisfied_lib_or_provider,None)
+
+
[OE-core] [PATCH] insane: file-rdeps: Readability only
Mostly just longer (and hopefully more meaningful) variable names.
Signed-off-by: Douglas Royds
---
meta/classes/insane.bbclass | 84 -
1 file changed, 45 insertions(+), 39 deletions(-)
diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass
index 9605ac2bae..b72740e5d4 100644
--- a/meta/classes/insane.bbclass
+++ b/meta/classes/insane.bbclass
@@ -707,57 +707,60 @@ def package_qa_check_rdepends(pkg, pkgdest, skip,
taskdeps, packages, d):
package_qa_handle_error("build-deps", error_msg, d)
if "file-rdeps" not in skip:
-ignored_file_rdeps = set(['/bin/sh', '/usr/bin/env',
'rtld(GNU_HASH)'])
+ignored_executable_rdeps = set(['/bin/sh', '/usr/bin/env',
'rtld(GNU_HASH)'])
if bb.data.inherits_class('nativesdk', d):
-ignored_file_rdeps |= set(['/bin/bash', '/usr/bin/perl',
'perl'])
-# For Saving the FILERDEPENDS
-filerdepends = {}
-rdep_data = oe.packagedata.read_subpkgdata(pkg, d)
-for key in rdep_data:
-if key.startswith("FILERDEPENDS_"):
-for subkey in bb.utils.explode_deps(rdep_data[key]):
-if subkey not in ignored_file_rdeps and \
-not subkey.startswith('perl('):
-# We already know it starts with FILERDEPENDS_
-filerdepends[subkey] = key[13:]
-
-if filerdepends:
-done = rdepends[:]
-# Add the rprovides of itself
-if pkg not in done:
-done.insert(0, pkg)
+ignored_executable_rdeps |= set(['/bin/bash', '/usr/bin/perl',
'perl'])
+
+# A dictionary of libs (or other providers) and one executable
that depends on each
+unsatisfied_rdepends = {}
+current_pkg_rdep_data = oe.packagedata.read_subpkgdata(pkg, d)
+for executable_that_rdepends in current_pkg_rdep_data:
+if executable_that_rdepends.startswith("FILERDEPENDS_"):
+for target_dependency in
bb.utils.explode_deps(current_pkg_rdep_data[executable_that_rdepends]):
+if target_dependency not in ignored_executable_rdeps \
+ and not target_dependency.startswith('perl('):
+unsatisfied_rdepends[target_dependency] =
executable_that_rdepends[len('FILERDEPENDS_'):]
+
+if unsatisfied_rdepends:
+rdep_packages_to_consider = rdepends[:]
+# Each package effectively rprovides itself
+if pkg not in rdep_packages_to_consider:
+rdep_packages_to_consider.insert(0, pkg)
# The python is not a package, but python-core provides it, so
# skip checking /usr/bin/python if python is in the rdeps, in
# case there is a RDEPENDS_pkg = "python" in the recipe.
-for py in [ d.getVar('MLPREFIX') + "python", "python" ]:
-if py in done:
-filerdepends.pop("/usr/bin/python",None)
-done.remove(py)
-for rdep in done:
-# The file dependencies may contain package names, e.g.,
-# perl
-filerdepends.pop(rdep,None)
-
-# For Saving the FILERPROVIDES, RPROVIDES and FILES_INFO
-rdep_data = oe.packagedata.read_subpkgdata(rdep, d)
-for key in rdep_data:
+for python_package in [ d.getVar('MLPREFIX') + "python",
"python" ]:
+if python_package in rdep_packages_to_consider:
+unsatisfied_rdepends.pop("/usr/bin/python",None)
+rdep_packages_to_consider.remove(python_package)
+
+for dependent_package in rdep_packages_to_consider:
+# Dependencies might contain package names, e.g. perl
+unsatisfied_rdepends.pop(dependent_package,None)
+
+dependent_pkg_rdep_data =
oe.packagedata.read_subpkgdata(dependent_package, d)
+for key in dependent_pkg_rdep_data:
if key.startswith("FILERPROVIDES_") or
key.startswith("RPROVIDES_"):
-for subkey in
bb.utils.explode_deps(rdep_data[key]):
-filerdepends.pop(subkey,None)
-# Add the files list to the rprovides
+for satisfied_lib_or_provider in
bb.utils.explode_deps(dependent_pkg_rdep_data[key]):
+
unsatisfied_rdepends.pop(satisfied_lib_or_provider,None)
+
+
[OE-core] [PATCH] icecc: Don't use icecc when INHIBIT_DEFAULT_DEPS is set
We don't have a compiler, so no icecc.
Silences a spew of warnings of the form:
do_configure: Cannot use icecc: could not get ICECC_CC or ICECC_CXX
Signed-off-by: Douglas Royds
---
meta/classes/icecc.bbclass | 4
1 file changed, 4 insertions(+)
diff --git a/meta/classes/icecc.bbclass b/meta/classes/icecc.bbclass
index e74d798fd0..bc3d6f4cc8 100644
--- a/meta/classes/icecc.bbclass
+++ b/meta/classes/icecc.bbclass
@@ -138,6 +138,10 @@ def use_icecc(bb,d):
if icecc_is_cross_canadian(bb, d):
return "no"
+if d.getVar('INHIBIT_DEFAULT_DEPS', False):
+# We don't have a compiler, so no icecc
+return "no"
+
pn = d.getVar('PN')
bpn = d.getVar('BPN')
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH] icecc: Export ICECC_CC and friends via wrapper-script
By exporting ICECC_CC, ICECC_CXX, and ICECC_VERSION in a wrapper-script,
and putting this wrapper-script in the PATH, the Makefiles generated by CMake or
the autotools are able to function correctly outside of bitbake.
This provides a convenient developer workflow in which the
modify-compile-unittest cycle can happen directly in the ${B} directory.
The `rm -f $ICE_PATH/$compiler` line is transitional,
and can go at some later date (October 2020 or later, perhaps).
Signed-off-by: Douglas Royds
---
meta/classes/icecc.bbclass | 32
1 file changed, 20 insertions(+), 12 deletions(-)
diff --git a/meta/classes/icecc.bbclass b/meta/classes/icecc.bbclass
index 9dbb4b1559..e74d798fd0 100644
--- a/meta/classes/icecc.bbclass
+++ b/meta/classes/icecc.bbclass
@@ -352,17 +352,6 @@ set_icecc_env() {
return
fi
-# Create symlinks to icecc in the recipe-sysroot directory
-mkdir -p ${ICE_PATH}
-if [ -n "${KERNEL_CC}" ]; then
-compilers="${@get_cross_kernel_cc(bb,d)}"
-else
-compilers="${HOST_PREFIX}gcc ${HOST_PREFIX}g++"
-fi
-for compiler in $compilers; do
-ln -sf ${ICECC_BIN} ${ICE_PATH}/$compiler
-done
-
ICECC_CC="${@icecc_get_and_check_tool(bb, d, "gcc")}"
ICECC_CXX="${@icecc_get_and_check_tool(bb, d, "g++")}"
# cannot use icecc_get_and_check_tool here because it assumes as without
target_sys prefix
@@ -381,6 +370,26 @@ set_icecc_env() {
return
fi
+# Create symlinks to icecc and wrapper-scripts in the recipe-sysroot
directory
+mkdir -p $ICE_PATH/symlinks
+if [ -n "${KERNEL_CC}" ]; then
+compilers="${@get_cross_kernel_cc(bb,d)}"
+else
+compilers="${HOST_PREFIX}gcc ${HOST_PREFIX}g++"
+fi
+for compiler in $compilers; do
+ln -sf $ICECC_BIN $ICE_PATH/symlinks/$compiler
+rm -f $ICE_PATH/$compiler
+cat <<-__EOF__ > $ICE_PATH/$compiler
+ #!/bin/sh -e
+ export ICECC_VERSION=$ICECC_VERSION
+ export ICECC_CC=$ICECC_CC
+ export ICECC_CXX=$ICECC_CXX
+ $ICE_PATH/symlinks/$compiler "\$@"
+ __EOF__
+chmod 775 $ICE_PATH/$compiler
+done
+
ICECC_AS="`${ICECC_CC} -print-prog-name=as`"
# for target recipes should return something like:
#
/OE/tmp-eglibc/sysroots/x86_64-linux/usr/libexec/arm920tt-oe-linux-gnueabi/gcc/arm-oe-linux-gnueabi/4.8.2/as
@@ -413,7 +422,6 @@ set_icecc_env() {
export CCACHE_PATH="$PATH"
export CCACHE_DISABLE="1"
-export ICECC_VERSION ICECC_CC ICECC_CXX
export PATH="$ICE_PATH:$PATH"
bbnote "Using icecc path: $ICE_PATH"
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 3/5] cmake.bbclass: append includedir to implicit include dirs
An error in my analysis below: We *do* delete the dependency files in
the in-tree build case as well. The side-effect is masked in both
in-tree and out-of-tree build cases except here at Tait, where we don't
delete the entire build at configure time, due to a 20min+ rebuild time,
even with the benefit of icecc.
The rest of my concern still applies: This setting has no effect in the
target case, and only an unfortunate side effect in the -native case.
I'm not clear what the benefit is.
On 11/07/19 2:51 PM, Douglas Royds wrote:
This commit is having an unintended side-effect in the -native (and
probably nativesdk) case.
In the target build, $includedir is normally /usr/include,
fully-qualified. This path is already in CMake's list of implicit
include directories, and we don't include any header files from the
build PC's /usr/include in a target build in any case. This addition
to CMAKE_C/CXX_IMPLICIT_INCLUDE_DIRECTORIES has no effect.
In the -native case, the $includedir is set to STAGING_INCDIR_NATIVE,
but this path has already been added to the BUILD_CPPFLAGS as a system
include directory, so there will already be no warnings for any header
file in the STAGING_INCDIR_NATIVE.
There is a nasty side-effect in the -native case: CMake excludes
headers in the CMAKE_C/CXX_IMPLICIT_INCLUDE_DIRECTORIES from its
generated dependency files, meaning that a change in any library
header file will not trigger a recompile of affected source files in
the dependent CMake component. In out-of-tree builds this isn't a
problem, as cmake.bbclass deletes the *entire* ${B} directory at
configure time, but where this is not the case, the build breaks with
any change in library headers.
I haven't examined the nativesdk case. The $includedir is set off to
$SDKPATHNATIVE/usr/include, but this path is not added as a system
include dir. The same problem will apply as in the -native case, that
CMake will not generate dependencies for headers staged in the
SDKPATHNATIVE.
Was nativesdk perhaps the intended case for this commit? Is there a
better way to silence warnings in this case? Do we need to silence
library header warnings at all? Should we instead add
$SDKPATHNATIVE/usr/include as a system include dir via the
BUILDSDK_CPPFLAGS?
I examined this problem using the Unix Makefiles generator. CMake
appears to equally exclude these headers from the ninja *.o.d
dependency files, though I haven't examined it closely.
On 30/11/18 1:21 AM, Mikko Rapeli wrote:
From: Michael Ho
This resolves issues with paths being marked as system includes that
differ from /usr/include but are considered implicit by the toolchain.
This enables developers to add directories to system includes
to supress compiler compiler warnings from them.
Signed-off-by: Michael Ho
Cc: Pascal Bach
---
meta/classes/cmake.bbclass | 4
1 file changed, 4 insertions(+)
diff --git a/meta/classes/cmake.bbclass b/meta/classes/cmake.bbclass
index fd40a98..485aea6 100644
--- a/meta/classes/cmake.bbclass
+++ b/meta/classes/cmake.bbclass
@@ -108,6 +108,10 @@ list(APPEND CMAKE_MODULE_PATH
"${STAGING_DATADIR}/cmake/Modules/")
# add for non /usr/lib libdir, e.g. /usr/lib64
set( CMAKE_LIBRARY_PATH ${libdir} ${base_libdir})
+# add include dir to implicit includes in case it differs from
/usr/include
+list(APPEND CMAKE_C_IMPLICIT_INCLUDE_DIRECTORIES ${includedir})
+list(APPEND CMAKE_CXX_IMPLICIT_INCLUDE_DIRECTORIES ${includedir})
+
EOF
}
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 3/5] cmake.bbclass: append includedir to implicit include dirs
This commit is having an unintended side-effect in the -native (and
probably nativesdk) case.
In the target build, $includedir is normally /usr/include,
fully-qualified. This path is already in CMake's list of implicit
include directories, and we don't include any header files from the
build PC's /usr/include in a target build in any case. This addition to
CMAKE_C/CXX_IMPLICIT_INCLUDE_DIRECTORIES has no effect.
In the -native case, the $includedir is set to STAGING_INCDIR_NATIVE,
but this path has already been added to the BUILD_CPPFLAGS as a system
include directory, so there will already be no warnings for any header
file in the STAGING_INCDIR_NATIVE.
There is a nasty side-effect in the -native case: CMake excludes headers
in the CMAKE_C/CXX_IMPLICIT_INCLUDE_DIRECTORIES from its generated
dependency files, meaning that a change in any library header file will
not trigger a recompile of affected source files in the dependent CMake
component. In out-of-tree builds this isn't a problem, as cmake.bbclass
deletes the *entire* ${B} directory at configure time, but where this is
not the case, the build breaks with any change in library headers.
I haven't examined the nativesdk case. The $includedir is set off to
$SDKPATHNATIVE/usr/include, but this path is not added as a system
include dir. The same problem will apply as in the -native case, that
CMake will not generate dependencies for headers staged in the
SDKPATHNATIVE.
Was nativesdk perhaps the intended case for this commit? Is there a
better way to silence warnings in this case? Do we need to silence
library header warnings at all? Should we instead add
$SDKPATHNATIVE/usr/include as a system include dir via the
BUILDSDK_CPPFLAGS?
I examined this problem using the Unix Makefiles generator. CMake
appears to equally exclude these headers from the ninja *.o.d dependency
files, though I haven't examined it closely.
On 30/11/18 1:21 AM, Mikko Rapeli wrote:
From: Michael Ho
This resolves issues with paths being marked as system includes that
differ from /usr/include but are considered implicit by the toolchain.
This enables developers to add directories to system includes
to supress compiler compiler warnings from them.
Signed-off-by: Michael Ho
Cc: Pascal Bach
---
meta/classes/cmake.bbclass | 4
1 file changed, 4 insertions(+)
diff --git a/meta/classes/cmake.bbclass b/meta/classes/cmake.bbclass
index fd40a98..485aea6 100644
--- a/meta/classes/cmake.bbclass
+++ b/meta/classes/cmake.bbclass
@@ -108,6 +108,10 @@ list(APPEND CMAKE_MODULE_PATH
"${STAGING_DATADIR}/cmake/Modules/")
# add for non /usr/lib libdir, e.g. /usr/lib64
set( CMAKE_LIBRARY_PATH ${libdir} ${base_libdir})
+# add include dir to implicit includes in case it differs from /usr/include
+list(APPEND CMAKE_C_IMPLICIT_INCLUDE_DIRECTORIES ${includedir})
+list(APPEND CMAKE_CXX_IMPLICIT_INCLUDE_DIRECTORIES ${includedir})
+
EOF
}
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] The state of reproducible Builds
On 2/07/19 3:58 AM, Joshua Watt wrote: 1. Testing RPM and IPK package formats. I think RPMs will be pretty easy; IPKs might be more challenging since AFAIK the tools that make them don't generate reproducible output to begin with. This has not been my experience. I have been building reproducible ipks, indeed, it is the hashsums of the ipks that I've been examining. In most cases, the correct SOURCE_DATE_EPOCH is enough, but there have been cases where I've had to correct upstream projects to cope with the SOURCE_DATE_EPOCH or avoid the effect of differing uname settings. 1. HOSTTOOLS differences. There are a lot of tools listed in HOSTTOOLS, and unfortunately some of them have version dependent output and are used for target builds (the one I've currently stumbled upon is pod2man, but I'm sure there are others). Unfortunately, one could probably argue that HOSTTOOLS is somewhat antithetical to the above statement, at least in regard to target builds. Any host tool output that "leaks" into the target build output can result in a non-reproducible build across hosts, and possibly should be avoided; the alternative is to use (or mandate) the corresponding -native recipe that provides that tool as a DEPENDS so that the controlled internally built version is used instead. Note that this only really applies target builds, not -native (or nativesdk right now). -native recipes would obviously need more HOSTTOOLS to help bootstrap the system. I suspect this would require reworking how HOSTOOLS works so that they can be split into two categories somehow; the tools that have "ubiquitous and stable" interfaces and are fine for all recipes (e.g. cat, sed, true, rm, etc.) and those that are variable and should only be used for -native builds (e.g. pod2man, rpcgen(?), chrpath(?), tar(?)... others?). Anyone have thoughts on this? Perhaps reproducibility is the decision-point for adding a tool to the HOSTTOOLS: If the precise version of the tool has no impact on reproducibility (eg. cat, sed, and even gawk), it is a good candidate for the HOSTTOOLS. pod2man shouldn't be in the HOSTTOOLS, because we need to control the version. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH] json-c: Backport --disable-werror patch to allow compilation under icecc
icecc preprocesses source files locally before shipping them off to be compiled
on remote hosts. This preprocessing removes comments, including /* fallthough */
comments in switch statements that normally prevent an implicit-fallthrough
warning, see https://github.com/icecc/icecream/issues/419
Rather than turning off -Werror, the upstream project has implemented a
configure option, --disable-werror, in response to Ross's
https://github.com/json-c/json-c/issues/489
This patch from
https://github.com/json-c/json-c/commit/21c886534f8927fdc0fb5f8647394f3e0e0874b8
Upstream-Status: Backport [Not yet released]
Signed-off-by: Douglas Royds
---
...d-disable-werror-option-to-configure.patch | 45 +++
meta/recipes-devtools/json-c/json-c_0.13.1.bb | 8 +++-
2 files changed, 51 insertions(+), 2 deletions(-)
create mode 100644
meta/recipes-devtools/json-c/json-c/add-disable-werror-option-to-configure.patch
diff --git
a/meta/recipes-devtools/json-c/json-c/add-disable-werror-option-to-configure.patch
b/meta/recipes-devtools/json-c/json-c/add-disable-werror-option-to-configure.patch
new file mode 100644
index 00..0c20c8458a
--- /dev/null
+++
b/meta/recipes-devtools/json-c/json-c/add-disable-werror-option-to-configure.patch
@@ -0,0 +1,45 @@
+json-c: Backport --disable-werror patch to allow compilation under icecc
+
+icecc preprocesses source files locally before shipping them off to be compiled
+on remote hosts. This preprocessing removes comments, including /* fallthough
*/
+comments in switch statements that normally prevent an implicit-fallthrough
+warning, see https://github.com/icecc/icecream/issues/419
+
+Rather than turning off -Werror, the upstream project has implemented a
+configure option, --disable-werror, in response to Ross's
+https://github.com/json-c/json-c/issues/489
+
+This patch from
+https://github.com/json-c/json-c/commit/21c886534f8927fdc0fb5f8647394f3e0e0874b8
+
+Upstream-Status: Backport [Not yet released]
+Signed-off-by: Douglas Royds
+
+From 21c886534f8927fdc0fb5f8647394f3e0e0874b8 Mon Sep 17 00:00:00 2001
+From: Pierce Lopez
+Date: Sun, 9 Jun 2019 10:52:08 -0400
+Subject: [PATCH] build: add --disable-werror option to configure
+
+to omit -Werror compiler option
+---
+ configure.ac | 7 ++-
+ 1 file changed, 6 insertions(+), 1 deletion(-)
+
+diff --git a/configure.ac b/configure.ac
+index 272ea6af9c..798fd5b747 100644
+--- a/configure.ac
b/configure.ac
+@@ -165,7 +165,12 @@ AS_IF([test "x$enable_Bsymbolic" = "xcheck"],
+ AS_IF([test "x$enable_Bsymbolic" = "xyes"],
[JSON_BSYMBOLIC_LDFLAGS=-Wl[,]-Bsymbolic-functions])
+ AC_SUBST(JSON_BSYMBOLIC_LDFLAGS)
+
+-AX_APPEND_COMPILE_FLAGS([-Wall -Werror -Wcast-qual
-Wno-error=deprecated-declarations])
++AC_ARG_ENABLE([werror],
++AS_HELP_STRING([--disable-werror], [avoid treating compiler warnings as
fatal errors]))
++
++AS_IF([test "x$enable_werror" != "xno"], [AX_APPEND_COMPILE_FLAGS([-Werror])])
++
++AX_APPEND_COMPILE_FLAGS([-Wall -Wcast-qual
-Wno-error=deprecated-declarations])
+ AX_APPEND_COMPILE_FLAGS([-Wextra -Wwrite-string -Wno-unused-parameter])
+ AX_APPEND_COMPILE_FLAGS([-D_GNU_SOURCE])
+
diff --git a/meta/recipes-devtools/json-c/json-c_0.13.1.bb
b/meta/recipes-devtools/json-c/json-c_0.13.1.bb
index 5b10e68297..9d8f2e7870 100644
--- a/meta/recipes-devtools/json-c/json-c_0.13.1.bb
+++ b/meta/recipes-devtools/json-c/json-c_0.13.1.bb
@@ -4,7 +4,9 @@ HOMEPAGE = "https://github.com/json-c/json-c/wiki;
LICENSE = "MIT"
LIC_FILES_CHKSUM = "file://COPYING;md5=de54b60fbbc35123ba193fea8ee216f2"
-SRC_URI = "https://s3.amazonaws.com/json-c_releases/releases/${BP}.tar.gz;
+SRC_URI = "https://s3.amazonaws.com/json-c_releases/releases/${BP}.tar.gz \
+ file://add-disable-werror-option-to-configure.patch \
+ "
SRC_URI[md5sum] = "04969ad59cc37bddd83741a08b98f350"
SRC_URI[sha256sum] =
"b87e608d4d3f7bfdd36ef78d56d53c74e66ab278d318b71e6002a369d36f4873"
@@ -20,7 +22,9 @@ RPROVIDES_${PN} = "libjson"
inherit autotools
-EXTRA_OECONF = "--enable-rdrand"
+EXTRA_OECONF = "--disable-werror \
+--enable-rdrand \
+"
do_configure_prepend() {
# Clean up autoconf cruft that should not be in the tarball
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] json-c: update to current upstream head, with --disable-werror
On 11/06/19 4:57 PM, Adrian Bunk wrote: On Tue, Jun 11, 2019 at 03:12:09PM +1200, Douglas Royds via Openembedded-core wrote: On 11/06/19 3:05 PM, Kang Kai wrote: On 2019/6/11 上午11:04, Douglas Royds wrote: On 11/06/19 2:46 PM, Kang Kai wrote: ... Use option '-M' of git format-patch may make the patch more clear. And why not just backport the 'disable-werror' commit? Re '-M': True, fair point. Re the backport: There hasn't been a release in over a year, they don't seem to have any plans to do so (see https://github.com/json-c/json-c/issues/487), and this was easier. If you have a substantial objection, yes, I could submit it again with a patch instead of the update. Just consider footprint, git repo will take more space than tar ball. But no substantial objection. We do also lose the RECIPE_UPSTREAM_VERSION functionality by switching to the git HEAD, ie. there'll be no notification when they do (eventually) release a new version. I'm in two minds. Opinions? The biggest worry is actually whether some random git snapshot is as stable as a release. Unless there is a good reason why a git snapshot is better than release plus backported patch, the default should be to not switch to using a git snapshot. Starting to follow git snapshots brings the risk of frequent regressions since it follows the latest upstream development. Fair enough, patch it is. Watch this space. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] json-c: update to current upstream head, with --disable-werror
On 11/06/19 3:05 PM, Kang Kai wrote:
On 2019/6/11 上午11:04, Douglas Royds wrote:
On 11/06/19 2:46 PM, Kang Kai wrote:
On 2019/6/11 上午9:57, Douglas Royds via Openembedded-core wrote:
Upstream json-c haven't made a release since March 2018.
Adopt the current HEAD revision, pulling it directly from git.
icecc preprocesses source files locally before shipping them off to
be compiled
on remote hosts. This preprocessing removes comments, including /*
fallthough */
comments in switch statements that normally prevent an
implicit-fallthrough
warning, see https://github.com/icecc/icecream/issues/419
Rather than turning off -Werror by patching configure.ac, the
upstream project
has implemented a configure option, --disable-werror, in response
to Ross's
https://github.com/json-c/json-c/issues/489
Signed-off-by: Douglas Royds
---
meta/recipes-devtools/json-c/json-c_0.13.1.bb | 30
---
meta/recipes-devtools/json-c/json-c_git.bb | 19
Use option '-M' of git format-patch may make the patch more clear.
And why not just backport the 'disable-werror' commit?
Re '-M': True, fair point.
Re the backport: There hasn't been a release in over a year, they
don't seem to have any plans to do so (see
https://github.com/json-c/json-c/issues/487), and this was easier. If
you have a substantial objection, yes, I could submit it again with a
patch instead of the update.
Just consider footprint, git repo will take more space than tar ball.
But no substantial objection.
We do also lose the RECIPE_UPSTREAM_VERSION functionality by switching
to the git HEAD, ie. there'll be no notification when they do
(eventually) release a new version.
I'm in two minds. Opinions?
Kai
Regards,
Kai
2 files changed, 19 insertions(+), 30 deletions(-)
delete mode 100644 meta/recipes-devtools/json-c/json-c_0.13.1.bb
create mode 100644 meta/recipes-devtools/json-c/json-c_git.bb
diff --git a/meta/recipes-devtools/json-c/json-c_0.13.1.bb
b/meta/recipes-devtools/json-c/json-c_0.13.1.bb
deleted file mode 100644
index 5b10e68297..00
--- a/meta/recipes-devtools/json-c/json-c_0.13.1.bb
+++ /dev/null
@@ -1,30 +0,0 @@
-SUMMARY = "C bindings for apps which will manipulate JSON data"
-DESCRIPTION = "JSON-C implements a reference counting object model
that allows you to easily construct JSON objects in C."
-HOMEPAGE = "https://github.com/json-c/json-c/wiki;
-LICENSE = "MIT"
-LIC_FILES_CHKSUM =
"file://COPYING;md5=de54b60fbbc35123ba193fea8ee216f2"
-
-SRC_URI =
"https://s3.amazonaws.com/json-c_releases/releases/${BP}.tar.gz;
-SRC_URI[md5sum] = "04969ad59cc37bddd83741a08b98f350"
-SRC_URI[sha256sum] =
"b87e608d4d3f7bfdd36ef78d56d53c74e66ab278d318b71e6002a369d36f4873"
-
-UPSTREAM_CHECK_REGEX = "json-c-(?P\d+(\.\d+)+).tar"
-# json-c releases page is fetching the list of releases in some
weird XML format
-# from https://s3.amazonaws.com/json-c_releases and processes it
with javascript :-/
-#UPSTREAM_CHECK_URI =
"https://s3.amazonaws.com/json-c_releases/releases/index.html;
-RECIPE_UPSTREAM_VERSION = "0.13.1"
-RECIPE_UPSTREAM_DATE = "Mar 04, 2018"
-CHECK_DATE = "May 02, 2018"
-
-RPROVIDES_${PN} = "libjson"
-
-inherit autotools
-
-EXTRA_OECONF = "--enable-rdrand"
-
-do_configure_prepend() {
- # Clean up autoconf cruft that should not be in the tarball
- rm -f ${S}/config.status
-}
-
-BBCLASSEXTEND = "native nativesdk"
diff --git a/meta/recipes-devtools/json-c/json-c_git.bb
b/meta/recipes-devtools/json-c/json-c_git.bb
new file mode 100644
index 00..07daa5ba11
--- /dev/null
+++ b/meta/recipes-devtools/json-c/json-c_git.bb
@@ -0,0 +1,19 @@
+SUMMARY = "C bindings for apps which will manipulate JSON data"
+DESCRIPTION = "JSON-C implements a reference counting object model
that allows you to easily construct JSON objects in C."
+HOMEPAGE = "https://github.com/json-c/json-c/wiki;
+LICENSE = "MIT"
+LIC_FILES_CHKSUM =
"file://COPYING;md5=de54b60fbbc35123ba193fea8ee216f2"
+
+SRC_URI = "git://github.com/json-c/json-c.git"
+SRCREV = "07ea04e65193c3e5c902c5b79421d5fa48ff67c7"
+S = "${WORKDIR}/git"
+
+RPROVIDES_${PN} = "libjson"
+
+inherit autotools
+
+EXTRA_OECONF = "--disable-werror \
+ --enable-rdrand \
+ "
+
+BBCLASSEXTEND = "native nativesdk"
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] json-c: update to current upstream head, with --disable-werror
On 11/06/19 2:46 PM, Kang Kai wrote:
On 2019/6/11 上午9:57, Douglas Royds via Openembedded-core wrote:
Upstream json-c haven't made a release since March 2018.
Adopt the current HEAD revision, pulling it directly from git.
icecc preprocesses source files locally before shipping them off to
be compiled
on remote hosts. This preprocessing removes comments, including /*
fallthough */
comments in switch statements that normally prevent an
implicit-fallthrough
warning, see https://github.com/icecc/icecream/issues/419
Rather than turning off -Werror by patching configure.ac, the
upstream project
has implemented a configure option, --disable-werror, in response to
Ross's
https://github.com/json-c/json-c/issues/489
Signed-off-by: Douglas Royds
---
meta/recipes-devtools/json-c/json-c_0.13.1.bb | 30 ---
meta/recipes-devtools/json-c/json-c_git.bb | 19
Use option '-M' of git format-patch may make the patch more clear. And
why not just backport the 'disable-werror' commit?
Re '-M': True, fair point.
Re the backport: There hasn't been a release in over a year, they don't
seem to have any plans to do so (see
https://github.com/json-c/json-c/issues/487), and this was easier. If
you have a substantial objection, yes, I could submit it again with a
patch instead of the update.
Regards,
Kai
2 files changed, 19 insertions(+), 30 deletions(-)
delete mode 100644 meta/recipes-devtools/json-c/json-c_0.13.1.bb
create mode 100644 meta/recipes-devtools/json-c/json-c_git.bb
diff --git a/meta/recipes-devtools/json-c/json-c_0.13.1.bb
b/meta/recipes-devtools/json-c/json-c_0.13.1.bb
deleted file mode 100644
index 5b10e68297..00
--- a/meta/recipes-devtools/json-c/json-c_0.13.1.bb
+++ /dev/null
@@ -1,30 +0,0 @@
-SUMMARY = "C bindings for apps which will manipulate JSON data"
-DESCRIPTION = "JSON-C implements a reference counting object model
that allows you to easily construct JSON objects in C."
-HOMEPAGE = "https://github.com/json-c/json-c/wiki;
-LICENSE = "MIT"
-LIC_FILES_CHKSUM =
"file://COPYING;md5=de54b60fbbc35123ba193fea8ee216f2"
-
-SRC_URI =
"https://s3.amazonaws.com/json-c_releases/releases/${BP}.tar.gz;
-SRC_URI[md5sum] = "04969ad59cc37bddd83741a08b98f350"
-SRC_URI[sha256sum] =
"b87e608d4d3f7bfdd36ef78d56d53c74e66ab278d318b71e6002a369d36f4873"
-
-UPSTREAM_CHECK_REGEX = "json-c-(?P\d+(\.\d+)+).tar"
-# json-c releases page is fetching the list of releases in some
weird XML format
-# from https://s3.amazonaws.com/json-c_releases and processes it
with javascript :-/
-#UPSTREAM_CHECK_URI =
"https://s3.amazonaws.com/json-c_releases/releases/index.html;
-RECIPE_UPSTREAM_VERSION = "0.13.1"
-RECIPE_UPSTREAM_DATE = "Mar 04, 2018"
-CHECK_DATE = "May 02, 2018"
-
-RPROVIDES_${PN} = "libjson"
-
-inherit autotools
-
-EXTRA_OECONF = "--enable-rdrand"
-
-do_configure_prepend() {
- # Clean up autoconf cruft that should not be in the tarball
- rm -f ${S}/config.status
-}
-
-BBCLASSEXTEND = "native nativesdk"
diff --git a/meta/recipes-devtools/json-c/json-c_git.bb
b/meta/recipes-devtools/json-c/json-c_git.bb
new file mode 100644
index 00..07daa5ba11
--- /dev/null
+++ b/meta/recipes-devtools/json-c/json-c_git.bb
@@ -0,0 +1,19 @@
+SUMMARY = "C bindings for apps which will manipulate JSON data"
+DESCRIPTION = "JSON-C implements a reference counting object model
that allows you to easily construct JSON objects in C."
+HOMEPAGE = "https://github.com/json-c/json-c/wiki;
+LICENSE = "MIT"
+LIC_FILES_CHKSUM =
"file://COPYING;md5=de54b60fbbc35123ba193fea8ee216f2"
+
+SRC_URI = "git://github.com/json-c/json-c.git"
+SRCREV = "07ea04e65193c3e5c902c5b79421d5fa48ff67c7"
+S = "${WORKDIR}/git"
+
+RPROVIDES_${PN} = "libjson"
+
+inherit autotools
+
+EXTRA_OECONF = "--disable-werror \
+ --enable-rdrand \
+ "
+
+BBCLASSEXTEND = "native nativesdk"
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH v2] json-c: Disable icecc to avoid implicit-fallthrough warning as error
Please disregard this one, I've sent a recipe update to pick up the new --disable-werror configure option. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH] json-c: update to current upstream head, with --disable-werror
Upstream json-c haven't made a release since March 2018.
Adopt the current HEAD revision, pulling it directly from git.
icecc preprocesses source files locally before shipping them off to be compiled
on remote hosts. This preprocessing removes comments, including /* fallthough */
comments in switch statements that normally prevent an implicit-fallthrough
warning, see https://github.com/icecc/icecream/issues/419
Rather than turning off -Werror by patching configure.ac, the upstream project
has implemented a configure option, --disable-werror, in response to Ross's
https://github.com/json-c/json-c/issues/489
Signed-off-by: Douglas Royds
---
meta/recipes-devtools/json-c/json-c_0.13.1.bb | 30 ---
meta/recipes-devtools/json-c/json-c_git.bb| 19
2 files changed, 19 insertions(+), 30 deletions(-)
delete mode 100644 meta/recipes-devtools/json-c/json-c_0.13.1.bb
create mode 100644 meta/recipes-devtools/json-c/json-c_git.bb
diff --git a/meta/recipes-devtools/json-c/json-c_0.13.1.bb
b/meta/recipes-devtools/json-c/json-c_0.13.1.bb
deleted file mode 100644
index 5b10e68297..00
--- a/meta/recipes-devtools/json-c/json-c_0.13.1.bb
+++ /dev/null
@@ -1,30 +0,0 @@
-SUMMARY = "C bindings for apps which will manipulate JSON data"
-DESCRIPTION = "JSON-C implements a reference counting object model that allows
you to easily construct JSON objects in C."
-HOMEPAGE = "https://github.com/json-c/json-c/wiki;
-LICENSE = "MIT"
-LIC_FILES_CHKSUM = "file://COPYING;md5=de54b60fbbc35123ba193fea8ee216f2"
-
-SRC_URI = "https://s3.amazonaws.com/json-c_releases/releases/${BP}.tar.gz;
-SRC_URI[md5sum] = "04969ad59cc37bddd83741a08b98f350"
-SRC_URI[sha256sum] =
"b87e608d4d3f7bfdd36ef78d56d53c74e66ab278d318b71e6002a369d36f4873"
-
-UPSTREAM_CHECK_REGEX = "json-c-(?P\d+(\.\d+)+).tar"
-# json-c releases page is fetching the list of releases in some weird XML
format
-# from https://s3.amazonaws.com/json-c_releases and processes it with
javascript :-/
-#UPSTREAM_CHECK_URI =
"https://s3.amazonaws.com/json-c_releases/releases/index.html;
-RECIPE_UPSTREAM_VERSION = "0.13.1"
-RECIPE_UPSTREAM_DATE = "Mar 04, 2018"
-CHECK_DATE = "May 02, 2018"
-
-RPROVIDES_${PN} = "libjson"
-
-inherit autotools
-
-EXTRA_OECONF = "--enable-rdrand"
-
-do_configure_prepend() {
-# Clean up autoconf cruft that should not be in the tarball
-rm -f ${S}/config.status
-}
-
-BBCLASSEXTEND = "native nativesdk"
diff --git a/meta/recipes-devtools/json-c/json-c_git.bb
b/meta/recipes-devtools/json-c/json-c_git.bb
new file mode 100644
index 00..07daa5ba11
--- /dev/null
+++ b/meta/recipes-devtools/json-c/json-c_git.bb
@@ -0,0 +1,19 @@
+SUMMARY = "C bindings for apps which will manipulate JSON data"
+DESCRIPTION = "JSON-C implements a reference counting object model that allows
you to easily construct JSON objects in C."
+HOMEPAGE = "https://github.com/json-c/json-c/wiki;
+LICENSE = "MIT"
+LIC_FILES_CHKSUM = "file://COPYING;md5=de54b60fbbc35123ba193fea8ee216f2"
+
+SRC_URI = "git://github.com/json-c/json-c.git"
+SRCREV = "07ea04e65193c3e5c902c5b79421d5fa48ff67c7"
+S = "${WORKDIR}/git"
+
+RPROVIDES_${PN} = "libjson"
+
+inherit autotools
+
+EXTRA_OECONF = "--disable-werror \
+--enable-rdrand \
+"
+
+BBCLASSEXTEND = "native nativesdk"
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v2] json-c: Disable icecc to avoid implicit-fallthrough warning as error
icecc preprocesses source files locally before shipping them off to be compiled
on remote hosts. This preprocessing removes comments, including /* fallthrough
*/
comments in switch statements that normally prevent the implicit-fallthrough
warning.
Rather than turning off -Werror by patching configure.ac, it is simpler to
disable icecc completely for json-c. There are very few source files to compile,
so the compilation is quick even without icecc.
See https://github.com/icecc/icecream/issues/419
Signed-off-by: Douglas Royds
---
meta/classes/icecc.bbclass | 1 +
1 file changed, 1 insertion(+)
diff --git a/meta/classes/icecc.bbclass b/meta/classes/icecc.bbclass
index edb0e10434..c0f21bdbfd 100644
--- a/meta/classes/icecc.bbclass
+++ b/meta/classes/icecc.bbclass
@@ -76,6 +76,7 @@ ICECC_ENV_DEBUG ??= ""
# target-sdk-provides-dummy - ${HOST_PREFIX} is empty which triggers the "NULL
# prefix" error.
ICECC_SYSTEM_PACKAGE_BL += "\
+json-c \
libgcc-initial \
target-sdk-provides-dummy \
"
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] json-c: Disable icecc to avoid implicit-fallthrough warning as error
__attribute__((fallthrough)) wouldn't help us with gcc 5.4 in the
-native case, of course.
I'll resubmit the patch using ICECC_SYSTEM_PACKAGE_BL
On 7/06/19 9:15 AM, Joshua Watt wrote:
FWIW: I've fixed a few upstream recipes that were doing this to use
the __attribute__((fallthrough)) instead of a comment.
Anyway, usually the correct fix is to add the package to
ICECC_SYSTEM_PACKAGE_BL in icecc.bbclass; it is maybe not ideal but
ICECC_DISABLED is the user control for enabling and disabling icecream
globally, not a per-recipe flag.
The current blacklist is woefully outdated, I really need to get
around to updating it
On 6/6/19 4:03 PM, Douglas Royds via Openembedded-core wrote:
I struggled to find a solution that would work in the -native case,
as we need to support gcc as old as 5.4 (Ubuntu 16.04).
The problem is somewhat specific to json-c, as -Werror is hard-coded
into configure.ac in this package.
On 6/06/19 9:45 PM, Burton, Ross wrote:
Considering the pain that the gcc upgrade introducing that warning
caused I'll be *very* surprised if this problem is limited to json-c.
Could the icecc class forcibly disable that warning instead?
Ross
On Thu, 6 Jun 2019 at 03:06, Douglas Royds via Openembedded-core
wrote:
icecc preprocesses source files locally before shipping them off to
be compiled
on remote hosts. This preprocessing removes comments, including /*
fallthrough */
comments in switch statements that normally prevent the
implicit-fallthrough
warning.
Rather than turning off -Werror by patching configure.ac, it is
simpler to
disable icecc completely for json-c. There are very few source
files to compile,
so the compilation is quick even without icecc.
See https://github.com/icecc/icecream/issues/419
Signed-off-by: Douglas Royds
---
meta/recipes-devtools/json-c/json-c_0.13.1.bb | 1 +
1 file changed, 1 insertion(+)
diff --git a/meta/recipes-devtools/json-c/json-c_0.13.1.bb
b/meta/recipes-devtools/json-c/json-c_0.13.1.bb
index 5b10e68297..8d2a20352d 100644
--- a/meta/recipes-devtools/json-c/json-c_0.13.1.bb
+++ b/meta/recipes-devtools/json-c/json-c_0.13.1.bb
@@ -21,6 +21,7 @@ RPROVIDES_${PN} = "libjson"
inherit autotools
EXTRA_OECONF = "--enable-rdrand"
+ICECC_DISABLED = "1"
do_configure_prepend() {
# Clean up autoconf cruft that should not be in the tarball
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] json-c: Disable icecc to avoid implicit-fallthrough warning as error
I struggled to find a solution that would work in the -native case, as
we need to support gcc as old as 5.4 (Ubuntu 16.04).
The problem is somewhat specific to json-c, as -Werror is hard-coded
into configure.ac in this package.
On 6/06/19 9:45 PM, Burton, Ross wrote:
Considering the pain that the gcc upgrade introducing that warning
caused I'll be *very* surprised if this problem is limited to json-c.
Could the icecc class forcibly disable that warning instead?
Ross
On Thu, 6 Jun 2019 at 03:06, Douglas Royds via Openembedded-core
wrote:
icecc preprocesses source files locally before shipping them off to be compiled
on remote hosts. This preprocessing removes comments, including /* fallthrough
*/
comments in switch statements that normally prevent the implicit-fallthrough
warning.
Rather than turning off -Werror by patching configure.ac, it is simpler to
disable icecc completely for json-c. There are very few source files to compile,
so the compilation is quick even without icecc.
See https://github.com/icecc/icecream/issues/419
Signed-off-by: Douglas Royds
---
meta/recipes-devtools/json-c/json-c_0.13.1.bb | 1 +
1 file changed, 1 insertion(+)
diff --git a/meta/recipes-devtools/json-c/json-c_0.13.1.bb
b/meta/recipes-devtools/json-c/json-c_0.13.1.bb
index 5b10e68297..8d2a20352d 100644
--- a/meta/recipes-devtools/json-c/json-c_0.13.1.bb
+++ b/meta/recipes-devtools/json-c/json-c_0.13.1.bb
@@ -21,6 +21,7 @@ RPROVIDES_${PN} = "libjson"
inherit autotools
EXTRA_OECONF = "--enable-rdrand"
+ICECC_DISABLED = "1"
do_configure_prepend() {
# Clean up autoconf cruft that should not be in the tarball
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH v2] json-c: Make implicit-fallthrough only a warning for compilation under icecc
Please disregard this one, I have submitted a newer patch under a new
subject-line:
json-c: Disable icecc to avoid implicit-fallthrough warning as error
On 6/06/19 1:13 PM, Douglas Royds wrote:
icecc preprocesses source files locally before shipping them off to be compiled
on remote hosts. This preprocessing removes comments, including /* fallthrough
*/
comments in switch statements that normally prevent the implicit-fallthrough
warning. Rather than turning off -Werror completely, just reduce
implicit-fallthrough to a warning only.
In the -native case, we might be building on an earlier gcc, eg. gcc 5.4
under Ubuntu 16.04, so we do disable -Werror completely.
See https://github.com/icecc/icecream/issues/419
Signed-off-by: Douglas Royds
---
meta/recipes-devtools/json-c/json-c_0.13.1.bb | 3 +++
1 file changed, 3 insertions(+)
diff --git a/meta/recipes-devtools/json-c/json-c_0.13.1.bb
b/meta/recipes-devtools/json-c/json-c_0.13.1.bb
index 5b10e68297..87a87aec44 100644
--- a/meta/recipes-devtools/json-c/json-c_0.13.1.bb
+++ b/meta/recipes-devtools/json-c/json-c_0.13.1.bb
@@ -22,6 +22,9 @@ inherit autotools
EXTRA_OECONF = "--enable-rdrand"
+TARGET_CPPFLAGS =+ "-Wno-error=implicit-fallthrough"
+BUILD_CPPFLAGS =+ "-Wno-error"
+
do_configure_prepend() {
# Clean up autoconf cruft that should not be in the tarball
rm -f ${S}/config.status
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH] json-c: Disable icecc to avoid implicit-fallthrough warning as error
icecc preprocesses source files locally before shipping them off to be compiled
on remote hosts. This preprocessing removes comments, including /* fallthrough
*/
comments in switch statements that normally prevent the implicit-fallthrough
warning.
Rather than turning off -Werror by patching configure.ac, it is simpler to
disable icecc completely for json-c. There are very few source files to compile,
so the compilation is quick even without icecc.
See https://github.com/icecc/icecream/issues/419
Signed-off-by: Douglas Royds
---
meta/recipes-devtools/json-c/json-c_0.13.1.bb | 1 +
1 file changed, 1 insertion(+)
diff --git a/meta/recipes-devtools/json-c/json-c_0.13.1.bb
b/meta/recipes-devtools/json-c/json-c_0.13.1.bb
index 5b10e68297..8d2a20352d 100644
--- a/meta/recipes-devtools/json-c/json-c_0.13.1.bb
+++ b/meta/recipes-devtools/json-c/json-c_0.13.1.bb
@@ -21,6 +21,7 @@ RPROVIDES_${PN} = "libjson"
inherit autotools
EXTRA_OECONF = "--enable-rdrand"
+ICECC_DISABLED = "1"
do_configure_prepend() {
# Clean up autoconf cruft that should not be in the tarball
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v2] json-c: Make implicit-fallthrough only a warning for compilation under icecc
icecc preprocesses source files locally before shipping them off to be compiled
on remote hosts. This preprocessing removes comments, including /* fallthrough
*/
comments in switch statements that normally prevent the implicit-fallthrough
warning. Rather than turning off -Werror completely, just reduce
implicit-fallthrough to a warning only.
In the -native case, we might be building on an earlier gcc, eg. gcc 5.4
under Ubuntu 16.04, so we do disable -Werror completely.
See https://github.com/icecc/icecream/issues/419
Signed-off-by: Douglas Royds
---
meta/recipes-devtools/json-c/json-c_0.13.1.bb | 3 +++
1 file changed, 3 insertions(+)
diff --git a/meta/recipes-devtools/json-c/json-c_0.13.1.bb
b/meta/recipes-devtools/json-c/json-c_0.13.1.bb
index 5b10e68297..87a87aec44 100644
--- a/meta/recipes-devtools/json-c/json-c_0.13.1.bb
+++ b/meta/recipes-devtools/json-c/json-c_0.13.1.bb
@@ -22,6 +22,9 @@ inherit autotools
EXTRA_OECONF = "--enable-rdrand"
+TARGET_CPPFLAGS =+ "-Wno-error=implicit-fallthrough"
+BUILD_CPPFLAGS =+ "-Wno-error"
+
do_configure_prepend() {
# Clean up autoconf cruft that should not be in the tarball
rm -f ${S}/config.status
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH] json-c: Make implicit-fallthrough only a warning for compilation under icecc
icecc preprocesses source files locally before shipping them off to be compiled
on remote hosts. This preprocessing removes comments, including /* fallthough */
comments in switch statements that normally prevent the implicit-fallthrough
warning. Rather than turning off -Werror completely, just reduce
implicit-fallthrough to a warning only.
See https://github.com/icecc/icecream/issues/419
Signed-off-by: Douglas Royds
---
meta/recipes-devtools/json-c/json-c_0.13.1.bb | 3 +++
1 file changed, 3 insertions(+)
diff --git a/meta/recipes-devtools/json-c/json-c_0.13.1.bb
b/meta/recipes-devtools/json-c/json-c_0.13.1.bb
index 5b10e68297..ddda7375d3 100644
--- a/meta/recipes-devtools/json-c/json-c_0.13.1.bb
+++ b/meta/recipes-devtools/json-c/json-c_0.13.1.bb
@@ -22,6 +22,9 @@ inherit autotools
EXTRA_OECONF = "--enable-rdrand"
+TARGET_CPPFLAGS =+ "-Wno-error=implicit-fallthrough"
+BUILD_CPPFLAGS =+ "-Wno-error=implicit-fallthrough"
+
do_configure_prepend() {
# Clean up autoconf cruft that should not be in the tarball
rm -f ${S}/config.status
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] distutils: Tidy and simplify for readability
On 14/05/19 6:41 PM, Martin Hundebøll wrote: On 14/05/2019 08.16, Douglas Royds via Openembedded-core wrote: Can anyone explain why I appear on patchwork as "Andrii Bordunov via Openembedded-core"? Yes. Because the DMARC policy for your domain is "quarantine": % dig _dmarc.taitradio.com TXT | grep -oE 'p=(reject|quarantine|none)' p=quarantine If the mailing list simply forwards your mails, they will fail validation, and thus be held back by the receiving mail servers. To avoid this, Mailman changes the "From:" header to "Douglas Royds via Openembedded-core ". Now the email address seen by patchwork is the same for everyone with either "reject" or "quarantine" as DMARC policy. I guess patchwork has that address associated with Andrii (maybe he came first). More info from mailman: https://wiki.list.org/DEV/DMARC Great, thanks for that. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] distutils: Tidy and simplify for readability
Can anyone explain why I appear on patchwork as "Andrii Bordunov via Openembedded-core"? See https://patchwork.openembedded.org/series/17604/ -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH] distutils: Tidy and simplify for readability
Line lengths, remove duplication, and use the PYTHON variable provided by
pythonnative.bbclass.
Coincidentally fixes a dormant defect in distutils3.bbclass in which we were
sedding for STAGING_BINDIR_NATIVE/python-python3/python3.
Signed-off-by: Douglas Royds
---
meta/classes/distutils.bbclass | 26 +++---
meta/classes/distutils3.bbclass | 22 --
2 files changed, 19 insertions(+), 29 deletions(-)
diff --git a/meta/classes/distutils.bbclass b/meta/classes/distutils.bbclass
index b5c9c2fbbd..3759b58263 100644
--- a/meta/classes/distutils.bbclass
+++ b/meta/classes/distutils.bbclass
@@ -52,25 +52,20 @@ distutils_do_install() {
# support filenames with *spaces*
# only modify file if it contains path and recompile it
-find ${D} -name "*.py" -exec grep -q ${D} {} \; -exec sed -i -e
s:${D}::g {} \; -exec ${STAGING_BINDIR_NATIVE}/python-native/python
-mcompileall {} \;
+find ${D} -name "*.py" -exec grep -q ${D} {} \; \
+ -exec sed -i -e s:${D}::g {} \; \
+ -exec
${STAGING_BINDIR_NATIVE}/python-native/python -mcompileall {} \;
-if test -e ${D}${bindir} ; then
-for i in ${D}${bindir}/* ; do \
-sed -i -e
s:${STAGING_BINDIR_NATIVE}/python-native/python:${USRBINPATH}/env\
${DISTUTILS_PYTHON}:g $i
+for i in ${D}${bindir}/* ${D}${sbindir}/*; do
+if [ -f "$i" ]; then
+sed -i -e s:${PYTHON}:${USRBINPATH}/env\ ${DISTUTILS_PYTHON}:g
$i
sed -i -e s:${STAGING_BINDIR_NATIVE}:${bindir}:g $i
-done
-fi
-
-if [ -e ${D}${sbindir} ]; then
-for i in ${D}${sbindir}/* ; do \
-sed -i -e
s:${STAGING_BINDIR_NATIVE}/python-native/python:${USRBINPATH}/env\
${DISTUTILS_PYTHON}:g $i
-sed -i -e s:${STAGING_BINDIR_NATIVE}:${bindir}:g $i
-done
-fi
+fi
+done
rm -f ${D}${PYTHON_SITEPACKAGES_DIR}/easy-install.pth
rm -f ${D}${PYTHON_SITEPACKAGES_DIR}/site.py*
-
+
#
# FIXME: Bandaid against wrong datadir computation
#
@@ -80,7 +75,8 @@ distutils_do_install() {
fi
# Fix backport modules
- if [ -e
${STAGING_LIBDIR}/${PYTHON_DIR}/site-packages/backports/__init__.py ] && [ -e
${D}${PYTHON_SITEPACKAGES_DIR}/backports/__init__.py ]; then
+ if [ -e
${STAGING_LIBDIR}/${PYTHON_DIR}/site-packages/backports/__init__.py ] &&
+ [ -e ${D}${PYTHON_SITEPACKAGES_DIR}/backports/__init__.py ]; then
rm ${D}${PYTHON_SITEPACKAGES_DIR}/backports/__init__.py;
rm ${D}${PYTHON_SITEPACKAGES_DIR}/backports/__init__.pyc;
fi
diff --git a/meta/classes/distutils3.bbclass b/meta/classes/distutils3.bbclass
index 42e7f5ae53..05a24bfe26 100644
--- a/meta/classes/distutils3.bbclass
+++ b/meta/classes/distutils3.bbclass
@@ -56,24 +56,18 @@ distutils3_do_install() {
bbfatal_log "'${PYTHON_PN} setup.py install ${DISTUTILS_INSTALL_ARGS}'
execution failed."
# support filenames with *spaces*
-find ${D} -name "*.py" -exec grep -q ${D} {} \; -exec sed -i -e
s:${D}::g {} \;
+find ${D} -name "*.py" -exec grep -q ${D} {} \; \
+ -exec sed -i -e s:${D}::g {} \;
-if test -e ${D}${bindir} ; then
-for i in ${D}${bindir}/* ; do \
-sed -i -e
s:${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-native/${PYTHON_PN}:${USRBINPATH}/env\
${DISTUTILS_PYTHON}:g $i
+for i in ${D}${bindir}/* ${D}${sbindir}/*; do
+if [ -f "$i" ]; then
+sed -i -e s:${PYTHON}:${USRBINPATH}/env\ ${DISTUTILS_PYTHON}:g
$i
sed -i -e s:${STAGING_BINDIR_NATIVE}:${bindir}:g $i
-done
-fi
-
-if test -e ${D}${sbindir}; then
-for i in ${D}${sbindir}/* ; do \
-sed -i -e
s:${STAGING_BINDIR_NATIVE}/python-${PYTHON_PN}/${PYTHON_PN}:${USRBINPATH}/env\
${DISTUTILS_PYTHON}:g $i
-sed -i -e s:${STAGING_BINDIR_NATIVE}:${bindir}:g $i
-done
-fi
+fi
+done
rm -f ${D}${PYTHON_SITEPACKAGES_DIR}/easy-install.pth
-
+
#
# FIXME: Bandaid against wrong datadir computation
#
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v3] distutils: Run python from the PATH in the -native case as well
The python distutils generate a python wrapper script for each package,
containing shebang lines pointing to the python executable.
In our case, this is a fully-qualified path to python-native in the
recipe-sysroot-native.
Ubuntu 18.04 restricts the useful length of the shebang line to 125
characters, and Ubuntu 16.04 restricts it to 77. In both cases, the
staged python script fails to run due to the length of the path to
the python-native executable.
Replace the shebang line with nativepython or nativepython3 as appropriate.
The nativepython symlink is installed by the python-native recipe:
#!/usr/bin/env nativepython
We were already doing this for on-target distutils components.
This change applies the sed-line to -native distutils components as well.
In this way, -native clients of these components can invoke the wrapper scripts
directly, without themselves needing to inherit pythonnative.
This works around a known setuptools issue:
https://github.com/pypa/setuptools/issues/494
Even once this issue has been resolved upstream,
we will still need to replace `python` with `nativepython`
Signed-off-by: Douglas Royds
---
meta/classes/distutils.bbclass | 11 +--
meta/classes/distutils3.bbclass | 7 +--
2 files changed, 10 insertions(+), 8 deletions(-)
diff --git a/meta/classes/distutils.bbclass b/meta/classes/distutils.bbclass
index 9862731493..b5c9c2fbbd 100644
--- a/meta/classes/distutils.bbclass
+++ b/meta/classes/distutils.bbclass
@@ -9,6 +9,9 @@ DISTUTILS_INSTALL_ARGS ?= "--root=${D} \
--install-lib=${PYTHON_SITEPACKAGES_DIR} \
--install-data=${datadir}"
+DISTUTILS_PYTHON = "python"
+DISTUTILS_PYTHON_class-native = "nativepython"
+
distutils_do_configure() {
if [ "${CLEANBROKEN}" != "1" ] ; then
NO_FETCH_BUILD=1 \
@@ -53,18 +56,14 @@ distutils_do_install() {
if test -e ${D}${bindir} ; then
for i in ${D}${bindir}/* ; do \
-if [ ${PN} != "${BPN}-native" ]; then
- sed -i -e
s:${STAGING_BINDIR_NATIVE}/python-native/python:${USRBINPATH}/env\ python:g $i
- fi
+sed -i -e
s:${STAGING_BINDIR_NATIVE}/python-native/python:${USRBINPATH}/env\
${DISTUTILS_PYTHON}:g $i
sed -i -e s:${STAGING_BINDIR_NATIVE}:${bindir}:g $i
done
fi
if [ -e ${D}${sbindir} ]; then
for i in ${D}${sbindir}/* ; do \
-if [ ${PN} != "${BPN}-native" ]; then
- sed -i -e
s:${STAGING_BINDIR_NATIVE}/python-native/python:${USRBINPATH}/env\ python:g $i
- fi
+sed -i -e
s:${STAGING_BINDIR_NATIVE}/python-native/python:${USRBINPATH}/env\
${DISTUTILS_PYTHON}:g $i
sed -i -e s:${STAGING_BINDIR_NATIVE}:${bindir}:g $i
done
fi
diff --git a/meta/classes/distutils3.bbclass b/meta/classes/distutils3.bbclass
index 834e322474..42e7f5ae53 100644
--- a/meta/classes/distutils3.bbclass
+++ b/meta/classes/distutils3.bbclass
@@ -10,6 +10,9 @@ DISTUTILS_INSTALL_ARGS ?= "--root=${D} \
--install-lib=${PYTHON_SITEPACKAGES_DIR} \
--install-data=${datadir}"
+DISTUTILS_PYTHON = "python3"
+DISTUTILS_PYTHON_class-native = "nativepython3"
+
distutils3_do_configure() {
if [ "${CLEANBROKEN}" != "1" ] ; then
NO_FETCH_BUILD=1 \
@@ -57,14 +60,14 @@ distutils3_do_install() {
if test -e ${D}${bindir} ; then
for i in ${D}${bindir}/* ; do \
-sed -i -e
s:${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-native/${PYTHON_PN}:${USRBINPATH}/env\
${PYTHON_PN}:g $i
+sed -i -e
s:${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-native/${PYTHON_PN}:${USRBINPATH}/env\
${DISTUTILS_PYTHON}:g $i
sed -i -e s:${STAGING_BINDIR_NATIVE}:${bindir}:g $i
done
fi
if test -e ${D}${sbindir}; then
for i in ${D}${sbindir}/* ; do \
-sed -i -e
s:${STAGING_BINDIR_NATIVE}/python-${PYTHON_PN}/${PYTHON_PN}:${USRBINPATH}/env\
${PYTHON_PN}:g $i
+sed -i -e
s:${STAGING_BINDIR_NATIVE}/python-${PYTHON_PN}/${PYTHON_PN}:${USRBINPATH}/env\
${DISTUTILS_PYTHON}:g $i
sed -i -e s:${STAGING_BINDIR_NATIVE}:${bindir}:g $i
done
fi
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v2] distutils: Run python from the PATH in the -native case as well
The python distutils generate a python wrapper script for each package,
containing shebang lines pointing to the python executable.
In our case, this is a fully-qualified path to python-native in the
recipe-sysroot-native.
Ubuntu 18.04 restricts the useful length of the shebang line to 125
characters, and Ubuntu 16.04 restricts it to 77. In both cases, the
staged python script fails to run due to the length of the path to
the python-native executable.
Replace the shebang line with nativepython or nativepython3 as appropriate.
The nativepython symlink is installed by the python-native recipe:
#!/usr/bin/env nativepython
We were already doing this for on-target distutils components.
This change applies the sed-line to -native distutils components as well.
In this way, -native clients of these components can invoke the wrapper scripts
directly, without themselves needing to inherit pythonnative.
This works around a known setuptools issue:
https://github.com/pypa/setuptools/issues/494
Even once this issue has been resolved upstream,
we will still need to replace `python` with `nativepython`
Signed-off-by: Douglas Royds
---
meta/classes/distutils.bbclass | 11 +--
meta/classes/distutils3.bbclass | 7 +--
2 files changed, 10 insertions(+), 8 deletions(-)
diff --git a/meta/classes/distutils.bbclass b/meta/classes/distutils.bbclass
index 9862731493..8d258daea4 100644
--- a/meta/classes/distutils.bbclass
+++ b/meta/classes/distutils.bbclass
@@ -9,6 +9,9 @@ DISTUTILS_INSTALL_ARGS ?= "--root=${D} \
--install-lib=${PYTHON_SITEPACKAGES_DIR} \
--install-data=${datadir}"
+DISTUTILS_PYTHON = "${PYTHON_PN}"
+DISTUTILS_PYTHON_class-native = "native${PYTHON_PN}"
+
distutils_do_configure() {
if [ "${CLEANBROKEN}" != "1" ] ; then
NO_FETCH_BUILD=1 \
@@ -53,18 +56,14 @@ distutils_do_install() {
if test -e ${D}${bindir} ; then
for i in ${D}${bindir}/* ; do \
-if [ ${PN} != "${BPN}-native" ]; then
- sed -i -e
s:${STAGING_BINDIR_NATIVE}/python-native/python:${USRBINPATH}/env\ python:g $i
- fi
+sed -i -e
s:${STAGING_BINDIR_NATIVE}/python-native/python:${USRBINPATH}/env\
${DISTUTILS_PYTHON}:g $i
sed -i -e s:${STAGING_BINDIR_NATIVE}:${bindir}:g $i
done
fi
if [ -e ${D}${sbindir} ]; then
for i in ${D}${sbindir}/* ; do \
-if [ ${PN} != "${BPN}-native" ]; then
- sed -i -e
s:${STAGING_BINDIR_NATIVE}/python-native/python:${USRBINPATH}/env\ python:g $i
- fi
+sed -i -e
s:${STAGING_BINDIR_NATIVE}/python-native/python:${USRBINPATH}/env\
${DISTUTILS_PYTHON}:g $i
sed -i -e s:${STAGING_BINDIR_NATIVE}:${bindir}:g $i
done
fi
diff --git a/meta/classes/distutils3.bbclass b/meta/classes/distutils3.bbclass
index 834e322474..bae536d343 100644
--- a/meta/classes/distutils3.bbclass
+++ b/meta/classes/distutils3.bbclass
@@ -10,6 +10,9 @@ DISTUTILS_INSTALL_ARGS ?= "--root=${D} \
--install-lib=${PYTHON_SITEPACKAGES_DIR} \
--install-data=${datadir}"
+DISTUTILS_PYTHON = "${PYTHON_PN}"
+DISTUTILS_PYTHON_class-native = "native${PYTHON_PN}"
+
distutils3_do_configure() {
if [ "${CLEANBROKEN}" != "1" ] ; then
NO_FETCH_BUILD=1 \
@@ -57,14 +60,14 @@ distutils3_do_install() {
if test -e ${D}${bindir} ; then
for i in ${D}${bindir}/* ; do \
-sed -i -e
s:${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-native/${PYTHON_PN}:${USRBINPATH}/env\
${PYTHON_PN}:g $i
+sed -i -e
s:${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-native/${PYTHON_PN}:${USRBINPATH}/env\
${DISTUTILS_PYTHON}:g $i
sed -i -e s:${STAGING_BINDIR_NATIVE}:${bindir}:g $i
done
fi
if test -e ${D}${sbindir}; then
for i in ${D}${sbindir}/* ; do \
-sed -i -e
s:${STAGING_BINDIR_NATIVE}/python-${PYTHON_PN}/${PYTHON_PN}:${USRBINPATH}/env\
${PYTHON_PN}:g $i
+sed -i -e
s:${STAGING_BINDIR_NATIVE}/python-${PYTHON_PN}/${PYTHON_PN}:${USRBINPATH}/env\
${DISTUTILS_PYTHON}:g $i
sed -i -e s:${STAGING_BINDIR_NATIVE}:${bindir}:g $i
done
fi
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH] distutils: Run python from the PATH in the -native case as well
The python distutils generate a python wrapper script for each package.
These python scripts have shebang lines pointing to the python executable.
In our case, this is a fully-qualified path to python-native in the
recipe-sysroot-native.
Ubuntu 18.04 restricts the useful length of the shebang line to 125
characters, and Ubuntu 16.04 restricts it to 77. In both cases, the
staged python script fails to run due to the length of the path to
the python-native executable.
Replace the shebang line with:
#!/usr/bin/env python
We were already doing this for on-target distutils components.
This change applies the sed-line to -native distutils components as
well.
The python executable must be in the PATH.
Client components must inherit pythonnative.
Signed-off-by: Douglas Royds
---
meta/classes/distutils.bbclass | 8 ++--
1 file changed, 2 insertions(+), 6 deletions(-)
diff --git a/meta/classes/distutils.bbclass b/meta/classes/distutils.bbclass
index 9862731493..e7d79271e3 100644
--- a/meta/classes/distutils.bbclass
+++ b/meta/classes/distutils.bbclass
@@ -53,18 +53,14 @@ distutils_do_install() {
if test -e ${D}${bindir} ; then
for i in ${D}${bindir}/* ; do \
-if [ ${PN} != "${BPN}-native" ]; then
- sed -i -e
s:${STAGING_BINDIR_NATIVE}/python-native/python:${USRBINPATH}/env\ python:g $i
- fi
+sed -i -e
s:${STAGING_BINDIR_NATIVE}/python-native/python:${USRBINPATH}/env\ python:g $i
sed -i -e s:${STAGING_BINDIR_NATIVE}:${bindir}:g $i
done
fi
if [ -e ${D}${sbindir} ]; then
for i in ${D}${sbindir}/* ; do \
-if [ ${PN} != "${BPN}-native" ]; then
- sed -i -e
s:${STAGING_BINDIR_NATIVE}/python-native/python:${USRBINPATH}/env\ python:g $i
- fi
+sed -i -e
s:${STAGING_BINDIR_NATIVE}/python-native/python:${USRBINPATH}/env\ python:g $i
sed -i -e s:${STAGING_BINDIR_NATIVE}:${bindir}:g $i
done
fi
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v2] cmake: Support Eclipse and other cmake generators
From: Nikhil Pal Singh
Support project-file generators such as CodeBlocks, CodeLite,
Eclipse, Sublime, and Kate for both make and Ninja build systems.
The following generators are listed in cmake --help:
Unix Makefiles = Generates standard UNIX makefiles.
Ninja= Generates build.ninja files.
Watcom WMake = Generates Watcom WMake makefiles.
CodeBlocks - Ninja = Generates CodeBlocks project files.
CodeBlocks - Unix Makefiles = Generates CodeBlocks project files.
CodeLite - Ninja = Generates CodeLite project files.
CodeLite - Unix Makefiles= Generates CodeLite project files.
Sublime Text 2 - Ninja = Generates Sublime Text 2 project files.
Sublime Text 2 - Unix Makefiles
= Generates Sublime Text 2 project files.
Kate - Ninja = Generates Kate project files.
Kate - Unix Makefiles= Generates Kate project files.
Eclipse CDT4 - Ninja = Generates Eclipse CDT 4.0 project files.
Eclipse CDT4 - Unix Makefiles= Generates Eclipse CDT 4.0 project files.
All but one of these contain one of the strings, "Unix Makefiles" or "Ninja".
In each of these cases, cmake generates the Makefiles (or ninja files
respectively),
and also the appropriate project files, eg. .project and .cproject for Eclipse.
A user can set OECMAKE_GENERATOR in their local.conf to any
one of these strings, except "Watcom WMake" (not supported).
Signed-off-by: Nikhil Pal Singh
---
meta/classes/cmake.bbclass | 9 +
1 file changed, 5 insertions(+), 4 deletions(-)
diff --git a/meta/classes/cmake.bbclass b/meta/classes/cmake.bbclass
index e16630434e..d3f0d70847 100644
--- a/meta/classes/cmake.bbclass
+++ b/meta/classes/cmake.bbclass
@@ -10,13 +10,14 @@ OECMAKE_GENERATOR ?= "Ninja"
python() {
generator = d.getVar("OECMAKE_GENERATOR")
-if generator == "Unix Makefiles":
-args = "-G 'Unix Makefiles' -DCMAKE_MAKE_PROGRAM=" + d.getVar("MAKE")
+if "Unix Makefiles" in generator:
+args = "-G '" + generator + "' -DCMAKE_MAKE_PROGRAM=" +
d.getVar("MAKE")
d.setVar("OECMAKE_GENERATOR_ARGS", args)
d.setVarFlag("do_compile", "progress", "percent")
-elif generator == "Ninja":
+elif "Ninja" in generator:
+args = "-G '" + generator + "' -DCMAKE_MAKE_PROGRAM=ninja"
d.appendVar("DEPENDS", " ninja-native")
-d.setVar("OECMAKE_GENERATOR_ARGS", "-G Ninja
-DCMAKE_MAKE_PROGRAM=ninja")
+d.setVar("OECMAKE_GENERATOR_ARGS", args)
d.setVarFlag("do_compile", "progress", r"outof:^\[(\d+)/(\d+)\]\s+")
else:
bb.fatal("Unknown CMake Generator %s" % generator)
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] cmake: Support Eclipse and other cmake generators
On 9/04/19 9:04 AM, Burton, Ross wrote:
On Mon, 8 Apr 2019 at 21:56, Douglas Royds wrote:
python() {
generator = d.getVar("OECMAKE_GENERATOR")
-if generator == "Unix Makefiles":
-args = "-G 'Unix Makefiles' -DCMAKE_MAKE_PROGRAM=" + d.getVar("MAKE")
+if "Unix Makefiles" in generator:
+args = "-G '" + generator + "' -DCMAKE_MAKE_PROGRAM=" +
d.getVar("MAKE")
So are there other generators such as "Unix Makefiles - Something
Else" that needs to trigger this code path? A real world example
would be useful.
else:
bb.fatal("Unknown CMake Generator %s" % generator)
Because obviously at this point, arbitrary generators are very much
not supported yet.
Ross
Here's the complete list, from cmake --help:
The following generators are available on this platform:
Unix Makefiles = Generates standard UNIX makefiles.
Ninja = Generates build.ninja files.
Watcom WMake = Generates Watcom WMake makefiles.
CodeBlocks - Ninja = Generates CodeBlocks project files.
CodeBlocks - Unix Makefiles = Generates CodeBlocks project files.
CodeLite - Ninja = Generates CodeLite project files.
CodeLite - Unix Makefiles = Generates CodeLite project files.
Sublime Text 2 - Ninja = Generates Sublime Text 2 project
files.
Sublime Text 2 - Unix Makefiles
= Generates Sublime Text 2 project
files.
Kate - Ninja = Generates Kate project files.
Kate - Unix Makefiles = Generates Kate project files.
Eclipse CDT4 - Ninja = Generates Eclipse CDT 4.0 project
files.
Eclipse CDT4 - Unix Makefiles= Generates Eclipse CDT 4.0 project
files.
All but one of them contain one of the strings, "Unix Makefiles" or
"Ninja". In each of these cases, they generate the Makefiles (or ninja
files respectively), but also generate the appropriate project files,
eg. .project and .cproject for Eclipse.
A user can set their OECMAKE_GENERATOR to any one of these strings,
except "Watcom WMake".
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH] cmake: Support Eclipse and other cmake generators
From: Nikhil Pal Singh
As per Generators information mentioned in the cmake --help file
only "Unix Makefiles" and "Ninja" generators are supported as
standard project files.
With this change generic solution can be created for all the other
supported generators such as CodeBlocks, CodeLite, Eclipse CDT4,
KDevelop3, Kate and others for make or Ninja build systems.
It will also give and option to set OECMAKE_GENERATOR in local.conf
file for configuring any specific generators.
Signed-off-by: Nikhil Pal Singh
---
meta/classes/cmake.bbclass | 9 +
1 file changed, 5 insertions(+), 4 deletions(-)
diff --git a/meta/classes/cmake.bbclass b/meta/classes/cmake.bbclass
index e16630434e..d3f0d70847 100644
--- a/meta/classes/cmake.bbclass
+++ b/meta/classes/cmake.bbclass
@@ -10,13 +10,14 @@ OECMAKE_GENERATOR ?= "Ninja"
python() {
generator = d.getVar("OECMAKE_GENERATOR")
-if generator == "Unix Makefiles":
-args = "-G 'Unix Makefiles' -DCMAKE_MAKE_PROGRAM=" + d.getVar("MAKE")
+if "Unix Makefiles" in generator:
+args = "-G '" + generator + "' -DCMAKE_MAKE_PROGRAM=" +
d.getVar("MAKE")
d.setVar("OECMAKE_GENERATOR_ARGS", args)
d.setVarFlag("do_compile", "progress", "percent")
-elif generator == "Ninja":
+elif "Ninja" in generator:
+args = "-G '" + generator + "' -DCMAKE_MAKE_PROGRAM=ninja"
d.appendVar("DEPENDS", " ninja-native")
-d.setVar("OECMAKE_GENERATOR_ARGS", "-G Ninja
-DCMAKE_MAKE_PROGRAM=ninja")
+d.setVar("OECMAKE_GENERATOR_ARGS", args)
d.setVarFlag("do_compile", "progress", r"outof:^\[(\d+)/(\d+)\]\s+")
else:
bb.fatal("Unknown CMake Generator %s" % generator)
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v3] cmake: Reduce verbosity for make invocation
New variable OECMAKE_VERBOSE_MAKEFILE defaults to True.
Since the dawn of time, we have set CMAKE_VERBOSE_MAKEFILE=1 (True) in
cmake.bbclass.
Back in 2016, we also explicitly set VERBOSE=1 in cmake_do_compile(),
to ensure that make (and ninja) output were verbose in log.do_compile.
Setting OECMAKE_VERBOSE_MAKEFILE to False means that make (or ninja)
invocations from the command-line are non-verbose,
giving CMake's default human-readable output on the terminal instead.
The user can still invoke VERBOSE=1 make if they do want verbose output.
This has no effect on the verbose output that goes into the logs.
Signed-off-by: Douglas Royds
---
meta/classes/cmake.bbclass | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/meta/classes/cmake.bbclass b/meta/classes/cmake.bbclass
index fa7f68c99b..0d44a74046 100644
--- a/meta/classes/cmake.bbclass
+++ b/meta/classes/cmake.bbclass
@@ -62,6 +62,7 @@ EXTRA_OECMAKE_BUILD_prepend_task-install =
"${PARALLEL_MAKEINST} "
OECMAKE_TARGET_COMPILE ?= "all"
OECMAKE_TARGET_INSTALL ?= "install"
+OECMAKE_VERBOSE_MAKEFILE ?= "True"
# CMake expects target architectures in the format of uname(2),
# which do not always match TARGET_ARCH, so all the necessary
@@ -164,7 +165,7 @@ cmake_do_configure() {
-DCMAKE_INSTALL_DATAROOTDIR:PATH=${@os.path.relpath(d.getVar('datadir'),
d.getVar('prefix'))} \
-DCMAKE_INSTALL_SO_NO_EXE=0 \
-DCMAKE_TOOLCHAIN_FILE=${WORKDIR}/toolchain.cmake \
- -DCMAKE_VERBOSE_MAKEFILE=1 \
+ -DCMAKE_VERBOSE_MAKEFILE=${OECMAKE_VERBOSE_MAKEFILE} \
-DCMAKE_NO_SYSTEM_FROM_IMPORTED=1 \
${EXTRA_OECMAKE} \
-Wno-dev
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v2] cmake: Reduce verbosity for make invocation
New variable OECMAKE_VERBOSE_MAKEFILE defaults to True.
Since the dawn of time, we have set CMAKE_VERBOSE_MAKEFILE=1 (True) in
cmake.bbclass.
Back in 2016, we also explicitly set VERBOSE=1 in cmake_do_compile(),
to ensure that make (and ninja) output were verbose in log.do_compile.
Setting OECMAKE_VERBOSE_MAKEFILE to False means that make (or ninja)
invocations from the command-line are non-verbose,
giving CMake's default human-readable output on the terminal instead.
The user can still invoke VERBOSE=1 make if they do want verbose output.
This has no effect on the verbose output that goes into the logs.
---
meta/classes/cmake.bbclass | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/meta/classes/cmake.bbclass b/meta/classes/cmake.bbclass
index fa7f68c99b..0d44a74046 100644
--- a/meta/classes/cmake.bbclass
+++ b/meta/classes/cmake.bbclass
@@ -62,6 +62,7 @@ EXTRA_OECMAKE_BUILD_prepend_task-install =
"${PARALLEL_MAKEINST} "
OECMAKE_TARGET_COMPILE ?= "all"
OECMAKE_TARGET_INSTALL ?= "install"
+OECMAKE_VERBOSE_MAKEFILE ?= "True"
# CMake expects target architectures in the format of uname(2),
# which do not always match TARGET_ARCH, so all the necessary
@@ -164,7 +165,7 @@ cmake_do_configure() {
-DCMAKE_INSTALL_DATAROOTDIR:PATH=${@os.path.relpath(d.getVar('datadir'),
d.getVar('prefix'))} \
-DCMAKE_INSTALL_SO_NO_EXE=0 \
-DCMAKE_TOOLCHAIN_FILE=${WORKDIR}/toolchain.cmake \
- -DCMAKE_VERBOSE_MAKEFILE=1 \
+ -DCMAKE_VERBOSE_MAKEFILE=${OECMAKE_VERBOSE_MAKEFILE} \
-DCMAKE_NO_SYSTEM_FROM_IMPORTED=1 \
${EXTRA_OECMAKE} \
-Wno-dev
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] cmake: Reduce verbosity for make invocation
On 12/03/19 5:09 PM, Khem Raj wrote: On Mon, Mar 11, 2019 at 5:47 PM Douglas Royds wrote: On 12/03/19 6:18 AM, Khem Raj wrote: On Sun, Mar 10, 2019 at 10:53 PM Douglas Royds wrote: Since the dawn of time, we have set CMAKE_VERBOSE_MAKEFILE=1 in cmake.bbclass. Back in 2016, we also explicitly set VERBOSE=1 in cmake_do_compile(), to ensure that make (and ninja) output were verbose in log.do_compile. Turning off CMAKE_VERBOSE_MAKEFILE=1 means that make (or ninja) invocations from the command-line are non-verbose, giving CMake's default human-readable output on the terminal instead. The user can still invoke VERBOSE=1 make if they do want verbose output. This has no effect on the verbose output that goes into the logs. I think it would be better to have knob to turn is on and off both on console and log files. If you want it on for some reason, you can turn it on in your local.conf: EXTRA_OECMAKE_append=" -DCMAKE_VERBOSE_MAKEFILE=1" Alternatively, you can turn verbosity on for a single invocation (this is what I do when I need it): make VERBOSE=1 You can also set an environment variable to make it always verbose, if you want that for some reason: VERBOSE=1 make Do we really need a new variable for this? its fine if we can ensure consistent behaviour, I dont like the fact that cmdline output would be different than whats stored in log files. OK, that's a fair point. I'll update it with a variable. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] cmake: Reduce verbosity for make invocation
On 12/03/19 6:18 AM, Khem Raj wrote: On Sun, Mar 10, 2019 at 10:53 PM Douglas Royds wrote: Since the dawn of time, we have set CMAKE_VERBOSE_MAKEFILE=1 in cmake.bbclass. Back in 2016, we also explicitly set VERBOSE=1 in cmake_do_compile(), to ensure that make (and ninja) output were verbose in log.do_compile. Turning off CMAKE_VERBOSE_MAKEFILE=1 means that make (or ninja) invocations from the command-line are non-verbose, giving CMake's default human-readable output on the terminal instead. The user can still invoke VERBOSE=1 make if they do want verbose output. This has no effect on the verbose output that goes into the logs. I think it would be better to have knob to turn is on and off both on console and log files. If you want it on for some reason, you can turn it on in your local.conf: EXTRA_OECMAKE_append=" -DCMAKE_VERBOSE_MAKEFILE=1" Alternatively, you can turn verbosity on for a single invocation (this is what I do when I need it): make VERBOSE=1 You can also set an environment variable to make it always verbose, if you want that for some reason: VERBOSE=1 make Do we really need a new variable for this? -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH] cmake: Reduce verbosity for make invocation
Since the dawn of time, we have set CMAKE_VERBOSE_MAKEFILE=1 in cmake.bbclass.
Back in 2016, we also explicitly set VERBOSE=1 in cmake_do_compile(),
to ensure that make (and ninja) output were verbose in log.do_compile.
Turning off CMAKE_VERBOSE_MAKEFILE=1 means that make (or ninja)
invocations from the command-line are non-verbose,
giving CMake's default human-readable output on the terminal instead.
The user can still invoke VERBOSE=1 make if they do want verbose output.
This has no effect on the verbose output that goes into the logs.
Signed-off-by: Douglas Royds
---
meta/classes/cmake.bbclass | 1 -
1 file changed, 1 deletion(-)
diff --git a/meta/classes/cmake.bbclass b/meta/classes/cmake.bbclass
index fa7f68c99b..e16630434e 100644
--- a/meta/classes/cmake.bbclass
+++ b/meta/classes/cmake.bbclass
@@ -164,7 +164,6 @@ cmake_do_configure() {
-DCMAKE_INSTALL_DATAROOTDIR:PATH=${@os.path.relpath(d.getVar('datadir'),
d.getVar('prefix'))} \
-DCMAKE_INSTALL_SO_NO_EXE=0 \
-DCMAKE_TOOLCHAIN_FILE=${WORKDIR}/toolchain.cmake \
- -DCMAKE_VERBOSE_MAKEFILE=1 \
-DCMAKE_NO_SYSTEM_FROM_IMPORTED=1 \
${EXTRA_OECMAKE} \
-Wno-dev
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH] icecc: Slightly more verbose log messages in the success case
The path containing the symlinks to icecc itself.
The path to the build env tarball.
Signed-off-by: Douglas Royds
---
meta/classes/icecc.bbclass | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/meta/classes/icecc.bbclass b/meta/classes/icecc.bbclass
index aebcc44667..6d003dccef 100644
--- a/meta/classes/icecc.bbclass
+++ b/meta/classes/icecc.bbclass
@@ -405,7 +405,8 @@ set_icecc_env() {
export ICECC_VERSION ICECC_CC ICECC_CXX
export PATH="$ICE_PATH:$PATH"
-bbnote "Using icecc"
+bbnote "Using icecc path: $ICE_PATH"
+bbnote "Using icecc tarball: $ICECC_VERSION"
}
do_configure_prepend() {
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH] libpam: libpamc is licensed under its own BSD-style licence
Or alternatively GPL, the same as the top-level Linux-PAM COPYING.
Signed-off-by: Douglas Royds
---
meta/recipes-extended/pam/libpam_1.3.0.bb | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/meta/recipes-extended/pam/libpam_1.3.0.bb
b/meta/recipes-extended/pam/libpam_1.3.0.bb
index 5c45460c3a..c124e3bb02 100644
--- a/meta/recipes-extended/pam/libpam_1.3.0.bb
+++ b/meta/recipes-extended/pam/libpam_1.3.0.bb
@@ -7,7 +7,9 @@ SECTION = "base"
# /etc/pam.d comes from Debian libpam-runtime in 2009-11 (at that time
# libpam-runtime-1.0.1 is GPLv2+), by openembedded
LICENSE = "GPLv2+ | BSD"
-LIC_FILES_CHKSUM = "file://COPYING;md5=7eb5c1bf854e8881005d673599ee74d3"
+LIC_FILES_CHKSUM = "file://COPYING;md5=7eb5c1bf854e8881005d673599ee74d3 \
+
file://libpamc/License;md5=a4da476a14c093fdc73be3c3c9ba8fb3 \
+"
SRC_URI = "http://linux-pam.org/library/Linux-PAM-${PV}.tar.bz2 \
file://99_pam \
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] cmake.bbclass: make cleaning of build directory on configure optional
On 18/01/19 12:59 PM, Randy MacLeod wrote:
On 1/16/19 7:38 PM, Douglas Royds wrote:
On 17/01/19 12:01 PM, Richard Purdie wrote:
On Thu, 2019-01-17 at 11:51 +1300, Douglas Royds wrote:
From: Sam Nobs
The do_configure task is sensitive to changes in recipes and
configuration files.
This can be time consuming because cmake.bbclass deletes the ${B}
directory
at the beginning of the do_configure task. CMake figures
out what to do when it's run again, so unless your cmake files are
buggy, there shouldn't
be any need to erase the build directory and start afresh.
This change allows you to turn this behaviour off, either globally or
on a per-recipe basis, e.g.
OECMAKE_CLEAN_BUILDDIR_ON_CONFIGURE_pn-yourrecipe = ""
Signed-off-by: Sam Nobs
---
meta/classes/cmake.bbclass | 23 ++-
1 file changed, 18 insertions(+), 5 deletions(-)
This may give a speed boost for some cases but it does come at a
potential risk of determinism issues.
I appreciate you've made it configurable but that actually further
concerns me as we now have two code paths which potentially both have
to be tested.
Is this change really worth it?
Cheers,
Richard
For us, the benefit is beyond question - on our proprietary C++
component, the compile from scratch after a configure is punishing,
10s of minutes per MACHINE.
We are counting on binary reproducibility to flush out any
determinism issues with the CMake code, that is, we will be comparing
our incremental builds with our clean builds. Note also that we are
deleting all the generated CMake artefacts, but leaving the object
files behind. CMake will force a complete rebuild should there be any
change in the CFLAGS and friends.
Douglas,
What workflow are you trying to optimize?
If you're strictly changing existing source files
then you shouldn't need to run configure again so I'm confused.
Also, have you tried using ccache as per:
d014c8c11f cmake.bbclass: Make it work with ccache
and if so why doesn't that work for you?
Thanks,
../Randy
We are not only changing existing source files, we routinely add or
rename files or entire directories. This is a current product, very much
under active development. You're absolutely right though, we should
rarely need to reconfigure, indeed a compile (normal behaviour under
externalsrc) will cause CMake to notice (under the bonnet) that it needs
to rebuild_cache, which is all we need.
The problem is that reconfiguration happens quite frequently for other
reasons, notably changes in any library that our proprietary components
depend on (some of them also proprietary), or non-whitelisted changes in
OE variables or functions.
We use icecc, which gives similar performance to ccache, it's quite
impressive. I cheerfully concede that we could use ccache to reduce the
load on our icecc network, and for a further marginal improvement. There
is however a great deal of linking to be done, and regrettably our
non-trivial code base might have less than ideal build dependencies
within it (within the cmake code), leading to less than perfect
parallelism in compilation and linking ... Unnecessary rebuilding from
scratch imposes an unwelcome burden on our developers, and quite some
muttering in the ranks. I wish to feel safe making my cup of coffee.
None of this is a problem if we don't delete object files at
configure-time. I see no benefit from doing so. Our proposed patch
deletes only CMake artefacts, which should be adequate in all cases.
Hope this clarifies things,
Douglas.
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] cmake.bbclass: make cleaning of build directory on configure optional
On 17/01/19 12:01 PM, Richard Purdie wrote:
On Thu, 2019-01-17 at 11:51 +1300, Douglas Royds wrote:
From: Sam Nobs
The do_configure task is sensitive to changes in recipes and
configuration files.
This can be time consuming because cmake.bbclass deletes the ${B}
directory
at the beginning of the do_configure task. CMake figures
out what to do when it's run again, so unless your cmake files are
buggy, there shouldn't
be any need to erase the build directory and start afresh.
This change allows you to turn this behaviour off, either globally or
on a per-recipe basis, e.g.
OECMAKE_CLEAN_BUILDDIR_ON_CONFIGURE_pn-yourrecipe = ""
Signed-off-by: Sam Nobs
---
meta/classes/cmake.bbclass | 23 ++-
1 file changed, 18 insertions(+), 5 deletions(-)
This may give a speed boost for some cases but it does come at a
potential risk of determinism issues.
I appreciate you've made it configurable but that actually further
concerns me as we now have two code paths which potentially both have
to be tested.
Is this change really worth it?
Cheers,
Richard
For us, the benefit is beyond question - on our proprietary C++
component, the compile from scratch after a configure is punishing, 10s
of minutes per MACHINE.
We are counting on binary reproducibility to flush out any determinism
issues with the CMake code, that is, we will be comparing our
incremental builds with our clean builds. Note also that we are deleting
all the generated CMake artefacts, but leaving the object files behind.
CMake will force a complete rebuild should there be any change in the
CFLAGS and friends.
diff --git a/meta/classes/cmake.bbclass b/meta/classes/cmake.bbclass
index fa7f68c99b..b9ec68ac9a 100644
--- a/meta/classes/cmake.bbclass
+++ b/meta/classes/cmake.bbclass
@@ -63,6 +63,11 @@ EXTRA_OECMAKE_BUILD_prepend_task-install =
"${PARALLEL_MAKEINST}"
OECMAKE_TARGET_COMPILE ?= "all"
OECMAKE_TARGET_INSTALL ?= "install"
+# Clean the build directory before configuring.
+# Some recipes don't want/need this as cmake is perfectly
+# capable of figuring out what to do, so we allow it to be turned
off
+OECMAKE_CLEAN_BUILDDIR_ON_CONFIGURE ??= "true"
+
# CMake expects target architectures in the format of uname(2),
# which do not always match TARGET_ARCH, so all the necessary
# conversions should happen here.
@@ -133,12 +138,20 @@ cmake_do_configure() {
bbnote "cmake.bbclass no longer uses
OECMAKE_BUILDPATH. The default behaviour is now out-of-tree builds
with B=WORKDIR/build."
fi
- if [ "${S}" != "${B}" ]; then
- rm -rf ${B}
- mkdir -p ${B}
- cd ${B}
+ if [ ${@oe.types.boolean('${OECMAKE_CLEAN_BUILDDIR_ON_CONFIGURE
}')} = True ]; then
+ if [ "${S}" != "${B}" ]; then
+ rm -rf ${B}
+ mkdir -p ${B}
+ cd ${B}
+ else
+ # For an in-tree build, delete all object files
and CMake artefacts
+ find ${B} -name CMakeFiles -or -name Makefile
-or -name cmake_install.cmake -or -name CMakeCache.txt -delete
+ fi
else
- find ${B} -name CMakeFiles -or -name Makefile -or -name
cmake_install.cmake -or -name CMakeCache.txt -delete
+ # Selectively delete CMake artefacts to ensure that -c
configure has a consistent result
+ rm -f CMakeCache.txt
+ cmake_version=$(cmake --version | head -1 | sed
's#^[^0-9]*\([0-9.]*\).*$#\1#' )
+ rm -fr CMakeFiles/$cmake_version/
fi
# Just like autotools cmake can use a site file to cache result
that need generated binaries to run
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH] cmake.bbclass: make cleaning of build directory on configure optional
From: Sam Nobs
The do_configure task is sensitive to changes in recipes and configuration
files.
This can be time consuming because cmake.bbclass deletes the ${B} directory
at the beginning of the do_configure task. CMake figures
out what to do when it's run again, so unless your cmake files are buggy, there
shouldn't
be any need to erase the build directory and start afresh.
This change allows you to turn this behaviour off, either globally or on a
per-recipe basis, e.g.
OECMAKE_CLEAN_BUILDDIR_ON_CONFIGURE_pn-yourrecipe = ""
Signed-off-by: Sam Nobs
---
meta/classes/cmake.bbclass | 23 ++-
1 file changed, 18 insertions(+), 5 deletions(-)
diff --git a/meta/classes/cmake.bbclass b/meta/classes/cmake.bbclass
index fa7f68c99b..b9ec68ac9a 100644
--- a/meta/classes/cmake.bbclass
+++ b/meta/classes/cmake.bbclass
@@ -63,6 +63,11 @@ EXTRA_OECMAKE_BUILD_prepend_task-install =
"${PARALLEL_MAKEINST} "
OECMAKE_TARGET_COMPILE ?= "all"
OECMAKE_TARGET_INSTALL ?= "install"
+# Clean the build directory before configuring.
+# Some recipes don't want/need this as cmake is perfectly
+# capable of figuring out what to do, so we allow it to be turned off
+OECMAKE_CLEAN_BUILDDIR_ON_CONFIGURE ??= "true"
+
# CMake expects target architectures in the format of uname(2),
# which do not always match TARGET_ARCH, so all the necessary
# conversions should happen here.
@@ -133,12 +138,20 @@ cmake_do_configure() {
bbnote "cmake.bbclass no longer uses OECMAKE_BUILDPATH. The
default behaviour is now out-of-tree builds with B=WORKDIR/build."
fi
- if [ "${S}" != "${B}" ]; then
- rm -rf ${B}
- mkdir -p ${B}
- cd ${B}
+ if [ ${@oe.types.boolean('${OECMAKE_CLEAN_BUILDDIR_ON_CONFIGURE}')} =
True ]; then
+ if [ "${S}" != "${B}" ]; then
+ rm -rf ${B}
+ mkdir -p ${B}
+ cd ${B}
+ else
+ # For an in-tree build, delete all object files and
CMake artefacts
+ find ${B} -name CMakeFiles -or -name Makefile -or -name
cmake_install.cmake -or -name CMakeCache.txt -delete
+ fi
else
- find ${B} -name CMakeFiles -or -name Makefile -or -name
cmake_install.cmake -or -name CMakeCache.txt -delete
+ # Selectively delete CMake artefacts to ensure that -c
configure has a consistent result
+ rm -f CMakeCache.txt
+ cmake_version=$(cmake --version | head -1 | sed
's#^[^0-9]*\([0-9.]*\).*$#\1#' )
+ rm -fr CMakeFiles/$cmake_version/
fi
# Just like autotools cmake can use a site file to cache result that
need generated binaries to run
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] patch: reproducibility: Fix host umask leakage
I haven't sent this as a "v2", because of the change of subject line.
On 21/12/18 12:10 PM, Douglas Royds wrote:
Some patch files create entirely new files, so their permissions are subject to
the host umask. If such a file is later installed into a package with no change
in permissions, it breaks the reproducibility of the package.
This was observed on libpam, for instance: The patch file
pam-security-abstract-securetty-handling.patch creates a new file
(tty_secure.c). This file is later copied into the -dbg package with no change
in permissions.
Signed-off-by: Douglas Royds
---
meta/classes/patch.bbclass | 1 +
1 file changed, 1 insertion(+)
diff --git a/meta/classes/patch.bbclass b/meta/classes/patch.bbclass
index 3e0a181821..cd241f1c84 100644
--- a/meta/classes/patch.bbclass
+++ b/meta/classes/patch.bbclass
@@ -153,6 +153,7 @@ python patch_do_patch() {
patch_do_patch[vardepsexclude] = "PATCHRESOLVE"
addtask patch after do_unpack
+do_patch[umask] = "022"
do_patch[dirs] = "${WORKDIR}"
do_patch[depends] = "${PATCHDEPENDENCY}"
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH] patch: reproducibility: Fix host umask leakage
Some patch files create entirely new files, so their permissions are subject to
the host umask. If such a file is later installed into a package with no change
in permissions, it breaks the reproducibility of the package.
This was observed on libpam, for instance: The patch file
pam-security-abstract-securetty-handling.patch creates a new file
(tty_secure.c). This file is later copied into the -dbg package with no change
in permissions.
Signed-off-by: Douglas Royds
---
meta/classes/patch.bbclass | 1 +
1 file changed, 1 insertion(+)
diff --git a/meta/classes/patch.bbclass b/meta/classes/patch.bbclass
index 3e0a181821..cd241f1c84 100644
--- a/meta/classes/patch.bbclass
+++ b/meta/classes/patch.bbclass
@@ -153,6 +153,7 @@ python patch_do_patch() {
patch_do_patch[vardepsexclude] = "PATCHRESOLVE"
addtask patch after do_unpack
+do_patch[umask] = "022"
do_patch[dirs] = "${WORKDIR}"
do_patch[depends] = "${PATCHDEPENDENCY}"
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] libpam: reproducibility: Fix host umask leakage at patch-time
On 21/12/18 9:37 AM, Burton, Ross wrote: On Thu, 20 Dec 2018 at 20:26, Douglas Royds wrote: libpam is unusual in having a patch that creates an entirely new file (tty_secure.c). If that patch eventually goes, then this umask setting can go as well. I haven't come across any other cases of a patch creating a file, but if there are, then we could certainly move this setting. Perhaps we should wait until others emerge. I definitely wouldn't want to make the assumption that libpam is the only recipe of all time that will create a file. No indeed. There are very likely others already, libpam is just the one I happened to stumble across. I'll shift the setting. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] libpam: reproducibility: Fix host umask leakage at patch-time
libpam is unusual in having a patch that creates an entirely new file
(tty_secure.c). If that patch eventually goes, then this umask setting
can go as well.
I haven't come across any other cases of a patch creating a file, but if
there are, then we could certainly move this setting. Perhaps we should
wait until others emerge.
On 21/12/18 2:04 AM, Burton, Ross wrote:
Why is this libpam specific and not a general issue that the umask
needs to be set in patch.bbclass?
Ross
On Thu, 20 Dec 2018 at 05:41, Douglas Royds wrote:
The patch file pam-security-abstract-securetty-handling.patch creates a new
file (tty_secure.c) at patch-time, so its permissions are subject to the host
umask. This file is later copied into the -dbg package with no change in
permissions.
Signed-off-by: Douglas Royds
---
meta/recipes-extended/pam/libpam_1.3.0.bb | 3 +++
1 file changed, 3 insertions(+)
diff --git a/meta/recipes-extended/pam/libpam_1.3.0.bb
b/meta/recipes-extended/pam/libpam_1.3.0.bb
index 3aec2cdb4c..292b570e11 100644
--- a/meta/recipes-extended/pam/libpam_1.3.0.bb
+++ b/meta/recipes-extended/pam/libpam_1.3.0.bb
@@ -49,6 +49,9 @@ S = "${WORKDIR}/Linux-PAM-${PV}"
inherit autotools gettext pkgconfig
+# We create a new file (tty_secure.c) from
pam-security-abstract-securetty-handling.patch
+do_patch[umask] = "022"
+
PACKAGECONFIG[audit] = "--enable-audit,--disable-audit,audit,"
PACKAGES += "${PN}-runtime ${PN}-xtests"
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH] libpam: reproducibility: Fix host umask leakage at patch-time
The patch file pam-security-abstract-securetty-handling.patch creates a new
file (tty_secure.c) at patch-time, so its permissions are subject to the host
umask. This file is later copied into the -dbg package with no change in
permissions.
Signed-off-by: Douglas Royds
---
meta/recipes-extended/pam/libpam_1.3.0.bb | 3 +++
1 file changed, 3 insertions(+)
diff --git a/meta/recipes-extended/pam/libpam_1.3.0.bb
b/meta/recipes-extended/pam/libpam_1.3.0.bb
index 3aec2cdb4c..292b570e11 100644
--- a/meta/recipes-extended/pam/libpam_1.3.0.bb
+++ b/meta/recipes-extended/pam/libpam_1.3.0.bb
@@ -49,6 +49,9 @@ S = "${WORKDIR}/Linux-PAM-${PV}"
inherit autotools gettext pkgconfig
+# We create a new file (tty_secure.c) from
pam-security-abstract-securetty-handling.patch
+do_patch[umask] = "022"
+
PACKAGECONFIG[audit] = "--enable-audit,--disable-audit,audit,"
PACKAGES += "${PN}-runtime ${PN}-xtests"
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 5/5] icecc: patchelf is needed by icecc-create-env
Although we could potentially build patchelf and all its dependencies,
they would all have to be blacklisted to avoid trying to build them with
icecc. We use the host patchelf instead.
Signed-off-by: Douglas Royds
---
meta/classes/icecc.bbclass | 6 +-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/meta/classes/icecc.bbclass b/meta/classes/icecc.bbclass
index 49597175ca..7d94525d31 100644
--- a/meta/classes/icecc.bbclass
+++ b/meta/classes/icecc.bbclass
@@ -38,7 +38,7 @@ BB_HASHBASE_WHITELIST += "ICECC_PARALLEL_MAKE ICECC_DISABLED
ICECC_USER_PACKAGE_
ICECC_ENV_EXEC ?= "${STAGING_BINDIR_NATIVE}/icecc-create-env"
-HOSTTOOLS_NONFATAL += "icecc"
+HOSTTOOLS_NONFATAL += "icecc patchelf"
# This version can be incremented when changes are made to the environment that
# invalidate the version on the compile nodes. Changing it will cause a new
@@ -326,6 +326,10 @@ set_icecc_env() {
bbwarn "Cannot use icecc: icecc binary not found"
return
fi
+if [ -z "$(which patchelf patchelf-uninative)" ]; then
+bbwarn "Cannot use icecc: patchelf not found"
+return
+fi
# Create symlinks to icecc in the recipe-sysroot directory
mkdir -p ${ICE_PATH}
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 2/5] icecc: Trivial simplification
Signed-off-by: Douglas Royds
---
meta/classes/icecc.bbclass | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/meta/classes/icecc.bbclass b/meta/classes/icecc.bbclass
index b251e9b5aa..d8a7894887 100644
--- a/meta/classes/icecc.bbclass
+++ b/meta/classes/icecc.bbclass
@@ -305,7 +305,7 @@ def icecc_get_and_check_tool(bb, d, tool):
if t:
link_path = icecc_get_tool_link(t, d)
if link_path == get_icecc(d):
-bb.error("%s is a symlink to %s in PATH and this prevents icecc
from working" % (t, get_icecc(d)))
+bb.error("%s is a symlink to %s in PATH and this prevents icecc
from working" % (t, link_path))
return ""
else:
return t
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 4/5] icecc: Don't generate recipe-sysroot symlinks at recipe-parsing time
The python function icecc_path() was being invoked inline by set_icecc_env(),
meaning that it was being invoked at recipe-parsing time.
As a side-effect, icecc_path() was creating the recipe-sysroot directory and
symlinking icecc into it. Because this was done at parsing time (rather than
configure time), we were generating otherwise-empty WORKDIRs for *all* parsed
recipes, and for all virtual classes (-native, -nativesdk).
In my build, this generated more than 800 of these otherwise-empty WORKDIRs.
I have simplified icecc_path() to return only the intended path to the icecc
symlinks in the recipe-sysroot, with no side-effect.
We then create the directory and the icecc symlinks at configure time.
Because get_cross_kernel_cc() is still invoked at parse-time,
it needs a guard-clause for the non-kernel case.
We are now finding the host icecc at do_configure time,
so icecc needs to be in the HOSTTOOLS. I have made this non-fatal,
so that we can still inherit icecc without icecc installed.
Signed-off-by: Douglas Royds
---
meta/classes/icecc.bbclass | 62 +++---
1 file changed, 25 insertions(+), 37 deletions(-)
diff --git a/meta/classes/icecc.bbclass b/meta/classes/icecc.bbclass
index 2171e11db0..49597175ca 100644
--- a/meta/classes/icecc.bbclass
+++ b/meta/classes/icecc.bbclass
@@ -38,6 +38,8 @@ BB_HASHBASE_WHITELIST += "ICECC_PARALLEL_MAKE ICECC_DISABLED
ICECC_USER_PACKAGE_
ICECC_ENV_EXEC ?= "${STAGING_BINDIR_NATIVE}/icecc-create-env"
+HOSTTOOLS_NONFATAL += "icecc"
+
# This version can be incremented when changes are made to the environment that
# invalidate the version on the compile nodes. Changing it will cause a new
# environment to be created.
@@ -98,9 +100,11 @@ DEPENDS_prepend += "${@icecc_dep_prepend(d)} "
get_cross_kernel_cc[vardepsexclude] += "KERNEL_CC"
def get_cross_kernel_cc(bb,d):
-kernel_cc = d.getVar('KERNEL_CC')
+if not icecc_is_kernel(bb, d):
+return None
# evaluate the expression by the shell if necessary
+kernel_cc = d.getVar('KERNEL_CC')
if '`' in kernel_cc or '$(' in kernel_cc:
import subprocess
kernel_cc = subprocess.check_output("echo %s" % kernel_cc,
shell=True).decode("utf-8")[:-1]
@@ -113,38 +117,6 @@ def get_cross_kernel_cc(bb,d):
def get_icecc(d):
return d.getVar('ICECC_PATH') or bb.utils.which(os.getenv("PATH"), "icecc")
-def create_path(compilers, bb, d):
-"""
-Create Symlinks for the icecc in the staging directory
-"""
-staging = os.path.join(d.expand('${STAGING_BINDIR}'), "ice")
-if icecc_is_kernel(bb, d):
-staging += "-kernel"
-
-#check if the icecc path is set by the user
-icecc = get_icecc(d)
-
-# Create the dir if necessary
-try:
-os.stat(staging)
-except:
-try:
-os.makedirs(staging)
-except:
-pass
-
-for compiler in compilers:
-gcc_path = os.path.join(staging, compiler)
-try:
-os.stat(gcc_path)
-except:
-try:
-os.symlink(icecc, gcc_path)
-except:
-pass
-
-return staging
-
def use_icecc(bb,d):
if d.getVar('ICECC_DISABLED') == "1":
# don't even try it, when explicitly disabled
@@ -248,12 +220,11 @@ def icecc_path(bb,d):
# don't create unnecessary directories when icecc is disabled
return
+staging = os.path.join(d.expand('${STAGING_BINDIR}'), "ice")
if icecc_is_kernel(bb, d):
-return create_path( [get_cross_kernel_cc(bb,d), ], bb, d)
+staging += "-kernel"
-else:
-prefix = d.expand('${HOST_PREFIX}')
-return create_path( [prefix+"gcc", prefix+"g++"], bb, d)
+return staging
def icecc_get_external_tool(bb, d, tool):
external_toolchain_bindir =
d.expand('${EXTERNAL_TOOLCHAIN}${bindir_cross}')
@@ -350,6 +321,23 @@ set_icecc_env() {
return
fi
+ICECC_BIN="${@get_icecc(d)}"
+if [ -z "${ICECC_BIN}" ]; then
+bbwarn "Cannot use icecc: icecc binary not found"
+return
+fi
+
+# Create symlinks to icecc in the recipe-sysroot directory
+mkdir -p ${ICE_PATH}
+if [ -n "${KERNEL_CC}" ]; then
+compilers="${@get_cross_kernel_cc(bb,d)}"
+else
+compilers="${HOST_PREFIX}gcc ${HOST_PREFIX}g++"
+fi
+for compiler in $compilers; do
+ln -sf ${ICECC_BIN} ${ICE_PATH}/$compiler
+done
+
ICECC_CC="${@icecc_get_and_check_tool(bb, d, "gcc")}"
ICECC_CXX="${@icecc_get_and_check_tool(bb, d, "g++")}"
# cannot use icecc_get_and_check_tool here because it assumes as without
target_sys prefix
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 0/5] icecc: Don't generate recipe-sysroot symlinks at recipe-parsing time
The python function icecc_path() was being invoked inline by set_icecc_env(), meaning that it was being invoked at recipe-parsing time. As a side-effect, icecc_path() was creating the recipe-sysroot directory and symlinking icecc into it. Because this was done at parsing time (rather than configure time), we were generating otherwise-empty WORKDIRs for *all* parsed recipes, and for all virtual classes (-native, -nativesdk). In my build, this generated more than 800 of these otherwise-empty WORKDIRs. Because we are now finding the host icecc at do_configure time, icecc needs to be in the HOSTTOOLS. I have made this non-fatal, so that we can still inherit icecc without icecc installed. Likewise, although we could potentially build patchelf and all its dependencies, they would all have to be blacklisted to avoid trying to build them with icecc. We use the host patchelf instead. You will only receive (the flood of) warnings about the absence of build host icecc and patchelf if you both inherit icecc and have ICECC_DISABLED = "". I have borrowed the patchelf-uninative bit from OE's modified icecc-create-env script, but I haven't tested it on a uninative build. If two bitbake processes try to generate the ICECC_VERSION tarball at the same time, the thread that fails to get the lock will wait 30 sec for the first to finish. A syntax error meant that this was not happening and in particular, if tarball generation failed (eg. for lack of patchelf), it did so silently. Douglas Royds (5): icecc: readlink -f on the recipe-sysroot gcc/g++ icecc: Trivial simplification icecc: Syntax error meant that we weren't waiting for tarball generation icecc: Don't generate recipe-sysroot symlinks at recipe-parsing time icecc: patchelf is needed by icecc-create-env meta/classes/icecc.bbclass | 72 +- 1 file changed, 32 insertions(+), 40 deletions(-) -- 2.17.1 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 1/5] icecc: readlink -f on the recipe-sysroot gcc/g++
We were accidentally doing a readlink -f on simply 'gcc', for instance
Signed-off-by: Douglas Royds
---
meta/classes/icecc.bbclass | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/meta/classes/icecc.bbclass b/meta/classes/icecc.bbclass
index 2b189232cb..b251e9b5aa 100644
--- a/meta/classes/icecc.bbclass
+++ b/meta/classes/icecc.bbclass
@@ -303,7 +303,7 @@ def icecc_get_and_check_tool(bb, d, tool):
# compiler environment package.
t = icecc_get_tool(bb, d, tool)
if t:
-link_path = icecc_get_tool_link(tool, d)
+link_path = icecc_get_tool_link(t, d)
if link_path == get_icecc(d):
bb.error("%s is a symlink to %s in PATH and this prevents icecc
from working" % (t, get_icecc(d)))
return ""
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 3/5] icecc: Syntax error meant that we weren't waiting for tarball generation
If two bitbake processes try to generate the ICECC_VERSION tarball at the same
time, the thread that fails to get the lock will wait 30 sec for the first to
finish. A syntax error meant that this was not happening, and in particular, if
tarball generation failed (eg. for lack of patchelf), it did so silently.
Signed-off-by: Douglas Royds
---
meta/classes/icecc.bbclass | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/meta/classes/icecc.bbclass b/meta/classes/icecc.bbclass
index d8a7894887..2171e11db0 100644
--- a/meta/classes/icecc.bbclass
+++ b/meta/classes/icecc.bbclass
@@ -387,7 +387,7 @@ set_icecc_env() {
${ICECC_ENV_EXEC} ${ICECC_ENV_DEBUG} "${ICECC_CC}" "${ICECC_CXX}"
"${ICECC_AS}" "${ICECC_VERSION}"
then
touch "${ICECC_VERSION}.done"
-elif [ ! wait_for_file "${ICECC_VERSION}.done" 30 ]
+elif ! wait_for_file "${ICECC_VERSION}.done" 30
then
# locking failed so wait for ${ICECC_VERSION}.done to appear
bbwarn "Timeout waiting for ${ICECC_VERSION}.done"
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 1/2] acl: trivial: whitespace only
Yes, brilliant. Once he has those upgrades sorted out, these ones will
be completely redundant, I believe.
On 15/12/18 3:54 AM, Burton, Ross wrote:
Enrico just submitted some upgrades to acl and attr which touch the
recipes quite dramatically, could you rebase all these patches on top
of that? Several will just disappear, for example ptest support was
removed.
Ross
On Fri, 14 Dec 2018 at 01:20, Douglas Royds wrote:
Signed-off-by: Douglas Royds
---
meta/recipes-support/attr/acl_2.2.52.bb | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/meta/recipes-support/attr/acl_2.2.52.bb
b/meta/recipes-support/attr/acl_2.2.52.bb
index 6bc77d868d..7f5883a5d7 100644
--- a/meta/recipes-support/attr/acl_2.2.52.bb
+++ b/meta/recipes-support/attr/acl_2.2.52.bb
@@ -47,6 +47,7 @@ do_install_ptest() {
install -m 644 ${S}/include/builddefs ${S}/include/buildmacros
${S}/include/buildrules ${D}${PTEST_PATH}/include/
}
-RDEPENDS_${PN}-ptest = "acl bash coreutils perl perl-module-filehandle
perl-module-getopt-std perl-module-posix shadow"
+RDEPENDS_${PN}-ptest = "acl bash coreutils \
+perl perl-module-filehandle perl-module-getopt-std
perl-module-posix shadow"
BBCLASSEXTEND = "native nativesdk"
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 1/2] acl: trivial: whitespace only
Signed-off-by: Douglas Royds
---
meta/recipes-support/attr/acl_2.2.52.bb | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/meta/recipes-support/attr/acl_2.2.52.bb
b/meta/recipes-support/attr/acl_2.2.52.bb
index 6bc77d868d..7f5883a5d7 100644
--- a/meta/recipes-support/attr/acl_2.2.52.bb
+++ b/meta/recipes-support/attr/acl_2.2.52.bb
@@ -47,6 +47,7 @@ do_install_ptest() {
install -m 644 ${S}/include/builddefs ${S}/include/buildmacros
${S}/include/buildrules ${D}${PTEST_PATH}/include/
}
-RDEPENDS_${PN}-ptest = "acl bash coreutils perl perl-module-filehandle
perl-module-getopt-std perl-module-posix shadow"
+RDEPENDS_${PN}-ptest = "acl bash coreutils \
+perl perl-module-filehandle perl-module-getopt-std
perl-module-posix shadow"
BBCLASSEXTEND = "native nativesdk"
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 2/2] acl ptest: Also RDEPENDS on e2fsprogs-mke2fs
The tests use mkfs.ext3
Signed-off-by: Douglas Royds
---
meta/recipes-support/attr/acl_2.2.52.bb | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/meta/recipes-support/attr/acl_2.2.52.bb
b/meta/recipes-support/attr/acl_2.2.52.bb
index 7f5883a5d7..4c30da8182 100644
--- a/meta/recipes-support/attr/acl_2.2.52.bb
+++ b/meta/recipes-support/attr/acl_2.2.52.bb
@@ -47,7 +47,7 @@ do_install_ptest() {
install -m 644 ${S}/include/builddefs ${S}/include/buildmacros
${S}/include/buildrules ${D}${PTEST_PATH}/include/
}
-RDEPENDS_${PN}-ptest = "acl bash coreutils \
+RDEPENDS_${PN}-ptest = "acl bash coreutils e2fsprogs-mke2fs \
perl perl-module-filehandle perl-module-getopt-std
perl-module-posix shadow"
BBCLASSEXTEND = "native nativesdk"
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 2/3] attr: Move unique-to-attr run-ptest into attr/ subdir
There is a separate, wildly different run-ptest for acl.
This is not a shared script.
Signed-off-by: Douglas Royds
---
meta/recipes-support/attr/{files => attr}/run-ptest | 0
1 file changed, 0 insertions(+), 0 deletions(-)
rename meta/recipes-support/attr/{files => attr}/run-ptest (100%)
diff --git a/meta/recipes-support/attr/files/run-ptest
b/meta/recipes-support/attr/attr/run-ptest
similarity index 100%
rename from meta/recipes-support/attr/files/run-ptest
rename to meta/recipes-support/attr/attr/run-ptest
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 1/3] attr: Move attr-only patches into the attr/ subdir
These are not shared patches between attr and acl.
Signed-off-by: Douglas Royds
---
.../Remove-the-attr.5-man-page-moved-to-man-pages.patch | 0
.../attr/{files => attr}/Remove-the-section-2-man-pages.patch | 0
2 files changed, 0 insertions(+), 0 deletions(-)
rename meta/recipes-support/attr/{files =>
attr}/Remove-the-attr.5-man-page-moved-to-man-pages.patch (100%)
rename meta/recipes-support/attr/{files =>
attr}/Remove-the-section-2-man-pages.patch (100%)
diff --git
a/meta/recipes-support/attr/files/Remove-the-attr.5-man-page-moved-to-man-pages.patch
b/meta/recipes-support/attr/attr/Remove-the-attr.5-man-page-moved-to-man-pages.patch
similarity index 100%
rename from
meta/recipes-support/attr/files/Remove-the-attr.5-man-page-moved-to-man-pages.patch
rename to
meta/recipes-support/attr/attr/Remove-the-attr.5-man-page-moved-to-man-pages.patch
diff --git
a/meta/recipes-support/attr/files/Remove-the-section-2-man-pages.patch
b/meta/recipes-support/attr/attr/Remove-the-section-2-man-pages.patch
similarity index 100%
rename from meta/recipes-support/attr/files/Remove-the-section-2-man-pages.patch
rename to meta/recipes-support/attr/attr/Remove-the-section-2-man-pages.patch
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 3/3] acl: reproducibility: copy configure.ac into ${S} at patch-time
Previously, we were unpacking our local configure.ac file directly into ${S} at
unpack time, which meant that the SOURCE_DATE_EPOCH was being set to the local
timestamp of that file.
Signed-off-by: Douglas Royds
---
meta/recipes-support/attr/acl_2.2.52.bb | 7 ++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/meta/recipes-support/attr/acl_2.2.52.bb
b/meta/recipes-support/attr/acl_2.2.52.bb
index 6bc77d868d..b10767776d 100644
--- a/meta/recipes-support/attr/acl_2.2.52.bb
+++ b/meta/recipes-support/attr/acl_2.2.52.bb
@@ -11,7 +11,7 @@ LIC_FILES_CHKSUM =
"file://doc/COPYING;md5=c781d70ed2b4d48995b790403217a249 \
DEPENDS = "attr"
SRC_URI = "${SAVANNAH_GNU_MIRROR}/acl/${BP}.src.tar.gz \
- file://configure.ac;subdir=${BP} \
+ file://configure.ac \
file://run-ptest \
file://acl-fix-the-order-of-expected-output-of-getfacl.patch \
file://test-fix-insufficient-quoting-of.patch \
@@ -25,6 +25,11 @@ SRC_URI[sha256sum] =
"179074bb0580c06c4b4137be4c5a92a701583277967acdb5546043c787
require ea-acl.inc
+copy_configure_ac_file() {
+ cp -v ${WORKDIR}/configure.ac ${S}
+}
+do_patch[prefuncs] += "copy_configure_ac_file"
+
# avoid RPATH hardcode to staging dir
do_configure_append() {
sed -i ${S}/config.status -e
s,^\\\(hardcode_into_libs=\\\).*$,\\1\'no\',
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v4 8/8] attr: Strip build host paths and tool versions from ptest files
Using the new PTEST_BUILD_HOST_FILES mechanism.
Specifically remove reference to build host rpm version, if any.
Signed-off-by: Douglas Royds
---
meta/recipes-support/attr/attr.inc | 9 +++--
1 file changed, 3 insertions(+), 6 deletions(-)
diff --git a/meta/recipes-support/attr/attr.inc
b/meta/recipes-support/attr/attr.inc
index 9fc0f33cce..cfa38a7196 100644
--- a/meta/recipes-support/attr/attr.inc
+++ b/meta/recipes-support/attr/attr.inc
@@ -29,6 +29,9 @@ ALTERNATIVE_PRIORITY = "100"
ALTERNATIVE_${PN} = "setfattr"
ALTERNATIVE_TARGET[setfattr] = "${bindir}/setfattr"
+PTEST_BUILD_HOST_FILES = "builddefs"
+PTEST_BUILD_HOST_PATTERN = "^RPM"
+
do_install_ptest() {
tar -c --exclude=ext test/ | ( cd ${D}${PTEST_PATH} && tar -xf - )
mkdir ${D}${PTEST_PATH}/include
@@ -36,12 +39,6 @@ do_install_ptest() {
do cp ${S}/include/$i ${D}${PTEST_PATH}/include/; \
done
sed -e 's|; @echo|; echo|' -i ${D}${PTEST_PATH}/test/Makefile
-
- # Remove any build host references
- sed -e "s:--sysroot=${STAGING_DIR_TARGET}::g" \
- -e 's:${HOSTTOOLS_DIR}/::g' \
- -e 's:${RECIPE_SYSROOT_NATIVE}::g' \
- -i ${D}${PTEST_PATH}/include/builddefs
}
RDEPENDS_${PN}-ptest = "attr coreutils perl-module-filehandle
perl-module-getopt-std perl-module-posix"
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v4 6/8] acl: Use install instead of cp
Signed-off-by: Douglas Royds
---
meta/recipes-support/attr/acl_2.2.52.bb | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/meta/recipes-support/attr/acl_2.2.52.bb
b/meta/recipes-support/attr/acl_2.2.52.bb
index 8b89de9b42..318308a009 100644
--- a/meta/recipes-support/attr/acl_2.2.52.bb
+++ b/meta/recipes-support/attr/acl_2.2.52.bb
@@ -41,8 +41,8 @@ inherit ptest
do_install_ptest() {
tar -c --exclude=nfs test/ | ( cd ${D}${PTEST_PATH} && tar -xf - )
- mkdir ${D}${PTEST_PATH}/include
- cp ${S}/include/builddefs ${S}/include/buildmacros
${S}/include/buildrules ${D}${PTEST_PATH}/include/
+ install -d ${D}${PTEST_PATH}/include
+ install -m 644 ${S}/include/builddefs ${S}/include/buildmacros
${S}/include/buildrules ${D}${PTEST_PATH}/include/
# Remove any build host references
sed -e "s:--sysroot=${STAGING_DIR_TARGET}::g" \
-e 's:${HOSTTOOLS_DIR}/::g' \
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v4 5/8] strace: Strip build host tool details from installed ptest Makefile
Adopt new PTEST_BUILD_HOST_PATTERN mechanism.
Signed-off-by: Douglas Royds
---
meta/recipes-devtools/strace/strace_4.25.bb | 11 +--
1 file changed, 1 insertion(+), 10 deletions(-)
diff --git a/meta/recipes-devtools/strace/strace_4.25.bb
b/meta/recipes-devtools/strace/strace_4.25.bb
index 117ebfbf72..58f6cf6e30 100644
--- a/meta/recipes-devtools/strace/strace_4.25.bb
+++ b/meta/recipes-devtools/strace/strace_4.25.bb
@@ -32,6 +32,7 @@ EXTRA_OECONF += "--enable-mpers=no"
CFLAGS_append_libc-musl = " -Dsigcontext_struct=sigcontext"
TESTDIR = "tests"
+PTEST_BUILD_HOST_PATTERN =
"^(DEB_CHANGELOGTIME|RPM_CHANGELOGTIME|WARN_CFLAGS_FOR_BUILD)"
do_install_append() {
# We don't ship strace-graph here because it needs perl
@@ -46,16 +47,6 @@ do_install_ptest() {
oe_runmake -C ${TESTDIR} install-ptest BUILDDIR=${B}
DESTDIR=${D}${PTEST_PATH} TESTDIR=${TESTDIR}
install -m 755 ${S}/test-driver ${D}${PTEST_PATH}
install -m 644 ${B}/config.h ${D}${PTEST_PATH}
- sed -i -e '/^src/s/strace.*[1-9]/ptest/' \
- -e 's,--sysroot=${STAGING_DIR_TARGET},,g' \
- -e 's|${DEBUG_PREFIX_MAP}||g' \
- -e 's:${HOSTTOOLS_DIR}/::g' \
- -e 's:${RECIPE_SYSROOT_NATIVE}::g' \
- -e 's:${RECIPE_SYSROOT}::g' \
- -e 's:${BASE_WORKDIR}/${MULTIMACH_TARGET_SYS}::g' \
- -e '/^DEB_CHANGELOGTIME/d' \
- -e '/^RPM_CHANGELOGTIME/d' \
- ${D}/${PTEST_PATH}/${TESTDIR}/Makefile
}
RDEPENDS_${PN}-ptest += "make coreutils grep gawk sed"
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v4 7/8] acl: Strip build host tool details from installed ptest file
Adopt the PTEST_BUILD_HOST_FILES mechanism to strip build host paths from the
installed builddefs file. Also strip the record of the build host rpm version.
Signed-off-by: Douglas Royds
---
meta/recipes-support/attr/acl_2.2.52.bb | 7 ++-
1 file changed, 2 insertions(+), 5 deletions(-)
diff --git a/meta/recipes-support/attr/acl_2.2.52.bb
b/meta/recipes-support/attr/acl_2.2.52.bb
index 318308a009..6bc77d868d 100644
--- a/meta/recipes-support/attr/acl_2.2.52.bb
+++ b/meta/recipes-support/attr/acl_2.2.52.bb
@@ -39,15 +39,12 @@ do_install_append() {
inherit ptest
+PTEST_BUILD_HOST_FILES = "builddefs"
+PTEST_BUILD_HOST_PATTERN = "^RPM"
do_install_ptest() {
tar -c --exclude=nfs test/ | ( cd ${D}${PTEST_PATH} && tar -xf - )
install -d ${D}${PTEST_PATH}/include
install -m 644 ${S}/include/builddefs ${S}/include/buildmacros
${S}/include/buildrules ${D}${PTEST_PATH}/include/
- # Remove any build host references
- sed -e "s:--sysroot=${STAGING_DIR_TARGET}::g" \
- -e 's:${HOSTTOOLS_DIR}/::g' \
- -e 's:${RECIPE_SYSROOT_NATIVE}::g' \
- -i ${D}${PTEST_PATH}/include/builddefs
}
RDEPENDS_${PN}-ptest = "acl bash coreutils perl perl-module-filehandle
perl-module-getopt-std perl-module-posix shadow"
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v4 4/8] openssl: Strip perl version from installed ptest configdata.pm file
Signed-off-by: Douglas Royds
---
meta/recipes-connectivity/openssl/openssl_1.1.1a.bb | 1 +
1 file changed, 1 insertion(+)
diff --git a/meta/recipes-connectivity/openssl/openssl_1.1.1a.bb
b/meta/recipes-connectivity/openssl/openssl_1.1.1a.bb
index deeb865a6f..042e023de5 100644
--- a/meta/recipes-connectivity/openssl/openssl_1.1.1a.bb
+++ b/meta/recipes-connectivity/openssl/openssl_1.1.1a.bb
@@ -157,6 +157,7 @@ do_install_append_class-nativesdk () {
}
PTEST_BUILD_HOST_FILES += "configdata.pm"
+PTEST_BUILD_HOST_PATTERN = "perl_version ="
do_install_ptest () {
# Prune the build tree
rm -f ${B}/fuzz/*.* ${B}/test/*.*
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v4 0/8] ptest: Strip build host paths from any installed Makefile
Only one change from v1: Simplified commit acl: Use install instead of cp Douglas Royds (8): ptest: Reproducibility: Strip build host paths from any installed Makefile ptest: Optionally strip build host tool info from installed files openssl ptest: Strip build host paths from configdata.pm openssl: Strip perl version from installed ptest configdata.pm file strace: Strip build host tool details from installed ptest Makefile acl: Use install instead of cp acl: Strip build host tool details from installed ptest file attr: Strip build host paths and tool versions from ptest files meta/classes/ptest.bbclass | 18 ++ .../openssl/openssl_1.1.1a.bb | 2 ++ meta/recipes-devtools/strace/strace_4.25.bb| 11 +-- meta/recipes-support/attr/acl_2.2.52.bb| 11 --- meta/recipes-support/attr/attr.inc | 9 +++-- 5 files changed, 28 insertions(+), 23 deletions(-) -- 2.17.1 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v4 1/8] ptest: Reproducibility: Strip build host paths from any installed Makefile
A common pattern for ptest is to install a Makefile that provides a make check
target. These generated Makefiles are normally full of build host paths, to
hosttools, and to files and scripts within the source or build tree. They also
commonly include the CFLAGS and CPPFLAGS. None of these build host paths
can possibly work on-target, so stripping them has no effect on the success of
the resulting ptests.
Stripping the HOSTTOOLS_DIR has this effect, for instance:
-MKDIR_P = /home/douglas/workspace/upstream/build/tmp/hosttools/mkdir -p
+MKDIR_P = mkdir -p
The curious WORKDIR/*= pattern avoids non-parseable -fdebug-prefix-map patterns
in the CFLAGS by adding an arbitrary ".", eg:
-fdebug-prefix-map=.=/usr/src/debug/rsyslog/8.37.0-r0
In all other cases, we simply remove the WORKDIR, eg:
CPP = arm-tait-linux-gnueabi-gcc -E --sysroot=recipe-sysroot ...
Signed-off-by: Douglas Royds
---
meta/classes/ptest.bbclass | 13 +
1 file changed, 13 insertions(+)
diff --git a/meta/classes/ptest.bbclass b/meta/classes/ptest.bbclass
index 9d8a79854a..fa518f6d45 100644
--- a/meta/classes/ptest.bbclass
+++ b/meta/classes/ptest.bbclass
@@ -3,6 +3,8 @@ DESCRIPTION_${PN}-ptest ?= "${DESCRIPTION} \
This package contains a test directory ${PTEST_PATH} for package test
purposes."
PTEST_PATH ?= "${libdir}/${BPN}/ptest"
+PTEST_BUILD_HOST_FILES ?= "Makefile"
+
FILES_${PN}-ptest = "${PTEST_PATH}"
SECTION_${PN}-ptest = "devel"
ALLOW_EMPTY_${PN}-ptest = "1"
@@ -45,6 +47,17 @@ do_install_ptest_base() {
fi
do_install_ptest
chown -R root:root ${D}${PTEST_PATH}
+
+# Strip build host paths from any installed Makefile
+for filename in ${PTEST_BUILD_HOST_FILES}; do
+for installed_ptest_file in $(find ${D}${PTEST_PATH} -type f -name
$filename); do
+bbnote "Stripping host paths from: $installed_ptest_file"
+sed -e 's#${HOSTTOOLS_DIR}/*##g' \
+-e 's#${WORKDIR}/*=#.=#g' \
+-e 's#${WORKDIR}/*##g' \
+-i $installed_ptest_file
+done
+done
}
do_configure_ptest_base[dirs] = "${B}"
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v4 3/8] openssl ptest: Strip build host paths from configdata.pm
This file contains CC, CPP, CFLAGS, CXXFLAGS and the like.
Signed-off-by: Douglas Royds
---
meta/recipes-connectivity/openssl/openssl_1.1.1a.bb | 1 +
1 file changed, 1 insertion(+)
diff --git a/meta/recipes-connectivity/openssl/openssl_1.1.1a.bb
b/meta/recipes-connectivity/openssl/openssl_1.1.1a.bb
index 3629f7e6a1..deeb865a6f 100644
--- a/meta/recipes-connectivity/openssl/openssl_1.1.1a.bb
+++ b/meta/recipes-connectivity/openssl/openssl_1.1.1a.bb
@@ -156,6 +156,7 @@ do_install_append_class-nativesdk () {
sed 's|/usr/lib/ssl/|/usr/lib/ssl-1.1/|g' -i
${D}${SDKPATHNATIVE}/environment-setup.d/openssl.sh
}
+PTEST_BUILD_HOST_FILES += "configdata.pm"
do_install_ptest () {
# Prune the build tree
rm -f ${B}/fuzz/*.* ${B}/test/*.*
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v4 2/8] ptest: Optionally strip build host tool info from installed files
Several packages (eg. acl, attr, openssl) install Makefiles or other build host
configuration files for ptest so that they can run `make check`, for instance.
These build host files commonly include paths and versions of build host tools,
(eg. rpm or perl) whose presence and version depends on the build host distro.
Optionally strip lines from installed PTEST_BUILD_HOST_FILES using an
extended regex pattern defined in PTEST_BUILD_HOST_PATTERN. This has no
effect on the on-target ptest.
Signed-off-by: Douglas Royds
---
meta/classes/ptest.bbclass | 5 +
1 file changed, 5 insertions(+)
diff --git a/meta/classes/ptest.bbclass b/meta/classes/ptest.bbclass
index fa518f6d45..97865c9338 100644
--- a/meta/classes/ptest.bbclass
+++ b/meta/classes/ptest.bbclass
@@ -4,6 +4,7 @@ This package contains a test directory ${PTEST_PATH} for
package test purposes."
PTEST_PATH ?= "${libdir}/${BPN}/ptest"
PTEST_BUILD_HOST_FILES ?= "Makefile"
+PTEST_BUILD_HOST_PATTERN ?= ""
FILES_${PN}-ptest = "${PTEST_PATH}"
SECTION_${PN}-ptest = "devel"
@@ -56,6 +57,10 @@ do_install_ptest_base() {
-e 's#${WORKDIR}/*=#.=#g' \
-e 's#${WORKDIR}/*##g' \
-i $installed_ptest_file
+if [ -n "${PTEST_BUILD_HOST_PATTERN}" ]; then
+ sed -E '/${PTEST_BUILD_HOST_PATTERN}/d' \
+ -i $installed_ptest_file
+fi
done
done
}
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 5/8] acl: Use install instead of cp
On 28/11/18 9:04 PM, Peter Kjellerstedt wrote: -Original Message- From: openembedded-core-boun...@lists.openembedded.org On Behalf Of Douglas Royds Sent: den 28 november 2018 05:53 To: openembedded-core@lists.openembedded.org Subject: [OE-core] [PATCH 5/8] acl: Use install instead of cp Also, hard-tabs begone Shell-code in OE-Core is expected to be indented using tabs, so don't do this. (For reference, OpenEmbedded uses four spaces to indent shell-code). Additionally, it makes it harder to see what you actually changed. Fair point. I shouldn't have muddled the two together. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v3 5/8] strace: Strip build host tool details from installed ptest Makefile
Adopt new PTEST_BUILD_HOST_PATTERN mechanism.
Signed-off-by: Douglas Royds
Signed-off-by: Richard Purdie
---
meta/recipes-devtools/strace/strace_4.25.bb | 11 +--
1 file changed, 1 insertion(+), 10 deletions(-)
diff --git a/meta/recipes-devtools/strace/strace_4.25.bb
b/meta/recipes-devtools/strace/strace_4.25.bb
index 117ebfbf72..58f6cf6e30 100644
--- a/meta/recipes-devtools/strace/strace_4.25.bb
+++ b/meta/recipes-devtools/strace/strace_4.25.bb
@@ -32,6 +32,7 @@ EXTRA_OECONF += "--enable-mpers=no"
CFLAGS_append_libc-musl = " -Dsigcontext_struct=sigcontext"
TESTDIR = "tests"
+PTEST_BUILD_HOST_PATTERN =
"^(DEB_CHANGELOGTIME|RPM_CHANGELOGTIME|WARN_CFLAGS_FOR_BUILD)"
do_install_append() {
# We don't ship strace-graph here because it needs perl
@@ -46,16 +47,6 @@ do_install_ptest() {
oe_runmake -C ${TESTDIR} install-ptest BUILDDIR=${B}
DESTDIR=${D}${PTEST_PATH} TESTDIR=${TESTDIR}
install -m 755 ${S}/test-driver ${D}${PTEST_PATH}
install -m 644 ${B}/config.h ${D}${PTEST_PATH}
- sed -i -e '/^src/s/strace.*[1-9]/ptest/' \
- -e 's,--sysroot=${STAGING_DIR_TARGET},,g' \
- -e 's|${DEBUG_PREFIX_MAP}||g' \
- -e 's:${HOSTTOOLS_DIR}/::g' \
- -e 's:${RECIPE_SYSROOT_NATIVE}::g' \
- -e 's:${RECIPE_SYSROOT}::g' \
- -e 's:${BASE_WORKDIR}/${MULTIMACH_TARGET_SYS}::g' \
- -e '/^DEB_CHANGELOGTIME/d' \
- -e '/^RPM_CHANGELOGTIME/d' \
- ${D}/${PTEST_PATH}/${TESTDIR}/Makefile
}
RDEPENDS_${PN}-ptest += "make coreutils grep gawk sed"
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v3 6/8] acl: Use install instead of cp
Signed-off-by: Douglas Royds
---
meta/recipes-support/attr/acl_2.2.52.bb | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/meta/recipes-support/attr/acl_2.2.52.bb
b/meta/recipes-support/attr/acl_2.2.52.bb
index 8b89de9b42..318308a009 100644
--- a/meta/recipes-support/attr/acl_2.2.52.bb
+++ b/meta/recipes-support/attr/acl_2.2.52.bb
@@ -41,8 +41,8 @@ inherit ptest
do_install_ptest() {
tar -c --exclude=nfs test/ | ( cd ${D}${PTEST_PATH} && tar -xf - )
- mkdir ${D}${PTEST_PATH}/include
- cp ${S}/include/builddefs ${S}/include/buildmacros
${S}/include/buildrules ${D}${PTEST_PATH}/include/
+ install -d ${D}${PTEST_PATH}/include
+ install -m 644 ${S}/include/builddefs ${S}/include/buildmacros
${S}/include/buildrules ${D}${PTEST_PATH}/include/
# Remove any build host references
sed -e "s:--sysroot=${STAGING_DIR_TARGET}::g" \
-e 's:${HOSTTOOLS_DIR}/::g' \
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v3 8/8] attr: Strip build host paths and tool versions from ptest files
Using the new PTEST_BUILD_HOST_FILES mechanism.
Specifically remove reference to build host rpm version, if any.
Signed-off-by: Douglas Royds
---
meta/recipes-support/attr/attr.inc | 9 +++--
1 file changed, 3 insertions(+), 6 deletions(-)
diff --git a/meta/recipes-support/attr/attr.inc
b/meta/recipes-support/attr/attr.inc
index 9fc0f33cce..cfa38a7196 100644
--- a/meta/recipes-support/attr/attr.inc
+++ b/meta/recipes-support/attr/attr.inc
@@ -29,6 +29,9 @@ ALTERNATIVE_PRIORITY = "100"
ALTERNATIVE_${PN} = "setfattr"
ALTERNATIVE_TARGET[setfattr] = "${bindir}/setfattr"
+PTEST_BUILD_HOST_FILES = "builddefs"
+PTEST_BUILD_HOST_PATTERN = "^RPM"
+
do_install_ptest() {
tar -c --exclude=ext test/ | ( cd ${D}${PTEST_PATH} && tar -xf - )
mkdir ${D}${PTEST_PATH}/include
@@ -36,12 +39,6 @@ do_install_ptest() {
do cp ${S}/include/$i ${D}${PTEST_PATH}/include/; \
done
sed -e 's|; @echo|; echo|' -i ${D}${PTEST_PATH}/test/Makefile
-
- # Remove any build host references
- sed -e "s:--sysroot=${STAGING_DIR_TARGET}::g" \
- -e 's:${HOSTTOOLS_DIR}/::g' \
- -e 's:${RECIPE_SYSROOT_NATIVE}::g' \
- -i ${D}${PTEST_PATH}/include/builddefs
}
RDEPENDS_${PN}-ptest = "attr coreutils perl-module-filehandle
perl-module-getopt-std perl-module-posix"
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v3 3/8] openssl ptest: Strip build host paths from configdata.pm
This file contains CC, CPP, CFLAGS, CXXFLAGS and the like.
Signed-off-by: Douglas Royds
Signed-off-by: Richard Purdie
---
meta/recipes-connectivity/openssl/openssl_1.1.1a.bb | 1 +
1 file changed, 1 insertion(+)
diff --git a/meta/recipes-connectivity/openssl/openssl_1.1.1a.bb
b/meta/recipes-connectivity/openssl/openssl_1.1.1a.bb
index 3629f7e6a1..deeb865a6f 100644
--- a/meta/recipes-connectivity/openssl/openssl_1.1.1a.bb
+++ b/meta/recipes-connectivity/openssl/openssl_1.1.1a.bb
@@ -156,6 +156,7 @@ do_install_append_class-nativesdk () {
sed 's|/usr/lib/ssl/|/usr/lib/ssl-1.1/|g' -i
${D}${SDKPATHNATIVE}/environment-setup.d/openssl.sh
}
+PTEST_BUILD_HOST_FILES += "configdata.pm"
do_install_ptest () {
# Prune the build tree
rm -f ${B}/fuzz/*.* ${B}/test/*.*
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v3 7/8] acl: Strip build host tool details from installed ptest file
Adopt the PTEST_BUILD_HOST_FILES mechanism to strip build host paths from the
installed builddefs file. Also strip the record of the build host rpm version.
Signed-off-by: Douglas Royds
---
meta/recipes-support/attr/acl_2.2.52.bb | 7 ++-
1 file changed, 2 insertions(+), 5 deletions(-)
diff --git a/meta/recipes-support/attr/acl_2.2.52.bb
b/meta/recipes-support/attr/acl_2.2.52.bb
index 318308a009..6bc77d868d 100644
--- a/meta/recipes-support/attr/acl_2.2.52.bb
+++ b/meta/recipes-support/attr/acl_2.2.52.bb
@@ -39,15 +39,12 @@ do_install_append() {
inherit ptest
+PTEST_BUILD_HOST_FILES = "builddefs"
+PTEST_BUILD_HOST_PATTERN = "^RPM"
do_install_ptest() {
tar -c --exclude=nfs test/ | ( cd ${D}${PTEST_PATH} && tar -xf - )
install -d ${D}${PTEST_PATH}/include
install -m 644 ${S}/include/builddefs ${S}/include/buildmacros
${S}/include/buildrules ${D}${PTEST_PATH}/include/
- # Remove any build host references
- sed -e "s:--sysroot=${STAGING_DIR_TARGET}::g" \
- -e 's:${HOSTTOOLS_DIR}/::g' \
- -e 's:${RECIPE_SYSROOT_NATIVE}::g' \
- -i ${D}${PTEST_PATH}/include/builddefs
}
RDEPENDS_${PN}-ptest = "acl bash coreutils perl perl-module-filehandle
perl-module-getopt-std perl-module-posix shadow"
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v3 0/8] ptest: Strip build host paths from any installed Makefile
Only one change from v1: Simplified commit acl: Use install instead of cp Douglas Royds (8): ptest: Reproducibility: Strip build host paths from any installed Makefile ptest: Optionally strip build host tool info from installed files openssl ptest: Strip build host paths from configdata.pm openssl: Strip perl version from installed ptest configdata.pm file strace: Strip build host tool details from installed ptest Makefile acl: Use install instead of cp acl: Strip build host tool details from installed ptest file attr: Strip build host paths and tool versions from ptest files meta/classes/ptest.bbclass | 18 ++ .../openssl/openssl_1.1.1a.bb | 2 ++ meta/recipes-devtools/strace/strace_4.25.bb| 11 +-- meta/recipes-support/attr/acl_2.2.52.bb| 11 --- meta/recipes-support/attr/attr.inc | 9 +++-- 5 files changed, 28 insertions(+), 23 deletions(-) -- 2.17.1 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v3 2/8] ptest: Optionally strip build host tool info from installed files
Several packages (eg. acl, attr, openssl) install Makefiles or other build host
configuration files for ptest so that they can run `make check`, for instance.
These build host files commonly include paths and versions of build host tools,
(eg. rpm or perl) whose presence and version depends on the build host distro.
Optionally strip lines from installed PTEST_BUILD_HOST_FILES using an
extended regex pattern defined in PTEST_BUILD_HOST_PATTERN. This has no
effect on the on-target ptest.
Signed-off-by: Douglas Royds
Signed-off-by: Richard Purdie
---
meta/classes/ptest.bbclass | 5 +
1 file changed, 5 insertions(+)
diff --git a/meta/classes/ptest.bbclass b/meta/classes/ptest.bbclass
index fa518f6d45..97865c9338 100644
--- a/meta/classes/ptest.bbclass
+++ b/meta/classes/ptest.bbclass
@@ -4,6 +4,7 @@ This package contains a test directory ${PTEST_PATH} for
package test purposes."
PTEST_PATH ?= "${libdir}/${BPN}/ptest"
PTEST_BUILD_HOST_FILES ?= "Makefile"
+PTEST_BUILD_HOST_PATTERN ?= ""
FILES_${PN}-ptest = "${PTEST_PATH}"
SECTION_${PN}-ptest = "devel"
@@ -56,6 +57,10 @@ do_install_ptest_base() {
-e 's#${WORKDIR}/*=#.=#g' \
-e 's#${WORKDIR}/*##g' \
-i $installed_ptest_file
+if [ -n "${PTEST_BUILD_HOST_PATTERN}" ]; then
+ sed -E '/${PTEST_BUILD_HOST_PATTERN}/d' \
+ -i $installed_ptest_file
+fi
done
done
}
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v3 4/8] openssl: Strip perl version from installed ptest configdata.pm file
Signed-off-by: Douglas Royds
Signed-off-by: Richard Purdie
---
meta/recipes-connectivity/openssl/openssl_1.1.1a.bb | 1 +
1 file changed, 1 insertion(+)
diff --git a/meta/recipes-connectivity/openssl/openssl_1.1.1a.bb
b/meta/recipes-connectivity/openssl/openssl_1.1.1a.bb
index deeb865a6f..042e023de5 100644
--- a/meta/recipes-connectivity/openssl/openssl_1.1.1a.bb
+++ b/meta/recipes-connectivity/openssl/openssl_1.1.1a.bb
@@ -157,6 +157,7 @@ do_install_append_class-nativesdk () {
}
PTEST_BUILD_HOST_FILES += "configdata.pm"
+PTEST_BUILD_HOST_PATTERN = "perl_version ="
do_install_ptest () {
# Prune the build tree
rm -f ${B}/fuzz/*.* ${B}/test/*.*
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v3 1/8] ptest: Reproducibility: Strip build host paths from any installed Makefile
A common pattern for ptest is to install a Makefile that provides a make check
target. These generated Makefiles are normally full of build host paths, to
hosttools, and to files and scripts within the source or build tree. They also
commonly include the CFLAGS and CPPFLAGS. None of these build host paths
can possibly work on-target, so stripping them has no effect on the success of
the resulting ptests.
Stripping the HOSTTOOLS_DIR has this effect, for instance:
-MKDIR_P = /home/douglas/workspace/upstream/build/tmp/hosttools/mkdir -p
+MKDIR_P = mkdir -p
The curious WORKDIR/*= pattern avoids non-parseable -fdebug-prefix-map patterns
in the CFLAGS by adding an arbitrary ".", eg:
-fdebug-prefix-map=.=/usr/src/debug/rsyslog/8.37.0-r0
In all other cases, we simply remove the WORKDIR, eg:
CPP = arm-tait-linux-gnueabi-gcc -E --sysroot=recipe-sysroot ...
Signed-off-by: Douglas Royds
Signed-off-by: Richard Purdie
---
meta/classes/ptest.bbclass | 13 +
1 file changed, 13 insertions(+)
diff --git a/meta/classes/ptest.bbclass b/meta/classes/ptest.bbclass
index 9d8a79854a..fa518f6d45 100644
--- a/meta/classes/ptest.bbclass
+++ b/meta/classes/ptest.bbclass
@@ -3,6 +3,8 @@ DESCRIPTION_${PN}-ptest ?= "${DESCRIPTION} \
This package contains a test directory ${PTEST_PATH} for package test
purposes."
PTEST_PATH ?= "${libdir}/${BPN}/ptest"
+PTEST_BUILD_HOST_FILES ?= "Makefile"
+
FILES_${PN}-ptest = "${PTEST_PATH}"
SECTION_${PN}-ptest = "devel"
ALLOW_EMPTY_${PN}-ptest = "1"
@@ -45,6 +47,17 @@ do_install_ptest_base() {
fi
do_install_ptest
chown -R root:root ${D}${PTEST_PATH}
+
+# Strip build host paths from any installed Makefile
+for filename in ${PTEST_BUILD_HOST_FILES}; do
+for installed_ptest_file in $(find ${D}${PTEST_PATH} -type f -name
$filename); do
+bbnote "Stripping host paths from: $installed_ptest_file"
+sed -e 's#${HOSTTOOLS_DIR}/*##g' \
+-e 's#${WORKDIR}/*=#.=#g' \
+-e 's#${WORKDIR}/*##g' \
+-i $installed_ptest_file
+done
+done
}
do_configure_ptest_base[dirs] = "${B}"
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v2 0/3] ptest: Strip build host paths from any installed Makefile
Only one change from v1: Simplified commit acl: Use install instead of cp Douglas Royds (3): acl: Use install instead of cp acl: Strip build host tool details from installed ptest file attr: Strip build host paths and tool versions from ptest files meta/recipes-support/attr/acl_2.2.52.bb | 11 --- meta/recipes-support/attr/attr.inc | 9 +++-- 2 files changed, 7 insertions(+), 13 deletions(-) -- 2.17.1 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v2 2/3] acl: Strip build host tool details from installed ptest file
Adopt the PTEST_BUILD_HOST_FILES mechanism to strip build host paths from the
installed builddefs file. Also strip the record of the build host rpm version.
Signed-off-by: Douglas Royds
---
meta/recipes-support/attr/acl_2.2.52.bb | 7 ++-
1 file changed, 2 insertions(+), 5 deletions(-)
diff --git a/meta/recipes-support/attr/acl_2.2.52.bb
b/meta/recipes-support/attr/acl_2.2.52.bb
index 318308a009..6bc77d868d 100644
--- a/meta/recipes-support/attr/acl_2.2.52.bb
+++ b/meta/recipes-support/attr/acl_2.2.52.bb
@@ -39,15 +39,12 @@ do_install_append() {
inherit ptest
+PTEST_BUILD_HOST_FILES = "builddefs"
+PTEST_BUILD_HOST_PATTERN = "^RPM"
do_install_ptest() {
tar -c --exclude=nfs test/ | ( cd ${D}${PTEST_PATH} && tar -xf - )
install -d ${D}${PTEST_PATH}/include
install -m 644 ${S}/include/builddefs ${S}/include/buildmacros
${S}/include/buildrules ${D}${PTEST_PATH}/include/
- # Remove any build host references
- sed -e "s:--sysroot=${STAGING_DIR_TARGET}::g" \
- -e 's:${HOSTTOOLS_DIR}/::g' \
- -e 's:${RECIPE_SYSROOT_NATIVE}::g' \
- -i ${D}${PTEST_PATH}/include/builddefs
}
RDEPENDS_${PN}-ptest = "acl bash coreutils perl perl-module-filehandle
perl-module-getopt-std perl-module-posix shadow"
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v2 1/3] acl: Use install instead of cp
Signed-off-by: Douglas Royds
---
meta/recipes-support/attr/acl_2.2.52.bb | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/meta/recipes-support/attr/acl_2.2.52.bb
b/meta/recipes-support/attr/acl_2.2.52.bb
index 8b89de9b42..318308a009 100644
--- a/meta/recipes-support/attr/acl_2.2.52.bb
+++ b/meta/recipes-support/attr/acl_2.2.52.bb
@@ -41,8 +41,8 @@ inherit ptest
do_install_ptest() {
tar -c --exclude=nfs test/ | ( cd ${D}${PTEST_PATH} && tar -xf - )
- mkdir ${D}${PTEST_PATH}/include
- cp ${S}/include/builddefs ${S}/include/buildmacros
${S}/include/buildrules ${D}${PTEST_PATH}/include/
+ install -d ${D}${PTEST_PATH}/include
+ install -m 644 ${S}/include/builddefs ${S}/include/buildmacros
${S}/include/buildrules ${D}${PTEST_PATH}/include/
# Remove any build host references
sed -e "s:--sysroot=${STAGING_DIR_TARGET}::g" \
-e 's:${HOSTTOOLS_DIR}/::g' \
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v2 3/3] attr: Strip build host paths and tool versions from ptest files
Using the new PTEST_BUILD_HOST_FILES mechanism.
Specifically remove reference to build host rpm version, if any.
Signed-off-by: Douglas Royds
---
meta/recipes-support/attr/attr.inc | 9 +++--
1 file changed, 3 insertions(+), 6 deletions(-)
diff --git a/meta/recipes-support/attr/attr.inc
b/meta/recipes-support/attr/attr.inc
index 9fc0f33cce..cfa38a7196 100644
--- a/meta/recipes-support/attr/attr.inc
+++ b/meta/recipes-support/attr/attr.inc
@@ -29,6 +29,9 @@ ALTERNATIVE_PRIORITY = "100"
ALTERNATIVE_${PN} = "setfattr"
ALTERNATIVE_TARGET[setfattr] = "${bindir}/setfattr"
+PTEST_BUILD_HOST_FILES = "builddefs"
+PTEST_BUILD_HOST_PATTERN = "^RPM"
+
do_install_ptest() {
tar -c --exclude=ext test/ | ( cd ${D}${PTEST_PATH} && tar -xf - )
mkdir ${D}${PTEST_PATH}/include
@@ -36,12 +39,6 @@ do_install_ptest() {
do cp ${S}/include/$i ${D}${PTEST_PATH}/include/; \
done
sed -e 's|; @echo|; echo|' -i ${D}${PTEST_PATH}/test/Makefile
-
- # Remove any build host references
- sed -e "s:--sysroot=${STAGING_DIR_TARGET}::g" \
- -e 's:${HOSTTOOLS_DIR}/::g' \
- -e 's:${RECIPE_SYSROOT_NATIVE}::g' \
- -i ${D}${PTEST_PATH}/include/builddefs
}
RDEPENDS_${PN}-ptest = "attr coreutils perl-module-filehandle
perl-module-getopt-std perl-module-posix"
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 8/8] strace: Strip build host tool details from installed ptest Makefile
Adopt new PTEST_BUILD_HOST_PATTERN mechanism.
Signed-off-by: Douglas Royds
---
meta/recipes-devtools/strace/strace_4.25.bb | 11 +--
1 file changed, 1 insertion(+), 10 deletions(-)
diff --git a/meta/recipes-devtools/strace/strace_4.25.bb
b/meta/recipes-devtools/strace/strace_4.25.bb
index 117ebfbf72..58f6cf6e30 100644
--- a/meta/recipes-devtools/strace/strace_4.25.bb
+++ b/meta/recipes-devtools/strace/strace_4.25.bb
@@ -32,6 +32,7 @@ EXTRA_OECONF += "--enable-mpers=no"
CFLAGS_append_libc-musl = " -Dsigcontext_struct=sigcontext"
TESTDIR = "tests"
+PTEST_BUILD_HOST_PATTERN =
"^(DEB_CHANGELOGTIME|RPM_CHANGELOGTIME|WARN_CFLAGS_FOR_BUILD)"
do_install_append() {
# We don't ship strace-graph here because it needs perl
@@ -46,16 +47,6 @@ do_install_ptest() {
oe_runmake -C ${TESTDIR} install-ptest BUILDDIR=${B}
DESTDIR=${D}${PTEST_PATH} TESTDIR=${TESTDIR}
install -m 755 ${S}/test-driver ${D}${PTEST_PATH}
install -m 644 ${B}/config.h ${D}${PTEST_PATH}
- sed -i -e '/^src/s/strace.*[1-9]/ptest/' \
- -e 's,--sysroot=${STAGING_DIR_TARGET},,g' \
- -e 's|${DEBUG_PREFIX_MAP}||g' \
- -e 's:${HOSTTOOLS_DIR}/::g' \
- -e 's:${RECIPE_SYSROOT_NATIVE}::g' \
- -e 's:${RECIPE_SYSROOT}::g' \
- -e 's:${BASE_WORKDIR}/${MULTIMACH_TARGET_SYS}::g' \
- -e '/^DEB_CHANGELOGTIME/d' \
- -e '/^RPM_CHANGELOGTIME/d' \
- ${D}/${PTEST_PATH}/${TESTDIR}/Makefile
}
RDEPENDS_${PN}-ptest += "make coreutils grep gawk sed"
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 5/8] acl: Use install instead of cp
Also, hard-tabs begone
Signed-off-by: Douglas Royds
---
meta/recipes-support/attr/acl_2.2.52.bb | 24
1 file changed, 12 insertions(+), 12 deletions(-)
diff --git a/meta/recipes-support/attr/acl_2.2.52.bb
b/meta/recipes-support/attr/acl_2.2.52.bb
index 8b89de9b42..260975d93e 100644
--- a/meta/recipes-support/attr/acl_2.2.52.bb
+++ b/meta/recipes-support/attr/acl_2.2.52.bb
@@ -27,27 +27,27 @@ require ea-acl.inc
# avoid RPATH hardcode to staging dir
do_configure_append() {
- sed -i ${S}/config.status -e
s,^\\\(hardcode_into_libs=\\\).*$,\\1\'no\',
- ${S}/config.status
+sed -i ${S}/config.status -e s,^\\\(hardcode_into_libs=\\\).*$,\\1\'no\',
+${S}/config.status
}
# libdir should point to .la
do_install_append() {
- sed -i ${D}${libdir}/libacl.la -e \
- s,^libdir=\'${base_libdir}\'$,libdir=\'${libdir}\',
+sed -i ${D}${libdir}/libacl.la \
+-e s,^libdir=\'${base_libdir}\'$,libdir=\'${libdir}\',
}
inherit ptest
do_install_ptest() {
- tar -c --exclude=nfs test/ | ( cd ${D}${PTEST_PATH} && tar -xf - )
- mkdir ${D}${PTEST_PATH}/include
- cp ${S}/include/builddefs ${S}/include/buildmacros
${S}/include/buildrules ${D}${PTEST_PATH}/include/
- # Remove any build host references
- sed -e "s:--sysroot=${STAGING_DIR_TARGET}::g" \
- -e 's:${HOSTTOOLS_DIR}/::g' \
- -e 's:${RECIPE_SYSROOT_NATIVE}::g' \
- -i ${D}${PTEST_PATH}/include/builddefs
+tar -c --exclude=nfs test/ | ( cd ${D}${PTEST_PATH} && tar -xf - )
+install -d ${D}${PTEST_PATH}/include
+install -m 644 ${S}/include/builddefs ${S}/include/buildmacros
${S}/include/buildrules ${D}${PTEST_PATH}/include/
+# Remove any build host references
+sed -e "s:--sysroot=${STAGING_DIR_TARGET}::g" \
+-e 's:${HOSTTOOLS_DIR}/::g' \
+-e 's:${RECIPE_SYSROOT_NATIVE}::g' \
+-i ${D}${PTEST_PATH}/include/builddefs
}
RDEPENDS_${PN}-ptest = "acl bash coreutils perl perl-module-filehandle
perl-module-getopt-std perl-module-posix shadow"
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 7/8] attr: Strip build host paths and tool versions from ptest files
Using the new PTEST_BUILD_HOST_FILES mechanism.
Specifically remove reference to build host rpm version, if any.
Signed-off-by: Douglas Royds
---
meta/recipes-support/attr/attr.inc | 9 +++--
1 file changed, 3 insertions(+), 6 deletions(-)
diff --git a/meta/recipes-support/attr/attr.inc
b/meta/recipes-support/attr/attr.inc
index 9fc0f33cce..cfa38a7196 100644
--- a/meta/recipes-support/attr/attr.inc
+++ b/meta/recipes-support/attr/attr.inc
@@ -29,6 +29,9 @@ ALTERNATIVE_PRIORITY = "100"
ALTERNATIVE_${PN} = "setfattr"
ALTERNATIVE_TARGET[setfattr] = "${bindir}/setfattr"
+PTEST_BUILD_HOST_FILES = "builddefs"
+PTEST_BUILD_HOST_PATTERN = "^RPM"
+
do_install_ptest() {
tar -c --exclude=ext test/ | ( cd ${D}${PTEST_PATH} && tar -xf - )
mkdir ${D}${PTEST_PATH}/include
@@ -36,12 +39,6 @@ do_install_ptest() {
do cp ${S}/include/$i ${D}${PTEST_PATH}/include/; \
done
sed -e 's|; @echo|; echo|' -i ${D}${PTEST_PATH}/test/Makefile
-
- # Remove any build host references
- sed -e "s:--sysroot=${STAGING_DIR_TARGET}::g" \
- -e 's:${HOSTTOOLS_DIR}/::g' \
- -e 's:${RECIPE_SYSROOT_NATIVE}::g' \
- -i ${D}${PTEST_PATH}/include/builddefs
}
RDEPENDS_${PN}-ptest = "attr coreutils perl-module-filehandle
perl-module-getopt-std perl-module-posix"
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 6/8] acl: Strip build host tool details from installed ptest file
Adopt the PTEST_BUILD_HOST_FILES mechanism to strip build host paths from the
installed builddefs file. Also strip the record of the build host rpm version.
Signed-off-by: Douglas Royds
---
meta/recipes-support/attr/acl_2.2.52.bb | 7 ++-
1 file changed, 2 insertions(+), 5 deletions(-)
diff --git a/meta/recipes-support/attr/acl_2.2.52.bb
b/meta/recipes-support/attr/acl_2.2.52.bb
index 260975d93e..82c9277c92 100644
--- a/meta/recipes-support/attr/acl_2.2.52.bb
+++ b/meta/recipes-support/attr/acl_2.2.52.bb
@@ -39,15 +39,12 @@ do_install_append() {
inherit ptest
+PTEST_BUILD_HOST_FILES = "builddefs"
+PTEST_BUILD_HOST_PATTERN = "^RPM"
do_install_ptest() {
tar -c --exclude=nfs test/ | ( cd ${D}${PTEST_PATH} && tar -xf - )
install -d ${D}${PTEST_PATH}/include
install -m 644 ${S}/include/builddefs ${S}/include/buildmacros
${S}/include/buildrules ${D}${PTEST_PATH}/include/
-# Remove any build host references
-sed -e "s:--sysroot=${STAGING_DIR_TARGET}::g" \
--e 's:${HOSTTOOLS_DIR}/::g' \
--e 's:${RECIPE_SYSROOT_NATIVE}::g' \
--i ${D}${PTEST_PATH}/include/builddefs
}
RDEPENDS_${PN}-ptest = "acl bash coreutils perl perl-module-filehandle
perl-module-getopt-std perl-module-posix shadow"
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 1/8] ptest: Reproducibility: Strip build host paths from any installed Makefile
A common pattern for ptest is to install a Makefile that provides a make check
target. These generated Makefiles are normally full of build host paths, to
hosttools, and to files and scripts within the source or build tree. They also
commonly include the CFLAGS and CPPFLAGS. None of these build host paths
can possibly work on-target, so stripping them has no effect on the success of
the resulting ptests.
Stripping the HOSTTOOLS_DIR has this effect, for instance:
-MKDIR_P = /home/douglas/workspace/upstream/build/tmp/hosttools/mkdir -p
+MKDIR_P = mkdir -p
The curious WORKDIR/*= pattern avoids non-parseable -fdebug-prefix-map patterns
in the CFLAGS by adding an arbitrary ".", eg:
-fdebug-prefix-map=.=/usr/src/debug/rsyslog/8.37.0-r0
In all other cases, we simply remove the WORKDIR, eg:
CPP = arm-tait-linux-gnueabi-gcc -E --sysroot=recipe-sysroot ...
Signed-off-by: Douglas Royds
---
meta/classes/ptest.bbclass | 13 +
1 file changed, 13 insertions(+)
diff --git a/meta/classes/ptest.bbclass b/meta/classes/ptest.bbclass
index 9d8a79854a..fa518f6d45 100644
--- a/meta/classes/ptest.bbclass
+++ b/meta/classes/ptest.bbclass
@@ -3,6 +3,8 @@ DESCRIPTION_${PN}-ptest ?= "${DESCRIPTION} \
This package contains a test directory ${PTEST_PATH} for package test
purposes."
PTEST_PATH ?= "${libdir}/${BPN}/ptest"
+PTEST_BUILD_HOST_FILES ?= "Makefile"
+
FILES_${PN}-ptest = "${PTEST_PATH}"
SECTION_${PN}-ptest = "devel"
ALLOW_EMPTY_${PN}-ptest = "1"
@@ -45,6 +47,17 @@ do_install_ptest_base() {
fi
do_install_ptest
chown -R root:root ${D}${PTEST_PATH}
+
+# Strip build host paths from any installed Makefile
+for filename in ${PTEST_BUILD_HOST_FILES}; do
+for installed_ptest_file in $(find ${D}${PTEST_PATH} -type f -name
$filename); do
+bbnote "Stripping host paths from: $installed_ptest_file"
+sed -e 's#${HOSTTOOLS_DIR}/*##g' \
+-e 's#${WORKDIR}/*=#.=#g' \
+-e 's#${WORKDIR}/*##g' \
+-i $installed_ptest_file
+done
+done
}
do_configure_ptest_base[dirs] = "${B}"
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 2/8] ptest: Optionally strip build host tool info from installed files
Several packages (eg. acl, attr, openssl) install Makefiles or other build host
configuration files for ptest so that they can run `make check`, for instance.
These build host files commonly include paths and versions of build host tools,
(eg. rpm or perl) whose presence and version depends on the build host distro.
Optionally strip lines from installed PTEST_BUILD_HOST_FILES using an
extended regex pattern defined in PTEST_BUILD_HOST_PATTERN. This has no
effect on the on-target ptest.
Signed-off-by: Douglas Royds
---
meta/classes/ptest.bbclass | 5 +
1 file changed, 5 insertions(+)
diff --git a/meta/classes/ptest.bbclass b/meta/classes/ptest.bbclass
index fa518f6d45..97865c9338 100644
--- a/meta/classes/ptest.bbclass
+++ b/meta/classes/ptest.bbclass
@@ -4,6 +4,7 @@ This package contains a test directory ${PTEST_PATH} for
package test purposes."
PTEST_PATH ?= "${libdir}/${BPN}/ptest"
PTEST_BUILD_HOST_FILES ?= "Makefile"
+PTEST_BUILD_HOST_PATTERN ?= ""
FILES_${PN}-ptest = "${PTEST_PATH}"
SECTION_${PN}-ptest = "devel"
@@ -56,6 +57,10 @@ do_install_ptest_base() {
-e 's#${WORKDIR}/*=#.=#g' \
-e 's#${WORKDIR}/*##g' \
-i $installed_ptest_file
+if [ -n "${PTEST_BUILD_HOST_PATTERN}" ]; then
+ sed -E '/${PTEST_BUILD_HOST_PATTERN}/d' \
+ -i $installed_ptest_file
+fi
done
done
}
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 0/8] ptest: Strip build host paths from any installed Makefile
PTEST_BUILD_HOST_FILES A common pattern for ptest is to install a Makefile that provides a make check target. These generated Makefiles are normally full of build host paths, to hosttools, and to files and scripts within the source or build tree. They also commonly include the CFLAGS and CPPFLAGS. None of these build host paths can possibly work on-target, so stripping them has no effect on the success of the resulting ptests. PTEST_BUILD_HOST_FILES is a whitespace-separated list of filenames within the PTEST_PATH that should have build host paths stripped from them at do_install_ptest_base time. It defaults to "Makefile". PTEST_BUILD_HOST_PATTERN Several packages install Makefiles or other build host configuration files for ptest so that they can run `make check`, for instance. These build host files commonly include paths and versions of build host tools, (eg. rpm or perl) whose presence and version depends on the build host distro. PTEST_BUILD_HOST_PATTERN is an extended regex pattern used to optionally strip lines from installed PTEST_BUILD_HOST_FILES. Douglas Royds (8): ptest: Reproducibility: Strip build host paths from any installed Makefile ptest: Optionally strip build host tool info from installed files openssl ptest: Strip build host paths from configdata.pm openssl: Strip perl version from installed ptest configdata.pm file acl: Use install instead of cp acl: Strip build host tool details from installed ptest file attr: Strip build host paths and tool versions from ptest files strace: Strip build host tool details from installed ptest Makefile meta/classes/ptest.bbclass| 18 .../openssl/openssl_1.1.1a.bb | 2 ++ meta/recipes-devtools/strace/strace_4.25.bb | 11 +- meta/recipes-support/attr/acl_2.2.52.bb | 21 --- meta/recipes-support/attr/attr.inc| 9 +++- 5 files changed, 33 insertions(+), 28 deletions(-) -- 2.17.1 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 4/8] openssl: Strip perl version from installed ptest configdata.pm file
Signed-off-by: Douglas Royds
---
meta/recipes-connectivity/openssl/openssl_1.1.1a.bb | 1 +
1 file changed, 1 insertion(+)
diff --git a/meta/recipes-connectivity/openssl/openssl_1.1.1a.bb
b/meta/recipes-connectivity/openssl/openssl_1.1.1a.bb
index deeb865a6f..042e023de5 100644
--- a/meta/recipes-connectivity/openssl/openssl_1.1.1a.bb
+++ b/meta/recipes-connectivity/openssl/openssl_1.1.1a.bb
@@ -157,6 +157,7 @@ do_install_append_class-nativesdk () {
}
PTEST_BUILD_HOST_FILES += "configdata.pm"
+PTEST_BUILD_HOST_PATTERN = "perl_version ="
do_install_ptest () {
# Prune the build tree
rm -f ${B}/fuzz/*.* ${B}/test/*.*
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 3/8] openssl ptest: Strip build host paths from configdata.pm
This file contains CC, CPP, CFLAGS, CXXFLAGS and the like.
Signed-off-by: Douglas Royds
---
meta/recipes-connectivity/openssl/openssl_1.1.1a.bb | 1 +
1 file changed, 1 insertion(+)
diff --git a/meta/recipes-connectivity/openssl/openssl_1.1.1a.bb
b/meta/recipes-connectivity/openssl/openssl_1.1.1a.bb
index 3629f7e6a1..deeb865a6f 100644
--- a/meta/recipes-connectivity/openssl/openssl_1.1.1a.bb
+++ b/meta/recipes-connectivity/openssl/openssl_1.1.1a.bb
@@ -156,6 +156,7 @@ do_install_append_class-nativesdk () {
sed 's|/usr/lib/ssl/|/usr/lib/ssl-1.1/|g' -i
${D}${SDKPATHNATIVE}/environment-setup.d/openssl.sh
}
+PTEST_BUILD_HOST_FILES += "configdata.pm"
do_install_ptest () {
# Prune the build tree
rm -f ${B}/fuzz/*.* ${B}/test/*.*
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 2/2] reproducible: Don't look for youngest file when no source tarball
Some packages (eg. init-ifupdown) take their source files entirely from
openembedded-core, that is, they download no source tarball.
These recipes either don't use S at all (ie. it is empty at unpack time),
or they set S = WORKDIR (as in init-ifupdown).
Looking at the file timestamps in the WORKDIR causes a non-reproducible
SOURCE_DATE_EPOCH, as files taken from file:// URIs do not have
reproducible timestamps.
If S == WORKDIR, we are better to assume that there is no source tarball,
and to fall back to a fixed timestamp for the SOURCE_DATE_EPOCH.
This makes the init-ifupdown build reproducible.
Signed-off-by: Douglas Royds
---
meta/classes/reproducible_build.bbclass | 13 -
1 file changed, 8 insertions(+), 5 deletions(-)
diff --git a/meta/classes/reproducible_build.bbclass
b/meta/classes/reproducible_build.bbclass
index a0fd4d656b..8788ad7145 100644
--- a/meta/classes/reproducible_build.bbclass
+++ b/meta/classes/reproducible_build.bbclass
@@ -22,7 +22,10 @@
# 3. Use the mtime of "known" files such as NEWS, CHANGLELOG, ...
#This works for well-kept repositories distributed via tarball.
#
-# 4. If the above steps fail, use the modification time of the youngest file
in the source tree.
+# 4. Use the modification time of the youngest file in the source tree, if
there is one.
+#This will be the newest file from the distribution tarball, if any.
+#
+# 5. Fall back to a fixed timestamp.
#
# Once the value of SOURCE_DATE_EPOCH is determined, it is stored in the
recipe's SDE_FILE.
# If none of these mechanisms are suitable, replace the
do_deploy_source_date_epoch task
@@ -104,15 +107,15 @@ def get_source_date_epoch_from_git(d, sourcedir):
return source_date_epoch
def get_source_date_epoch_from_youngest_file(d, sourcedir):
+if sourcedir == d.getVar('WORKDIR'):
+ # These sources are almost certainly not from a tarball
+ return None
+
# Do it the hard way: check all files and find the youngest one...
source_date_epoch = None
newest_file = None
-# Just in case S = WORKDIR
-exclude = set(["build", "image", "license-destdir", "patches", "pseudo",
- "recipe-sysroot", "recipe-sysroot-native",
"sysroot-destdir", "temp"])
for root, dirs, files in os.walk(sourcedir, topdown=True):
files = [f for f in files if not f[0] == '.']
-dirs[:] = [d for d in dirs if d not in exclude]
for fname in files:
filename = os.path.join(root, fname)
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 1/2] reproducible: Refactor: Break out fixed_source_date_epoch() function
Signed-off-by: Douglas Royds
---
meta/classes/reproducible_build.bbclass | 9 +
1 file changed, 5 insertions(+), 4 deletions(-)
diff --git a/meta/classes/reproducible_build.bbclass
b/meta/classes/reproducible_build.bbclass
index 0eb696ac78..a0fd4d656b 100644
--- a/meta/classes/reproducible_build.bbclass
+++ b/meta/classes/reproducible_build.bbclass
@@ -128,6 +128,10 @@ def get_source_date_epoch_from_youngest_file(d, sourcedir):
bb.debug(1, "Newest file found: %s" % newest_file)
return source_date_epoch
+def fixed_source_date_epoch():
+bb.debug(1, "No tarball or git repo found to determine SOURCE_DATE_EPOCH")
+return 0
+
python do_create_source_date_epoch_stamp() {
epochfile = d.getVar('SDE_FILE')
if os.path.isfile(epochfile):
@@ -139,11 +143,8 @@ python do_create_source_date_epoch_stamp() {
get_source_date_epoch_from_git(d, sourcedir) or
get_source_date_epoch_from_known_files(d, sourcedir) or
get_source_date_epoch_from_youngest_file(d, sourcedir) or
-0 # Last resort
+fixed_source_date_epoch() # Last resort
)
-if source_date_epoch == 0:
-# empty folder, not a single file ...
-bb.debug(1, "No files found to determine SOURCE_DATE_EPOCH")
bb.debug(1, "SOURCE_DATE_EPOCH: %d" % source_date_epoch)
bb.utils.mkdirhier(d.getVar('SDE_DIR'))
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v3] boost-context: Reproducibility: Set .file section for all *_elf_gas.S files
Add a .file directive explicitly for all *_elf_gas.S files to prevent the linker adding a host build-system path as a FILE symbol to the object file. This replaces the existing patch that added the .file directive to a small subset of these files. Upstream-Status: Submitted [https://github.com/boostorg/context/issues/91] Signed-off-by: Douglas Royds --- ..._64_sysv_elf_gas.S-set-.file-section.patch | 48 ...lity-add-file-directive-to-assembler.patch | 243 ++ meta/recipes-support/boost/boost_1.68.0.bb| 2 +- 3 files changed, 244 insertions(+), 49 deletions(-) delete mode 100644 meta/recipes-support/boost/boost/0001-make_x86_64_sysv_elf_gas.S-set-.file-section.patch create mode 100644 meta/recipes-support/boost/boost/reproducibility-add-file-directive-to-assembler.patch diff --git a/meta/recipes-support/boost/boost/0001-make_x86_64_sysv_elf_gas.S-set-.file-section.patch b/meta/recipes-support/boost/boost/0001-make_x86_64_sysv_elf_gas.S-set-.file-section.patch deleted file mode 100644 index c2ac49e58f..00 --- a/meta/recipes-support/boost/boost/0001-make_x86_64_sysv_elf_gas.S-set-.file-section.patch +++ /dev/null @@ -1,48 +0,0 @@ -From c99f798407e44c86e9f64f5b1adf3b4d0549eef2 Mon Sep 17 00:00:00 2001 -From: Mingli Yu -Date: Fri, 15 Jun 2018 16:21:56 +0800 -Subject: [PATCH] make_x86_64_sysv_elf_gas.S: set .file section - -Set .file section explicitly for .S files to avoid -the linker introduces the host path in symbols for -object files whose source file is .S -Otherwise, there is a host path in the symbols as below: -$ readelf --wide --symbols /my-build/boost/1.67.0-r0/boost_1_67_0/x86_64-poky-linux/boost/bin.v2/libs/context/build/aca09349fdb84d131321425f6c3a38ed/libboost_context.so.1.67.0 - -42: 0 FILE LOCAL DEFAULT ABS /my-build/boost/1.67.0-r0/boost_1_67_0/x86_64-poky-linux/boost/bin.v2/libs/context/build/aca09349fdb84d131321425f6c3a38ed/asm/make_x86_64_sysv_elf_gas.o - -Upstream-Status: Pending - -Signed-off-by: Mingli Yu - libs/context/src/asm/make_i386_sysv_elf_gas.S | 1 + - libs/context/src/asm/make_x86_64_sysv_elf_gas.S | 1 + - 2 files changed, 2 insertions(+) - -diff --git a/libs/context/src/asm/make_i386_sysv_elf_gas.S b/libs/context/src/asm/make_i386_sysv_elf_gas.S -index de77e88..b76de26 100644 a/libs/context/src/asm/make_i386_sysv_elf_gas.S -+++ b/libs/context/src/asm/make_i386_sysv_elf_gas.S -@@ -24,6 +24,7 @@ - * * - / - -+.file "make_i386_sysv_elf_gas.S" - .text - .globl make_fcontext - .align 2 -diff --git a/libs/context/src/asm/make_x86_64_sysv_elf_gas.S b/libs/context/src/asm/make_x86_64_sysv_elf_gas.S -index 25a0c00..0ef3756 100644 a/libs/context/src/asm/make_x86_64_sysv_elf_gas.S -+++ b/libs/context/src/asm/make_x86_64_sysv_elf_gas.S -@@ -24,6 +24,7 @@ - * * - / - -+.file "make_x86_64_sysv_elf_gas.S" - .text - .globl make_fcontext - .type make_fcontext,@function --- -2.7.4 - diff --git a/meta/recipes-support/boost/boost/reproducibility-add-file-directive-to-assembler.patch b/meta/recipes-support/boost/boost/reproducibility-add-file-directive-to-assembler.patch new file mode 100644 index 00..155653316c --- /dev/null +++ b/meta/recipes-support/boost/boost/reproducibility-add-file-directive-to-assembler.patch @@ -0,0 +1,243 @@ +Author: Douglas Royds +Date: Thu Nov 22 09:34:22 2018 +1300 + +Add a .file directive explicitly for all *_elf_gas.S files to prevent the linker +adding a host build-system path as a FILE symbol to the object file. + +This replaces the existing patch that added the .file directive to a small +subset of these files. + +Upstream-Status: Submitted [https://github.com/boostorg/context/issues/91] +Signed-off-by: Douglas Royds + +diff -ur boost_1_68_0.original/libs/context/src/asm/jump_arm64_aapcs_elf_gas.S boost_1_68_0/libs/context/src/asm/jump_arm64_aapcs_elf_gas.S +--- boost_1_68_0.original/libs/context/src/asm/jump_arm64_aapcs_elf_gas.S 2018-11-21 16:31:18.601760893 +1300 boost_1_68_0/libs/context/src/asm/jump_arm64_aapcs_elf_gas.S 2018-11-22 09:04:27.900876941 +1300 +@@ -51,6 +51,7 @@ + * * + ***/ + ++.file "jump_arm64_aapcs_elf_gas.S" + .text + .align 2 + .global jump_fcontext +diff -ur boost_1_68_0.original/libs/context/src/asm/jump_arm_aapcs_elf_gas.S boost_1_68_0/libs/context/src/asm/jump_arm_aapcs_elf_gas.S +--- boost_1_68_0.original/libs/context/src/asm/jump_arm_aapcs_elf_gas.S 2018-11-21 16:31:18.601760893 +1300 boost_1_68_0/libs/context/sr
Re: [OE-core] ✗ patchtest: failure for boost-context: Reproducibility: Set .file section for all *_elf_gas.S files (rev2)
On 23/11/18 2:24 AM, Burton, Ross wrote: On Thu, 22 Nov 2018 at 00:38, Douglas Royds wrote: On 22/11/18 1:33 PM, Patchwork wrote: ... * Issue Upstream-Status is Submitted, but it is not mentioned where [test_upstream_status_presence_format] Suggested fixInclude where reproducibility-add-file-directive-to-assembler.patch was submitted Yes it is ... Current Upstream-Status: Submitted https://github.com/boostorg/context/issues/91 Standard format Upstream-Status: Submitted [where] The [] is not syntactic markers but literal characters, i.e. it expects to see: Upstream-Status: Submitted [https://github.com/boostorg/context/issues/91] Yes, this test could be considered overly pedantic, but I'm also aware that raising and formalising this would be a bikeshed of epic proportions... Ross Great, thanks, my mistake. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH] ptest: Reproducibility: Take control of umask
The build host umask was leaking into the thing-ptest packages
at do_install_ptest() time.
Signed-off-by: Douglas Royds
---
meta/classes/ptest.bbclass | 1 +
1 file changed, 1 insertion(+)
diff --git a/meta/classes/ptest.bbclass b/meta/classes/ptest.bbclass
index c19f65b9bb..9d8a79854a 100644
--- a/meta/classes/ptest.bbclass
+++ b/meta/classes/ptest.bbclass
@@ -59,6 +59,7 @@ addtask install_ptest_base after do_install before
do_package do_populate_sy
python () {
if not bb.data.inherits_class('native', d) and not
bb.data.inherits_class('cross', d):
d.setVarFlag('do_install_ptest_base', 'fakeroot', '1')
+d.setVarFlag('do_install_ptest_base', 'umask', '022')
# Remove all '*ptest_base' tasks when ptest is not enabled
if not(d.getVar('PTEST_ENABLED') == "1"):
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] ✗ patchtest: failure for boost-context: Reproducibility: Set .file section for all *_elf_gas.S files (rev2)
On 22/11/18 1:33 PM, Patchwork wrote: ... * Issue Upstream-Status is Submitted, but it is not mentioned where [test_upstream_status_presence_format] Suggested fixInclude where reproducibility-add-file-directive-to-assembler.patch was submitted Yes it is ... Current Upstream-Status: Submitted https://github.com/boostorg/context/issues/91 Standard format Upstream-Status: Submitted [where] Huh? -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
