Re: [OE-core] boost 1.69 -> 1.71 upgrade for 3.0?
Hi,
On 22.08.19 21:19, Randy MacLeod wrote:
> I'm checking if you have done any work on upgrading the
> boost recipe?
Not long ago I've looked at updating the boost recipe to 1.71.0-beta1
(when 1.71.0 wasn't released yet) because of some bugs encountered with
1.69.0 (and 1.70.0 being buggy too). See the attached patch (it is based
on poky warrior).
It was not trivial due to changes to bjam bootstrap scripts, but I got
it working. I'd be interested in updating it to the final 1.71.0 too,
including backporting to warrior.
Regards,
Daniel
>From 2f8eb6379ff926cc53d1b2a24a2b93c5ea5268fa Mon Sep 17 00:00:00 2001
From: Daniel Klauer
Date: Mon, 26 Aug 2019 11:01:00 +0200
Subject: [PATCH] boost: Update to 1.71.0-beta1
Some notes:
- bjam bootstrap scripts were changed, and the patch for that had to be
adjusted. tools/build/src/engine/build.sh now supports an --debug option
which the bjam-native recipe can use to get the debug build of b2.
- gcc.jam patch had to be refreshed because boost added more "cpu-flags"
lines.
- since 1.70.0, boost includes new cmake config files which are packaged
in ${PN}-dev now so they make it into the SDK. (although 1.71.0 is needed
to fix some bugs in them)
- The recipe had to be adjusted more than usual to support the boost beta
version.
---
..._1.69.0.bb => bjam-native_1.71.0.beta1.bb} | 5 +-
...oost-1.69.0.inc => boost-1.71.0.beta1.inc} | 12 ++---
meta/recipes-support/boost/boost.inc | 2 +-
...h-instruction-set-flags-we-do-that-o.patch | 23 +
...{boost_1.69.0.bb => boost_1.71.0.beta1.bb} | 0
.../0001-Build-debug-version-of-bjam.patch| 25 ++
.../files/bjam-native-build-bjam.debug.patch | 50 ---
7 files changed, 45 insertions(+), 72 deletions(-)
rename meta/recipes-support/boost/{bjam-native_1.69.0.bb => bjam-native_1.71.0.beta1.bb} (70%)
rename meta/recipes-support/boost/{boost-1.69.0.inc => boost-1.71.0.beta1.inc} (59%)
rename meta/recipes-support/boost/{boost_1.69.0.bb => boost_1.71.0.beta1.bb} (100%)
create mode 100644 meta/recipes-support/boost/files/0001-Build-debug-version-of-bjam.patch
delete mode 100644 meta/recipes-support/boost/files/bjam-native-build-bjam.debug.patch
diff --git a/meta/recipes-support/boost/bjam-native_1.69.0.bb b/meta/recipes-support/boost/bjam-native_1.71.0.beta1.bb
similarity index 70%
rename from meta/recipes-support/boost/bjam-native_1.69.0.bb
rename to meta/recipes-support/boost/bjam-native_1.71.0.beta1.bb
index 94f96e62d8..30279a73e6 100644
--- a/meta/recipes-support/boost/bjam-native_1.69.0.bb
+++ b/meta/recipes-support/boost/bjam-native_1.71.0.beta1.bb
@@ -5,8 +5,7 @@ SECTION = "devel"
inherit native
-SRC_URI += "file://bjam-native-build-bjam.debug.patch \
-"
+SRC_URI += "file://0001-Build-debug-version-of-bjam.patch"
do_compile() {
./bootstrap.sh --with-toolset=gcc
@@ -15,5 +14,5 @@ do_compile() {
do_install() {
install -d ${D}${bindir}/
# install unstripped version for bjam
-install -c -m 755 bjam.debug ${D}${bindir}/bjam
+install -c -m 755 b2 ${D}${bindir}/bjam
}
diff --git a/meta/recipes-support/boost/boost-1.69.0.inc b/meta/recipes-support/boost/boost-1.71.0.beta1.inc
similarity index 59%
rename from meta/recipes-support/boost/boost-1.69.0.inc
rename to meta/recipes-support/boost/boost-1.71.0.beta1.inc
index 923436b1e0..d2e2a11457 100644
--- a/meta/recipes-support/boost/boost-1.69.0.inc
+++ b/meta/recipes-support/boost/boost-1.71.0.beta1.inc
@@ -7,15 +7,11 @@ HOMEPAGE = "http://www.boost.org/";
LICENSE = "BSL-1.0 & MIT & Python-2.0"
LIC_FILES_CHKSUM = "file://LICENSE_1_0.txt;md5=e4224ccaecb14d942c71d31bef20d78c"
-BOOST_VER = "${@"_".join(d.getVar("PV").split("."))}"
-BOOST_MAJ = "${@"_".join(d.getVar("PV").split(".")[0:2])}"
-BOOST_P = "boost_${BOOST_VER}"
-
-SRC_URI = "${SOURCEFORGE_MIRROR}/project/boost/boost/${PV}/${BOOST_P}.tar.bz2"
-SRC_URI[md5sum] = "a1332494397bf48332cb152abfefcec2"
-SRC_URI[sha256sum] = "8f32d4617390d1c2d16f26a27ab60d97807b35440d45891fa340fc2648b04406"
+SRC_URI = "${SOURCEFORGE_MIRROR}/project/boost/boost/${PV}/boost_1_71_0_b1.tar.bz2"
+SRC_URI[md5sum] = "f74e2064eef26e108f304680b21a9a82"
+SRC_URI[sha256sum] = "a93d7066fc1514995aec0e726532a219447dd39b0ffa7cb97ca5736f16b0488b"
UPSTREAM_CHECK_URI = "http://www.boost.org/users/download/";
UPSTREAM_CHECK_REGEX = "boostorg/release/(?P.*)/source/"
-S = "${WORKDIR}/${BOOST_P}"
+S = "${WORKDIR}/boost_1_71_0"
diff --git a/meta/recipes-support/boost/boost.inc b/meta/recipes-support/boost/boost.inc
index 9be3717fd6..6de940265c 100644
--- a/meta/recipes-support/boost/boost.inc
+++ b/meta/recipes-support/boost/boost.inc
@@ -80,7 +80,7 @@ FILE
[OE-core] [PATCH 2/2] deploy.bbclass: Clean DEPLOYDIR before do_deploy
It seems convenient for do_deploy to clean up ${DEPLOYDIR} (its output
directory) before running, just like do_install cleans up ${D} before
running. This way we can be sure that a recipe's do_deploy output is not
accidentally contaminated by previously existing files in DEPLOYDIR in
case of incremental builds.
All recipes using deploy.bbclass (grep -r 'inherit .*deploy') in poky,
meta-openembedded and meta-freescale look to me like they either benefit
from this or are at least not affected negatively by it. The only exception
I've noticed was uboot-sign.bbclass, which was however fixed by the
previous patch.
I tested this by running "bitbake virtual/kernel u-boot":
1. on patched, but clean poky source tree
2. after adding "touch ${DEPLOYDIR}/foo-${PN}" to linux-yocto's/u-boot's
do_deploy, to manually trigger an incremental build of do_deploy only.
3. after removing the modifications again.
The foo-${PN} files existed in tmp/deploy/images/qemux86-64 after step 2,
and not anymore after step 3, as expected. Thus the cleandirs still works.
Signed-off-by: Daniel Klauer
---
meta/classes/deploy.bbclass| 1 +
meta/classes/kernel.bbclass| 2 --
meta/recipes-core/meta/signing-keys.bb | 2 --
meta/recipes-core/ovmf/ovmf_git.bb | 1 -
4 files changed, 1 insertion(+), 5 deletions(-)
diff --git a/meta/classes/deploy.bbclass b/meta/classes/deploy.bbclass
index 6d52908783..737c26122b 100644
--- a/meta/classes/deploy.bbclass
+++ b/meta/classes/deploy.bbclass
@@ -8,4 +8,5 @@ python do_deploy_setscene () {
}
addtask do_deploy_setscene
do_deploy[dirs] = "${DEPLOYDIR} ${B}"
+do_deploy[cleandirs] = "${DEPLOYDIR}"
do_deploy[stamp-extra-info] = "${MACHINE_ARCH}"
diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index bf3674238f..aa339ace7e 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -706,8 +706,6 @@ kernel_do_deploy() {
done
fi
}
-do_deploy[cleandirs] = "${DEPLOYDIR}"
-do_deploy[dirs] = "${DEPLOYDIR} ${B}"
do_deploy[prefuncs] += "package_get_auto_pr"
addtask deploy after do_populate_sysroot do_packagedata
diff --git a/meta/recipes-core/meta/signing-keys.bb
b/meta/recipes-core/meta/signing-keys.bb
index 1e1c7e3459..5bab94aa36 100644
--- a/meta/recipes-core/meta/signing-keys.bb
+++ b/meta/recipes-core/meta/signing-keys.bb
@@ -67,8 +67,6 @@ do_deploy () {
fi
}
do_deploy[sstate-outputdirs] = "${DEPLOY_DIR_RPM}"
-# cleandirs should possibly be in deploy.bbclass but we need it
-do_deploy[cleandirs] = "${DEPLOYDIR}"
# clear stamp-extra-info since MACHINE_ARCH is normally put there by
# deploy.bbclass
do_deploy[stamp-extra-info] = ""
diff --git a/meta/recipes-core/ovmf/ovmf_git.bb
b/meta/recipes-core/ovmf/ovmf_git.bb
index b569b593fc..2c804381ee 100644
--- a/meta/recipes-core/ovmf/ovmf_git.bb
+++ b/meta/recipes-core/ovmf/ovmf_git.bb
@@ -223,7 +223,6 @@ do_deploy[depends] += "${DEPLOYDEP}"
do_deploy() {
}
-do_deploy[cleandirs] = "${DEPLOYDIR}"
do_deploy_class-target() {
# For use with "runqemu ovmf".
for i in \
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 1/2] uboot-sign: Refactor do_deploy prefunc to do_deploy_prepend
bitbake removes cleandirs once per prefunc and then again for the actual
task. By moving the concat_dtb step here from prefunc to main task we can
add
do_deploy[cleandirs] = "${DEPLOYDIR}"
to deploy.bbclass without losing the files produced by concat_dtb.
It looks like using do_deploy(_append) was the original goal anyways.
I tested this with poky, putting the following in local.conf:
UBOOT_SIGN_ENABLE = "1"
KERNEL_CLASSES = " kernel-fitimage "
KERNEL_IMAGETYPE = "fitImage"
and then doing
bitbake core-image-minimal
bitbake u-boot
It builds successfully, the kernel and uboot recipes' temp/run.do_deploy
scripts look good (kernel's do_deploy is not broken, and uboot's do_deploy
still calls concat_dtb).
Signed-off-by: Daniel Klauer
---
meta/classes/uboot-sign.bbclass | 11 ++-
1 file changed, 6 insertions(+), 5 deletions(-)
diff --git a/meta/classes/uboot-sign.bbclass b/meta/classes/uboot-sign.bbclass
index 982ed46d01..713196df41 100644
--- a/meta/classes/uboot-sign.bbclass
+++ b/meta/classes/uboot-sign.bbclass
@@ -117,15 +117,16 @@ do_install_append() {
fi
}
+do_deploy_prepend_pn-${UBOOT_PN}() {
+ if [ "${UBOOT_SIGN_ENABLE}" = "1" -a -n "${UBOOT_DTB_BINARY}" ]; then
+ concat_dtb
+ fi
+}
+
python () {
if d.getVar('UBOOT_SIGN_ENABLE') == '1' and d.getVar('PN') ==
d.getVar('UBOOT_PN') and d.getVar('UBOOT_DTB_BINARY'):
kernel_pn = d.getVar('PREFERRED_PROVIDER_virtual/kernel')
# Make "bitbake u-boot -cdeploy" deploys the signed u-boot.dtb
d.appendVarFlag('do_deploy', 'depends', ' %s:do_deploy' % kernel_pn)
-
-# kernerl's do_deploy is a litle special, so we can't use
-# do_deploy_append, otherwise it would override
-# kernel_do_deploy.
-d.appendVarFlag('do_deploy', 'prefuncs', ' concat_dtb')
}
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 1/2] uboot-sign: Refactor do_deploy prefunc to do_deploy_prepend
> That commit message isn't correct.
>
> It removes cleandirs for the prefunc at the start of the prefunc and
> cleandirs for the main function at the start of that main function.
Oh ok, so the prefuncs/postfuncs can have their own [cleandirs], that's
interesting.
How about the following (trying to find a good way to explain it):
-
When inherited by the u-boot recipe (UBOOT_PN), uboot-sign.bbclass adds
a concat_dtb step, which places additional files into ${DEPLOYDIR}
before do_deploy.
The use of prefuncs for this prevents us from adding
do_deploy[cleandirs] = "${DEPLOYDIR}"
because that would remove the files produced by the prefunc.
(Each of prefuncs/task/postfuncs functions has their own cleandirs, and
each function's cleandirs are removed at the start of that function.)
Thus, it seems good to make concat_dtb a part of do_deploy, such that it
runs after removal of do_deploy's cleandirs. As before, care is taken to
not interfere with the kernel's do_deploy definition, since concat_dtb
was only needed for u-boot.
-
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v2 1/2] uboot-sign: Refactor do_deploy prefunc to do_deploy_prepend
When inherited by the u-boot recipe (UBOOT_PN), uboot-sign.bbclass adds
a concat_dtb step, which places additional files into ${DEPLOYDIR}
before do_deploy.
The use of prefuncs for this prevents us from adding
do_deploy[cleandirs] = "${DEPLOYDIR}"
because that would remove the files produced by the prefunc.
Thus, it seems good to make concat_dtb a part of do_deploy, such that it
runs after removal of do_deploy's cleandirs. As before, care is taken to
not interfere with the kernel's do_deploy definition, since concat_dtb
was only needed for u-boot.
I tested this with poky, putting the following in local.conf:
UBOOT_SIGN_ENABLE = "1"
KERNEL_CLASSES = " kernel-fitimage "
KERNEL_IMAGETYPE = "fitImage"
and then doing
bitbake core-image-minimal
bitbake u-boot
It built successfully and concat_dtb was still called.
Signed-off-by: Daniel Klauer
---
meta/classes/uboot-sign.bbclass | 11 ++-
1 file changed, 6 insertions(+), 5 deletions(-)
diff --git a/meta/classes/uboot-sign.bbclass b/meta/classes/uboot-sign.bbclass
index 982ed46d01..713196df41 100644
--- a/meta/classes/uboot-sign.bbclass
+++ b/meta/classes/uboot-sign.bbclass
@@ -117,15 +117,16 @@ do_install_append() {
fi
}
+do_deploy_prepend_pn-${UBOOT_PN}() {
+ if [ "${UBOOT_SIGN_ENABLE}" = "1" -a -n "${UBOOT_DTB_BINARY}" ]; then
+ concat_dtb
+ fi
+}
+
python () {
if d.getVar('UBOOT_SIGN_ENABLE') == '1' and d.getVar('PN') ==
d.getVar('UBOOT_PN') and d.getVar('UBOOT_DTB_BINARY'):
kernel_pn = d.getVar('PREFERRED_PROVIDER_virtual/kernel')
# Make "bitbake u-boot -cdeploy" deploys the signed u-boot.dtb
d.appendVarFlag('do_deploy', 'depends', ' %s:do_deploy' % kernel_pn)
-
-# kernerl's do_deploy is a litle special, so we can't use
-# do_deploy_append, otherwise it would override
-# kernel_do_deploy.
-d.appendVarFlag('do_deploy', 'prefuncs', ' concat_dtb')
}
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v2] image.bbclass: Fix debug output for rootfs size
The debug output showed the wrong variable. Signed-off-by: Daniel Klauer --- meta/classes/image.bbclass | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass index 7fa4ff20bd..f4633da3d5 100644 --- a/meta/classes/image.bbclass +++ b/meta/classes/image.bbclass @@ -526,7 +526,7 @@ def get_rootfs_size(d): base_size = size_kb * overhead_factor bb.debug(1, '%f = %d * %f' % (base_size, size_kb, overhead_factor)) base_size2 = max(base_size, rootfs_req_size) + rootfs_extra_space -bb.debug(1, '%f = max(%f, %d)[%f] + %d' % (base_size2, base_size, rootfs_req_size, max(base_size, rootfs_req_size), overhead_factor)) +bb.debug(1, '%f = max(%f, %d)[%f] + %d' % (base_size2, base_size, rootfs_req_size, max(base_size, rootfs_req_size), rootfs_extra_space)) base_size = base_size2 if base_size != int(base_size): -- 2.17.1 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 2/2] deploy.bbclass: Clean DEPLOYDIR before do_deploy
Ping - any chance of getting this merged? (refering to https://patchwork.openembedded.org/series/19533/) I think it would still be useful, especially for meta-freescale, which uses do_deploy() in many recipes. If not, then I can update the meta-freescale pull request that triggered this patch (https://github.com/Freescale/meta-freescale/pull/139). Thanks. -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#140014): https://lists.openembedded.org/g/openembedded-core/message/140014 Mute This Topic: https://lists.openembedded.org/mt/72383135/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH 1/2] uboot-sign: Refactor do_deploy prefunc to do_deploy_prepend
When inherited by the u-boot recipe (UBOOT_PN), uboot-sign.bbclass adds
a concat_dtb step, which places additional files into ${DEPLOYDIR}
before do_deploy. By turning this from a prefunc into a part of the normal
do_deploy function, it becomes possible to use
do_deploy[cleandirs] = "${DEPLOYDIR}"
in the future, without deleting the files produced by concat_dtb.
As before, care is taken to not interfere with the kernel's do_deploy
definition, since concat_dtb was only needed for u-boot.
Signed-off-by: Daniel Klauer
---
meta/classes/uboot-sign.bbclass | 11 ++-
1 file changed, 6 insertions(+), 5 deletions(-)
diff --git a/meta/classes/uboot-sign.bbclass b/meta/classes/uboot-sign.bbclass
index 982ed46d01..713196df41 100644
--- a/meta/classes/uboot-sign.bbclass
+++ b/meta/classes/uboot-sign.bbclass
@@ -117,15 +117,16 @@ do_install_append() {
fi
}
+do_deploy_prepend_pn-${UBOOT_PN}() {
+ if [ "${UBOOT_SIGN_ENABLE}" = "1" -a -n "${UBOOT_DTB_BINARY}" ]; then
+ concat_dtb
+ fi
+}
+
python () {
if d.getVar('UBOOT_SIGN_ENABLE') == '1' and d.getVar('PN') ==
d.getVar('UBOOT_PN') and d.getVar('UBOOT_DTB_BINARY'):
kernel_pn = d.getVar('PREFERRED_PROVIDER_virtual/kernel')
# Make "bitbake u-boot -cdeploy" deploys the signed u-boot.dtb
d.appendVarFlag('do_deploy', 'depends', ' %s:do_deploy' % kernel_pn)
-
-# kernerl's do_deploy is a litle special, so we can't use
-# do_deploy_append, otherwise it would override
-# kernel_do_deploy.
-d.appendVarFlag('do_deploy', 'prefuncs', ' concat_dtb')
}
--
2.20.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#140132):
https://lists.openembedded.org/g/openembedded-core/message/140132
Mute This Topic: https://lists.openembedded.org/mt/75211795/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH 2/2] deploy.bbclass: Clean DEPLOYDIR before do_deploy
do_deploy should clean up ${DEPLOYDIR} before running, just like do_install
cleans up ${D} before running. This reduces the risk of DEPLOYDIR being
accidentally contaminated by files from previous runs, possibly even with
different config, in case of incremental builds.
It is convenient to have this in deploy.bbclass, so it doesn't have to be
duplicated in every recipe, considering for example meta-freescale, which
has 23 affected recipes.
All recipes using deploy.bbclass (grep -r 'inherit .*deploy') in poky,
meta-openembedded and meta-freescale look like they either benefit from
this or are at least not affected negatively by it. The only exception
I've noticed was uboot-sign.bbclass, which was however fixed by the
previous patch.
Signed-off-by: Daniel Klauer
---
meta/classes/deploy.bbclass| 1 +
meta/classes/kernel.bbclass| 2 --
meta/recipes-core/meta/signing-keys.bb | 2 --
meta/recipes-core/ovmf/ovmf_git.bb | 1 -
4 files changed, 1 insertion(+), 5 deletions(-)
diff --git a/meta/classes/deploy.bbclass b/meta/classes/deploy.bbclass
index 6d52908783..737c26122b 100644
--- a/meta/classes/deploy.bbclass
+++ b/meta/classes/deploy.bbclass
@@ -8,4 +8,5 @@ python do_deploy_setscene () {
}
addtask do_deploy_setscene
do_deploy[dirs] = "${DEPLOYDIR} ${B}"
+do_deploy[cleandirs] = "${DEPLOYDIR}"
do_deploy[stamp-extra-info] = "${MACHINE_ARCH}"
diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index a1219f3156..301e3b86b2 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -729,8 +729,6 @@ kernel_do_deploy() {
done
fi
}
-do_deploy[cleandirs] = "${DEPLOYDIR}"
-do_deploy[dirs] = "${DEPLOYDIR} ${B}"
do_deploy[prefuncs] += "package_get_auto_pr"
addtask deploy after do_populate_sysroot do_packagedata
diff --git a/meta/recipes-core/meta/signing-keys.bb
b/meta/recipes-core/meta/signing-keys.bb
index 1e1c7e3459..5bab94aa36 100644
--- a/meta/recipes-core/meta/signing-keys.bb
+++ b/meta/recipes-core/meta/signing-keys.bb
@@ -67,8 +67,6 @@ do_deploy () {
fi
}
do_deploy[sstate-outputdirs] = "${DEPLOY_DIR_RPM}"
-# cleandirs should possibly be in deploy.bbclass but we need it
-do_deploy[cleandirs] = "${DEPLOYDIR}"
# clear stamp-extra-info since MACHINE_ARCH is normally put there by
# deploy.bbclass
do_deploy[stamp-extra-info] = ""
diff --git a/meta/recipes-core/ovmf/ovmf_git.bb
b/meta/recipes-core/ovmf/ovmf_git.bb
index 7292ebbb77..84547630e8 100644
--- a/meta/recipes-core/ovmf/ovmf_git.bb
+++ b/meta/recipes-core/ovmf/ovmf_git.bb
@@ -220,7 +220,6 @@ do_deploy[depends] += "${DEPLOYDEP}"
do_deploy() {
}
-do_deploy[cleandirs] = "${DEPLOYDIR}"
do_deploy_class-target() {
# For use with "runqemu ovmf".
for i in \
--
2.20.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#140133):
https://lists.openembedded.org/g/openembedded-core/message/140133
Mute This Topic: https://lists.openembedded.org/mt/75211798/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] package.bbclass behaviour with multiple providers of a shared library
Hi oe-core devs, looking at package.bbclass and in particular the read_shlib_providers() function (although, without having fully understood it): For scanning the SHLIBSDIRS, it uses os.listdir() but seemingly without sorting. There is a comment saying "the last one found wins". Couldn't this cause "undefined" behaviour in case there are multiple providers of a shared library? (meaning sometimes one provider wins, sometimes the others, since they are in separate files in the SHLIBSDIRS) Specifically, I saw a build failure, which I can't seem to reproduce anymore now, with meta-freescale's openssl-qoriq, which packages libcrypto.so.1.1/libssl.so.1.1 in libcrypto/libssl packages but also in a openssl-qoriq-ptest package. .../tmp/pkgdata/lx2160ardb/shlibs2$ grep -r 'libssl\.so\.1\.1\|libcrypto\.so\.1\.1' openssl-qoriq-ptest.list:libssl.so.1.1:/usr/lib/openssl-qoriq/ptest:1.1.0g openssl-qoriq-ptest.list:libcrypto.so.1.1:/usr/lib/openssl-qoriq/ptest:1.1.0g libcrypto.list:libcrypto.so.1.1:/usr/lib:1.1.0g libssl.list:libssl.so.1.1:/usr/lib:1.1.0g Kind regards, Daniel pEpkey.asc Description: application/pgp-keys -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [warrior][PATCH] package: Improve determinism
From: Richard Purdie
Its possible in cases with multiple shlib providers we were not being
deterministic. Add in a couple of sorted() calls to fix the shlibs and
pkgconfig cases with this potential issue.
Signed-off-by: Richard Purdie
---
meta/classes/package.bbclass | 2 +-
meta/lib/oe/package.py | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/meta/classes/package.bbclass b/meta/classes/package.bbclass
index 8b89fb1129..e67bb5bd97 100644
--- a/meta/classes/package.bbclass
+++ b/meta/classes/package.bbclass
@@ -1924,7 +1924,7 @@ python package_do_pkgconfig () {
for dir in reversed(shlibs_dirs):
if not os.path.exists(dir):
continue
-for file in os.listdir(dir):
+for file in sorted(os.listdir(dir)):
m = re.match(r'^(.*)\.pclist$', file)
if m:
pkg = m.group(1)
diff --git a/meta/lib/oe/package.py b/meta/lib/oe/package.py
index b59513227b..b8585d4253 100644
--- a/meta/lib/oe/package.py
+++ b/meta/lib/oe/package.py
@@ -265,7 +265,7 @@ def read_shlib_providers(d):
bb.debug(2, "Reading shlib providers in %s" % (dir))
if not os.path.exists(dir):
continue
-for file in os.listdir(dir):
+for file in sorted(os.listdir(dir)):
m = list_re.match(file)
if m:
dep_pkg = m.group(1)
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] package.bbclass behaviour with multiple providers of a shared library
On 25.07.19 23:57, Richard Purdie wrote: > We definitely should sort to make things deterministic. Since I don't > want to forget this I've added a patch into my testing queue. OK, thanks! It would be great to have the patch backported to warrior too, so I just re-sent your patch for [warrior] for that. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH] image.bbclass: Fix debug output for rootfs size
The debug output didn't match the calculation - it showed a wrong value. --- meta/classes/image.bbclass | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass index 682858dc95..25b0ead9bf 100644 --- a/meta/classes/image.bbclass +++ b/meta/classes/image.bbclass @@ -529,7 +529,7 @@ def get_rootfs_size(d): base_size = size_kb * overhead_factor bb.debug(1, '%f = %d * %f' % (base_size, size_kb, overhead_factor)) base_size2 = max(base_size, rootfs_req_size) + rootfs_extra_space -bb.debug(1, '%f = max(%f, %d)[%f] + %d' % (base_size2, base_size, rootfs_req_size, max(base_size, rootfs_req_size), overhead_factor)) +bb.debug(1, '%f = max(%f, %d)[%f] + %d' % (base_size2, base_size, rootfs_req_size, max(base_size, rootfs_req_size), rootfs_extra_space)) base_size = base_size2 if base_size != int(base_size): -- 2.17.1 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH] cmake.bbclass: Add ${COREBASE}/scripts to CMAKE_FIND_ROOT_PATH
${COREBASE}/scripts contains a "git" wrapper disabling fakeroot/pseudo.
This patch allows CMake to find ${COREBASE}/scripts/git instead of
${HOSTTOOLS_DIR}/git. This is needed for git invocations during do_install,
since do_install is a fakeroot task, and otherwise all git commands fail
with "fatal: detected dubious ownership in repository ...".
I don't know how common it is for CMake projects to invoke git during the
install phase intentionally. It's probably more common to do this during
the configure phase. However, the install step may re-run the configure
step, if some dependencies changed.
In my case, this happened in incremental Yocto builds which reran
do_install and repopulated parts of the recipe-sysroot during that,
without first rerunning do_configure or do_compile. One of the dependencies
changed (but only changing a file in some unrelated sub-package of it which
was not even installed into the recipe-sysroot), causing the dependant's
recipe-sysroot to be repopulated during do_install and thus causing the
CMake project to be reconfigured during do_install.
Signed-off-by: Daniel Klauer
---
meta/classes-recipe/cmake.bbclass | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/meta/classes-recipe/cmake.bbclass
b/meta/classes-recipe/cmake.bbclass
index 3d3781ef33..e1c3d7ddb5 100644
--- a/meta/classes-recipe/cmake.bbclass
+++ b/meta/classes-recipe/cmake.bbclass
@@ -124,7 +124,7 @@ set( CMAKE_CXX_LINK_FLAGS "${OECMAKE_CXX_LINK_FLAGS}" CACHE
STRING "LDFLAGS" )
# only search in the paths provided so cmake doesnt pick
# up libraries and tools from the native build machine
-set( CMAKE_FIND_ROOT_PATH ${STAGING_DIR_HOST} ${STAGING_DIR_NATIVE}
${CROSS_DIR} ${OECMAKE_PERLNATIVE_DIR} ${OECMAKE_EXTRA_ROOT_PATH}
${EXTERNAL_TOOLCHAIN} ${HOSTTOOLS_DIR})
+set( CMAKE_FIND_ROOT_PATH ${STAGING_DIR_HOST} ${STAGING_DIR_NATIVE}
${CROSS_DIR} ${OECMAKE_PERLNATIVE_DIR} ${OECMAKE_EXTRA_ROOT_PATH}
${EXTERNAL_TOOLCHAIN} ${COREBASE}/scripts ${HOSTTOOLS_DIR} )
set( CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY )
set( CMAKE_FIND_ROOT_PATH_MODE_PROGRAM ${OECMAKE_FIND_ROOT_PATH_MODE_PROGRAM} )
set( CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY )
--
2.34.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#198462):
https://lists.openembedded.org/g/openembedded-core/message/198462
Mute This Topic: https://lists.openembedded.org/mt/105575001/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH] cmake.bbclass: Add ${COREBASE}/scripts to CMAKE_FIND_ROOT_PATH
Hi,
>> ${COREBASE}/scripts contains a "git" wrapper disabling fakeroot/pseudo.
>> This patch allows CMake to find ${COREBASE}/scripts/git instead of
>> ${HOSTTOOLS_DIR}/git. This is needed for git invocations during do_install,
>> since do_install is a fakeroot task, and otherwise all git commands fail
>> with "fatal: detected dubious ownership in repository ...".
>>
>> I don't know how common it is for CMake projects to invoke git during the
>> install phase intentionally. It's probably more common to do this during
>> the configure phase. However, the install step may re-run the configure
>> step, if some dependencies changed.
>
>
> If a package build system has such dependencies they should be better
> expressed in recipes and linked using bitbake task level dependency mechanism
>
> I wonder why git is needed during install phase
> Can you explain a bit more on that
>
> And since change it in a class it means it has to be a pattern
It's just about running "git rev-parse HEAD" or similar, not acquiring external
dependencies with "git clone", FetchContent download, etc., if that's what you
meant. "dependencies" referred to timestamps of files checked by make/ninja.
The affected CMake projects are using code like the following to query Git
revision information (for storing into header files and binaries):
find_package(Git REQUIRED)
execute_process(COMMAND "${GIT_EXECUTABLE}" rev-parse HEAD ...)
set_property(DIRECTORY APPEND PROPERTY CMAKE_CONFIGURE_DEPENDS
"${CMAKE_CURRENT_SOURCE_DIR}/.git/HEAD")
${S} is owned by the user that runs bitbake, so cmake succeeds running such git
commands during do_configure, but they fail during do_install because it's
under fakeroot/pseudo.
This can be fixed by allowing CMake to use the git wrapper script which
temporarily disables fakeroot. It's already in PATH anyways (see PATH:prepend =
"${COREBASE}/scripts..." in bitbake.conf), but CMake's find_package(Git) only
respects CMAKE_FIND_ROOT_PATH. The description of commit af27c81e "scripts:
Make git intercept global" hints at this type of situation.
As for why cmake re-runs during do_install, I'm still trying to piece it
together and reduce it to a minimal example. I can try to explain what I know
so far:
Basically, it's normal for a CMake project to be re-configured by "make" or
"make install" (or ninja equivalents), if for example the CMakeLists.txt or
files referenced by CMAKE_CONFIGURE_DEPENDS were modified, and an existing
build directory was re-used. I'm not really surprised to see this case
occurring in Yocto aswell. Presumably it can only happen in incremental Yocto
builds if do_fetch/do_unpack/do_patch were already run previously and don't
need to run again.
What I did:
1. have a recipe for a library
2. have a recipe for a program which depends on that library (and of courses
uses DEPENDS += "library")
3. run bitbake to build the image
4. modify library:do_install (to install a different configuration file into
/etc)
5. modify program:do_install (same)
6. run bitbake to build the image
My observations for the 2nd bitbake run (after I modified the two do_install
functions):
6.1. library:do_configure did not run again (makes sense, since I didn't change
anything there)
6.2. library:do_compile did not run again (same)
6.3. library:do_install was re-run (due to the changes for the new config file
in /etc)
6.4. library:do_package was re-run (but presumably only the main library
package changed, not the library-dev or library-static packages)
6.5. program:do_configure did not run again (not sure why not, since library
recipe changed, but maybe bitbake noticed through the library packaging that
only /etc changed and not the library or its header files)
6.6. program:do_compile did not run again (same)
6.7. program:do_install was re-run
6.7.1. The library was re-populated into the program's recipe-sysroot ("NOTE:
exists in sysroot, but is stale")
6.7.2. "ninja install" re-ran CMake to re-configure the project (I'm guessing
due to the re-population of the recipe-sysroot), this is where the git commands
failed.
Regards,
Daniel
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#198548):
https://lists.openembedded.org/g/openembedded-core/message/198548
Mute This Topic: https://lists.openembedded.org/mt/105575001/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
