Re: [OE-core] boost 1.69 -> 1.71 upgrade for 3.0?
Hi, On 22.08.19 21:19, Randy MacLeod wrote: > I'm checking if you have done any work on upgrading the > boost recipe? Not long ago I've looked at updating the boost recipe to 1.71.0-beta1 (when 1.71.0 wasn't released yet) because of some bugs encountered with 1.69.0 (and 1.70.0 being buggy too). See the attached patch (it is based on poky warrior). It was not trivial due to changes to bjam bootstrap scripts, but I got it working. I'd be interested in updating it to the final 1.71.0 too, including backporting to warrior. Regards, Daniel >From 2f8eb6379ff926cc53d1b2a24a2b93c5ea5268fa Mon Sep 17 00:00:00 2001 From: Daniel Klauer Date: Mon, 26 Aug 2019 11:01:00 +0200 Subject: [PATCH] boost: Update to 1.71.0-beta1 Some notes: - bjam bootstrap scripts were changed, and the patch for that had to be adjusted. tools/build/src/engine/build.sh now supports an --debug option which the bjam-native recipe can use to get the debug build of b2. - gcc.jam patch had to be refreshed because boost added more "cpu-flags" lines. - since 1.70.0, boost includes new cmake config files which are packaged in ${PN}-dev now so they make it into the SDK. (although 1.71.0 is needed to fix some bugs in them) - The recipe had to be adjusted more than usual to support the boost beta version. --- ..._1.69.0.bb => bjam-native_1.71.0.beta1.bb} | 5 +- ...oost-1.69.0.inc => boost-1.71.0.beta1.inc} | 12 ++--- meta/recipes-support/boost/boost.inc | 2 +- ...h-instruction-set-flags-we-do-that-o.patch | 23 + ...{boost_1.69.0.bb => boost_1.71.0.beta1.bb} | 0 .../0001-Build-debug-version-of-bjam.patch| 25 ++ .../files/bjam-native-build-bjam.debug.patch | 50 --- 7 files changed, 45 insertions(+), 72 deletions(-) rename meta/recipes-support/boost/{bjam-native_1.69.0.bb => bjam-native_1.71.0.beta1.bb} (70%) rename meta/recipes-support/boost/{boost-1.69.0.inc => boost-1.71.0.beta1.inc} (59%) rename meta/recipes-support/boost/{boost_1.69.0.bb => boost_1.71.0.beta1.bb} (100%) create mode 100644 meta/recipes-support/boost/files/0001-Build-debug-version-of-bjam.patch delete mode 100644 meta/recipes-support/boost/files/bjam-native-build-bjam.debug.patch diff --git a/meta/recipes-support/boost/bjam-native_1.69.0.bb b/meta/recipes-support/boost/bjam-native_1.71.0.beta1.bb similarity index 70% rename from meta/recipes-support/boost/bjam-native_1.69.0.bb rename to meta/recipes-support/boost/bjam-native_1.71.0.beta1.bb index 94f96e62d8..30279a73e6 100644 --- a/meta/recipes-support/boost/bjam-native_1.69.0.bb +++ b/meta/recipes-support/boost/bjam-native_1.71.0.beta1.bb @@ -5,8 +5,7 @@ SECTION = "devel" inherit native -SRC_URI += "file://bjam-native-build-bjam.debug.patch \ -" +SRC_URI += "file://0001-Build-debug-version-of-bjam.patch" do_compile() { ./bootstrap.sh --with-toolset=gcc @@ -15,5 +14,5 @@ do_compile() { do_install() { install -d ${D}${bindir}/ # install unstripped version for bjam -install -c -m 755 bjam.debug ${D}${bindir}/bjam +install -c -m 755 b2 ${D}${bindir}/bjam } diff --git a/meta/recipes-support/boost/boost-1.69.0.inc b/meta/recipes-support/boost/boost-1.71.0.beta1.inc similarity index 59% rename from meta/recipes-support/boost/boost-1.69.0.inc rename to meta/recipes-support/boost/boost-1.71.0.beta1.inc index 923436b1e0..d2e2a11457 100644 --- a/meta/recipes-support/boost/boost-1.69.0.inc +++ b/meta/recipes-support/boost/boost-1.71.0.beta1.inc @@ -7,15 +7,11 @@ HOMEPAGE = "http://www.boost.org/"; LICENSE = "BSL-1.0 & MIT & Python-2.0" LIC_FILES_CHKSUM = "file://LICENSE_1_0.txt;md5=e4224ccaecb14d942c71d31bef20d78c" -BOOST_VER = "${@"_".join(d.getVar("PV").split("."))}" -BOOST_MAJ = "${@"_".join(d.getVar("PV").split(".")[0:2])}" -BOOST_P = "boost_${BOOST_VER}" - -SRC_URI = "${SOURCEFORGE_MIRROR}/project/boost/boost/${PV}/${BOOST_P}.tar.bz2" -SRC_URI[md5sum] = "a1332494397bf48332cb152abfefcec2" -SRC_URI[sha256sum] = "8f32d4617390d1c2d16f26a27ab60d97807b35440d45891fa340fc2648b04406" +SRC_URI = "${SOURCEFORGE_MIRROR}/project/boost/boost/${PV}/boost_1_71_0_b1.tar.bz2" +SRC_URI[md5sum] = "f74e2064eef26e108f304680b21a9a82" +SRC_URI[sha256sum] = "a93d7066fc1514995aec0e726532a219447dd39b0ffa7cb97ca5736f16b0488b" UPSTREAM_CHECK_URI = "http://www.boost.org/users/download/"; UPSTREAM_CHECK_REGEX = "boostorg/release/(?P.*)/source/" -S = "${WORKDIR}/${BOOST_P}" +S = "${WORKDIR}/boost_1_71_0" diff --git a/meta/recipes-support/boost/boost.inc b/meta/recipes-support/boost/boost.inc index 9be3717fd6..6de940265c 100644 --- a/meta/recipes-support/boost/boost.inc +++ b/meta/recipes-support/boost/boost.inc @@ -80,7 +80,7 @@ FILE
[OE-core] [PATCH 2/2] deploy.bbclass: Clean DEPLOYDIR before do_deploy
It seems convenient for do_deploy to clean up ${DEPLOYDIR} (its output directory) before running, just like do_install cleans up ${D} before running. This way we can be sure that a recipe's do_deploy output is not accidentally contaminated by previously existing files in DEPLOYDIR in case of incremental builds. All recipes using deploy.bbclass (grep -r 'inherit .*deploy') in poky, meta-openembedded and meta-freescale look to me like they either benefit from this or are at least not affected negatively by it. The only exception I've noticed was uboot-sign.bbclass, which was however fixed by the previous patch. I tested this by running "bitbake virtual/kernel u-boot": 1. on patched, but clean poky source tree 2. after adding "touch ${DEPLOYDIR}/foo-${PN}" to linux-yocto's/u-boot's do_deploy, to manually trigger an incremental build of do_deploy only. 3. after removing the modifications again. The foo-${PN} files existed in tmp/deploy/images/qemux86-64 after step 2, and not anymore after step 3, as expected. Thus the cleandirs still works. Signed-off-by: Daniel Klauer --- meta/classes/deploy.bbclass| 1 + meta/classes/kernel.bbclass| 2 -- meta/recipes-core/meta/signing-keys.bb | 2 -- meta/recipes-core/ovmf/ovmf_git.bb | 1 - 4 files changed, 1 insertion(+), 5 deletions(-) diff --git a/meta/classes/deploy.bbclass b/meta/classes/deploy.bbclass index 6d52908783..737c26122b 100644 --- a/meta/classes/deploy.bbclass +++ b/meta/classes/deploy.bbclass @@ -8,4 +8,5 @@ python do_deploy_setscene () { } addtask do_deploy_setscene do_deploy[dirs] = "${DEPLOYDIR} ${B}" +do_deploy[cleandirs] = "${DEPLOYDIR}" do_deploy[stamp-extra-info] = "${MACHINE_ARCH}" diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass index bf3674238f..aa339ace7e 100644 --- a/meta/classes/kernel.bbclass +++ b/meta/classes/kernel.bbclass @@ -706,8 +706,6 @@ kernel_do_deploy() { done fi } -do_deploy[cleandirs] = "${DEPLOYDIR}" -do_deploy[dirs] = "${DEPLOYDIR} ${B}" do_deploy[prefuncs] += "package_get_auto_pr" addtask deploy after do_populate_sysroot do_packagedata diff --git a/meta/recipes-core/meta/signing-keys.bb b/meta/recipes-core/meta/signing-keys.bb index 1e1c7e3459..5bab94aa36 100644 --- a/meta/recipes-core/meta/signing-keys.bb +++ b/meta/recipes-core/meta/signing-keys.bb @@ -67,8 +67,6 @@ do_deploy () { fi } do_deploy[sstate-outputdirs] = "${DEPLOY_DIR_RPM}" -# cleandirs should possibly be in deploy.bbclass but we need it -do_deploy[cleandirs] = "${DEPLOYDIR}" # clear stamp-extra-info since MACHINE_ARCH is normally put there by # deploy.bbclass do_deploy[stamp-extra-info] = "" diff --git a/meta/recipes-core/ovmf/ovmf_git.bb b/meta/recipes-core/ovmf/ovmf_git.bb index b569b593fc..2c804381ee 100644 --- a/meta/recipes-core/ovmf/ovmf_git.bb +++ b/meta/recipes-core/ovmf/ovmf_git.bb @@ -223,7 +223,6 @@ do_deploy[depends] += "${DEPLOYDEP}" do_deploy() { } -do_deploy[cleandirs] = "${DEPLOYDIR}" do_deploy_class-target() { # For use with "runqemu ovmf". for i in \ -- 2.17.1 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH 1/2] uboot-sign: Refactor do_deploy prefunc to do_deploy_prepend
bitbake removes cleandirs once per prefunc and then again for the actual task. By moving the concat_dtb step here from prefunc to main task we can add do_deploy[cleandirs] = "${DEPLOYDIR}" to deploy.bbclass without losing the files produced by concat_dtb. It looks like using do_deploy(_append) was the original goal anyways. I tested this with poky, putting the following in local.conf: UBOOT_SIGN_ENABLE = "1" KERNEL_CLASSES = " kernel-fitimage " KERNEL_IMAGETYPE = "fitImage" and then doing bitbake core-image-minimal bitbake u-boot It builds successfully, the kernel and uboot recipes' temp/run.do_deploy scripts look good (kernel's do_deploy is not broken, and uboot's do_deploy still calls concat_dtb). Signed-off-by: Daniel Klauer --- meta/classes/uboot-sign.bbclass | 11 ++- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/meta/classes/uboot-sign.bbclass b/meta/classes/uboot-sign.bbclass index 982ed46d01..713196df41 100644 --- a/meta/classes/uboot-sign.bbclass +++ b/meta/classes/uboot-sign.bbclass @@ -117,15 +117,16 @@ do_install_append() { fi } +do_deploy_prepend_pn-${UBOOT_PN}() { + if [ "${UBOOT_SIGN_ENABLE}" = "1" -a -n "${UBOOT_DTB_BINARY}" ]; then + concat_dtb + fi +} + python () { if d.getVar('UBOOT_SIGN_ENABLE') == '1' and d.getVar('PN') == d.getVar('UBOOT_PN') and d.getVar('UBOOT_DTB_BINARY'): kernel_pn = d.getVar('PREFERRED_PROVIDER_virtual/kernel') # Make "bitbake u-boot -cdeploy" deploys the signed u-boot.dtb d.appendVarFlag('do_deploy', 'depends', ' %s:do_deploy' % kernel_pn) - -# kernerl's do_deploy is a litle special, so we can't use -# do_deploy_append, otherwise it would override -# kernel_do_deploy. -d.appendVarFlag('do_deploy', 'prefuncs', ' concat_dtb') } -- 2.17.1 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 1/2] uboot-sign: Refactor do_deploy prefunc to do_deploy_prepend
> That commit message isn't correct. > > It removes cleandirs for the prefunc at the start of the prefunc and > cleandirs for the main function at the start of that main function. Oh ok, so the prefuncs/postfuncs can have their own [cleandirs], that's interesting. How about the following (trying to find a good way to explain it): - When inherited by the u-boot recipe (UBOOT_PN), uboot-sign.bbclass adds a concat_dtb step, which places additional files into ${DEPLOYDIR} before do_deploy. The use of prefuncs for this prevents us from adding do_deploy[cleandirs] = "${DEPLOYDIR}" because that would remove the files produced by the prefunc. (Each of prefuncs/task/postfuncs functions has their own cleandirs, and each function's cleandirs are removed at the start of that function.) Thus, it seems good to make concat_dtb a part of do_deploy, such that it runs after removal of do_deploy's cleandirs. As before, care is taken to not interfere with the kernel's do_deploy definition, since concat_dtb was only needed for u-boot. - -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v2 1/2] uboot-sign: Refactor do_deploy prefunc to do_deploy_prepend
When inherited by the u-boot recipe (UBOOT_PN), uboot-sign.bbclass adds a concat_dtb step, which places additional files into ${DEPLOYDIR} before do_deploy. The use of prefuncs for this prevents us from adding do_deploy[cleandirs] = "${DEPLOYDIR}" because that would remove the files produced by the prefunc. Thus, it seems good to make concat_dtb a part of do_deploy, such that it runs after removal of do_deploy's cleandirs. As before, care is taken to not interfere with the kernel's do_deploy definition, since concat_dtb was only needed for u-boot. I tested this with poky, putting the following in local.conf: UBOOT_SIGN_ENABLE = "1" KERNEL_CLASSES = " kernel-fitimage " KERNEL_IMAGETYPE = "fitImage" and then doing bitbake core-image-minimal bitbake u-boot It built successfully and concat_dtb was still called. Signed-off-by: Daniel Klauer --- meta/classes/uboot-sign.bbclass | 11 ++- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/meta/classes/uboot-sign.bbclass b/meta/classes/uboot-sign.bbclass index 982ed46d01..713196df41 100644 --- a/meta/classes/uboot-sign.bbclass +++ b/meta/classes/uboot-sign.bbclass @@ -117,15 +117,16 @@ do_install_append() { fi } +do_deploy_prepend_pn-${UBOOT_PN}() { + if [ "${UBOOT_SIGN_ENABLE}" = "1" -a -n "${UBOOT_DTB_BINARY}" ]; then + concat_dtb + fi +} + python () { if d.getVar('UBOOT_SIGN_ENABLE') == '1' and d.getVar('PN') == d.getVar('UBOOT_PN') and d.getVar('UBOOT_DTB_BINARY'): kernel_pn = d.getVar('PREFERRED_PROVIDER_virtual/kernel') # Make "bitbake u-boot -cdeploy" deploys the signed u-boot.dtb d.appendVarFlag('do_deploy', 'depends', ' %s:do_deploy' % kernel_pn) - -# kernerl's do_deploy is a litle special, so we can't use -# do_deploy_append, otherwise it would override -# kernel_do_deploy. -d.appendVarFlag('do_deploy', 'prefuncs', ' concat_dtb') } -- 2.17.1 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v2] image.bbclass: Fix debug output for rootfs size
The debug output showed the wrong variable. Signed-off-by: Daniel Klauer --- meta/classes/image.bbclass | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass index 7fa4ff20bd..f4633da3d5 100644 --- a/meta/classes/image.bbclass +++ b/meta/classes/image.bbclass @@ -526,7 +526,7 @@ def get_rootfs_size(d): base_size = size_kb * overhead_factor bb.debug(1, '%f = %d * %f' % (base_size, size_kb, overhead_factor)) base_size2 = max(base_size, rootfs_req_size) + rootfs_extra_space -bb.debug(1, '%f = max(%f, %d)[%f] + %d' % (base_size2, base_size, rootfs_req_size, max(base_size, rootfs_req_size), overhead_factor)) +bb.debug(1, '%f = max(%f, %d)[%f] + %d' % (base_size2, base_size, rootfs_req_size, max(base_size, rootfs_req_size), rootfs_extra_space)) base_size = base_size2 if base_size != int(base_size): -- 2.17.1 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH 2/2] deploy.bbclass: Clean DEPLOYDIR before do_deploy
Ping - any chance of getting this merged? (refering to https://patchwork.openembedded.org/series/19533/) I think it would still be useful, especially for meta-freescale, which uses do_deploy() in many recipes. If not, then I can update the meta-freescale pull request that triggered this patch (https://github.com/Freescale/meta-freescale/pull/139). Thanks. -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#140014): https://lists.openembedded.org/g/openembedded-core/message/140014 Mute This Topic: https://lists.openembedded.org/mt/72383135/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH 1/2] uboot-sign: Refactor do_deploy prefunc to do_deploy_prepend
When inherited by the u-boot recipe (UBOOT_PN), uboot-sign.bbclass adds a concat_dtb step, which places additional files into ${DEPLOYDIR} before do_deploy. By turning this from a prefunc into a part of the normal do_deploy function, it becomes possible to use do_deploy[cleandirs] = "${DEPLOYDIR}" in the future, without deleting the files produced by concat_dtb. As before, care is taken to not interfere with the kernel's do_deploy definition, since concat_dtb was only needed for u-boot. Signed-off-by: Daniel Klauer --- meta/classes/uboot-sign.bbclass | 11 ++- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/meta/classes/uboot-sign.bbclass b/meta/classes/uboot-sign.bbclass index 982ed46d01..713196df41 100644 --- a/meta/classes/uboot-sign.bbclass +++ b/meta/classes/uboot-sign.bbclass @@ -117,15 +117,16 @@ do_install_append() { fi } +do_deploy_prepend_pn-${UBOOT_PN}() { + if [ "${UBOOT_SIGN_ENABLE}" = "1" -a -n "${UBOOT_DTB_BINARY}" ]; then + concat_dtb + fi +} + python () { if d.getVar('UBOOT_SIGN_ENABLE') == '1' and d.getVar('PN') == d.getVar('UBOOT_PN') and d.getVar('UBOOT_DTB_BINARY'): kernel_pn = d.getVar('PREFERRED_PROVIDER_virtual/kernel') # Make "bitbake u-boot -cdeploy" deploys the signed u-boot.dtb d.appendVarFlag('do_deploy', 'depends', ' %s:do_deploy' % kernel_pn) - -# kernerl's do_deploy is a litle special, so we can't use -# do_deploy_append, otherwise it would override -# kernel_do_deploy. -d.appendVarFlag('do_deploy', 'prefuncs', ' concat_dtb') } -- 2.20.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#140132): https://lists.openembedded.org/g/openembedded-core/message/140132 Mute This Topic: https://lists.openembedded.org/mt/75211795/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH 2/2] deploy.bbclass: Clean DEPLOYDIR before do_deploy
do_deploy should clean up ${DEPLOYDIR} before running, just like do_install cleans up ${D} before running. This reduces the risk of DEPLOYDIR being accidentally contaminated by files from previous runs, possibly even with different config, in case of incremental builds. It is convenient to have this in deploy.bbclass, so it doesn't have to be duplicated in every recipe, considering for example meta-freescale, which has 23 affected recipes. All recipes using deploy.bbclass (grep -r 'inherit .*deploy') in poky, meta-openembedded and meta-freescale look like they either benefit from this or are at least not affected negatively by it. The only exception I've noticed was uboot-sign.bbclass, which was however fixed by the previous patch. Signed-off-by: Daniel Klauer --- meta/classes/deploy.bbclass| 1 + meta/classes/kernel.bbclass| 2 -- meta/recipes-core/meta/signing-keys.bb | 2 -- meta/recipes-core/ovmf/ovmf_git.bb | 1 - 4 files changed, 1 insertion(+), 5 deletions(-) diff --git a/meta/classes/deploy.bbclass b/meta/classes/deploy.bbclass index 6d52908783..737c26122b 100644 --- a/meta/classes/deploy.bbclass +++ b/meta/classes/deploy.bbclass @@ -8,4 +8,5 @@ python do_deploy_setscene () { } addtask do_deploy_setscene do_deploy[dirs] = "${DEPLOYDIR} ${B}" +do_deploy[cleandirs] = "${DEPLOYDIR}" do_deploy[stamp-extra-info] = "${MACHINE_ARCH}" diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass index a1219f3156..301e3b86b2 100644 --- a/meta/classes/kernel.bbclass +++ b/meta/classes/kernel.bbclass @@ -729,8 +729,6 @@ kernel_do_deploy() { done fi } -do_deploy[cleandirs] = "${DEPLOYDIR}" -do_deploy[dirs] = "${DEPLOYDIR} ${B}" do_deploy[prefuncs] += "package_get_auto_pr" addtask deploy after do_populate_sysroot do_packagedata diff --git a/meta/recipes-core/meta/signing-keys.bb b/meta/recipes-core/meta/signing-keys.bb index 1e1c7e3459..5bab94aa36 100644 --- a/meta/recipes-core/meta/signing-keys.bb +++ b/meta/recipes-core/meta/signing-keys.bb @@ -67,8 +67,6 @@ do_deploy () { fi } do_deploy[sstate-outputdirs] = "${DEPLOY_DIR_RPM}" -# cleandirs should possibly be in deploy.bbclass but we need it -do_deploy[cleandirs] = "${DEPLOYDIR}" # clear stamp-extra-info since MACHINE_ARCH is normally put there by # deploy.bbclass do_deploy[stamp-extra-info] = "" diff --git a/meta/recipes-core/ovmf/ovmf_git.bb b/meta/recipes-core/ovmf/ovmf_git.bb index 7292ebbb77..84547630e8 100644 --- a/meta/recipes-core/ovmf/ovmf_git.bb +++ b/meta/recipes-core/ovmf/ovmf_git.bb @@ -220,7 +220,6 @@ do_deploy[depends] += "${DEPLOYDEP}" do_deploy() { } -do_deploy[cleandirs] = "${DEPLOYDIR}" do_deploy_class-target() { # For use with "runqemu ovmf". for i in \ -- 2.20.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#140133): https://lists.openembedded.org/g/openembedded-core/message/140133 Mute This Topic: https://lists.openembedded.org/mt/75211798/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] package.bbclass behaviour with multiple providers of a shared library
Hi oe-core devs, looking at package.bbclass and in particular the read_shlib_providers() function (although, without having fully understood it): For scanning the SHLIBSDIRS, it uses os.listdir() but seemingly without sorting. There is a comment saying "the last one found wins". Couldn't this cause "undefined" behaviour in case there are multiple providers of a shared library? (meaning sometimes one provider wins, sometimes the others, since they are in separate files in the SHLIBSDIRS) Specifically, I saw a build failure, which I can't seem to reproduce anymore now, with meta-freescale's openssl-qoriq, which packages libcrypto.so.1.1/libssl.so.1.1 in libcrypto/libssl packages but also in a openssl-qoriq-ptest package. .../tmp/pkgdata/lx2160ardb/shlibs2$ grep -r 'libssl\.so\.1\.1\|libcrypto\.so\.1\.1' openssl-qoriq-ptest.list:libssl.so.1.1:/usr/lib/openssl-qoriq/ptest:1.1.0g openssl-qoriq-ptest.list:libcrypto.so.1.1:/usr/lib/openssl-qoriq/ptest:1.1.0g libcrypto.list:libcrypto.so.1.1:/usr/lib:1.1.0g libssl.list:libssl.so.1.1:/usr/lib:1.1.0g Kind regards, Daniel pEpkey.asc Description: application/pgp-keys -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [warrior][PATCH] package: Improve determinism
From: Richard Purdie Its possible in cases with multiple shlib providers we were not being deterministic. Add in a couple of sorted() calls to fix the shlibs and pkgconfig cases with this potential issue. Signed-off-by: Richard Purdie --- meta/classes/package.bbclass | 2 +- meta/lib/oe/package.py | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/meta/classes/package.bbclass b/meta/classes/package.bbclass index 8b89fb1129..e67bb5bd97 100644 --- a/meta/classes/package.bbclass +++ b/meta/classes/package.bbclass @@ -1924,7 +1924,7 @@ python package_do_pkgconfig () { for dir in reversed(shlibs_dirs): if not os.path.exists(dir): continue -for file in os.listdir(dir): +for file in sorted(os.listdir(dir)): m = re.match(r'^(.*)\.pclist$', file) if m: pkg = m.group(1) diff --git a/meta/lib/oe/package.py b/meta/lib/oe/package.py index b59513227b..b8585d4253 100644 --- a/meta/lib/oe/package.py +++ b/meta/lib/oe/package.py @@ -265,7 +265,7 @@ def read_shlib_providers(d): bb.debug(2, "Reading shlib providers in %s" % (dir)) if not os.path.exists(dir): continue -for file in os.listdir(dir): +for file in sorted(os.listdir(dir)): m = list_re.match(file) if m: dep_pkg = m.group(1) -- 2.17.1 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] package.bbclass behaviour with multiple providers of a shared library
On 25.07.19 23:57, Richard Purdie wrote: > We definitely should sort to make things deterministic. Since I don't > want to forget this I've added a patch into my testing queue. OK, thanks! It would be great to have the patch backported to warrior too, so I just re-sent your patch for [warrior] for that. -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH] image.bbclass: Fix debug output for rootfs size
The debug output didn't match the calculation - it showed a wrong value. --- meta/classes/image.bbclass | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass index 682858dc95..25b0ead9bf 100644 --- a/meta/classes/image.bbclass +++ b/meta/classes/image.bbclass @@ -529,7 +529,7 @@ def get_rootfs_size(d): base_size = size_kb * overhead_factor bb.debug(1, '%f = %d * %f' % (base_size, size_kb, overhead_factor)) base_size2 = max(base_size, rootfs_req_size) + rootfs_extra_space -bb.debug(1, '%f = max(%f, %d)[%f] + %d' % (base_size2, base_size, rootfs_req_size, max(base_size, rootfs_req_size), overhead_factor)) +bb.debug(1, '%f = max(%f, %d)[%f] + %d' % (base_size2, base_size, rootfs_req_size, max(base_size, rootfs_req_size), rootfs_extra_space)) base_size = base_size2 if base_size != int(base_size): -- 2.17.1 -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH] cmake.bbclass: Add ${COREBASE}/scripts to CMAKE_FIND_ROOT_PATH
${COREBASE}/scripts contains a "git" wrapper disabling fakeroot/pseudo. This patch allows CMake to find ${COREBASE}/scripts/git instead of ${HOSTTOOLS_DIR}/git. This is needed for git invocations during do_install, since do_install is a fakeroot task, and otherwise all git commands fail with "fatal: detected dubious ownership in repository ...". I don't know how common it is for CMake projects to invoke git during the install phase intentionally. It's probably more common to do this during the configure phase. However, the install step may re-run the configure step, if some dependencies changed. In my case, this happened in incremental Yocto builds which reran do_install and repopulated parts of the recipe-sysroot during that, without first rerunning do_configure or do_compile. One of the dependencies changed (but only changing a file in some unrelated sub-package of it which was not even installed into the recipe-sysroot), causing the dependant's recipe-sysroot to be repopulated during do_install and thus causing the CMake project to be reconfigured during do_install. Signed-off-by: Daniel Klauer --- meta/classes-recipe/cmake.bbclass | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meta/classes-recipe/cmake.bbclass b/meta/classes-recipe/cmake.bbclass index 3d3781ef33..e1c3d7ddb5 100644 --- a/meta/classes-recipe/cmake.bbclass +++ b/meta/classes-recipe/cmake.bbclass @@ -124,7 +124,7 @@ set( CMAKE_CXX_LINK_FLAGS "${OECMAKE_CXX_LINK_FLAGS}" CACHE STRING "LDFLAGS" ) # only search in the paths provided so cmake doesnt pick # up libraries and tools from the native build machine -set( CMAKE_FIND_ROOT_PATH ${STAGING_DIR_HOST} ${STAGING_DIR_NATIVE} ${CROSS_DIR} ${OECMAKE_PERLNATIVE_DIR} ${OECMAKE_EXTRA_ROOT_PATH} ${EXTERNAL_TOOLCHAIN} ${HOSTTOOLS_DIR}) +set( CMAKE_FIND_ROOT_PATH ${STAGING_DIR_HOST} ${STAGING_DIR_NATIVE} ${CROSS_DIR} ${OECMAKE_PERLNATIVE_DIR} ${OECMAKE_EXTRA_ROOT_PATH} ${EXTERNAL_TOOLCHAIN} ${COREBASE}/scripts ${HOSTTOOLS_DIR} ) set( CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY ) set( CMAKE_FIND_ROOT_PATH_MODE_PROGRAM ${OECMAKE_FIND_ROOT_PATH_MODE_PROGRAM} ) set( CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY ) -- 2.34.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#198462): https://lists.openembedded.org/g/openembedded-core/message/198462 Mute This Topic: https://lists.openembedded.org/mt/105575001/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH] cmake.bbclass: Add ${COREBASE}/scripts to CMAKE_FIND_ROOT_PATH
Hi, >> ${COREBASE}/scripts contains a "git" wrapper disabling fakeroot/pseudo. >> This patch allows CMake to find ${COREBASE}/scripts/git instead of >> ${HOSTTOOLS_DIR}/git. This is needed for git invocations during do_install, >> since do_install is a fakeroot task, and otherwise all git commands fail >> with "fatal: detected dubious ownership in repository ...". >> >> I don't know how common it is for CMake projects to invoke git during the >> install phase intentionally. It's probably more common to do this during >> the configure phase. However, the install step may re-run the configure >> step, if some dependencies changed. > > > If a package build system has such dependencies they should be better > expressed in recipes and linked using bitbake task level dependency mechanismĀ > > I wonder why git is needed during install phase > Can you explain a bit more on thatĀ > > And since change it in a class it means it has to be a patternĀ It's just about running "git rev-parse HEAD" or similar, not acquiring external dependencies with "git clone", FetchContent download, etc., if that's what you meant. "dependencies" referred to timestamps of files checked by make/ninja. The affected CMake projects are using code like the following to query Git revision information (for storing into header files and binaries): find_package(Git REQUIRED) execute_process(COMMAND "${GIT_EXECUTABLE}" rev-parse HEAD ...) set_property(DIRECTORY APPEND PROPERTY CMAKE_CONFIGURE_DEPENDS "${CMAKE_CURRENT_SOURCE_DIR}/.git/HEAD") ${S} is owned by the user that runs bitbake, so cmake succeeds running such git commands during do_configure, but they fail during do_install because it's under fakeroot/pseudo. This can be fixed by allowing CMake to use the git wrapper script which temporarily disables fakeroot. It's already in PATH anyways (see PATH:prepend = "${COREBASE}/scripts..." in bitbake.conf), but CMake's find_package(Git) only respects CMAKE_FIND_ROOT_PATH. The description of commit af27c81e "scripts: Make git intercept global" hints at this type of situation. As for why cmake re-runs during do_install, I'm still trying to piece it together and reduce it to a minimal example. I can try to explain what I know so far: Basically, it's normal for a CMake project to be re-configured by "make" or "make install" (or ninja equivalents), if for example the CMakeLists.txt or files referenced by CMAKE_CONFIGURE_DEPENDS were modified, and an existing build directory was re-used. I'm not really surprised to see this case occurring in Yocto aswell. Presumably it can only happen in incremental Yocto builds if do_fetch/do_unpack/do_patch were already run previously and don't need to run again. What I did: 1. have a recipe for a library 2. have a recipe for a program which depends on that library (and of courses uses DEPENDS += "library") 3. run bitbake to build the image 4. modify library:do_install (to install a different configuration file into /etc) 5. modify program:do_install (same) 6. run bitbake to build the image My observations for the 2nd bitbake run (after I modified the two do_install functions): 6.1. library:do_configure did not run again (makes sense, since I didn't change anything there) 6.2. library:do_compile did not run again (same) 6.3. library:do_install was re-run (due to the changes for the new config file in /etc) 6.4. library:do_package was re-run (but presumably only the main library package changed, not the library-dev or library-static packages) 6.5. program:do_configure did not run again (not sure why not, since library recipe changed, but maybe bitbake noticed through the library packaging that only /etc changed and not the library or its header files) 6.6. program:do_compile did not run again (same) 6.7. program:do_install was re-run 6.7.1. The library was re-populated into the program's recipe-sysroot ("NOTE: exists in sysroot, but is stale") 6.7.2. "ninja install" re-ran CMake to re-configure the project (I'm guessing due to the re-population of the recipe-sysroot), this is where the git commands failed. Regards, Daniel -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#198548): https://lists.openembedded.org/g/openembedded-core/message/198548 Mute This Topic: https://lists.openembedded.org/mt/105575001/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-