[OE-core] [RFC PATCH] kernel-yocto: Support in-tree config fragments
[This is currently an RFC as it depends on a separate patch to add
support for config fragments with .config file extensions in
yocto-kernel-tools:
https://lists.yoctoproject.org/g/linux-yocto/message/14431]
In Linux 6.8, the in-tree rv32_defconfig file for 32-bit RISC-V support
was dropped in favour of a Makefile target which combines the RISC-V
defconfig with a config fragment '32-bit.config'. So to build for 32-bit
RISC-V using the in-tree configuration from an upstream kernel, we need
to be able to enable this 32-bit config fragment.
We can support this via KERNEL_FEATURES if we add the in-tree arch
config directory (arch/${ARCH}/configs) to the search path.
While we're here, let's also add the generic config directory
(kernel/configs) to the search path so in-tree config fragments such as
'hardening.config' can be used.
Signed-off-by: Paul Barker
---
meta/classes-recipe/kernel-yocto.bbclass | 3 +++
1 file changed, 3 insertions(+)
diff --git a/meta/classes-recipe/kernel-yocto.bbclass
b/meta/classes-recipe/kernel-yocto.bbclass
index 6d5c3b6327b6..66602d359404 100644
--- a/meta/classes-recipe/kernel-yocto.bbclass
+++ b/meta/classes-recipe/kernel-yocto.bbclass
@@ -248,6 +248,9 @@ do_kernel_metadata() {
fi
done
+ # allow in-tree config fragments to be used in KERNEL_FEATURES
+ includes="$includes -I${S}/arch/${ARCH}/configs -I${S}/kernel/configs"
+
# expand kernel features into their full path equivalents
bsp_definition=$(spp ${includes} --find -DKMACHINE=${KMACHINE}
-DKTYPE=${LINUX_KERNEL_TYPE})
if [ -z "$bsp_definition" ]; then
--
2.39.5
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#205183):
https://lists.openembedded.org/g/openembedded-core/message/205183
Mute This Topic: https://lists.openembedded.org/mt/108776739/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [master/scarthgap PATCH] meta-ide-support: Mark recipe as MACHINE-specific
On 06/09/2024 14:48, Richard Purdie wrote:
> On Thu, 2024-09-05 at 23:40 +0100, Richard Purdie via lists.openembedded.org
> wrote:
>> On Thu, 2024-09-05 at 21:21 +0100, Paul Barker via lists.openembedded.org
>> wrote:
>>> meta-ide-support:do_write_test_data dumps the bitbake data dictionary to
>>> a file using export2json(). As this obviously includes the value of
>>> MACHINE, and other MACHINE-specific variables, the recipe needs to be
>>> marked as MACHINE-specific.
>>>
>>> Signed-off-by: Paul Barker
>>> ---
>>> meta/recipes-core/meta/meta-ide-support.bb | 1 +
>>> 1 file changed, 1 insertion(+)
>>>
>>> diff --git a/meta/recipes-core/meta/meta-ide-support.bb
>>> b/meta/recipes-core/meta/meta-ide-support.bb
>>> index d85aa120c08b..7ed422ce4ddb 100644
>>> --- a/meta/recipes-core/meta/meta-ide-support.bb
>>> +++ b/meta/recipes-core/meta/meta-ide-support.bb
>>> @@ -1,6 +1,7 @@
>>> SUMMARY = "Integrated Development Environment support"
>>> DESCRIPTION = "Meta package for ensuring the build directory contains all
>>> appropriate toolchain packages for using an IDE"
>>> LICENSE = "MIT"
>>> +PACKAGE_ARCH = "${MACHINE_ARCH}"
>>>
>>> DEPENDS = "virtual/libc gdb-cross-${TARGET_ARCH} qemu-native
>>> qemu-helper-native unfs3-native cmake-native autoconf-native
>>> automake-native meson-native intltool-native pkgconfig-native"
>>> RM_WORK_EXCLUDE += "${PN}"
>>
>> Something about this is making our toolchain tests unhappy, e.g.:
>>
>> https://valkyrie.yoctoproject.org/#/builders/42/builds/86/steps/12/logs/stdio
>>
>> I've not looked into it at all, just reporting what we've need to fix
>> in order to be able to consider merging this.
>
> It can be fixed with:
>
> diff --git a/meta/lib/oeqa/selftest/cases/meta_ide.py
> b/meta/lib/oeqa/selftest/cases/meta_ide.py
> index ffe0d2604d1..5a17ca52ea1 100644
> --- a/meta/lib/oeqa/selftest/cases/meta_ide.py
> +++ b/meta/lib/oeqa/selftest/cases/meta_ide.py
> @@ -20,8 +20,8 @@ class MetaIDE(OESelftestTestCase):
> bitbake('meta-ide-support')
> bitbake('build-sysroots -c build_native_sysroot')
> bitbake('build-sysroots -c build_target_sysroot')
> -bb_vars = get_bb_vars(['MULTIMACH_TARGET_SYS', 'DEPLOY_DIR_IMAGE',
> 'COREBASE'])
> -cls.environment_script = 'environment-setup-%s' %
> bb_vars['MULTIMACH_TARGET_SYS']
> +bb_vars = get_bb_vars(['MACHINE_ARCH', 'TARGET_VENDOR', 'TARGET_OS',
> 'DEPLOY_DIR_IMAGE', 'COREBASE'])
> +cls.environment_script = 'environment-setup-%s%s-%s' %
> (bb_vars['MACHINE_ARCH'], bb_vars['TARGET_VENDOR'], bb_vars['TARGET_OS'])
> cls.deploydir = bb_vars['DEPLOY_DIR_IMAGE']
> cls.environment_script_path = '%s/%s' % (cls.deploydir,
> cls.environment_script)
> cls.corebasedir = bb_vars['COREBASE']
>
> Whether that is a good/right fix, I'm not sure...
>
> Cheers,
>
> Richard
>
Hi Richard, Steve,
I see this has been committed as 3be2bc8a9b0c in openembedded-core [1]
for master/Styhead. Thanks for fixing the test case and merging this.
We saw the same issue when running yocto-check-layer in Scarthgap, do
you think this is suitable for backporting?
[1]:
https://git.openembedded.org/openembedded-core/commit/?id=3be2bc8a9b0c9d6a178329c8b451a6bedf255d6c
Thanks,
--
Paul Barker
OpenPGP_0x27F4B3459F002257.asc
Description: OpenPGP public key
OpenPGP_signature.asc
Description: OpenPGP digital signature
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#205113):
https://lists.openembedded.org/g/openembedded-core/message/205113
Mute This Topic: https://lists.openembedded.org/mt/108292901/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [master/scarthgap PATCH] meta-ide-support: Mark recipe as MACHINE-specific
meta-ide-support:do_write_test_data dumps the bitbake data dictionary to
a file using export2json(). As this obviously includes the value of
MACHINE, and other MACHINE-specific variables, the recipe needs to be
marked as MACHINE-specific.
Signed-off-by: Paul Barker
---
meta/recipes-core/meta/meta-ide-support.bb | 1 +
1 file changed, 1 insertion(+)
diff --git a/meta/recipes-core/meta/meta-ide-support.bb
b/meta/recipes-core/meta/meta-ide-support.bb
index d85aa120c08b..7ed422ce4ddb 100644
--- a/meta/recipes-core/meta/meta-ide-support.bb
+++ b/meta/recipes-core/meta/meta-ide-support.bb
@@ -1,6 +1,7 @@
SUMMARY = "Integrated Development Environment support"
DESCRIPTION = "Meta package for ensuring the build directory contains all
appropriate toolchain packages for using an IDE"
LICENSE = "MIT"
+PACKAGE_ARCH = "${MACHINE_ARCH}"
DEPENDS = "virtual/libc gdb-cross-${TARGET_ARCH} qemu-native
qemu-helper-native unfs3-native cmake-native autoconf-native automake-native
meson-native intltool-native pkgconfig-native"
RM_WORK_EXCLUDE += "${PN}"
--
2.43.0
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#204243):
https://lists.openembedded.org/g/openembedded-core/message/204243
Mute This Topic: https://lists.openembedded.org/mt/108292901/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [Openembedded-architecture] New mailing list for layer patches
On 28/03/2024 14:36, Ross Burton wrote: > Hi, > > At the moment if a layer doesn’t have enough volume to justify a dedicated > mailing list for patches (see, for example, meta-...@lists.yoctoproject.org > <mailto:meta-...@lists.yoctoproject.org>) then the convention is that the > patches can be sent to yo...@lists.yoctoproject.org > <mailto:yo...@lists.yoctoproject.org>. > > However, that’s also the list that we encourage users of Yocto to use for > help, so the list has a mix of patches for various layers and people asking > for help, which isn’t ideal: users who want help don’t want to be flooded > with patches for layers they’re not using, layer maintainers may not want to > see the requests for help, and people who want the status reports have to > receive patches and support. > > The Yocto TSC has been discussing this, and as a solution to this we’ve just > created a new mailing list: yocto-patc...@lists.yoctoproject.org > <mailto:yocto-patc...@lists.yoctoproject.org>. This list is specifically for > Yocto-related projects which don’t have their own mailing list or other patch > submission process. We ask that all layers currently using yocto@ for > patches move to yocto-patches@, so that yocto@ can be used purely for > discussion and other non-patch related subjects. I think this is a great change, along with the new yocto-status@ list it will make things easier to follow :) Do you know if yocto-patches@ will be available via lore.kernel.org? Thanks, -- Paul Barker OpenPGP_0x27F4B3459F002257.asc Description: OpenPGP public key OpenPGP_signature.asc Description: OpenPGP digital signature -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#197868): https://lists.openembedded.org/g/openembedded-core/message/197868 Mute This Topic: https://lists.openembedded.org/mt/105284126/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH] kernel: Fix check_oldest_kernel
The check_oldest_kernel() function requires utsrelease.h to be
generated. This file is generated during do_compile, so we need to delay
calling check_oldest_kernel() until after this.
With this change in place, I now see the expected warning when building
Linux 5.10.y.
Signed-off-by: Paul Barker
---
meta/classes-recipe/kernel.bbclass | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/meta/classes-recipe/kernel.bbclass
b/meta/classes-recipe/kernel.bbclass
index db4461e551e4..b084d6d69dc1 100644
--- a/meta/classes-recipe/kernel.bbclass
+++ b/meta/classes-recipe/kernel.bbclass
@@ -645,7 +645,7 @@ python check_oldest_kernel() {
}
check_oldest_kernel[vardepsexclude] += "OLDEST_KERNEL KERNEL_VERSION"
-do_configure[prefuncs] += "check_oldest_kernel"
+do_compile[postfuncs] += "check_oldest_kernel"
KERNEL_LOCALVERSION ??= ""
--
2.44.0
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#197052):
https://lists.openembedded.org/g/openembedded-core/message/197052
Mute This Topic: https://lists.openembedded.org/mt/104910570/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [Openembedded-architecture] [RFC PATCH] Add genericarm64 MACHINE using upstream defconfig
On 21/02/2024 10:57, Ross Burton wrote: > From: Ross Burton > > This is a new 64-bit "generic" Arm machine, that expects the hardware to > be SystemReady IR compatible. This is slightly forward-leaning as there's > not a _lot_ of SystemReady hardware in the wild, but most modern boards > are and the number will only grow. Also, this is the only way to have a > 'generic' machine as without standardised bootloaders and firmware it > would be impossible. > > The base machine configuration isn't that exciting: it's a fully featured > machine that supports most things, booting via UEFI and an initramfs. > > However, the kernel is more interesting. This RFC uses the upstream defconfig > because unlike some other platforms, the arm64 defconfig is actively > maintained with the goal of being a 'boots on most hardware' configuration. > My argument is: why would we duplicate that effort? > > The "linux-yocto way" is configuration fragments and after a week of > hair-pulling I do actually have fragments that boot on a BeaglePlay, but > to say this was a tiresome and frustrating exercise would be understating it. > > So, a request for comments: is it acceptable to use the upstream defconfig in > a reference BSP? Personally I'm torn: the Yocto way is fragments not > monolithic > configs, but repeating the effort to fragmentise the configuration and then > also have it sufficiently modular that it can be used in pieces - instead of > just being a large file split up into smaller files - is a lot of effort for > what might end up being minimal gain. My fear is we end up with a fragmented > configuration that can't be easily modified without breaking some platforms, > and badly copies what the defconfig already does. I am in favour of this - I think the "genericarm64" machine should use the in-tree defconfig so that it can support the widest array of hardware. If someone wants to trim down the kernel for a particular platform then they should probably create a specific MACHINE anyway. If we take the other approach of building up the kernel config from fragments, how would we know that all SystemReady IR capable systems will be supported? Yocto Project doesn't have the resources to test every platform. For the Renesas RZ SoCs I work on these days, the in-tree defconfig is the configuration we test with the mainline kernel. Thanks, -- Paul Barker OpenPGP_0x27F4B3459F002257.asc Description: OpenPGP public key OpenPGP_signature.asc Description: OpenPGP digital signature -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#195979): https://lists.openembedded.org/g/openembedded-core/message/195979 Mute This Topic: https://lists.openembedded.org/mt/104489784/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] Yocto Project Status 07 November 2023 (WW45)
On Tue, Nov 07, 2023 at 10:52:53AM -0500, Neal Caidin wrote: >The Yocto Project website has had a refresh (www.yoctoproject.org) The new website looks great! Just a quick note - the links at the bottom of the page are mostly going to https://yocto.dev-lfprojects3.linuxfoundation.org/. E.g. "Technical Overview" goes to https://yocto.dev-lfprojects3.linuxfoundation.org/development/technical-overview/ Thanks, Paul signature.asc Description: PGP signature -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#190309): https://lists.openembedded.org/g/openembedded-core/message/190309 Mute This Topic: https://lists.openembedded.org/mt/102445104/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [dunfell][PATCH RESEND] kernel.bbclass: Add force flag to rm calls
From: Ryan Eatmon
The latest 6.5 kernels do not appear to create the source file in
${D}${nonarch_base_libdir}/modules/${KERNEL_VERSION}/source so the
recipe errors out when trying to remove it. Simple fix is to add the
-f (force) flag to the call.
(From OE-Core rev: 2e669bf797b15d803e7d6a700e449bdc467a4bcc)
Signed-off-by: Ryan Eatmon
Signed-off-by: Richard Purdie
Signed-off-by: Alexander Sverdlin
Signed-off-by: Steve Sakoman
Backported from kirkstone commit 7e177848f97e.
Signed-off-by: Paul Barker
---
Re-sending this for the mailing list. My first send was dropped as I
wasn't subscribed to the list from this email address.
Yes, I do crazy things like building mainline kernels on dunfell.
This fix is unlikely to have any negative effects though, so I think
it's good to apply to dunfell.
meta/classes/kernel.bbclass | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index ba5b6cf384a0..ca7530095e12 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -405,8 +405,8 @@ kernel_do_install() {
unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS MACHINE
if (grep -q -i -e '^CONFIG_MODULES=y$' .config); then
oe_runmake DEPMOD=echo
MODLIB=${D}${nonarch_base_libdir}/modules/${KERNEL_VERSION}
INSTALL_FW_PATH=${D}${nonarch_base_libdir}/firmware modules_install
- rm "${D}${nonarch_base_libdir}/modules/${KERNEL_VERSION}/build"
- rm "${D}${nonarch_base_libdir}/modules/${KERNEL_VERSION}/source"
+ rm -f
"${D}${nonarch_base_libdir}/modules/${KERNEL_VERSION}/build"
+ rm -f
"${D}${nonarch_base_libdir}/modules/${KERNEL_VERSION}/source"
# If the kernel/ directory is empty remove it to prevent QA
issues
rmdir --ignore-fail-on-non-empty
"${D}${nonarch_base_libdir}/modules/${KERNEL_VERSION}/kernel"
else
--
2.39.2
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#188872):
https://lists.openembedded.org/g/openembedded-core/message/188872
Mute This Topic: https://lists.openembedded.org/mt/101870886/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [dunfell PATCH] licenses: Handle newer SPDX license names
License names were updated in commit 2456f523cfbb (after the dunfell release) to match the current SPDX license list. We don't want to do any wholesale renaming on the dunfell stable branch but we should add mappings for the newer '*-only' names to allow for layers which support both dunfell and newer releases. Signed-off-by: Paul Barker --- meta/conf/licenses.conf | 7 +++ 1 file changed, 7 insertions(+) diff --git a/meta/conf/licenses.conf b/meta/conf/licenses.conf index 5b309eb3851f..0149b1dc4445 100644 --- a/meta/conf/licenses.conf +++ b/meta/conf/licenses.conf @@ -13,24 +13,31 @@ SPDXLICENSEMAP[AGPL-3] = "AGPL-3.0" SPDXLICENSEMAP[AGPLv3] = "AGPL-3.0" SPDXLICENSEMAP[AGPLv3.0] = "AGPL-3.0" +SPDXLICENSEMAP[AGPL-3.0-only] = "AGPL-3.0" # GPL variations SPDXLICENSEMAP[GPL-1] = "GPL-1.0" SPDXLICENSEMAP[GPLv1] = "GPL-1.0" SPDXLICENSEMAP[GPLv1.0] = "GPL-1.0" +SPDXLICENSEMAP[GPL-1.0-only] = "GPL-1.0" SPDXLICENSEMAP[GPL-2] = "GPL-2.0" SPDXLICENSEMAP[GPLv2] = "GPL-2.0" SPDXLICENSEMAP[GPLv2.0] = "GPL-2.0" +SPDXLICENSEMAP[GPL-2.0-only] = "GPL-2.0" SPDXLICENSEMAP[GPL-3] = "GPL-3.0" SPDXLICENSEMAP[GPLv3] = "GPL-3.0" SPDXLICENSEMAP[GPLv3.0] = "GPL-3.0" +SPDXLICENSEMAP[GPL-3.0-only] = "GPL-3.0" #LGPL variations SPDXLICENSEMAP[LGPLv2] = "LGPL-2.0" SPDXLICENSEMAP[LGPLv2.0] = "LGPL-2.0" +SPDXLICENSEMAP[LGPL-2.0-only] = "LGPL-2.0" SPDXLICENSEMAP[LGPL2.1] = "LGPL-2.1" SPDXLICENSEMAP[LGPLv2.1] = "LGPL-2.1" +SPDXLICENSEMAP[LGPL-2.1-only] = "LGPL-2.1" SPDXLICENSEMAP[LGPLv3] = "LGPL-3.0" +SPDXLICENSEMAP[LGPL-3.0-only] = "LGPL-3.0" #MPL variations SPDXLICENSEMAP[MPL-1] = "MPL-1.0" -- 2.25.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#171220): https://lists.openembedded.org/g/openembedded-core/message/171220 Mute This Topic: https://lists.openembedded.org/mt/94013682/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH 5/5] lib: Add copyright statements to files without one
On 11/08/2022 08:59, Richard Purdie wrote: On Thu, 2022-08-11 at 08:57 +0100, Paul Barker wrote: On 10/08/2022 20:45, Richard Purdie wrote: diff --git a/meta/lib/oeqa/runtime/cases/rtc.py b/meta/lib/oeqa/runtime/cases/rtc.py index c4e66813249..223cf7b95d9 100644 --- a/meta/lib/oeqa/runtime/cases/rtc.py +++ b/meta/lib/oeqa/runtime/cases/rtc.py @@ -1,3 +1,8 @@ +\# Looks like an extra '\' has slipped in here. Thanks, well spotted! I've fixed in the branch. I've also checked master-next with `reuse lint`, obviously it complains that license text files aren't in the places the reuse tool wants them to be and that's unimportant, but it does highlight some other low hanging fruit: * The following are missing SPDX headers: * scripts/lib/devtool/menuconfig.py * meta-skeleton/recipes-kernel/hello-mod/files/hello.c * The following are missing copyright headers: * scripts/lib/argparse_oe.py * scripts/lib/wic/plugins/source/bootimg-partition.py * scripts/lib/wic/plugins/source/empty.py * scripts/lib/wic/plugins/source/isoimage-isohybrid.py * scripts/lib/wic/plugins/source/rawcopy.py * Most files under meta/conf can probaby have copyright and license headers added without any confusion. * `reuse lint` will accept the following header as valid, which could be a path forward for recipes if other folks think it's clear enough: # Recipe: Copyright OpenEmbedded Contributors # Recipe: SPDX-License-Identifier: MIT Thanks, -- Paul Barker OpenPGP_0x74975C81B7E66BAC.asc Description: OpenPGP public key OpenPGP_signature Description: OpenPGP digital signature -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#169235): https://lists.openembedded.org/g/openembedded-core/message/169235 Mute This Topic: https://lists.openembedded.org/mt/92944317/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH 5/5] lib: Add copyright statements to files without one
On 10/08/2022 20:45, Richard Purdie wrote: diff --git a/meta/lib/oeqa/runtime/cases/rtc.py b/meta/lib/oeqa/runtime/cases/rtc.py index c4e66813249..223cf7b95d9 100644 --- a/meta/lib/oeqa/runtime/cases/rtc.py +++ b/meta/lib/oeqa/runtime/cases/rtc.py @@ -1,3 +1,8 @@ +\# Looks like an extra '\' has slipped in here. Thanks, -- Paul Barker OpenPGP_0x74975C81B7E66BAC.asc Description: OpenPGP public key OpenPGP_signature Description: OpenPGP digital signature -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#169233): https://lists.openembedded.org/g/openembedded-core/message/169233 Mute This Topic: https://lists.openembedded.org/mt/92944317/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [honister PATCH] kernel: introduce python3-dtschema-wrapper
On 14/01/2022 13:21, Paul Barker wrote: From: Bruce Ashfield The 5.16 kernel introduced mandatory schema checking on any dtb file built through the kernel. That funcionality is provided via python3-dt-schema. The dependencies to enable that functionality is not small, and may not always be desired (in particular on architectures that do not support dtbs, or in development cycles). It may also be useful for allowing a non-conformant dts to be compiled. This commit introduces a set of wrapper scripts that when added as a depenency to the kernel, can pass both the validation testing and validation steps of a dts. Signed-off-by: Bruce Ashfield Signed-off-by: Richard Purdie Backported from oe-core commit 2566563ad49d. Signed-off-by: Paul Barker I'm sending a gentle ping on this as I haven't heard any reply or seen this land in honister yet. Any feedback? Thanks, -- Paul Barker Principal Software Engineer SanCloud Ltd e: paul.bar...@sancloud.com w: https://sancloud.co.uk/ OpenPGP_0xA67255DFCCE62ECD.asc Description: OpenPGP public key OpenPGP_signature Description: OpenPGP digital signature -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#161121): https://lists.openembedded.org/g/openembedded-core/message/161121 Mute This Topic: https://lists.openembedded.org/mt/88420169/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] Inclusive Language Proposal for YP/OE
toml-do-not-exit-on-panic.patch Also, there are a few others outside of OE that should probably be patched too. Branch Names The “master” branches on the relevant OpenEmbedded and Yocto Project git trees will be changed to an alternative name at some point in the future. The current preferred name is “devel”. There is no time table for this currently, and there is no obligation or requirement to change the branch name for any downstream project which is beyond the project’s remit. The layer index may need some modification to handle different layers having different names for the in-development branch. We could implement the layer index changes first to support other layers which rename their master branch. I'm going to bite the bullet with meta-linux-mainline and rename the master branch to "dev" this week to see what happens. Similarly, there is no need to change any recipes that are using a “master” branch as part of the SRC_URI. Those are outside the scope of YP/OE and this effort. Note These changes are only to bitbake and OE-Core. There is no requirement to change any other layers but we’d note consistency is encouraged and helpful to users. Helping If you would like to help, please put your name by the items in question on the inclusive language wiki page. https://wiki.yoctoproject.org/wiki/Inclusive_language Thanks Special thanks to Richard Purdie, Michael Opdenacker. Marta Rybczynska, Scott Murray, Jan-Simon Moeller, Saul Wold, and Armin Kuster for providing their time, technical details, text, and feedback on this task. Thanks all, it's great to see this moving forward! -- Paul Barker Principal Software Engineer SanCloud Ltd e: paul.bar...@sancloud.com w: https://sancloud.co.uk/ OpenPGP_0xA67255DFCCE62ECD.asc Description: OpenPGP public key OpenPGP_signature Description: OpenPGP digital signature -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#160912): https://lists.openembedded.org/g/openembedded-core/message/160912 Mute This Topic: https://lists.openembedded.org/mt/88650128/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH 1/2] kernel: Rework kernel make flag to variable mappings
On 14/10/2021 21:26, Richard Purdie wrote: In 2017 we added changes to pass the BUILD_CFLAGS into the kernel via BUILD_CC. This isn't really correct and the upstream kernel now has places to pass build cflags, ldflags and more. Update our kernel make flags to correctly use the kernel's variables. This addresses concerns raised by kernel developers. If this breaks some usecase please report it so we can work out how to fix it properly. FYI, this change back in Oct broke the build for Linux 5.10.y, 5.4.y, 4.19.y & 4.4.y kernels where CONFIG_UNWINDER_ORC=y (which is set by the in-tree defconfig for x86-64). The build failures sat on my backburner for a long time due to other commitments but I finally tracked them down to this commit and looked for solutions. For 5.10.y, 5.4.y & 4.19.y there is an upstream kernel patch which can be backported which will address this. See https://lore.kernel.org/stable/04598430-7383-b725-2f5f-3f2b16aac...@sancloud.com/ and replies for more info. Hopefully these branches will be fixed with the next round of stable kernel releases now the backport has been requested. For 4.14.y there is no trivial fix upstream. The backport I did for the other branches depends on the KBUILD_HOSTLDFLAGS variable being set in the kernel Makefile which was added after 4.14.y. Personally, I'm happy to leave 4.14.y broken in this case and workaround the issue in meta-linux-mainline by disabling CONFIG_UNWINDER_ORC for this series. I don't think any changes are needed in oe-core, the info is now here in case it affects others though. Thanks, -- Paul Barker Principal Software Engineer SanCloud Ltd e: paul.bar...@sancloud.com w: https://sancloud.co.uk/ -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#160569): https://lists.openembedded.org/g/openembedded-core/message/160569 Mute This Topic: https://lists.openembedded.org/mt/86324180/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [dunfell PATCH] kernel: introduce python3-dtschema-wrapper
From: Bruce Ashfield
The 5.16 kernel introduced mandatory schema checking on any dtb file
built through the kernel.
That funcionality is provided via python3-dt-schema.
The dependencies to enable that functionality is not small, and may
not always be desired (in particular on architectures that do not
support dtbs, or in development cycles). It may also be useful for
allowing a non-conformant dts to be compiled.
This commit introduces a set of wrapper scripts that when added
as a depenency to the kernel, can pass both the validation testing
and validation steps of a dts.
Signed-off-by: Bruce Ashfield
Signed-off-by: Richard Purdie
Backported from oe-core commit 2566563ad49d.
Signed-off-by: Paul Barker
---
meta/conf/distro/include/maintainers.inc | 1 +
.../python3-dtschema-wrapper/dt-doc-validate | 20 +++
.../dtc/python3-dtschema-wrapper/dt-mk-schema | 20 +++
.../dtc/python3-dtschema-wrapper/dt-validate | 20 +++
.../dtc/python3-dtschema-wrapper_2021.10.bb | 17
5 files changed, 78 insertions(+)
create mode 100644
meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-doc-validate
create mode 100644
meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-mk-schema
create mode 100644 meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-validate
create mode 100644 meta/recipes-kernel/dtc/python3-dtschema-wrapper_2021.10.bb
diff --git a/meta/conf/distro/include/maintainers.inc
b/meta/conf/distro/include/maintainers.inc
index 895cf894872b..1575fce8c715 100644
--- a/meta/conf/distro/include/maintainers.inc
+++ b/meta/conf/distro/include/maintainers.inc
@@ -576,6 +576,7 @@ RECIPE_MAINTAINER_pn-python3 = "Oleksandr Kravchuk
+# License: MIT (see COPYING.MIT at the root of the repository for terms)
+
+for arg; do
+case "$arg" in
+--version)
+echo "v2021.10"
+;;
+esac
+done
+
+# TBD: left for future consideration
+# exec dt-doc-validate.real "$@"
+
+# we always succeed
+exit 0
+
diff --git a/meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-mk-schema
b/meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-mk-schema
new file mode 100644
index ..24b89d861982
--- /dev/null
+++ b/meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-mk-schema
@@ -0,0 +1,20 @@
+#!/bin/sh
+# dt-mk-schema wrapper to allow kernel dt-validation to pass
+#
+# Copyright (C) 2021 Bruce Ashfield
+# License: MIT (see COPYING.MIT at the root of the repository for terms)
+
+for arg; do
+case "$arg" in
+--version)
+echo "v2021.10"
+;;
+esac
+done
+
+# TBD: left for future consideration
+# exec dt-mk-schema.real "$@"
+
+# we always succeed
+exit 0
+
diff --git a/meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-validate
b/meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-validate
new file mode 100644
index ..8a4710a7eda1
--- /dev/null
+++ b/meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-validate
@@ -0,0 +1,20 @@
+#!/bin/sh
+# dt-validate wrapper to allow kernel dt-validation to pass
+#
+# Copyright (C) 2021 Bruce Ashfield
+# License: MIT (see COPYING.MIT at the root of the repository for terms)
+
+for arg; do
+case "$arg" in
+--version)
+echo "v2021.10"
+;;
+esac
+done
+
+# TBD: left for future consideration
+# exec dt-validate.real "$@"
+
+# we always succeed
+exit 0
+
diff --git a/meta/recipes-kernel/dtc/python3-dtschema-wrapper_2021.10.bb
b/meta/recipes-kernel/dtc/python3-dtschema-wrapper_2021.10.bb
new file mode 100644
index ..c869274d09fa
--- /dev/null
+++ b/meta/recipes-kernel/dtc/python3-dtschema-wrapper_2021.10.bb
@@ -0,0 +1,17 @@
+DESCRIPTION = "Wrapper for tooling for devicetree validation using YAML and
jsonschema"
+HOMEPAGE = "https://yoctoproject.org";
+LICENSE = "MIT"
+LIC_FILES_CHKSUM =
"file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"
+
+SRC_URI = "file://dt-doc-validate \
+ file://dt-mk-schema \
+ file://dt-validate"
+
+do_install() {
+install -d ${D}${bindir}/
+install -m 755 ${WORKDIR}/dt-doc-validate ${D}${bindir}/
+install -m 755 ${WORKDIR}/dt-mk-schema ${D}${bindir}/
+install -m 755 ${WORKDIR}/dt-validate ${D}${bindir}/
+}
+
+BBCLASSEXTEND = "native nativesdk"
--
2.34.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#160567):
https://lists.openembedded.org/g/openembedded-core/message/160567
Mute This Topic: https://lists.openembedded.org/mt/88420183/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [honister PATCH] kernel: introduce python3-dtschema-wrapper
From: Bruce Ashfield
The 5.16 kernel introduced mandatory schema checking on any dtb file
built through the kernel.
That funcionality is provided via python3-dt-schema.
The dependencies to enable that functionality is not small, and may
not always be desired (in particular on architectures that do not
support dtbs, or in development cycles). It may also be useful for
allowing a non-conformant dts to be compiled.
This commit introduces a set of wrapper scripts that when added
as a depenency to the kernel, can pass both the validation testing
and validation steps of a dts.
Signed-off-by: Bruce Ashfield
Signed-off-by: Richard Purdie
Backported from oe-core commit 2566563ad49d.
Signed-off-by: Paul Barker
---
meta/conf/distro/include/maintainers.inc | 1 +
.../python3-dtschema-wrapper/dt-doc-validate | 20 +++
.../dtc/python3-dtschema-wrapper/dt-mk-schema | 20 +++
.../dtc/python3-dtschema-wrapper/dt-validate | 20 +++
.../dtc/python3-dtschema-wrapper_2021.10.bb | 17
5 files changed, 78 insertions(+)
create mode 100644
meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-doc-validate
create mode 100644
meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-mk-schema
create mode 100644 meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-validate
create mode 100644 meta/recipes-kernel/dtc/python3-dtschema-wrapper_2021.10.bb
diff --git a/meta/conf/distro/include/maintainers.inc
b/meta/conf/distro/include/maintainers.inc
index b3b7711a0c48..2b54d2d12f9b 100644
--- a/meta/conf/distro/include/maintainers.inc
+++ b/meta/conf/distro/include/maintainers.inc
@@ -592,6 +592,7 @@ RECIPE_MAINTAINER:pn-python3-cython = "Oleksandr Kravchuk
+# License: MIT (see COPYING.MIT at the root of the repository for terms)
+
+for arg; do
+case "$arg" in
+--version)
+echo "v2021.10"
+;;
+esac
+done
+
+# TBD: left for future consideration
+# exec dt-doc-validate.real "$@"
+
+# we always succeed
+exit 0
+
diff --git a/meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-mk-schema
b/meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-mk-schema
new file mode 100644
index ..24b89d861982
--- /dev/null
+++ b/meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-mk-schema
@@ -0,0 +1,20 @@
+#!/bin/sh
+# dt-mk-schema wrapper to allow kernel dt-validation to pass
+#
+# Copyright (C) 2021 Bruce Ashfield
+# License: MIT (see COPYING.MIT at the root of the repository for terms)
+
+for arg; do
+case "$arg" in
+--version)
+echo "v2021.10"
+;;
+esac
+done
+
+# TBD: left for future consideration
+# exec dt-mk-schema.real "$@"
+
+# we always succeed
+exit 0
+
diff --git a/meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-validate
b/meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-validate
new file mode 100644
index ..8a4710a7eda1
--- /dev/null
+++ b/meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-validate
@@ -0,0 +1,20 @@
+#!/bin/sh
+# dt-validate wrapper to allow kernel dt-validation to pass
+#
+# Copyright (C) 2021 Bruce Ashfield
+# License: MIT (see COPYING.MIT at the root of the repository for terms)
+
+for arg; do
+case "$arg" in
+--version)
+echo "v2021.10"
+;;
+esac
+done
+
+# TBD: left for future consideration
+# exec dt-validate.real "$@"
+
+# we always succeed
+exit 0
+
diff --git a/meta/recipes-kernel/dtc/python3-dtschema-wrapper_2021.10.bb
b/meta/recipes-kernel/dtc/python3-dtschema-wrapper_2021.10.bb
new file mode 100644
index ..c869274d09fa
--- /dev/null
+++ b/meta/recipes-kernel/dtc/python3-dtschema-wrapper_2021.10.bb
@@ -0,0 +1,17 @@
+DESCRIPTION = "Wrapper for tooling for devicetree validation using YAML and
jsonschema"
+HOMEPAGE = "https://yoctoproject.org";
+LICENSE = "MIT"
+LIC_FILES_CHKSUM =
"file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"
+
+SRC_URI = "file://dt-doc-validate \
+ file://dt-mk-schema \
+ file://dt-validate"
+
+do_install() {
+install -d ${D}${bindir}/
+install -m 755 ${WORKDIR}/dt-doc-validate ${D}${bindir}/
+install -m 755 ${WORKDIR}/dt-mk-schema ${D}${bindir}/
+install -m 755 ${WORKDIR}/dt-validate ${D}${bindir}/
+}
+
+BBCLASSEXTEND = "native nativesdk"
--
2.34.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#160566):
https://lists.openembedded.org/g/openembedded-core/message/160566
Mute This Topic: https://lists.openembedded.org/mt/88420169/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [FOSDEM standholders] FOSDEM 2022 stands - an update
Hi all, I'm unable to take the lead on organising a virtual stand for OpenEmbedded at FOSDEM 2022. I'm forwarding this mail to the lists to see if anyone else would like to take over - you'll need to review how the virtual stands work, update the website content as detailed below and be around during FOSDEM 2022 to "run" the stand. At least one commit to the website content needs to be made before the end of the year to confirm that OpenEmbedded will have a stand. The repositories we used last year were: * https://gitlab.com/openembedded/community/fosdem-stand-content * https://gitlab.com/openembedded/community/fosdem-stand-static I can grant permissions on those repositories to whoever needs them. I also recommend we email sta...@fosdem.org to let them know who will be running the stand this year so that the standholders mailing list can be updated. Thanks, Paul Forwarded Message Subject: [FOSDEM standholders] FOSDEM 2022 stands - an update Date: Mon, 20 Dec 2021 15:58:21 + From: Alasdair G Kergon To: standhold...@lists.fosdem.org We've decided to streamline the process for the 2022 event by reusing the system and data that is already in place. The 2021 stands website has been archived into archive.fosdem.org. stands.fosdem.org is now available to use again in 2022 and the direct links there from fosdem.org have been removed for now. The front page hasn't yet been updated. If you had a stand at FOSDEM 2021, and wish to have one at FOSDEM 2022, then all you need to do is to start updating your static content the same way as you did last year. If you don't want a stand, please send an email to sta...@fosdem.org and we will remove it. We will also remove stands that haven't had at least one change made by 31st December. PLEASE MAKE AT LEAST ONE COMMIT BEFORE 31ST DECEMBER. If you need metadata updating that you can't change yourself, either send pull requests againt the main repository or email sta...@fosdem.org. The system for videos/events is not ready yet and this won't be set up until the final list of stands is confirmed. New stands will be by invitation only and limited in number because of the amount of manual administration involved. If you think there's a deserving case for an invitation, contact sta...@fosdem.org. Alasdair ___ standholders mailing list standhold...@lists.fosdem.org https://lists.fosdem.org/listinfo/standholders -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#159960): https://lists.openembedded.org/g/openembedded-core/message/159960 Mute This Topic: https://lists.openembedded.org/mt/87899744/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core][PATCH 0/4] Move crate fetcher to OE_IMPORTS
On 20/12/2021 22:34, Richard Purdie wrote: On Mon, 2021-12-20 at 17:01 +, Paul Barker wrote: On 17/12/2021 15:36, Matt Madison wrote: This patch series switches to using the OE_IMPORTS mechanism for installing the crate fetcher. This simplifies the addition and clears up issues with interactions between SRCPV and the shared-state fetching logic. Matt Madison (4): lib/crate.py: install crate fetcher on import base.bbclass: add crate to OE_IMPORTS cargo_common.bbclass: remove inherit of crate-fetch classes: remove crate-fetch.bbclass meta/classes/base.bbclass | 2 +- meta/classes/cargo_common.bbclass | 2 -- meta/classes/crate-fetch.bbclass | 28 meta/lib/crate.py | 2 ++ 4 files changed, 3 insertions(+), 31 deletions(-) delete mode 100644 meta/classes/crate-fetch.bbclass Would a better solution here be to move the crate fetcher into lib/bb/fetch2/ in bitbake? I've been wondering about this too. Does anyone remember how the fetcher lines up against the general fetcher requirements we have and is there a list of anything that remained to be done before it could be added to bitbake? If I remember correctly, it just needed some test cases writing. The crate fetcher is a subclass of the wget fetcher with changes to how urldata is set up and how unpack is handled, so it should integrate well into bitbake. Thanks, -- Paul Barker Principal Software Engineer SanCloud Ltd e: paul.bar...@sancloud.com w: https://sancloud.co.uk/ -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#159920): https://lists.openembedded.org/g/openembedded-core/message/159920 Mute This Topic: https://lists.openembedded.org/mt/87791267/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core][PATCH 0/4] Move crate fetcher to OE_IMPORTS
On 17/12/2021 15:36, Matt Madison wrote: This patch series switches to using the OE_IMPORTS mechanism for installing the crate fetcher. This simplifies the addition and clears up issues with interactions between SRCPV and the shared-state fetching logic. Matt Madison (4): lib/crate.py: install crate fetcher on import base.bbclass: add crate to OE_IMPORTS cargo_common.bbclass: remove inherit of crate-fetch classes: remove crate-fetch.bbclass meta/classes/base.bbclass | 2 +- meta/classes/cargo_common.bbclass | 2 -- meta/classes/crate-fetch.bbclass | 28 meta/lib/crate.py | 2 ++ 4 files changed, 3 insertions(+), 31 deletions(-) delete mode 100644 meta/classes/crate-fetch.bbclass Would a better solution here be to move the crate fetcher into lib/bb/fetch2/ in bitbake? -- Paul Barker Principal Software Engineer SanCloud Ltd e: paul.bar...@sancloud.com w: https://sancloud.co.uk/ OpenPGP_0xA67255DFCCE62ECD.asc Description: OpenPGP public key OpenPGP_signature Description: OpenPGP digital signature -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#159892): https://lists.openembedded.org/g/openembedded-core/message/159892 Mute This Topic: https://lists.openembedded.org/mt/87791267/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [dunfell][PATCH 00/14] Backport mandatory dtschema handling for device trees built through the kernel.
On 01/12/2021 21:33, Bruce Ashfield wrote: On Wed, Dec 1, 2021 at 3:34 PM Bhupesh Sharma wrote: This is a backport request for dunfell, while picking up only the skeletal support for allowing mandatory dtschema handling for device trees built through the kernel, introduced from kernel version 5.16 onwards. The problem with adding this support to dunfell, is obvious in the number of patches involved. We've also said/known that at some point that we won't be able to support all new kernel versions with the LTS release. This may be the start of drawing that line. Also, we've had to do some license tweaks in master just today for one of the added packages (idna), so that would be needed as well. The validation isn't absolutely critical, I'd suggest that just the wrappers and pkg-config fixes would be a smaller footprint to allow the kernel to build, without the new package/feature additions to the dunfell release. The new packages (and full validation) could still possibly be done through a secondary or mixin layer. I'm not sure which way to go on this, but I thought I'd offer those points for discussion. I've been thinking about this for meta-linux-mainline. Backporting python3-dtschema-wrapper seems like the way to go, either into a very small mixin layer or into oe-core. Thanks, -- Paul Barker e: p...@pbarker.dev w: https://pbarker.dev/ -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#159286): https://lists.openembedded.org/g/openembedded-core/message/159286 Mute This Topic: https://lists.openembedded.org/mt/87439224/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [Hardknott backport request] kernel-yocto: Simplify no git repo case in do_kernel_checkout
Hi Anuj,
Please could you consider backporting the following commit to hardknott:
af2a9c92d449 ("kernel-yocto: Simplify no git repo case in do_kernel_checkout")
This has already been backport to dunfell. The patch cherry-picks
cleanly to hardknott.
Thanks,
--
Paul Barker
https://pbarker.dev/
pgpFyZGqNgYYi.pgp
Description: OpenPGP digital signature
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#155481):
https://lists.openembedded.org/g/openembedded-core/message/155481
Mute This Topic: https://lists.openembedded.org/mt/85246331/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core][PATCH] 1/2] wic: Add --no-fstab-update part option
On Tue, 10 Aug 2021 15:06:02 +0200
Daniel Gomez wrote:
> When embedding a rootfs image (e.g. 'rootfs-dir') as a partition we might
> want to keep the stock fstab for that image. In such case, use
> this option to not update the fstab and use the stock one instead.
>
> This option allows to specify which partitions get the fstab updated
> and which get the stock fstab.
>
> The option matches the argument you can pass to wic itself where the
> same action is performed but for all the partitions.
>
> Example:
> part /export --source rootfs --rootfs-dir=hockeycam-image --fstype=ext4
> --label export --align 1024 --no-fstab-update
>
> part / --source rootfs --fstype=ext4 --label rootfs --align 1024
>
> Signed-off-by: Daniel Gomez
> ---
> scripts/lib/wic/help.py | 3 +++
> scripts/lib/wic/ksparser.py | 1 +
> scripts/lib/wic/partition.py | 5 +++--
> 3 files changed, 7 insertions(+), 2 deletions(-)
>
> diff --git a/scripts/lib/wic/help.py b/scripts/lib/wic/help.py
> index 991e5094bb..5c3d278d4e 100644
> --- a/scripts/lib/wic/help.py
> +++ b/scripts/lib/wic/help.py
> @@ -991,6 +991,9 @@ DESCRIPTION
> multiple partitions and we want to keep the
> right
> permissions and usernames in all the partitions.
>
> + --no-fstab-update: This option is specific to wic. It does not
> update the
> +'/etc/fstab' stock file for the given partition.
> +
> --extra-space: This option is specific to wic. It adds extra
> space after the space filled by the content
> of the partition. The final size can go
> diff --git a/scripts/lib/wic/ksparser.py b/scripts/lib/wic/ksparser.py
> index 7a4cc83af5..0df9eb0d05 100644
> --- a/scripts/lib/wic/ksparser.py
> +++ b/scripts/lib/wic/ksparser.py
> @@ -185,6 +185,7 @@ class KickStart():
> part.add_argument('--use-uuid', action='store_true')
> part.add_argument('--uuid')
> part.add_argument('--fsuuid')
> +part.add_argument('--no-fstab-update', action='store_true')
>
> bootloader = subparsers.add_parser('bootloader')
> bootloader.add_argument('--append')
> diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
> index e0b2c5bdf2..ab304f1b2a 100644
> --- a/scripts/lib/wic/partition.py
> +++ b/scripts/lib/wic/partition.py
> @@ -54,6 +54,7 @@ class Partition():
> self.uuid = args.uuid
> self.fsuuid = args.fsuuid
> self.type = args.type
> +self.no_fstab_update = args.no_fstab_update
> self.updated_fstab_path = None
> self.has_fstab = False
> self.update_fstab_in_rootfs = False
> @@ -286,7 +287,7 @@ class Partition():
> (self.fstype, extraopts, rootfs, label_str, self.fsuuid,
> rootfs_dir)
> exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
>
> -if self.updated_fstab_path and self.has_fstab:
> +if self.updated_fstab_path and self.has_fstab and not
> self.no_fstab_update:
> debugfs_script_path = os.path.join(cr_workdir, "debugfs_script")
> with open(debugfs_script_path, "w") as f:
> f.write("cd etc\n")
> @@ -350,7 +351,7 @@ class Partition():
> mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (rootfs, rootfs_dir)
> exec_native_cmd(mcopy_cmd, native_sysroot)
>
> -if self.updated_fstab_path and self.has_fstab:
> +if self.updated_fstab_path and self.has_fstab and not
> self.no_fstab_update:
> mcopy_cmd = "mcopy -i %s %s ::/etc/fstab" % (rootfs,
> self.updated_fstab_path)
> exec_native_cmd(mcopy_cmd, native_sysroot)
>
This looks like the right approach. However, you're still missing the
handling for filesystems other than ext2/3/4 & msdos. See
http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/scripts/lib/wic/plugins/source/rootfs.py#n218
and grep to see if there are any other uses of updated_fstab_path.
Thanks,
--
Paul Barker
https://pbarker.dev/
pgpEWdyh_IMLl.pgp
Description: OpenPGP digital signature
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154684):
https://lists.openembedded.org/g/openembedded-core/message/154684
Mute This Topic: https://lists.openembedded.org/mt/84792172/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH] pypi: Allow override of PyPI archive name
Some packages on PyPI don't follow the usual expectations for archive
naming. For example, the archive for asyncio-mqtt 0.10.0 is named
asyncio_mqtt-0.10.0.tar.gz (with an underscore instead of the dash used
in the package name).
To handle these edge cases a new PYPI_ARCHIVE_NAME variable is
introduced. By default this is set to the expected archive name based on
the PyPI package name, version and extension but it can be set to a
different value if needed in a recipe which inherits the pypi class.
Signed-off-by: Paul Barker
---
meta/classes/pypi.bbclass | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/meta/classes/pypi.bbclass b/meta/classes/pypi.bbclass
index 272c220bca..9405d58601 100644
--- a/meta/classes/pypi.bbclass
+++ b/meta/classes/pypi.bbclass
@@ -8,12 +8,12 @@ def pypi_package(d):
PYPI_PACKAGE ?= "${@pypi_package(d)}"
PYPI_PACKAGE_EXT ?= "tar.gz"
+PYPI_ARCHIVE_NAME ?= "${PYPI_PACKAGE}-${PV}.${PYPI_PACKAGE_EXT}"
def pypi_src_uri(d):
package = d.getVar('PYPI_PACKAGE')
-package_ext = d.getVar('PYPI_PACKAGE_EXT')
-pv = d.getVar('PV')
-return 'https://files.pythonhosted.org/packages/source/%s/%s/%s-%s.%s' %
(package[0], package, package, pv, package_ext)
+archive_name = d.getVar('PYPI_ARCHIVE_NAME')
+return 'https://files.pythonhosted.org/packages/source/%s/%s/%s' %
(package[0], package, archive_name)
PYPI_SRC_URI ?= "${@pypi_src_uri(d)}"
--
2.31.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154678):
https://lists.openembedded.org/g/openembedded-core/message/154678
Mute This Topic: https://lists.openembedded.org/mt/84790391/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH] u-boot: Package extlinux.conf separately
A separate u-boot-extlinux package is created for the extlinux.conf file
so that it can be installed on its own if needed. If this package is
populated, it is added as a dependency of the main u-boot package so
that installing just u-boot still results in the extlinux.conf file
being present in the rootfs.
Signed-off-by: Paul Barker
---
meta/recipes-bsp/u-boot/u-boot.inc | 5 -
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/meta/recipes-bsp/u-boot/u-boot.inc
b/meta/recipes-bsp/u-boot/u-boot.inc
index 4340b17cb6..971fdbb102 100644
--- a/meta/recipes-bsp/u-boot/u-boot.inc
+++ b/meta/recipes-bsp/u-boot/u-boot.inc
@@ -210,7 +210,7 @@ do_install () {
fi
}
-PACKAGE_BEFORE_PN += "${PN}-env"
+PACKAGE_BEFORE_PN += "${PN}-env ${PN}-extlinux"
RPROVIDES:${PN}-env += "u-boot-default-env"
ALLOW_EMPTY:${PN}-env = "1"
@@ -219,6 +219,9 @@ FILES:${PN}-env = " \
${sysconfdir}/fw_env.config \
"
+FILES:${PN}-extlinux =
"${UBOOT_EXTLINUX_INSTALL_DIR}/${UBOOT_EXTLINUX_CONF_NAME}"
+RDEPENDS:${PN} += "${@bb.utils.contains('UBOOT_EXTLINUX', '1',
'${PN}-extlinux', '', d)}"
+
FILES:${PN} = "/boot ${datadir}"
RDEPENDS:${PN} += "${PN}-env"
--
2.31.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154660):
https://lists.openembedded.org/g/openembedded-core/message/154660
Mute This Topic: https://lists.openembedded.org/mt/84776988/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core][RFC][PATCH][dunfell] wic: Add --use-stock-fstab option
On Fri, 6 Aug 2021 11:23:37 +0200
Daniel Gomez wrote:
> When embeddding rootfs image (e.g. 'rootfs-dir') as a partition we
> might want to keep the stock fstab for that image. In such case, use
> this option to not update the fstab and use the stock one instead.
>
> This option allows to specify which partitions get the updated fstab
> and which get the stock fstab.
>
> Example:
> part /export --source rootfs --rootfs-dir=hockeycam-image
> --fstype=ext4 --label export --align 1024 --use-stock-fstab
>
> part / --source rootfs --fstype=ext4 --label rootfs --align 1024
>
> Signed-off-by: Daniel Gomez
> ---
>
> The commit-id ce682a73b7447652f898ce1d1d0416a456df5416 reverted the
> functionality to restore the stock/original fstab file in a rootfs
> partition image (wic plugin).
>
> Add back such functionality with an option to the user to select which
> fstab will be used for each partition by adding the --use-stock-fstab.
> In this case, the user will select whether or not the stock fstab will
> be updated or kept.
>
> If you agree with the patch I'll adapt it to the master branch.
>
> Daniel
>
> scripts/lib/wic/ksparser.py | 1 +
> scripts/lib/wic/partition.py | 3 ++-
> 2 files changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/scripts/lib/wic/ksparser.py b/scripts/lib/wic/ksparser.py
> index 452a160232..64933e0a5b 100644
> --- a/scripts/lib/wic/ksparser.py
> +++ b/scripts/lib/wic/ksparser.py
> @@ -184,6 +184,7 @@ class KickStart():
> part.add_argument('--use-uuid', action='store_true')
> part.add_argument('--uuid')
> part.add_argument('--fsuuid')
> +part.add_argument('--use-stock-fstab', action='store_true')
>
> bootloader = subparsers.add_parser('bootloader')
> bootloader.add_argument('--append')
> diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
> index 85f9847047..c9e45773de 100644
> --- a/scripts/lib/wic/partition.py
> +++ b/scripts/lib/wic/partition.py
> @@ -54,6 +54,7 @@ class Partition():
> self.uuid = args.uuid
> self.fsuuid = args.fsuuid
> self.type = args.type
> +self.use_stock_fstab = args.use_stock_fstab
> self.updated_fstab_path = None
> self.has_fstab = False
> self.update_fstab_in_rootfs = False
> @@ -286,7 +287,7 @@ class Partition():
> (self.fstype, extraopts, rootfs, label_str, self.fsuuid,
> rootfs_dir)
> exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
>
> -if self.updated_fstab_path and self.has_fstab:
> +if self.updated_fstab_path and self.has_fstab and not
> self.use_stock_fstab:
> debugfs_script_path = os.path.join(cr_workdir, "debugfs_script")
> with open(debugfs_script_path, "w") as f:
> f.write("cd etc\n")
> --
> 2.30.2
>
You're only addressing ext2/3/4 partitions here. The patch needs to be
extended to cover the msdos partition case and the
update_fstab_in_rootfs case in the rootfs source plugin.
I also recommend changing the option in ksparser to '--no-fstab-update'
to match the argument you can pass to wic itself. That way you can pass
'--no-fstab-update' to wic to set the option for all partitions, or
include it in the wks file to set the option per-partition.
Thanks,
--
Paul Barker
https://pbarker.dev/
pgpnp1YzDVrHE.pgp
Description: OpenPGP digital signature
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154545):
https://lists.openembedded.org/g/openembedded-core/message/154545
Mute This Topic: https://lists.openembedded.org/mt/84704596/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH] kernel-yocto: Simplify no git repo case in do_kernel_checkout
If the kernel sources are not fetched via git, a local git repository is
created in do_kernel_checkout. In this case we know that there will be
no remote branches and we will already be on the correct branch (since
only one branch will exist). So we can simplify things by skipping these
steps.
This also removes the assumption that the default git branch name will
be "master". Prior to this change, the final git checkout command in
do_kernel_checkout could fail if a local git repo was created and the
user had changed init.defaultBranch in their gitconfig.
Signed-off-by: Paul Barker
---
meta/classes/kernel-yocto.bbclass | 30 +++---
1 file changed, 15 insertions(+), 15 deletions(-)
diff --git a/meta/classes/kernel-yocto.bbclass
b/meta/classes/kernel-yocto.bbclass
index 0df61cdef0..18b77e2b9f 100644
--- a/meta/classes/kernel-yocto.bbclass
+++ b/meta/classes/kernel-yocto.bbclass
@@ -358,6 +358,21 @@ do_kernel_checkout() {
fi
fi
cd ${S}
+
+ # convert any remote branches to local tracking ones
+ for i in `git branch -a --no-color | grep remotes | grep -v
HEAD`; do
+ b=`echo $i | cut -d' ' -f2 | sed 's%remotes/origin/%%'`;
+ git show-ref --quiet --verify -- "refs/heads/$b"
+ if [ $? -ne 0 ]; then
+ git branch $b $i > /dev/null
+ fi
+ done
+
+ # Create a working tree copy of the kernel by checking out a
branch
+ machine_branch="${@ get_machine_branch(d, "${KBRANCH}" )}"
+
+ # checkout and clobber any unimportant files
+ git checkout -f ${machine_branch}
else
# case: we have no git repository at all.
# To support low bandwidth options for building the kernel,
we'll just
@@ -379,21 +394,6 @@ do_kernel_checkout() {
git commit -q -m "baseline commit: creating repo for
${PN}-${PV}"
git clean -d -f
fi
-
- # convert any remote branches to local tracking ones
- for i in `git branch -a --no-color | grep remotes | grep -v HEAD`; do
- b=`echo $i | cut -d' ' -f2 | sed 's%remotes/origin/%%'`;
- git show-ref --quiet --verify -- "refs/heads/$b"
- if [ $? -ne 0 ]; then
- git branch $b $i > /dev/null
- fi
- done
-
- # Create a working tree copy of the kernel by checking out a branch
- machine_branch="${@ get_machine_branch(d, "${KBRANCH}" )}"
-
- # checkout and clobber any unimportant files
- git checkout -f ${machine_branch}
}
do_kernel_checkout[dirs] = "${S} ${WORKDIR}"
--
2.31.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154365):
https://lists.openembedded.org/g/openembedded-core/message/154365
Mute This Topic: https://lists.openembedded.org/mt/84608327/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH] linux-yocto: Fix devupstream SRCREV assignment
Fixes the following error when building linux-yocto-upstream: Fetcher failure: Unable to find revision 570e7aedcd00e7e8a218e37c3dfb3b1fda83d111 in branch v5.13/base even from upstream The error occurred as the SRCREV_machine assignment earlier in the recipe was being used instead of the mainline commit hash we want to use for linux-yocto-upstream. Signed-off-by: Paul Barker --- meta/recipes-kernel/linux/linux-yocto_5.13.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meta/recipes-kernel/linux/linux-yocto_5.13.bb b/meta/recipes-kernel/linux/linux-yocto_5.13.bb index df385f912c..f43e738c69 100644 --- a/meta/recipes-kernel/linux/linux-yocto_5.13.bb +++ b/meta/recipes-kernel/linux/linux-yocto_5.13.bb @@ -31,7 +31,7 @@ SRCREV_meta ?= "4dcaa4bcd042d398e210b702449dd35fdf9f0078" # normal PREFERRED_VERSION settings. BBCLASSEXTEND = "devupstream:target" DEFAULT_PREFERENCE_class-devupstream = "-1" -SRCREV_class-devupstream ?= "f86aa267e180b23d8d24ee6886bd3bf9a360112e" +SRCREV_machine_class-devupstream ?= "f86aa267e180b23d8d24ee6886bd3bf9a360112e" PN_class-devupstream = "linux-yocto-upstream" KBRANCH_class-devupstream = "v5.13/base" -- 2.26.2 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#154013): https://lists.openembedded.org/g/openembedded-core/message/154013 Mute This Topic: https://lists.openembedded.org/mt/84341857/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH] linux-yocto/5.13: add devupstream support
On Tue, 6 Jul 2021 13:58:21 -0400 "Bruce Ashfield" wrote: > From: Bruce Ashfield > > We are always getting questions about building -stable, or mainline > master. Rather than introducing a separate set of recipes, we can > facilitate this sort of testing by using the existing devupstream > bbclass support. > > To build an unpatched or otherwise modifed -stable of 5.13, simply > set the linux-yocto preferred version to: > > upstream+5.13% > > And your wish will be granted, as the build will use v5.13/base > which matches the main linux-yocto version, simply with no extra > changes applied. > > Signed-off-by: Bruce Ashfield This looks like a good thing to have in oe-core, in the basic case of just wanting the latest mainline as-is you'd not need to add extra layers any longer. meta-linux-mainline would still fill a niche as it provides recipes for older stable kernels and supports using mainline & newer stable kernels on dunfell & hardknott. I guess the meta repository (yocto-kernel-cache) would still be cloned and the upstream kernel could still be modified by patches or config fragments enabled by KERNEL_FEATURES. That may be worth noting in the documentation if I'm right. For my use-case where I want to use the exact upstream defconfig I would probably need to override this, e.g. `KERNEL_FEATURES_forcevariable = ""`, to make use of this. I also guess this would also allow you to follow newer kernel releases as they're added to linux-yocto recipes by setting the preferred version to 'upstream%' (or maybe 'upstream+%' to be more restrictive). That would then avoid the build breaking when the 5.13 recipe disappears and is replaced by 5.14 or something else later. Thanks, -- Paul Barker https://pbarker.dev/ pgpYwwMxtWjZZ.pgp Description: OpenPGP digital signature -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#153628): https://lists.openembedded.org/g/openembedded-core/message/153628 Mute This Topic: https://lists.openembedded.org/mt/84026594/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH 0/8] Safe PEP8
On Tue, 29 Jun 2021 at 17:19, Persian Prince wrote: > > @Paul Barker > > I spent days and nights for testing the outcome of my PEP8.sh script and I > used it on more than 100 repos to check the results, what you see now is the > result of that. > > People at https://github.com/OpenVisionE2 , https://github.com/OpenPLi , > https://github.com/oe-alliance and more other teams like ViX, ATV and ... > trusted my work and all those are using the changed codes for months! > > This is why I write "Safe PEP8" not just "PEP8" as if you just run a normal > PEP8 on your repos you'll end up with dead ones. > > If you check https://github.com/openembedded/meta-openembedded you're using > my changes for 3 months on this repo also so I don't get you being rude. Please start with https://docs.yoctoproject.org/dev-manual/common-tasks.html#making-changes-to-the-yocto-project which documents the way to make contributions to the project. If you're using an automated tool please give a quick description of it alongside the changes, preferably with a link to its source code. You may also want to check https://git.openembedded.org/openembedded-core/tree/scripts/oe-pylint. Running that script should flag up errors which we do want to see fixed. If there are more warnings & errors to consider then these should be added to that script or to a pylint/pep8/flake8 config file so that we can run this against all new patches in the future. -- Paul Barker Konsulko Group -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#153433): https://lists.openembedded.org/g/openembedded-core/message/153433 Mute This Topic: https://lists.openembedded.org/mt/83869185/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH 0/8] Safe PEP8
On Tue, 29 Jun 2021 at 15:08, Armin Kuster wrote: > > I am forwarding a pull request; > https://github.com/openembedded/openembedded-core/pull/71 Looking at the comments on that pull request, "I did this on more than 100 repos" suggests this hasn't been done with any attention to the specifics of openembedded-core/bitbake. In my opinion that approach is a waste of maintainer's time. -- Paul Barker Konsulko Group -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#153432): https://lists.openembedded.org/g/openembedded-core/message/153432 Mute This Topic: https://lists.openembedded.org/mt/83869185/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH] prservice: Use new connect API
The new prserv connect() function decouples the code in oe-core from the
exact classes and implementation details used within bitbake. This
allows us to more easily switch over to a new asyncrpc based prservice.
Signed-off-by: Paul Barker
---
meta/lib/oe/prservice.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/meta/lib/oe/prservice.py b/meta/lib/oe/prservice.py
index fcdbe66c19..15ce060ff6 100644
--- a/meta/lib/oe/prservice.py
+++ b/meta/lib/oe/prservice.py
@@ -7,7 +7,7 @@ def prserv_make_conn(d, check = False):
host_params = list([_f for _f in (d.getVar("PRSERV_HOST") or
'').split(':') if _f])
try:
conn = None
-conn = prserv.serv.PRServerConnection(host_params[0],
int(host_params[1]))
+conn = prserv.serv.connect(host_params[0], int(host_params[1]))
if check:
if not conn.ping():
raise Exception('service not available')
--
2.26.2
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#151059):
https://lists.openembedded.org/g/openembedded-core/message/151059
Mute This Topic: https://lists.openembedded.org/mt/82425692/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core][dunfell V3 00/46] Patch review - pseudo file mode corruption fix V3
On Tue, 26 Jan 2021 at 14:42, Steve Sakoman wrote:
>
> Issues with undetected file mode corruption in pseudo have been identified.
>
> Fixes have been merged into master and gatesgarth over the past several
> months and
> things seem to have stabilized enough that we can consider backporting these
> fixes
> to dunfell.
>
> This is a somewhat more invasive change than normal. Specifically the user
> will
> be required to clean TMPDIR after this merge. It may also expose issues with
> other layers, though master and gategarth have probably helped to pave the
> way for
> this change.
>
> Personally I think that this fixes a serious enough issue that we should
> consider
> this series for dunfell.
>
> This third version removes:
>
> bitbake.conf: Remove TERM from default BB_HASHBASE_WHITELIST
>
> and hence the need for any bitbake changes.
>
> Let me know if you you are aware of any other pseudo issues which would
> necessitate
> delaying the merge of this series.
>
> Please have comments back by Thursday evening.
>
> Passed a-full on autobuilder:
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/1784
>
> The following changes since commit 897822233faef0f8f35dc1d8a39e1c4bc0550f1e:
>
> ppp: Whitelist CVE-2020-15704 (2021-01-19 04:22:10 -1000)
>
> are available in the Git repository at:
>
> git://git.openembedded.org/openembedded-core-contrib stable/dunfell-nut
>
> http://cgit.openembedded.org/openembedded-core-contrib/log/?h=stable/dunfell-nut
>
> Jack Mitchell (1):
> distutils3: allow setup.py to be run from a different directory to
> ${S}
>
> Joe Slater (1):
> pseudo: fix renaming to self
>
> Martin Jansa (2):
> base.bbclass: use os.path.normpath instead of just comparing WORKDIR
> and S as strings
> license.bbclass: Add COMMON_LICENSE_DIR and LICENSE_PATH dirs to
> PSEUDO_IGNORE_PATHS
>
> Mingli Yu (2):
> tcl: adapt to potential pseudo changes
> bitbake.conf: Exclude ${CCACHE_DIR} from pseudo database
>
> Paul Barker (8):
> bitbake.conf: Prevent pyc file generation in pseudo context
> wic: Add workdir argument
> wic: Allow exec_native_cmd to run HOSTTOOLS
> wic: Ensure internal workdir is not reused
> image_types_wic: Move wic working directory
> wic: Update pseudo db when excluding content from rootfs
> wic: Copy rootfs dir if fstab needs updating
> wic: Optimise fstab modification for ext2/3/4 and msdos partitions
>
> Peter Kjellerstedt (4):
> pseudo: Simplify pseudo_client_ignore_path_chroot()
> lib/oe/path: Add canonicalize()
> bitbake.conf: Canonicalize paths in PSEUDO_IGNORE_PATHS
> wic: Pass canonicalized paths in PSEUDO_IGNORE_PATHS
>
> Ricardo Ribalda Delgado (4):
> wic: Fix permissions when using exclude or include path
> wic: Fix multi images .wks with bitbake
> wic: Avoid creating invalid pseudo directory
> wic: Add --change-directory argument
>
> Richard Purdie (23):
> pseudo: Switch to oe-core branch in git repo
> pseudo: merge in fixes for setfacl issue
> pseudo: Update to add OFC fcntl lock updates
> pseudo: Ignore mismatched inodes from the db
> pseudo: Add support for ignoring paths from the pseudo DB
> pseudo: Abort on mismatch patch
> psuedo: Add tracking of linked files for fds
> pseudo: Fix xattr segfault
> pseudo: Add may unlink patch
> pseudo: Add pathfix patch
> base/bitbake.conf: Enable pseudo path filtering
> wic: Handle new PSEUDO_IGNORE_PATHS variable
> pseudo: Fix statx function usage
> bitbake.conf: Extend PSEUDO_IGNORE_PATHS to ${COREBASE}/meta
> abi_version,sanity: Tell users TMPDIR must be clean after pseudo
> changes
> pseudo: Update to account for patches merged on branch
> pseudo: Upgrade to include mkostemp64 wrapper
> oeqa/selftest/runtime_test: Exclude gpg directory from pseudo database
> uninative: Don't use single sstate for pseudo-native
> pseudo: Drop patches merged into upstream branch
> bitbake.conf: Add /run/ to PSEUDO_IGNORE_PATHS
> pseudo: Add lchmod wrapper
> pseudo: Update for arm host and memleak fixes/cleanup
>
> Tomasz Dziendzielski (1):
> pseudo: Update to print PSEUDO_LOGFILE in abort message on path
> mismatches
>
> meta-selftest/lib/pseudo_pyc_test1.py | 1 +
> meta-selftest/lib/pseudo_pyc_test2.py | 1 +
> .../pseudo-pyc-test/pseudo-pyc-test.bb| 15 ++
> meta/classes/archiver.bbclass | 2 +-
> meta/classes/base.bbclass | 6 +
> meta/classes/distutils3.bbclass | 10 +-
> meta/classes/image_types_wic.bbclass | 20 ++-
> meta/classes/license.
Re: [OE-core][dunfell 00/47] Patch review - pseudo file mode corruption fix V2
On Mon, 25 Jan 2021 at 16:25, Steve Sakoman wrote: > > On Mon, Jan 25, 2021 at 5:42 AM Paul Barker wrote: > > > > On Mon, 25 Jan 2021 at 14:51, Steve Sakoman wrote: > > > > > > Issues with undetected file mode corruption in pseudo have been > > > identified. > > > > > > Fixes have been merged into master and gatesgarth over the past several > > > months and > > > things seem to have stabilized enough that we can consider backporting > > > these fixes > > > to dunfell. > > > > > > This is a somewhat more invasive change than normal. Specifically the > > > user will > > > be required to clean TMPDIR after this merge. It may also expose issues > > > with > > > other layers, though master and gategarth have probably helped to pave > > > the way for > > > this change. > > > > > > Personally I think that this fixes a serious enough issue that we should > > > consider > > > this series for dunfell. > > > > > > This second version includes the wic patches which have been the gating > > > item for > > > merging this series. > > > > > > Note also that there is an associated bitbake patch for stable/1.46: > > > > > > lib/bb/utils.py: Do not preserve TERM in the environment > > > > > > This is due to taking: > > > > > > bitbake.conf: Remove TERM from default BB_HASHBASE_WHITELIST > > > > > > which was needed to do clean cherry-picks of the pseudo changes. > > > > If this is just brought in to allow clean cherry-picks I think it > > would be better left out and instead do the small bit of rebasing > > required in the patches. We don't know what downstream users are doing > > with TERM so minimising unnecessary changes in the stable branch is > > preferable. > > OK, will do. > > > Everything else looks good to me at a glance. > > Thanks for reviewing. I'll wait for more feedback and send a v3. > This is one series we really want to get right! Agreed. It's a difficult balance. I want stable branches to be as conservative as possible and not risk making breaking changes but this series fixes an important issue (risk of silently creating images with incorrect permissions). On balance it does make sense to backport these pretty invasive changes. Thanks, -- Paul Barker Konsulko Group -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#147219): https://lists.openembedded.org/g/openembedded-core/message/147219 Mute This Topic: https://lists.openembedded.org/mt/80104914/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core][dunfell 00/47] Patch review - pseudo file mode corruption fix V2
On Mon, 25 Jan 2021 at 14:51, Steve Sakoman wrote: > > Issues with undetected file mode corruption in pseudo have been identified. > > Fixes have been merged into master and gatesgarth over the past several > months and > things seem to have stabilized enough that we can consider backporting these > fixes > to dunfell. > > This is a somewhat more invasive change than normal. Specifically the user > will > be required to clean TMPDIR after this merge. It may also expose issues with > other layers, though master and gategarth have probably helped to pave the > way for > this change. > > Personally I think that this fixes a serious enough issue that we should > consider > this series for dunfell. > > This second version includes the wic patches which have been the gating item > for > merging this series. > > Note also that there is an associated bitbake patch for stable/1.46: > > lib/bb/utils.py: Do not preserve TERM in the environment > > This is due to taking: > > bitbake.conf: Remove TERM from default BB_HASHBASE_WHITELIST > > which was needed to do clean cherry-picks of the pseudo changes. If this is just brought in to allow clean cherry-picks I think it would be better left out and instead do the small bit of rebasing required in the patches. We don't know what downstream users are doing with TERM so minimising unnecessary changes in the stable branch is preferable. Everything else looks good to me at a glance. -- Paul Barker Konsulko Group -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#147217): https://lists.openembedded.org/g/openembedded-core/message/147217 Mute This Topic: https://lists.openembedded.org/mt/80104914/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [yocto] OpenEmbedded Virtual Stand at FOSDEM 2021
On Thu, 14 Jan 2021 at 11:02, Robert ber...@yocto.user wrote: > > Hi, > > On 13/01/2021 23:33, Paul Barker wrote: > > Hi folks, > > > > * Any participants in the project who want to help host the Matrix > > chat room between 09:00 and 18:00 each day of the event. This will > > likely involve introducing the project to folks dropping in the chat > > who aren't familiar with OpenEmbedded, answering basic questions and > > chatting about example uses of the project. You don't need to be a > > long-standing expert in the project to help out here! If you can do a > > couple of hours or a half day please let us know. > > Let's try to come up with some schedule when I should be there and just > let me know how the join the chat. > > > > > * Any contributions of video content to go along with the static web > > pages. I'm planning to record some short introductory video content > > but other contributions would also be welcome. Details on how to > > upload videos is expected in the near future but for now it would be > > good to just collect folks who are interested so we can discuss this > > further. > > > > If you're interested in any of the above please reply to me and/or the > > list. I look forward to virtually seeing many of you at FOSDEM 2021! > > What kind of video would you like? A small video which explains a > specific topic? > > Something like this?[1] > > [1] https://www.youtube.com/watch?v=kmpEN953pzs&feature=youtu.be I'd say video content for FOSDEM should be around the level of the things we'd show on the stand, short demos or introductions which can be understood by people not too familiar with the details of the project. I'm going to record a short "Introducing OpenEmbedded" presentation based around the way I usually pitch the project to folks who turn up at our FOSDEM stand and ask what the project is about. I may also do a quick build & boot demo with a Raspberry Pi if I have time to record that. Thanks, -- Paul Barker Konsulko Group -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#147124): https://lists.openembedded.org/g/openembedded-core/message/147124 Mute This Topic: https://lists.openembedded.org/mt/80031982/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH v2 5/5] wic: Optimise fstab modification for ext2/3/4 and msdos partitions
The fix for [Yocto #13994] required the rootfs directory to be copied
(using hardlinks if possible) when modifying the fstab file under wic.
We can optimise this copy away for filesystems where we have the tools
to modify the contents of the partition image after it is created. For
ext2/3/4 filesystems we have the debugfs tool and for msdos/vfat
filesystems we have the mcopy tool. So for any of these filesystems we
skip the modification of the fstab file in the rootfs directory (and
skip the associated copy unless it is otherwise necessary) and update
the contents of fstab directly in the partition image.
Signed-off-by: Paul Barker
---
scripts/lib/wic/partition.py | 27 +++-
scripts/lib/wic/plugins/source/rootfs.py | 9 +---
2 files changed, 28 insertions(+), 8 deletions(-)
diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
index f59eceb23d..e574f40c47 100644
--- a/scripts/lib/wic/partition.py
+++ b/scripts/lib/wic/partition.py
@@ -55,6 +55,8 @@ class Partition():
self.fsuuid = args.fsuuid
self.type = args.type
self.updated_fstab_path = None
+self.has_fstab = False
+self.update_fstab_in_rootfs = False
self.lineno = lineno
self.source_file = ""
@@ -125,6 +127,8 @@ class Partition():
partition command parameters.
"""
self.updated_fstab_path = updated_fstab_path
+if self.updated_fstab_path and not (self.fstype.startswith("ext") or
self.fstype == "msdos"):
+self.update_fstab_in_rootfs = True
if not self.source:
if not self.size and not self.fixed_size:
@@ -250,7 +254,7 @@ class Partition():
prefix = "ext" if self.fstype.startswith("ext") else self.fstype
method = getattr(self, "prepare_rootfs_" + prefix)
-method(rootfs, oe_builddir, rootfs_dir, native_sysroot, pseudo)
+method(rootfs, cr_workdir, oe_builddir, rootfs_dir, native_sysroot,
pseudo)
self.source_file = rootfs
# get the rootfs size in the right units for kickstart (kB)
@@ -258,7 +262,7 @@ class Partition():
out = exec_cmd(du_cmd)
self.size = int(out.split()[0])
-def prepare_rootfs_ext(self, rootfs, oe_builddir, rootfs_dir,
+def prepare_rootfs_ext(self, rootfs, cr_workdir, oe_builddir, rootfs_dir,
native_sysroot, pseudo):
"""
Prepare content for an ext2/3/4 rootfs partition.
@@ -282,10 +286,19 @@ class Partition():
(self.fstype, extraopts, rootfs, label_str, self.fsuuid,
rootfs_dir)
exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
+if self.updated_fstab_path and self.has_fstab:
+debugfs_script_path = os.path.join(cr_workdir, "debugfs_script")
+with open(debugfs_script_path, "w") as f:
+f.write("cd etc\n")
+f.write("rm fstab\n")
+f.write("write %s fstab\n" % (self.updated_fstab_path))
+debugfs_cmd = "debugfs -w -f %s %s" % (debugfs_script_path, rootfs)
+exec_native_cmd(debugfs_cmd, native_sysroot)
+
mkfs_cmd = "fsck.%s -pvfD %s" % (self.fstype, rootfs)
exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
-def prepare_rootfs_btrfs(self, rootfs, oe_builddir, rootfs_dir,
+def prepare_rootfs_btrfs(self, rootfs, cr_workdir, oe_builddir, rootfs_dir,
native_sysroot, pseudo):
"""
Prepare content for a btrfs rootfs partition.
@@ -308,7 +321,7 @@ class Partition():
self.mkfs_extraopts, self.fsuuid, rootfs)
exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
-def prepare_rootfs_msdos(self, rootfs, oe_builddir, rootfs_dir,
+def prepare_rootfs_msdos(self, rootfs, cr_workdir, oe_builddir, rootfs_dir,
native_sysroot, pseudo):
"""
Prepare content for a msdos/vfat rootfs partition.
@@ -337,12 +350,16 @@ class Partition():
mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (rootfs, rootfs_dir)
exec_native_cmd(mcopy_cmd, native_sysroot)
+if self.updated_fstab_path and self.has_fstab:
+mcopy_cmd = "mcopy -i %s %s ::/etc/fstab" % (rootfs,
self.updated_fstab_path)
+exec_native_cmd(mcopy_cmd, native_sysroot)
+
chmod_cmd = "chmod 644 %s" % rootfs
exec_cmd(chmod_cmd)
prepare_rootfs_vfat = prepare_rootfs_msdos
-def prepare_rootfs_squashfs(self, rootfs, oe_builddir, rootfs_dir,
+def prepare_rootfs_squashfs(self, rootfs, cr_workdir, oe_builddir,
rootfs_dir,
native_sysroot, pseudo):
"""
Prepare content
[OE-core] [PATCH v2 4/5] wic: Copy rootfs dir if fstab needs updating
By default, wic updates the /etc/fstab in the rootfs to include details
of additional partitions described in the selected wks file. If this
modification is performed in place, other tasks which create an image
file from the rootfs directory (e.g. do_image_tar and do_image_ext4)
will pick up the modified fstab file which would not be appropriate for
those images as they do not include the additional partitions described
in the wks file. wic does undo modifications to the fstab file once it
has finished creating the filesystem image, however this leaves open a
race condition if one of the other tasks reads the contents of the fstab
file from the rootfs directory between the point where wic modifies the
fstab file and the point where wic restores the files original content.
This could be solved by adding a lockfile for tasks which use the rootfs
directory to ensure that no other such task is reading the rootfs
directory while do_image_wic is running. This would serialize several
do_image_* tasks and result in slower builds, especially for large
images. Another drawback of this solution is that it is hard to
selectively optimise - adding lockfiles to do_image_* tasks would result
in these tasks always being serialized even if no fstab modification
will take place.
An alternative solution is to copy the rootfs directory when fstab needs
to be modified. The code to do this in wic already exists as it is
needed when including or excluding content in the rootfs. This still
results in an impact on build times but the copy uses hardlinks if
possible (so little data is actually copied) and we can make selective
optimisations to improve things. The rootfs copy will only take place if
fstab modification is required (or if it was already needed to include
or exclude rootfs content). We can also follow up with further
optimisations after this commit. So this second solution is chosen.
Fixes [Yocto #13994]
Signed-off-by: Paul Barker
---
scripts/lib/wic/partition.py | 5 +++-
scripts/lib/wic/plugins/imager/direct.py | 36
scripts/lib/wic/plugins/source/rootfs.py | 17 +--
3 files changed, 29 insertions(+), 29 deletions(-)
diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
index 286c7867cb..f59eceb23d 100644
--- a/scripts/lib/wic/partition.py
+++ b/scripts/lib/wic/partition.py
@@ -54,6 +54,7 @@ class Partition():
self.uuid = args.uuid
self.fsuuid = args.fsuuid
self.type = args.type
+self.updated_fstab_path = None
self.lineno = lineno
self.source_file = ""
@@ -118,11 +119,13 @@ class Partition():
return self.fixed_size if self.fixed_size else self.size
def prepare(self, creator, cr_workdir, oe_builddir, rootfs_dir,
-bootimg_dir, kernel_dir, native_sysroot):
+bootimg_dir, kernel_dir, native_sysroot, updated_fstab_path):
"""
Prepare content for individual partitions, depending on
partition command parameters.
"""
+self.updated_fstab_path = updated_fstab_path
+
if not self.source:
if not self.size and not self.fixed_size:
raise WicError("The %s partition has a size of zero. Please "
diff --git a/scripts/lib/wic/plugins/imager/direct.py
b/scripts/lib/wic/plugins/imager/direct.py
index f107e60089..7e1c1c03ab 100644
--- a/scripts/lib/wic/plugins/imager/direct.py
+++ b/scripts/lib/wic/plugins/imager/direct.py
@@ -58,7 +58,7 @@ class DirectPlugin(ImagerPlugin):
self.compressor = options.compressor
self.bmap = options.bmap
self.no_fstab_update = options.no_fstab_update
-self.original_fstab = None
+self.updated_fstab_path = None
self.name = "%s-%s" % (os.path.splitext(os.path.basename(wks_file))[0],
strftime("%Y%m%d%H%M"))
@@ -100,11 +100,8 @@ class DirectPlugin(ImagerPlugin):
finally:
self.cleanup()
-def _write_fstab(self, image_rootfs):
-"""overriden to generate fstab (temporarily) in rootfs. This is called
-from _create, make sure it doesn't get called from
-BaseImage.create()
-"""
+def update_fstab(self, image_rootfs):
+"""Assume partition order same as in wks"""
if not image_rootfs:
return
@@ -114,18 +111,9 @@ class DirectPlugin(ImagerPlugin):
with open(fstab_path) as fstab:
fstab_lines = fstab.readlines()
-self.original_fstab = fstab_lines.copy()
-
-if self._update_fstab(fstab_lines, self.parts):
-with open(fstab_path, "w") as fstab:
-fstab.writelines(fstab_lines)
-else:
-self.original_fstab = None
-def _update_fstab(self, fstab_lines, parts):
-
[OE-core] [PATCH v2 3/5] wic: Update pseudo db when excluding content from rootfs
To exclude content from the rootfs, wic makes a copy (using hardlinks if possible) of the rootfs directory and associated pseudo db, then removes files & directories as needed. However if these files and directories are removed using the python functions os.remove and shutil.rmtree, the copied pseudo db will not be updated correctly. For files copied from the original rootfs, if hardlinks were used successfully when copying the rootfs this should mean that the relevant inodes can't be reused and so the risk of pseudo aborts should be avoided. However, this logic doesn't apply for directories (as they can't be hardlinked) or for files added via the '--include-path' argument (as they weren't present in the original rootfs) and so there remains some risk of inodes being reused and the pseudo db becoming corrupted. To fix this, use the 'rm' command under pseudo when removing files & directories from the copied rootfs to ensure that the copied pseudo db is updated. Signed-off-by: Paul Barker --- scripts/lib/wic/plugins/source/rootfs.py | 13 - 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/scripts/lib/wic/plugins/source/rootfs.py b/scripts/lib/wic/plugins/source/rootfs.py index f1db83f8a1..afb1eb9202 100644 --- a/scripts/lib/wic/plugins/source/rootfs.py +++ b/scripts/lib/wic/plugins/source/rootfs.py @@ -199,17 +199,20 @@ class RootfsPlugin(SourcePlugin): if not os.path.lexists(full_path): continue +if new_pseudo: +pseudo = cls.__get_pseudo(native_sysroot, new_rootfs, new_pseudo) +else: +pseudo = None if path.endswith(os.sep): # Delete content only. for entry in os.listdir(full_path): full_entry = os.path.join(full_path, entry) -if os.path.isdir(full_entry) and not os.path.islink(full_entry): -shutil.rmtree(full_entry) -else: -os.remove(full_entry) +rm_cmd = "rm -rf %s" % (full_entry) +exec_native_cmd(rm_cmd, native_sysroot, pseudo) else: # Delete whole directory. -shutil.rmtree(full_path) +rm_cmd = "rm -rf %s" % (full_path) +exec_native_cmd(rm_cmd, native_sysroot, pseudo) part.prepare_rootfs(cr_workdir, oe_builddir, new_rootfs or part.rootfs_dir, native_sysroot, -- 2.26.2 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#146996): https://lists.openembedded.org/g/openembedded-core/message/146996 Mute This Topic: https://lists.openembedded.org/mt/79955375/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH v2 2/5] image_types_wic: Move wic working directory
By default the wic working directory is placed under the output
directory. When invoking wic under bitbake, the wic output directory is
added to PSEUDO_PATHS_IGNORE to avoid issues with files being removed
from outside a pseudo environment (see oe-core commit ad8f5532ff).
However, wic will copy the rootfs directory into its working directory
if it needs to add or remove content before creating a filesystem image.
This copy of the rootfs directory must be tracked by pseudo in order to
keep the permissions correct in the resulting image. So we can't have
the wic working directory under a path in PSEUDO_PATHS_IGNORE unless
we like broken permissions.
To fix this the new '-w' argument to wic is used to move the working
directory away from the output directory.
Note that wic deletes the temporary working directory automatically
when it finishes creating an image so users won't normally see the
'tmp-wic' directory under WORKDIR.
Fixes [Yocto #14129]
Signed-off-by: Paul Barker
---
meta/classes/image_types_wic.bbclass | 8 +++-
1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/meta/classes/image_types_wic.bbclass
b/meta/classes/image_types_wic.bbclass
index 000ee4249f..49be1da77a 100644
--- a/meta/classes/image_types_wic.bbclass
+++ b/meta/classes/image_types_wic.bbclass
@@ -29,11 +29,17 @@ WIC_CREATE_EXTRA_ARGS ?= ""
IMAGE_CMD_wic () {
out="${IMGDEPLOYDIR}/${IMAGE_NAME}"
build_wic="${WORKDIR}/build-wic"
+ tmp_wic="${WORKDIR}/tmp-wic"
wks="${WKS_FULL_PATH}"
+ if [ -e "$tmp_wic" ]; then
+ # Ensure we don't have any junk leftover from a previously
interrupted
+ # do_image_wic execution
+ rm -rf "$tmp_wic"
+ fi
if [ -z "$wks" ]; then
bbfatal "No kickstart files from WKS_FILES were found:
${WKS_FILES}. Please set WKS_FILE or WKS_FILES appropriately."
fi
- BUILDDIR="${TOPDIR}" PSEUDO_UNLOAD=1 wic create "$wks" --vars
"${STAGING_DIR}/${MACHINE}/imgdata/" -e "${IMAGE_BASENAME}" -o "$build_wic/"
${WIC_CREATE_EXTRA_ARGS}
+ BUILDDIR="${TOPDIR}" PSEUDO_UNLOAD=1 wic create "$wks" --vars
"${STAGING_DIR}/${MACHINE}/imgdata/" -e "${IMAGE_BASENAME}" -o "$build_wic/" -w
"$tmp_wic" ${WIC_CREATE_EXTRA_ARGS}
mv "$build_wic/$(basename "${wks%.wks}")"*.direct
"$out${IMAGE_NAME_SUFFIX}.wic"
}
IMAGE_CMD_wic[vardepsexclude] = "WKS_FULL_PATH WKS_FILES TOPDIR"
--
2.26.2
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#146995):
https://lists.openembedded.org/g/openembedded-core/message/146995
Mute This Topic: https://lists.openembedded.org/mt/79955373/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH v2 1/5] wic: Ensure internal workdir is not reused
If a path is specified for the internal wic working directory using
the -w/--workdir argument then it must not already exist. Re-using a
previous workdir could easily result in rootfs and intermediate files
from a previous build being added to the current image.
Signed-off-by: Paul Barker
---
scripts/lib/wic/plugins/imager/direct.py | 12 +++-
1 file changed, 11 insertions(+), 1 deletion(-)
diff --git a/scripts/lib/wic/plugins/imager/direct.py
b/scripts/lib/wic/plugins/imager/direct.py
index b329568c7a..f107e60089 100644
--- a/scripts/lib/wic/plugins/imager/direct.py
+++ b/scripts/lib/wic/plugins/imager/direct.py
@@ -62,7 +62,7 @@ class DirectPlugin(ImagerPlugin):
self.name = "%s-%s" % (os.path.splitext(os.path.basename(wks_file))[0],
strftime("%Y%m%d%H%M"))
-self.workdir = options.workdir or tempfile.mkdtemp(dir=self.outdir,
prefix='tmp.wic.')
+self.workdir = self.setup_workdir(options.workdir)
self._image = None
self.ptable_format = self.ks.bootloader.ptable
self.parts = self.ks.partitions
@@ -78,6 +78,16 @@ class DirectPlugin(ImagerPlugin):
self._image = PartitionedImage(image_path, self.ptable_format,
self.parts, self.native_sysroot)
+def setup_workdir(self, workdir):
+if workdir:
+if os.path.exists(workdir):
+raise WicError("Internal workdir '%s' specified in wic
arguments already exists!" % (workdir))
+
+os.makedirs(workdir)
+return workdir
+else:
+return tempfile.mkdtemp(dir=self.outdir, prefix='tmp.wic.')
+
def do_create(self):
"""
Plugin entry point.
--
2.26.2
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#146994):
https://lists.openembedded.org/g/openembedded-core/message/146994
Mute This Topic: https://lists.openembedded.org/mt/79955372/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH] devtool: Fix source extraction for gcc shared source
On Tue, 19 Jan 2021 at 15:51, Tomasz Dziendzielski
wrote:
>
> If do_patch task is disabled then prepare do_configure dependencies to
> fetch external sources and create symlink to ${S} in devtool workspace.
>
> [YOCTO #13036]
>
> Signed-off-by: Tomasz Dziendzielski
> ---
> .../gcc/gcc-shared-source.inc | 1 +
> scripts/lib/devtool/standard.py | 30 ---
> 2 files changed, 27 insertions(+), 4 deletions(-)
>
> diff --git a/meta/recipes-devtools/gcc/gcc-shared-source.inc
> b/meta/recipes-devtools/gcc/gcc-shared-source.inc
> index aac4b49313..9ef80f2074 100644
> --- a/meta/recipes-devtools/gcc/gcc-shared-source.inc
> +++ b/meta/recipes-devtools/gcc/gcc-shared-source.inc
> @@ -2,6 +2,7 @@ do_fetch() {
> :
> }
> do_fetch[noexec] = "1"
> +do_patch[noexec] = "1"
Have you tested whether the archiver class is still working after this
change? If not it'd be helpful to run `oe-selftest -r archiver` to
check.
--
Paul Barker
Konsulko Group
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#146993):
https://lists.openembedded.org/g/openembedded-core/message/146993
Mute This Topic: https://lists.openembedded.org/mt/79954315/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [meta-oe][PATCH v2 0/3] kernel-fitimage: adding support for Initramfs bundle and u-boot script
On Tue, 19 Jan 2021 at 10:53, Abdellatif El Khlifi wrote: > > From: Abdellatif El Khlifi > > This is the version 2 of adding support to Initramfs bundles and u-boot > script in the FIT image. > > This patchset provides the following: > > - Initramfs bundle support in the FIT image kernel class > - Allowing to use the Initramfs bundle with the FIT image in the kernel class > - A self-contained test case allowing to validate the Initramfs bundle > support in the FIT image > - u-boot script support in the FIT image kernel class > > A 4th patch will be sent to the Yocto documentation which describes the new > features. > > Abdellatif El Khlifi (3): > kernel-fitimage: adding support for Initramfs bundle and u-boot script > kernel: skip installing fitImage when using Initramfs bundles > oeqa/selftest/imagefeatures: adding fitImage initramfs bundle testcase > > meta/classes/kernel-fitimage.bbclass | 160 +++ > meta/classes/kernel.bbclass | 16 ++- > meta/lib/oeqa/selftest/cases/fitimage.py | 132 +++ > 3 files changed, 284 insertions(+), 24 deletions(-) > > -- > 2.17.1 I think the `[meta-oe]` tag in the email subject is wrong here. You don't need to resend the patch though as that isn't included in the commit message. This is just a note for next time. Thanks, -- Paul Barker Konsulko Group -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#146976): https://lists.openembedded.org/g/openembedded-core/message/146976 Mute This Topic: https://lists.openembedded.org/mt/79948575/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH] appliance: Add vhd & vhdx support
dimg/wic.vmdk/wic.qcow2/wic.vdi/ramfs/cpio.gz...
> + runqemu qemux86
> iso/hddimg/wic.vmdk/wic.vhd/wic.vhdx/wic.qcow2/wic.vdi/ramfs/cpio.gz...
>runqemu qemux86 qemuparams="-m 256"
>runqemu qemux86 bootparams="psplash=false"
>runqemu path/to/-.wic
>runqemu path/to/-.wic.vmdk
> + runqemu path/to/-.wic.vhdx
> + runqemu path/to/-.wic.vhd
> """)
>
> def check_tun():
> @@ -179,7 +181,7 @@ class BaseConfig(object):
> self.portlocks = {}
> self.bitbake_e = ''
> self.snapshot = False
> -self.wictypes = ('wic', 'wic.vmdk', 'wic.qcow2', 'wic.vdi')
> +self.wictypes = ('wic', 'wic.vmdk', 'wic.qcow2', 'wic.vdi',
> "wic.vhd", "wic.vhdx")
> self.fstypes = ('ext2', 'ext3', 'ext4', 'jffs2', 'nfs', 'btrfs',
> 'cpio.gz', 'cpio', 'ramfs', 'tar.bz2', 'tar.gz')
> self.vmtypes = ('hddimg', 'iso')
> @@ -1203,7 +1205,7 @@ class BaseConfig(object):
> return
> if 'wic.' in self.fstype:
> self.fstype = self.fstype[4:]
> -rootfs_format = self.fstype if self.fstype in ('vmdk', 'qcow2',
> 'vdi') else 'raw'
> +rootfs_format = self.fstype if self.fstype in ('vmdk', 'vhd',
> 'vhdx', 'qcow2', 'vdi') else 'raw'
>
> qb_rootfs_opt = self.get('QB_ROOTFS_OPT')
> if qb_rootfs_opt:
I actually recommend splitting the runqemu changes off into another
patch. So you'd have one patch for image_types, one patch for bitbake
and one patch for runqemu.
Thanks,
--
Paul Barker
Konsulko Group
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#146870):
https://lists.openembedded.org/g/openembedded-core/message/146870
Mute This Topic: https://lists.openembedded.org/mt/79674371/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] OpenEmbedded Virtual Stand at FOSDEM 2021
Hi folks, OpenEmbedded has been accepted to hold a "virtual stand" at FOSDEM 2021 which is taking place online this year on Sat 6th Feb & Sun 7th Feb, based around the CET timezone. Most of you will be familiar with FOSDEM by now, but if not let me give a quick bit of background on the event. This is the Free and Open Source Developers' European Meeting and usually takes place in Brussels in early February each year. The event won't be happening in-person in Brussels this year due to the current pandemic but I think it's still important to take advantage of this opportunity of connecting with other contributors to OpenEmbedded & Yocto Project as well as other members and projects within the wider open source community. For more information see the FOSDEM 2021 website at https://fosdem.org/2021/. This virtual stand will consist of a Matrix chat room (likely with Jitsi video chat) along with hosting space for static web pages & video within the FOSDEM 2021 website. The details on all of this are still being figured out but right now we've got enough information to start planning our involvement. What we're looking for is: * Any participants in the project who want to help host the Matrix chat room between 09:00 and 18:00 each day of the event. This will likely involve introducing the project to folks dropping in the chat who aren't familiar with OpenEmbedded, answering basic questions and chatting about example uses of the project. You don't need to be a long-standing expert in the project to help out here! If you can do a couple of hours or a half day please let us know. * Any folks who would like to help build the static web pages for the stands section of the FOSDEM 2021 website. FOSDEM is using the Hugo static website generator so the content will be written as Markdown files which are converted to html by hugo. Further information is available in the README file at https://github.com/FOSDEM/stands-website/blob/master/README.md. I'm hoping that someone is willing to take the lead on this as I'm already very busy in the runup to FOSDEM. I'm available to help set up git repositories, test things out and review content. * Any contributions of video content to go along with the static web pages. I'm planning to record some short introductory video content but other contributions would also be welcome. Details on how to upload videos is expected in the near future but for now it would be good to just collect folks who are interested so we can discuss this further. If you're interested in any of the above please reply to me and/or the list. I look forward to virtually seeing many of you at FOSDEM 2021! Thanks, -- Paul Barker Konsulko Group -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#146662): https://lists.openembedded.org/g/openembedded-core/message/146662 Mute This Topic: https://lists.openembedded.org/mt/79662007/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH] rootfs_ipk: allow do_populate_sdk in parallel to do_rootfs
On Tue, 12 Jan 2021 at 09:21, Richard Purdie wrote: > > On Mon, 2021-01-11 at 13:45 +0100, Michael Ho wrote: > > From: Michael Ho > > > > Switch do_populate_sdk for the ipk package manager to use a separate target > > opkg config file and separate the lockfiles restricting do_rootfs and > > do_populate_sdk from running in parallel. > > > > This way if an image recipe includes a dependency to do_populate_sdk by > > default then it will run in parallel to do_rootfs saving time compared to > > the > > sequential execution. > > > > Signed-off-by: Michael Ho > > --- > > meta/classes/package_ipk.bbclass | 1 + > > meta/classes/rootfs_ipk.bbclass| 4 ++-- > > meta/lib/oe/package_manager/ipk/sdk.py | 6 ++ > > 3 files changed, 9 insertions(+), 2 deletions(-) > > I have to admit I'm very nervous about this change. The races we've > seen betweem rootfs and sdk can be quite unusual. > > I did put this in for testing and we saw: > > https://autobuilder.yoctoproject.org/typhoon/#/builders/65/builds/2938 > > which in the pseudo.log shows: > > path mismatch [2 links]: ino 372706913 db > '/home/pokybuild/yocto-worker/beaglebone/build/build/tmp/work/beaglebone_yocto-poky-linux-gnueabi/core-image-sato/1.0-r0/tmp-wic' > req > '/home/pokybuild/yocto-worker/beaglebone/build/build/tmp/work/beaglebone_yocto-poky-linux-gnueabi/core-image-sato/1.0-r0/sdk/image/opt/poky/3.2+snapshot/sysroots/cortexa8hf-neon-poky-linux-gnueabi/usr/src/debug/puzzles'. > path mismatch [2 links]: ino 372706913 db > '/home/pokybuild/yocto-worker/beaglebone/build/build/tmp/work/beaglebone_yocto-poky-linux-gnueabi/core-image-sato/1.0-r0/tmp-wic' > req > '/home/pokybuild/yocto-worker/beaglebone/build/build/tmp/work/beaglebone_yocto-poky-linux-gnueabi/core-image-sato/1.0-r0/sdk/image/opt/poky/3.2+snapshot/sysroots/cortexa8hf-neon-poky-linux-gnueabi/usr/src/debug/puzzles'. > > its hard to know if this is due to this patch or possibly Paul's wic > changes as both were in this test series. I suspect it won't reproduce > every time since its a race. Those path mismatches do seem to show that the `tmp-wic` directory was removed outside of a pseudo context. That would point the finger at my patches. wic does delete its temporary working directory when exiting (http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/scripts/lib/wic/plugins/imager/direct.py?h=master-next&id=b86e5b32f767b69bb770f6060cb784b15b147f22#n268) but that should be ok as wic itself is running under pseudo when invoked by bitbake. So when do_image_wic finishes, the path to `tmp-wic` should no longer be present in the pseudo db. The error seen here may be due to a race condition but the underlying question of whether `tmp-wic` is removed outside a pseudo context (leaving the path in the db) should hopefully be deterministic. Is there a way to query the pseudo db to see if a path is present or not? I'd like to try that after running do_image_wic. Thanks, -- Paul Barker Konsulko Group -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#146613): https://lists.openembedded.org/g/openembedded-core/message/146613 Mute This Topic: https://lists.openembedded.org/mt/79594382/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH 1/6] wic: Add workdir argument
On Mon, 11 Jan 2021 at 14:22, Paul Barker wrote: > > On Mon, 11 Jan 2021 at 14:17, Lee, Chee Yang wrote: > > > > You might want to update scripts/lib/wic/help.py with description for this > > new features too > > /me looks at help.py > > That is really frustrating. The help output should be generated from > the argument parsers instead of carrying hand-written duplicate > information. I'll send a follow up patch to add to the documentation in help.py. -- Paul Barker Konsulko Group -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#146594): https://lists.openembedded.org/g/openembedded-core/message/146594 Mute This Topic: https://lists.openembedded.org/mt/79592782/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH 1/6] wic: Add workdir argument
On Mon, 11 Jan 2021 at 14:17, Lee, Chee Yang wrote: > > You might want to update scripts/lib/wic/help.py with description for this > new features too /me looks at help.py That is really frustrating. The help output should be generated from the argument parsers instead of carrying hand-written duplicate information. -- Paul Barker Konsulko Group -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#146593): https://lists.openembedded.org/g/openembedded-core/message/146593 Mute This Topic: https://lists.openembedded.org/mt/79592782/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-Core][PATCH v3] oe-pkgdata-util: Added a test to verify oe-pkgdata-util without parameters
On Mon, 11 Jan 2021 at 12:19, Milan Shah wrote: > > Hi All, > > This is a gentle reminder to review this patch. Looks like this is also in master already: https://git.openembedded.org/openembedded-core/commit/?id=6b4e2eafa6eb71ca94ccc8a18d05b473b352367c -- Paul Barker Konsulko Group -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#146587): https://lists.openembedded.org/g/openembedded-core/message/146587 Mute This Topic: https://lists.openembedded.org/mt/79418977/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH 6/6] wic: Optimise fstab modification for ext2/3/4 and msdos partitions
On Mon, 11 Jan 2021 at 10:37, Alexander Kanavin wrote: > > Oh right nevermind - it's an optimization, not an actual fix which is > elsewhere. Yes, for other fstypes we just fallback on copying the rootfs directory (using hardlinks if possible). -- Paul Barker Konsulko Group -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#146585): https://lists.openembedded.org/g/openembedded-core/message/146585 Mute This Topic: https://lists.openembedded.org/mt/79592788/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core][dunfell 11/12] selftest: Add argument to keep build dir
On Mon, 11 Jan 2021 at 00:46, Steve Sakoman wrote: > > From: Paul Barker > > The oe-selftest code already keeps the selftest build directory in place > if any tests failed. By default the build directory is deleted if all > tests pass but there may be cases where it's desirable to keep this > directory around, for example to compare intermediate files between > passing and failing test runs. > > Signed-off-by: Paul Barker > Signed-off-by: Richard Purdie > (cherry picked from commit 67aa7069dbe8f5f5f186eb67708ece5c4bd42976) > Signed-off-by: Steve Sakoman I was going to suggest backporting this one even though it is not explicitly a bugfix as it's low risk and helps with testing. Thanks for picking this up. Anuj, could we also get this backported to gatesgarth? Thanks, -- Paul Barker Konsulko Group -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#146582): https://lists.openembedded.org/g/openembedded-core/message/146582 Mute This Topic: https://lists.openembedded.org/mt/79585739/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH 6/6] wic: Optimise fstab modification for ext2/3/4 and msdos partitions
The fix for [Yocto #13994] required the rootfs directory to be copied
(using hardlinks if possible) when modifying the fstab file under wic.
We can optimise this copy away for filesystems where we have the tools
to modify the contents of the partition image after it is created. For
ext2/3/4 filesystems we have the debugfs tool and for msdos/vfat
filesystems we have the mcopy tool. So for any of these filesystems we
skip the modification of the fstab file in the rootfs directory (and
skip the associated copy unless it is otherwise necessary) and update
the contents of fstab directly in the partition image.
Signed-off-by: Paul Barker
---
scripts/lib/wic/partition.py | 27 +++-
scripts/lib/wic/plugins/source/rootfs.py | 9 +---
2 files changed, 28 insertions(+), 8 deletions(-)
diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
index f59eceb23d..e574f40c47 100644
--- a/scripts/lib/wic/partition.py
+++ b/scripts/lib/wic/partition.py
@@ -55,6 +55,8 @@ class Partition():
self.fsuuid = args.fsuuid
self.type = args.type
self.updated_fstab_path = None
+self.has_fstab = False
+self.update_fstab_in_rootfs = False
self.lineno = lineno
self.source_file = ""
@@ -125,6 +127,8 @@ class Partition():
partition command parameters.
"""
self.updated_fstab_path = updated_fstab_path
+if self.updated_fstab_path and not (self.fstype.startswith("ext") or
self.fstype == "msdos"):
+self.update_fstab_in_rootfs = True
if not self.source:
if not self.size and not self.fixed_size:
@@ -250,7 +254,7 @@ class Partition():
prefix = "ext" if self.fstype.startswith("ext") else self.fstype
method = getattr(self, "prepare_rootfs_" + prefix)
-method(rootfs, oe_builddir, rootfs_dir, native_sysroot, pseudo)
+method(rootfs, cr_workdir, oe_builddir, rootfs_dir, native_sysroot,
pseudo)
self.source_file = rootfs
# get the rootfs size in the right units for kickstart (kB)
@@ -258,7 +262,7 @@ class Partition():
out = exec_cmd(du_cmd)
self.size = int(out.split()[0])
-def prepare_rootfs_ext(self, rootfs, oe_builddir, rootfs_dir,
+def prepare_rootfs_ext(self, rootfs, cr_workdir, oe_builddir, rootfs_dir,
native_sysroot, pseudo):
"""
Prepare content for an ext2/3/4 rootfs partition.
@@ -282,10 +286,19 @@ class Partition():
(self.fstype, extraopts, rootfs, label_str, self.fsuuid,
rootfs_dir)
exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
+if self.updated_fstab_path and self.has_fstab:
+debugfs_script_path = os.path.join(cr_workdir, "debugfs_script")
+with open(debugfs_script_path, "w") as f:
+f.write("cd etc\n")
+f.write("rm fstab\n")
+f.write("write %s fstab\n" % (self.updated_fstab_path))
+debugfs_cmd = "debugfs -w -f %s %s" % (debugfs_script_path, rootfs)
+exec_native_cmd(debugfs_cmd, native_sysroot)
+
mkfs_cmd = "fsck.%s -pvfD %s" % (self.fstype, rootfs)
exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
-def prepare_rootfs_btrfs(self, rootfs, oe_builddir, rootfs_dir,
+def prepare_rootfs_btrfs(self, rootfs, cr_workdir, oe_builddir, rootfs_dir,
native_sysroot, pseudo):
"""
Prepare content for a btrfs rootfs partition.
@@ -308,7 +321,7 @@ class Partition():
self.mkfs_extraopts, self.fsuuid, rootfs)
exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
-def prepare_rootfs_msdos(self, rootfs, oe_builddir, rootfs_dir,
+def prepare_rootfs_msdos(self, rootfs, cr_workdir, oe_builddir, rootfs_dir,
native_sysroot, pseudo):
"""
Prepare content for a msdos/vfat rootfs partition.
@@ -337,12 +350,16 @@ class Partition():
mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (rootfs, rootfs_dir)
exec_native_cmd(mcopy_cmd, native_sysroot)
+if self.updated_fstab_path and self.has_fstab:
+mcopy_cmd = "mcopy -i %s %s ::/etc/fstab" % (rootfs,
self.updated_fstab_path)
+exec_native_cmd(mcopy_cmd, native_sysroot)
+
chmod_cmd = "chmod 644 %s" % rootfs
exec_cmd(chmod_cmd)
prepare_rootfs_vfat = prepare_rootfs_msdos
-def prepare_rootfs_squashfs(self, rootfs, oe_builddir, rootfs_dir,
+def prepare_rootfs_squashfs(self, rootfs, cr_workdir, oe_builddir,
rootfs_dir,
native_sysroot, pseudo):
"""
Prepare content
[OE-core] [PATCH 5/6] wic: Copy rootfs dir if fstab needs updating
By default, wic updates the /etc/fstab in the rootfs to include details
of additional partitions described in the selected wks file. If this
modification is performed in place, other tasks which create an image
file from the rootfs directory (e.g. do_image_tar and do_image_ext4)
will pick up the modified fstab file which would not be appropriate for
those images as they do not include the additional partitions described
in the wks file. wic does undo modifications to the fstab file once it
has finished creating the filesystem image, however this leaves open a
race condition if one of the other tasks reads the contents of the fstab
file from the rootfs directory between the point where wic modifies the
fstab file and the point where wic restores the files original content.
This could be solved by adding a lockfile for tasks which use the rootfs
directory to ensure that no other such task is reading the rootfs
directory while do_image_wic is running. This would serialize several
do_image_* tasks and result in slower builds, especially for large
images. Another drawback of this solution is that it is hard to
selectively optimise - adding lockfiles to do_image_* tasks would result
in these tasks always being serialized even if no fstab modification
will take place.
An alternative solution is to copy the rootfs directory when fstab needs
to be modified. The code to do this in wic already exists as it is
needed when including or excluding content in the rootfs. This still
results in an impact on build times but the copy uses hardlinks if
possible (so little data is actually copied) and we can make selective
optimisations to improve things. The rootfs copy will only take place if
fstab modification is required (or if it was already needed to include
or exclude rootfs content). We can also follow up with further
optimisations after this commit. So this second solution is chosen.
Fixes [Yocto #13994]
Signed-off-by: Paul Barker
---
scripts/lib/wic/partition.py | 5 +++-
scripts/lib/wic/plugins/imager/direct.py | 36
scripts/lib/wic/plugins/source/rootfs.py | 17 +--
3 files changed, 29 insertions(+), 29 deletions(-)
diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
index 286c7867cb..f59eceb23d 100644
--- a/scripts/lib/wic/partition.py
+++ b/scripts/lib/wic/partition.py
@@ -54,6 +54,7 @@ class Partition():
self.uuid = args.uuid
self.fsuuid = args.fsuuid
self.type = args.type
+self.updated_fstab_path = None
self.lineno = lineno
self.source_file = ""
@@ -118,11 +119,13 @@ class Partition():
return self.fixed_size if self.fixed_size else self.size
def prepare(self, creator, cr_workdir, oe_builddir, rootfs_dir,
-bootimg_dir, kernel_dir, native_sysroot):
+bootimg_dir, kernel_dir, native_sysroot, updated_fstab_path):
"""
Prepare content for individual partitions, depending on
partition command parameters.
"""
+self.updated_fstab_path = updated_fstab_path
+
if not self.source:
if not self.size and not self.fixed_size:
raise WicError("The %s partition has a size of zero. Please "
diff --git a/scripts/lib/wic/plugins/imager/direct.py
b/scripts/lib/wic/plugins/imager/direct.py
index b329568c7a..d0f228567e 100644
--- a/scripts/lib/wic/plugins/imager/direct.py
+++ b/scripts/lib/wic/plugins/imager/direct.py
@@ -58,7 +58,7 @@ class DirectPlugin(ImagerPlugin):
self.compressor = options.compressor
self.bmap = options.bmap
self.no_fstab_update = options.no_fstab_update
-self.original_fstab = None
+self.updated_fstab_path = None
self.name = "%s-%s" % (os.path.splitext(os.path.basename(wks_file))[0],
strftime("%Y%m%d%H%M"))
@@ -90,11 +90,8 @@ class DirectPlugin(ImagerPlugin):
finally:
self.cleanup()
-def _write_fstab(self, image_rootfs):
-"""overriden to generate fstab (temporarily) in rootfs. This is called
-from _create, make sure it doesn't get called from
-BaseImage.create()
-"""
+def update_fstab(self, image_rootfs):
+"""Assume partition order same as in wks"""
if not image_rootfs:
return
@@ -104,18 +101,9 @@ class DirectPlugin(ImagerPlugin):
with open(fstab_path) as fstab:
fstab_lines = fstab.readlines()
-self.original_fstab = fstab_lines.copy()
-
-if self._update_fstab(fstab_lines, self.parts):
-with open(fstab_path, "w") as fstab:
-fstab.writelines(fstab_lines)
-else:
-self.original_fstab = None
-def _update_fstab(self, fstab_lines, parts):
-
[OE-core] [PATCH 3/6] wic: Allow exec_native_cmd to run HOSTTOOLS
This allows programs from HOSTTOOLS (e.g. 'install', 'rm', 'mv', etc) to
be more easily executed by wic. Without this change only programs from
an actual *-native recipe built by bitbake can be executed by wic.
Signed-off-by: Paul Barker
---
meta/classes/image_types_wic.bbclass | 2 +-
scripts/lib/wic/misc.py | 7 +--
2 files changed, 6 insertions(+), 3 deletions(-)
diff --git a/meta/classes/image_types_wic.bbclass
b/meta/classes/image_types_wic.bbclass
index a5ed780ceb..48f015de9b 100644
--- a/meta/classes/image_types_wic.bbclass
+++ b/meta/classes/image_types_wic.bbclass
@@ -3,7 +3,7 @@
WICVARS ?= "\
BBLAYERS IMGDEPLOYDIR DEPLOY_DIR_IMAGE FAKEROOTCMD IMAGE_BASENAME
IMAGE_EFI_BOOT_FILES IMAGE_BOOT_FILES \
IMAGE_LINK_NAME IMAGE_ROOTFS INITRAMFS_FSTYPES INITRD INITRD_LIVE
ISODIR RECIPE_SYSROOT_NATIVE \
- ROOTFS_SIZE STAGING_DATADIR STAGING_DIR STAGING_LIBDIR TARGET_SYS \
+ ROOTFS_SIZE STAGING_DATADIR STAGING_DIR STAGING_LIBDIR TARGET_SYS
HOSTTOOLS_DIR \
KERNEL_IMAGETYPE MACHINE INITRAMFS_IMAGE INITRAMFS_IMAGE_BUNDLE
INITRAMFS_LINK_NAME APPEND \
ASSUME_PROVIDED PSEUDO_IGNORE_PATHS"
diff --git a/scripts/lib/wic/misc.py b/scripts/lib/wic/misc.py
index 4b08d649c6..75b219cd3f 100644
--- a/scripts/lib/wic/misc.py
+++ b/scripts/lib/wic/misc.py
@@ -138,9 +138,12 @@ def exec_native_cmd(cmd_and_args, native_sysroot,
pseudo=""):
if pseudo:
cmd_and_args = pseudo + cmd_and_args
-native_paths = "%s/sbin:%s/usr/sbin:%s/usr/bin:%s/bin" % \
+hosttools_dir = get_bitbake_var("HOSTTOOLS_DIR")
+
+native_paths = "%s/sbin:%s/usr/sbin:%s/usr/bin:%s/bin:%s" % \
(native_sysroot, native_sysroot,
-native_sysroot, native_sysroot)
+native_sysroot, native_sysroot,
+hosttools_dir)
native_cmd_and_args = "export PATH=%s:$PATH;%s" % \
(native_paths, cmd_and_args)
--
2.26.2
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#146578):
https://lists.openembedded.org/g/openembedded-core/message/146578
Mute This Topic: https://lists.openembedded.org/mt/79592784/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH 4/6] wic: Update pseudo db when excluding content from rootfs
To exclude content from the rootfs, wic makes a copy (using hardlinks if possible) of the rootfs directory and associated pseudo db, then removes files & directories as needed. However if these files and directories are removed using the python functions os.remove and shutil.rmtree, the copied pseudo db will not be updated correctly. For files copied from the original rootfs, if hardlinks were used successfully when copying the rootfs this should mean that the relevant inodes can't be reused and so the risk of pseudo aborts should be avoided. However, this logic doesn't apply for directories (as they can't be hardlinked) or for files added via the '--include-path' argument (as they weren't present in the original rootfs) and so there remains some risk of inodes being reused and the pseudo db becoming corrupted. To fix this, use the 'rm' command under pseudo when removing files & directories from the copied rootfs to ensure that the copied pseudo db is updated. Signed-off-by: Paul Barker --- scripts/lib/wic/plugins/source/rootfs.py | 13 - 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/scripts/lib/wic/plugins/source/rootfs.py b/scripts/lib/wic/plugins/source/rootfs.py index f1db83f8a1..afb1eb9202 100644 --- a/scripts/lib/wic/plugins/source/rootfs.py +++ b/scripts/lib/wic/plugins/source/rootfs.py @@ -199,17 +199,20 @@ class RootfsPlugin(SourcePlugin): if not os.path.lexists(full_path): continue +if new_pseudo: +pseudo = cls.__get_pseudo(native_sysroot, new_rootfs, new_pseudo) +else: +pseudo = None if path.endswith(os.sep): # Delete content only. for entry in os.listdir(full_path): full_entry = os.path.join(full_path, entry) -if os.path.isdir(full_entry) and not os.path.islink(full_entry): -shutil.rmtree(full_entry) -else: -os.remove(full_entry) +rm_cmd = "rm -rf %s" % (full_entry) +exec_native_cmd(rm_cmd, native_sysroot, pseudo) else: # Delete whole directory. -shutil.rmtree(full_path) +rm_cmd = "rm -rf %s" % (full_path) +exec_native_cmd(rm_cmd, native_sysroot, pseudo) part.prepare_rootfs(cr_workdir, oe_builddir, new_rootfs or part.rootfs_dir, native_sysroot, -- 2.26.2 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#146579): https://lists.openembedded.org/g/openembedded-core/message/146579 Mute This Topic: https://lists.openembedded.org/mt/79592785/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH 1/6] wic: Add workdir argument
This allows the path for the temporary workdir used by wic to be set
when running wic from bitbake or directly from the command line.
Signed-off-by: Paul Barker
---
scripts/lib/wic/plugins/imager/direct.py | 2 +-
scripts/wic | 2 ++
2 files changed, 3 insertions(+), 1 deletion(-)
diff --git a/scripts/lib/wic/plugins/imager/direct.py
b/scripts/lib/wic/plugins/imager/direct.py
index 0ca67b77d5..b329568c7a 100644
--- a/scripts/lib/wic/plugins/imager/direct.py
+++ b/scripts/lib/wic/plugins/imager/direct.py
@@ -62,7 +62,7 @@ class DirectPlugin(ImagerPlugin):
self.name = "%s-%s" % (os.path.splitext(os.path.basename(wks_file))[0],
strftime("%Y%m%d%H%M"))
-self.workdir = tempfile.mkdtemp(dir=self.outdir, prefix='tmp.wic.')
+self.workdir = options.workdir or tempfile.mkdtemp(dir=self.outdir,
prefix='tmp.wic.')
self._image = None
self.ptable_format = self.ks.bootloader.ptable
self.parts = self.ks.partitions
diff --git a/scripts/wic b/scripts/wic
index 24700f380f..a741aed364 100755
--- a/scripts/wic
+++ b/scripts/wic
@@ -312,6 +312,8 @@ def wic_init_parser_create(subparser):
subparser.add_argument("-o", "--outdir", dest="outdir", default='.',
help="name of directory to create image in")
+subparser.add_argument("-w", "--workdir",
+ help="temporary workdir to use for intermediate files")
subparser.add_argument("-e", "--image-name", dest="image_name",
help="name of the image to use the artifacts from "
"e.g. core-image-sato")
--
2.26.2
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#146576):
https://lists.openembedded.org/g/openembedded-core/message/146576
Mute This Topic: https://lists.openembedded.org/mt/79592782/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH 2/6] image_types_wic: Move wic working directory
By default the wic working directory is placed under the output
directory. When invoking wic under bitbake, the wic output directory is
added to PSEUDO_PATHS_IGNORE to avoid issues with files being removed
from outside a pseudo environment (see oe-core commit ad8f5532ff).
However, wic will copy the rootfs directory into its working directory
if it needs to add or remove content before creating a filesystem image.
This copy of the rootfs directory must be tracked by pseudo in order to
keep the permissions correct in the resulting image. So we can't have
the wic working directory under a path in PSEUDO_PATHS_IGNORE unless
we like broken permissions.
To fix this the new '-w' argument to wic is used to move the working
directory away from the output directory.
Note that wic deletes the temporary working directory automatically
when it finishes creating an image so users won't normally see the
'tmp-wic' directory under WORKDIR.
Fixes [Yocto #14129]
Signed-off-by: Paul Barker
---
meta/classes/image_types_wic.bbclass | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/meta/classes/image_types_wic.bbclass
b/meta/classes/image_types_wic.bbclass
index 286e0f5d54..a5ed780ceb 100644
--- a/meta/classes/image_types_wic.bbclass
+++ b/meta/classes/image_types_wic.bbclass
@@ -29,15 +29,16 @@ WIC_CREATE_EXTRA_ARGS ?= ""
IMAGE_CMD_wic () {
out="${IMGDEPLOYDIR}/${IMAGE_NAME}"
build_wic="${WORKDIR}/build-wic"
+ tmp_wic="${WORKDIR}/tmp-wic"
wks="${WKS_FULL_PATH}"
if [ -z "$wks" ]; then
bbfatal "No kickstart files from WKS_FILES were found:
${WKS_FILES}. Please set WKS_FILE or WKS_FILES appropriately."
fi
- BUILDDIR="${TOPDIR}" PSEUDO_UNLOAD=1 wic create "$wks" --vars
"${STAGING_DIR}/${MACHINE}/imgdata/" -e "${IMAGE_BASENAME}" -o "$build_wic/"
${WIC_CREATE_EXTRA_ARGS}
+ BUILDDIR="${TOPDIR}" PSEUDO_UNLOAD=1 wic create "$wks" --vars
"${STAGING_DIR}/${MACHINE}/imgdata/" -e "${IMAGE_BASENAME}" -o "$build_wic/" -w
"$tmp_wic" ${WIC_CREATE_EXTRA_ARGS}
mv "$build_wic/$(basename "${wks%.wks}")"*.direct
"$out${IMAGE_NAME_SUFFIX}.wic"
}
IMAGE_CMD_wic[vardepsexclude] = "WKS_FULL_PATH WKS_FILES TOPDIR"
-do_image_wic[cleandirs] = "${WORKDIR}/build-wic"
+do_image_wic[cleandirs] = "${WORKDIR}/build-wic ${WORKDIR}/tmp-wic"
PSEUDO_IGNORE_PATHS .= ",${WORKDIR}/build-wic"
--
2.26.2
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#146577):
https://lists.openembedded.org/g/openembedded-core/message/146577
Mute This Topic: https://lists.openembedded.org/mt/79592783/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH] selftest: Add argument to keep build dir
The oe-selftest code already keeps the selftest build directory in place
if any tests failed. By default the build directory is deleted if all
tests pass but there may be cases where it's desirable to keep this
directory around, for example to compare intermediate files between
passing and failing test runs.
Signed-off-by: Paul Barker
---
meta/lib/oeqa/selftest/context.py | 17 +
1 file changed, 13 insertions(+), 4 deletions(-)
diff --git a/meta/lib/oeqa/selftest/context.py
b/meta/lib/oeqa/selftest/context.py
index dd3609c1d6..1659926975 100644
--- a/meta/lib/oeqa/selftest/context.py
+++ b/meta/lib/oeqa/selftest/context.py
@@ -34,7 +34,7 @@ class NonConcurrentTestSuite(unittest.TestSuite):
(builddir, newbuilddir) = self.setupfunc("-st", None, self.suite)
ret = super().run(result)
os.chdir(builddir)
-if newbuilddir and ret.wasSuccessful():
+if newbuilddir and ret.wasSuccessful() and self.removefunc:
self.removefunc(newbuilddir)
def removebuilddir(d):
@@ -54,7 +54,7 @@ def removebuilddir(d):
bb.utils.prunedir(d, ionice=True)
class OESelftestTestContext(OETestContext):
-def __init__(self, td=None, logger=None, machines=None, config_paths=None,
newbuilddir=None):
+def __init__(self, td=None, logger=None, machines=None, config_paths=None,
newbuilddir=None, keep_builddir=None):
super(OESelftestTestContext, self).__init__(td, logger)
self.machines = machines
@@ -62,6 +62,11 @@ class OESelftestTestContext(OETestContext):
self.config_paths = config_paths
self.newbuilddir = newbuilddir
+if keep_builddir:
+self.removebuilddir = None
+else:
+self.removebuilddir = removebuilddir
+
def setup_builddir(self, suffix, selftestdir, suite):
builddir = os.environ['BUILDDIR']
if not selftestdir:
@@ -119,9 +124,9 @@ class OESelftestTestContext(OETestContext):
if processes:
from oeqa.core.utils.concurrencytest import ConcurrentTestSuite
-return ConcurrentTestSuite(suites, processes, self.setup_builddir,
removebuilddir)
+return ConcurrentTestSuite(suites, processes, self.setup_builddir,
self.removebuilddir)
else:
-return NonConcurrentTestSuite(suites, processes,
self.setup_builddir, removebuilddir)
+return NonConcurrentTestSuite(suites, processes,
self.setup_builddir, self.removebuilddir)
def runTests(self, processes=None, machine=None, skips=[]):
if machine:
@@ -179,6 +184,9 @@ class OESelftestTestContextExecutor(OETestContextExecutor):
action='append', default=None,
help='Exclude all (unhidden) tests that match any of the
specified tag(s). (exclude applies before select)')
+parser.add_argument('-K', '--keep-builddir', action='store_true',
+help='Keep the test build directory even if all tests pass')
+
parser.add_argument('-B', '--newbuilddir', help='New build directory
to use for tests.')
parser.add_argument('-v', '--verbose', action='store_true')
parser.set_defaults(func=self.run)
@@ -236,6 +244,7 @@ class OESelftestTestContextExecutor(OETestContextExecutor):
self.tc_kwargs['init']['config_paths']['localconf'] =
os.path.join(builddir, "conf/local.conf")
self.tc_kwargs['init']['config_paths']['bblayers'] =
os.path.join(builddir, "conf/bblayers.conf")
self.tc_kwargs['init']['newbuilddir'] = args.newbuilddir
+self.tc_kwargs['init']['keep_builddir'] = args.keep_builddir
def tag_filter(tags):
if args.exclude_tags:
--
2.26.2
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#146473):
https://lists.openembedded.org/g/openembedded-core/message/146473
Mute This Topic: https://lists.openembedded.org/mt/79500166/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-Core][PATCH v2] oe-pkgdata-util: Added a test to verify oe-pkgdata-util without parameters
On Thu, 24 Dec 2020 at 10:01, Milan Shah wrote:
>
> A test is implemented on poky/meta/lib/oeqa/selftest/pkgdata.py to test
> the scenario when oe-pkgdata-util is executed without parameters and
> help is displayed.
>
> See [YOCTO #10726] for detailed bug information.
>
> Signed-off-by: Milan Shah
> ---
> meta/lib/oeqa/selftest/cases/pkgdata.py | 6 ++
> 1 file changed, 6 insertions(+)
>
> diff --git a/meta/lib/oeqa/selftest/cases/pkgdata.py
> b/meta/lib/oeqa/selftest/cases/pkgdata.py
> index 833a180..94ee1f5 100644
> --- a/meta/lib/oeqa/selftest/cases/pkgdata.py
> +++ b/meta/lib/oeqa/selftest/cases/pkgdata.py
> @@ -218,3 +218,9 @@ class OePkgdataUtilTests(OESelftestTestCase):
> def test_specify_pkgdatadir(self):
> result = runCmd('oe-pkgdata-util -p %s lookup-pkg zlib' %
> get_bb_var('PKGDATA_DIR'))
> self.assertEqual(result.output, 'libz1')
> +
> +def test_no_param(self):
> +result = runCmd('oe-pkgdata-util', ignore_status=True)
> +self.assertEqual(result.status, 2, "Status different than 2. output:
> %s" % result.output)
> +currpos = result.output.find('usage: oe-pkgdata-util [-h] [-d] [-p
> PKGDATA_DIR] ')
In my opinion this is still too restrictive a test, it would need
updating when any further arguments are added to the oe-pkgdata-util
tool. I'd recommend just checking for "usage: oe-pkgdata-util" and
ignoring the rest of the line.
Thanks,
--
Paul Barker
Konsulko Group
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#146337):
https://lists.openembedded.org/g/openembedded-core/message/146337
Mute This Topic: https://lists.openembedded.org/mt/79199395/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] meta-oe Warrior postgresql build error
poky-linux/postgresql/11.5-r0/image/usr/share/postgresql/extension/' > make[1]: Leaving directory > '/home/dingo/STX/build/tmp/work/corei7-64-poky-linux/postgresql/11.5-r0/build/contrib/ltree_plpython' > make[1]: Leaving directory > '/home/dingo/STX/build/tmp/work/corei7-64-poky-linux/postgresql/11.5-r0/build/contrib/jsonb_plpython' > make: Leaving directory > '/home/dingo/STX/build/tmp/work/corei7-64-poky-linux/postgresql/11.5-r0/build/contrib' > sed: can't read > /home/dingo/STX/build/tmp/work/corei7-64-poky-linux/postgresql/11.5-r0/image/var/lib/postgresql/.bash_profile: > No such file or directory > WARNING: exit code 2 from a shell command. > ERROR: Function failed: do_install (log file is located at > /home/dingo/STX/build/tmp/work/corei7-64-poky-linux/postgresql/11.5-r0/temp/log.do_install.25533) > dingo@ubuntu:~/STX/build$ Looks like a large stack of layers there. Is this failure reproducible with just poky and meta-openembedded? Have you checked for any bbappends to the postgresql recipe? The warrior branch is now EOL so you should upgrade to dunfell or gatesgarth if you can. Thanks, -- Paul Barker Konsulko Group -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#146082): https://lists.openembedded.org/g/openembedded-core/message/146082 Mute This Topic: https://lists.openembedded.org/mt/79177432/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core][dunfell 00/28] Patch review - pseudo file mode corruption fix series
On Tue, 22 Dec 2020 at 14:57, Erik Botö wrote:
>
> On Tue, Dec 22, 2020 at 2:50 PM Paul Barker wrote:
> >
> > On Tue, 22 Dec 2020 at 10:03, Richard Purdie
> > wrote:
> > >
> > > On Tue, 2020-12-22 at 10:08 +0100, Erik Boto wrote:
> > > > There's a bug affecting wic in combination with --exclude-path /
> > > > --include-path that would then be backported into the LTS version.
> > > > See
> > > > https://bugzilla.yoctoproject.org/show_bug.cgi?id=14129.
> > > >
> > > > Maybe the issue that is being fixed is more serious then the issue
> > > > being introduced so it's a valid trade-off, I just wanted to mention
> > > > it so it's known.
> > >
> > > I've been suggesting Steve hold off this until we sorted out master,
> > > having a patch queue ready is helpful though so we don't lose track of
> > > all the patches.
> > >
> > > There were some fixes from Peter/Paul around pseudo which we needed and
> > > have now merged which leaves the above issue...
> > >
> > > Cheers,
> > >
> > > Richard
> >
> > Looking at that bug (#14129) I wonder why "${WORKDIR}/build-wic" was
> > added to PSEUDO_IGNORE_PATHS in
> > https://git.openembedded.org/openembedded-core/commit/?id=ad8f5532ffaead9a5ad13e1034fe9e5e1b7979f4.
> > Is anything likely to explode if we drop that addition?
>
> Yes, if you try to build an image a second time it breaks and you have
> to "bitbake -c clean " in order to build it successfully again.
> I don't recall the exact error at this time, but it was related to the
> partition files wic created under build-wic.
Ok. Sounds like we need to move the directory wic uses when copying
the rootfs out of "${WORKDIR}/build-wic" so that it can be tracked by
pseudo. The partition files should stay in the build-wic directory.
--
Paul Barker
Konsulko Group
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#146065):
https://lists.openembedded.org/g/openembedded-core/message/146065
Mute This Topic: https://lists.openembedded.org/mt/78686037/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core][dunfell 00/28] Patch review - pseudo file mode corruption fix series
On Tue, 22 Dec 2020 at 10:03, Richard Purdie
wrote:
>
> On Tue, 2020-12-22 at 10:08 +0100, Erik Boto wrote:
> > There's a bug affecting wic in combination with --exclude-path /
> > --include-path that would then be backported into the LTS version.
> > See
> > https://bugzilla.yoctoproject.org/show_bug.cgi?id=14129.
> >
> > Maybe the issue that is being fixed is more serious then the issue
> > being introduced so it's a valid trade-off, I just wanted to mention
> > it so it's known.
>
> I've been suggesting Steve hold off this until we sorted out master,
> having a patch queue ready is helpful though so we don't lose track of
> all the patches.
>
> There were some fixes from Peter/Paul around pseudo which we needed and
> have now merged which leaves the above issue...
>
> Cheers,
>
> Richard
Looking at that bug (#14129) I wonder why "${WORKDIR}/build-wic" was
added to PSEUDO_IGNORE_PATHS in
https://git.openembedded.org/openembedded-core/commit/?id=ad8f5532ffaead9a5ad13e1034fe9e5e1b7979f4.
Is anything likely to explode if we drop that addition?
Thanks,
--
Paul Barker
Konsulko Group
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#146052):
https://lists.openembedded.org/g/openembedded-core/message/146052
Mute This Topic: https://lists.openembedded.org/mt/78686037/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH] kernel-devicetree: Introduce KERNEL_DEVICETREE_FLAGS to pass dtc flags
On Tue, 22 Dec 2020 at 10:42, Ovidiu Panait wrote:
>
> Currently DTC_FLAGS kernel makefile parameter can be specified directly on the
> command line by adding it to KERNEL_EXTRA_ARGS. However, this prevents
> scripts/Makefile.lib logic from appending flags that silence dtc warnings (all
> assignments done from within a makefile, to a variable specified on the
> command
> line, are ignored).
>
> Because of this, the do_compile log is cluttered with dtc warnings that should
> only be printed when compiling with W="123":
> ...
> /soc: node has a reg or ranges property, but no unit name
> /soc/gpu: missing or empty reg/ranges property
> /soc/firmware/gpio: missing or empty reg/ranges property
> ...
>
> To fix this, introduce the dedicated KERNEL_DEVICETREE_FLAGS variable to hold
> dtc flags and export DTC_FLAGS in the environment before generating the dtbs
> (make allows "+=" operations on variables that come from the environment, so
> the warnings are silenced properly).
I'd prefer KERNEL_DTC_FLAGS but maybe that's just arguing over the
colour of the bikeshed.
>
> Signed-off-by: Ovidiu Panait
> ---
> meta/classes/kernel-devicetree.bbclass | 7 +++
> 1 file changed, 7 insertions(+)
>
> diff --git a/meta/classes/kernel-devicetree.bbclass
> b/meta/classes/kernel-devicetree.bbclass
> index 81dda8003f..2066096a9a 100644
> --- a/meta/classes/kernel-devicetree.bbclass
> +++ b/meta/classes/kernel-devicetree.bbclass
> @@ -9,6 +9,9 @@ FILES_${KERNEL_PACKAGE_NAME}-image-zimage-bundle =
> "/${KERNEL_IMAGEDEST}/zImage-
> # Generate kernel+devicetree bundle
> KERNEL_DEVICETREE_BUNDLE ?= "0"
>
> +# dtc flags passed via DTC_FLAGS env variable
> +KERNEL_DEVICETREE_FLAGS ?= ""
> +
> normalize_dtb () {
> dtb="$1"
> if echo $dtb | grep -q '/dts/'; then
> @@ -50,6 +53,10 @@ do_configure_append() {
> }
>
> do_compile_append() {
> + if [ ! -z "${KERNEL_DEVICETREE_FLAGS}" ]; then
> + export DTC_FLAGS="${KERNEL_DEVICETREE_FLAGS}"
> + fi
> +
> for dtbf in ${KERNEL_DEVICETREE}; do
> dtb=`normalize_dtb "$dtbf"`
> oe_runmake $dtb CC="${KERNEL_CC} $cc_extra "
> LD="${KERNEL_LD}" ${KERNEL_EXTRA_ARGS}
> --
> 2.17.1
>
>
>
>
--
Paul Barker
Konsulko Group
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#146050):
https://lists.openembedded.org/g/openembedded-core/message/146050
Mute This Topic: https://lists.openembedded.org/mt/79146426/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [meta-oe][PATCH] oe-pkgdata-util: Added a test to verify oe-pkgdata-util without parameters
On Tue, 22 Dec 2020 at 11:25, Milan Shah wrote:
>
> A test is implemented on poky/meta/lib/oeqa/selftest/pkgdata.py to test
> the scenario when oe-pkgdata-util is executed without parameters and
> help is displayed.
>
> See [YOCTO #10726] for detailed bug information.
>
> Upstream-Status: Submitted
This tag doesn't make sense here, you can just drop it. We only need
this tag for actual patch files added to the repository to be applied
to upstream software.
>
> Signed-off-by: Milan Shah
> ---
> meta/lib/oeqa/selftest/cases/pkgdata.py | 6 ++
> 1 file changed, 6 insertions(+)
>
> diff --git a/meta/lib/oeqa/selftest/cases/pkgdata.py
> b/meta/lib/oeqa/selftest/cases/pkgdata.py
> index 833a180..d3a5519 100644
> --- a/meta/lib/oeqa/selftest/cases/pkgdata.py
> +++ b/meta/lib/oeqa/selftest/cases/pkgdata.py
> @@ -218,3 +218,9 @@ class OePkgdataUtilTests(OESelftestTestCase):
> def test_specify_pkgdatadir(self):
> result = runCmd('oe-pkgdata-util -p %s lookup-pkg zlib' %
> get_bb_var('PKGDATA_DIR'))
> self.assertEqual(result.output, 'libz1')
> +
> +def test_no_param(self):
> +result = runCmd('oe-pkgdata-util', ignore_status=True)
> +self.assertEqual(result.status, 2, "Status different than 2. output:
> %s" % result.output)
> +self.assertIn('oe-pkgdata-util: error: the following arguments are
> required: ', result.output)
> +self.assertIn('usage: oe-pkgdata-util [-h] [-d] [-p PKGDATA_DIR]
> ', result.output)
I don't like these assertln calls - they just make it more difficult
to change the help output of the oe-pkgdata-util tool if additional
options are added in the future.
I'm not really sure what this test case is expected to catch but
perhaps just checking for the return code of 2 and the string "usage:
oe-pkgdata-util" somewhere in the output is enough.
Thanks,
--
Paul Barker
Konsulko Group
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#146049):
https://lists.openembedded.org/g/openembedded-core/message/146049
Mute This Topic: https://lists.openembedded.org/mt/79146837/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH] bitbake.conf: Prevent pyc file generation in pseudo context
This also effectively reverts commit b6d30c21b0:
bitbake.conf: Extend PSEUDO_IGNORE_PATHS to ${COREBASE}/meta
The contents of ${COREBASE}/meta were ignored as pyc files could be
generated for the contents of the lib subdirectory if python modules
were imported within a pseudo context. However this doesn't protect us
from pyc files being generated in the lib directories for other layers.
It's far better to tell python not to produce pyc files when running
under pseudo (by setting the PYTHONDONTWRITEBYTECODE variable) as this
will cover any location where pyc files could possibly be created. This
variable is set in FAKEROOTBASEENV so that it applies to the
bitbake-worker instance for fakeroot tasks, preventing pyc files from
being generated for imports in python tasks themselves.
Also add a test case to ensure that pyc files are not created in tasks
which are executed under pseudo.
Signed-off-by: Paul Barker
---
meta-selftest/lib/pseudo_pyc_test1.py | 1 +
meta-selftest/lib/pseudo_pyc_test2.py | 1 +
.../pseudo-pyc-test/pseudo-pyc-test.bb| 15 +++
meta/conf/bitbake.conf| 4 +--
meta/lib/oeqa/selftest/cases/pseudo.py| 27 +++
5 files changed, 46 insertions(+), 2 deletions(-)
create mode 100644 meta-selftest/lib/pseudo_pyc_test1.py
create mode 100644 meta-selftest/lib/pseudo_pyc_test2.py
create mode 100644
meta-selftest/recipes-test/pseudo-pyc-test/pseudo-pyc-test.bb
create mode 100644 meta/lib/oeqa/selftest/cases/pseudo.py
diff --git a/meta-selftest/lib/pseudo_pyc_test1.py
b/meta-selftest/lib/pseudo_pyc_test1.py
new file mode 100644
index 00..b59abdd536
--- /dev/null
+++ b/meta-selftest/lib/pseudo_pyc_test1.py
@@ -0,0 +1 @@
+STRING = "pseudo_pyc_test1"
diff --git a/meta-selftest/lib/pseudo_pyc_test2.py
b/meta-selftest/lib/pseudo_pyc_test2.py
new file mode 100644
index 00..fb67a978e0
--- /dev/null
+++ b/meta-selftest/lib/pseudo_pyc_test2.py
@@ -0,0 +1 @@
+STRING = "pseudo_pyc_test2"
diff --git a/meta-selftest/recipes-test/pseudo-pyc-test/pseudo-pyc-test.bb
b/meta-selftest/recipes-test/pseudo-pyc-test/pseudo-pyc-test.bb
new file mode 100644
index 00..12dc91a8f3
--- /dev/null
+++ b/meta-selftest/recipes-test/pseudo-pyc-test/pseudo-pyc-test.bb
@@ -0,0 +1,15 @@
+SUMMARY = "pseudo env test"
+LICENSE = "MIT"
+LIC_FILES_CHKSUM =
"file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"
+
+INHIBIT_DEFAULT_DEPS = "1"
+
+python do_compile() {
+import pseudo_pyc_test1
+print(pseudo_pyc_test1.STRING)
+}
+
+python do_install() {
+import pseudo_pyc_test2
+print(pseudo_pyc_test2.STRING)
+}
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 0d38eac094..4ce224a90c 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -685,13 +685,13 @@ SRC_URI = ""
PSEUDO_LOCALSTATEDIR ?= "${WORKDIR}/pseudo/"
PSEUDO_PASSWD ?= "${STAGING_DIR_TARGET}:${PSEUDO_SYSROOT}"
PSEUDO_SYSROOT = "${COMPONENTS_DIR}/${BUILD_ARCH}/pseudo-native"
-PSEUDO_IGNORE_PATHS =
"/usr/,/etc/,/lib,/dev/,${T},${WORKDIR}/recipe-sysroot,${SSTATE_DIR},${STAMPS_DIR},${WORKDIR}/pkgdata-sysroot,${TMPDIR}/sstate-control,${DEPLOY_DIR},${WORKDIR}/deploy-,${TMPDIR}/buildstats,${WORKDIR}/sstate-build-package_,${WORKDIR}/sstate-install-package_,${WORKDIR}/sstate-build-image_complete,${TMPDIR}/sysroots-components,${BUILDHISTORY_DIR},${TMPDIR}/pkgdata,${TOPDIR}/cache,${COREBASE}/scripts,${COREBASE}/meta,${CCACHE_DIR}"
+PSEUDO_IGNORE_PATHS =
"/usr/,/etc/,/lib,/dev/,${T},${WORKDIR}/recipe-sysroot,${SSTATE_DIR},${STAMPS_DIR},${WORKDIR}/pkgdata-sysroot,${TMPDIR}/sstate-control,${DEPLOY_DIR},${WORKDIR}/deploy-,${TMPDIR}/buildstats,${WORKDIR}/sstate-build-package_,${WORKDIR}/sstate-install-package_,${WORKDIR}/sstate-build-image_complete,${TMPDIR}/sysroots-components,${BUILDHISTORY_DIR},${TMPDIR}/pkgdata,${TOPDIR}/cache,${COREBASE}/scripts,${CCACHE_DIR}"
export PSEUDO_DISABLED = "1"
#export PSEUDO_PREFIX = "${STAGING_DIR_NATIVE}${prefix_native}"
#export PSEUDO_BINDIR = "${STAGING_DIR_NATIVE}${bindir_native}"
#export PSEUDO_LIBDIR = "${STAGING_DIR_NATIVE}$PSEUDOBINDIR/../lib/pseudo/lib
-FAKEROOTBASEENV = "PSEUDO_BINDIR=${PSEUDO_SYSROOT}${bindir_native}
PSEUDO_LIBDIR=${PSEUDO_SYSROOT}${prefix_native}/lib/pseudo/lib
PSEUDO_PREFIX=${PSEUDO_SYSROOT}${prefix_native}
PSEUDO_IGNORE_PATHS=${PSEUDO_IGNORE_PATHS} PSEUDO_DISABLED=1"
+FAKEROOTBASEENV = "PSEUDO_BINDIR=${PSEUDO_SYSROOT}${bindir_native}
PSEUDO_LIBDIR=${PSEUDO_SYSROOT}${prefix_native}/lib/pseudo/lib
PSEUDO_PREFIX=${PSEUDO_SYSROOT}${prefix_native}
PSEUDO_IGNORE_PATHS=${PSEUDO_IGNORE_PATHS} PSEUDO_DISABLED=1
PYTHONDONTWRITEBYTECODE=1"
FAKEROOTCMD = "${PSEUDO_SYSROOT}${bindir_native}/pseudo"
FAKEROOTENV = "PSEUDO_PREFIX=${PSEUDO_SY
Re: [OE-core] [PATCH] Revert "bitbake.conf: Add all layers (from BBLAYERS) to PSEUDO_IGNORE_PATHS"
On Wed, 9 Dec 2020 at 12:26, Richard Purdie
wrote:
>
> This reverts commit cc30537bb4d0815c4d69f0b1a1a8a200085902d5.
>
> There are issues with this change where the build directory may be included
> within the paths included in BBLAYERS, which is particularly problematic
> where people are using kas.
>
> An alternative to this to handle the pyc file issue is being worked on but
> revert this for now whilst that fix is developed.
> ---
> meta/conf/bitbake.conf | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
> index a353dc5d6b2..812cd051021 100644
> --- a/meta/conf/bitbake.conf
> +++ b/meta/conf/bitbake.conf
> @@ -685,7 +685,7 @@ SRC_URI = ""
> PSEUDO_LOCALSTATEDIR ?= "${WORKDIR}/pseudo/"
> PSEUDO_PASSWD ?= "${STAGING_DIR_TARGET}:${PSEUDO_SYSROOT}"
> PSEUDO_SYSROOT = "${COMPONENTS_DIR}/${BUILD_ARCH}/pseudo-native"
> -PSEUDO_IGNORE_PATHS =
> "/usr/,/etc/,/lib,/dev/,${T},${WORKDIR}/recipe-sysroot,${SSTATE_DIR},${STAMPS_DIR},${WORKDIR}/pkgdata-sysroot,${TMPDIR}/sstate-control,${DEPLOY_DIR},${WORKDIR}/deploy-,${TMPDIR}/buildstats,${WORKDIR}/sstate-build-package_,${WORKDIR}/sstate-install-package_,${WORKDIR}/sstate-build-image_complete,${TMPDIR}/sysroots-components,${BUILDHISTORY_DIR},${TMPDIR}/pkgdata,${TOPDIR}/cache,${COREBASE}/scripts,${@','.join(d.getVar('BBLAYERS').split())},${CCACHE_DIR}"
> +PSEUDO_IGNORE_PATHS =
> "/usr/,/etc/,/lib,/dev/,${T},${WORKDIR}/recipe-sysroot,${SSTATE_DIR},${STAMPS_DIR},${WORKDIR}/pkgdata-sysroot,${TMPDIR}/sstate-control,${DEPLOY_DIR},${WORKDIR}/deploy-,${TMPDIR}/buildstats,${WORKDIR}/sstate-build-package_,${WORKDIR}/sstate-install-package_,${WORKDIR}/sstate-build-image_complete,${TMPDIR}/sysroots-components,${BUILDHISTORY_DIR},${TMPDIR}/pkgdata,${TOPDIR}/cache,${COREBASE}/scripts,${COREBASE}/meta,${CCACHE_DIR}"
>
> export PSEUDO_DISABLED = "1"
> #export PSEUDO_PREFIX = "${STAGING_DIR_NATIVE}${prefix_native}"
> --
> 2.25.1
>
Thanks Richard, I had few other builds to run so haven't had a chance
to test this locally yet.
--
Paul Barker
Konsulko Group
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#145422):
https://lists.openembedded.org/g/openembedded-core/message/145422
Mute This Topic: https://lists.openembedded.org/mt/78827518/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCHv3 2/5] bitbake.conf: Add all layers (from BBLAYERS) to PSEUDO_IGNORE_PATHS
On Mon, 7 Dec 2020 at 12:46, Richard Purdie
wrote:
>
> On Mon, 2020-12-07 at 12:04 +, Paul Barker wrote:
> > On Mon, 7 Dec 2020 at 10:29, Paul Barker
> > wrote:
> > To follow up some more: The entries in PSEUDO_IGNORE_PATHS are
> > treated
> > as string prefixes within pseudo. So if
> > "/home/pbarker/Projects/Yocto/meta-linux-mainline" is added to the
> > ignore list it will exclude not just
> > "/home/pbarker/Projects/Yocto/meta-linux-mainline/build" but also
> > ""/home/pbarker/Projects/Yocto/meta-linux-mainline-build".
> >
> > I wonder if some more of those entries should have trailing slashes.
>
> In most (all?) cases it was very deliberate FWIW...
That does make sense. Ignoring "${COREBASE}/meta" will also ignore
most layers unpacked or cloned within the poky directory as their
names start with "meta". However that does miss layers if you use a
different directory structure which is what Peter's patch addresses
(though I'm still not sure if there's an actual build failure with
some layers which it is intended to fix or if it's just to make things
consistent).
The issue comes when you clone a layer as the top-level of your
working tree and build within that. That's how I work with
meta-sancloud & meta-linux-mainline. It's also what happens if you
build using the kas config in meta-raspberrypi. So it's not uncommon.
Investigating why the layer directories are being ignored I found this
commit added the ignore of "${COREBASE}/meta":
commit e0cb6dd689a362d8433caa14cc5a9fdd5eb44923
Author: Richard Purdie
Date: Wed Oct 7 23:08:45 2020 +0100
bitbake.conf: Extend PSEUDO_IGNORE_PATHS to ${COREBASE}/meta
Unfortunately, .pyc files can be generated in meta/lib/oe
which corrupt the pseudo
database so we need to extend the ignore list to cover this as well.
Signed-off-by: Richard Purdie
Could we instead ignore "${LAYERDIR}/lib" for each layer?
An alternative would be to detect the case where TOPDIR or TMPDIR is
beneath an entry in BBLAYERS and handle that as a special case.
Thanks,
--
Paul Barker
Konsulko Group
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#145376):
https://lists.openembedded.org/g/openembedded-core/message/145376
Mute This Topic: https://lists.openembedded.org/mt/78640454/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCHv3 2/5] bitbake.conf: Add all layers (from BBLAYERS) to PSEUDO_IGNORE_PATHS
On Mon, 7 Dec 2020 at 10:29, Paul Barker wrote:
>
> On Tue, 1 Dec 2020 at 18:12, Peter Kjellerstedt
> wrote:
> >
> > Instead of ignoring ${COREBASE}/meta in PSEUDO_IGNORE_PATHS (which may
> > or may not ignore all layers depending on how they are named and placed
> > under ${COREBASE}), ignore all layers.
> >
> > Signed-off-by: Peter Kjellerstedt
> > ---
> > meta/conf/bitbake.conf | 2 +-
> > 1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
> > index 0d38eac094..9742fe4fe2 100644
> > --- a/meta/conf/bitbake.conf
> > +++ b/meta/conf/bitbake.conf
> > @@ -685,7 +685,7 @@ SRC_URI = ""
> > PSEUDO_LOCALSTATEDIR ?= "${WORKDIR}/pseudo/"
> > PSEUDO_PASSWD ?= "${STAGING_DIR_TARGET}:${PSEUDO_SYSROOT}"
> > PSEUDO_SYSROOT = "${COMPONENTS_DIR}/${BUILD_ARCH}/pseudo-native"
> > -PSEUDO_IGNORE_PATHS =
> > "/usr/,/etc/,/lib,/dev/,${T},${WORKDIR}/recipe-sysroot,${SSTATE_DIR},${STAMPS_DIR},${WORKDIR}/pkgdata-sysroot,${TMPDIR}/sstate-control,${DEPLOY_DIR},${WORKDIR}/deploy-,${TMPDIR}/buildstats,${WORKDIR}/sstate-build-package_,${WORKDIR}/sstate-install-package_,${WORKDIR}/sstate-build-image_complete,${TMPDIR}/sysroots-components,${BUILDHISTORY_DIR},${TMPDIR}/pkgdata,${TOPDIR}/cache,${COREBASE}/scripts,${COREBASE}/meta,${CCACHE_DIR}"
> > +PSEUDO_IGNORE_PATHS =
> > "/usr/,/etc/,/lib,/dev/,${T},${WORKDIR}/recipe-sysroot,${SSTATE_DIR},${STAMPS_DIR},${WORKDIR}/pkgdata-sysroot,${TMPDIR}/sstate-control,${DEPLOY_DIR},${WORKDIR}/deploy-,${TMPDIR}/buildstats,${WORKDIR}/sstate-build-package_,${WORKDIR}/sstate-install-package_,${WORKDIR}/sstate-build-image_complete,${TMPDIR}/sysroots-components,${BUILDHISTORY_DIR},${TMPDIR}/pkgdata,${TOPDIR}/cache,${COREBASE}/scripts,${@','.join(d.getVar('BBLAYERS').split())},${CCACHE_DIR}"
>
> This has now landed in master and unfortunately it breaks the way I
> run builds. I often end up building inside a layer directory. For
> example the one that failed this morning has:
>
> BBLAYERS ?= " \
> /home/pbarker/Projects/Yocto/meta-linux-mainline \
> /home/pbarker/Projects/Yocto/meta-linux-mainline/layers/poky/meta \
>
> /home/pbarker/Projects/Yocto/meta-linux-mainline/layers/poky/meta-poky"
>
> TOPDIR is "/home/pbarker/Projects/Yocto/meta-linux-mainline/build" and
> TMPDIR is "${TOPDIR}/tmp". So everything under TMPDIR is now ignored
> by pseudo and various do_install/do_package tasks fail.
>
> I don't think this will just affect me as this is a common pattern,
> especially when using something like kas to setup builds.
>
> Before we revert this I'd like to understand the problem this patch is
> trying to fix. Is there a specific reason why you need to ignore all
> layer directories? I'm not 100% sure why "${COREBASE}/meta" is ignored
> anyway though I guess it is to reduce pseudo db size and so speed up
> builds.
To follow up some more: The entries in PSEUDO_IGNORE_PATHS are treated
as string prefixes within pseudo. So if
"/home/pbarker/Projects/Yocto/meta-linux-mainline" is added to the
ignore list it will exclude not just
"/home/pbarker/Projects/Yocto/meta-linux-mainline/build" but also
""/home/pbarker/Projects/Yocto/meta-linux-mainline-build".
I wonder if some more of those entries should have trailing slashes.
--
Paul Barker
Konsulko Group
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#145342):
https://lists.openembedded.org/g/openembedded-core/message/145342
Mute This Topic: https://lists.openembedded.org/mt/78640454/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCHv3 2/5] bitbake.conf: Add all layers (from BBLAYERS) to PSEUDO_IGNORE_PATHS
On Tue, 1 Dec 2020 at 18:12, Peter Kjellerstedt
wrote:
>
> Instead of ignoring ${COREBASE}/meta in PSEUDO_IGNORE_PATHS (which may
> or may not ignore all layers depending on how they are named and placed
> under ${COREBASE}), ignore all layers.
>
> Signed-off-by: Peter Kjellerstedt
> ---
> meta/conf/bitbake.conf | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
> index 0d38eac094..9742fe4fe2 100644
> --- a/meta/conf/bitbake.conf
> +++ b/meta/conf/bitbake.conf
> @@ -685,7 +685,7 @@ SRC_URI = ""
> PSEUDO_LOCALSTATEDIR ?= "${WORKDIR}/pseudo/"
> PSEUDO_PASSWD ?= "${STAGING_DIR_TARGET}:${PSEUDO_SYSROOT}"
> PSEUDO_SYSROOT = "${COMPONENTS_DIR}/${BUILD_ARCH}/pseudo-native"
> -PSEUDO_IGNORE_PATHS =
> "/usr/,/etc/,/lib,/dev/,${T},${WORKDIR}/recipe-sysroot,${SSTATE_DIR},${STAMPS_DIR},${WORKDIR}/pkgdata-sysroot,${TMPDIR}/sstate-control,${DEPLOY_DIR},${WORKDIR}/deploy-,${TMPDIR}/buildstats,${WORKDIR}/sstate-build-package_,${WORKDIR}/sstate-install-package_,${WORKDIR}/sstate-build-image_complete,${TMPDIR}/sysroots-components,${BUILDHISTORY_DIR},${TMPDIR}/pkgdata,${TOPDIR}/cache,${COREBASE}/scripts,${COREBASE}/meta,${CCACHE_DIR}"
> +PSEUDO_IGNORE_PATHS =
> "/usr/,/etc/,/lib,/dev/,${T},${WORKDIR}/recipe-sysroot,${SSTATE_DIR},${STAMPS_DIR},${WORKDIR}/pkgdata-sysroot,${TMPDIR}/sstate-control,${DEPLOY_DIR},${WORKDIR}/deploy-,${TMPDIR}/buildstats,${WORKDIR}/sstate-build-package_,${WORKDIR}/sstate-install-package_,${WORKDIR}/sstate-build-image_complete,${TMPDIR}/sysroots-components,${BUILDHISTORY_DIR},${TMPDIR}/pkgdata,${TOPDIR}/cache,${COREBASE}/scripts,${@','.join(d.getVar('BBLAYERS').split())},${CCACHE_DIR}"
This has now landed in master and unfortunately it breaks the way I
run builds. I often end up building inside a layer directory. For
example the one that failed this morning has:
BBLAYERS ?= " \
/home/pbarker/Projects/Yocto/meta-linux-mainline \
/home/pbarker/Projects/Yocto/meta-linux-mainline/layers/poky/meta \
/home/pbarker/Projects/Yocto/meta-linux-mainline/layers/poky/meta-poky"
TOPDIR is "/home/pbarker/Projects/Yocto/meta-linux-mainline/build" and
TMPDIR is "${TOPDIR}/tmp". So everything under TMPDIR is now ignored
by pseudo and various do_install/do_package tasks fail.
I don't think this will just affect me as this is a common pattern,
especially when using something like kas to setup builds.
Before we revert this I'd like to understand the problem this patch is
trying to fix. Is there a specific reason why you need to ignore all
layer directories? I'm not 100% sure why "${COREBASE}/meta" is ignored
anyway though I guess it is to reduce pseudo db size and so speed up
builds.
Thanks,
--
Paul Barker
Konsulko Group
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#145341):
https://lists.openembedded.org/g/openembedded-core/message/145341
Mute This Topic: https://lists.openembedded.org/mt/78640454/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] Automatic cap on BB_NUMBER_THREADS and PARALLEL_MAKE
On Thu, 3 Dec 2020 at 17:48, Ross Burton wrote: > > Hi, > > Currently, BB_NUMBER_THREADS and PARALLEL_MAKE use the number of cores > available unless told otherwise. This was a good idea six years > ago[1] but some modern machines are moving to very large core counts. > > For example, 88 core dual Xeons are fairly common. A ThunderX2 has 256 > cores (2 sockets, 4 hyperthreads per physical core). The Ampere Altra > is dual socket 2*80=160 cores. > > At this level of parallelisation the sheer amount of I/O from the > unpack storm is quite excessive. As a strawman argument, I propose a > hard cap to the default BB_NUMBER_THREADS of -- and I'm literally > making up numbers here -- 32. Maybe 64. Comments? This is really going to depend on what storage technology you're using. I used to limit both at 8 when I was using traditional HDDs. With NVMe drives I see no need to limit at the top end of the systems I've built on (12c/24t). -- Paul Barker Konsulko Group -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#145248): https://lists.openembedded.org/g/openembedded-core/message/145248 Mute This Topic: https://lists.openembedded.org/mt/78690216/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] [OE-core][PATCH] kernel-dummy: fix executing unexpected tasks
On Wed, 25 Nov 2020 at 16:56, Andrej Valek wrote:
>
> - refactor kernel dummy recipe
> - remove unneeded COPYING.GPL file -> change license to close
> - remove unneeded tasks (fetch, unpack, pack)
> - do not run unneeded tasks, but keep deps (configure, compile, install)
> - save files into sstate
> - fix: ERROR: Task linux-dummy.do_fetch attempted to execute unexpectedly
>
> Signed-off-by: Andrej Valek
> ---
> meta/recipes-kernel/linux/linux-dummy.bb | 28 +-
> meta/recipes-kernel/linux/linux-dummy/COPYING.GPL | 339
> --
> 2 files changed, 12 insertions(+), 355 deletions(-)
> delete mode 100644 meta/recipes-kernel/linux/linux-dummy/COPYING.GPL
>
> diff --git a/meta/recipes-kernel/linux/linux-dummy.bb
> b/meta/recipes-kernel/linux/linux-dummy.bb
> index 62cf6f5ea6..e98a29d5c6 100644
> --- a/meta/recipes-kernel/linux/linux-dummy.bb
> +++ b/meta/recipes-kernel/linux/linux-dummy.bb
> @@ -4,11 +4,12 @@ provider for virtual/kernel to satisfy dependencies for
> situations \
> where you wish to build the kernel externally from the build system."
> SECTION = "kernel"
>
> -LICENSE = "GPLv2"
> -LIC_FILES_CHKSUM =
> "file://${WORKDIR}/COPYING.GPL;md5=751419260aa954499f7abaabaa882bbe"
> +LICENSE = "CLOSED"
I don't think "CLOSED" is a good choice to use in the core metadata.
The usual way to handle recipes like this which don't require external
source code is to say that they have the same license as the core
metadata itself (MIT) and reference the license text in
"meta/COPYING.MIT". See for example
meta/recipes-bsp/formfactor/formfactor_0.0.bb.
Thanks,
--
Paul Barker
Konsulko Group
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#144986):
https://lists.openembedded.org/g/openembedded-core/message/144986
Mute This Topic: https://lists.openembedded.org/mt/78504065/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH 1/2] poky.conf: Add opensuseleap-15.2 and fedora-33 to tested distros
On Tue, 24 Nov 2020 at 15:14, Paul Barker wrote: > > The autobuilder contains workers for both of these distros. > > Signed-off-by: Paul Barker > --- > meta-poky/conf/distro/poky.conf | 2 ++ > 1 file changed, 2 insertions(+) > > diff --git a/meta-poky/conf/distro/poky.conf b/meta-poky/conf/distro/poky.conf > index 31dc110bf6..bdf46f4038 100644 > --- a/meta-poky/conf/distro/poky.conf > +++ b/meta-poky/conf/distro/poky.conf > @@ -52,12 +52,14 @@ SANITY_TESTED_DISTROS ?= " \ > fedora-30 \n \ > fedora-31 \n \ > fedora-32 \n \ > +fedora-33 \n \ > centos-7 \n \ > centos-8 \n \ > debian-8 \n \ > debian-9 \n \ > debian-10 \n \ > opensuseleap-15.1 \n \ > +opensuseleap-15.2 \n \ > " > # add poky sanity bbclass > INHERIT += "poky-sanity" > -- > 2.26.2 > Sorry, wrong mailing list. -- Paul Barker Konsulko Group -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#144953): https://lists.openembedded.org/g/openembedded-core/message/144953 Mute This Topic: https://lists.openembedded.org/mt/78478908/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH 2/2] poky.conf: Drop fedora-30 from tested distros
The autobuilder no longer contains a worker for this distro. Signed-off-by: Paul Barker --- meta-poky/conf/distro/poky.conf | 1 - 1 file changed, 1 deletion(-) diff --git a/meta-poky/conf/distro/poky.conf b/meta-poky/conf/distro/poky.conf index bdf46f4038..35c9b1a27c 100644 --- a/meta-poky/conf/distro/poky.conf +++ b/meta-poky/conf/distro/poky.conf @@ -49,7 +49,6 @@ SANITY_TESTED_DISTROS ?= " \ ubuntu-16.04 \n \ ubuntu-18.04 \n \ ubuntu-20.04 \n \ -fedora-30 \n \ fedora-31 \n \ fedora-32 \n \ fedora-33 \n \ -- 2.26.2 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#144952): https://lists.openembedded.org/g/openembedded-core/message/144952 Mute This Topic: https://lists.openembedded.org/mt/78478909/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH 1/2] poky.conf: Add opensuseleap-15.2 and fedora-33 to tested distros
The autobuilder contains workers for both of these distros. Signed-off-by: Paul Barker --- meta-poky/conf/distro/poky.conf | 2 ++ 1 file changed, 2 insertions(+) diff --git a/meta-poky/conf/distro/poky.conf b/meta-poky/conf/distro/poky.conf index 31dc110bf6..bdf46f4038 100644 --- a/meta-poky/conf/distro/poky.conf +++ b/meta-poky/conf/distro/poky.conf @@ -52,12 +52,14 @@ SANITY_TESTED_DISTROS ?= " \ fedora-30 \n \ fedora-31 \n \ fedora-32 \n \ +fedora-33 \n \ centos-7 \n \ centos-8 \n \ debian-8 \n \ debian-9 \n \ debian-10 \n \ opensuseleap-15.1 \n \ +opensuseleap-15.2 \n \ " # add poky sanity bbclass INHERIT += "poky-sanity" -- 2.26.2 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#144951): https://lists.openembedded.org/g/openembedded-core/message/144951 Mute This Topic: https://lists.openembedded.org/mt/78478908/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [master/gatesgarth/dunfell PATCH] local.conf.sample: Update list of qemu machines
On Tue, 24 Nov 2020 at 12:44, Paul Barker wrote: > > The `qemuarmv5` and `qemuriscv64` machines have been support since the > warrior and zeus releases respectively. Add them to the local.conf > sample so that they're easier for new users to discover. > > Signed-off-by: Paul Barker > --- > meta/conf/local.conf.sample | 2 ++ > 1 file changed, 2 insertions(+) > > diff --git a/meta/conf/local.conf.sample b/meta/conf/local.conf.sample > index 22d43b20d4..742c3e3cb4 100644 > --- a/meta/conf/local.conf.sample > +++ b/meta/conf/local.conf.sample > @@ -19,11 +19,13 @@ > # > #MACHINE ?= "qemuarm" > #MACHINE ?= "qemuarm64" > +#MACHINE ?= "qemuarmv5" > #MACHINE ?= "qemumips" > #MACHINE ?= "qemumips64" > #MACHINE ?= "qemuppc" > #MACHINE ?= "qemux86" > #MACHINE ?= "qemux86-64" > +#MACHINE ?= "qemuriscv64" > # > # This sets the default machine to be qemux86-64 if no other machine is > selected: > MACHINE ??= "qemux86-64" > -- > 2.26.2 > After further discussion on the poky list, let's drop this for now. We don't have sufficient test coverage at the moment for either of these machines to list them in the sample config file. -- Paul Barker Konsulko Group -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#144949): https://lists.openembedded.org/g/openembedded-core/message/144949 Mute This Topic: https://lists.openembedded.org/mt/78475719/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [master/gatesgarth/dunfell PATCH] local.conf.sample: Update list of qemu machines
The `qemuarmv5` and `qemuriscv64` machines have been support since the warrior and zeus releases respectively. Add them to the local.conf sample so that they're easier for new users to discover. Signed-off-by: Paul Barker --- meta/conf/local.conf.sample | 2 ++ 1 file changed, 2 insertions(+) diff --git a/meta/conf/local.conf.sample b/meta/conf/local.conf.sample index 22d43b20d4..742c3e3cb4 100644 --- a/meta/conf/local.conf.sample +++ b/meta/conf/local.conf.sample @@ -19,11 +19,13 @@ # #MACHINE ?= "qemuarm" #MACHINE ?= "qemuarm64" +#MACHINE ?= "qemuarmv5" #MACHINE ?= "qemumips" #MACHINE ?= "qemumips64" #MACHINE ?= "qemuppc" #MACHINE ?= "qemux86" #MACHINE ?= "qemux86-64" +#MACHINE ?= "qemuriscv64" # # This sets the default machine to be qemux86-64 if no other machine is selected: MACHINE ??= "qemux86-64" -- 2.26.2 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#144947): https://lists.openembedded.org/g/openembedded-core/message/144947 Mute This Topic: https://lists.openembedded.org/mt/78475719/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] archiver.bbclass and GIT repositories
On Tue, 28 Jul 2020 at 13:56, Christian Eggers wrote: > > In my project, the linux kernel (together with dozens of our changes) is kept > in a company internal GIT repository. The GPLv2 requires the following: > > > 2a) You must cause the modified files to carry prominent notices stating > > that you changed the files and the date of any change. > > My idea is, that the GIT history (and diff) should be sufficient for > documenting > our changes to the linux source code. > > But when I unpack the .tar.gz file which was deploy by archiver.bbclass, > I get the following: > > # tar -xf my-linux-kernel.tar.gz > # cd /git > # git log > > > error: object directory > > /srv/gitlab-runner/downloads/[..]/yocto/git2/[xxx].git/objects does not > > exist; check .git/objects/info/alternates > > fatal: bad object HEAD > > Note: The project has been built on another machine. > > From the bitbake user manual, 4.3.5, Git Fetcher: > > "This bare clone is then cloned into the work directory during the unpack > > stage when a specific tree is checked out. This is done using alternates and > > by reference to minimize the amount of duplicate data on the disk and make > > the unpack process fast." > > It looks like the .git directory within the tar archive doesn't contain > the necessary information required for using the "git" command on > the extracted "working copy". If I understood the bitbake doc correctly, > the working copy is cloned "by reference" from a "bare" clone which > resided outside the archived directory (in the downloads dir). > > So I have three questions: > 1. Is it possible to archive the git project in a way, that the GIT history > can be examined later? > 2. If this is not possible, I would have to put my changes in an > "external" patch file alongside the bitbake recipe, right? > 3. Is there any value archiving the .git directory at all? What configuration are you using to enable the archiver? There may be ways to fix this by changing the configuration. Also, which Yocto Project version/branch are you using? Thanks, -- Paul Barker Konsulko Group -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#141080): https://lists.openembedded.org/g/openembedded-core/message/141080 Mute This Topic: https://lists.openembedded.org/mt/75843071/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH v4 01/12] Move rpm manifest to its own subdir
On Tue, 14 Jul 2020 at 08:02, Fredrik Gustafsson
wrote:
>
> Thank you, just let me know if I can make it any easier to review this,
> I know it's a big patch.
Hi Fredrik,
Sorry for the delays here, I've been unable to get back to this until
today due to other commitments and a few health issues.
Here's my feedback on the series as a whole. Overall I think it's
excellent, I much prefer the result where the code is grouped by
package manager (opkg, dpkg or rpm) to the existing code where it's
spread across package_manager.py, rootfs.py, manifest.py and sdk.py.
1) Please edit the first line of each commit message to follow the
style guide at
https://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines.
2) Please also fix the following whitespace errors reported when
applying these patches:
$ git am ../v4-Move-rpm-manifest-to-its-...-and-11-more.mbox
Applying: Move rpm manifest to its own subdir
.git/rebase-apply/patch:148: new blank line at EOF.
+
warning: 1 line adds whitespace errors.
Applying: Move ipk manifest to its own subdir
.git/rebase-apply/patch:187: new blank line at EOF.
+
warning: 1 line adds whitespace errors.
Applying: Move deb manifest to its own subdir
Applying: Move rpm rootfs to its own dir
.git/rebase-apply/patch:161: new blank line at EOF.
+
warning: 1 line adds whitespace errors.
Applying: Move ipk rootfs to its own dir
Applying: Move deb rootfs to its own dir
.git/rebase-apply/patch:222: new blank line at EOF.
+
warning: 1 line adds whitespace errors.
Applying: Move rpm sdk to its own dir
.git/rebase-apply/patch:127: new blank line at EOF.
+
warning: 1 line adds whitespace errors.
Applying: Move ipk sdk to its own dir
.git/rebase-apply/patch:42: trailing whitespace.
self.d.getVar("ALL_MULTILIB_PACKAGE_ARCHS"),
.git/rebase-apply/patch:109: new blank line at EOF.
+
warning: 2 lines add whitespace errors.
Applying: Move deb sdk to its own dir
.git/rebase-apply/patch:108: new blank line at EOF.
+
warning: 1 line adds whitespace errors.
Applying: Move rpm package manager to its own dir
.git/rebase-apply/patch:840: new blank line at EOF.
+
warning: 1 line adds whitespace errors.
Applying: Move ipk package manager to its own dir
.git/rebase-apply/patch:951: new blank line at EOF.
+
warning: 1 line adds whitespace errors.
Applying: Move deb package manager to its own dir
.git/rebase-apply/patch:1042: new blank line at EOF.
+
warning: 1 line adds whitespace errors.
3) I'd recommend the new module keeps the name package_manager. You
can do this by moving the existing code into
package_manager/__init__.py and then splitting it up inside that
directory. Imports and wrappers in the __init__.py file can be used to
ensure no other bit of the code needs to worry about the changes.
4) Please avoid unnecessarily using imports inside functions. In
rootfs.py, manifest.py and sdk.py you can import the required classes
at the top level. Functions like create_rootfs(), populate_sdk(), etc
should not require any changes as part of this patch series.
I'll run as much of the selftest suite as I can on this series as-is
to see if there are any other issues.
Thanks,
--
Paul Barker
Konsulko Group
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#140791):
https://lists.openembedded.org/g/openembedded-core/message/140791
Mute This Topic: https://lists.openembedded.org/mt/75277396/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [poky][master][PATCH] systemd: Fix codesonar warnings
continue; > +--- origcode/src/udev/udev-rules.c 2020-07-09 18:05:13.624021008 +0530 > modifcode/src/udev/udev-rules.c2020-07-09 18:03:03.227995790 +0530 > +@@ -726,7 +726,7 @@ > + return -1; > + > + /* skip whitespace */ > +-while (isspace(linepos[0]) || linepos[0] == ',') > ++while (isspace((unsigned char) linepos[0]) || linepos[0] == ',') > + linepos++; > + > + /* get the key */ > +@@ -738,7 +738,7 @@ > + linepos++; > + if (linepos[0] == '\0') > + return -1; > +-if (isspace(linepos[0])) > ++if (isspace((unsigned char) linepos[0])) > + break; > + if (linepos[0] == '=') > + break; > +@@ -751,7 +751,7 @@ > + temp = linepos; > + > + /* skip whitespace after key */ > +-while (isspace(linepos[0])) > ++while (isspace((unsigned char) linepos[0])) > + linepos++; > + if (linepos[0] == '\0') > + return -1; This patch seems to have got mangled by your email client, please re-send via `git send-email`. > This message contains information that may be privileged or confidential and > is the property of the KPIT Technologies Ltd. It is intended only for the > person to whom it is addressed. If you are not the intended recipient, you > are not authorized to read, print, retain copy, disseminate, distribute, or > use this message or any part thereof. If you receive this message in error, > please notify the sender immediately and delete all copies of this message. > KPIT Technologies Ltd. does not accept any liability for virus infected mails. I'm not comfortable seeing footers like this on patches, it suggests that the code above is subject to additional restrictions which would make it unsuitable for inclusion in an open source project. -- Paul Barker Konsulko Group -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#140518): https://lists.openembedded.org/g/openembedded-core/message/140518 Mute This Topic: https://lists.openembedded.org/mt/75416263/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH v4 01/12] Move rpm manifest to its own subdir
On Thu, 9 Jul 2020 at 11:42, Fredrik Gustafsson wrote: > > Hi, > is there something I can do to make this series move forward? > I guess it's waiting for at least one person to review it in detail. It's in my backlog, I should get to it on Monday. -- Paul Barker Konsulko Group -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#140485): https://lists.openembedded.org/g/openembedded-core/message/140485 Mute This Topic: https://lists.openembedded.org/mt/75277396/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] Add package managers as a plugin
On Thu, 2 Jul 2020 at 20:29, Fredrik Gustafsson wrote: > > OE-core today has three different package managers, the well-known formats deb > and rpm is supported as well as ipkg that is good for embedded devices. > > When building and having a good cache hit, a significant amount of time is > spent in the phase of generating a rootfs, which is really about the > performance of the package manager. To save build time and also get a > package manager that is suitanle for use on targets where flash memory is a > concern, support for apk is suggested. > > However, it might or might not be what's wanted for OE-core since it increases > the test matrix. Therefore I will refactor the package management code to > allow a layer to add a new package manager without editing the meta layer. > > This refactor will be divided into multiple patch series to be easier to > review, > this is the second patch series. Sorry to be a pain but I think this series suffers from the opposite problem to the previous version - it's only a partial move of some of the code. I'd much rather review a series which does the complete refactor of the existing code into separate modules without making any functional changes. I'll let others give their opinions as well, I'm happy to review this one if we do want to look at this piecemeal. -- Paul Barker Konsulko Group -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#140246): https://lists.openembedded.org/g/openembedded-core/message/140246 Mute This Topic: https://lists.openembedded.org/mt/75057633/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [oe-core][PATCH 1/1] systemd-conf: unpack to $S
On Thu, 2 Jul 2020 at 13:31, Richard Purdie wrote: > > On Wed, 2020-07-01 at 15:11 -0700, Joe Slater wrote: > > systemd-conf copies some files to the target but they > > are in WORKDIR and not visible to the archiver. Unpack > > them to S. > > > > Signed-off-by: Joe Slater > > --- > > .../systemd/systemd-conf_245.6.bb | 20 +-- > > > > 1 file changed, 10 insertions(+), 10 deletions(-) > > Whilst these patches are kind of ok but I can't help feeling the > archiver really needs fixing... Agreed. Joe, how are you configuring the archiver here? Also, are you not providing your layers along with the copies of the upstream source code? If you're providing the layers then you shouldn't need to worry about files like these being picked up by the archiver. Thanks, -- Paul Barker Konsulko Group -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#140229): https://lists.openembedded.org/g/openembedded-core/message/140229 Mute This Topic: https://lists.openembedded.org/mt/75246650/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH v3] nopackages.bbclass: Get tasks from variable
On Wed, 1 Jul 2020 at 11:12, Richard Purdie
wrote:
>
> On Wed, 2020-07-01 at 11:11 +0100, Richard Purdie via
> lists.openembedded.org wrote:
> > On Wed, 2020-07-01 at 10:54 +0100, Richard Purdie via
> > lists.openembedded.org wrote:
> > > On Wed, 2020-07-01 at 10:48 +0100, Paul Barker wrote:
> > > > On Wed, 1 Jul 2020 at 09:46, Fredrik Gustafsson
> > > > wrote:
> > > > > Get tasks from a variable instead of having them hardcoded.
> > > > > This
> > > > > will
> > > > > enable other layers to add tasks that should be deleted when
> > > > > nopackages
> > > > > is used.
> > > > >
> > > > > Signed-off-by: Fredrik Gustafsson
> > > > > ---
> > > > > meta/classes/nopackages.bbclass | 19 +++
> > > > > 1 file changed, 7 insertions(+), 12 deletions(-)
> > > > >
> > > > > diff --git a/meta/classes/nopackages.bbclass
> > > > > b/meta/classes/nopackages.bbclass
> > > > > index 559f5078bd..4c9fe510c5 100644
> > > > > --- a/meta/classes/nopackages.bbclass
> > > > > +++ b/meta/classes/nopackages.bbclass
> > > > > @@ -1,12 +1,7 @@
> > > > > -deltask do_package
> > > > > -deltask do_package_write_rpm
> > > > > -deltask do_package_write_ipk
> > > > > -deltask do_package_write_deb
> > > > > -deltask do_package_qa
> > > > > -deltask do_packagedata
> > > > > -deltask do_package_setscene
> > > > > -deltask do_package_write_rpm_setscene
> > > > > -deltask do_package_write_ipk_setscene
> > > > > -deltask do_package_write_deb_setscene
> > > > > -deltask do_package_qa_setscene
> > > > > -deltask do_packagedata_setscene
> > > > > +NO_PACKAGES_DELTASKS += "do_package do_package_write_rpm
> > > > > do_package_write_ipk do_package_write_deb do_package_qa
> > > > > do_packagedata do_package_setscene
> > > > > do_package_write_rpm_setscene
> > > > > do_package_write_ipk_setscene do_package_write_deb_setscene
> > > > > do_package_qa_setscene do_packagedata_setscene"
> > > >
> > > > Not sure if `+=` was present in the previous versions, I only
> > > > just
> > > > noticed it now. I think `?=` is a better choice so it can be
> > > > overridden easily. Other than that, this looks good.
> > > >
> > > > > +
> > > > > +python () {
> > > > > +list = d.getVar("NO_PACKAGES_DELTASKS").split()
> > > > > +for task in list:
> > > > > +bb.build.deltask(task, d)
> > > > > +}
> > >
> > > I'm just sad this is having to use anonymous python. Its slow and
> > > I'd
> > > like to see less of it, not more, given people's complaints about
> > > parsing speed.
> > >
> > > I did make a suggestion about how this could be done but that isn't
> > > an option. I might sort out such a patch.
> >
> > http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/t222&id=46a01aef15b19431bd9f2924537723481e0151e5
> >
> > is a patch which lets deltask become a bit more flexible. I've not
> > tested it much but see it if works/helps.
>
> Sorry:
> http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/t222&id=fd24e59fc886e6642bc3c8fb2941a37a1cc4042a
>
> Cheers,
>
> Richard (in need of caffeine)
I like this approach.
Fredrik, what's the reason for using a variable here again? Is it just
to be able to add tasks to the list or do you also need the ability to
remove tasks from the list?
Thanks,
--
Paul Barker
Konsulko Group
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#140173):
https://lists.openembedded.org/g/openembedded-core/message/140173
Mute This Topic: https://lists.openembedded.org/mt/75231136/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH v3] nopackages.bbclass: Get tasks from variable
On Wed, 1 Jul 2020 at 09:46, Fredrik Gustafsson
wrote:
>
> Get tasks from a variable instead of having them hardcoded. This will
> enable other layers to add tasks that should be deleted when nopackages
> is used.
>
> Signed-off-by: Fredrik Gustafsson
> ---
> meta/classes/nopackages.bbclass | 19 +++
> 1 file changed, 7 insertions(+), 12 deletions(-)
>
> diff --git a/meta/classes/nopackages.bbclass b/meta/classes/nopackages.bbclass
> index 559f5078bd..4c9fe510c5 100644
> --- a/meta/classes/nopackages.bbclass
> +++ b/meta/classes/nopackages.bbclass
> @@ -1,12 +1,7 @@
> -deltask do_package
> -deltask do_package_write_rpm
> -deltask do_package_write_ipk
> -deltask do_package_write_deb
> -deltask do_package_qa
> -deltask do_packagedata
> -deltask do_package_setscene
> -deltask do_package_write_rpm_setscene
> -deltask do_package_write_ipk_setscene
> -deltask do_package_write_deb_setscene
> -deltask do_package_qa_setscene
> -deltask do_packagedata_setscene
> +NO_PACKAGES_DELTASKS += "do_package do_package_write_rpm
> do_package_write_ipk do_package_write_deb do_package_qa do_packagedata
> do_package_setscene do_package_write_rpm_setscene
> do_package_write_ipk_setscene do_package_write_deb_setscene
> do_package_qa_setscene do_packagedata_setscene"
Not sure if `+=` was present in the previous versions, I only just
noticed it now. I think `?=` is a better choice so it can be
overridden easily. Other than that, this looks good.
> +
> +python () {
> +list = d.getVar("NO_PACKAGES_DELTASKS").split()
> +for task in list:
> +bb.build.deltask(task, d)
> +}
> --
> 2.20.1
--
Paul Barker
Konsulko Group
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#140169):
https://lists.openembedded.org/g/openembedded-core/message/140169
Mute This Topic: https://lists.openembedded.org/mt/75231136/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] Status of Go support
On Thu, 25 Jun 2020 at 16:36, Stefan Herbrechtsmeier wrote: > > Am 24.06.20 um 22:24 schrieb Otavio Salvador: > > Em qua., 24 de jun. de 2020 às 17:17, Stefan Herbrechtsmeier > > escreveu: > >> The go-mod.bbclass download data from the internet during do_compile. If > >> you cut the internet connection after a do_fetch the do_compile failed: > > > > Yes, this was the better we could get without hacking go mod tool > > itself. It would be nice if you can provide an alternative way to > > improve it. > > > > What are the alternative? > > a) Add a fetcher to bitbake to create a vendor directory via `go mod > vendor`? > b) Add a `create_go.py` to recipetool and create a recipe per module? > > The solutions are related to the npm (a) and python (b) solution. > > I don't like a) because it breaks the best practices of OE (for example > one major version per project and avoid embedded dependencies). On the > other side b) isn't so simple because of the circular dependency between > go projects. This means we need a package (recipe) for the source and > the binary. (a) may not be in line with past best practices but I think it's better than no change. It at least gives the archiver and licensing classes a chance to capture the relevant artifacts (if configured correctly) and so do proper license compliance and/or maintain a source mirror. I think the Python method of one recipe per dependency works well for Python since it's not common to distribute a built artifact which statically links all those dependencies together. That's often not the case with rust and golang projects though. I still think one recipe per upstream project with dependencies fully expressed in the bitbake recipes would be ideal but as others have said that's fighting a losing battle. -- Paul Barker Konsulko Group -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#139965): https://lists.openembedded.org/g/openembedded-core/message/139965 Mute This Topic: https://lists.openembedded.org/mt/75063635/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] Distro name
On Thu, 25 Jun 2020 at 11:52, Pankaj Vinadrao Joshi
wrote:
>
> [Edited Message Follows]
>
> Hi,
> i have built image for raspberrypi4-64 using openembedded-core and i am
> trying to run some test specific to lkp but i am getting issue with the distro
> which is nodistro.0 but the test needs distro for it
>
> i am attaching the output of os-realease
> ID=nodistro
> NAME="OpenEmbedded"
> VERSION="nodistro.0"
> VERSION_ID=nodistro.0
> PRETTY_NAME="OpenEmbedded nodistro.0"
>
> i am getting following error with respect to lkp
>
> ERROR: "Not a supported system, cannot install packages."
>
> detect_system distro=$_system_name_lowercase
> DISTRO=$distro export DISTRO
> [ -x $LKP_SRC/distro/installer/$distro ] || { echo "Not a supported
> system, cannot install packages." exit 1 }
>
>
> can somebody help me how i can add distro for raspberrypi??
If you're unsure what a distro is in the context of Yocto Project /
OpenEmbedded then I recommend you find one of the many tutorials
online and work through it. Also, his question is not specific to
openembedded-core, please use a more appropriate list like
https://lists.yoctoproject.org/g/yocto for questions like this.
--
Paul Barker
Konsulko Group
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#139949):
https://lists.openembedded.org/g/openembedded-core/message/139949
Mute This Topic: https://lists.openembedded.org/mt/75100327/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH v2 26/26] package_manager: Rename PkgsList classes
kgsList(d,
> rootfs_dir).list_pkgs()
>
> if __name__ == "__main__":
> """
> diff --git a/meta/lib/oe/sdk.py b/meta/lib/oe/sdk.py
> index 1fac0bed5b..7b17afde78 100644
> --- a/meta/lib/oe/sdk.py
> +++ b/meta/lib/oe/sdk.py
> @@ -116,19 +116,8 @@ def sdk_list_installed_packages(d, target,
> rootfs_dir=None):
>
> rootfs_dir = [sdk_output, os.path.join(sdk_output,
> target_path)][target is True]
>
> -from oe.package_managers.rpm.package_manager import RpmPkgsList
> -from oe.package_managers.ipk.package_manager import OpkgPkgsList
> -from oe.package_managers.deb.package_manager import DpkgPkgsList
> -img_type = d.getVar('IMAGE_PKGTYPE')
> -if img_type == "rpm":
> -arch_var = ["SDK_PACKAGE_ARCHS", None][target is True]
> -os_var = ["SDK_OS", None][target is True]
> -return RpmPkgsList(d, rootfs_dir).list_pkgs()
> -elif img_type == "ipk":
> -conf_file_var = ["IPKGCONF_SDK", "IPKGCONF_TARGET"][target is True]
> -return OpkgPkgsList(d, rootfs_dir,
> d.getVar(conf_file_var)).list_pkgs()
> -elif img_type == "deb":
> -return DpkgPkgsList(d, rootfs_dir).list_pkgs()
> +import importlib
> +return importlib.import_module('oe.package_managers.' +
> d.getVar('IMAGE_PKGTYPE') + '.package_manager').PkgsList(d,
> rootfs_dir).list_pkgs()
>
> def populate_sdk(d, manifest_dir=None):
> env_bkp = os.environ.copy()
Same as my previous comments, let's just do a move first and add the
dynamic loading & class renaming later.
--
Paul Barker
Konsulko Group
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#139947):
https://lists.openembedded.org/g/openembedded-core/message/139947
Mute This Topic: https://lists.openembedded.org/mt/75100012/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH v2 22/26] package_manager: Rename Indexer classes
ib/oe/package_managers/rpm/package_manager.py
> b/meta/lib/oe/package_managers/rpm/package_manager.py
> index 025e8cdfd2..e781676183 100644
> --- a/meta/lib/oe/package_managers/rpm/package_manager.py
> +++ b/meta/lib/oe/package_managers/rpm/package_manager.py
> @@ -384,8 +384,11 @@ class RpmIndexer(Indexer):
>
> self.d.getVar('PACKAGE_FEED_GPG_PASSPHRASE_FILE'),
> armor=is_ascii_sig)
>
> -class RpmSubdirIndexer(RpmIndexer):
> +class PkgIndexer(RpmIndexer):
> def write_index(self):
> +if self.deploy_dir == "":
> +self.deploy_dir = self.d.getVar('DEPLOY_DIR_RPM')
> +
> bb.note("Generating package index for %s" %(self.deploy_dir))
> self.do_write_index(self.deploy_dir)
> for entry in os.walk(self.deploy_dir):
As per previous comments, let's delay this for a later series. Also
please keep whitespace changes separate from functional changes.
--
Paul Barker
Konsulko Group
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#139946):
https://lists.openembedded.org/g/openembedded-core/message/139946
Mute This Topic: https://lists.openembedded.org/mt/75100016/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH v2 21/26] package_manager.py: Move DpkgIndexer
On Thu, 25 Jun 2020 at 11:22, Fredrik Gustafsson wrote: > > Let the code that is only used by opkg package manager live in that Typo here: This commit moves the dpkg indexer not the opkg one. -- Paul Barker Konsulko Group -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#139945): https://lists.openembedded.org/g/openembedded-core/message/139945 Mute This Topic: https://lists.openembedded.org/mt/75100013/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH v2 14/26] rootfs.py: Dynamic load of rootfs
pkg_name = m_pkg.group(1)
> -#Make sure we reset other variables
> -pkg_status_match = False
> -pkg_depends = ""
> -elif m_status is not None:
> -#New status matched
> -pkg_status_match = True
> -elif m_depends is not None:
> -#New depends macthed
> -pkg_depends = m_depends.group(1)
> -else:
> -pass
> -
> -#Now check if we can process package depends and postinst
> -if "" != pkg_name and pkg_status_match:
> -pkgs[pkg_name] = _get_pkg_depends_list(pkg_depends)
> -else:
> -#Not enough information
> -pass
> -
> -# remove package dependencies not in postinsts
> -pkg_names = list(pkgs.keys())
> -for pkg_name in pkg_names:
> -deps = pkgs[pkg_name][:]
> -
> -for d in deps:
> -if d not in pkg_names:
> -pkgs[pkg_name].remove(d)
> -
> -return pkgs
> -
> -def _get_delayed_postinsts_common(self, status_file):
> -def _dep_resolve(graph, node, resolved, seen):
> -seen.append(node)
> -
> -for edge in graph[node]:
> -if edge not in resolved:
> -if edge in seen:
> -raise RuntimeError("Packages %s and %s have " \
> -"a circular dependency in postinsts
> scripts." \
> -% (node, edge))
> -_dep_resolve(graph, edge, resolved, seen)
> -
> -resolved.append(node)
> -
> -pkg_list = []
> -
> -pkgs = None
> -if not self.d.getVar('PACKAGE_INSTALL').strip():
> -bb.note("Building empty image")
> -else:
> -pkgs = self._get_pkgs_postinsts(status_file)
> -if pkgs:
> -root = "__packagegroup_postinst__"
> -pkgs[root] = list(pkgs.keys())
> -_dep_resolve(pkgs, root, pkg_list, [])
> -pkg_list.remove(root)
> -
> -if len(pkg_list) == 0:
> -return None
> -
> -return pkg_list
> -
> -def _save_postinsts_common(self, dst_postinst_dir, src_postinst_dir):
> -if bb.utils.contains("IMAGE_FEATURES", "package-management",
> - True, False, self.d):
> -return
> -num = 0
> -for p in self._get_delayed_postinsts():
> -bb.utils.mkdirhier(dst_postinst_dir)
> -
> -if os.path.exists(os.path.join(src_postinst_dir, p +
> ".postinst")):
> -shutil.copy(os.path.join(src_postinst_dir, p + ".postinst"),
> -os.path.join(dst_postinst_dir, "%03d-%s" % (num,
> p)))
> -
> -num += 1
> -
> -def get_class_for_type(imgtype):
> -from oe.package_managers.rpm.rootfs import RpmRootfs
> -from oe.package_managers.ipk.rootfs import OpkgRootfs
> -from oe.package_managers.deb.rootfs import DpkgRootfs
> -return {"rpm": RpmRootfs,
> -"ipk": OpkgRootfs,
> -"deb": DpkgRootfs}[imgtype]
> -
> def variable_depends(d, manifest_dir=None):
> -img_type = d.getVar('IMAGE_PKGTYPE')
> -cls = get_class_for_type(img_type)
> -return cls._depends_list()
> +import importlib
> +return importlib.import_module('oe.package_managers.' +
> d.getVar('IMAGE_PKGTYPE') + '.rootfs').PkgRootfs._depends_list()
>
> def create_rootfs(d, manifest_dir=None, progress_reporter=None,
> logcatcher=None):
> env_bkp = os.environ.copy()
>
> -img_type = d.getVar('IMAGE_PKGTYPE')
> -from oe.package_managers.rpm.rootfs import RpmRootfs
> -from oe.package_managers.ipk.rootfs import OpkgRootfs
> -from oe.package_managers.deb.rootfs import DpkgRootfs
> -if img_type == "rpm":
> -RpmRootfs(d, manifest_dir, progress_reporter, logcatcher).create()
> -elif img_type == "ipk":
> -OpkgRootfs(d, manifest_dir, progress_reporter, logcatcher).create()
> -elif img_type == "deb":
> -DpkgRootfs(d, manifest_dir, progress_reporter, logcatcher).create()
> +import importlib
> +return importlib.import_module('oe.package_managers.' +
> d.getVar('IMAGE_PKGTYPE') + '.rootfs').PkgRootfs(d, manifest_dir,
> progress_reporter, logcatcher).create()
>
> os.environ.clear()
> os.environ.update(env_bkp)
> --
> 2.20.1
Again, let's delay this to a later series.
--
Paul Barker
Konsulko Group
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#139943):
https://lists.openembedded.org/g/openembedded-core/message/139943
Mute This Topic: https://lists.openembedded.org/mt/75100017/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH v2 18/26] package_manager: Rename RpmPM to PkgPM, etc.
> - self.pkg_archs)
> +self.pm = PkgPM(d,
> +self.image_rootfs,
> +self.opkg_conf,
> +self.pkg_archs)
> else:
> -self.pm = OpkgPM(d,
> - self.image_rootfs,
> - self.opkg_conf,
> - self.pkg_archs)
> +self.pm = PkgPM(d,
> +self.image_rootfs,
> +self.opkg_conf,
> +self.pkg_archs)
> self.pm.recover_packaging_data()
>
> bb.utils.remove(self.d.getVar('MULTILIB_TEMP_ROOTFS'), True)
> diff --git a/meta/lib/oe/package_managers/ipk/sdk.py
> b/meta/lib/oe/package_managers/ipk/sdk.py
> index bd7bab6ebe..b237148d15 100644
> --- a/meta/lib/oe/package_managers/ipk/sdk.py
> +++ b/meta/lib/oe/package_managers/ipk/sdk.py
> @@ -4,6 +4,7 @@
>
> from oe.sdk import *
> from oe.package_managers.ipk.manifest import *
> +from oe.package_managers.ipk.package_manager import *
>
> class PkgSdk(Sdk):
> def __init__(self, d, manifest_dir=None):
> @@ -21,11 +22,11 @@ class PkgSdk(Sdk):
> if "sdk_ext" in d.getVar("BB_RUNTASK"):
> ipk_repo_workdir = "oe-sdk-ext-repo"
>
> -self.target_pm = OpkgPM(d, self.sdk_target_sysroot, self.target_conf,
> +self.target_pm = PkgPM(d, self.sdk_target_sysroot, self.target_conf,
> self.d.getVar("ALL_MULTILIB_PACKAGE_ARCHS"),
> ipk_repo_workdir=ipk_repo_workdir)
>
> -self.host_pm = OpkgPM(d, self.sdk_host_sysroot, self.host_conf,
> +self.host_pm = PkgPM(d, self.sdk_host_sysroot, self.host_conf,
>self.d.getVar("SDK_PACKAGE_ARCHS"),
> ipk_repo_workdir=ipk_repo_workdir)
>
> diff --git a/meta/lib/oe/package_managers/rpm/package_manager.py
> b/meta/lib/oe/package_managers/rpm/package_manager.py
> index f34190f5bd..306c3336ca 100644
> --- a/meta/lib/oe/package_managers/rpm/package_manager.py
> +++ b/meta/lib/oe/package_managers/rpm/package_manager.py
> @@ -4,7 +4,7 @@
>
> from oe.package_manager import *
>
> -class RpmPM(PackageManager):
> +class PkgPM(PackageManager):
> def __init__(self,
> d,
> target_rootfs,
> @@ -15,7 +15,7 @@ class RpmPM(PackageManager):
> rpm_repo_workdir="oe-rootfs-repo",
> filterbydependencies=True,
> needfeed=True):
> -super(RpmPM, self).__init__(d, target_rootfs)
> +super(PkgPM, self).__init__(d, target_rootfs)
> self.target_vendor = target_vendor
> self.task_name = task_name
> if arch_var == None:
> diff --git a/meta/lib/oe/package_managers/rpm/rootfs.py
> b/meta/lib/oe/package_managers/rpm/rootfs.py
> index 7b9e176bcb..26232e6ffb 100644
> --- a/meta/lib/oe/package_managers/rpm/rootfs.py
> +++ b/meta/lib/oe/package_managers/rpm/rootfs.py
> @@ -3,6 +3,8 @@
> #
>
> from oe.rootfs import *
> +from oe.package_managers.rpm.package_manager import *
> +from oe.package_managers.rpm.manifest import *
>
> class PkgRootfs(Rootfs):
> def __init__(self, d, manifest_dir, progress_reporter=None,
> logcatcher=None):
> @@ -11,10 +13,8 @@ class PkgRootfs(Rootfs):
> r'|exit 1|ERROR: |Error: |Error |ERROR '\
> r'|Failed |Failed: |Failed$|Failed\(\d+\):)'
>
> -import importlib
> -self.manifest =
> importlib.import_module('oe.package_managers.rpm.manifest').PkgManifest(d,
> manifest_dir)
> -
> -self.pm = RpmPM(d,
> +self.manifest = PkgManifest(d, manifest_dir)
> +self.pm = PkgPM(d,
> d.getVar('IMAGE_ROOTFS'),
> self.d.getVar('TARGET_VENDOR')
> )
> diff --git a/meta/lib/oe/package_managers/rpm/sdk.py
> b/meta/lib/oe/package_managers/rpm/sdk.py
> index d2f7447657..abd3ce5569 100644
> --- a/meta/lib/oe/package_managers/rpm/sdk.py
> +++ b/meta/lib/oe/package_managers/rpm/sdk.py
> @@ -4,6 +4,7 @@
>
> from oe.sdk import *
> from oe.package_managers.rpm.manifest import *
> +from oe.package_managers.rpm.package_manager import *
>
> class PkgSdk(Sdk):
> def __init__(self, d, manifest_dir=None, rpm_workdir="oe-sdk-repo"):
> @@ -18,14 +19,14 @@ class PkgSdk(Sdk):
> if "sdk_ext" in d.getVar("BB_RUNTASK"):
> rpm_repo_workdir = "oe-sdk-ext-repo"
>
> -self.target_pm = RpmPM(d,
> +self.target_pm = PkgPM(d,
> self.sdk_target_sysroot,
> self.d.getVar('TARGET_VENDOR'),
> 'target',
> rpm_repo_workdir=rpm_repo_workdir
> )
>
> -self.host_pm = RpmPM(d,
> +self.host_pm = PkgPM(d,
> self.sdk_host_sysroot,
> self.d.getVar('SDK_VENDOR'),
> 'host',
I think this should be delayed to a later patch series. Let's get
everything moved first using the current naming.
--
Paul Barker
Konsulko Group
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#139944):
https://lists.openembedded.org/g/openembedded-core/message/139944
Mute This Topic: https://lists.openembedded.org/mt/7515/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH v2 06/26] manifest.py: Dynamic load of manifest
anifest(d, manifest_dir)
> +import importlib
> +self.manifest =
> importlib.import_module('oe.package_managers.deb.manifest').PkgManifest(d,
> manifest_dir)
> self.pm = DpkgPM(d, d.getVar('IMAGE_ROOTFS'),
> d.getVar('PACKAGE_ARCHS'),
> d.getVar('DPKG_ARCH'))
> @@ -699,8 +698,8 @@ class OpkgRootfs(DpkgOpkgRootfs):
> super(OpkgRootfs, self).__init__(d, progress_reporter, logcatcher)
> self.log_check_regex = '(exit 1|Collected errors)'
>
> -from oe.package_managers.ipk.manifest import OpkgManifest
> -self.manifest = OpkgManifest(d, manifest_dir)
> +import importlib
> +self.manifest =
> importlib.import_module('oe.package_managers.ipk.manifest').PkgManifest(d,
> manifest_dir)
> self.opkg_conf = self.d.getVar("IPKGCONF_TARGET")
> self.pkg_archs = self.d.getVar("ALL_MULTILIB_PACKAGE_ARCHS")
>
> diff --git a/meta/lib/oe/sdk.py b/meta/lib/oe/sdk.py
> index 70451a0b3a..18f6c16bd2 100644
> --- a/meta/lib/oe/sdk.py
> +++ b/meta/lib/oe/sdk.py
> @@ -114,11 +114,9 @@ class RpmSdk(Sdk):
> def __init__(self, d, manifest_dir=None, rpm_workdir="oe-sdk-repo"):
> super(RpmSdk, self).__init__(d, manifest_dir)
>
> -from oe.package_managers.rpm.manifest import RpmManifest
> -self.target_manifest = RpmManifest(d, self.manifest_dir,
> - Manifest.MANIFEST_TYPE_SDK_TARGET)
> -self.host_manifest = RpmManifest(d, self.manifest_dir,
> - Manifest.MANIFEST_TYPE_SDK_HOST)
> +import importlib
> +self.target_manifest =
> importlib.import_module('oe.package_managers.rpm.manifest').PkgManifest(d,
> self.manifest_dir, Manifest.MANIFEST_TYPE_SDK_TARGET)
> +self.host_manifest =
> importlib.import_module('oe.package_managers.rpm.manifest').PkgManifest(d,
> self.manifest_dir, Manifest.MANIFEST_TYPE_SDK_HOST)
>
> rpm_repo_workdir = "oe-sdk-repo"
> if "sdk_ext" in d.getVar("BB_RUNTASK"):
> @@ -232,14 +230,9 @@ class OpkgSdk(Sdk):
> if "sdk_ext" in d.getVar("BB_RUNTASK"):
> ipk_repo_workdir = "oe-sdk-ext-repo"
>
> -from oe.package_managers.ipk.manifest import OpkgManifest
> -self.target_pm = OpkgPM(d, self.sdk_target_sysroot, self.target_conf,
> -self.d.getVar("ALL_MULTILIB_PACKAGE_ARCHS"),
> -ipk_repo_workdir=ipk_repo_workdir)
> -
> -self.host_pm = OpkgPM(d, self.sdk_host_sysroot, self.host_conf,
> - self.d.getVar("SDK_PACKAGE_ARCHS"),
> -ipk_repo_workdir=ipk_repo_workdir)
> +import importlib
> +self.target_manifest =
> importlib.import_module('oe.package_managers.ipk.manifest').PkgManifest(d,
> self.sdk_target_sysroot, self.target_conf,
> self.d.getVar("ALL_MULTILIB_PACKAGE_ARCHS"),
> ipk_repo_workdir=ipk_repo_workdir)
> +self.host_manifest =
> importlib.import_module('oe.package_managers.ipk.manifest').PkgManifest(d,
> self.sdk_host_sysroot, self.host_conf, self.d.getVar("SDK_PACKAGE_ARCHS"),
> ipk_repo_workdir=ipk_repo_workdir)
>
> def _populate_sysroot(self, pm, manifest):
> pkgs_to_install = manifest.parse_initial_manifest()
> @@ -310,11 +303,9 @@ class DpkgSdk(Sdk):
> self.target_conf_dir = os.path.join(self.d.getVar("APTCONF_TARGET"),
> "apt")
> self.host_conf_dir = os.path.join(self.d.getVar("APTCONF_TARGET"),
> "apt-sdk")
>
> -from oe.package_managers.deb.manifest import DpkgManifest
> -self.target_manifest = DpkgManifest(d, self.manifest_dir,
> -
> Manifest.MANIFEST_TYPE_SDK_TARGET)
> -self.host_manifest = DpkgManifest(d, self.manifest_dir,
> - Manifest.MANIFEST_TYPE_SDK_HOST)
> +import importlib
> + self.target_manifest =
> importlib.import_module('oe.package_managers.deb.manifest').PkgManifest(d,
> self.manifest_dir, Manifest.MANIFEST_TYPE_SDK_TARGET)
> +self.host_manifest =
> importlib.import_module('oe.package_managers.deb.manifest').PkgManifest(d,
> self.manifest_dir, Manifest.MANIFEST_TYPE_SDK_HOST)
>
> deb_repo_workdir = "oe-sdk-repo"
> if "sdk_ext" in d.getVar("BB_RUNTASK"):
> --
> 2.20.1
This is a functional change rather than just moving code. I recommend
the first patch series here just moves the relevant code and adds
additional imports, redirection, etc as needed. Once that's reviewed
and merged then we can look at a follow up series to make things more
dynamic and support package managers being added by other layers.
Mixing the functional changes and the code moves in a single series
makes review & testing difficult.
I also think we'd need a test case for this. Perhaps a dummy package
manager can be added in the meta-selftest layer and a selftest case
can confirm that this is loaded.
--
Paul Barker
Konsulko Group
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#139941):
https://lists.openembedded.org/g/openembedded-core/message/139941
Mute This Topic: https://lists.openembedded.org/mt/7509/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH v2 02/26] package_managers: Add directory structure
On Thu, 25 Jun 2020 at 11:22, Fredrik Gustafsson wrote: > > This is first part of a refactoring to split up the package managers to > different files making it easier to extend OE-Core with more package > managers. > > Signed-off-by: Fredrik Gustafsson > --- > meta/lib/oe/package_managers/deb/__init__.py | 3 +++ > meta/lib/oe/package_managers/ipk/__init__.py | 3 +++ > meta/lib/oe/package_managers/rpm/__init__.py | 3 +++ > 3 files changed, 9 insertions(+) > create mode 100644 meta/lib/oe/package_managers/deb/__init__.py > create mode 100644 meta/lib/oe/package_managers/ipk/__init__.py > create mode 100644 meta/lib/oe/package_managers/rpm/__init__.py > > diff --git a/meta/lib/oe/package_managers/deb/__init__.py > b/meta/lib/oe/package_managers/deb/__init__.py > new file mode 100644 > index 00..a2094304c9 > --- /dev/null > +++ b/meta/lib/oe/package_managers/deb/__init__.py > @@ -0,0 +1,3 @@ > +# > +# SPDX-License-Identifier: GPL-2.0-only > +# > diff --git a/meta/lib/oe/package_managers/ipk/__init__.py > b/meta/lib/oe/package_managers/ipk/__init__.py > new file mode 100644 > index 00..a2094304c9 > --- /dev/null > +++ b/meta/lib/oe/package_managers/ipk/__init__.py > @@ -0,0 +1,3 @@ > +# > +# SPDX-License-Identifier: GPL-2.0-only > +# > diff --git a/meta/lib/oe/package_managers/rpm/__init__.py > b/meta/lib/oe/package_managers/rpm/__init__.py > new file mode 100644 > index 00..a2094304c9 > --- /dev/null > +++ b/meta/lib/oe/package_managers/rpm/__init__.py > @@ -0,0 +1,3 @@ > +# > +# SPDX-License-Identifier: GPL-2.0-only > +# These directories can be created as needed in later patches, there's no need to break this out into a separate commit. -- Paul Barker Konsulko Group -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#139940): https://lists.openembedded.org/g/openembedded-core/message/139940 Mute This Topic: https://lists.openembedded.org/mt/7505/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH v2 10/26] sdk.py: Dynamic load of sdk
On Thu, 25 Jun 2020 at 11:22, Fredrik Gustafsson
wrote:
>
> Decide which sdk we should load in run time without any hard coded
> values but look at which package type that is used.
>
> Signed-off-by: Fredrik Gustafsson
> ---
> meta/lib/oe/package_managers/deb/sdk.py | 4 ++--
> meta/lib/oe/package_managers/ipk/sdk.py | 4 ++--
> meta/lib/oe/package_managers/rpm/sdk.py | 4 ++--
> meta/lib/oe/sdk.py | 13 ++---
> 4 files changed, 8 insertions(+), 17 deletions(-)
>
> diff --git a/meta/lib/oe/package_managers/deb/sdk.py
> b/meta/lib/oe/package_managers/deb/sdk.py
> index 154ec5ab17..950455988a 100644
> --- a/meta/lib/oe/package_managers/deb/sdk.py
> +++ b/meta/lib/oe/package_managers/deb/sdk.py
> @@ -5,9 +5,9 @@
> from oe.sdk import *
> from oe.package_managers.deb.manifest import *
>
> -class DpkgSdk(Sdk):
> +class PkgSdk(Sdk):
> def __init__(self, d, manifest_dir=None):
> -super(DpkgSdk, self).__init__(d, manifest_dir)
> +super(PkgSdk, self).__init__(d, manifest_dir)
>
> self.target_conf_dir = os.path.join(self.d.getVar("APTCONF_TARGET"),
> "apt")
> self.host_conf_dir = os.path.join(self.d.getVar("APTCONF_TARGET"),
> "apt-sdk")
> diff --git a/meta/lib/oe/package_managers/ipk/sdk.py
> b/meta/lib/oe/package_managers/ipk/sdk.py
> index 4862616c08..bd7bab6ebe 100644
> --- a/meta/lib/oe/package_managers/ipk/sdk.py
> +++ b/meta/lib/oe/package_managers/ipk/sdk.py
> @@ -5,9 +5,9 @@
> from oe.sdk import *
> from oe.package_managers.ipk.manifest import *
>
> -class OpkgSdk(Sdk):
> +class PkgSdk(Sdk):
> def __init__(self, d, manifest_dir=None):
> -super(OpkgSdk, self).__init__(d, manifest_dir)
> +super(PkgSdk, self).__init__(d, manifest_dir)
>
> self.target_conf = self.d.getVar("IPKGCONF_TARGET")
> self.host_conf = self.d.getVar("IPKGCONF_SDK")
> diff --git a/meta/lib/oe/package_managers/rpm/sdk.py
> b/meta/lib/oe/package_managers/rpm/sdk.py
> index fc120b6171..d2f7447657 100644
> --- a/meta/lib/oe/package_managers/rpm/sdk.py
> +++ b/meta/lib/oe/package_managers/rpm/sdk.py
> @@ -5,9 +5,9 @@
> from oe.sdk import *
> from oe.package_managers.rpm.manifest import *
>
> -class RpmSdk(Sdk):
> +class PkgSdk(Sdk):
> def __init__(self, d, manifest_dir=None, rpm_workdir="oe-sdk-repo"):
> -super(RpmSdk, self).__init__(d, manifest_dir)
> +super(PkgSdk, self).__init__(d, manifest_dir)
>
> self.target_manifest = PkgManifest(d, self.manifest_dir,
> Manifest.MANIFEST_TYPE_SDK_TARGET)
> diff --git a/meta/lib/oe/sdk.py b/meta/lib/oe/sdk.py
> index d8a00c04d1..3b7869f071 100644
> --- a/meta/lib/oe/sdk.py
> +++ b/meta/lib/oe/sdk.py
> @@ -130,17 +130,8 @@ def sdk_list_installed_packages(d, target,
> rootfs_dir=None):
> def populate_sdk(d, manifest_dir=None):
> env_bkp = os.environ.copy()
>
> -img_type = d.getVar('IMAGE_PKGTYPE')
> -
> -from oe.package_managers.rpm.sdk import RpmSdk
> -from oe.package_managers.ipk.sdk import OpkgSdk
> -from oe.package_managers.deb.sdk import DpkgSdk
> -if img_type == "rpm":
> -RpmSdk(d, manifest_dir).populate()
> -elif img_type == "ipk":
> -OpkgSdk(d, manifest_dir).populate()
> -elif img_type == "deb":
> - DpkgSdk(d, manifest_dir).populate()
> +import importlib
> +importlib.import_module('oe.package_managers.' +
> d.getVar('IMAGE_PKGTYPE') + '.sdk').PkgSdk(d, manifest_dir).populate()
>
> os.environ.clear()
> os.environ.update(env_bkp)
See my comments on the patch to add dynamic loading of the manifest
class - the same comments apply here.
--
Paul Barker
Konsulko Group
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#139942):
https://lists.openembedded.org/g/openembedded-core/message/139942
Mute This Topic: https://lists.openembedded.org/mt/7511/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH v2 01/26] nopackages.bbclass: Get tasks from variable
On Thu, 25 Jun 2020 at 11:22, Fredrik Gustafsson wrote: > > Get tasks from a variable instead of having them hardcoded. This will > enable other layers to add tasks that should be deleted when nopackages > is used. > > Signed-off-by: Fredrik Gustafsson > --- > meta/classes/nopackages.bbclass | 19 +++ > 1 file changed, 7 insertions(+), 12 deletions(-) > > diff --git a/meta/classes/nopackages.bbclass b/meta/classes/nopackages.bbclass > index 559f5078bd..f3ef5f 100644 > --- a/meta/classes/nopackages.bbclass > +++ b/meta/classes/nopackages.bbclass > @@ -1,12 +1,7 @@ > -deltask do_package > -deltask do_package_write_rpm > -deltask do_package_write_ipk > -deltask do_package_write_deb > -deltask do_package_qa > -deltask do_packagedata > -deltask do_package_setscene > -deltask do_package_write_rpm_setscene > -deltask do_package_write_ipk_setscene > -deltask do_package_write_deb_setscene > -deltask do_package_qa_setscene > -deltask do_packagedata_setscene > +NO_PACKAGES += "do_package do_package_write_rpm do_package_write_ipk > do_package_write_deb do_package_qa do_packagedata do_package_setscene > do_package_write_rpm_setscene do_package_write_ipk_setscene > do_package_write_deb_setscene do_package_qa_setscene do_packagedata_setscene" Could we use a more descriptive name here, e.g. NO_PACKAGES_DELTASKS? -- Paul Barker Konsulko Group -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#139939): https://lists.openembedded.org/g/openembedded-core/message/139939 Mute This Topic: https://lists.openembedded.org/mt/7504/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] Add package managers as a plugin
On Thu, 25 Jun 2020 at 11:21, Fredrik Gustafsson wrote: > > Poky today has three different package managers, the well-known formats deb > and rpm is supported as well as ipkg that is good for embedded devices. > > When building and having a good cache hit, a significant amount of time is > spent in the phase of generating a rootfs, which is really about the > performance of the package manager. ipkg is way slower than deb or rpm. To > save build time and also get a package manager that is suitanle for use on > targets where flash memory is a concern, support for apk is suggested. > > However, it might or might not be what's wanted for Poky since it increases > the test matrix. Therefore this patch series refactors the package > management code so that it's possible to add more package managers in other > own layer. I will send another patch serie that will add apk. > > Perfomance metrics below, (note that this includes build times). > > APK > === > bitbake core-image-minimal > 15.84s user 2.60s system 0% cpu 1:26:19.21 total > 16.01s user 2.58s system 0% cpu 1:26:03.72 total > 15.69s user 2.61s system 0% cpu 1:26:45.45 total > > bitbake core-image-minimal -c cleansstate > bitbake core-image-minimal -c clean > bitbake core-image-minimal -f -c do_rootfs > 0.55s user 0.06s system 4% cpu 14.236 total > 0.54s user 0.08s system 4% cpu 15.247 total > 0.52s user 0.15s system 4% cpu 15.143 total > > RPM > === > bitbake core-image-minimal > 18.57s user 3.09s system 0% cpu 1:31:29.09 total > 18.58s user 3.08s system 0% cpu 1:30:53.80 total > 18.20s user 3.31s system 0% cpu 1:31:06.69 total > > bitbake core-image-minimal -c cleansstate > bitbake core-image-minimal -c clean > bitbake core-image-minimal -c do_rootfs -f > 0.58s user 0.10s system 3% cpu 19.470 total > 0.57s user 0.09s system 3% cpu 19.480 total > 0.60s user 0.07s system 3% cpu 20.381 total Some initial review comments incoming on all these patches. I've just done a surface level reading, I think the way you're submitting this still needs some work before it's in a state where we can properly review the changes in detail. Right now it's too big and too easy to get confused when looking through 26 large patches. I'm very supportive of tidying this up and making it more dynamic, we just need to make sure we don't break things along the way. Thanks, -- Paul Barker Konsulko Group -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#139938): https://lists.openembedded.org/g/openembedded-core/message/139938 Mute This Topic: https://lists.openembedded.org/mt/75057633/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] package bzip2-ptest requires /usr/bin/sh, but no providers found in RDEPENDS_bzip2-ptest
On Thu, 25 Jun 2020 at 08:04, Rahul Kumar wrote:
>
> Hi khem Raj,
>
> Create a patch to update.sample where it asks for /bin/sh instead
>
> I don't think we can create a patch to update.sample file because this is git
> hidden file.
>
> .git/hooks/update.sample
>
> In my Patch fetching bzip2-tests.git repository as mentioned below Is there
> any issue with below line.
> SRC_URI = "https://sourceware.org/pub/${BPN}/${BPN}-${PV}.tar.gz \
>git://sourceware.org/git/bzip2-tests.git;name=bzip2-tests \
>
> At my local system i am not able to face this issue but when this patch built
> with open source auto-builder then reporting below error
> do_package_qa: QA Issue:
> /usr/lib/bzip2/ptest/bzip2-tests/.git/hooks/update.sample contained in
> package bzip2-ptest requires /usr/bin/sh, but no providers found in
> RDEPENDS_bzip2-ptest? [file-rdeps]
> step1b: ERROR: bzip2-1.0.8-r0 do_package_qa: QA run found fatal errors.
> Please consider fixing them.
>
> I have stuck here. I am not aware why do_package_qa is reporting errors with
> git hidden files.
Do you need to install the whole git repository? Typically this issue
wouldn't be seen as .git would not be part of the installed package.
Perhaps instead of running `cp -rf $(srcdir)/../git
$(DESTDIR)/bzip2-tests` you need to copy a more specific subset of
files & directories.
--
Paul Barker
Konsulko Group
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#139904):
https://lists.openembedded.org/g/openembedded-core/message/139904
Mute This Topic: https://lists.openembedded.org/mt/75044252/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH 2/2] lib/oe: Split package manager code to multiple files
On Tue, 23 Jun 2020 at 13:12, Fredrik Gustafsson wrote: > > > > From: Richard Purdie > Sent: Tuesday, June 23, 2020 2:02 PM > To: Fredrik Gustafsson; openembedded-core@lists.openembedded.org > Cc: tools-cfpbuild-internal; Hugo Cedervall; Fredrik Gustafsson > Subject: Re: [OE-core] [PATCH 2/2] lib/oe: Split package manager code to > multiple files > > On Tue, 2020-06-23 at 13:13 +0200, Fredrik Gustafsson wrote: > > Today OE-Core has support for three package managers, even if there > > are many more package managers that could be interesting to use. > > Adding a new package manager to OE-Core would increase the test matrix > > significantly. In order to let users use their favorite package manager > > without need for it to have support from upstream, this patch refactors > > the package manager code so that it's easy to add a new package > > manager in another layer. > > > > This patch is mostly moving code and duplicating some code (because of > > the hard coupling between deb and ipk). > > > > Signed-off-by: Fredrik Gustafsson > > --- > > meta/classes/populate_sdk_base.bbclass| 11 +- > > meta/lib/oe/manifest.py | 149 +- > > meta/lib/oe/package_manager.py| 1402 + > > meta/lib/oe/package_managers/deb/manifest.py | 31 + > > .../package_managers/deb/package_manager.py | 719 + > > meta/lib/oe/package_managers/deb/rootfs.py| 213 +++ > > meta/lib/oe/package_managers/deb/sdk.py | 97 ++ > > .../ipk/.package_manager.py.swo | Bin 0 -> 45056 bytes > > meta/lib/oe/package_managers/ipk/manifest.py | 79 + > > .../package_managers/ipk/package_manager.py | 588 +++ > > meta/lib/oe/package_managers/ipk/rootfs.py| 395 + > > meta/lib/oe/package_managers/ipk/sdk.py | 104 ++ > > meta/lib/oe/package_managers/rpm/manifest.py | 62 + > > .../package_managers/rpm/package_manager.py | 423 + > > meta/lib/oe/package_managers/rpm/rootfs.py| 154 ++ > > meta/lib/oe/package_managers/rpm/sdk.py | 104 ++ > > meta/lib/oe/rootfs.py | 631 +--- > > meta/lib/oe/sdk.py| 311 +--- > > meta/lib/oeqa/utils/package_manager.py| 29 +- > > meta/recipes-core/meta/package-index.bb |2 +- > > 20 files changed, 3014 insertions(+), 2490 deletions(-) > > I'm afraid this patch is basically impossible to review. There are > warning signs like the inclusion of the binary swo file. The hint that > code gets duplicated also worries me, even if you abstract this, there > shouldn't be reason to do that. > > If we do this its going to have to be a more granular series where the > patches *just* move code to new locations so we can see/understand what > other changes are being made as I simply don't trust this patch, sorry > :(. > > Cheers, > > Richard > > > Hi, > sorry about the swo file that's a mistake on my side. The duplicated code is > simply so that deb and ipk shares a class. So my choice was to either > duplicate that class (and let them diverge which is fine) or in the core > package_manager.py file have a class that is not used by all package managers > (rpm doesn't use it). > > This patch is only code moves and class renames. I know it's very hard to > review, that's a real problem. Let me see if I can split it up some more. I really like the idea here but agree with Richard that the patch is impossible to review as is. I'd recommend moving things over in stages and keeping the package_manager.py file around until we're completely finished. Recipes like package-index.bb should not need to change at all - leave a shim layer in place with the old import path. I see you split the code into package_manager.py, rootfs.py, sdk.py and manifest.py for each package manager. That suggests you could start by just splitting out the rootfs code from each package manager, leaving the rest in package_manager.py. Then on the next commit move the sdk handling, etc. That would make each commit easier to reason about. I'd also recommend adding a __init__.py file in the ipk, rpm and deb directories so it's easier to import things when needed. Thanks, -- Paul Barker Konsulko Group -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#139832): https://lists.openembedded.org/g/openembedded-core/message/139832 Mute This Topic: https://lists.openembedded.org/mt/75057635/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH] toolchain-shar-relocate.sh: check for environment-setup beforehand
On Wed, 17 Jun 2020 at 21:48, Awais Belal wrote: > > The script runs a 'cat' on the script and if it isn't present in the > sdk the cat command waits on the std input and hence the installation > process simply sits there. That sort of error would typically be caused by the variable being unset rather than the variable being set to the path of a nonexistent file. I don't know much about the context this script runs in though so I may be missing something obvious. > > Signed-off-by: Awais Belal > --- > meta/files/toolchain-shar-relocate.sh | 6 ++ > 1 file changed, 6 insertions(+) > > diff --git a/meta/files/toolchain-shar-relocate.sh > b/meta/files/toolchain-shar-relocate.sh > index e3c10018ef..02a05664c6 100644 > --- a/meta/files/toolchain-shar-relocate.sh > +++ b/meta/files/toolchain-shar-relocate.sh > @@ -3,6 +3,12 @@ if ! xargs --version > /dev/null 2>&1; then > exit 1 > fi > > +# check if we have a valid env-setup script > +if [ ! -f "$env_setup_script" ]; then > + echo "Main environment-setup file not found. Abort!" > + exit 1 > +fi > + > # fix dynamic loader paths in all ELF SDK binaries > native_sysroot=$($SUDO_EXEC cat $env_setup_script |grep > 'OECORE_NATIVE_SYSROOT='|cut -d'=' -f2|tr -d '"') > dl_path=$($SUDO_EXEC find $native_sysroot/lib -name "ld-linux*") > -- > 2.17.1 -- Paul Barker Konsulko Group -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#139627): https://lists.openembedded.org/g/openembedded-core/message/139627 Mute This Topic: https://lists.openembedded.org/mt/74946187/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH v2 1/2] selftest: git-submodule-test: New recipe for testing a gitsm SRC_URI
On Fri, 12 Jun 2020 at 08:34, Richard Purdie
wrote:
>
> Hi Paul,
>
> On Thu, 2020-06-04 at 18:33 +0100, Paul Barker wrote:
> > Signed-off-by: Paul Barker
> > ---
> > .../recipes-test/git-submodule-test/git-submodule-test.bb | 8
> > 1 file changed, 8 insertions(+)
> > create mode 100644
> > meta-selftest/recipes-test/git-submodule-test/git-submodule-test.bb
> >
> > diff --git
> > a/meta-selftest/recipes-test/git-submodule-test/git-submodule-test.bb
> > b/meta-selftest/recipes-test/git-submodule-test/git-submodule-test.bb
> > new file mode 100644
> > index 00..08089b4186
> > --- /dev/null
> > +++ b/meta-selftest/recipes-test/git-submodule-test/git-submodule-test.bb
> > @@ -0,0 +1,8 @@
> > +SUMMARY = "Test recipe for fetching git submodules"
> > +LICENSE = "MIT"
> > +LIC_FILES_CHKSUM =
> > "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302"
> > +
> > +INHIBIT_DEFAULT_DEPS = "1"
> > +
> > +SRC_URI = "gitsm://git.yoctoproject.org/git-submodule-test"
> > +SRCREV = "a2885dd7d25380d23627e7544b7bbb55014b16ee"
>
> I didn't spot it at first but unfortunately this recipe is breaking the
> autobuilder. I think there is a problem with gitsm:// modules not
> mirroring correctly.
>
> Here you see the a-quick build run:
>
> bitbake universe -c fetch -k
> (meta-selftest is included and SOURCE_MIRROR_FETCH = '1' set)
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/85/builds/1055/steps/10/logs/step1b
>
> you do see do_fetch called for this recipe. Then later in the build it
> does:
>
> oe-selftest -r buildoptions.SourceMirroring.test_yocto_source_mirror
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/85/builds/1055/steps/12/logs/step1d
>
> The intent is that we check YP can build from its mirrors. The test is
> say it cannot.
>
> Any ideas on what might be wrong?
ovmf also uses git submodules and that seems to work. Also, I can see
the relevant mirror tarballs are present on
downloads.yoctoproject.org:
http://downloads.yoctoproject.org/mirror/sources/git2_git.openembedded.org.bitbake.tar.gz
http://downloads.yoctoproject.org/mirror/sources/git2_git.yoctoproject.org.bitbake-gitsm-test1.tar.gz
http://downloads.yoctoproject.org/mirror/sources/git2_git.yoctoproject.org.bitbake-gitsm-test2.tar.gz
http://downloads.yoctoproject.org/mirror/sources/git2_git.yoctoproject.org.git-submodule-test.tar.gz
So it's not immediately obvious what's wrong there. I'll run that
selftest case locally and see if I can reproduce the issue.
Thanks,
--
Paul Barker
Konsulko Group
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#139435):
https://lists.openembedded.org/g/openembedded-core/message/139435
Mute This Topic: https://lists.openembedded.org/mt/74676446/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] Changing kernel to the Mainline kernel ,linux-stable instead of linux-raspberrypi
On Thu, 11 Jun 2020 at 20:16, Pankaj Vinadrao Joshi wrote: > > > Thanks for response..You have provided the link ,But i want to build > specifically for 5.4.3 only ,where i should change in the existing file so > that i will be able to build the image for same since i am very new to yocto > can you help me out ?? Why do you want to use an old patch release in the 5.4 series? You should be using the latest patch release in a stable series so that you've got all the bug and security fixes. Thanks, -- Paul Barker Konsulko Group -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#139427): https://lists.openembedded.org/g/openembedded-core/message/139427 Mute This Topic: https://lists.openembedded.org/mt/74824292/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
