[OE-core] [RFC PATCH] kernel-yocto: Support in-tree config fragments
[This is currently an RFC as it depends on a separate patch to add
support for config fragments with .config file extensions in
yocto-kernel-tools:
https://lists.yoctoproject.org/g/linux-yocto/message/14431]
In Linux 6.8, the in-tree rv32_defconfig file for 32-bit RISC-V support
was dropped in favour of a Makefile target which combines the RISC-V
defconfig with a config fragment '32-bit.config'. So to build for 32-bit
RISC-V using the in-tree configuration from an upstream kernel, we need
to be able to enable this 32-bit config fragment.
We can support this via KERNEL_FEATURES if we add the in-tree arch
config directory (arch/${ARCH}/configs) to the search path.
While we're here, let's also add the generic config directory
(kernel/configs) to the search path so in-tree config fragments such as
'hardening.config' can be used.
Signed-off-by: Paul Barker
---
meta/classes-recipe/kernel-yocto.bbclass | 3 +++
1 file changed, 3 insertions(+)
diff --git a/meta/classes-recipe/kernel-yocto.bbclass
b/meta/classes-recipe/kernel-yocto.bbclass
index 6d5c3b6327b6..66602d359404 100644
--- a/meta/classes-recipe/kernel-yocto.bbclass
+++ b/meta/classes-recipe/kernel-yocto.bbclass
@@ -248,6 +248,9 @@ do_kernel_metadata() {
fi
done
+ # allow in-tree config fragments to be used in KERNEL_FEATURES
+ includes="$includes -I${S}/arch/${ARCH}/configs -I${S}/kernel/configs"
+
# expand kernel features into their full path equivalents
bsp_definition=$(spp ${includes} --find -DKMACHINE=${KMACHINE}
-DKTYPE=${LINUX_KERNEL_TYPE})
if [ -z "$bsp_definition" ]; then
--
2.39.5
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#205183):
https://lists.openembedded.org/g/openembedded-core/message/205183
Mute This Topic: https://lists.openembedded.org/mt/108776739/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [master/scarthgap PATCH] meta-ide-support: Mark recipe as MACHINE-specific
On 06/09/2024 14:48, Richard Purdie wrote:
> On Thu, 2024-09-05 at 23:40 +0100, Richard Purdie via lists.openembedded.org
> wrote:
>> On Thu, 2024-09-05 at 21:21 +0100, Paul Barker via lists.openembedded.org
>> wrote:
>>> meta-ide-support:do_write_test_data dumps the bitbake data dictionary to
>>> a file using export2json(). As this obviously includes the value of
>>> MACHINE, and other MACHINE-specific variables, the recipe needs to be
>>> marked as MACHINE-specific.
>>>
>>> Signed-off-by: Paul Barker
>>> ---
>>> meta/recipes-core/meta/meta-ide-support.bb | 1 +
>>> 1 file changed, 1 insertion(+)
>>>
>>> diff --git a/meta/recipes-core/meta/meta-ide-support.bb
>>> b/meta/recipes-core/meta/meta-ide-support.bb
>>> index d85aa120c08b..7ed422ce4ddb 100644
>>> --- a/meta/recipes-core/meta/meta-ide-support.bb
>>> +++ b/meta/recipes-core/meta/meta-ide-support.bb
>>> @@ -1,6 +1,7 @@
>>> SUMMARY = "Integrated Development Environment support"
>>> DESCRIPTION = "Meta package for ensuring the build directory contains all
>>> appropriate toolchain packages for using an IDE"
>>> LICENSE = "MIT"
>>> +PACKAGE_ARCH = "${MACHINE_ARCH}"
>>>
>>> DEPENDS = "virtual/libc gdb-cross-${TARGET_ARCH} qemu-native
>>> qemu-helper-native unfs3-native cmake-native autoconf-native
>>> automake-native meson-native intltool-native pkgconfig-native"
>>> RM_WORK_EXCLUDE += "${PN}"
>>
>> Something about this is making our toolchain tests unhappy, e.g.:
>>
>> https://valkyrie.yoctoproject.org/#/builders/42/builds/86/steps/12/logs/stdio
>>
>> I've not looked into it at all, just reporting what we've need to fix
>> in order to be able to consider merging this.
>
> It can be fixed with:
>
> diff --git a/meta/lib/oeqa/selftest/cases/meta_ide.py
> b/meta/lib/oeqa/selftest/cases/meta_ide.py
> index ffe0d2604d1..5a17ca52ea1 100644
> --- a/meta/lib/oeqa/selftest/cases/meta_ide.py
> +++ b/meta/lib/oeqa/selftest/cases/meta_ide.py
> @@ -20,8 +20,8 @@ class MetaIDE(OESelftestTestCase):
> bitbake('meta-ide-support')
> bitbake('build-sysroots -c build_native_sysroot')
> bitbake('build-sysroots -c build_target_sysroot')
> -bb_vars = get_bb_vars(['MULTIMACH_TARGET_SYS', 'DEPLOY_DIR_IMAGE',
> 'COREBASE'])
> -cls.environment_script = 'environment-setup-%s' %
> bb_vars['MULTIMACH_TARGET_SYS']
> +bb_vars = get_bb_vars(['MACHINE_ARCH', 'TARGET_VENDOR', 'TARGET_OS',
> 'DEPLOY_DIR_IMAGE', 'COREBASE'])
> +cls.environment_script = 'environment-setup-%s%s-%s' %
> (bb_vars['MACHINE_ARCH'], bb_vars['TARGET_VENDOR'], bb_vars['TARGET_OS'])
> cls.deploydir = bb_vars['DEPLOY_DIR_IMAGE']
> cls.environment_script_path = '%s/%s' % (cls.deploydir,
> cls.environment_script)
> cls.corebasedir = bb_vars['COREBASE']
>
> Whether that is a good/right fix, I'm not sure...
>
> Cheers,
>
> Richard
>
Hi Richard, Steve,
I see this has been committed as 3be2bc8a9b0c in openembedded-core [1]
for master/Styhead. Thanks for fixing the test case and merging this.
We saw the same issue when running yocto-check-layer in Scarthgap, do
you think this is suitable for backporting?
[1]:
https://git.openembedded.org/openembedded-core/commit/?id=3be2bc8a9b0c9d6a178329c8b451a6bedf255d6c
Thanks,
--
Paul Barker
OpenPGP_0x27F4B3459F002257.asc
Description: OpenPGP public key
OpenPGP_signature.asc
Description: OpenPGP digital signature
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#205113):
https://lists.openembedded.org/g/openembedded-core/message/205113
Mute This Topic: https://lists.openembedded.org/mt/108292901/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [master/scarthgap PATCH] meta-ide-support: Mark recipe as MACHINE-specific
meta-ide-support:do_write_test_data dumps the bitbake data dictionary to
a file using export2json(). As this obviously includes the value of
MACHINE, and other MACHINE-specific variables, the recipe needs to be
marked as MACHINE-specific.
Signed-off-by: Paul Barker
---
meta/recipes-core/meta/meta-ide-support.bb | 1 +
1 file changed, 1 insertion(+)
diff --git a/meta/recipes-core/meta/meta-ide-support.bb
b/meta/recipes-core/meta/meta-ide-support.bb
index d85aa120c08b..7ed422ce4ddb 100644
--- a/meta/recipes-core/meta/meta-ide-support.bb
+++ b/meta/recipes-core/meta/meta-ide-support.bb
@@ -1,6 +1,7 @@
SUMMARY = "Integrated Development Environment support"
DESCRIPTION = "Meta package for ensuring the build directory contains all
appropriate toolchain packages for using an IDE"
LICENSE = "MIT"
+PACKAGE_ARCH = "${MACHINE_ARCH}"
DEPENDS = "virtual/libc gdb-cross-${TARGET_ARCH} qemu-native
qemu-helper-native unfs3-native cmake-native autoconf-native automake-native
meson-native intltool-native pkgconfig-native"
RM_WORK_EXCLUDE += "${PN}"
--
2.43.0
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#204243):
https://lists.openembedded.org/g/openembedded-core/message/204243
Mute This Topic: https://lists.openembedded.org/mt/108292901/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH 1/1] uboot-sign: fix counters in do_uboot_assemble_fitimage
Without unsetting `j` and `k` for each `UBOOT_MACHINE`, `j` and `k`
are incremented in the same frequency as `i` and therefore `$j -eq $i`
and `$k -eq $i` is always true for the first `type` from `UBOOT_CONFIG`
and the first `binary` from `UBOOT_BINARIES`.
Signed-off-by: Paul Gerber
---
meta/classes-recipe/uboot-sign.bbclass | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/meta/classes-recipe/uboot-sign.bbclass
b/meta/classes-recipe/uboot-sign.bbclass
index c8e097f2f2..3e5f5dcf66 100644
--- a/meta/classes-recipe/uboot-sign.bbclass
+++ b/meta/classes-recipe/uboot-sign.bbclass
@@ -356,8 +356,9 @@ do_uboot_assemble_fitimage() {
fi
if [ -n "${UBOOT_CONFIG}" ]; then
- unset i j k
+ unset i
for config in ${UBOOT_MACHINE}; do
+ unset j k
i=$(expr $i + 1);
for type in ${UBOOT_CONFIG}; do
j=$(expr $j + 1);
--
2.44.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#203910):
https://lists.openembedded.org/g/openembedded-core/message/203910
Mute This Topic: https://lists.openembedded.org/mt/108158713/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH v2 0/1] classes: add new retain class for retaining build results
classes: add new retain class for retaining build results Add a fairly simple class to tar up the work directory on task failure to a configurable location - useful when the build environment is ephemeral (e.g. an autobuilder/build pipeline). The directories that are retained can be configured, and you can also choose to save directories on every build as well if desired. [This is a significant reworking of a prototype I sent as an RFC back in 2020 (AKA "workdir_save"). Changes since then include deferring tarball creation until the end, more config flexibility, and the addition of test cases.] Changes since v1: * Use ;prefix= instead of : to specify tarball prefix, to be consistent with other similar usages and avoid potential confusion with overrides * Use prefix of "workdir" for default work directory tarballs * Fix dir name within tarball to not have .tar.gz extension * oe-selftest: fix test_retain_misc to be deterministic (should solve failures seen on the autobuilder) and tweak tests for robustness in a few places * oe-selftest: add detail to failure messages in test_retain_misc and test_retain_failure The following changes since commit 662396533177b72cc1d83e95841b27f7e42dcb20: create-spdx-3.0/populate_sdk_base: Add SDK_CLASSES inherit mechanism to fix tarball SPDX manifests (2024-07-26 12:26:24 +0100) are available in the git repository at: https://git.yoctoproject.org/poky-contrib paule/retain https://git.yoctoproject.org/poky-contrib/log/?h=paule/retain Paul Eggleton (1): classes: add new retain class for retaining build results meta/classes-global/retain.bbclass | 182 + meta/lib/oeqa/selftest/cases/retain.py | 241 + 2 files changed, 423 insertions(+) create mode 100644 meta/classes-global/retain.bbclass create mode 100644 meta/lib/oeqa/selftest/cases/retain.py -- 1.8.3.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#202722): https://lists.openembedded.org/g/openembedded-core/message/202722 Mute This Topic: https://lists.openembedded.org/mt/107659949/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH v2 1/1] classes: add new retain class for retaining build results
From: Paul Eggleton
If you are running your builds inside an environment where you don't
have access to the build tree (e.g. an autobuilder where you can only
download final artifacts such as images), then debugging build failures
can be difficult - you can't examine log files, the source tree or
output files. When enabled, by default this class will retain the work
directory for any recipe that has a task failure in the form of a
tarball, and can also be configured to save other directories on failure
or always.
It puts these tarballs in a configurable location (${TMPDIR}/retained by
default), where they can be picked up by a separate process and made
available as downloadable artifacts.
Signed-off-by: Paul Eggleton
---
meta/classes-global/retain.bbclass | 182 +
meta/lib/oeqa/selftest/cases/retain.py | 241 +
2 files changed, 423 insertions(+)
create mode 100644 meta/classes-global/retain.bbclass
create mode 100644 meta/lib/oeqa/selftest/cases/retain.py
diff --git a/meta/classes-global/retain.bbclass
b/meta/classes-global/retain.bbclass
new file mode 100644
index 000..46e8c25
--- /dev/null
+++ b/meta/classes-global/retain.bbclass
@@ -0,0 +1,182 @@
+# Creates a tarball of the work directory for a recipe when one of its
+# tasks fails, or any other nominated directories.
+# Useful in cases where the environment in which builds are run is
+# ephemeral or otherwise inaccessible for examination during
+# debugging.
+#
+# To enable, simply add the following to your configuration:
+#
+# INHERIT += "retain"
+#
+# You can specify the recipe-specific directories to save upon failure
+# or always (space-separated) e.g.:
+#
+# RETAIN_DIRS_FAILURE = "${WORKDIR};prefix=workdir"# default
+# RETAIN_DIRS_ALWAYS = "${T}"
+#
+# Naturally you can use overrides to limit it to a specific recipe:
+# RETAIN_DIRS_ALWAYS:pn-somerecipe = "${T}"
+#
+# You can also specify global (non-recipe-specific) directories to save:
+#
+# RETAIN_DIRS_GLOBAL_FAILURE = "${LOG_DIR}"
+# RETAIN_DIRS_GLOBAL_ALWAYS = "${BUILDSTATS_BASE}"
+#
+# If you wish to use a different tarball name prefix than the default of
+# the directory name, you can do so by specifying a ;prefix= followed by
+# the desired prefix (no spaces) in any of the RETAIN_DIRS_* variables.
+# e.g. to always save the log files with a "recipelogs" as the prefix for
+# the tarball of ${T} you would do this:
+#
+# RETAIN_DIRS_ALWAYS = "${T};prefix=recipelogs"
+#
+# Notes:
+# * For this to be useful you also need corresponding logic in your build
+# orchestration tool to pick up any files written out to RETAIN_OUTDIR
+# (with the other assumption being that no files are present there at
+# the start of the build, since there is no logic to purge old files).
+# * Work directories can be quite large, so saving them can take some time
+# and of course space.
+# * Tarball creation is deferred to the end of the build, thus you will
+# get the state at the end, not immediately upon failure.
+# * Extra directories must naturally be populated at the time the retain
+# class goes to save them (build completion); to try ensure this for
+# things that are also saved on build completion (e.g. buildstats), put
+# the INHERIT += "retain" after the INHERIT += lines for the class that
+# is writing out the data that you wish to save.
+# * The tarballs have the tarball name as a top-level directory so that
+# multiple tarballs can be extracted side-by-side easily.
+#
+# Copyright (c) 2020, 2024 Microsoft Corporation
+#
+# SPDX-License-Identifier: GPL-2.0-only
+#
+
+RETAIN_OUTDIR ?= "${TMPDIR}/retained"
+RETAIN_DIRS_FAILURE ?= "${WORKDIR};prefix=workdir"
+RETAIN_DIRS_ALWAYS ?= ""
+RETAIN_DIRS_GLOBAL_FAILURE ?= ""
+RETAIN_DIRS_GLOBAL_ALWAYS ?= ""
+RETAIN_TARBALL_SUFFIX ?= "${DATETIME}.tar.gz"
+RETAIN_ENABLED ?= "1"
+
+
+def retain_retain_dir(desc, tarprefix, path, tarbasepath, d):
+import datetime
+
+outdir = d.getVar('RETAIN_OUTDIR')
+bb.utils.mkdirhier(outdir)
+suffix = d.getVar('RETAIN_TARBALL_SUFFIX')
+tarname = '%s_%s' % (tarprefix, suffix)
+tarfp = os.path.join(outdir, '%s' % tarname)
+tardir = os.path.relpath(path, tarbasepath)
+cmdargs = ['tar', 'cfa', tarfp]
+# Prefix paths within the tarball with the tarball name so that
+# multiple tarballs can be extracted side-by-side
+tarname_noext = os.path.splitext(tarname)[0]
+if tarname_noext.endswith('.tar'):
+tarname_noext = tarname_noext[:-4]
+cmdargs += ['--transform', 's:^:%s/:' % tarname_noext]
+cmdargs += [tardir]
+try:
+bb.process.run(cmdargs, cwd=tarbasepath)
+except bb.process.ExecutionError as e:
+
[OE-core] [PATCH 0/1] classes: add new retain class for retaining build results
Add a fairly simple class to tar up the work directory on task failure to a configurable location - useful when the build environment is ephemeral (e.g. an autobuilder/build pipeline). The directories that are retained can be configured, and you can also choose to save directories on every build as well if desired. [This is a significant reworking of a prototype I sent as an RFC back in 2020 (AKA "workdir_save"). Changes since then include deferring tarball creation until the end, more config flexibility, and the addition of test cases.] The following changes since commit 662396533177b72cc1d83e95841b27f7e42dcb20: create-spdx-3.0/populate_sdk_base: Add SDK_CLASSES inherit mechanism to fix tarball SPDX manifests (2024-07-26 12:26:24 +0100) are available in the git repository at: https://git.yoctoproject.org/poky-contrib paule/retain https://git.yoctoproject.org/poky-contrib/log/?h=paule/retain (should have been openembedded-core-contrib I know, but for some reason I wasn't able to push there.) Paul Eggleton (1): classes: add new retain class for retaining build results meta/classes-global/retain.bbclass | 162 meta/lib/oeqa/selftest/cases/retain.py | 219 + 2 files changed, 381 insertions(+) create mode 100644 meta/classes-global/retain.bbclass create mode 100644 meta/lib/oeqa/selftest/cases/retain.py -- 1.8.3.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#202573): https://lists.openembedded.org/g/openembedded-core/message/202573 Mute This Topic: https://lists.openembedded.org/mt/107600987/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH 1/1] classes: add new retain class for retaining build results
From: Paul Eggleton
If you are running your builds inside an environment where you don't
have access to the build tree (e.g. an autobuilder where you can only
download final artifacts such as images), then debugging build failures
can be difficult - you can't examine log files, the source tree or
output files. When enabled, by default this class will retain the work
directory for any recipe that has a task failure in the form of a
tarball, and can also be configured to save other directories on failure
or always.
It puts these tarballs in a configurable location (${TMPDIR}/retained by
default), where they can be picked up by a separate process and made
available as downloadable artifacts.
Signed-off-by: Paul Eggleton
---
meta/classes-global/retain.bbclass | 162
meta/lib/oeqa/selftest/cases/retain.py | 219 +
2 files changed, 381 insertions(+)
create mode 100644 meta/classes-global/retain.bbclass
create mode 100644 meta/lib/oeqa/selftest/cases/retain.py
diff --git a/meta/classes-global/retain.bbclass
b/meta/classes-global/retain.bbclass
new file mode 100644
index 000..f71c6a2
--- /dev/null
+++ b/meta/classes-global/retain.bbclass
@@ -0,0 +1,162 @@
+# Creates a tarball of the work directory for a recipe when one of its
+# tasks fails, or any other nominated directories.
+# Useful in cases where the environment in which builds are run is
+# ephemeral or otherwise inaccessible for examination during
+# debugging.
+#
+# To enable, simply add the following to your configuration:
+#
+# INHERIT += "retain"
+#
+# You can specify the recipe-specific directories to save upon failure
+# or always (space-separated) e.g.:
+#
+# RETAIN_DIRS_FAILURE = "${WORKDIR}"# default
+# RETAIN_DIRS_ALWAYS = "${T}"
+#
+# Naturally you can use overrides to limit it to a specific recipe:
+# RETAIN_DIRS_ALWAYS:pn-somerecipe = "${T}"
+#
+# You can also specify global (non-recipe-specific) directories to save:
+#
+# RETAIN_DIRS_GLOBAL_FAILURE = "${LOG_DIR}"
+# RETAIN_DIRS_GLOBAL_ALWAYS = "${BUILDSTATS_BASE}"
+#
+# If you wish to use a different tarball name prefix you can do so by
+# adding a : followed by the desired prefix (no spaces) in any of the
+# RETAIN_DIRS variables. e.g. to use "buildlogs" as the prefix for the
+# tarball of ${T} you would do this:
+#
+# RETAIN_DIRS_FAILURE = "${T}:buildlogs"
+#
+# Notes:
+# * For this to be useful you also need corresponding logic in your build
+# orchestration tool to pick up any files written out to RETAIN_OUTDIR
+# (with the other assumption being that no files are present there at
+# the start of the build, since there is no logic to purge old files).
+# * Work directories can be quite large, so saving them can take some time
+# and of course space.
+# * Tarball creation is deferred to the end of the build, thus you will
+# get the state at the end, not immediately upon failure.
+# * Extra directories must naturally be populated at the time the retain
+# class goes to save them (build completion); to try ensure this for
+# things that are also saved on build completion (e.g. buildstats), put
+# the INHERIT += "retain" after the INHERIT += lines for the class that
+# is writing out the data that you wish to save.
+# * The tarballs have the tarball name as a top-level directory so that
+# multiple tarballs can be extracted side-by-side easily.
+#
+# Copyright (c) 2020, 2024 Microsoft Corporation
+#
+# SPDX-License-Identifier: GPL-2.0-only
+#
+
+RETAIN_OUTDIR ?= "${TMPDIR}/retained"
+RETAIN_DIRS_FAILURE ?= "${WORKDIR}"
+RETAIN_DIRS_ALWAYS ?= ""
+RETAIN_DIRS_GLOBAL_FAILURE ?= ""
+RETAIN_DIRS_GLOBAL_ALWAYS ?= ""
+RETAIN_TARBALL_SUFFIX ?= "${DATETIME}.tar.gz"
+RETAIN_ENABLED ?= "1"
+
+
+def retain_retain_dir(desc, tarprefix, path, tarbasepath, d):
+import datetime
+
+outdir = d.getVar('RETAIN_OUTDIR')
+bb.utils.mkdirhier(outdir)
+suffix = d.getVar('RETAIN_TARBALL_SUFFIX')
+tarname = '%s_%s' % (tarprefix, suffix)
+tarfp = os.path.join(outdir, '%s' % tarname)
+tardir = os.path.relpath(path, tarbasepath)
+cmdargs = ['tar', 'cfa', tarfp]
+# Prefix paths within the tarball with the tarball name so that
+# multiple tarballs can be extracted side-by-side
+cmdargs += ['--transform', 's:^:%s/:' % tarname]
+cmdargs += [tardir]
+try:
+bb.process.run(cmdargs, cwd=tarbasepath)
+except bb.process.ExecutionError as e:
+# It is possible for other tasks to be writing to the workdir
+# while we are tarring it up, in which case tar will return 1,
+# but we don't care in this situation (tar returns 2 for other
+# errors so we we will see those)
+if e.exitcode !
Re: [OE-core] [Openembedded-architecture] New mailing list for layer patches
On 28/03/2024 14:36, Ross Burton wrote: > Hi, > > At the moment if a layer doesn’t have enough volume to justify a dedicated > mailing list for patches (see, for example, meta-...@lists.yoctoproject.org > <mailto:meta-...@lists.yoctoproject.org>) then the convention is that the > patches can be sent to yo...@lists.yoctoproject.org > <mailto:yo...@lists.yoctoproject.org>. > > However, that’s also the list that we encourage users of Yocto to use for > help, so the list has a mix of patches for various layers and people asking > for help, which isn’t ideal: users who want help don’t want to be flooded > with patches for layers they’re not using, layer maintainers may not want to > see the requests for help, and people who want the status reports have to > receive patches and support. > > The Yocto TSC has been discussing this, and as a solution to this we’ve just > created a new mailing list: yocto-patc...@lists.yoctoproject.org > <mailto:yocto-patc...@lists.yoctoproject.org>. This list is specifically for > Yocto-related projects which don’t have their own mailing list or other patch > submission process. We ask that all layers currently using yocto@ for > patches move to yocto-patches@, so that yocto@ can be used purely for > discussion and other non-patch related subjects. I think this is a great change, along with the new yocto-status@ list it will make things easier to follow :) Do you know if yocto-patches@ will be available via lore.kernel.org? Thanks, -- Paul Barker OpenPGP_0x27F4B3459F002257.asc Description: OpenPGP public key OpenPGP_signature.asc Description: OpenPGP digital signature -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#197868): https://lists.openembedded.org/g/openembedded-core/message/197868 Mute This Topic: https://lists.openembedded.org/mt/105284126/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH] kernel: Fix check_oldest_kernel
The check_oldest_kernel() function requires utsrelease.h to be
generated. This file is generated during do_compile, so we need to delay
calling check_oldest_kernel() until after this.
With this change in place, I now see the expected warning when building
Linux 5.10.y.
Signed-off-by: Paul Barker
---
meta/classes-recipe/kernel.bbclass | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/meta/classes-recipe/kernel.bbclass
b/meta/classes-recipe/kernel.bbclass
index db4461e551e4..b084d6d69dc1 100644
--- a/meta/classes-recipe/kernel.bbclass
+++ b/meta/classes-recipe/kernel.bbclass
@@ -645,7 +645,7 @@ python check_oldest_kernel() {
}
check_oldest_kernel[vardepsexclude] += "OLDEST_KERNEL KERNEL_VERSION"
-do_configure[prefuncs] += "check_oldest_kernel"
+do_compile[postfuncs] += "check_oldest_kernel"
KERNEL_LOCALVERSION ??= ""
--
2.44.0
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#197052):
https://lists.openembedded.org/g/openembedded-core/message/197052
Mute This Topic: https://lists.openembedded.org/mt/104910570/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [Openembedded-architecture] [RFC PATCH] Add genericarm64 MACHINE using upstream defconfig
On 21/02/2024 10:57, Ross Burton wrote: > From: Ross Burton > > This is a new 64-bit "generic" Arm machine, that expects the hardware to > be SystemReady IR compatible. This is slightly forward-leaning as there's > not a _lot_ of SystemReady hardware in the wild, but most modern boards > are and the number will only grow. Also, this is the only way to have a > 'generic' machine as without standardised bootloaders and firmware it > would be impossible. > > The base machine configuration isn't that exciting: it's a fully featured > machine that supports most things, booting via UEFI and an initramfs. > > However, the kernel is more interesting. This RFC uses the upstream defconfig > because unlike some other platforms, the arm64 defconfig is actively > maintained with the goal of being a 'boots on most hardware' configuration. > My argument is: why would we duplicate that effort? > > The "linux-yocto way" is configuration fragments and after a week of > hair-pulling I do actually have fragments that boot on a BeaglePlay, but > to say this was a tiresome and frustrating exercise would be understating it. > > So, a request for comments: is it acceptable to use the upstream defconfig in > a reference BSP? Personally I'm torn: the Yocto way is fragments not > monolithic > configs, but repeating the effort to fragmentise the configuration and then > also have it sufficiently modular that it can be used in pieces - instead of > just being a large file split up into smaller files - is a lot of effort for > what might end up being minimal gain. My fear is we end up with a fragmented > configuration that can't be easily modified without breaking some platforms, > and badly copies what the defconfig already does. I am in favour of this - I think the "genericarm64" machine should use the in-tree defconfig so that it can support the widest array of hardware. If someone wants to trim down the kernel for a particular platform then they should probably create a specific MACHINE anyway. If we take the other approach of building up the kernel config from fragments, how would we know that all SystemReady IR capable systems will be supported? Yocto Project doesn't have the resources to test every platform. For the Renesas RZ SoCs I work on these days, the in-tree defconfig is the configuration we test with the mainline kernel. Thanks, -- Paul Barker OpenPGP_0x27F4B3459F002257.asc Description: OpenPGP public key OpenPGP_signature.asc Description: OpenPGP digital signature -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#195979): https://lists.openembedded.org/g/openembedded-core/message/195979 Mute This Topic: https://lists.openembedded.org/mt/104489784/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] Yocto Project Status 07 November 2023 (WW45)
On Tue, Nov 07, 2023 at 10:52:53AM -0500, Neal Caidin wrote: >The Yocto Project website has had a refresh (www.yoctoproject.org) The new website looks great! Just a quick note - the links at the bottom of the page are mostly going to https://yocto.dev-lfprojects3.linuxfoundation.org/. E.g. "Technical Overview" goes to https://yocto.dev-lfprojects3.linuxfoundation.org/development/technical-overview/ Thanks, Paul signature.asc Description: PGP signature -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#190309): https://lists.openembedded.org/g/openembedded-core/message/190309 Mute This Topic: https://lists.openembedded.org/mt/102445104/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] Summary of the remaining 6.5 kernel serial issue (and 6.5 summary)
So, here is an update for those not following on IRC: [Re: [OE-core] Summary of the remaining 6.5 kernel serial issue (and 6.5 summary)] On 11/10/2023 (Wed 00:50) Paul Gortmaker wrote: [snip details of getting RP's chunk of AB python working as a 1% reproducer] > Next steps: > > Each run takes approximately 20s, and with a 1.5% success factor, a > minimum of 150 clean runs is needed before one can say "OK, *maybe* this > kernel currently under test is OK. That translates to over an hour to > get a data point you have some faith in as being "good". [Still a heck > of a lot better than doing 150 *builds* though!] > > On the other hand, just one instance of missing login on ttyS1 is > concretely "bad" and hence the test run can be stopped as soon as one > instance of that is seen. > > This "good is really maybe perhaps good, but bad is for sure bad" type > of bisects that I love to hate. > > Since the data points are "expensive", I probably won't just attempt a > mindless brute force bisect. I'd probably do something more like... > > --ensure modules (or lack of) don't impact the results, so I don't have > to rebundle rootfs images. This turned out to be true - didn't need modules. > --maybe see if I can build with host gcc vs. using devshell/sysroot gcc > and still see the issue. Although really, kernel build time optimization > won't be the bottleneck in this particular case, so not worthwhile? I didn't bother with the above, since build time was not significant here. > --check vanilla v6.5 (i.e. v6.5.0 -- no linux-stable content) so I know > whether to dig around in v6.5.0 --> v6.5.5, or dig around in the > much much larger v6.4 --> v6.5 additions. I expect it to be the > latter, i.e. digging through mainline, but who knows. As expected, it followed v6.5.0 and linux-stable isn't to blame here. > --assuming mainline introduced it, instead of a brute force bisect, I'd > look for merges by Linus from gregKH pull requests for the tty and > serial subsystems in the merge window (v6.4 ---> v6.5-rc1) and test > both sides of that/those merge(s). This also largely went according to plan. I started with the tty merge: v6.5-rc1~74: 868a9fd94807 Merge tag 'tty-6.5-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/tty and did burrow down into the 60-odd commits within. One complication was that there were interim mainline merges into the dev stream: $ git log --oneline --merges v6.5-rc1~74^2 ^v6.5-rc1~74^1 2076b2a4a6b7 Merge 6.4-rc5 into tty-next 0e4daea31d83 Merge 6.4-rc3 into tty-next ..which usually indicate a dependency on work from another subsystem. Fortunately the test pass/fail boundary didn't land on one of those. > --if things haven't completely fallen apart by that point, I'd be left > with a relatively small set of subsystem specific commits from a single > pull request which then could be bisected. We did somehow manage to get here with just 60 commits to consider. > > Sometimes in the past, when I've got down to that small set of commits, > the culprit almost jumps out at you when scanning over the shortlogs. While I didn't immediately spot the commit via the shortlog, extensive test iterations did give us confidence in pinpointing a single commit: 84a9582fd203 serial: core: Start managing serial controllers to enable runtime PM *** < FAIL (33/1436) ae62c49c0cef serial: 8250: omap: convert to modern PM ops < PASS (0/1593) > Anyway, that is where my thoughts are at. But of course we don't even > yet know if this "test" is even accurately reflecting the OEQA/AB issue! > So nobody should get their hopes up too early. I think now, we can say with a solid degree of confidence that the PM commit gets to own the responsibility of causing the ttyS1 getty not flushing the login banner and prompt. Hence why I did over 1000 runs on both the PASS and FAIL commit boundary. Unfortunately, there is a lot in that commit ; also a lot in the discussion and the twelve iterations before acceptance that I've not even looked at yet: https://lore.kernel.org/linux-serial/1431f5b4-fb39-483b-9314-ed2b7c118...@gmail.com/T/#t My "next steps" at this point would be to try and attempt to break down that relatively large commit into multiple "sub-commits" in order to use the AB test harness portion to better get an idea of which change component is the root cause of the ttyS1 getty stall/seizure. I won't get to that until Saturday at the earliest - going AFK after sending this. However, Mikko already opened a dialog with the linux-serial folks a few days ago: https://lore.kernel
Re: [OE-core] Summary of the remaining 6.5 kernel serial issue (and 6.5 summary)
n can be stopped as soon as one instance of that is seen. This "good is really maybe perhaps good, but bad is for sure bad" type of bisects that I love to hate. Since the data points are "expensive", I probably won't just attempt a mindless brute force bisect. I'd probably do something more like... --ensure modules (or lack of) don't impact the results, so I don't have to rebundle rootfs images. --maybe see if I can build with host gcc vs. using devshell/sysroot gcc and still see the issue. Although really, kernel build time optimization won't be the bottleneck in this particular case, so not worthwhile? --check vanilla v6.5 (i.e. v6.5.0 -- no linux-stable content) so I know whether to dig around in v6.5.0 --> v6.5.5, or dig around in the much much larger v6.4 --> v6.5 additions. I expect it to be the latter, i.e. digging through mainline, but who knows. --assuming mainline introduced it, instead of a brute force bisect, I'd look for merges by Linus from gregKH pull requests for the tty and serial subsystems in the merge window (v6.4 ---> v6.5-rc1) and test both sides of that/those merge(s). --if things haven't completely fallen apart by that point, I'd be left with a relatively small set of subsystem specific commits from a single pull request which then could be bisected. Sometimes in the past, when I've got down to that small set of commits, the culprit almost jumps out at you when scanning over the shortlogs. Anyway, that is where my thoughts are at. But of course we don't even yet know if this "test" is even accurately reflecting the OEQA/AB issue! So nobody should get their hopes up too early. As a side note, and something that kind of was briefly discussed on IRC, it seems to make sense to someday informally "support" the idea of developers being able to use a chunk of the OEQA like this in a stand-alone mode - since we always seem to be chasing issues that only show up in AB output - with people struggling to reproduce elsewhere. Paul. -- -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#188941): https://lists.openembedded.org/g/openembedded-core/message/188941 Mute This Topic: https://lists.openembedded.org/mt/101824562/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [dunfell][PATCH RESEND] kernel.bbclass: Add force flag to rm calls
From: Ryan Eatmon
The latest 6.5 kernels do not appear to create the source file in
${D}${nonarch_base_libdir}/modules/${KERNEL_VERSION}/source so the
recipe errors out when trying to remove it. Simple fix is to add the
-f (force) flag to the call.
(From OE-Core rev: 2e669bf797b15d803e7d6a700e449bdc467a4bcc)
Signed-off-by: Ryan Eatmon
Signed-off-by: Richard Purdie
Signed-off-by: Alexander Sverdlin
Signed-off-by: Steve Sakoman
Backported from kirkstone commit 7e177848f97e.
Signed-off-by: Paul Barker
---
Re-sending this for the mailing list. My first send was dropped as I
wasn't subscribed to the list from this email address.
Yes, I do crazy things like building mainline kernels on dunfell.
This fix is unlikely to have any negative effects though, so I think
it's good to apply to dunfell.
meta/classes/kernel.bbclass | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index ba5b6cf384a0..ca7530095e12 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -405,8 +405,8 @@ kernel_do_install() {
unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS MACHINE
if (grep -q -i -e '^CONFIG_MODULES=y$' .config); then
oe_runmake DEPMOD=echo
MODLIB=${D}${nonarch_base_libdir}/modules/${KERNEL_VERSION}
INSTALL_FW_PATH=${D}${nonarch_base_libdir}/firmware modules_install
- rm "${D}${nonarch_base_libdir}/modules/${KERNEL_VERSION}/build"
- rm "${D}${nonarch_base_libdir}/modules/${KERNEL_VERSION}/source"
+ rm -f
"${D}${nonarch_base_libdir}/modules/${KERNEL_VERSION}/build"
+ rm -f
"${D}${nonarch_base_libdir}/modules/${KERNEL_VERSION}/source"
# If the kernel/ directory is empty remove it to prevent QA
issues
rmdir --ignore-fail-on-non-empty
"${D}${nonarch_base_libdir}/modules/${KERNEL_VERSION}/kernel"
else
--
2.39.2
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#188872):
https://lists.openembedded.org/g/openembedded-core/message/188872
Mute This Topic: https://lists.openembedded.org/mt/101870886/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core][mickledore 00/18] Patch review
[Re: [OE-core][mickledore 00/18] Patch review] On 17/08/2023 (Thu 23:17) Paul
Gortmaker via lists.openembedded.org wrote:
> [Re: [OE-core][mickledore 00/18] Patch review] On 16/08/2023 (Wed 07:50)
> Steve Sakoman via lists.openembedded.org wrote:
>
> > On Tue, Aug 15, 2023 at 6:24???AM Steve Sakoman via
> > lists.openembedded.org
> > wrote:
> > >
> > > Please review this set of changes for mickledore and have comments back by
> > > end of day Thursday, August 17.
>
> [...]
>
> > > Bruce Ashfield (2):
> > > linux-yocto/6.1: update to v6.1.41
> > > linux-yocto/6.1: update to v6.1.43
> >
> > I'm seeing intermittent failures with these 6.1 "stable" version bumps
> > (issues are also seen in master).
> >
> > So I am dropping these for now till we can fix the problem.
>
> aka "NOHZ tick-stop error: local softirq work is pending, handler #80!!!"
>
> For those not on IRC, the "cause" was tracked down to a v6.1.39 change:
>
> https://git.kernel.org/pub/scm/linux/kernel/git/stable/stable-queue.git/tree/releases/6.1.39/tick-rcu-fix-bogus-ratelimit-condition.patch
>
> [linux-stable commit ID 77cc52f1b8d7 in v6.1]
>
> The bad news - when you look at that commit, you realize the softirq-80
> problem has been there for an as-yet undetermined length of time, but
> was never output because of the bogus/broken "less than 10" check.
>
> The "less than 10" was introduced in v5.18, so that sets a lower bound,
> but I've not yet tested if the pending-80 was there immediately at 5.18
> or appeared somewhere between v5.18 and v6.1 - stay tuned.
For the mail archives - it turns out the issue was there immediately at
v5.18, based on my testing results. - I'm optimistic (for once) that we
have this softirq-80 issue solved. But we'll have to wait and see.
If folks are interested, more details are in the commit log here:
https://lore.kernel.org/lkml/20230818200757.1808398-1-paul.gortma...@windriver.com/
Paul.
--
>
> The same change/issue 1st appears in v6.4.4 stable kernel [7fe63c29cb].
>
> Paul.
> --
>
> >
> > I reran a-full without the version bumps and all tests passed:
> >
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/5742
> >
> > Steve
> >
> > > Dmitry Baryshkov (1):
> > > linux-firmware: split platform-specific Adreno shaders to separate
> > > packages
> > >
> > > Joel Stanley (1):
> > > kernel: don't fail if Modules.symvers doesn't exist
> > >
> > > Marek Vasut (1):
> > > linux-firmware: Fix mediatek mt7601u firmware path
> > >
> > > Mark Hatle (1):
> > > tcf-agent: Update to 1.8.0 release
> > >
> > > Richard Purdie (1):
> > > oeqa/ssh: Further improve process exit handling
> > >
> > > Ross Burton (1):
> > > openssh: upgrade to 9.3p2
> > >
> > > Sudip Mukherjee (1):
> > > bind: upgrade to v9.18.17
> > >
> > > Yogita Urade (3):
> > > qemu: fix CVE-2023-3301
> > > qemu: fix CVE-2023-3255
> > > qemu: fix CVE-2023-2861
> > >
> > > sanjana (1):
> > > binutils: stable 2.40 branch updates
> > >
> > > meta/classes-recipe/kernel.bbclass| 4 +-
> > > meta/conf/distro/include/maintainers.inc | 2 +-
> > > meta/lib/oeqa/core/target/ssh.py | 5 +-
> > > ...1-avoid-start-failure-with-bind-user.patch | 0
> > > ...d-V-and-start-log-hide-build-options.patch | 0
> > > ...ching-for-json-headers-searches-sysr.patch | 0
> > > .../bind/{bind-9.18.16 => bind}/bind9 | 0
> > > .../bind/{bind-9.18.16 => bind}/conf.patch| 0
> > > .../generate-rndc-key.sh | 0
> > > ...t.d-add-support-for-read-only-rootfs.patch | 0
> > > .../make-etc-initd-bind-stop-work.patch | 0
> > > .../bind/{bind-9.18.16 => bind}/named.service | 0
> > > .../bind/{bind_9.18.16.bb => bind_9.18.17.bb} | 4 +-
> > > .../{openssh_9.3p1.bb => openssh_9.3p2.bb}| 2 +-
> > > meta/recipes-core/glibc/glibc-locale.inc | 8 +-
> > > .../binutils/binutils-2.40.inc| 2 +-
> > > meta/recipes-devtools/qemu/qemu.inc | 3 +
> > > .../qemu/qemu/CVE-2023-2861.patch | 171 ++
> > > .../qemu/qemu/CVE-2023-3255.patch | 65 +++
> > > .../qemu/qemu/CVE-2023-3301.patch
Re: [OE-core] Dilemma on changes - merge or not to merge (e.g. 6.4)
[Re: [OE-core] Dilemma on changes - merge or not to merge (e.g. 6.4)] On
16/08/2023 (Wed 09:55) Rasmus Villemoes wrote:
> On 15/08/2023 15.08, Paul Gortmaker via lists.openembedded.org wrote:
> > [Dilemma on changes - merge or not to merge (e.g. 6.4)] On 14/08/2023 (Mon
> > 10:54) Richard Purdie wrote:
> >
> >> Remaining are:
> >> * an error upon boot on preempt-rt on qemux86-64
> >> (e.g.
> >> https://autobuilder.yoctoproject.org/typhoon/#/builders/72/builds/7616/steps/36/logs/stdio)
> >> We'll probably just have to ignore it in parselogs as it has been??
> >> around for a while and nobody seems interested in fixing it upstream.
> >
> > Just back from vacation and I see an internal report of 10-ish at boot
>
> it seems to be rate-limited to 10 per boot, so it should never appear
> more than those 10ish times:
>
> static bool report_idle_softirq(void)
> {
> ...
> if (ratelimit >= 10)
> return false;
> ...
> ratelimit++;
> ...
Amusingly enough - you were looking right at the problem. Just a few
stable kernels ago, it was inadvertently ratelimited to zero. :-P
https://lists.openembedded.org/g/openembedded-core/message/186343
Paul.
--
> }
>
> (it's all non-atomic/lockfree, so ofc it could just happen to get
> emitted 11 or 12 times if the stars align just right...)
>
> >
> > NOHZ tick-stop error: local softirq work is pending, handler #80!!!
> >
> > ..on the 6.1.43-rt10-yocto-preempt-rt kernel, on real hardware. So it
> > seems we can't blame that one entirely on v6.4 kernel (or qemu).
> >
> > We used to get (late 3.x and 4.x era) pretty common "NOHZ: local softirq
> > pending" messages even on common/popular distro kernels. But I haven't
> > seen those for a long time and they didn't scream "error" or have the
> > alarmist three exclamation marks either.
>
> FWIW, we're also seeing exactly that "NOHZ tick-stop error" message on
> 6.4.6-rt8 running on a couple of different imx8mp based boards.
>
> Rasmus
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#186344):
https://lists.openembedded.org/g/openembedded-core/message/186344
Mute This Topic: https://lists.openembedded.org/mt/100733646/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core][mickledore 00/18] Patch review
[Re: [OE-core][mickledore 00/18] Patch review] On 16/08/2023 (Wed 07:50) Steve
Sakoman via lists.openembedded.org wrote:
> On Tue, Aug 15, 2023 at 6:24???AM Steve Sakoman via
> lists.openembedded.org
> wrote:
> >
> > Please review this set of changes for mickledore and have comments back by
> > end of day Thursday, August 17.
[...]
> > Bruce Ashfield (2):
> > linux-yocto/6.1: update to v6.1.41
> > linux-yocto/6.1: update to v6.1.43
>
> I'm seeing intermittent failures with these 6.1 "stable" version bumps
> (issues are also seen in master).
>
> So I am dropping these for now till we can fix the problem.
aka "NOHZ tick-stop error: local softirq work is pending, handler #80!!!"
For those not on IRC, the "cause" was tracked down to a v6.1.39 change:
https://git.kernel.org/pub/scm/linux/kernel/git/stable/stable-queue.git/tree/releases/6.1.39/tick-rcu-fix-bogus-ratelimit-condition.patch
[linux-stable commit ID 77cc52f1b8d7 in v6.1]
The bad news - when you look at that commit, you realize the softirq-80
problem has been there for an as-yet undetermined length of time, but
was never output because of the bogus/broken "less than 10" check.
The "less than 10" was introduced in v5.18, so that sets a lower bound,
but I've not yet tested if the pending-80 was there immediately at 5.18
or appeared somewhere between v5.18 and v6.1 - stay tuned.
The same change/issue 1st appears in v6.4.4 stable kernel [7fe63c29cb].
Paul.
--
>
> I reran a-full without the version bumps and all tests passed:
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/5742
>
> Steve
>
> > Dmitry Baryshkov (1):
> > linux-firmware: split platform-specific Adreno shaders to separate
> > packages
> >
> > Joel Stanley (1):
> > kernel: don't fail if Modules.symvers doesn't exist
> >
> > Marek Vasut (1):
> > linux-firmware: Fix mediatek mt7601u firmware path
> >
> > Mark Hatle (1):
> > tcf-agent: Update to 1.8.0 release
> >
> > Richard Purdie (1):
> > oeqa/ssh: Further improve process exit handling
> >
> > Ross Burton (1):
> > openssh: upgrade to 9.3p2
> >
> > Sudip Mukherjee (1):
> > bind: upgrade to v9.18.17
> >
> > Yogita Urade (3):
> > qemu: fix CVE-2023-3301
> > qemu: fix CVE-2023-3255
> > qemu: fix CVE-2023-2861
> >
> > sanjana (1):
> > binutils: stable 2.40 branch updates
> >
> > meta/classes-recipe/kernel.bbclass| 4 +-
> > meta/conf/distro/include/maintainers.inc | 2 +-
> > meta/lib/oeqa/core/target/ssh.py | 5 +-
> > ...1-avoid-start-failure-with-bind-user.patch | 0
> > ...d-V-and-start-log-hide-build-options.patch | 0
> > ...ching-for-json-headers-searches-sysr.patch | 0
> > .../bind/{bind-9.18.16 => bind}/bind9 | 0
> > .../bind/{bind-9.18.16 => bind}/conf.patch| 0
> > .../generate-rndc-key.sh | 0
> > ...t.d-add-support-for-read-only-rootfs.patch | 0
> > .../make-etc-initd-bind-stop-work.patch | 0
> > .../bind/{bind-9.18.16 => bind}/named.service | 0
> > .../bind/{bind_9.18.16.bb => bind_9.18.17.bb} | 4 +-
> > .../{openssh_9.3p1.bb => openssh_9.3p2.bb}| 2 +-
> > meta/recipes-core/glibc/glibc-locale.inc | 8 +-
> > .../binutils/binutils-2.40.inc| 2 +-
> > meta/recipes-devtools/qemu/qemu.inc | 3 +
> > .../qemu/qemu/CVE-2023-2861.patch | 171 ++
> > .../qemu/qemu/CVE-2023-3255.patch | 65 +++
> > .../qemu/qemu/CVE-2023-3301.patch | 65 +++
> > .../tcf-agent/tcf-agent_git.bb| 4 +-
> > .../linux-firmware/linux-firmware_20230625.bb | 28 ++-
> > .../linux/linux-yocto-rt_6.1.bb | 6 +-
> > .../linux/linux-yocto-tiny_6.1.bb | 6 +-
> > meta/recipes-kernel/linux/linux-yocto_6.1.bb | 28 +--
> > meta/recipes-support/curl/curl/disable-tests | 2 +
> > meta/recipes-support/curl/curl/run-ptest | 2 +-
> > scripts/lib/devtool/upgrade.py| 2 +
> > 28 files changed, 366 insertions(+), 43 deletions(-)
> > rename meta/recipes-connectivity/bind/{bind-9.18.16 =>
> > bind}/0001-avoid-start-failure-with-bind-user.patch (100%)
> > rename meta/recipes-connectivity/bind/{bind-9.18.16 =>
> > bind}/0001-named-lwresd-V-and-start-log-hide-build-options.patch (100%)
> > rename meta/recipes-connectivity/bind/{bind-9.18.16 =>
> > bind}/bind-ensure-searching-
Re: [OE-core] Dilemma on changes - merge or not to merge (e.g. 6.4)
[Dilemma on changes - merge or not to merge (e.g. 6.4)] On 14/08/2023 (Mon 10:54) Richard Purdie wrote: > I'm becoming a little weary/wary of some of the changes that are coming > in. The challenge is that once they merge, issues become the problem of > a very small number of people. > > My current dilemma is the 6.4 kernel. People would like it, we'd really > ideally use it for the next release but there are issues. > > I've worked through a few, at least pinning down where the issues were > then resolving them with the help of others (thanks Bruce, Jon, Ross). > > Remaining are: > * an error upon boot on preempt-rt on qemux86-64 > (e.g. > https://autobuilder.yoctoproject.org/typhoon/#/builders/72/builds/7616/steps/36/logs/stdio) > We'll probably just have to ignore it in parselogs as it has been?? > around for a while and nobody seems interested in fixing it upstream. Just back from vacation and I see an internal report of 10-ish at boot NOHZ tick-stop error: local softirq work is pending, handler #80!!! ..on the 6.1.43-rt10-yocto-preempt-rt kernel, on real hardware. So it seems we can't blame that one entirely on v6.4 kernel (or qemu). We used to get (late 3.x and 4.x era) pretty common "NOHZ: local softirq pending" messages even on common/popular distro kernels. But I haven't seen those for a long time and they didn't scream "error" or have the alarmist three exclamation marks either. I'll see if I can dig into that further. This instance is new to me, so any additional context or information I might not turn up myself would be useful. > * some random hangs: > > https://autobuilder.yoctoproject.org/typhoon/#/builders/148/builds/349/steps/12/logs/stdio > > https://autobuilder.yoctoproject.org/typhoon/#/builders/148/builds/354/steps/12/logs/stdio > > The latter are rare and intermittent, mainly taking out CI test builds. > Most people aren't affected by them, find them hard to reproduce let > alone fix and will ignore them. That will leave me/Bruce/PaulG holding > the pieces. Ugh. The RCU one is ugly and the Silent Boot Death one is no better. Nobody likes SBD cases. They suck. > > I know Bruce spends a ton of time debugging weird things just to get > the kernel to the point we can even consider merging and nobody ever > really sees or appreciates that work :(. Well, not "nobody". There are at least two people who have a good idea of what Bruce does. :-P Paul. -- > > Systemd was a similar challenge recently, multiple patches causing > multiple issues with a significant impact on CI. In that case the > issues weren't intermittent so resolution wasn't so bad. > > Rust and reproducibility??was given a pass so the rest of the changes > could merge for it. That just meant there was less pressure and the > reproducibility issue is still there with people saying its too hard. > That issue is now spreading down the chain to other recipes. > > The toolchain test reports have thousands of failures nobody is really > looking at. Similarly the now consistent ltp controllers failures > (previously the reports weren't even consistent!). > > I'm worried the access control patches changing the tar format are > going to destablise and once merged, people will move on to other > things leaving any remaining intermittent issues to me. Already we're > seeing things like sstate being blamed as it is easiest to do that. I > end up having to "prove" it isn't that. > > There are intermittent ptests on the autobuilder too. I took mdadm > ptest patches on the basis there was help to fix them. We are still see > a lot of failures in CI from there. The glib-networking intermittent > failures continue, I know Trevor has tried to dig into those but he is > alone in doing it in code which isn't easy to navigate (and I don't > know how to help there). > > As an idea of impact, every time one of these things fails in CI, > someone has triage that failure. The bug triage team has to triage the > bugs too. > > I don't know how we fix this but we really could do with more people > able to dive in and help with these intermittent issues. I'm really > really apprehensive about merging some patches as I can just tell > they're going to cause pain :(. > > Cheers, > > Richard > -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#186063): https://lists.openembedded.org/g/openembedded-core/message/186063 Mute This Topic: https://lists.openembedded.org/mt/100733646/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH 1/2] qemux86/conf: TEST ONLY. Drop smp boot
[Re: [PATCH 1/2] qemux86/conf: TEST ONLY. Drop smp boot] On 09/06/2023 (Fri 10:08) Bruce Ashfield wrote: > On Fri, Jun 9, 2023 at 2:06???AM Richard Purdie > wrote: > > [...] > > FWIW, this patch didn't change much. With it applied we still see: > > > ok. so at least we can rule out it being specific to secondary cpu startup. I'll own some of that. I was looking forward in the dmesg where secondary CPU init was, and ignoring looking backwards. I have a new thought. I still could be completely off base. I updated the case with my theory. https://bugzilla.yoctoproject.org/show_bug.cgi?id=15138 Might totally be another wild goose chase. Who knows... Paul. -- -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#182614): https://lists.openembedded.org/g/openembedded-core/message/182614 Mute This Topic: https://lists.openembedded.org/mt/99414662/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH] scripts: fix buildstats diff/summary hard bound to host python3
Somehow these two got left behind and hence on older hosts that are using buildtools for a newer python - they will still fail. Signed-off-by: Paul Gortmaker diff --git a/scripts/buildstats-diff b/scripts/buildstats-diff index 2f6498ab6743..c9aa76a8fafc 100755 --- a/scripts/buildstats-diff +++ b/scripts/buildstats-diff @@ -1,4 +1,4 @@ -#!/usr/bin/python3 +#!/usr/bin/env python3 # # Script for comparing buildstats from two different builds # diff --git a/scripts/buildstats-summary b/scripts/buildstats-summary index f521d78650b4..b10c671b29bd 100755 --- a/scripts/buildstats-summary +++ b/scripts/buildstats-summary @@ -1,4 +1,4 @@ -#! /usr/bin/python3 +#!/usr/bin/env python3 # # Dump a summary of the specified buildstats to the terminal, filtering and # sorting by walltime. -- 2.37.3 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#180832): https://lists.openembedded.org/g/openembedded-core/message/180832 Mute This Topic: https://lists.openembedded.org/mt/98665140/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCHv3] python3-manifest.json: Fix re in core
Hi Leon
On Friday, 4 November 2022 01:27:55 NZST Leon Anavi wrote:
> Following the update from 3.10.6 to 3.11.0, the source code of
> module re for regular expression operations has been moved to
> Lib/re/. File python3-manifest.json has been modified according
> to the new path.
>
> According to the instructions the new python3-manifest.json
> version was created by running bitbake python3 -c create_manifest.
> Because of this it also updated other files and rdepends in it.
Maybe you're already aware, but just in case not, this patch appears to have
effectively undone the change you made earlier ("Move urllib to netclient"):
https://git.openembedded.org/openembedded-core/commit/?id=e74ab2c144042b0a9f788b1b15f11b2800e2c7a1
Cheers
Paul
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#180234):
https://lists.openembedded.org/g/openembedded-core/message/180234
Mute This Topic: https://lists.openembedded.org/mt/94755089/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH 09/14] image-artifact-names: add IMAGE_MACHINE_SUFFIX variable
Hi Martin
On Tuesday, 14 March 2023 01:15:36 NZST Martin Jansa wrote:
> * to make it easier for projects to avoid default -${MACHINE} suffix if
> the ${MACHINE} named DEPLOY_DIR_IMAGE works better for them
>
> * also use IMAGE_LINK_NAME in IMAGE_NAME to make it more clear
> that IMAGE_NAME is the same as IMAGE_LINK_NAME but with version
> suffix
>
> * adding it as separate variable helps us to catch the cases
> where we didn't respect ${IMAGE_LINK_NAME} variable and just used
> the common default ${IMAGE_BASENAME}-${MACHINE}.
>
> [YOCTO #12937]
>
> Signed-off-by: Martin Jansa
> ---
> meta/classes-recipe/image-artifact-names.bbclass | 15 ---
> meta/classes-recipe/kernel-artifact-names.bbclass | 2 +-
> 2 files changed, 13 insertions(+), 4 deletions(-)
>
> diff --git a/meta/classes-recipe/image-artifact-names.bbclass
> b/meta/classes-recipe/image-artifact-names.bbclass index
> 9dc25b6dde..ac2376d59a 100644
> --- a/meta/classes-recipe/image-artifact-names.bbclass
> +++ b/meta/classes-recipe/image-artifact-names.bbclass
> @@ -11,11 +11,20 @@
> IMAGE_BASENAME ?= "${PN}"
> IMAGE_VERSION_SUFFIX ?= "-${DATETIME}"
> IMAGE_VERSION_SUFFIX[vardepsexclude] += "DATETIME SOURCE_DATE_EPOCH"
> -IMAGE_NAME ?= "${IMAGE_BASENAME}-${MACHINE}${IMAGE_VERSION_SUFFIX}"
> -IMAGE_LINK_NAME ?= "${IMAGE_BASENAME}-${MACHINE}"
> +IMAGE_NAME ?= "${IMAGE_LINK_NAME}${IMAGE_VERSION_SUFFIX}"
> +IMAGE_LINK_NAME ?= "${IMAGE_BASENAME}${IMAGE_MACHINE_SUFFIX}"
So there's a minor unfortunate side-effect of this in that you can no longer
just set IMAGE_LINK_NAME = "" to drop the symlinks - a minority use case, but
something I have used. Of course you can just re-set the value of IMAGE_NAME.
I am making a note in the manual and migration guide.
Cheers
Paul
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#180204):
https://lists.openembedded.org/g/openembedded-core/message/180204
Mute This Topic: https://lists.openembedded.org/mt/97578959/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH] kernel: improve initramfs bundle processing time
[[OE-core] [PATCH] kernel: improve initramfs bundle processing time] On
14/04/2023 (Fri 15:29) Bruce Ashfield via lists.openembedded.org wrote:
> From: Bruce Ashfield
>
> This is a partial fix for bugzilla 15059
> [https://bugzilla.yoctoproject.org/show_bug.cgi?id=15059]
>
> It has been noted by several people that when an initramfs is bundled:
>
> - a lot of the kernel is rebuilt
> - it takes a really long time
>
> When looking at the logs, the second kernel compilation (that performs
> the bundle) is not using the parallel make settings, and builds with
> -j1.
>
> We are already explicitly passing PARALLEL_MAKE when building kernel
> modules, and by extending that explicit use to the main kernel
> compilation, we ensure that we always get a parallel build.
>
> Build times chnaged from more than 30 minutes for the bundle, to
> 3 minutes in local testing.
I've seen this a bunch of times in the stuff I'm working on, and the
incremental fix to go from 30m to 3m is super welcome. Thanks.
> The question of whether or not too much is rebuilding during the
> bundle step is still an open question, but with this tweak, at least
> the build time is back in the realm of acceptable.
Thank you for the incremental fix. If I can help with the
over-rebuilding aspect, I will. Let me know.
Paul.
--
>
> Signed-off-by: Bruce Ashfield
> ---
> meta/classes-recipe/kernel.bbclass | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/meta/classes-recipe/kernel.bbclass
> b/meta/classes-recipe/kernel.bbclass
> index aefa0d21bc..e634eabd49 100644
> --- a/meta/classes-recipe/kernel.bbclass
> +++ b/meta/classes-recipe/kernel.bbclass
> @@ -382,7 +382,7 @@ kernel_do_compile() {
>
> use_alternate_initrd=CONFIG_INITRAMFS_SOURCE=${B}/usr/${INITRAMFS_IMAGE_NAME}.cpio
> fi
> for typeformake in ${KERNEL_IMAGETYPE_FOR_MAKE} ; do
> - oe_runmake ${typeformake} ${KERNEL_EXTRA_ARGS}
> $use_alternate_initrd
> + oe_runmake ${PARALLEL_MAKE} ${typeformake} ${KERNEL_EXTRA_ARGS}
> $use_alternate_initrd
> done
> }
>
> --
> 2.25.1
>
>
>
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#180112):
https://lists.openembedded.org/g/openembedded-core/message/180112
Mute This Topic: https://lists.openembedded.org/mt/98269754/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] mesa-native fails on master
[Re: [OE-core] mesa-native fails on master] On 04/04/2023 (Tue 02:39) Chen Qi via lists.openembedded.org wrote: > I just met the same issue. My host is ubuntu18, gcc version is 7.5.0. Note that ubuntu-18.04 is End-of-Life in another three weeks (unless you buy additional support) - in case people weren't keeping track. The ubuntu-20.04 uses gcc-9.4.0 by default. Paul. -- > > Regards, > Qi > > -Original Message- > From: openembedded-core@lists.openembedded.org > On Behalf Of Mark Hatle > Sent: Tuesday, April 4, 2023 10:25 AM > To: Patches and discussions about the oe-core layer > > Subject: [OE-core] mesa-native fails on master > > It's been a few weeks since I last built master, but starting with today's > pull I'm getting errors building mesa-native -- which pretty much kills the > whole build. > > I'm on an Ubuntu 18.04 host, and have the 'buildtools-tarball' from the last > 4.1.3 loaded so I can get to the minimum version of python to run a build. > > (I tried with buildtools-tarball-extended and that DOES appear to work. If > this is expected, you can ignore the rest of this. But I expected the system > to complain before trying to build everything if the compiler wasn't new > enough.) > > > The error is: > > brw_simd_selection.cpp:205:7: sorry, unimplemented: non-trivial designated > initializers not supported > > > The host gcc is 7.5.0 > > What is strange is I had 2 successful builds, and then it started failing. > All > fresh builds. I've no idea why one was successful and the others failed. > I'm > wondering if maybe there is some sort of race in mesa, but lowered my > parallel > build (-j) to 1 and I'm still getting it. > > > Reproducer: > > git clone https://git.yoctoproject.org/poky > > cd poky > > wget > http://downloads.yoctoproject.org/releases/yocto/yocto-4.1.3/buildtools/x86_64-buildtools-nativesdk-standalone-4.1.3.sh > > bash x86_64-buildtools-nativesdk-standalone-4.1.3.sh -d buildtools -y > > . ./buildtools/environment-... > > . ./oe-init-build-env > > bitbake mesa-native > > > > > -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#179691): https://lists.openembedded.org/g/openembedded-core/message/179691 Mute This Topic: https://lists.openembedded.org/mt/98052337/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] create-spdx and sstate
Hi Joshua / all We've been having an issue with the create-spdx class if we share sstate between two configurations - one where gcc-cross- has a dependency and one where it doesn't (specifically, one where the abicheck class in meta- binaryaudit is inherited and the other where it isn't; that influences DEPENDS). The result is that if you build the configuration with the dependency then the one where it doesn't (in separate build dirs with the same sstate cache), image_combine_spdx fails because it can't find the SPDX data file for the dependency as it was not built in the second configuration. It seems that create-spdx looks at BB_TASKDEPDATA to get dependencies and then adds BB_TASKDEPDATA to vardepsexclude, thus the dependencies changing does not cause the task to be re-executed. However, I assume a variable dependency on BB_TASKDEPDATA might be impractical, thus why it was excluded in the first place. Do we instead add an explicit dependency on DEPENDS? I'm happy to come up with a patch if we can determine what the correct fix is. (FWIW we're still using dunfell, but I don't see any changes in master that alter this particular behaviour.) Thanks Paul -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#175549): https://lists.openembedded.org/g/openembedded-core/message/175549 Mute This Topic: https://lists.openembedded.org/mt/96080598/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [RFC] [PATCH] Allow fitimage + initramfs rebuild to be accelerated
Hi Richard
On Thursday, 10 November 2022 11:14:24 NZDT Richard Purdie wrote:
> We have tried really hard to avoid putting all of the kernel build
> output into sstate. The main reason for that was that recompiling the
> kernel from source took around the same amount of time as compressing
> it and moving it around in sstate, it was huge.
>
> If we're going to start putting all the kernel bits into sstate, we
> would probably rethink kernel-devsrc and a few other pieces too. I'm
> not sure we should do that but they're all connected. Perhaps we do
> give in and do that but I doubt people will like the build times or
> sstate size increases :(.
Right, and I had avoided putting too much into sstate with this patch -
basically just vmlinux and the contents of arch/${ARCH}/boot (via the perhaps
a bit broadly named KERNEL_OUTPUT_DIR) - possibly even that could be trimmed a
bit. For our case building the kernel takes a significant time because (long
story short) there are actually two kernels building (debug and regular
flavours) and there are a number of items that depend upon them -> they also
get rebuilt if we don't untangle this a little bit.
I'm now wondering if part of the solution here would be to move some of the fit
image + initramfs assembly to the initramfs image context. That would mean
that do_deploy would need to save away everything necessary to do that, but
that shouldn't be a huge amount of data.
> I'd also observe that we don't have good tests for a lot of these
> different use cases. The code has slowly been turning into an if/else
> labyrinth and it is very unclear which usecases are being used by
> people. We have seen a rise in test cases and I have pushed for them
> where I can but the situation isn't great and is a big worry whenever
> we make changes in here.
I'm all for adding additional tests, certainly, and I'm happy to at least some
of that work. I'd need to get a better understanding of some of the other use
cases though.
> Adding in yet further if/else paths with magic variables to control
> them isn't filling me with confidence.
I understand that. I was hoping to figure out a way to avoid adding significant
extra complexity.
> The recent work Sean Anderson did on fitimage with u-boot looked like a
> promising de-cluttering of some of the maze.
Indeed.
Paul
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#173104):
https://lists.openembedded.org/g/openembedded-core/message/173104
Mute This Topic: https://lists.openembedded.org/mt/94747626/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [RFC] [PATCH] Allow fitimage + initramfs rebuild to be accelerated
On Thursday, 10 November 2022 08:53:02 NZDT Martin Jansa wrote: > https://bugzilla.yoctoproject.org/show_bug.cgi?id=12937 might be > alternative solution which we at LGE were using for different usecase than > fit image, but in theory should work as well. > > Basically the initramfs do_deploy could be without version and then the > hardlink with datetime (or any other version) is created in fast > do_deploy_links task. Unless I've missed something I don't think this can help - in our case the content of the initramfs is changing, not just the filename due to the version. Cheers Paul -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#173103): https://lists.openembedded.org/g/openembedded-core/message/173103 Mute This Topic: https://lists.openembedded.org/mt/94747626/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [RFC] [PATCH] Allow fitimage + initramfs rebuild to be accelerated
Hi folks
Any comments? Alternative approaches?
Thanks
Paul
- original message -
Subject: [OE-core] [RFC] [PATCH] Allow fitimage + initramfs rebuild to be
accelerated
Date: Thursday, 3 November 2022, 13:23:51 NZDT
From: Paul Eggleton
To: openembedded-core@lists.openembedded.org
We have a usecase where our initramfs changes on every build (since it
contains a version number which is based on the date), and we're
incorporating that into a fitImage. Most of the time our builds are done
from shared state and we want them to be as efficient as possible.
Currently, when the initramfs signature is different this setup requires
the kernel to be recompiled on every build, which is not ideal. I've
come up with the following rough patch to address this, which does the
following:
* Avoid the dependency of do_bundle_initramfs on ${INITRAMFS_IMAGE} if
we are going to be building that for the fit image later anyway
* Add a do_preserve_kernel_build task to preserve the files needed to
re-run do_assemble_fitimage_initramfs in a separate location from
${B}, which can be saved to sstate
* Add a do_deploy_fitimage task to deploy just the files written out by
do_assemble_fitimage_initramfs to ${DEPLOY_DIR_IMAGE}; this allows us
to preserve less in sstate than we would have to if we allowed
do_deploy to rerun. Of course the downside is we then need to add a
dependency from the image's do_build on the kernel's
do_deploy_fitimage, and I had to introduce a variable to control that
which is a bit awkward.
This patch is quite rough and I'd like to take suggestions on how to
improve it. One of my challenges is I don't have a complete grasp of all
the usage modes supported by this code so I'd appreciate folks who do
taking a look.
Thanks!
Paul
---
meta/classes-recipe/image.bbclass | 5 +--
meta/classes-recipe/kernel-fitimage.bbclass | 55 -
meta/classes-recipe/kernel.bbclass | 2 +-
3 files changed, 51 insertions(+), 11 deletions(-)
diff --git a/meta/classes-recipe/image.bbclass
b/meta/classes-recipe/image.bbclass
index e387645..cedd913 100644
--- a/meta/classes-recipe/image.bbclass
+++ b/meta/classes-recipe/image.bbclass
@@ -137,13 +137,14 @@ def rootfs_variables(d):
do_rootfs[vardeps] += "${@rootfs_variables(d)}"
+IMAGE_FITIMAGE_DEPEND = "${@'virtual/kernel:do_deploy_fitimage' if
d.getVar('IMAGE_FITIMAGE') == '1' else ''}"
+
# This is needed to have kernel image in DEPLOY_DIR.
# This follows many common usecases and user expectations.
# But if you are building an image which doesn't need the kernel image at all,
# you can unset this variable manually.
KERNEL_DEPLOY_DEPEND ?= "virtual/kernel:do_deploy"
-do_build[depends] += "${KERNEL_DEPLOY_DEPEND}"
-
+do_build[depends] += "${KERNEL_DEPLOY_DEPEND} ${IMAGE_FITIMAGE_DEPEND}"
python () {
def extraimage_getdepends(task):
diff --git a/meta/classes-recipe/kernel-fitimage.bbclass
b/meta/classes-recipe/kernel-fitimage.bbclass
index 7980910..40406fc 100644
--- a/meta/classes-recipe/kernel-fitimage.bbclass
+++ b/meta/classes-recipe/kernel-fitimage.bbclass
@@ -731,10 +731,32 @@ do_install:append() {
fi
}
+python do_preserve_kernel_build() {
+archdir = d.getVar('KERNEL_OUTPUT_DIR')
+src = d.getVar('B')
+dst = d.getVar('INITRAMFSDIR')
+oe.path.copyhardlinktree(os.path.join(src, archdir), os.path.join(dst,
archdir))
+for fn in ['vmlinux', 'fit-image.its']:
+oe.path.copyhardlink(os.path.join(src, fn), os.path.join(dst, fn))
+}
+
+INITRAMFSDIR = "${WORKDIR}/initramfs-work"
+SSTATETASKS += "do_preserve_kernel_build"
+do_preserve_kernel_build[cleandirs] = "${INITRAMFSDIR}"
+do_preserve_kernel_build[sstate-plaindirs] = "${INITRAMFSDIR}"
+
+python do_preserve_kernel_build_setscene () {
+sstate_setscene(d)
+}
+addtask preserve_kernel_build_setscene
+
+
+addtask preserve_kernel_build after do_bundle_initramfs
+
do_assemble_fitimage_initramfs() {
if echo ${KERNEL_IMAGETYPES} | grep -wq "fitImage" && \
test -n "${INITRAMFS_IMAGE}" ; then
- cd ${B}
+ cd ${INITRAMFSDIR}
if [ "${INITRAMFS_IMAGE_BUNDLE}" = "1" ]; then
fitimage_assemble fit-image-${INITRAMFS_IMAGE}.its
fitImage-bundle ""
ln -sf fitImage-bundle
${B}/${KERNEL_OUTPUT_DIR}/fitImage
@@ -744,7 +766,9 @@ do_assemble_fitimage_initramfs() {
fi
}
-addtask assemble_fitimage_initramfs before do_deploy after do_bundle_initramfs
+addtask assemble_fitimage_initramfs after do_preserve_kernel_build
+
+do_assemble_fitimage_initramfs[depends] +=
"virtual/${TARGET_PREFIX}binutils:do_populate_sysroot"
[OE-core] [RFC] [PATCH] Allow fitimage + initramfs rebuild to be accelerated
From: Paul Eggleton
We have a usecase where our initramfs changes on every build (since it
contains a version number which is based on the date), and we're
incorporating that into a fitImage. Most of the time our builds are done
from shared state and we want them to be as efficient as possible.
Currently, when the initramfs signature is different this setup requires
the kernel to be recompiled on every build, which is not ideal. I've
come up with the following rough patch to address this, which does the
following:
* Avoid the dependency of do_bundle_initramfs on ${INITRAMFS_IMAGE} if
we are going to be building that for the fit image later anyway
* Add a do_preserve_kernel_build task to preserve the files needed to
re-run do_assemble_fitimage_initramfs in a separate location from
${B}, which can be saved to sstate
* Add a do_deploy_fitimage task to deploy just the files written out by
do_assemble_fitimage_initramfs to ${DEPLOY_DIR_IMAGE}; this allows us
to preserve less in sstate than we would have to if we allowed
do_deploy to rerun. Of course the downside is we then need to add a
dependency from the image's do_build on the kernel's
do_deploy_fitimage, and I had to introduce a variable to control that
which is a bit awkward.
This patch is quite rough and I'd like to take suggestions on how to
improve it. One of my challenges is I don't have a complete grasp of all
the usage modes supported by this code so I'd appreciate folks who do
taking a look.
Thanks!
Paul
---
meta/classes-recipe/image.bbclass | 5 +--
meta/classes-recipe/kernel-fitimage.bbclass | 55 -
meta/classes-recipe/kernel.bbclass | 2 +-
3 files changed, 51 insertions(+), 11 deletions(-)
diff --git a/meta/classes-recipe/image.bbclass
b/meta/classes-recipe/image.bbclass
index e387645..cedd913 100644
--- a/meta/classes-recipe/image.bbclass
+++ b/meta/classes-recipe/image.bbclass
@@ -137,13 +137,14 @@ def rootfs_variables(d):
do_rootfs[vardeps] += "${@rootfs_variables(d)}"
+IMAGE_FITIMAGE_DEPEND = "${@'virtual/kernel:do_deploy_fitimage' if
d.getVar('IMAGE_FITIMAGE') == '1' else ''}"
+
# This is needed to have kernel image in DEPLOY_DIR.
# This follows many common usecases and user expectations.
# But if you are building an image which doesn't need the kernel image at all,
# you can unset this variable manually.
KERNEL_DEPLOY_DEPEND ?= "virtual/kernel:do_deploy"
-do_build[depends] += "${KERNEL_DEPLOY_DEPEND}"
-
+do_build[depends] += "${KERNEL_DEPLOY_DEPEND} ${IMAGE_FITIMAGE_DEPEND}"
python () {
def extraimage_getdepends(task):
diff --git a/meta/classes-recipe/kernel-fitimage.bbclass
b/meta/classes-recipe/kernel-fitimage.bbclass
index 7980910..40406fc 100644
--- a/meta/classes-recipe/kernel-fitimage.bbclass
+++ b/meta/classes-recipe/kernel-fitimage.bbclass
@@ -731,10 +731,32 @@ do_install:append() {
fi
}
+python do_preserve_kernel_build() {
+archdir = d.getVar('KERNEL_OUTPUT_DIR')
+src = d.getVar('B')
+dst = d.getVar('INITRAMFSDIR')
+oe.path.copyhardlinktree(os.path.join(src, archdir), os.path.join(dst,
archdir))
+for fn in ['vmlinux', 'fit-image.its']:
+oe.path.copyhardlink(os.path.join(src, fn), os.path.join(dst, fn))
+}
+
+INITRAMFSDIR = "${WORKDIR}/initramfs-work"
+SSTATETASKS += "do_preserve_kernel_build"
+do_preserve_kernel_build[cleandirs] = "${INITRAMFSDIR}"
+do_preserve_kernel_build[sstate-plaindirs] = "${INITRAMFSDIR}"
+
+python do_preserve_kernel_build_setscene () {
+sstate_setscene(d)
+}
+addtask preserve_kernel_build_setscene
+
+
+addtask preserve_kernel_build after do_bundle_initramfs
+
do_assemble_fitimage_initramfs() {
if echo ${KERNEL_IMAGETYPES} | grep -wq "fitImage" && \
test -n "${INITRAMFS_IMAGE}" ; then
- cd ${B}
+ cd ${INITRAMFSDIR}
if [ "${INITRAMFS_IMAGE_BUNDLE}" = "1" ]; then
fitimage_assemble fit-image-${INITRAMFS_IMAGE}.its
fitImage-bundle ""
ln -sf fitImage-bundle
${B}/${KERNEL_OUTPUT_DIR}/fitImage
@@ -744,7 +766,9 @@ do_assemble_fitimage_initramfs() {
fi
}
-addtask assemble_fitimage_initramfs before do_deploy after do_bundle_initramfs
+addtask assemble_fitimage_initramfs after do_preserve_kernel_build
+
+do_assemble_fitimage_initramfs[depends] +=
"virtual/${TARGET_PREFIX}binutils:do_populate_sysroot"
do_kernel_generate_rsa_keys() {
if [ "${UBOOT_SIGN_ENABLE}" = "0" ] && [ "${FIT_GENERATE_KEYS}" = "1"
]; then
@@ -793,20 +817,33 @@ do_kernel_generate_rsa_keys() {
addtask kernel_generate_rsa_keys befor
Re: [OE-core] [RFC PATCH 0/6] (e)SDK workflow directly in a Yocto build
Hi Alex On Wednesday, 22 June 2022 22:33:06 NZDT Alexander Kanavin wrote: > There's been a recent discussion about how we can make the Yocto SDK > experience better [1]. One of the ideas was to eliminate the SDK > as a separate artefact altogether and simply provide everything > that the SDK and eSDKs do directly in a yocto build. This does not > mean that people have to 'learn Yocto', but rather that the integrators > should provide a well-functioning sstate cache infrastructure (same as > with minimal eSDK, really), and a few wrapper scripts for setting up the > build and the SDK environment that run layer setup and bitbake behind the > scenes. FWIW I think this is quite clever - well done :) One question though: > 2. Run: > $ bitbake meta-ide-support > $ bitbake -c populate_sysroot gtk+3 > (or any other target or native item that the application developer would > need) > $ bitbake populate-sysroots Should this have been "build-sysroots"? "populate-sysroots" doesn't seem to be a valid target. It looks like this is in the SDK manual as well. Cheers Paul -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#171557): https://lists.openembedded.org/g/openembedded-core/message/171557 Mute This Topic: https://lists.openembedded.org/mt/91918831/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH 0/1] install-buildtools: support buildtools-make-tarball and update to 4.1
Add support to the install-buildtools script for the new make tarball. I noticed this while adding documentation for new features in 4.1. NOTE: also includes a bump to version 4.1; the side-effect of this is that the new defaults won't work until 4.1 is actually released, but I figured that was a reasonable tradeoff vs. only supporting what is in 4.0 by default. The following changes since commit 5200799866b92259e855051112520006e1c0: build-appliance-image: Update to master head revision (2022-09-30 10:39:16 +0100) are available in the git repository at: https://git.yoctoproject.org/poky-contrib paule/buildtools-installer-make http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=paule/buildtools-installer-make Paul Eggleton (1): install-buildtools: support buildtools-make-tarball and update to 4.1 scripts/install-buildtools | 20 1 file changed, 16 insertions(+), 4 deletions(-) -- 1.8.3.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#171523): https://lists.openembedded.org/g/openembedded-core/message/171523 Mute This Topic: https://lists.openembedded.org/mt/94177235/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH 1/1] install-buildtools: support buildtools-make-tarball and update to 4.1
From: Paul Eggleton
Support installing buildtools-make-tarball that is built in version 4.1
and later for build hosts with a broken make version. Also update the
default version values to 4.1.
Signed-off-by: Paul Eggleton
---
scripts/install-buildtools | 20
1 file changed, 16 insertions(+), 4 deletions(-)
diff --git a/scripts/install-buildtools b/scripts/install-buildtools
index 10c3d04..2218f3f 100755
--- a/scripts/install-buildtools
+++ b/scripts/install-buildtools
@@ -57,8 +57,8 @@ logger = scriptutils.logger_create(PROGNAME,
stream=sys.stdout)
DEFAULT_INSTALL_DIR = os.path.join(os.path.split(scripts_path)[0],'buildtools')
DEFAULT_BASE_URL = 'http://downloads.yoctoproject.org/releases/yocto'
-DEFAULT_RELEASE = 'yocto-3.4'
-DEFAULT_INSTALLER_VERSION = '3.4'
+DEFAULT_RELEASE = 'yocto-4.1'
+DEFAULT_INSTALLER_VERSION = '4.1'
DEFAULT_BUILDDATE = '202110XX'
# Python version sanity check
@@ -154,6 +154,8 @@ def main():
group.add_argument('--without-extended-buildtools', action='store_false',
dest='with_extended_buildtools',
help='disable extended buildtools (traditional
buildtools tarball)')
+group.add_argument('--make-only', action='store_true',
+ help='only install make tarball')
group = parser.add_mutually_exclusive_group()
group.add_argument('-c', '--check', help='enable checksum validation',
default=True, action='store_true')
@@ -170,6 +172,9 @@ def main():
args = parser.parse_args()
+if args.make_only:
+args.with_extended_buildtools = False
+
if args.debug:
logger.setLevel(logging.DEBUG)
elif args.quiet:
@@ -197,7 +202,10 @@ def main():
if not args.build_date:
logger.error("Milestone installers require --build-date")
else:
-if args.with_extended_buildtools:
+if args.make_only:
+filename =
"%s-buildtools-make-nativesdk-standalone-%s-%s.sh" % (
+arch, args.installer_version, args.build_date)
+elif args.with_extended_buildtools:
filename =
"%s-buildtools-extended-nativesdk-standalone-%s-%s.sh" % (
arch, args.installer_version, args.build_date)
else:
@@ -207,6 +215,8 @@ def main():
buildtools_url = "%s/milestones/%s/buildtools/%s" %
(base_url, args.release, safe_filename)
# regular release SDK
else:
+if args.make_only:
+filename = "%s-buildtools-make-nativesdk-standalone-%s.sh"
% (arch, args.installer_version)
if args.with_extended_buildtools:
filename =
"%s-buildtools-extended-nativesdk-standalone-%s.sh" % (arch,
args.installer_version)
else:
@@ -303,7 +313,9 @@ def main():
if args.with_extended_buildtools and not m:
logger.info("Ignoring --with-extended-buildtools as filename "
"does not contain 'extended'")
-if args.with_extended_buildtools and m:
+if args.make_only:
+tool = 'make'
+elif args.with_extended_buildtools and m:
tool = 'gcc'
else:
tool = 'tar'
--
1.8.3.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#171522):
https://lists.openembedded.org/g/openembedded-core/message/171522
Mute This Topic: https://lists.openembedded.org/mt/94177234/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [dunfell PATCH] licenses: Handle newer SPDX license names
License names were updated in commit 2456f523cfbb (after the dunfell release) to match the current SPDX license list. We don't want to do any wholesale renaming on the dunfell stable branch but we should add mappings for the newer '*-only' names to allow for layers which support both dunfell and newer releases. Signed-off-by: Paul Barker --- meta/conf/licenses.conf | 7 +++ 1 file changed, 7 insertions(+) diff --git a/meta/conf/licenses.conf b/meta/conf/licenses.conf index 5b309eb3851f..0149b1dc4445 100644 --- a/meta/conf/licenses.conf +++ b/meta/conf/licenses.conf @@ -13,24 +13,31 @@ SPDXLICENSEMAP[AGPL-3] = "AGPL-3.0" SPDXLICENSEMAP[AGPLv3] = "AGPL-3.0" SPDXLICENSEMAP[AGPLv3.0] = "AGPL-3.0" +SPDXLICENSEMAP[AGPL-3.0-only] = "AGPL-3.0" # GPL variations SPDXLICENSEMAP[GPL-1] = "GPL-1.0" SPDXLICENSEMAP[GPLv1] = "GPL-1.0" SPDXLICENSEMAP[GPLv1.0] = "GPL-1.0" +SPDXLICENSEMAP[GPL-1.0-only] = "GPL-1.0" SPDXLICENSEMAP[GPL-2] = "GPL-2.0" SPDXLICENSEMAP[GPLv2] = "GPL-2.0" SPDXLICENSEMAP[GPLv2.0] = "GPL-2.0" +SPDXLICENSEMAP[GPL-2.0-only] = "GPL-2.0" SPDXLICENSEMAP[GPL-3] = "GPL-3.0" SPDXLICENSEMAP[GPLv3] = "GPL-3.0" SPDXLICENSEMAP[GPLv3.0] = "GPL-3.0" +SPDXLICENSEMAP[GPL-3.0-only] = "GPL-3.0" #LGPL variations SPDXLICENSEMAP[LGPLv2] = "LGPL-2.0" SPDXLICENSEMAP[LGPLv2.0] = "LGPL-2.0" +SPDXLICENSEMAP[LGPL-2.0-only] = "LGPL-2.0" SPDXLICENSEMAP[LGPL2.1] = "LGPL-2.1" SPDXLICENSEMAP[LGPLv2.1] = "LGPL-2.1" +SPDXLICENSEMAP[LGPL-2.1-only] = "LGPL-2.1" SPDXLICENSEMAP[LGPLv3] = "LGPL-3.0" +SPDXLICENSEMAP[LGPL-3.0-only] = "LGPL-3.0" #MPL variations SPDXLICENSEMAP[MPL-1] = "MPL-1.0" -- 2.25.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#171220): https://lists.openembedded.org/g/openembedded-core/message/171220 Mute This Topic: https://lists.openembedded.org/mt/94013682/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH 03/10] zlib: Resolve CVE-2022-37434
On Monday, 15 August 2022 10:20:30 NZST Khem Raj wrote: > This is only seen in development branch as per [1] > > [1] > https://github.com/madler/zlib/commit/eff308af425b67093bab25f80f1ae950166be > ce1#commitcomment-80753451 CVE: CVE-2022-37434 It's a little confusing, but I think that CVE-2022-37434 does affect existing zlib releases - at least the patch does apply. My reading was that the upstream comment was referring to the *fix* (and thus the additional segfault issue that it introduced) was not yet in any zlib release. A look around suggests Ubuntu is treating CVE-2022-37434 as needing to be fixed at least. Cheers Paul -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#169346): https://lists.openembedded.org/g/openembedded-core/message/169346 Mute This Topic: https://lists.openembedded.org/mt/93025229/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH 5/5] lib: Add copyright statements to files without one
On 11/08/2022 08:59, Richard Purdie wrote: On Thu, 2022-08-11 at 08:57 +0100, Paul Barker wrote: On 10/08/2022 20:45, Richard Purdie wrote: diff --git a/meta/lib/oeqa/runtime/cases/rtc.py b/meta/lib/oeqa/runtime/cases/rtc.py index c4e66813249..223cf7b95d9 100644 --- a/meta/lib/oeqa/runtime/cases/rtc.py +++ b/meta/lib/oeqa/runtime/cases/rtc.py @@ -1,3 +1,8 @@ +\# Looks like an extra '\' has slipped in here. Thanks, well spotted! I've fixed in the branch. I've also checked master-next with `reuse lint`, obviously it complains that license text files aren't in the places the reuse tool wants them to be and that's unimportant, but it does highlight some other low hanging fruit: * The following are missing SPDX headers: * scripts/lib/devtool/menuconfig.py * meta-skeleton/recipes-kernel/hello-mod/files/hello.c * The following are missing copyright headers: * scripts/lib/argparse_oe.py * scripts/lib/wic/plugins/source/bootimg-partition.py * scripts/lib/wic/plugins/source/empty.py * scripts/lib/wic/plugins/source/isoimage-isohybrid.py * scripts/lib/wic/plugins/source/rawcopy.py * Most files under meta/conf can probaby have copyright and license headers added without any confusion. * `reuse lint` will accept the following header as valid, which could be a path forward for recipes if other folks think it's clear enough: # Recipe: Copyright OpenEmbedded Contributors # Recipe: SPDX-License-Identifier: MIT Thanks, -- Paul Barker OpenPGP_0x74975C81B7E66BAC.asc Description: OpenPGP public key OpenPGP_signature Description: OpenPGP digital signature -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#169235): https://lists.openembedded.org/g/openembedded-core/message/169235 Mute This Topic: https://lists.openembedded.org/mt/92944317/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH 5/5] lib: Add copyright statements to files without one
On 10/08/2022 20:45, Richard Purdie wrote: diff --git a/meta/lib/oeqa/runtime/cases/rtc.py b/meta/lib/oeqa/runtime/cases/rtc.py index c4e66813249..223cf7b95d9 100644 --- a/meta/lib/oeqa/runtime/cases/rtc.py +++ b/meta/lib/oeqa/runtime/cases/rtc.py @@ -1,3 +1,8 @@ +\# Looks like an extra '\' has slipped in here. Thanks, -- Paul Barker OpenPGP_0x74975C81B7E66BAC.asc Description: OpenPGP public key OpenPGP_signature Description: OpenPGP digital signature -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#169233): https://lists.openembedded.org/g/openembedded-core/message/169233 Mute This Topic: https://lists.openembedded.org/mt/92944317/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH] relocate_sdk.py: ensure interpreter size error causes relocation to fail
From: Paul Eggleton
If there is insufficent space to change the interpreter, we were
printing an error here but the overall script did not return an error
code, and thus the SDK installation appeared to succeed - but some of
the binaries will not be in a working state. Allow the relocation to
proceed (so we still get a full list of the failures) but error out at
the end so that the installation is halted.
Signed-off-by: Paul Eggleton
---
scripts/relocate_sdk.py | 10 --
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git a/scripts/relocate_sdk.py b/scripts/relocate_sdk.py
index 4ed8bfc..8a72872 100755
--- a/scripts/relocate_sdk.py
+++ b/scripts/relocate_sdk.py
@@ -104,11 +104,12 @@ def change_interpreter(elf_file_name):
if (len(new_dl_path) >= p_filesz):
print("ERROR: could not relocate %s, interp size = %i and %i
is needed." \
% (elf_file_name, p_memsz, len(new_dl_path) + 1))
-break
+return False
dl_path = new_dl_path + b("\0") * (p_filesz - len(new_dl_path))
f.seek(p_offset)
f.write(dl_path)
break
+return True
def change_dl_sysdirs(elf_file_name):
if arch == 32:
@@ -222,6 +223,7 @@ else:
executables_list = sys.argv[3:]
+errors = False
for e in executables_list:
perms = os.stat(e)[stat.ST_MODE]
if os.access(e, os.W_OK|os.R_OK):
@@ -247,7 +249,8 @@ for e in executables_list:
arch = get_arch()
if arch:
parse_elf_header()
-change_interpreter(e)
+if not change_interpreter(e):
+errors = True
change_dl_sysdirs(e)
""" change permissions back """
@@ -260,3 +263,6 @@ for e in executables_list:
print("New file size for %s is different. Looks like a relocation
error!", e)
sys.exit(-1)
+if errors:
+print("Relocation of one or more executables failed.")
+sys.exit(-1)
--
1.8.3.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#168878):
https://lists.openembedded.org/g/openembedded-core/message/168878
Mute This Topic: https://lists.openembedded.org/mt/92825384/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH v2 1/3] patch: handle if S points to a subdirectory of a git repo
From: Paul Eggleton
If PATCHTOOL = "git", SRC_URI fetches from a git repo and S points to
a subdirectory of the checked out sources, then we were erroneously
initialising the subdirectory as its own git repo. Check if the returned
top-level repo directory is a subdirectory of WORKDIR and do not
run initialise the source directory if that is the case.
(This was a regression introduced with OE-Core revision
6184b56a7a0fc6f5d19fdfb81e7453667f7da940, however we didn't have a test
that verified the behaviour.)
Signed-off-by: Paul Eggleton
---
meta/lib/oe/patch.py | 8
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/meta/lib/oe/patch.py b/meta/lib/oe/patch.py
index 95b915a..4ec9cae 100644
--- a/meta/lib/oe/patch.py
+++ b/meta/lib/oe/patch.py
@@ -299,10 +299,10 @@ class GitApplyTree(PatchTree):
PatchTree.__init__(self, dir, d)
self.commituser = d.getVar('PATCH_GIT_USER_NAME')
self.commitemail = d.getVar('PATCH_GIT_USER_EMAIL')
-if not self._isInitialized():
+if not self._isInitialized(d):
self._initRepo()
-def _isInitialized(self):
+def _isInitialized(self, d):
cmd = "git rev-parse --show-toplevel"
try:
output = runcmd(cmd.split(), self.dir).strip()
@@ -310,8 +310,8 @@ class GitApplyTree(PatchTree):
## runcmd returned non-zero which most likely means 128
## Not a git directory
return False
-## Make sure repo is in builddir to not break top-level git repos
-return os.path.samefile(output, self.dir)
+## Make sure repo is in builddir to not break top-level git repos, or
under workdir
+return os.path.samefile(output, self.dir) or
oe.path.is_path_parent(d.getVar('WORKDIR'), output)
def _initRepo(self):
runcmd("git init".split(), self.dir)
--
1.8.3.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#167949):
https://lists.openembedded.org/g/openembedded-core/message/167949
Mute This Topic: https://lists.openembedded.org/mt/92349705/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH v2 0/3] devtool fixes
A few fixes for devtool, mostly relating to recipes that fetch from git where S points to a subdirectory of the repo. Changes since v1: * Fixed erroneous addition of a separator when not needed (which broke one of the other devtool tests) * Fixed missing separator before patchdir= when updating original recipe * Added extra logic to the test to ensure we can update the original recipe properly (as well creating a bbappend that was already in the test). * Rebased and dropped pn- overrides fix since that has just been merged. The following changes since commit db22dbc1f66d2f76dab719597986cfd96cf18d71: linux-yocto/5.10: fix build_OID_registry/conmakehash buildpaths warning (2022-07-12 23:56:06 +0100) are available in the git repository at: git://git.openembedded.org/openembedded-core-contrib paule/devtool37-oe http://cgit.openembedded.org/openembedded-core-contrib/log/?h=paule/devtool37-oe Paul Eggleton (3): patch: handle if S points to a subdirectory of a git repo devtool: finish: handle patching when S points to subdir of a git repo oe-selftest: devtool: test modify git recipe building from a subdir meta/lib/oe/patch.py| 8 +-- meta/lib/oe/recipeutils.py | 9 ++- meta/lib/oeqa/selftest/cases/devtool.py | 114 +++- scripts/lib/devtool/standard.py | 25 +-- 4 files changed, 127 insertions(+), 29 deletions(-) -- 1.8.3.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#167951): https://lists.openembedded.org/g/openembedded-core/message/167951 Mute This Topic: https://lists.openembedded.org/mt/92349707/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH v2 3/3] oe-selftest: devtool: test modify git recipe building from a subdir
From: Paul Eggleton
Add a test that verifies that devtool modify + devtool finish do the
right thing on a recipe that fetches from git and sets S to point to
a subdirectory of the source tree. We have a few examples among the core
recipes, dos2unix is a convenient one so let's use that. (The test first
verifies that that is still true in case the recipe is changed in
future.)
Signed-off-by: Paul Eggleton
---
meta/lib/oeqa/selftest/cases/devtool.py | 114 +++-
1 file changed, 97 insertions(+), 17 deletions(-)
diff --git a/meta/lib/oeqa/selftest/cases/devtool.py
b/meta/lib/oeqa/selftest/cases/devtool.py
index ddf6c0c..34fc791 100644
--- a/meta/lib/oeqa/selftest/cases/devtool.py
+++ b/meta/lib/oeqa/selftest/cases/devtool.py
@@ -218,6 +218,34 @@ class DevtoolTestCase(OESelftestTestCase):
filelist.append(' '.join(splitline))
return filelist
+def _check_diff(self, diffoutput, addlines, removelines):
+"""Check output from 'git diff' matches expectation"""
+remaining_addlines = addlines[:]
+remaining_removelines = removelines[:]
+for line in diffoutput.splitlines():
+if line.startswith('+++') or line.startswith('---'):
+continue
+elif line.startswith('+'):
+matched = False
+for item in addlines:
+if re.match(item, line[1:].strip()):
+matched = True
+remaining_addlines.remove(item)
+break
+self.assertTrue(matched, 'Unexpected diff add line: %s' % line)
+elif line.startswith('-'):
+matched = False
+for item in removelines:
+if re.match(item, line[1:].strip()):
+matched = True
+remaining_removelines.remove(item)
+break
+self.assertTrue(matched, 'Unexpected diff remove line: %s' %
line)
+if remaining_addlines:
+self.fail('Expected added lines not found: %s' %
remaining_addlines)
+if remaining_removelines:
+self.fail('Expected removed lines not found: %s' %
remaining_removelines)
+
class DevtoolBase(DevtoolTestCase):
@@ -718,6 +746,7 @@ class DevtoolModifyTests(DevtoolBase):
self.assertTrue(bbclassextended, 'None of these recipes are
BBCLASSEXTENDed to native - need to adjust testrecipes list: %s' % ',
'.join(testrecipes))
self.assertTrue(inheritnative, 'None of these recipes do "inherit
native" - need to adjust testrecipes list: %s' % ', '.join(testrecipes))
+
def test_devtool_modify_localfiles_only(self):
# Check preconditions
testrecipe = 'base-files'
@@ -930,23 +959,7 @@ class DevtoolUpdateTests(DevtoolBase):
srcurilines[0] = 'SRC_URI = "' + srcurilines[0]
srcurilines.append('"')
removelines = ['SRCREV = ".*"'] + srcurilines
-for line in result.output.splitlines():
-if line.startswith('+++') or line.startswith('---'):
-continue
-elif line.startswith('+'):
-matched = False
-for item in addlines:
-if re.match(item, line[1:].strip()):
-matched = True
-break
-self.assertTrue(matched, 'Unexpected diff add line: %s' % line)
-elif line.startswith('-'):
-matched = False
-for item in removelines:
-if re.match(item, line[1:].strip()):
-matched = True
-break
-self.assertTrue(matched, 'Unexpected diff remove line: %s' %
line)
+self._check_diff(result.output, addlines, removelines)
# Now try with auto mode
runCmd('cd %s; git checkout %s %s' % (os.path.dirname(recipefile),
testrecipe, os.path.basename(recipefile)))
result = runCmd('devtool update-recipe %s' % testrecipe)
@@ -1316,6 +1329,73 @@ class DevtoolUpdateTests(DevtoolBase):
expected_status = []
self._check_repo_status(os.path.dirname(recipefile), expected_status)
+def test_devtool_finish_modify_git_subdir(self):
+# Check preconditions
+testrecipe = 'dos2unix'
+bb_vars = get_bb_vars(['SRC_URI', 'S', 'WORKDIR', 'FILE'], testrecipe)
+self.assertIn('git://', bb_vars['SRC_URI'], 'This test expects the %s
recipe to be a git recipe' % testrecipe)
+wor
[OE-core] [PATCH v2 2/3] devtool: finish: handle patching when S points to subdir of a git repo
From: Paul Eggleton
If devtool finish needs to create a patch and have it applied to the
sources for a recipe where S points to a subdirectory of the sources,
then the patch needs to be applied at the root of the repo i.e. we need
to add a patchdir= parameter to the SRC_URI entry.
Signed-off-by: Paul Eggleton
---
meta/lib/oe/recipeutils.py | 9 +++--
scripts/lib/devtool/standard.py | 25 +++--
2 files changed, 26 insertions(+), 8 deletions(-)
diff --git a/meta/lib/oe/recipeutils.py b/meta/lib/oe/recipeutils.py
index 872ff97..b04992c 100644
--- a/meta/lib/oe/recipeutils.py
+++ b/meta/lib/oe/recipeutils.py
@@ -666,7 +666,7 @@ def get_bbappend_path(d, destlayerdir, wildcardver=False):
return (appendpath, pathok)
-def bbappend_recipe(rd, destlayerdir, srcfiles, install=None,
wildcardver=False, machine=None, extralines=None, removevalues=None,
redirect_output=None):
+def bbappend_recipe(rd, destlayerdir, srcfiles, install=None,
wildcardver=False, machine=None, extralines=None, removevalues=None,
redirect_output=None, params=None):
"""
Writes a bbappend file for a recipe
Parameters:
@@ -696,6 +696,9 @@ def bbappend_recipe(rd, destlayerdir, srcfiles,
install=None, wildcardver=False,
redirect_output:
If specified, redirects writing the output file to the
specified directory (for dry-run purposes)
+params:
+Parameters to use when adding entries to SRC_URI. If specified,
+should be a list of dicts with the same length as srcfiles.
"""
if not removevalues:
@@ -762,12 +765,14 @@ def bbappend_recipe(rd, destlayerdir, srcfiles,
install=None, wildcardver=False,
copyfiles = {}
if srcfiles:
instfunclines = []
-for newfile, origsrcfile in srcfiles.items():
+for i, (newfile, origsrcfile) in enumerate(srcfiles.items()):
srcfile = origsrcfile
srcurientry = None
if not srcfile:
srcfile = os.path.basename(newfile)
srcurientry = 'file://%s' % srcfile
+if params and params[i]:
+srcurientry = '%s;%s' % (srcurientry, ';'.join('%s=%s' %
(k,v) for k,v in params[i].items()))
# Double-check it's not there already
# FIXME do we care if the entry is added by another bbappend
that might go away?
if not srcurientry in rd.getVar('SRC_URI').split():
diff --git a/scripts/lib/devtool/standard.py b/scripts/lib/devtool/standard.py
index e53569c..c98bfe8 100644
--- a/scripts/lib/devtool/standard.py
+++ b/scripts/lib/devtool/standard.py
@@ -1606,6 +1606,19 @@ def _update_recipe_patch(recipename, workspace, srctree,
rd, appendlayerdir, wil
if not os.path.exists(append):
raise DevtoolError('unable to find workspace bbappend for recipe %s' %
recipename)
+srctreebase = workspace[recipename]['srctreebase']
+relpatchdir = os.path.relpath(srctreebase, srctree)
+if relpatchdir == '.':
+patchdir_params = {}
+else:
+patchdir_params = {'patchdir': relpatchdir}
+
+def srcuri_entry(fname):
+if patchdir_params:
+paramstr = ';' + ';'.join('%s=%s' % (k,v) for k,v in
patchdir_params.items())
+else:
+paramstr = ''
+return 'file://%s%s' % (basepath, paramstr)
initial_rev, update_rev, changed_revs, filter_patches =
_get_patchset_revs(srctree, append, initial_rev, force_patch_refresh)
if not initial_rev:
@@ -1627,7 +1640,6 @@ def _update_recipe_patch(recipename, workspace, srctree,
rd, appendlayerdir, wil
new_f = {}
del_f = {}
else:
-srctreebase = workspace[recipename]['srctreebase']
upd_f, new_f, del_f = _export_local_files(srctree, rd,
local_files_dir, srctreebase)
remove_files = []
@@ -1663,14 +1675,15 @@ def _update_recipe_patch(recipename, workspace,
srctree, rd, appendlayerdir, wil
removedentries, remaining = _remove_file_entries(
srcuri, remove_files)
if removedentries or remaining:
-remaining = ['file://' + os.path.basename(item) for
+remaining = [srcuri_entry(os.path.basename(item)) for
item in remaining]
removevalues = {'SRC_URI': removedentries + remaining}
appendfile, destpath = oe.recipeutils.bbappend_recipe(
rd, appendlayerdir, files,
wildcardver=wildcard_version,
removevalues=removeva
Re: [OE-core] [PATCH 0/4] devtool fixes
On Tue, Jul 12, 2022 at 04:59:38PM +0100, Richard Purdie wrote: > On Mon, 2022-07-11 at 21:06 -0700, Paul Eggleton wrote: > > A few fixes for devtool, mostly relating to recipes that fetch from git > > where S points to a subdirectory of the repo. > > > > Note: I wasn't 100% sure if it was appropriate to be referencing WORKDIR > > in patch.py; let me know if I should try to rework that. > > It is probably ok. The code won't pick up on the variable usage from a > hash perspective there but it would be referenced elsewhere anyway. > > There was a failure which is probably from this series though: > > https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/3817/steps/14/logs/stdio > > > (I've not confirmed it was the series, I'm just guessing) Hmm, yes, it probably is. I did run the tests but somehow I didn't see this failure. I'll investigate further. Paul -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#167940): https://lists.openembedded.org/g/openembedded-core/message/167940 Mute This Topic: https://lists.openembedded.org/mt/92326549/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH 4/4] oe-selftest: devtool: test modify git recipe building from a subdir
From: Paul Eggleton
Add a test that verifies that devtool modify + devtool finish do the
right thing on a recipe that fetches from git and sets S to point to
a subdirectory of the source tree. We have a few examples among the core
recipes, dos2unix is a convenient one so let's use that. (The test first
verifies that that is still true in case the recipe is changed in
future.)
Signed-off-by: Paul Eggleton
---
meta/lib/oeqa/selftest/cases/devtool.py | 51 +
1 file changed, 51 insertions(+)
diff --git a/meta/lib/oeqa/selftest/cases/devtool.py
b/meta/lib/oeqa/selftest/cases/devtool.py
index ddf6c0c..ddab150 100644
--- a/meta/lib/oeqa/selftest/cases/devtool.py
+++ b/meta/lib/oeqa/selftest/cases/devtool.py
@@ -718,6 +718,7 @@ class DevtoolModifyTests(DevtoolBase):
self.assertTrue(bbclassextended, 'None of these recipes are
BBCLASSEXTENDed to native - need to adjust testrecipes list: %s' % ',
'.join(testrecipes))
self.assertTrue(inheritnative, 'None of these recipes do "inherit
native" - need to adjust testrecipes list: %s' % ', '.join(testrecipes))
+
def test_devtool_modify_localfiles_only(self):
# Check preconditions
testrecipe = 'base-files'
@@ -1316,6 +1317,56 @@ class DevtoolUpdateTests(DevtoolBase):
expected_status = []
self._check_repo_status(os.path.dirname(recipefile), expected_status)
+def test_devtool_finish_modify_git_subdir(self):
+# Check preconditions
+testrecipe = 'dos2unix'
+bb_vars = get_bb_vars(['SRC_URI', 'S', 'WORKDIR', 'FILE'], testrecipe)
+self.assertIn('git://', bb_vars['SRC_URI'], 'This test expects the %s
recipe to be a git recipe' % testrecipe)
+workdir_git = '%s/git/' % bb_vars['WORKDIR']
+if not bb_vars['S'].startswith(workdir_git):
+self.fail('This test expects the %s recipe to be building from a
subdirectory of the git repo' % testrecipe)
+subdir = bb_vars['S'].split(workdir_git, 1)[1]
+# Clean up anything in the workdir/sysroot/sstate cache
+bitbake('%s -c cleansstate' % testrecipe)
+# Try modifying a recipe
+tempdir = tempfile.mkdtemp(prefix='devtoolqa')
+self.track_for_cleanup(tempdir)
+self.track_for_cleanup(self.workspacedir)
+self.add_command_to_tearDown('bitbake -c clean %s' % testrecipe)
+self.add_command_to_tearDown('bitbake-layers remove-layer */workspace')
+result = runCmd('devtool modify %s -x %s' % (testrecipe, tempdir))
+testsrcfile = os.path.join(tempdir, subdir, 'dos2unix.c')
+self.assertExists(testsrcfile, 'Extracted source could not be found')
+self.assertExists(os.path.join(self.workspacedir, 'conf',
'layer.conf'), 'Workspace directory not created. devtool output: %s' %
result.output)
+self.assertNotExists(os.path.join(tempdir, subdir, '.git'),
'Subdirectory has been initialised as a git repo')
+# Check git repo
+self._check_src_repo(tempdir)
+# Modify file
+runCmd("sed -i '1s:^:/* Add a comment */\\n:' %s" % testsrcfile)
+result = runCmd('git commit -a -m "Add a comment"', cwd=tempdir)
+# Run devtool finish
+recipefile = bb_vars['FILE']
+recipedir = os.path.dirname(recipefile)
+res = re.search('recipes-.*', recipedir)
+self.assertTrue(res, 'Unable to find recipe subdirectory')
+recipesubdir = res[0]
+self.add_command_to_tearDown('rm -rf %s' %
os.path.join(self.testlayer_path, recipesubdir))
+result = runCmd('devtool finish %s meta-selftest' % testrecipe)
+# Check bbappend file contents
+appendfn = os.path.join(self.testlayer_path, recipesubdir,
'%s_%%.bbappend' % testrecipe)
+with open(appendfn, 'r') as f:
+appendlines = f.readlines()
+expected_appendlines = [
+'FILESEXTRAPATHS:prepend := "${THISDIR}/${PN}:"\n',
+'\n',
+'SRC_URI += "file://0001-Add-a-comment.patch;patchdir=.."\n',
+'\n'
+]
+self.assertEqual(appendlines, expected_appendlines)
+self.assertExists(os.path.join(os.path.dirname(appendfn), testrecipe,
'0001-Add-a-comment.patch'))
+# Try building
+bitbake('%s -c patch' % testrecipe)
+
class DevtoolExtractTests(DevtoolBase):
def test_devtool_extract(self):
--
1.8.3.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#167890):
https://lists.openembedded.org/g/openembedded-core/message/167890
Mute This Topic: https://lists.openembedded.org/mt/92326550/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH 2/4] devtool: finish: handle patching when S points to subdir of a git repo
From: Paul Eggleton
If devtool finish needs to create a patch and have it applied to the
sources for a recipe where S points to a subdirectory of the sources,
then the patch needs to be applied at the root of the repo i.e. we need
to add a patchdir= parameter to the SRC_URI entry.
Signed-off-by: Paul Eggleton
---
meta/lib/oe/recipeutils.py | 9 +++--
scripts/lib/devtool/standard.py | 25 +++--
2 files changed, 26 insertions(+), 8 deletions(-)
diff --git a/meta/lib/oe/recipeutils.py b/meta/lib/oe/recipeutils.py
index 872ff97..6755ae8 100644
--- a/meta/lib/oe/recipeutils.py
+++ b/meta/lib/oe/recipeutils.py
@@ -666,7 +666,7 @@ def get_bbappend_path(d, destlayerdir, wildcardver=False):
return (appendpath, pathok)
-def bbappend_recipe(rd, destlayerdir, srcfiles, install=None,
wildcardver=False, machine=None, extralines=None, removevalues=None,
redirect_output=None):
+def bbappend_recipe(rd, destlayerdir, srcfiles, install=None,
wildcardver=False, machine=None, extralines=None, removevalues=None,
redirect_output=None, params=None):
"""
Writes a bbappend file for a recipe
Parameters:
@@ -696,6 +696,9 @@ def bbappend_recipe(rd, destlayerdir, srcfiles,
install=None, wildcardver=False,
redirect_output:
If specified, redirects writing the output file to the
specified directory (for dry-run purposes)
+params:
+Parameters to use when adding entries to SRC_URI. If specified,
+should be a list of dicts with the same length as srcfiles.
"""
if not removevalues:
@@ -762,12 +765,14 @@ def bbappend_recipe(rd, destlayerdir, srcfiles,
install=None, wildcardver=False,
copyfiles = {}
if srcfiles:
instfunclines = []
-for newfile, origsrcfile in srcfiles.items():
+for i, (newfile, origsrcfile) in enumerate(srcfiles.items()):
srcfile = origsrcfile
srcurientry = None
if not srcfile:
srcfile = os.path.basename(newfile)
srcurientry = 'file://%s' % srcfile
+if params:
+srcurientry = '%s;%s' % (srcurientry, ';'.join('%s=%s' %
(k,v) for k,v in params[i].items()))
# Double-check it's not there already
# FIXME do we care if the entry is added by another bbappend
that might go away?
if not srcurientry in rd.getVar('SRC_URI').split():
diff --git a/scripts/lib/devtool/standard.py b/scripts/lib/devtool/standard.py
index 4b50e3c..a942ef3 100644
--- a/scripts/lib/devtool/standard.py
+++ b/scripts/lib/devtool/standard.py
@@ -1604,6 +1604,19 @@ def _update_recipe_patch(recipename, workspace, srctree,
rd, appendlayerdir, wil
if not os.path.exists(append):
raise DevtoolError('unable to find workspace bbappend for recipe %s' %
recipename)
+srctreebase = workspace[recipename]['srctreebase']
+relpatchdir = os.path.relpath(srctreebase, srctree)
+if relpatchdir == '.':
+patchdir_params = {}
+else:
+patchdir_params = {'patchdir': relpatchdir}
+
+def srcuri_entry(fname):
+if patchdir_params:
+paramstr = ';'.join('%s=%s' % (k,v) for k,v in
patchdir_params.items())
+else:
+paramstr = ''
+return 'file://%s%s' % (basepath, paramstr)
initial_rev, update_rev, changed_revs, filter_patches =
_get_patchset_revs(srctree, append, initial_rev, force_patch_refresh)
if not initial_rev:
@@ -1625,7 +1638,6 @@ def _update_recipe_patch(recipename, workspace, srctree,
rd, appendlayerdir, wil
new_f = {}
del_f = {}
else:
-srctreebase = workspace[recipename]['srctreebase']
upd_f, new_f, del_f = _export_local_files(srctree, rd,
local_files_dir, srctreebase)
remove_files = []
@@ -1661,14 +1673,15 @@ def _update_recipe_patch(recipename, workspace,
srctree, rd, appendlayerdir, wil
removedentries, remaining = _remove_file_entries(
srcuri, remove_files)
if removedentries or remaining:
-remaining = ['file://' + os.path.basename(item) for
+remaining = [srcuri_entry(os.path.basename(item)) for
item in remaining]
removevalues = {'SRC_URI': removedentries + remaining}
appendfile, destpath = oe.recipeutils.bbappend_recipe(
rd, appendlayerdir, files,
wildcardver=wildcard_version,
removevalues=removevalues,
-
[OE-core] [PATCH 3/4] devtool: ignore pn- overrides when determining SRC_URI overrides
From: Paul Eggleton
If (perhaps foolishly) at your configuration level you have e.g.
SRC_URI_append_pn-recipename = " file://patchname.patch"
and then run devtool modify on a different recipe, an error occurs:
INFO: SRC_URI contains some conditional appends/prepends - will create
branches to represent these
...
ERROR: [Errno 2] No such file or directory:
'/path/to/downloads/patchname.patch'
pn- overrides would not constitute an alternative configuration that we
should handle in this context, so just ignore them to avoid the issue.
Signed-off-by: Paul Eggleton
---
scripts/lib/devtool/standard.py | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/scripts/lib/devtool/standard.py b/scripts/lib/devtool/standard.py
index a942ef3..b0b484e 100644
--- a/scripts/lib/devtool/standard.py
+++ b/scripts/lib/devtool/standard.py
@@ -520,7 +520,9 @@ def _extract_source(srctree, keep_temp, devbranch, sync,
config, basepath, works
for event in history:
if not 'flag' in event:
if event['op'].startswith((':append[', ':prepend[')):
-
extra_overrides.append(event['op'].split('[')[1].split(']')[0])
+override = event['op'].split('[')[1].split(']')[0]
+if not override.startswith('pn-'):
+extra_overrides.append(override)
# We want to remove duplicate overrides. If a recipe had multiple
# SRC_URI_override += values it would cause mulitple instances of
# overrides. This doesn't play nicely with things like creating a
--
1.8.3.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#167893):
https://lists.openembedded.org/g/openembedded-core/message/167893
Mute This Topic: https://lists.openembedded.org/mt/92326553/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH 1/4] patch: handle if S points to a subdirectory of a git repo
From: Paul Eggleton
If PATCHTOOL = "git", SRC_URI fetches from a git repo and S points to
a subdirectory of the checked out sources, then we were erroneously
initialising the subdirectory as its own git repo. Check if the returned
top-level repo directory is a subdirectory of WORKDIR and do not
run initialise the source directory if that is the case.
(This was a regression introduced with OE-Core revision
6184b56a7a0fc6f5d19fdfb81e7453667f7da940, however we didn't have a test
that verified the behaviour.)
Signed-off-by: Paul Eggleton
---
meta/lib/oe/patch.py | 8
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/meta/lib/oe/patch.py b/meta/lib/oe/patch.py
index 95b915a..4ec9cae 100644
--- a/meta/lib/oe/patch.py
+++ b/meta/lib/oe/patch.py
@@ -299,10 +299,10 @@ class GitApplyTree(PatchTree):
PatchTree.__init__(self, dir, d)
self.commituser = d.getVar('PATCH_GIT_USER_NAME')
self.commitemail = d.getVar('PATCH_GIT_USER_EMAIL')
-if not self._isInitialized():
+if not self._isInitialized(d):
self._initRepo()
-def _isInitialized(self):
+def _isInitialized(self, d):
cmd = "git rev-parse --show-toplevel"
try:
output = runcmd(cmd.split(), self.dir).strip()
@@ -310,8 +310,8 @@ class GitApplyTree(PatchTree):
## runcmd returned non-zero which most likely means 128
## Not a git directory
return False
-## Make sure repo is in builddir to not break top-level git repos
-return os.path.samefile(output, self.dir)
+## Make sure repo is in builddir to not break top-level git repos, or
under workdir
+return os.path.samefile(output, self.dir) or
oe.path.is_path_parent(d.getVar('WORKDIR'), output)
def _initRepo(self):
runcmd("git init".split(), self.dir)
--
1.8.3.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#167891):
https://lists.openembedded.org/g/openembedded-core/message/167891
Mute This Topic: https://lists.openembedded.org/mt/92326551/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH 0/4] devtool fixes
A few fixes for devtool, mostly relating to recipes that fetch from git where S points to a subdirectory of the repo. Note: I wasn't 100% sure if it was appropriate to be referencing WORKDIR in patch.py; let me know if I should try to rework that. The following changes since commit db28cd0e1540e44db963108430205c8c0c817774: gperf: Switch to upstream patch (2022-07-09 20:58:31 +0100) are available in the git repository at: git://git.openembedded.org/openembedded-core-contrib paule/devtool37-oe http://cgit.openembedded.org/openembedded-core-contrib/log/?h=paule/devtool37-oe Paul Eggleton (4): patch: handle if S points to a subdirectory of a git repo devtool: finish: handle patching when S points to subdir of a git repo devtool: ignore pn- overrides when determining SRC_URI overrides oe-selftest: devtool: test modify git recipe building from a subdir meta/lib/oe/patch.py| 8 +++--- meta/lib/oe/recipeutils.py | 9 -- meta/lib/oeqa/selftest/cases/devtool.py | 51 + scripts/lib/devtool/standard.py | 29 ++- 4 files changed, 84 insertions(+), 13 deletions(-) -- 1.8.3.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#167889): https://lists.openembedded.org/g/openembedded-core/message/167889 Mute This Topic: https://lists.openembedded.org/mt/92326549/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH] devshell: introduce intercepts as per native / nativesdk
In a devshell, recent versions of git will complain if the repo is owned
by someone other than the current UID - consider this example:
--
bitbake -c devshell linux-yocto
[...]
kernel-source#git branch
fatal: unsafe repository
('/home/paul/poky/build-qemuarm64/tmp/work-shared/qemuarm64/kernel-source' is
owned by someone else)
To add an exception for this directory, call:
git config --global --add safe.directory
/home/paul/poky/build-qemuarm64/tmp/work-shared/qemuarm64/kernel-source
kernel-source#
--
Of course the devshell has UID zero and the "real" UID is for "paul" in
this case. And so recent git versions complain.
As the whole purpose of the devshell is to invoke a shell where development
can take place, having a non-functional git is clearly unacceptable.
Richard suggested we could use PSEUDO_UNLOAD=1 to evade this issue, and I
suggested we probably will see other similar instances like this and should
make use of PATH to intercept via devshell wrappers - conveniently we already
have examples of this.
Here, we copy the existing "ar" example and tune it to the needs of git to
combine Richard's suggestion and mine.
As such we now also can store commit logs and use send-email with our user
specific settings, instead of "root", so in additon to fixing basic
commands like "git branch" it should also increase general usefulness.
Cc: Richard Purdie
Signed-off-by: Paul Gortmaker
diff --git a/meta/classes/devshell.bbclass b/meta/classes/devshell.bbclass
index 62dc958d9a4a..7ac134e0950f 100644
--- a/meta/classes/devshell.bbclass
+++ b/meta/classes/devshell.bbclass
@@ -2,6 +2,8 @@ inherit terminal
DEVSHELL = "${SHELL}"
+PATH:prepend = "${COREBASE}/scripts/devshell-intercept:"
+
python do_devshell () {
if d.getVarFlag("do_devshell", "manualfakeroot"):
d.prependVar("DEVSHELL", "pseudo ")
diff --git a/scripts/devshell-intercept/git b/scripts/devshell-intercept/git
new file mode 100755
index ..8adf5c9ecb71
--- /dev/null
+++ b/scripts/devshell-intercept/git
@@ -0,0 +1,19 @@
+#!/usr/bin/env python3
+#
+# Wrapper around 'git' that doesn't think we are root
+
+import os
+import shutil
+import sys
+
+os.environ['PSEUDO_UNLOAD'] = '1'
+
+# calculate path to the real 'git'
+path = os.environ['PATH']
+path = path.replace(os.path.dirname(sys.argv[0]), '')
+real_git = shutil.which('git', path=path)
+
+if len(sys.argv) == 1:
+os.execl(real_git, 'git')
+
+os.execv(real_git, sys.argv)
--
2.17.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#164701):
https://lists.openembedded.org/g/openembedded-core/message/164701
Mute This Topic: https://lists.openembedded.org/mt/90585970/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH 04/12] recipetool: Change default paramter fallback_licenses of function split_pkg_licenses from None to []
Hi Stefan
Firstly, thanks for the work extending and improving this.
Secondly, apologies for dredging this up, but I just found this looking back
through the history for the release notes:
On Friday, 8 October 2021 20:42:20 NZST Stefan Herbrechtsmeier wrote:
> From: Stefan Herbrechtsmeier
>
> Signed-off-by: Stefan Herbrechtsmeier
> ---
>
> (no changes since v1)
>
> scripts/lib/recipetool/create.py | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/scripts/lib/recipetool/create.py
> b/scripts/lib/recipetool/create.py index 83cf25d9b7..893980a5ce 100644
> --- a/scripts/lib/recipetool/create.py
> +++ b/scripts/lib/recipetool/create.py
> @@ -1134,7 +1134,7 @@ def guess_license(srctree, d):
>
> return licenses
>
> -def split_pkg_licenses(licvalues, packages, outlines,
> fallback_licenses=None, pn='${PN}'): +def split_pkg_licenses(licvalues,
> packages, outlines, fallback_licenses=[], pn='${PN}'): """
> Given a list of (license, path, md5sum) as returned by guess_license(),
> a dict of package name to path mappings, write out a set of
Note that setting a mutable type (e.g. an empty list) as a default for a
function parameter is not good practice in Python - see:
https://florimond.dev/en/posts/2018/08/python-mutable-defaults-are-the-source-of-all-evil/
I'm aware we have done this already in a few places - git grep "^def.*=\[\]"
shows a few others that we probably ought to fix.
Paul
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#164370):
https://lists.openembedded.org/g/openembedded-core/message/164370
Mute This Topic: https://lists.openembedded.org/mt/86165224/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH v2] create-spdx: add support for SDKs
On Thursday, 27 January 2022 07:16:48 NZDT Andres Beltran wrote: > Currently, SPDX SBOMs are only created for images. Add support for > SDKs. Fix json indent when outputting SBOMs for better readability. Hey folks, gentle ping on this patch - any comments? Thanks Paul -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#161634): https://lists.openembedded.org/g/openembedded-core/message/161634 Mute This Topic: https://lists.openembedded.org/mt/88703214/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [poky][master][PATCHv2] buildhistory.bbclass: Enable exporting more recipe and package data
On Thursday, 10 February 2022 04:34:16 NZDT Richard Purdie wrote: > On Wed, 2022-02-09 at 14:27 +, mikko.rap...@bmw.de wrote: > > On Wed, Feb 09, 2022 at 01:40:22PM +, Richard Purdie wrote: > > > On Wed, 2022-02-09 at 13:27 +, mikko.rap...@bmw.de wrote: > > > > Hi, > > > > > > > > On Wed, Feb 09, 2022 at 12:23:39PM +, Richard Purdie wrote: > > > > > People have requested changes like this before and I rejected it as > > > > > I'm worried that allowing people to customise this code will just > > > > > fork the project into many different directions. > > > > > > > > It's the other way round. There are a lot of needs to extract metadata > > > > from > > > > build system into something where reports can be generated. > > > > > > I don't doubt that however buildhistory was written for a specific > > > purpose and if we start adding the ability to customise it heavily we > > > lose the ability for comparisions to be made, or sstate reuse and so > > > on. > > > > > > I'm partly channelling the original author's views on this since they > > > had some very specific thoughts on this change. I do sometimes wonder > > > if I should continue doing that though :/. > > > > Then how should yocto users export CVE_NAME, LICENSE, PN, PV, SRC_URI etc > > from the build system to generate SW bill of materials (BOM) for their > > product and track progress? > > > > Yes, SPDX can be the other answer but I don't find that human readable or > > working out of the box atm. > > buildhistory was not intended for SBOM generation, that is what create-spdx > is being developed for. They have two quite different intentions and trying > to turn one into the other is why I have concerns about this patch. > > For example, of we did go this way, next, we may need to either write a > converter of buildhistory to SPDX format, or change buildhistory to use SPDX > format so that it has a standard SBOM output form. This is not the > direction we want/need to go. FWIW I'll just chime in here as the original author[1] and say I agree with Richard. If folks are needing an alternative SBOM generation mechanism to SPDX, or have other use cases for extracting build information, then I'd prefer we take a step back and design such a thing properly. The original idea was (1) to expose changes in the output that might not readily apparent otherwise, and (2) expose selected "input" values that would (or could) materially influence the output, so you hopefully have a ready explanation for why an image / package increased in size or dependencies got added etc. I will accept that over time buildhistory has added things here and there that further the idea of using it for SBOM, folks no doubt have been using these for such purposes, and I'm not 100% opposed to making small additions where they make sense. However, even with this proposed extension, buildhistory is still not capable of recording sufficient information that (I believe) is expected in a modern SBOM - for example, the list of patches applied / CVEs resolved as a result - and I don't think it would be wise to extend it to do so. We certainly don't want to give folks the idea that buildhistory is good enough to resolve their SBOM requirements when we haven't designed it to do so, particularly as for many folks there are legal and regulatory concerns involved. Cheers Paul [1] based upon the earlier testlab and packagehistory classes that were written by others -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#161633): https://lists.openembedded.org/g/openembedded-core/message/161633 Mute This Topic: https://lists.openembedded.org/mt/89018266/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [honister PATCH] kernel: introduce python3-dtschema-wrapper
On 14/01/2022 13:21, Paul Barker wrote: From: Bruce Ashfield The 5.16 kernel introduced mandatory schema checking on any dtb file built through the kernel. That funcionality is provided via python3-dt-schema. The dependencies to enable that functionality is not small, and may not always be desired (in particular on architectures that do not support dtbs, or in development cycles). It may also be useful for allowing a non-conformant dts to be compiled. This commit introduces a set of wrapper scripts that when added as a depenency to the kernel, can pass both the validation testing and validation steps of a dts. Signed-off-by: Bruce Ashfield Signed-off-by: Richard Purdie Backported from oe-core commit 2566563ad49d. Signed-off-by: Paul Barker I'm sending a gentle ping on this as I haven't heard any reply or seen this land in honister yet. Any feedback? Thanks, -- Paul Barker Principal Software Engineer SanCloud Ltd e: paul.bar...@sancloud.com w: https://sancloud.co.uk/ OpenPGP_0xA67255DFCCE62ECD.asc Description: OpenPGP public key OpenPGP_signature Description: OpenPGP digital signature -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#161121): https://lists.openembedded.org/g/openembedded-core/message/161121 Mute This Topic: https://lists.openembedded.org/mt/88420169/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [yocto] Inclusive Language Proposal for YP/OE
On Tuesday, 25 January 2022 21:26:33 NZDT Paul Barker wrote: > The layer index may need some modification to handle different layers > having different names for the in-development branch. We could implement > the layer index changes first to support other layers which rename their > master branch. > > I'm going to bite the bullet with meta-linux-mainline and rename the > master branch to "dev" this week to see what happens. The layer index does already support this, it's just not exposed (at the time I introduced it - some time ago - I thought it might encourage people to deviate from the release branch names, but in hindsight that was probably not worth worrying about.) So at the moment an admin can set an alternative branch name. It would not be hard to expose it through the UI for layer maintainers though. Cheers, Paul -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#160954): https://lists.openembedded.org/g/openembedded-core/message/160954 Mute This Topic: https://lists.openembedded.org/mt/88687808/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] Inclusive Language Proposal for YP/OE
toml-do-not-exit-on-panic.patch Also, there are a few others outside of OE that should probably be patched too. Branch Names The “master” branches on the relevant OpenEmbedded and Yocto Project git trees will be changed to an alternative name at some point in the future. The current preferred name is “devel”. There is no time table for this currently, and there is no obligation or requirement to change the branch name for any downstream project which is beyond the project’s remit. The layer index may need some modification to handle different layers having different names for the in-development branch. We could implement the layer index changes first to support other layers which rename their master branch. I'm going to bite the bullet with meta-linux-mainline and rename the master branch to "dev" this week to see what happens. Similarly, there is no need to change any recipes that are using a “master” branch as part of the SRC_URI. Those are outside the scope of YP/OE and this effort. Note These changes are only to bitbake and OE-Core. There is no requirement to change any other layers but we’d note consistency is encouraged and helpful to users. Helping If you would like to help, please put your name by the items in question on the inclusive language wiki page. https://wiki.yoctoproject.org/wiki/Inclusive_language Thanks Special thanks to Richard Purdie, Michael Opdenacker. Marta Rybczynska, Scott Murray, Jan-Simon Moeller, Saul Wold, and Armin Kuster for providing their time, technical details, text, and feedback on this task. Thanks all, it's great to see this moving forward! -- Paul Barker Principal Software Engineer SanCloud Ltd e: paul.bar...@sancloud.com w: https://sancloud.co.uk/ OpenPGP_0xA67255DFCCE62ECD.asc Description: OpenPGP public key OpenPGP_signature Description: OpenPGP digital signature -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#160912): https://lists.openembedded.org/g/openembedded-core/message/160912 Mute This Topic: https://lists.openembedded.org/mt/88650128/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH 1/2] kernel: Rework kernel make flag to variable mappings
On 14/10/2021 21:26, Richard Purdie wrote: In 2017 we added changes to pass the BUILD_CFLAGS into the kernel via BUILD_CC. This isn't really correct and the upstream kernel now has places to pass build cflags, ldflags and more. Update our kernel make flags to correctly use the kernel's variables. This addresses concerns raised by kernel developers. If this breaks some usecase please report it so we can work out how to fix it properly. FYI, this change back in Oct broke the build for Linux 5.10.y, 5.4.y, 4.19.y & 4.4.y kernels where CONFIG_UNWINDER_ORC=y (which is set by the in-tree defconfig for x86-64). The build failures sat on my backburner for a long time due to other commitments but I finally tracked them down to this commit and looked for solutions. For 5.10.y, 5.4.y & 4.19.y there is an upstream kernel patch which can be backported which will address this. See https://lore.kernel.org/stable/04598430-7383-b725-2f5f-3f2b16aac...@sancloud.com/ and replies for more info. Hopefully these branches will be fixed with the next round of stable kernel releases now the backport has been requested. For 4.14.y there is no trivial fix upstream. The backport I did for the other branches depends on the KBUILD_HOSTLDFLAGS variable being set in the kernel Makefile which was added after 4.14.y. Personally, I'm happy to leave 4.14.y broken in this case and workaround the issue in meta-linux-mainline by disabling CONFIG_UNWINDER_ORC for this series. I don't think any changes are needed in oe-core, the info is now here in case it affects others though. Thanks, -- Paul Barker Principal Software Engineer SanCloud Ltd e: paul.bar...@sancloud.com w: https://sancloud.co.uk/ -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#160569): https://lists.openembedded.org/g/openembedded-core/message/160569 Mute This Topic: https://lists.openembedded.org/mt/86324180/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [dunfell PATCH] kernel: introduce python3-dtschema-wrapper
From: Bruce Ashfield
The 5.16 kernel introduced mandatory schema checking on any dtb file
built through the kernel.
That funcionality is provided via python3-dt-schema.
The dependencies to enable that functionality is not small, and may
not always be desired (in particular on architectures that do not
support dtbs, or in development cycles). It may also be useful for
allowing a non-conformant dts to be compiled.
This commit introduces a set of wrapper scripts that when added
as a depenency to the kernel, can pass both the validation testing
and validation steps of a dts.
Signed-off-by: Bruce Ashfield
Signed-off-by: Richard Purdie
Backported from oe-core commit 2566563ad49d.
Signed-off-by: Paul Barker
---
meta/conf/distro/include/maintainers.inc | 1 +
.../python3-dtschema-wrapper/dt-doc-validate | 20 +++
.../dtc/python3-dtschema-wrapper/dt-mk-schema | 20 +++
.../dtc/python3-dtschema-wrapper/dt-validate | 20 +++
.../dtc/python3-dtschema-wrapper_2021.10.bb | 17
5 files changed, 78 insertions(+)
create mode 100644
meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-doc-validate
create mode 100644
meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-mk-schema
create mode 100644 meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-validate
create mode 100644 meta/recipes-kernel/dtc/python3-dtschema-wrapper_2021.10.bb
diff --git a/meta/conf/distro/include/maintainers.inc
b/meta/conf/distro/include/maintainers.inc
index 895cf894872b..1575fce8c715 100644
--- a/meta/conf/distro/include/maintainers.inc
+++ b/meta/conf/distro/include/maintainers.inc
@@ -576,6 +576,7 @@ RECIPE_MAINTAINER_pn-python3 = "Oleksandr Kravchuk
+# License: MIT (see COPYING.MIT at the root of the repository for terms)
+
+for arg; do
+case "$arg" in
+--version)
+echo "v2021.10"
+;;
+esac
+done
+
+# TBD: left for future consideration
+# exec dt-doc-validate.real "$@"
+
+# we always succeed
+exit 0
+
diff --git a/meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-mk-schema
b/meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-mk-schema
new file mode 100644
index ..24b89d861982
--- /dev/null
+++ b/meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-mk-schema
@@ -0,0 +1,20 @@
+#!/bin/sh
+# dt-mk-schema wrapper to allow kernel dt-validation to pass
+#
+# Copyright (C) 2021 Bruce Ashfield
+# License: MIT (see COPYING.MIT at the root of the repository for terms)
+
+for arg; do
+case "$arg" in
+--version)
+echo "v2021.10"
+;;
+esac
+done
+
+# TBD: left for future consideration
+# exec dt-mk-schema.real "$@"
+
+# we always succeed
+exit 0
+
diff --git a/meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-validate
b/meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-validate
new file mode 100644
index ..8a4710a7eda1
--- /dev/null
+++ b/meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-validate
@@ -0,0 +1,20 @@
+#!/bin/sh
+# dt-validate wrapper to allow kernel dt-validation to pass
+#
+# Copyright (C) 2021 Bruce Ashfield
+# License: MIT (see COPYING.MIT at the root of the repository for terms)
+
+for arg; do
+case "$arg" in
+--version)
+echo "v2021.10"
+;;
+esac
+done
+
+# TBD: left for future consideration
+# exec dt-validate.real "$@"
+
+# we always succeed
+exit 0
+
diff --git a/meta/recipes-kernel/dtc/python3-dtschema-wrapper_2021.10.bb
b/meta/recipes-kernel/dtc/python3-dtschema-wrapper_2021.10.bb
new file mode 100644
index ..c869274d09fa
--- /dev/null
+++ b/meta/recipes-kernel/dtc/python3-dtschema-wrapper_2021.10.bb
@@ -0,0 +1,17 @@
+DESCRIPTION = "Wrapper for tooling for devicetree validation using YAML and
jsonschema"
+HOMEPAGE = "https://yoctoproject.org";
+LICENSE = "MIT"
+LIC_FILES_CHKSUM =
"file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"
+
+SRC_URI = "file://dt-doc-validate \
+ file://dt-mk-schema \
+ file://dt-validate"
+
+do_install() {
+install -d ${D}${bindir}/
+install -m 755 ${WORKDIR}/dt-doc-validate ${D}${bindir}/
+install -m 755 ${WORKDIR}/dt-mk-schema ${D}${bindir}/
+install -m 755 ${WORKDIR}/dt-validate ${D}${bindir}/
+}
+
+BBCLASSEXTEND = "native nativesdk"
--
2.34.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#160567):
https://lists.openembedded.org/g/openembedded-core/message/160567
Mute This Topic: https://lists.openembedded.org/mt/88420183/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [honister PATCH] kernel: introduce python3-dtschema-wrapper
From: Bruce Ashfield
The 5.16 kernel introduced mandatory schema checking on any dtb file
built through the kernel.
That funcionality is provided via python3-dt-schema.
The dependencies to enable that functionality is not small, and may
not always be desired (in particular on architectures that do not
support dtbs, or in development cycles). It may also be useful for
allowing a non-conformant dts to be compiled.
This commit introduces a set of wrapper scripts that when added
as a depenency to the kernel, can pass both the validation testing
and validation steps of a dts.
Signed-off-by: Bruce Ashfield
Signed-off-by: Richard Purdie
Backported from oe-core commit 2566563ad49d.
Signed-off-by: Paul Barker
---
meta/conf/distro/include/maintainers.inc | 1 +
.../python3-dtschema-wrapper/dt-doc-validate | 20 +++
.../dtc/python3-dtschema-wrapper/dt-mk-schema | 20 +++
.../dtc/python3-dtschema-wrapper/dt-validate | 20 +++
.../dtc/python3-dtschema-wrapper_2021.10.bb | 17
5 files changed, 78 insertions(+)
create mode 100644
meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-doc-validate
create mode 100644
meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-mk-schema
create mode 100644 meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-validate
create mode 100644 meta/recipes-kernel/dtc/python3-dtschema-wrapper_2021.10.bb
diff --git a/meta/conf/distro/include/maintainers.inc
b/meta/conf/distro/include/maintainers.inc
index b3b7711a0c48..2b54d2d12f9b 100644
--- a/meta/conf/distro/include/maintainers.inc
+++ b/meta/conf/distro/include/maintainers.inc
@@ -592,6 +592,7 @@ RECIPE_MAINTAINER:pn-python3-cython = "Oleksandr Kravchuk
+# License: MIT (see COPYING.MIT at the root of the repository for terms)
+
+for arg; do
+case "$arg" in
+--version)
+echo "v2021.10"
+;;
+esac
+done
+
+# TBD: left for future consideration
+# exec dt-doc-validate.real "$@"
+
+# we always succeed
+exit 0
+
diff --git a/meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-mk-schema
b/meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-mk-schema
new file mode 100644
index ..24b89d861982
--- /dev/null
+++ b/meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-mk-schema
@@ -0,0 +1,20 @@
+#!/bin/sh
+# dt-mk-schema wrapper to allow kernel dt-validation to pass
+#
+# Copyright (C) 2021 Bruce Ashfield
+# License: MIT (see COPYING.MIT at the root of the repository for terms)
+
+for arg; do
+case "$arg" in
+--version)
+echo "v2021.10"
+;;
+esac
+done
+
+# TBD: left for future consideration
+# exec dt-mk-schema.real "$@"
+
+# we always succeed
+exit 0
+
diff --git a/meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-validate
b/meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-validate
new file mode 100644
index ..8a4710a7eda1
--- /dev/null
+++ b/meta/recipes-kernel/dtc/python3-dtschema-wrapper/dt-validate
@@ -0,0 +1,20 @@
+#!/bin/sh
+# dt-validate wrapper to allow kernel dt-validation to pass
+#
+# Copyright (C) 2021 Bruce Ashfield
+# License: MIT (see COPYING.MIT at the root of the repository for terms)
+
+for arg; do
+case "$arg" in
+--version)
+echo "v2021.10"
+;;
+esac
+done
+
+# TBD: left for future consideration
+# exec dt-validate.real "$@"
+
+# we always succeed
+exit 0
+
diff --git a/meta/recipes-kernel/dtc/python3-dtschema-wrapper_2021.10.bb
b/meta/recipes-kernel/dtc/python3-dtschema-wrapper_2021.10.bb
new file mode 100644
index ..c869274d09fa
--- /dev/null
+++ b/meta/recipes-kernel/dtc/python3-dtschema-wrapper_2021.10.bb
@@ -0,0 +1,17 @@
+DESCRIPTION = "Wrapper for tooling for devicetree validation using YAML and
jsonschema"
+HOMEPAGE = "https://yoctoproject.org";
+LICENSE = "MIT"
+LIC_FILES_CHKSUM =
"file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420"
+
+SRC_URI = "file://dt-doc-validate \
+ file://dt-mk-schema \
+ file://dt-validate"
+
+do_install() {
+install -d ${D}${bindir}/
+install -m 755 ${WORKDIR}/dt-doc-validate ${D}${bindir}/
+install -m 755 ${WORKDIR}/dt-mk-schema ${D}${bindir}/
+install -m 755 ${WORKDIR}/dt-validate ${D}${bindir}/
+}
+
+BBCLASSEXTEND = "native nativesdk"
--
2.34.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#160566):
https://lists.openembedded.org/g/openembedded-core/message/160566
Mute This Topic: https://lists.openembedded.org/mt/88420169/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH] classes/create-spdx: handle if IMAGE_LINK_NAME is empty
From: Paul Eggleton
If IMAGE_LINK_NAME is set to "" (supported in other classes) then skip
creating symlinks for the spdx manifest files.
Signed-off-by: Paul Eggleton
---
meta/classes/create-spdx.bbclass | 10 ++
1 file changed, 6 insertions(+), 4 deletions(-)
diff --git a/meta/classes/create-spdx.bbclass b/meta/classes/create-spdx.bbclass
index 0a4db80..e44a204 100644
--- a/meta/classes/create-spdx.bbclass
+++ b/meta/classes/create-spdx.bbclass
@@ -870,8 +870,9 @@ python image_combine_spdx() {
with image_spdx_path.open("wb") as f:
doc.to_json(f, sort_keys=True)
-image_spdx_link = imgdeploydir / (image_link_name + ".spdx.json")
-image_spdx_link.symlink_to(os.path.relpath(image_spdx_path,
image_spdx_link.parent))
+if image_link_name:
+image_spdx_link = imgdeploydir / (image_link_name + ".spdx.json")
+image_spdx_link.symlink_to(os.path.relpath(image_spdx_path,
image_spdx_link.parent))
num_threads = int(d.getVar("BB_NUMBER_THREADS"))
@@ -942,8 +943,9 @@ python image_combine_spdx() {
tar.addfile(info, fileobj=index_str)
def make_image_link(target_path, suffix):
-link = imgdeploydir / (image_link_name + suffix)
-link.symlink_to(os.path.relpath(target_path, link.parent))
+if image_link_name:
+link = imgdeploydir / (image_link_name + suffix)
+link.symlink_to(os.path.relpath(target_path, link.parent))
make_image_link(spdx_tar_path, ".spdx.tar.zst")
--
1.8.3.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#160318):
https://lists.openembedded.org/g/openembedded-core/message/160318
Mute This Topic: https://lists.openembedded.org/mt/88315088/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [FOSDEM standholders] FOSDEM 2022 stands - an update
Hi all, I'm unable to take the lead on organising a virtual stand for OpenEmbedded at FOSDEM 2022. I'm forwarding this mail to the lists to see if anyone else would like to take over - you'll need to review how the virtual stands work, update the website content as detailed below and be around during FOSDEM 2022 to "run" the stand. At least one commit to the website content needs to be made before the end of the year to confirm that OpenEmbedded will have a stand. The repositories we used last year were: * https://gitlab.com/openembedded/community/fosdem-stand-content * https://gitlab.com/openembedded/community/fosdem-stand-static I can grant permissions on those repositories to whoever needs them. I also recommend we email sta...@fosdem.org to let them know who will be running the stand this year so that the standholders mailing list can be updated. Thanks, Paul Forwarded Message Subject: [FOSDEM standholders] FOSDEM 2022 stands - an update Date: Mon, 20 Dec 2021 15:58:21 + From: Alasdair G Kergon To: standhold...@lists.fosdem.org We've decided to streamline the process for the 2022 event by reusing the system and data that is already in place. The 2021 stands website has been archived into archive.fosdem.org. stands.fosdem.org is now available to use again in 2022 and the direct links there from fosdem.org have been removed for now. The front page hasn't yet been updated. If you had a stand at FOSDEM 2021, and wish to have one at FOSDEM 2022, then all you need to do is to start updating your static content the same way as you did last year. If you don't want a stand, please send an email to sta...@fosdem.org and we will remove it. We will also remove stands that haven't had at least one change made by 31st December. PLEASE MAKE AT LEAST ONE COMMIT BEFORE 31ST DECEMBER. If you need metadata updating that you can't change yourself, either send pull requests againt the main repository or email sta...@fosdem.org. The system for videos/events is not ready yet and this won't be set up until the final list of stands is confirmed. New stands will be by invitation only and limited in number because of the amount of manual administration involved. If you think there's a deserving case for an invitation, contact sta...@fosdem.org. Alasdair ___ standholders mailing list standhold...@lists.fosdem.org https://lists.fosdem.org/listinfo/standholders -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#159960): https://lists.openembedded.org/g/openembedded-core/message/159960 Mute This Topic: https://lists.openembedded.org/mt/87899744/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core][PATCH 0/4] Move crate fetcher to OE_IMPORTS
On 20/12/2021 22:34, Richard Purdie wrote: On Mon, 2021-12-20 at 17:01 +, Paul Barker wrote: On 17/12/2021 15:36, Matt Madison wrote: This patch series switches to using the OE_IMPORTS mechanism for installing the crate fetcher. This simplifies the addition and clears up issues with interactions between SRCPV and the shared-state fetching logic. Matt Madison (4): lib/crate.py: install crate fetcher on import base.bbclass: add crate to OE_IMPORTS cargo_common.bbclass: remove inherit of crate-fetch classes: remove crate-fetch.bbclass meta/classes/base.bbclass | 2 +- meta/classes/cargo_common.bbclass | 2 -- meta/classes/crate-fetch.bbclass | 28 meta/lib/crate.py | 2 ++ 4 files changed, 3 insertions(+), 31 deletions(-) delete mode 100644 meta/classes/crate-fetch.bbclass Would a better solution here be to move the crate fetcher into lib/bb/fetch2/ in bitbake? I've been wondering about this too. Does anyone remember how the fetcher lines up against the general fetcher requirements we have and is there a list of anything that remained to be done before it could be added to bitbake? If I remember correctly, it just needed some test cases writing. The crate fetcher is a subclass of the wget fetcher with changes to how urldata is set up and how unpack is handled, so it should integrate well into bitbake. Thanks, -- Paul Barker Principal Software Engineer SanCloud Ltd e: paul.bar...@sancloud.com w: https://sancloud.co.uk/ -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#159920): https://lists.openembedded.org/g/openembedded-core/message/159920 Mute This Topic: https://lists.openembedded.org/mt/87791267/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH v2 3/3] classes/qemuboot: allow IMAGE_LINK_NAME to be empty
From: Paul Eggleton
If IMAGE_LINK_NAME is empty (supported everywhere else) then do not
create the symlink for the .qemuboot file.
Signed-off-by: Paul Eggleton
---
meta/classes/qemuboot.bbclass | 7 +--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/meta/classes/qemuboot.bbclass b/meta/classes/qemuboot.bbclass
index 8cdb544..95cd1d6 100644
--- a/meta/classes/qemuboot.bbclass
+++ b/meta/classes/qemuboot.bbclass
@@ -118,7 +118,10 @@ python do_write_qemuboot_conf() {
import configparser
qemuboot = "%s/%s.qemuboot.conf" % (d.getVar('IMGDEPLOYDIR'),
d.getVar('IMAGE_NAME'))
-qemuboot_link = "%s/%s.qemuboot.conf" % (d.getVar('IMGDEPLOYDIR'),
d.getVar('IMAGE_LINK_NAME'))
+if d.getVar('IMAGE_LINK_NAME'):
+qemuboot_link = "%s/%s.qemuboot.conf" % (d.getVar('IMGDEPLOYDIR'),
d.getVar('IMAGE_LINK_NAME'))
+else:
+qemuboot_link = ""
finalpath = d.getVar("DEPLOY_DIR_IMAGE")
topdir = d.getVar('TOPDIR')
cf = configparser.ConfigParser()
@@ -153,7 +156,7 @@ python do_write_qemuboot_conf() {
with open(qemuboot, 'w') as f:
cf.write(f)
-if qemuboot_link != qemuboot:
+if qemuboot_link and qemuboot_link != qemuboot:
if os.path.lexists(qemuboot_link):
os.remove(qemuboot_link)
os.symlink(os.path.basename(qemuboot), qemuboot_link)
--
1.8.3.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#159901):
https://lists.openembedded.org/g/openembedded-core/message/159901
Mute This Topic: https://lists.openembedded.org/mt/87866490/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH v2 0/3] Enable replacing image symlinks with unversioned files
This patchset puts in the remaining pieces to allow disabling symlinks
and then renaming the real files to avoid having the version in their
names, for example:
IMAGE_LINK_NAME = ""
IMAGE_VERSION_SUFFIX = ""
IMAGE_NAME_SUFFIX = ""
INITRAMFS_LINK_NAME = ""
KERNEL_ARTIFACT_NAME = "${MACHINE}"
KERNEL_ARTIFACT_LINK_NAME = ""
KERNEL_FIT_BIN_EXT = ""
KERNEL_IMAGETYPE_SYMLINK = "0"
Changes since v1 (a year ago!):
* Add KERNEL_IMAGETYPE_SYMLINK to allow explicitly disabling kernel
image type named symlinks (e.g. "bzImage")
* Add variables to allow changing/disabling kernel artifact .bin extension
* Fix unconditional IMAGE_LINK_NAME usage in qemuboot.bbclass
The following changes since commit a5459e42f1a6be9c08f303653cc1f73514eca9ef:
oeqa/selftest/bbtests: Use YP sources mirror instead of GNU (2021-12-20
15:28:57 +)
are available in the git repository at:
git://git.openembedded.org/openembedded-core-contrib paule/kernel-symlinks
http://cgit.openembedded.org/openembedded-core-contrib/log/?h=paule/kernel-symlinks
Paul Eggleton (3):
classes/kernel*: allow disabling symlink creation
classes/kernel*: add variables to allow changing artifact extension
classes/qemuboot: allow IMAGE_LINK_NAME to be empty
meta/classes/kernel-artifact-names.bbclass | 5 +
meta/classes/kernel-devicetree.bbclass | 24
meta/classes/kernel-fitimage.bbclass | 20 ++--
meta/classes/kernel.bbclass| 20 ++--
meta/classes/qemuboot.bbclass | 7 +--
5 files changed, 54 insertions(+), 22 deletions(-)
--
1.8.3.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#159900):
https://lists.openembedded.org/g/openembedded-core/message/159900
Mute This Topic: https://lists.openembedded.org/mt/87866489/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH v2 1/3] classes/kernel*: allow disabling symlink creation
From: Paul Eggleton
Allow setting any of the *_LINK_NAME variables to empty string in order
to disable creating symlinks for kernel artifacts, as you can already
for filesystem images with IMAGE_LINK_NAME. Additionally, for the image
type named symlinks, add a KERNEL_IMAGETYPE_SYMLINK boolean variable
which you can set to 0 to disable those symlinks as well.
Signed-off-by: Paul Eggleton
---
meta/classes/kernel-artifact-names.bbclass | 1 +
meta/classes/kernel-devicetree.bbclass | 20 ++--
meta/classes/kernel-fitimage.bbclass | 16
meta/classes/kernel.bbclass| 16
4 files changed, 39 insertions(+), 14 deletions(-)
diff --git a/meta/classes/kernel-artifact-names.bbclass
b/meta/classes/kernel-artifact-names.bbclass
index a65cddd..e1d943b 100644
--- a/meta/classes/kernel-artifact-names.bbclass
+++ b/meta/classes/kernel-artifact-names.bbclass
@@ -11,6 +11,7 @@ KERNEL_ARTIFACT_LINK_NAME ?= "${MACHINE}"
KERNEL_IMAGE_NAME ?= "${KERNEL_ARTIFACT_NAME}"
KERNEL_IMAGE_LINK_NAME ?= "${KERNEL_ARTIFACT_LINK_NAME}"
+KERNEL_IMAGETYPE_SYMLINK ?= "1"
KERNEL_DTB_NAME ?= "${KERNEL_ARTIFACT_NAME}"
KERNEL_DTB_LINK_NAME ?= "${KERNEL_ARTIFACT_LINK_NAME}"
diff --git a/meta/classes/kernel-devicetree.bbclass
b/meta/classes/kernel-devicetree.bbclass
index a50ea4f..c8aaf74 100644
--- a/meta/classes/kernel-devicetree.bbclass
+++ b/meta/classes/kernel-devicetree.bbclass
@@ -83,21 +83,29 @@ do_deploy:append() {
dtb_base_name=`basename $dtb .$dtb_ext`
install -d $deployDir
install -m 0644
${D}/${KERNEL_IMAGEDEST}/$dtb_base_name.$dtb_ext
$deployDir/$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext
- ln -sf $dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext
$deployDir/$dtb_base_name.$dtb_ext
- ln -sf $dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext
$deployDir/$dtb_base_name-${KERNEL_DTB_LINK_NAME}.$dtb_ext
+ if [ "${KERNEL_IMAGETYPE_SYMLINK}" = "1" ] ; then
+ ln -sf $dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext
$deployDir/$dtb_base_name.$dtb_ext
+ fi
+ if [ -n "${KERNEL_DTB_LINK_NAME}" ] ; then
+ ln -sf $dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext
$deployDir/$dtb_base_name-${KERNEL_DTB_LINK_NAME}.$dtb_ext
+ fi
for type in ${KERNEL_IMAGETYPE_FOR_MAKE}; do
if [ "$type" = "zImage" ] && [
"${KERNEL_DEVICETREE_BUNDLE}" = "1" ]; then
cat ${D}/${KERNEL_IMAGEDEST}/$type \
$deployDir/$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext \
>
$deployDir/$type-$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext.bin
- ln -sf
$type-$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext.bin \
-
$deployDir/$type-$dtb_base_name-${KERNEL_DTB_LINK_NAME}.$dtb_ext.bin
+ if [ -n "${KERNEL_DTB_LINK_NAME}" ]; then
+ ln -sf
$type-$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext.bin \
+
$deployDir/$type-$dtb_base_name-${KERNEL_DTB_LINK_NAME}.$dtb_ext.bin
+ fi
if [ -e
"${KERNEL_OUTPUT_DIR}/${type}.initramfs" ]; then
cat
${KERNEL_OUTPUT_DIR}/${type}.initramfs \
$deployDir/$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext \
>
$deployDir/${type}-${INITRAMFS_NAME}-$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext.bin
- ln -sf
${type}-${INITRAMFS_NAME}-$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext.bin \
-
$deployDir/${type}-${INITRAMFS_NAME}-$dtb_base_name-${KERNEL_DTB_LINK_NAME}.$dtb_ext.bin
+ if [ -n "${KERNEL_DTB_LINK_NAME}" ];
then
+ ln -sf
${type}-${INITRAMFS_NAME}-$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext.bin \
+
$deployDir/${type}-${INITRAMFS_NAME}-$dtb_base_name-${KERNEL_DTB_LINK_NAME}.$dtb_ext.bin
+ fi
fi
fi
done
diff --git a/meta/classes/kernel-fitimage.bbclass
b/meta/classes/kernel-fitimage.bbclass
index 8718ce7..11dce67 100644
--- a/meta/classes/kernel-fitimage.bbclass
+++ b/meta/classes/kernel-fitimage.bbclass
@@ -722,22 +722,30 @@ kernel_do_deploy:append() {
if [ "${INITRAMFS_IMAGE_BUNDLE}" != &qu
[OE-core] [PATCH v2 2/3] classes/kernel*: add variables to allow changing artifact extension
From: Paul Eggleton
Allow .bin suffix to be removed (or changed) in the various artifact
filenames. Removing this extension is useful when trying to remove
symlinks and present only unversioned image files (especially for the
FIT image).
Signed-off-by: Paul Eggleton
---
meta/classes/kernel-artifact-names.bbclass | 4
meta/classes/kernel-devicetree.bbclass | 12 ++--
meta/classes/kernel-fitimage.bbclass | 8
meta/classes/kernel.bbclass| 10 +-
4 files changed, 19 insertions(+), 15 deletions(-)
diff --git a/meta/classes/kernel-artifact-names.bbclass
b/meta/classes/kernel-artifact-names.bbclass
index e1d943b..e77107c 100644
--- a/meta/classes/kernel-artifact-names.bbclass
+++ b/meta/classes/kernel-artifact-names.bbclass
@@ -8,16 +8,20 @@ inherit image-artifact-names
KERNEL_ARTIFACT_NAME ?=
"${PKGE}-${PKGV}-${PKGR}-${MACHINE}${IMAGE_VERSION_SUFFIX}"
KERNEL_ARTIFACT_LINK_NAME ?= "${MACHINE}"
+KERNEL_ARTIFACT_BIN_EXT ?= ".bin"
KERNEL_IMAGE_NAME ?= "${KERNEL_ARTIFACT_NAME}"
KERNEL_IMAGE_LINK_NAME ?= "${KERNEL_ARTIFACT_LINK_NAME}"
+KERNEL_IMAGE_BIN_EXT ?= "${KERNEL_ARTIFACT_BIN_EXT}"
KERNEL_IMAGETYPE_SYMLINK ?= "1"
KERNEL_DTB_NAME ?= "${KERNEL_ARTIFACT_NAME}"
KERNEL_DTB_LINK_NAME ?= "${KERNEL_ARTIFACT_LINK_NAME}"
+KERNEL_DTB_BIN_EXT ?= "${KERNEL_ARTIFACT_BIN_EXT}"
KERNEL_FIT_NAME ?= "${KERNEL_ARTIFACT_NAME}"
KERNEL_FIT_LINK_NAME ?= "${KERNEL_ARTIFACT_LINK_NAME}"
+KERNEL_FIT_BIN_EXT ?= "${KERNEL_ARTIFACT_BIN_EXT}"
MODULE_TARBALL_NAME ?= "${KERNEL_ARTIFACT_NAME}"
MODULE_TARBALL_LINK_NAME ?= "${KERNEL_ARTIFACT_LINK_NAME}"
diff --git a/meta/classes/kernel-devicetree.bbclass
b/meta/classes/kernel-devicetree.bbclass
index c8aaf74..b4338da 100644
--- a/meta/classes/kernel-devicetree.bbclass
+++ b/meta/classes/kernel-devicetree.bbclass
@@ -93,18 +93,18 @@ do_deploy:append() {
if [ "$type" = "zImage" ] && [
"${KERNEL_DEVICETREE_BUNDLE}" = "1" ]; then
cat ${D}/${KERNEL_IMAGEDEST}/$type \
$deployDir/$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext \
- >
$deployDir/$type-$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext.bin
+ >
$deployDir/$type-$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext${KERNEL_DTB_BIN_EXT}
if [ -n "${KERNEL_DTB_LINK_NAME}" ]; then
- ln -sf
$type-$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext.bin \
-
$deployDir/$type-$dtb_base_name-${KERNEL_DTB_LINK_NAME}.$dtb_ext.bin
+ ln -sf
$type-$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext${KERNEL_DTB_BIN_EXT} \
+
$deployDir/$type-$dtb_base_name-${KERNEL_DTB_LINK_NAME}.$dtb_ext${KERNEL_DTB_BIN_EXT}
fi
if [ -e
"${KERNEL_OUTPUT_DIR}/${type}.initramfs" ]; then
cat
${KERNEL_OUTPUT_DIR}/${type}.initramfs \
$deployDir/$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext \
- >
$deployDir/${type}-${INITRAMFS_NAME}-$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext.bin
+ >
$deployDir/${type}-${INITRAMFS_NAME}-$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext${KERNEL_DTB_BIN_EXT}
if [ -n "${KERNEL_DTB_LINK_NAME}" ];
then
- ln -sf
${type}-${INITRAMFS_NAME}-$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext.bin \
-
$deployDir/${type}-${INITRAMFS_NAME}-$dtb_base_name-${KERNEL_DTB_LINK_NAME}.$dtb_ext.bin
+ ln -sf
${type}-${INITRAMFS_NAME}-$dtb_base_name-${KERNEL_DTB_NAME}.$dtb_ext${KERNEL_DTB_BIN_EXT}
\
+
$deployDir/${type}-${INITRAMFS_NAME}-$dtb_base_name-${KERNEL_DTB_LINK_NAME}.$dtb_ext${KERNEL_DTB_BIN_EXT}
fi
fi
fi
diff --git a/meta/classes/kernel-fitimage.bbclass
b/meta/classes/kernel-fitimage.bbclass
index 11dce67..b0c971b 100644
--- a/meta/classes/kernel-fitimage.bbclass
+++ b/meta/classes/kernel-fitimage.bbclass
@@ -727,9 +727,9 @@ kernel_do_deploy:append() {
fi
echo "Copying linux.bin file..."
- install -m 0644 ${B}/linux.bin
$deployDir/fitImage-linux.bin-${KER
Re: [OE-core][PATCH 0/4] Move crate fetcher to OE_IMPORTS
On 17/12/2021 15:36, Matt Madison wrote: This patch series switches to using the OE_IMPORTS mechanism for installing the crate fetcher. This simplifies the addition and clears up issues with interactions between SRCPV and the shared-state fetching logic. Matt Madison (4): lib/crate.py: install crate fetcher on import base.bbclass: add crate to OE_IMPORTS cargo_common.bbclass: remove inherit of crate-fetch classes: remove crate-fetch.bbclass meta/classes/base.bbclass | 2 +- meta/classes/cargo_common.bbclass | 2 -- meta/classes/crate-fetch.bbclass | 28 meta/lib/crate.py | 2 ++ 4 files changed, 3 insertions(+), 31 deletions(-) delete mode 100644 meta/classes/crate-fetch.bbclass Would a better solution here be to move the crate fetcher into lib/bb/fetch2/ in bitbake? -- Paul Barker Principal Software Engineer SanCloud Ltd e: paul.bar...@sancloud.com w: https://sancloud.co.uk/ OpenPGP_0xA67255DFCCE62ECD.asc Description: OpenPGP public key OpenPGP_signature Description: OpenPGP digital signature -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#159892): https://lists.openembedded.org/g/openembedded-core/message/159892 Mute This Topic: https://lists.openembedded.org/mt/87791267/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [dunfell][PATCH 00/14] Backport mandatory dtschema handling for device trees built through the kernel.
On 01/12/2021 21:33, Bruce Ashfield wrote: On Wed, Dec 1, 2021 at 3:34 PM Bhupesh Sharma wrote: This is a backport request for dunfell, while picking up only the skeletal support for allowing mandatory dtschema handling for device trees built through the kernel, introduced from kernel version 5.16 onwards. The problem with adding this support to dunfell, is obvious in the number of patches involved. We've also said/known that at some point that we won't be able to support all new kernel versions with the LTS release. This may be the start of drawing that line. Also, we've had to do some license tweaks in master just today for one of the added packages (idna), so that would be needed as well. The validation isn't absolutely critical, I'd suggest that just the wrappers and pkg-config fixes would be a smaller footprint to allow the kernel to build, without the new package/feature additions to the dunfell release. The new packages (and full validation) could still possibly be done through a secondary or mixin layer. I'm not sure which way to go on this, but I thought I'd offer those points for discussion. I've been thinking about this for meta-linux-mainline. Backporting python3-dtschema-wrapper seems like the way to go, either into a very small mixin layer or into oe-core. Thanks, -- Paul Barker e: p...@pbarker.dev w: https://pbarker.dev/ -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#159286): https://lists.openembedded.org/g/openembedded-core/message/159286 Mute This Topic: https://lists.openembedded.org/mt/87439224/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH 0/3] SPDX: Add annotations to relationship
On Tuesday, 9 November 2021 08:01:38 NZDT Saul Wold wrote: > On 11/4/21 2:20 PM, Joshua Watt wrote: > > On 11/4/21 3:50 PM, Richard Purdie wrote: > >> On Thu, 2021-11-04 at 15:45 -0500, Joshua Watt wrote: > >>> On 11/4/21 3:43 PM, Richard Purdie wrote: > >>>> On Thu, 2021-11-04 at 20:00 +, Jose Quaresma wrote: > >>>>> Richard Purdie escreveu no dia > >>>>> quinta, > >>>>> > >>>>> 28/10/2021 à(s) 21:58: > >>>>>> On Thu, 2021-10-28 at 08:47 -1000, Steve Sakoman wrote: > >>>>>>> On Tue, Oct 26, 2021 at 10:41 PM Jose Quaresma > >>>>>>> > >>>>>> > >>>>>> wrote: > >>>>>>>> Hi all, > >>>>>>>> > >>>>>>>> There are any plans or is it possible to backport the SBOM/SPDX > >>>>>>>> to the > >>>>>> > >>>>>> dunfell branch? > >>>>>> > >>>>>>> I'm going to yield to Saul as to whether he thinks this is > >>>>>>> desirable/possible or not. > >>>>>> > >>>>>> The packagedata changes are pretty invasive unfortunately and > >>>>>> likely not > >>>>>> something you're going to want in dunfell sadly. > >>>>> > >>>>> Thanks for the clarification. > >>>> > >>>> I have been thinking a bit more about this. I did wonder if we > >>>> should consider a > >>>> mixin layer of some kind for it that could work with dunfell? > >>>> > >>>> We could host it, it is just a question of writing the mixin layer and > >>>> maintaining it. > >>> > >>> I don't think it's going to be possible with a pure mixin layer, since > >>> it relies on the extended package data? > >> > >> I suspect that could perhaps be patched in through a layer though? You > >> might > >> choose to drop the compression piece or do it differently for the > >> backport? > > > > I'm not sure if a layer could hook in well enough to get the data > > needed... maybe worth an experiment though > > Yeah, I am not sure an mixin could track the changes for package.bbclass > > > With a backport, I would probably either use GZip compression or no > > compression. The zstd compression was designed as a drop in replacement > > for Gzip if we wanted to go that route. > > I will say that we did something similar with Hardknott for WRLinux, but > did not propose it upstream as Hardknott was knot going to be supported > longer term. > > Having the spdx class standalone with the correctly backported changes > seems to be working FYI Andres and I have done this backport to dunfell - should I post it? That said, I did just take the hit on some of the invasive parts (e.g. LICENSE value changes). I think given regulatory requirements this is important for lots of folks, so we probably need to do something here. Happy to be part of it. Cheers Paul -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#158316): https://lists.openembedded.org/g/openembedded-core/message/158316 Mute This Topic: https://lists.openembedded.org/mt/86616599/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH 18/65] u-boot: upgrade 2021.07 -> 2021.10
On Sunday, 24 October 2021 06:33:29 NZDT Alexander Kanavin wrote: > On Thu, 21 Oct 2021 at 03:14, Paul Eggleton > > wrote: > > Not sure what the official protocol for this kind of thing is, but > > meta-arm will > > naturally need an update for this (as seen via autobuilder warnings). > > Yes, I think it's usually done after the fact though. Absolutely, I just wanted to make a note of it ahead of time. Cheers Paul -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#157316): https://lists.openembedded.org/g/openembedded-core/message/157316 Mute This Topic: https://lists.openembedded.org/mt/86442662/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH 18/65] u-boot: upgrade 2021.07 -> 2021.10
On Wednesday, 20 October 2021 04:33:32 NZDT Alexander Kanavin wrote:
> From: Alexander Kanavin
>
> Signed-off-by: Alexander Kanavin
> ---
> ...ls-image-host-fix-wrong-return-value.patch | 41 ---
> meta/recipes-bsp/u-boot/u-boot-common.inc | 2 +-
> .../u-boot/u-boot-tools_2021.07.bb| 4 --
> .../u-boot/u-boot-tools_2021.10.bb| 2 +
> .../{u-boot_2021.07.bb => u-boot_2021.10.bb} | 1 +
Not sure what the official protocol for this kind of thing is, but meta-arm
will
naturally need an update for this (as seen via autobuilder warnings).
Cheers
Paul
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#157270):
https://lists.openembedded.org/g/openembedded-core/message/157270
Mute This Topic: https://lists.openembedded.org/mt/86442662/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [Hardknott backport request] kernel-yocto: Simplify no git repo case in do_kernel_checkout
Hi Anuj,
Please could you consider backporting the following commit to hardknott:
af2a9c92d449 ("kernel-yocto: Simplify no git repo case in do_kernel_checkout")
This has already been backport to dunfell. The patch cherry-picks
cleanly to hardknott.
Thanks,
--
Paul Barker
https://pbarker.dev/
pgpFyZGqNgYYi.pgp
Description: OpenPGP digital signature
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#155481):
https://lists.openembedded.org/g/openembedded-core/message/155481
Mute This Topic: https://lists.openembedded.org/mt/85246331/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH] buildhistory: Label packages providing per-file dependencies in depends.dot
On Thursday, 19 August 2021 05:17:26 NZST Andres Beltran wrote: > Currently, depends.dot includes per-file dependencies but not the packages > providing those files. This makes it hard to obtain all package > dependencies by just looking at depends.dot. > > Parse the RPROVIDES and FILERPROVIDES fields from pkgdata to map each of > their values to the package providing the component. Include runtime > packages as dependencies in depends.dot, together with the component > provided by the package as a label. > > Signed-off-by: Andres Beltran Acked-by: Paul Eggleton (FWIW) Cheers Paul -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#155382): https://lists.openembedded.org/g/openembedded-core/message/155382 Mute This Topic: https://lists.openembedded.org/mt/84977439/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH] buildhistory: Add output file listing package information
On Friday, 13 August 2021 04:58:56 NZST Andres Beltran wrote: > Currently, buildhistory does not produce a single file combining relevant > information of installed packages. Produce an output file > "installed-package-info.txt" listing a package's runtime name, buildtime > name, its recipe, version, and size to avoid having to look up each package > externally. Leave the existing package list files as-is for backwards > compatibility. > > In order to support this efficiently, extend oe-pkgdata-util to accept > multiple keys for its read-value argument. Acked-by: Paul Eggleton (FWIW) Cheers Paul -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#155381): https://lists.openembedded.org/g/openembedded-core/message/155381 Mute This Topic: https://lists.openembedded.org/mt/84844548/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH] ltp: backport ioctl_ns05 fix from upstream
This false positive keeps showing up in our testing but the fix isn't
yet a part of a tagged release, and it is probably too late for doing
an uprev for the fall release anyway.
Signed-off-by: Paul Gortmaker
diff --git
a/meta/recipes-extended/ltp/ltp/0001-syscalls-ioctl_ns05.c-ioctl_ns06.c-Fix-too-small-buf.patch
b/meta/recipes-extended/ltp/ltp/0001-syscalls-ioctl_ns05.c-ioctl_ns06.c-Fix-too-small-buf.patch
new file mode 100644
index ..08b88a38f3ac
--- /dev/null
+++
b/meta/recipes-extended/ltp/ltp/0001-syscalls-ioctl_ns05.c-ioctl_ns06.c-Fix-too-small-buf.patch
@@ -0,0 +1,59 @@
+From af2b6f5ee6b171078b18246dd73f71cf6e350859 Mon Sep 17 00:00:00 2001
+From: Marius Hillenbrand
+Date: Mon, 19 Jul 2021 13:58:35 +0800
+Subject: [PATCH] syscalls/ioctl_ns05.c, ioctl_ns06.c: Fix too small buffer for
+ path
+
+commit af2b6f5ee6b171078b18246dd73f71cf6e350859 upstream.
+
+Resize the buffer used for paths into /proc/ to grant enough space
+for long PIDs. While at it, replace sprintf with snprintf to avoid
+buffer overflows if we ever ran out of space again.
+
+Fixes: #847
+Signed-off-by: Marius Hillenbrand
+Reviewed-by: Yang Xu
+Upstream-Status: Backport
+Signed-off-by: Paul Gortmaker
+
+diff --git a/testcases/kernel/syscalls/ioctl/ioctl_ns05.c
b/testcases/kernel/syscalls/ioctl/ioctl_ns05.c
+index a67ddbe2c66f..52613810c7ce 100644
+--- a/testcases/kernel/syscalls/ioctl/ioctl_ns05.c
b/testcases/kernel/syscalls/ioctl/ioctl_ns05.c
+@@ -59,10 +59,10 @@ static void run(void)
+ if (pid == -1)
+ tst_brk(TBROK | TERRNO, "ltp_clone failed");
+
+- char child_namespace[20];
++ char child_namespace[30];
+ int my_fd, child_fd, parent_fd;
+
+- sprintf(child_namespace, "/proc/%i/ns/pid", pid);
++ snprintf(child_namespace, sizeof(child_namespace), "/proc/%i/ns/pid",
pid);
+ my_fd = SAFE_OPEN("/proc/self/ns/pid", O_RDONLY);
+ child_fd = SAFE_OPEN(child_namespace, O_RDONLY);
+ parent_fd = ioctl(child_fd, NS_GET_PARENT);
+diff --git a/testcases/kernel/syscalls/ioctl/ioctl_ns06.c
b/testcases/kernel/syscalls/ioctl/ioctl_ns06.c
+index b6ac80208d02..c30f7de91e09 100644
+--- a/testcases/kernel/syscalls/ioctl/ioctl_ns06.c
b/testcases/kernel/syscalls/ioctl/ioctl_ns06.c
+@@ -51,14 +51,14 @@ static int child(void *arg LTP_ATTRIBUTE_UNUSED)
+
+ static void run(void)
+ {
+- char child_namespace[20];
++ char child_namespace[30];
+
+ pid_t pid = ltp_clone(CLONE_NEWUSER | SIGCHLD, &child, 0,
+ STACK_SIZE, child_stack);
+ if (pid == -1)
+ tst_brk(TBROK | TERRNO, "ltp_clone failed");
+
+- sprintf(child_namespace, "/proc/%i/ns/user", pid);
++ snprintf(child_namespace, sizeof(child_namespace), "/proc/%i/ns/user",
pid);
+ int my_fd, child_fd, parent_fd;
+
+ my_fd = SAFE_OPEN("/proc/self/ns/user", O_RDONLY);
+--
+2.32.0
+
diff --git a/meta/recipes-extended/ltp/ltp_20210524.bb
b/meta/recipes-extended/ltp/ltp_20210524.bb
index 702fa62b660d..20e2deffa5aa 100644
--- a/meta/recipes-extended/ltp/ltp_20210524.bb
+++ b/meta/recipes-extended/ltp/ltp_20210524.bb
@@ -31,6 +31,7 @@ SRCREV = "0fb171f2beddaf64bd27597577c206c0f892b3cd"
SRC_URI = "git://github.com/linux-test-project/ltp.git \
file://0001-Remove-OOM-tests-from-runtest-mm.patch \
+
file://0001-syscalls-ioctl_ns05.c-ioctl_ns06.c-Fix-too-small-buf.patch \
"
S = "${WORKDIR}/git"
--
2.32.0
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#155263):
https://lists.openembedded.org/g/openembedded-core/message/155263
Mute This Topic: https://lists.openembedded.org/mt/85120082/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core][PATCH] 1/2] wic: Add --no-fstab-update part option
On Tue, 10 Aug 2021 15:06:02 +0200
Daniel Gomez wrote:
> When embedding a rootfs image (e.g. 'rootfs-dir') as a partition we might
> want to keep the stock fstab for that image. In such case, use
> this option to not update the fstab and use the stock one instead.
>
> This option allows to specify which partitions get the fstab updated
> and which get the stock fstab.
>
> The option matches the argument you can pass to wic itself where the
> same action is performed but for all the partitions.
>
> Example:
> part /export --source rootfs --rootfs-dir=hockeycam-image --fstype=ext4
> --label export --align 1024 --no-fstab-update
>
> part / --source rootfs --fstype=ext4 --label rootfs --align 1024
>
> Signed-off-by: Daniel Gomez
> ---
> scripts/lib/wic/help.py | 3 +++
> scripts/lib/wic/ksparser.py | 1 +
> scripts/lib/wic/partition.py | 5 +++--
> 3 files changed, 7 insertions(+), 2 deletions(-)
>
> diff --git a/scripts/lib/wic/help.py b/scripts/lib/wic/help.py
> index 991e5094bb..5c3d278d4e 100644
> --- a/scripts/lib/wic/help.py
> +++ b/scripts/lib/wic/help.py
> @@ -991,6 +991,9 @@ DESCRIPTION
> multiple partitions and we want to keep the
> right
> permissions and usernames in all the partitions.
>
> + --no-fstab-update: This option is specific to wic. It does not
> update the
> +'/etc/fstab' stock file for the given partition.
> +
> --extra-space: This option is specific to wic. It adds extra
> space after the space filled by the content
> of the partition. The final size can go
> diff --git a/scripts/lib/wic/ksparser.py b/scripts/lib/wic/ksparser.py
> index 7a4cc83af5..0df9eb0d05 100644
> --- a/scripts/lib/wic/ksparser.py
> +++ b/scripts/lib/wic/ksparser.py
> @@ -185,6 +185,7 @@ class KickStart():
> part.add_argument('--use-uuid', action='store_true')
> part.add_argument('--uuid')
> part.add_argument('--fsuuid')
> +part.add_argument('--no-fstab-update', action='store_true')
>
> bootloader = subparsers.add_parser('bootloader')
> bootloader.add_argument('--append')
> diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
> index e0b2c5bdf2..ab304f1b2a 100644
> --- a/scripts/lib/wic/partition.py
> +++ b/scripts/lib/wic/partition.py
> @@ -54,6 +54,7 @@ class Partition():
> self.uuid = args.uuid
> self.fsuuid = args.fsuuid
> self.type = args.type
> +self.no_fstab_update = args.no_fstab_update
> self.updated_fstab_path = None
> self.has_fstab = False
> self.update_fstab_in_rootfs = False
> @@ -286,7 +287,7 @@ class Partition():
> (self.fstype, extraopts, rootfs, label_str, self.fsuuid,
> rootfs_dir)
> exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
>
> -if self.updated_fstab_path and self.has_fstab:
> +if self.updated_fstab_path and self.has_fstab and not
> self.no_fstab_update:
> debugfs_script_path = os.path.join(cr_workdir, "debugfs_script")
> with open(debugfs_script_path, "w") as f:
> f.write("cd etc\n")
> @@ -350,7 +351,7 @@ class Partition():
> mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (rootfs, rootfs_dir)
> exec_native_cmd(mcopy_cmd, native_sysroot)
>
> -if self.updated_fstab_path and self.has_fstab:
> +if self.updated_fstab_path and self.has_fstab and not
> self.no_fstab_update:
> mcopy_cmd = "mcopy -i %s %s ::/etc/fstab" % (rootfs,
> self.updated_fstab_path)
> exec_native_cmd(mcopy_cmd, native_sysroot)
>
This looks like the right approach. However, you're still missing the
handling for filesystems other than ext2/3/4 & msdos. See
http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/scripts/lib/wic/plugins/source/rootfs.py#n218
and grep to see if there are any other uses of updated_fstab_path.
Thanks,
--
Paul Barker
https://pbarker.dev/
pgpEWdyh_IMLl.pgp
Description: OpenPGP digital signature
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154684):
https://lists.openembedded.org/g/openembedded-core/message/154684
Mute This Topic: https://lists.openembedded.org/mt/84792172/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH] pypi: Allow override of PyPI archive name
Some packages on PyPI don't follow the usual expectations for archive
naming. For example, the archive for asyncio-mqtt 0.10.0 is named
asyncio_mqtt-0.10.0.tar.gz (with an underscore instead of the dash used
in the package name).
To handle these edge cases a new PYPI_ARCHIVE_NAME variable is
introduced. By default this is set to the expected archive name based on
the PyPI package name, version and extension but it can be set to a
different value if needed in a recipe which inherits the pypi class.
Signed-off-by: Paul Barker
---
meta/classes/pypi.bbclass | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/meta/classes/pypi.bbclass b/meta/classes/pypi.bbclass
index 272c220bca..9405d58601 100644
--- a/meta/classes/pypi.bbclass
+++ b/meta/classes/pypi.bbclass
@@ -8,12 +8,12 @@ def pypi_package(d):
PYPI_PACKAGE ?= "${@pypi_package(d)}"
PYPI_PACKAGE_EXT ?= "tar.gz"
+PYPI_ARCHIVE_NAME ?= "${PYPI_PACKAGE}-${PV}.${PYPI_PACKAGE_EXT}"
def pypi_src_uri(d):
package = d.getVar('PYPI_PACKAGE')
-package_ext = d.getVar('PYPI_PACKAGE_EXT')
-pv = d.getVar('PV')
-return 'https://files.pythonhosted.org/packages/source/%s/%s/%s-%s.%s' %
(package[0], package, package, pv, package_ext)
+archive_name = d.getVar('PYPI_ARCHIVE_NAME')
+return 'https://files.pythonhosted.org/packages/source/%s/%s/%s' %
(package[0], package, archive_name)
PYPI_SRC_URI ?= "${@pypi_src_uri(d)}"
--
2.31.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154678):
https://lists.openembedded.org/g/openembedded-core/message/154678
Mute This Topic: https://lists.openembedded.org/mt/84790391/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH] u-boot: Package extlinux.conf separately
A separate u-boot-extlinux package is created for the extlinux.conf file
so that it can be installed on its own if needed. If this package is
populated, it is added as a dependency of the main u-boot package so
that installing just u-boot still results in the extlinux.conf file
being present in the rootfs.
Signed-off-by: Paul Barker
---
meta/recipes-bsp/u-boot/u-boot.inc | 5 -
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/meta/recipes-bsp/u-boot/u-boot.inc
b/meta/recipes-bsp/u-boot/u-boot.inc
index 4340b17cb6..971fdbb102 100644
--- a/meta/recipes-bsp/u-boot/u-boot.inc
+++ b/meta/recipes-bsp/u-boot/u-boot.inc
@@ -210,7 +210,7 @@ do_install () {
fi
}
-PACKAGE_BEFORE_PN += "${PN}-env"
+PACKAGE_BEFORE_PN += "${PN}-env ${PN}-extlinux"
RPROVIDES:${PN}-env += "u-boot-default-env"
ALLOW_EMPTY:${PN}-env = "1"
@@ -219,6 +219,9 @@ FILES:${PN}-env = " \
${sysconfdir}/fw_env.config \
"
+FILES:${PN}-extlinux =
"${UBOOT_EXTLINUX_INSTALL_DIR}/${UBOOT_EXTLINUX_CONF_NAME}"
+RDEPENDS:${PN} += "${@bb.utils.contains('UBOOT_EXTLINUX', '1',
'${PN}-extlinux', '', d)}"
+
FILES:${PN} = "/boot ${datadir}"
RDEPENDS:${PN} += "${PN}-env"
--
2.31.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154660):
https://lists.openembedded.org/g/openembedded-core/message/154660
Mute This Topic: https://lists.openembedded.org/mt/84776988/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core][RFC][PATCH][dunfell] wic: Add --use-stock-fstab option
On Fri, 6 Aug 2021 11:23:37 +0200
Daniel Gomez wrote:
> When embeddding rootfs image (e.g. 'rootfs-dir') as a partition we
> might want to keep the stock fstab for that image. In such case, use
> this option to not update the fstab and use the stock one instead.
>
> This option allows to specify which partitions get the updated fstab
> and which get the stock fstab.
>
> Example:
> part /export --source rootfs --rootfs-dir=hockeycam-image
> --fstype=ext4 --label export --align 1024 --use-stock-fstab
>
> part / --source rootfs --fstype=ext4 --label rootfs --align 1024
>
> Signed-off-by: Daniel Gomez
> ---
>
> The commit-id ce682a73b7447652f898ce1d1d0416a456df5416 reverted the
> functionality to restore the stock/original fstab file in a rootfs
> partition image (wic plugin).
>
> Add back such functionality with an option to the user to select which
> fstab will be used for each partition by adding the --use-stock-fstab.
> In this case, the user will select whether or not the stock fstab will
> be updated or kept.
>
> If you agree with the patch I'll adapt it to the master branch.
>
> Daniel
>
> scripts/lib/wic/ksparser.py | 1 +
> scripts/lib/wic/partition.py | 3 ++-
> 2 files changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/scripts/lib/wic/ksparser.py b/scripts/lib/wic/ksparser.py
> index 452a160232..64933e0a5b 100644
> --- a/scripts/lib/wic/ksparser.py
> +++ b/scripts/lib/wic/ksparser.py
> @@ -184,6 +184,7 @@ class KickStart():
> part.add_argument('--use-uuid', action='store_true')
> part.add_argument('--uuid')
> part.add_argument('--fsuuid')
> +part.add_argument('--use-stock-fstab', action='store_true')
>
> bootloader = subparsers.add_parser('bootloader')
> bootloader.add_argument('--append')
> diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
> index 85f9847047..c9e45773de 100644
> --- a/scripts/lib/wic/partition.py
> +++ b/scripts/lib/wic/partition.py
> @@ -54,6 +54,7 @@ class Partition():
> self.uuid = args.uuid
> self.fsuuid = args.fsuuid
> self.type = args.type
> +self.use_stock_fstab = args.use_stock_fstab
> self.updated_fstab_path = None
> self.has_fstab = False
> self.update_fstab_in_rootfs = False
> @@ -286,7 +287,7 @@ class Partition():
> (self.fstype, extraopts, rootfs, label_str, self.fsuuid,
> rootfs_dir)
> exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
>
> -if self.updated_fstab_path and self.has_fstab:
> +if self.updated_fstab_path and self.has_fstab and not
> self.use_stock_fstab:
> debugfs_script_path = os.path.join(cr_workdir, "debugfs_script")
> with open(debugfs_script_path, "w") as f:
> f.write("cd etc\n")
> --
> 2.30.2
>
You're only addressing ext2/3/4 partitions here. The patch needs to be
extended to cover the msdos partition case and the
update_fstab_in_rootfs case in the rootfs source plugin.
I also recommend changing the option in ksparser to '--no-fstab-update'
to match the argument you can pass to wic itself. That way you can pass
'--no-fstab-update' to wic to set the option for all partitions, or
include it in the wks file to set the option per-partition.
Thanks,
--
Paul Barker
https://pbarker.dev/
pgpnp1YzDVrHE.pgp
Description: OpenPGP digital signature
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154545):
https://lists.openembedded.org/g/openembedded-core/message/154545
Mute This Topic: https://lists.openembedded.org/mt/84704596/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH] kernel-yocto: Simplify no git repo case in do_kernel_checkout
If the kernel sources are not fetched via git, a local git repository is
created in do_kernel_checkout. In this case we know that there will be
no remote branches and we will already be on the correct branch (since
only one branch will exist). So we can simplify things by skipping these
steps.
This also removes the assumption that the default git branch name will
be "master". Prior to this change, the final git checkout command in
do_kernel_checkout could fail if a local git repo was created and the
user had changed init.defaultBranch in their gitconfig.
Signed-off-by: Paul Barker
---
meta/classes/kernel-yocto.bbclass | 30 +++---
1 file changed, 15 insertions(+), 15 deletions(-)
diff --git a/meta/classes/kernel-yocto.bbclass
b/meta/classes/kernel-yocto.bbclass
index 0df61cdef0..18b77e2b9f 100644
--- a/meta/classes/kernel-yocto.bbclass
+++ b/meta/classes/kernel-yocto.bbclass
@@ -358,6 +358,21 @@ do_kernel_checkout() {
fi
fi
cd ${S}
+
+ # convert any remote branches to local tracking ones
+ for i in `git branch -a --no-color | grep remotes | grep -v
HEAD`; do
+ b=`echo $i | cut -d' ' -f2 | sed 's%remotes/origin/%%'`;
+ git show-ref --quiet --verify -- "refs/heads/$b"
+ if [ $? -ne 0 ]; then
+ git branch $b $i > /dev/null
+ fi
+ done
+
+ # Create a working tree copy of the kernel by checking out a
branch
+ machine_branch="${@ get_machine_branch(d, "${KBRANCH}" )}"
+
+ # checkout and clobber any unimportant files
+ git checkout -f ${machine_branch}
else
# case: we have no git repository at all.
# To support low bandwidth options for building the kernel,
we'll just
@@ -379,21 +394,6 @@ do_kernel_checkout() {
git commit -q -m "baseline commit: creating repo for
${PN}-${PV}"
git clean -d -f
fi
-
- # convert any remote branches to local tracking ones
- for i in `git branch -a --no-color | grep remotes | grep -v HEAD`; do
- b=`echo $i | cut -d' ' -f2 | sed 's%remotes/origin/%%'`;
- git show-ref --quiet --verify -- "refs/heads/$b"
- if [ $? -ne 0 ]; then
- git branch $b $i > /dev/null
- fi
- done
-
- # Create a working tree copy of the kernel by checking out a branch
- machine_branch="${@ get_machine_branch(d, "${KBRANCH}" )}"
-
- # checkout and clobber any unimportant files
- git checkout -f ${machine_branch}
}
do_kernel_checkout[dirs] = "${S} ${WORKDIR}"
--
2.31.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154365):
https://lists.openembedded.org/g/openembedded-core/message/154365
Mute This Topic: https://lists.openembedded.org/mt/84608327/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH] linux-yocto: Fix devupstream SRCREV assignment
Fixes the following error when building linux-yocto-upstream: Fetcher failure: Unable to find revision 570e7aedcd00e7e8a218e37c3dfb3b1fda83d111 in branch v5.13/base even from upstream The error occurred as the SRCREV_machine assignment earlier in the recipe was being used instead of the mainline commit hash we want to use for linux-yocto-upstream. Signed-off-by: Paul Barker --- meta/recipes-kernel/linux/linux-yocto_5.13.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meta/recipes-kernel/linux/linux-yocto_5.13.bb b/meta/recipes-kernel/linux/linux-yocto_5.13.bb index df385f912c..f43e738c69 100644 --- a/meta/recipes-kernel/linux/linux-yocto_5.13.bb +++ b/meta/recipes-kernel/linux/linux-yocto_5.13.bb @@ -31,7 +31,7 @@ SRCREV_meta ?= "4dcaa4bcd042d398e210b702449dd35fdf9f0078" # normal PREFERRED_VERSION settings. BBCLASSEXTEND = "devupstream:target" DEFAULT_PREFERENCE_class-devupstream = "-1" -SRCREV_class-devupstream ?= "f86aa267e180b23d8d24ee6886bd3bf9a360112e" +SRCREV_machine_class-devupstream ?= "f86aa267e180b23d8d24ee6886bd3bf9a360112e" PN_class-devupstream = "linux-yocto-upstream" KBRANCH_class-devupstream = "v5.13/base" -- 2.26.2 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#154013): https://lists.openembedded.org/g/openembedded-core/message/154013 Mute This Topic: https://lists.openembedded.org/mt/84341857/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH] linux-yocto/5.13: add devupstream support
On Tue, 6 Jul 2021 13:58:21 -0400 "Bruce Ashfield" wrote: > From: Bruce Ashfield > > We are always getting questions about building -stable, or mainline > master. Rather than introducing a separate set of recipes, we can > facilitate this sort of testing by using the existing devupstream > bbclass support. > > To build an unpatched or otherwise modifed -stable of 5.13, simply > set the linux-yocto preferred version to: > > upstream+5.13% > > And your wish will be granted, as the build will use v5.13/base > which matches the main linux-yocto version, simply with no extra > changes applied. > > Signed-off-by: Bruce Ashfield This looks like a good thing to have in oe-core, in the basic case of just wanting the latest mainline as-is you'd not need to add extra layers any longer. meta-linux-mainline would still fill a niche as it provides recipes for older stable kernels and supports using mainline & newer stable kernels on dunfell & hardknott. I guess the meta repository (yocto-kernel-cache) would still be cloned and the upstream kernel could still be modified by patches or config fragments enabled by KERNEL_FEATURES. That may be worth noting in the documentation if I'm right. For my use-case where I want to use the exact upstream defconfig I would probably need to override this, e.g. `KERNEL_FEATURES_forcevariable = ""`, to make use of this. I also guess this would also allow you to follow newer kernel releases as they're added to linux-yocto recipes by setting the preferred version to 'upstream%' (or maybe 'upstream+%' to be more restrictive). That would then avoid the build breaking when the 5.13 recipe disappears and is replaced by 5.14 or something else later. Thanks, -- Paul Barker https://pbarker.dev/ pgpYwwMxtWjZZ.pgp Description: OpenPGP digital signature -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#153628): https://lists.openembedded.org/g/openembedded-core/message/153628 Mute This Topic: https://lists.openembedded.org/mt/84026594/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH 0/8] Safe PEP8
On Tue, 29 Jun 2021 at 17:19, Persian Prince wrote: > > @Paul Barker > > I spent days and nights for testing the outcome of my PEP8.sh script and I > used it on more than 100 repos to check the results, what you see now is the > result of that. > > People at https://github.com/OpenVisionE2 , https://github.com/OpenPLi , > https://github.com/oe-alliance and more other teams like ViX, ATV and ... > trusted my work and all those are using the changed codes for months! > > This is why I write "Safe PEP8" not just "PEP8" as if you just run a normal > PEP8 on your repos you'll end up with dead ones. > > If you check https://github.com/openembedded/meta-openembedded you're using > my changes for 3 months on this repo also so I don't get you being rude. Please start with https://docs.yoctoproject.org/dev-manual/common-tasks.html#making-changes-to-the-yocto-project which documents the way to make contributions to the project. If you're using an automated tool please give a quick description of it alongside the changes, preferably with a link to its source code. You may also want to check https://git.openembedded.org/openembedded-core/tree/scripts/oe-pylint. Running that script should flag up errors which we do want to see fixed. If there are more warnings & errors to consider then these should be added to that script or to a pylint/pep8/flake8 config file so that we can run this against all new patches in the future. -- Paul Barker Konsulko Group -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#153433): https://lists.openembedded.org/g/openembedded-core/message/153433 Mute This Topic: https://lists.openembedded.org/mt/83869185/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH 0/8] Safe PEP8
On Tue, 29 Jun 2021 at 15:08, Armin Kuster wrote: > > I am forwarding a pull request; > https://github.com/openembedded/openembedded-core/pull/71 Looking at the comments on that pull request, "I did this on more than 100 repos" suggests this hasn't been done with any attention to the specifics of openembedded-core/bitbake. In my opinion that approach is a waste of maintainer's time. -- Paul Barker Konsulko Group -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#153432): https://lists.openembedded.org/g/openembedded-core/message/153432 Mute This Topic: https://lists.openembedded.org/mt/83869185/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [swat] ltp failures on autobuilder
[Re: [swat] ltp failures on autobuilder] On 11/06/2021 (Fri 14:19) Richard Purdie wrote: > On Fri, 2021-06-11 at 12:36 +0100, Richard Purdie via lists.yoctoproject.org > wrote: > > as a .cfg to the kernel and that still reproduced the crash. However: > > > > CONFIG_DEBUG_KERNEL=y > > CONFIG_CGROUP_DEBUG=y > > CONFIG_SCHED_DEBUG=y > > CONFIG_DEBUG_PREEMPT=y > > # CONFIG_RCU_TRACE is not set > > # CONFIG_X86_DEBUG_FPU is not set > > # CONFIG_CONSOLE_POLL is not set > > # CONFIG_DEBUG_INFO is not set > > # CONFIG_KGDB is not set > > # CONFIG_KGDB_HONOUR_BLOCKLIST is not set > > # CONFIG_KGDB_SERIAL_CONSOLE is not set > > # CONFIG_KGDB_LOW_LEVEL_TRAP is not set > > # CONFIG_KGDB_KDB is not set > > # CONFIG_KDB_KEYBOARD is not set > > # CONFIG_DEBUG_MISC is not set > > > > Isolated down to CONFIG_SCHED_DEBUG=y being the line which somehow "fixes" > the crash. I can enable all the above apart from that and we can reproduce > it. > > Also, I changed gatesgarth to use qemu 5.2.0 copied in from hardknott and that > breaks it. Dropping the 27 CVE patches "fixes" it again. It is possible it > is one of the CVE fixes. Continuing to try and isolate. For the mail archive trail, and for those not follwing the ongoing research on IRC, we are hopeful that this fixes it. https://lore.kernel.org/lkml/20210616125157.438837-1-paul.gortma...@windriver.com/ Paul. -- > > Cheers, > > Richard > -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#153007): https://lists.openembedded.org/g/openembedded-core/message/153007 Mute This Topic: https://lists.openembedded.org/mt/83466238/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH] prservice: Use new connect API
The new prserv connect() function decouples the code in oe-core from the
exact classes and implementation details used within bitbake. This
allows us to more easily switch over to a new asyncrpc based prservice.
Signed-off-by: Paul Barker
---
meta/lib/oe/prservice.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/meta/lib/oe/prservice.py b/meta/lib/oe/prservice.py
index fcdbe66c19..15ce060ff6 100644
--- a/meta/lib/oe/prservice.py
+++ b/meta/lib/oe/prservice.py
@@ -7,7 +7,7 @@ def prserv_make_conn(d, check = False):
host_params = list([_f for _f in (d.getVar("PRSERV_HOST") or
'').split(':') if _f])
try:
conn = None
-conn = prserv.serv.PRServerConnection(host_params[0],
int(host_params[1]))
+conn = prserv.serv.connect(host_params[0], int(host_params[1]))
if check:
if not conn.ping():
raise Exception('service not available')
--
2.26.2
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#151059):
https://lists.openembedded.org/g/openembedded-core/message/151059
Mute This Topic: https://lists.openembedded.org/mt/82425692/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core][PATCH v2] scripts/contrib/image-manifest: add new script
On Friday, 16 April 2021 08:31:58 NZST Paul Eggleton wrote: > Hi folks > > On Friday, 9 April 2021 06:03:36 NZST Anders Wallin wrote: > > image-manifest: script to generate product/image specific BOM > > > > The image-manifest script generates image specific reports based on > > an image manifest file. Currently there is data generated by buildhistory, > > pkgdata, and license manifest but this data is poorly formated and spread > > across multiple text files. This script can generate a single JSON output > > file that is machine readable by other tools. > > > > The manifest-info collects package information and stores the information > > in a tarball. manifest-info can be configured using a json configuration > > file. The default configuration including all possible options can be > > dumped using the dump-config subcommand. > > > > image-manifest takes an image manifest file as input to get the runtime > > dependencies. As an option image-manifest can also use the build > > dependency > > file, pn-buildlist, to get the build dependencies excluding native > > packages. > > > > This script extends the oe-image-manifest script [0] done by Paul Eggleton > > > > [0] > > https://github.com/intel/clear-linux-dissector-web/blob/master/layerindex/ > > st atic/files/oe-image-manifest > > So I've thought some more about this. At minimum I think this script should > be under contrib/ Hmm, as Richard points out, it *is* already under contrib - I think I got that mixed up with another script I'd looked at separately. Still, my other comments stand. Paul -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#150585): https://lists.openembedded.org/g/openembedded-core/message/150585 Mute This Topic: https://lists.openembedded.org/mt/81949320/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core][PATCH v2] scripts/contrib/image-manifest: add new script
Hi folks On Friday, 9 April 2021 06:03:36 NZST Anders Wallin wrote: > image-manifest: script to generate product/image specific BOM > > The image-manifest script generates image specific reports based on > an image manifest file. Currently there is data generated by buildhistory, > pkgdata, and license manifest but this data is poorly formated and spread > across multiple text files. This script can generate a single JSON output > file that is machine readable by other tools. > > The manifest-info collects package information and stores the information > in a tarball. manifest-info can be configured using a json configuration > file. The default configuration including all possible options can be > dumped using the dump-config subcommand. > > image-manifest takes an image manifest file as input to get the runtime > dependencies. As an option image-manifest can also use the build dependency > file, pn-buildlist, to get the build dependencies excluding native > packages. > > This script extends the oe-image-manifest script [0] done by Paul Eggleton > > [0] > https://github.com/intel/clear-linux-dissector-web/blob/master/layerindex/st > atic/files/oe-image-manifest So I've thought some more about this. At minimum I think this script should be under contrib/, and looking at the command line options it seems like there's been a bit of scope creep such that it overlaps with other tools that we have. As a matter of best practice, for reporting / compliance usage I think getting this information should be something that is integrated into the build process (a la buildhistory) rather than something that you run afterwards - that way there is less chance that the information doesn't match up with the images produced. I wrote the original script so that it would work with older releases where the logic wouldn't have been able to be practically added that way without otherwise needing to patch the code, so it kind of made sense to have it as a separate script that you could run. All that is not to say this script *shouldn't* be used, I just think that it shouldn't be something we have front-and-centre. Cheers, Paul -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#150582): https://lists.openembedded.org/g/openembedded-core/message/150582 Mute This Topic: https://lists.openembedded.org/mt/81949320/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [oe-core][PATCH v3 1/2] valgrind: update 3.16.1 -> 3.17.0
Hi Yi Fan
On Saturday, 27 March 2021 03:37:09 NZDT Yi Fan Yu wrote:
> Notable changes:
> * library is now in libexecdir instead of libdir
>
> Added patches:
> * Add musl.supp: missing musl.supp in 3.17.0
>
> Dropped backport patches:
> * nlcontrolc: found in c79180a3afcf65902e578646c3b716cc749db406
> * drd Fedora33: found in 15330adf7c2471fbaa6a0818db07078d81dbff97
> * lmw lswi ppc64le: found in 74b74174d572fee4015b8f4e326db3cd949bcdc3
>
> Other dropped patches
> * helgrind intercept: found in d2d54dbcc74244adfc0c80b40862edf2b82f53b9
> * drd musl fix: found in d2d54dbcc74244adfc0c80b40862edf2b82f53b9
>
> TESTING RESULTS:
> qemux86-64:
> FAIL: drd/tests/swapcontext
>
> 3.17.0 3.16.1
> ===
> TOTAL: 736726
> PASSED: 694688
> FAILED: 1 0
> SKIPPED: 41 38
Can you take a look at the following ptest failure:
AssertionError: Failed ptests:
{'valgrind': ['callgrind/tests/simwork-branch']}
https://autobuilder.yoctoproject.org/typhoon/#/builders/82/builds/1657/steps/12/logs/stdio
(To be fair I'm not 100% sure if this only started happening after your patch
though.)
Thanks
Paul
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#150031):
https://lists.openembedded.org/g/openembedded-core/message/150031
Mute This Topic: https://lists.openembedded.org/mt/81629456/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core][dunfell V3 00/46] Patch review - pseudo file mode corruption fix V3
On Tue, 26 Jan 2021 at 14:42, Steve Sakoman wrote:
>
> Issues with undetected file mode corruption in pseudo have been identified.
>
> Fixes have been merged into master and gatesgarth over the past several
> months and
> things seem to have stabilized enough that we can consider backporting these
> fixes
> to dunfell.
>
> This is a somewhat more invasive change than normal. Specifically the user
> will
> be required to clean TMPDIR after this merge. It may also expose issues with
> other layers, though master and gategarth have probably helped to pave the
> way for
> this change.
>
> Personally I think that this fixes a serious enough issue that we should
> consider
> this series for dunfell.
>
> This third version removes:
>
> bitbake.conf: Remove TERM from default BB_HASHBASE_WHITELIST
>
> and hence the need for any bitbake changes.
>
> Let me know if you you are aware of any other pseudo issues which would
> necessitate
> delaying the merge of this series.
>
> Please have comments back by Thursday evening.
>
> Passed a-full on autobuilder:
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/1784
>
> The following changes since commit 897822233faef0f8f35dc1d8a39e1c4bc0550f1e:
>
> ppp: Whitelist CVE-2020-15704 (2021-01-19 04:22:10 -1000)
>
> are available in the Git repository at:
>
> git://git.openembedded.org/openembedded-core-contrib stable/dunfell-nut
>
> http://cgit.openembedded.org/openembedded-core-contrib/log/?h=stable/dunfell-nut
>
> Jack Mitchell (1):
> distutils3: allow setup.py to be run from a different directory to
> ${S}
>
> Joe Slater (1):
> pseudo: fix renaming to self
>
> Martin Jansa (2):
> base.bbclass: use os.path.normpath instead of just comparing WORKDIR
> and S as strings
> license.bbclass: Add COMMON_LICENSE_DIR and LICENSE_PATH dirs to
> PSEUDO_IGNORE_PATHS
>
> Mingli Yu (2):
> tcl: adapt to potential pseudo changes
> bitbake.conf: Exclude ${CCACHE_DIR} from pseudo database
>
> Paul Barker (8):
> bitbake.conf: Prevent pyc file generation in pseudo context
> wic: Add workdir argument
> wic: Allow exec_native_cmd to run HOSTTOOLS
> wic: Ensure internal workdir is not reused
> image_types_wic: Move wic working directory
> wic: Update pseudo db when excluding content from rootfs
> wic: Copy rootfs dir if fstab needs updating
> wic: Optimise fstab modification for ext2/3/4 and msdos partitions
>
> Peter Kjellerstedt (4):
> pseudo: Simplify pseudo_client_ignore_path_chroot()
> lib/oe/path: Add canonicalize()
> bitbake.conf: Canonicalize paths in PSEUDO_IGNORE_PATHS
> wic: Pass canonicalized paths in PSEUDO_IGNORE_PATHS
>
> Ricardo Ribalda Delgado (4):
> wic: Fix permissions when using exclude or include path
> wic: Fix multi images .wks with bitbake
> wic: Avoid creating invalid pseudo directory
> wic: Add --change-directory argument
>
> Richard Purdie (23):
> pseudo: Switch to oe-core branch in git repo
> pseudo: merge in fixes for setfacl issue
> pseudo: Update to add OFC fcntl lock updates
> pseudo: Ignore mismatched inodes from the db
> pseudo: Add support for ignoring paths from the pseudo DB
> pseudo: Abort on mismatch patch
> psuedo: Add tracking of linked files for fds
> pseudo: Fix xattr segfault
> pseudo: Add may unlink patch
> pseudo: Add pathfix patch
> base/bitbake.conf: Enable pseudo path filtering
> wic: Handle new PSEUDO_IGNORE_PATHS variable
> pseudo: Fix statx function usage
> bitbake.conf: Extend PSEUDO_IGNORE_PATHS to ${COREBASE}/meta
> abi_version,sanity: Tell users TMPDIR must be clean after pseudo
> changes
> pseudo: Update to account for patches merged on branch
> pseudo: Upgrade to include mkostemp64 wrapper
> oeqa/selftest/runtime_test: Exclude gpg directory from pseudo database
> uninative: Don't use single sstate for pseudo-native
> pseudo: Drop patches merged into upstream branch
> bitbake.conf: Add /run/ to PSEUDO_IGNORE_PATHS
> pseudo: Add lchmod wrapper
> pseudo: Update for arm host and memleak fixes/cleanup
>
> Tomasz Dziendzielski (1):
> pseudo: Update to print PSEUDO_LOGFILE in abort message on path
> mismatches
>
> meta-selftest/lib/pseudo_pyc_test1.py | 1 +
> meta-selftest/lib/pseudo_pyc_test2.py | 1 +
> .../pseudo-pyc-test/pseudo-pyc-test.bb| 15 ++
> meta/classes/archiver.bbclass | 2 +-
> meta/classes/base.bbclass | 6 +
> meta/classes/distutils3.bbclass | 10 +-
> meta/classes/image_types_wic.bbclass | 20 ++-
> meta/classes/license.
Re: [OE-core][dunfell 00/47] Patch review - pseudo file mode corruption fix V2
On Mon, 25 Jan 2021 at 16:25, Steve Sakoman wrote: > > On Mon, Jan 25, 2021 at 5:42 AM Paul Barker wrote: > > > > On Mon, 25 Jan 2021 at 14:51, Steve Sakoman wrote: > > > > > > Issues with undetected file mode corruption in pseudo have been > > > identified. > > > > > > Fixes have been merged into master and gatesgarth over the past several > > > months and > > > things seem to have stabilized enough that we can consider backporting > > > these fixes > > > to dunfell. > > > > > > This is a somewhat more invasive change than normal. Specifically the > > > user will > > > be required to clean TMPDIR after this merge. It may also expose issues > > > with > > > other layers, though master and gategarth have probably helped to pave > > > the way for > > > this change. > > > > > > Personally I think that this fixes a serious enough issue that we should > > > consider > > > this series for dunfell. > > > > > > This second version includes the wic patches which have been the gating > > > item for > > > merging this series. > > > > > > Note also that there is an associated bitbake patch for stable/1.46: > > > > > > lib/bb/utils.py: Do not preserve TERM in the environment > > > > > > This is due to taking: > > > > > > bitbake.conf: Remove TERM from default BB_HASHBASE_WHITELIST > > > > > > which was needed to do clean cherry-picks of the pseudo changes. > > > > If this is just brought in to allow clean cherry-picks I think it > > would be better left out and instead do the small bit of rebasing > > required in the patches. We don't know what downstream users are doing > > with TERM so minimising unnecessary changes in the stable branch is > > preferable. > > OK, will do. > > > Everything else looks good to me at a glance. > > Thanks for reviewing. I'll wait for more feedback and send a v3. > This is one series we really want to get right! Agreed. It's a difficult balance. I want stable branches to be as conservative as possible and not risk making breaking changes but this series fixes an important issue (risk of silently creating images with incorrect permissions). On balance it does make sense to backport these pretty invasive changes. Thanks, -- Paul Barker Konsulko Group -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#147219): https://lists.openembedded.org/g/openembedded-core/message/147219 Mute This Topic: https://lists.openembedded.org/mt/80104914/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core][dunfell 00/47] Patch review - pseudo file mode corruption fix V2
On Mon, 25 Jan 2021 at 14:51, Steve Sakoman wrote: > > Issues with undetected file mode corruption in pseudo have been identified. > > Fixes have been merged into master and gatesgarth over the past several > months and > things seem to have stabilized enough that we can consider backporting these > fixes > to dunfell. > > This is a somewhat more invasive change than normal. Specifically the user > will > be required to clean TMPDIR after this merge. It may also expose issues with > other layers, though master and gategarth have probably helped to pave the > way for > this change. > > Personally I think that this fixes a serious enough issue that we should > consider > this series for dunfell. > > This second version includes the wic patches which have been the gating item > for > merging this series. > > Note also that there is an associated bitbake patch for stable/1.46: > > lib/bb/utils.py: Do not preserve TERM in the environment > > This is due to taking: > > bitbake.conf: Remove TERM from default BB_HASHBASE_WHITELIST > > which was needed to do clean cherry-picks of the pseudo changes. If this is just brought in to allow clean cherry-picks I think it would be better left out and instead do the small bit of rebasing required in the patches. We don't know what downstream users are doing with TERM so minimising unnecessary changes in the stable branch is preferable. Everything else looks good to me at a glance. -- Paul Barker Konsulko Group -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#147217): https://lists.openembedded.org/g/openembedded-core/message/147217 Mute This Topic: https://lists.openembedded.org/mt/80104914/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [yocto] OpenEmbedded Virtual Stand at FOSDEM 2021
On Thu, 14 Jan 2021 at 11:02, Robert ber...@yocto.user wrote: > > Hi, > > On 13/01/2021 23:33, Paul Barker wrote: > > Hi folks, > > > > * Any participants in the project who want to help host the Matrix > > chat room between 09:00 and 18:00 each day of the event. This will > > likely involve introducing the project to folks dropping in the chat > > who aren't familiar with OpenEmbedded, answering basic questions and > > chatting about example uses of the project. You don't need to be a > > long-standing expert in the project to help out here! If you can do a > > couple of hours or a half day please let us know. > > Let's try to come up with some schedule when I should be there and just > let me know how the join the chat. > > > > > * Any contributions of video content to go along with the static web > > pages. I'm planning to record some short introductory video content > > but other contributions would also be welcome. Details on how to > > upload videos is expected in the near future but for now it would be > > good to just collect folks who are interested so we can discuss this > > further. > > > > If you're interested in any of the above please reply to me and/or the > > list. I look forward to virtually seeing many of you at FOSDEM 2021! > > What kind of video would you like? A small video which explains a > specific topic? > > Something like this?[1] > > [1] https://www.youtube.com/watch?v=kmpEN953pzs&feature=youtu.be I'd say video content for FOSDEM should be around the level of the things we'd show on the stand, short demos or introductions which can be understood by people not too familiar with the details of the project. I'm going to record a short "Introducing OpenEmbedded" presentation based around the way I usually pitch the project to folks who turn up at our FOSDEM stand and ask what the project is about. I may also do a quick build & boot demo with a Raspberry Pi if I have time to record that. Thanks, -- Paul Barker Konsulko Group -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#147124): https://lists.openembedded.org/g/openembedded-core/message/147124 Mute This Topic: https://lists.openembedded.org/mt/80031982/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH v4] util-linux: split uuid in separate recipe to allow bootstrapping
On Wednesday, 20 January 2021 12:23:34 NZDT Richard Purdie wrote: > On Wed, 2021-01-20 at 12:13 +1300, Paul Eggleton wrote: > > On Wednesday, 20 January 2021 09:52:41 NZDT Richard Purdie wrote: > > > I think the one remaining issue here is the need to change the DEPENDS > > > of so many other recipes, likely not just here in this patch but in > > > other layers. I think if util-linux DEPENDS on util-linux-uuid that > > > might remove the need for those changes? That should still allow you to > > > break the circular dependency problem? > > > > I have to admit to a gap in my own knowledge of how our build system > > handles transitive dependencies. Of course the recipe sysroot should > > still get everything it needs in it even if the dependency is only > > indirectly included, in the back of my mind I have the impression that > > there are expectations that all dependencies are explicitly called out > > and there are subtle issues if they aren't, but that could be a mistaken > > impression on my part. > > I do wonder a little about that as well. As you say, sysroot > dependencies should handle this. Anything linking against libuuid > should also establish an package level runtime dependency through the > linkage so I think this should work. > > We definitely don't explicitly list every dependency in every recipe. > > If this can work, it makes the migration path for people easier so I > think its at least worth investigating/testing. OK, sure thing. > Just while I'm thinking, the PACKAGES_remove also bothers me a little. > Can we rearrange the variables so libuuid is only added in the libuuid > recipe variant? > > [the idea being that since we control the metadata in oe-core, we > shouldn't need to use _remove and can restructure so we don't need to, > they're hard to undo. I know we do use it in places sadly even in core] Yes, that is a good point. We should be able to avoid using _remove and I don't like using it either given how heavy a hammer it is. > > I agree that it would be better being separate, FWIW. > > > > > I am also worried this is going to break AUH and mean we have to > > > manually handle this recipe but again, I suspect there is little to be > > > done and we just have to deal with it. > > > > Could we perhaps fix the AUH to handle this properly? Do we need some kind > > of mechanism to get it to always upgrade the two recipes together or is > > that only part of the issue? > > I don't know for sure that AUH won't handle it, I just worry about it. > If it doesn't it definitely could be something we could fix there. I > just don't know of anyone with the time to spend on what is a marginal > corner case if it doesn't work. Well, on the other hand if we're "causing" a problem with the AUH with this change it does behoove us to try to resolve that. It is something I'd be prepared to look into at least. Cheers Paul -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#147012): https://lists.openembedded.org/g/openembedded-core/message/147012 Mute This Topic: https://lists.openembedded.org/mt/78861332/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH v4] util-linux: split uuid in separate recipe to allow bootstrapping
Hi Richard On Wednesday, 20 January 2021 09:52:41 NZDT Richard Purdie wrote: > I think the one remaining issue here is the need to change the DEPENDS > of so many other recipes, likely not just here in this patch but in > other layers. I think if util-linux DEPENDS on util-linux-uuid that > might remove the need for those changes? That should still allow you to > break the circular dependency problem? I have to admit to a gap in my own knowledge of how our build system handles transitive dependencies. Of course the recipe sysroot should still get everything it needs in it even if the dependency is only indirectly included, in the back of my mind I have the impression that there are expectations that all dependencies are explicitly called out and there are subtle issues if they aren't, but that could be a mistaken impression on my part. > I suspect libuuid should really be maintained/built as a separate > software project given the dependency problems but that isn't my > decision, we just have to deal with it. I agree that it would be better being separate, FWIW. > I am also worried this is going to break AUH and mean we have to > manually handle this recipe but again, I suspect there is little to be > done and we just have to deal with it. Could we perhaps fix the AUH to handle this properly? Do we need some kind of mechanism to get it to always upgrade the two recipes together or is that only part of the issue? Cheers Paul -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#147010): https://lists.openembedded.org/g/openembedded-core/message/147010 Mute This Topic: https://lists.openembedded.org/mt/78861332/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH v2 5/5] wic: Optimise fstab modification for ext2/3/4 and msdos partitions
The fix for [Yocto #13994] required the rootfs directory to be copied
(using hardlinks if possible) when modifying the fstab file under wic.
We can optimise this copy away for filesystems where we have the tools
to modify the contents of the partition image after it is created. For
ext2/3/4 filesystems we have the debugfs tool and for msdos/vfat
filesystems we have the mcopy tool. So for any of these filesystems we
skip the modification of the fstab file in the rootfs directory (and
skip the associated copy unless it is otherwise necessary) and update
the contents of fstab directly in the partition image.
Signed-off-by: Paul Barker
---
scripts/lib/wic/partition.py | 27 +++-
scripts/lib/wic/plugins/source/rootfs.py | 9 +---
2 files changed, 28 insertions(+), 8 deletions(-)
diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
index f59eceb23d..e574f40c47 100644
--- a/scripts/lib/wic/partition.py
+++ b/scripts/lib/wic/partition.py
@@ -55,6 +55,8 @@ class Partition():
self.fsuuid = args.fsuuid
self.type = args.type
self.updated_fstab_path = None
+self.has_fstab = False
+self.update_fstab_in_rootfs = False
self.lineno = lineno
self.source_file = ""
@@ -125,6 +127,8 @@ class Partition():
partition command parameters.
"""
self.updated_fstab_path = updated_fstab_path
+if self.updated_fstab_path and not (self.fstype.startswith("ext") or
self.fstype == "msdos"):
+self.update_fstab_in_rootfs = True
if not self.source:
if not self.size and not self.fixed_size:
@@ -250,7 +254,7 @@ class Partition():
prefix = "ext" if self.fstype.startswith("ext") else self.fstype
method = getattr(self, "prepare_rootfs_" + prefix)
-method(rootfs, oe_builddir, rootfs_dir, native_sysroot, pseudo)
+method(rootfs, cr_workdir, oe_builddir, rootfs_dir, native_sysroot,
pseudo)
self.source_file = rootfs
# get the rootfs size in the right units for kickstart (kB)
@@ -258,7 +262,7 @@ class Partition():
out = exec_cmd(du_cmd)
self.size = int(out.split()[0])
-def prepare_rootfs_ext(self, rootfs, oe_builddir, rootfs_dir,
+def prepare_rootfs_ext(self, rootfs, cr_workdir, oe_builddir, rootfs_dir,
native_sysroot, pseudo):
"""
Prepare content for an ext2/3/4 rootfs partition.
@@ -282,10 +286,19 @@ class Partition():
(self.fstype, extraopts, rootfs, label_str, self.fsuuid,
rootfs_dir)
exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
+if self.updated_fstab_path and self.has_fstab:
+debugfs_script_path = os.path.join(cr_workdir, "debugfs_script")
+with open(debugfs_script_path, "w") as f:
+f.write("cd etc\n")
+f.write("rm fstab\n")
+f.write("write %s fstab\n" % (self.updated_fstab_path))
+debugfs_cmd = "debugfs -w -f %s %s" % (debugfs_script_path, rootfs)
+exec_native_cmd(debugfs_cmd, native_sysroot)
+
mkfs_cmd = "fsck.%s -pvfD %s" % (self.fstype, rootfs)
exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
-def prepare_rootfs_btrfs(self, rootfs, oe_builddir, rootfs_dir,
+def prepare_rootfs_btrfs(self, rootfs, cr_workdir, oe_builddir, rootfs_dir,
native_sysroot, pseudo):
"""
Prepare content for a btrfs rootfs partition.
@@ -308,7 +321,7 @@ class Partition():
self.mkfs_extraopts, self.fsuuid, rootfs)
exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
-def prepare_rootfs_msdos(self, rootfs, oe_builddir, rootfs_dir,
+def prepare_rootfs_msdos(self, rootfs, cr_workdir, oe_builddir, rootfs_dir,
native_sysroot, pseudo):
"""
Prepare content for a msdos/vfat rootfs partition.
@@ -337,12 +350,16 @@ class Partition():
mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (rootfs, rootfs_dir)
exec_native_cmd(mcopy_cmd, native_sysroot)
+if self.updated_fstab_path and self.has_fstab:
+mcopy_cmd = "mcopy -i %s %s ::/etc/fstab" % (rootfs,
self.updated_fstab_path)
+exec_native_cmd(mcopy_cmd, native_sysroot)
+
chmod_cmd = "chmod 644 %s" % rootfs
exec_cmd(chmod_cmd)
prepare_rootfs_vfat = prepare_rootfs_msdos
-def prepare_rootfs_squashfs(self, rootfs, oe_builddir, rootfs_dir,
+def prepare_rootfs_squashfs(self, rootfs, cr_workdir, oe_builddir,
rootfs_dir,
native_sysroot, pseudo):
"""
Prepare content
[OE-core] [PATCH v2 4/5] wic: Copy rootfs dir if fstab needs updating
By default, wic updates the /etc/fstab in the rootfs to include details
of additional partitions described in the selected wks file. If this
modification is performed in place, other tasks which create an image
file from the rootfs directory (e.g. do_image_tar and do_image_ext4)
will pick up the modified fstab file which would not be appropriate for
those images as they do not include the additional partitions described
in the wks file. wic does undo modifications to the fstab file once it
has finished creating the filesystem image, however this leaves open a
race condition if one of the other tasks reads the contents of the fstab
file from the rootfs directory between the point where wic modifies the
fstab file and the point where wic restores the files original content.
This could be solved by adding a lockfile for tasks which use the rootfs
directory to ensure that no other such task is reading the rootfs
directory while do_image_wic is running. This would serialize several
do_image_* tasks and result in slower builds, especially for large
images. Another drawback of this solution is that it is hard to
selectively optimise - adding lockfiles to do_image_* tasks would result
in these tasks always being serialized even if no fstab modification
will take place.
An alternative solution is to copy the rootfs directory when fstab needs
to be modified. The code to do this in wic already exists as it is
needed when including or excluding content in the rootfs. This still
results in an impact on build times but the copy uses hardlinks if
possible (so little data is actually copied) and we can make selective
optimisations to improve things. The rootfs copy will only take place if
fstab modification is required (or if it was already needed to include
or exclude rootfs content). We can also follow up with further
optimisations after this commit. So this second solution is chosen.
Fixes [Yocto #13994]
Signed-off-by: Paul Barker
---
scripts/lib/wic/partition.py | 5 +++-
scripts/lib/wic/plugins/imager/direct.py | 36
scripts/lib/wic/plugins/source/rootfs.py | 17 +--
3 files changed, 29 insertions(+), 29 deletions(-)
diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
index 286c7867cb..f59eceb23d 100644
--- a/scripts/lib/wic/partition.py
+++ b/scripts/lib/wic/partition.py
@@ -54,6 +54,7 @@ class Partition():
self.uuid = args.uuid
self.fsuuid = args.fsuuid
self.type = args.type
+self.updated_fstab_path = None
self.lineno = lineno
self.source_file = ""
@@ -118,11 +119,13 @@ class Partition():
return self.fixed_size if self.fixed_size else self.size
def prepare(self, creator, cr_workdir, oe_builddir, rootfs_dir,
-bootimg_dir, kernel_dir, native_sysroot):
+bootimg_dir, kernel_dir, native_sysroot, updated_fstab_path):
"""
Prepare content for individual partitions, depending on
partition command parameters.
"""
+self.updated_fstab_path = updated_fstab_path
+
if not self.source:
if not self.size and not self.fixed_size:
raise WicError("The %s partition has a size of zero. Please "
diff --git a/scripts/lib/wic/plugins/imager/direct.py
b/scripts/lib/wic/plugins/imager/direct.py
index f107e60089..7e1c1c03ab 100644
--- a/scripts/lib/wic/plugins/imager/direct.py
+++ b/scripts/lib/wic/plugins/imager/direct.py
@@ -58,7 +58,7 @@ class DirectPlugin(ImagerPlugin):
self.compressor = options.compressor
self.bmap = options.bmap
self.no_fstab_update = options.no_fstab_update
-self.original_fstab = None
+self.updated_fstab_path = None
self.name = "%s-%s" % (os.path.splitext(os.path.basename(wks_file))[0],
strftime("%Y%m%d%H%M"))
@@ -100,11 +100,8 @@ class DirectPlugin(ImagerPlugin):
finally:
self.cleanup()
-def _write_fstab(self, image_rootfs):
-"""overriden to generate fstab (temporarily) in rootfs. This is called
-from _create, make sure it doesn't get called from
-BaseImage.create()
-"""
+def update_fstab(self, image_rootfs):
+"""Assume partition order same as in wks"""
if not image_rootfs:
return
@@ -114,18 +111,9 @@ class DirectPlugin(ImagerPlugin):
with open(fstab_path) as fstab:
fstab_lines = fstab.readlines()
-self.original_fstab = fstab_lines.copy()
-
-if self._update_fstab(fstab_lines, self.parts):
-with open(fstab_path, "w") as fstab:
-fstab.writelines(fstab_lines)
-else:
-self.original_fstab = None
-def _update_fstab(self, fstab_lines, parts):
-
[OE-core] [PATCH v2 3/5] wic: Update pseudo db when excluding content from rootfs
To exclude content from the rootfs, wic makes a copy (using hardlinks if possible) of the rootfs directory and associated pseudo db, then removes files & directories as needed. However if these files and directories are removed using the python functions os.remove and shutil.rmtree, the copied pseudo db will not be updated correctly. For files copied from the original rootfs, if hardlinks were used successfully when copying the rootfs this should mean that the relevant inodes can't be reused and so the risk of pseudo aborts should be avoided. However, this logic doesn't apply for directories (as they can't be hardlinked) or for files added via the '--include-path' argument (as they weren't present in the original rootfs) and so there remains some risk of inodes being reused and the pseudo db becoming corrupted. To fix this, use the 'rm' command under pseudo when removing files & directories from the copied rootfs to ensure that the copied pseudo db is updated. Signed-off-by: Paul Barker --- scripts/lib/wic/plugins/source/rootfs.py | 13 - 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/scripts/lib/wic/plugins/source/rootfs.py b/scripts/lib/wic/plugins/source/rootfs.py index f1db83f8a1..afb1eb9202 100644 --- a/scripts/lib/wic/plugins/source/rootfs.py +++ b/scripts/lib/wic/plugins/source/rootfs.py @@ -199,17 +199,20 @@ class RootfsPlugin(SourcePlugin): if not os.path.lexists(full_path): continue +if new_pseudo: +pseudo = cls.__get_pseudo(native_sysroot, new_rootfs, new_pseudo) +else: +pseudo = None if path.endswith(os.sep): # Delete content only. for entry in os.listdir(full_path): full_entry = os.path.join(full_path, entry) -if os.path.isdir(full_entry) and not os.path.islink(full_entry): -shutil.rmtree(full_entry) -else: -os.remove(full_entry) +rm_cmd = "rm -rf %s" % (full_entry) +exec_native_cmd(rm_cmd, native_sysroot, pseudo) else: # Delete whole directory. -shutil.rmtree(full_path) +rm_cmd = "rm -rf %s" % (full_path) +exec_native_cmd(rm_cmd, native_sysroot, pseudo) part.prepare_rootfs(cr_workdir, oe_builddir, new_rootfs or part.rootfs_dir, native_sysroot, -- 2.26.2 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#146996): https://lists.openembedded.org/g/openembedded-core/message/146996 Mute This Topic: https://lists.openembedded.org/mt/79955375/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH v2 2/5] image_types_wic: Move wic working directory
By default the wic working directory is placed under the output
directory. When invoking wic under bitbake, the wic output directory is
added to PSEUDO_PATHS_IGNORE to avoid issues with files being removed
from outside a pseudo environment (see oe-core commit ad8f5532ff).
However, wic will copy the rootfs directory into its working directory
if it needs to add or remove content before creating a filesystem image.
This copy of the rootfs directory must be tracked by pseudo in order to
keep the permissions correct in the resulting image. So we can't have
the wic working directory under a path in PSEUDO_PATHS_IGNORE unless
we like broken permissions.
To fix this the new '-w' argument to wic is used to move the working
directory away from the output directory.
Note that wic deletes the temporary working directory automatically
when it finishes creating an image so users won't normally see the
'tmp-wic' directory under WORKDIR.
Fixes [Yocto #14129]
Signed-off-by: Paul Barker
---
meta/classes/image_types_wic.bbclass | 8 +++-
1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/meta/classes/image_types_wic.bbclass
b/meta/classes/image_types_wic.bbclass
index 000ee4249f..49be1da77a 100644
--- a/meta/classes/image_types_wic.bbclass
+++ b/meta/classes/image_types_wic.bbclass
@@ -29,11 +29,17 @@ WIC_CREATE_EXTRA_ARGS ?= ""
IMAGE_CMD_wic () {
out="${IMGDEPLOYDIR}/${IMAGE_NAME}"
build_wic="${WORKDIR}/build-wic"
+ tmp_wic="${WORKDIR}/tmp-wic"
wks="${WKS_FULL_PATH}"
+ if [ -e "$tmp_wic" ]; then
+ # Ensure we don't have any junk leftover from a previously
interrupted
+ # do_image_wic execution
+ rm -rf "$tmp_wic"
+ fi
if [ -z "$wks" ]; then
bbfatal "No kickstart files from WKS_FILES were found:
${WKS_FILES}. Please set WKS_FILE or WKS_FILES appropriately."
fi
- BUILDDIR="${TOPDIR}" PSEUDO_UNLOAD=1 wic create "$wks" --vars
"${STAGING_DIR}/${MACHINE}/imgdata/" -e "${IMAGE_BASENAME}" -o "$build_wic/"
${WIC_CREATE_EXTRA_ARGS}
+ BUILDDIR="${TOPDIR}" PSEUDO_UNLOAD=1 wic create "$wks" --vars
"${STAGING_DIR}/${MACHINE}/imgdata/" -e "${IMAGE_BASENAME}" -o "$build_wic/" -w
"$tmp_wic" ${WIC_CREATE_EXTRA_ARGS}
mv "$build_wic/$(basename "${wks%.wks}")"*.direct
"$out${IMAGE_NAME_SUFFIX}.wic"
}
IMAGE_CMD_wic[vardepsexclude] = "WKS_FULL_PATH WKS_FILES TOPDIR"
--
2.26.2
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#146995):
https://lists.openembedded.org/g/openembedded-core/message/146995
Mute This Topic: https://lists.openembedded.org/mt/79955373/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH v2 1/5] wic: Ensure internal workdir is not reused
If a path is specified for the internal wic working directory using
the -w/--workdir argument then it must not already exist. Re-using a
previous workdir could easily result in rootfs and intermediate files
from a previous build being added to the current image.
Signed-off-by: Paul Barker
---
scripts/lib/wic/plugins/imager/direct.py | 12 +++-
1 file changed, 11 insertions(+), 1 deletion(-)
diff --git a/scripts/lib/wic/plugins/imager/direct.py
b/scripts/lib/wic/plugins/imager/direct.py
index b329568c7a..f107e60089 100644
--- a/scripts/lib/wic/plugins/imager/direct.py
+++ b/scripts/lib/wic/plugins/imager/direct.py
@@ -62,7 +62,7 @@ class DirectPlugin(ImagerPlugin):
self.name = "%s-%s" % (os.path.splitext(os.path.basename(wks_file))[0],
strftime("%Y%m%d%H%M"))
-self.workdir = options.workdir or tempfile.mkdtemp(dir=self.outdir,
prefix='tmp.wic.')
+self.workdir = self.setup_workdir(options.workdir)
self._image = None
self.ptable_format = self.ks.bootloader.ptable
self.parts = self.ks.partitions
@@ -78,6 +78,16 @@ class DirectPlugin(ImagerPlugin):
self._image = PartitionedImage(image_path, self.ptable_format,
self.parts, self.native_sysroot)
+def setup_workdir(self, workdir):
+if workdir:
+if os.path.exists(workdir):
+raise WicError("Internal workdir '%s' specified in wic
arguments already exists!" % (workdir))
+
+os.makedirs(workdir)
+return workdir
+else:
+return tempfile.mkdtemp(dir=self.outdir, prefix='tmp.wic.')
+
def do_create(self):
"""
Plugin entry point.
--
2.26.2
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#146994):
https://lists.openembedded.org/g/openembedded-core/message/146994
Mute This Topic: https://lists.openembedded.org/mt/79955372/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH] devtool: Fix source extraction for gcc shared source
On Tue, 19 Jan 2021 at 15:51, Tomasz Dziendzielski
wrote:
>
> If do_patch task is disabled then prepare do_configure dependencies to
> fetch external sources and create symlink to ${S} in devtool workspace.
>
> [YOCTO #13036]
>
> Signed-off-by: Tomasz Dziendzielski
> ---
> .../gcc/gcc-shared-source.inc | 1 +
> scripts/lib/devtool/standard.py | 30 ---
> 2 files changed, 27 insertions(+), 4 deletions(-)
>
> diff --git a/meta/recipes-devtools/gcc/gcc-shared-source.inc
> b/meta/recipes-devtools/gcc/gcc-shared-source.inc
> index aac4b49313..9ef80f2074 100644
> --- a/meta/recipes-devtools/gcc/gcc-shared-source.inc
> +++ b/meta/recipes-devtools/gcc/gcc-shared-source.inc
> @@ -2,6 +2,7 @@ do_fetch() {
> :
> }
> do_fetch[noexec] = "1"
> +do_patch[noexec] = "1"
Have you tested whether the archiver class is still working after this
change? If not it'd be helpful to run `oe-selftest -r archiver` to
check.
--
Paul Barker
Konsulko Group
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#146993):
https://lists.openembedded.org/g/openembedded-core/message/146993
Mute This Topic: https://lists.openembedded.org/mt/79954315/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
