Re: [yocto] running ptest without internet?
On Wed, 2024-07-31 at 20:50 +0200, Jan Vermaete via lists.yoctoproject.org wrote: > I was wondering if there is a convention or agreement on what to do > with tests under ptests that do need internet but are running without? > An example could be a unittest of a package about networking that does > a ping to an URL. We haven't had anyone raise that as a requirement and we don't have any markup of the tests for it. The challenge is that ptests vary significantly in how they're run so we probably can't control if they're executed or not on a per test basis in a general way, only overall at the recipe level. > For now, Yocto assumes there is always a network/internet? For ptests, yes. We do assume the rest of the build can run offline outside of fetching. > Or we could exclude those tests based on BB_NO_NETWORK? > Something else, or not in years a priority right now? You could just run the tests and create a list of known failures without network, then filter the results accordingly. You could also have a list of recipes whos ptests are known to access the internet and exclude those from your testing? Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#63641): https://lists.yoctoproject.org/g/yocto/message/63641 Mute This Topic: https://lists.yoctoproject.org/mt/107652041/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] Tuesday Tech Call - 30 mins on our autobuilder infrastructure - 30th July
On Mon, 2024-07-29 at 00:48 -0700, yanmei.guo via Lists.Yoctoproject.Org wrote: > Any more details about this tech call? > Time/meeting links ? In my email I pointed people at: """ https://www.yoctoproject.org/community/get-involved/ has the call details under "WEEKLY ENGINEERING SYNC". """ which says: """ WEEKLY ENGINEERING SYNC Tuesdays at 8:00 PT (Pacific Time) A technical call to discuss the current status of engineering topics, including the auto builder, documentation, builds, and releases. Call Details Meeting ID: 990 892 712 Passcode: 805990 International numbers available: https://zoom.us/u/abKnJRHtHt """ Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#63585): https://lists.yoctoproject.org/g/yocto/message/63585 Mute This Topic: https://lists.yoctoproject.org/mt/107539718/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] Tuesday Tech Call - 30 mins on our autobuilder infrastructure - 30th July
The project has been asked some questions about our testing infrastructure, how it is setup, what it does, how we configure it, why and so on. I think these are good questions and ones that a wider audience could be interested in hearing discussed so I'm proposing we use half of the next Tuesday tech call to talk a bit about the autobuilder and try and answer some of the questions. This won't be a formal presentation, there might be some graphics just to make things easier to picture. We all know I'm overloaded but sadly when it comes to the autobuilder, I'm probably the person who knows a chunk of this topic well enough to talk about it. I do worry that as soon as I agree to this, it will escalate into people wanting slides, recordings, new chapters in the testing manual and so on. They're all wonderful ideas, I just don't have the time/energy to do more than I'm doing to keep the project running day to day. If anyone wants to listen and then document things, that would be very helpful and welcome. I started to write down a description of the things we should cover and realised it could take a lot longer than 30 mins. I'm thinking we may have a second session in the meeting the week after, assuming people find it useful and we have more to cover. Please pass the message around to any colleagues who may be interested too. Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#63569): https://lists.yoctoproject.org/g/yocto/message/63569 Mute This Topic: https://lists.yoctoproject.org/mt/107539718/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] [qa-build-notification] QA notification for completed autobuilder build (yocto-5.1_M2.rc1)
On Wed, 2024-07-17 at 09:26 +0100, Richard Purdie via lists.yoctoproject.org wrote: > On Wed, 2024-07-17 at 04:45 +, Pokybuild User via > lists.yoctoproject.org wrote: > > > > A build flagged for QA (yocto-5.1_M2.rc1) was completed on the > > autobuilder and is available at: > > > > > > https://autobuilder.yocto.io/pub/releases/yocto-5.1_M2.rc1 > > > > > > Build URL: > > https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/7144 > > > > Build hash information: > > > > bitbake: 1d7360031164f04887c792fb0b2dd86c6ccfcc23 > > meta-agl: 3f3bbd3871e5a7a4e9e275e2c957b98515cc30c3 > > meta-arm: 14d24b5526298cdfd6db65b6e11b54804a6a553f > > meta-aws: 4d048eacb5cbdaeb3f37d0427cc8a810a8ba7b69 > > meta-clang: 1fdde55d951b7a8f1e537ccf56aca2dc90c40091 > > meta-intel: e97386254712047f27f5554da6470fe3fb65d07a > > meta-mingw: e4d377d5ddb62b265536bd33dbccfbb0904f8701 > > meta-openembedded: f8342855a440fa5c4194fc57285e8db9e034fcaa > > meta-virtualization: c138943fed29e768df990061c30d6f071b98ffc3 > > oecore: 758508614541c68dd6a2b7084e07287c71f40a8f > > poky: e18d60deb0496f7c91f2de900d6c024b45b7910a > > This build had three issues: > > ubuntu cve-check database issue in oe-selftest: > https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/6974 > > qemumips ssh test failure: > https://autobuilder.yoctoproject.org/typhoon/#/builders/102/builds/6318 > > meta-arm systemd boot issue: > https://autobuilder.yoctoproject.org/typhoon/#/builders/113/builds/5756 > > The ssh test failure on mips is likely load related. The meta-arm > issue > was seen before and appears intermittent. The cve-check issue has > been > seen periodically and needs investigation. > > I'm not sure if any of these should block QA for M2 or not, input > welcome (TSC esp.). Sadly, I feel relatively powerless to do much > about > them. The meta-arm issue is confirmed as intermittent as https://autobuilder.yoctoproject.org/typhoon/#/builders/113/builds/5759 passed and is the same config as the above failure. Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#63525): https://lists.yoctoproject.org/g/yocto/message/63525 Mute This Topic: https://lists.yoctoproject.org/mt/107269197/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] [qa-build-notification] QA notification for completed autobuilder build (yocto-5.1_M2.rc1)
On Wed, 2024-07-17 at 04:45 +, Pokybuild User via lists.yoctoproject.org wrote: > > A build flagged for QA (yocto-5.1_M2.rc1) was completed on the > autobuilder and is available at: > > > https://autobuilder.yocto.io/pub/releases/yocto-5.1_M2.rc1 > > > Build URL: > https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/7144 > > Build hash information: > > bitbake: 1d7360031164f04887c792fb0b2dd86c6ccfcc23 > meta-agl: 3f3bbd3871e5a7a4e9e275e2c957b98515cc30c3 > meta-arm: 14d24b5526298cdfd6db65b6e11b54804a6a553f > meta-aws: 4d048eacb5cbdaeb3f37d0427cc8a810a8ba7b69 > meta-clang: 1fdde55d951b7a8f1e537ccf56aca2dc90c40091 > meta-intel: e97386254712047f27f5554da6470fe3fb65d07a > meta-mingw: e4d377d5ddb62b265536bd33dbccfbb0904f8701 > meta-openembedded: f8342855a440fa5c4194fc57285e8db9e034fcaa > meta-virtualization: c138943fed29e768df990061c30d6f071b98ffc3 > oecore: 758508614541c68dd6a2b7084e07287c71f40a8f > poky: e18d60deb0496f7c91f2de900d6c024b45b7910a This build had three issues: ubuntu cve-check database issue in oe-selftest: https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/6974 qemumips ssh test failure: https://autobuilder.yoctoproject.org/typhoon/#/builders/102/builds/6318 meta-arm systemd boot issue: https://autobuilder.yoctoproject.org/typhoon/#/builders/113/builds/5756 The ssh test failure on mips is likely load related. The meta-arm issue was seen before and appears intermittent. The cve-check issue has been seen periodically and needs investigation. I'm not sure if any of these should block QA for M2 or not, input welcome (TSC esp.). Sadly, I feel relatively powerless to do much about them. Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#63520): https://lists.yoctoproject.org/g/yocto/message/63520 Mute This Topic: https://lists.yoctoproject.org/mt/107267678/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] [yocto-autobuilder-helper] [PATCH] scripts: Split ptest log extraction to separate script
The ptest extraction should really happen separately to the index generation,
move it to do so.
Add code to only consider directories changed in the last 7 days to make
the script faster.
Add an indexing target to config.json to trigger the script.
Signed-off-by: Richard Purdie
---
config.json | 6 +++
scripts/generate-ptest-logs.py | 78
scripts/generate-testresult-index.py | 14 -
3 files changed, 84 insertions(+), 14 deletions(-)
create mode 100755 scripts/generate-ptest-logs.py
diff --git a/config.json b/config.json
index 6907f26..07bbd8e 100644
--- a/config.json
+++ b/config.json
@@ -1343,6 +1343,12 @@
"EXTRACMDS": [ "${SCRIPTSDIR}/run-abint
${HELPERRESULTSDIR}/../../abint/"]
}
},
+"indexing" : {
+"step1" : {
+"shortname" : "Extract ptest logs",
+"EXTRACMDS" : ["${SCRIPTSDIR}/generate-ptest-logs.py
/srv/autobuilder/autobuilder.yocto.io/pub/non-release/"]
+}
+},
"meta-mingw" : {
"NEEDREPOS" : ["poky", "meta-mingw"],
"MACHINE" : "qemux86-64",
diff --git a/scripts/generate-ptest-logs.py b/scripts/generate-ptest-logs.py
new file mode 100755
index 000..9565f2f
--- /dev/null
+++ b/scripts/generate-ptest-logs.py
@@ -0,0 +1,78 @@
+#!/usr/bin/env python3
+#
+# Copyright Linux Foundation, Richard Purdie
+#
+# SPDX-License-Identifier: GPL-2.0-only
+#
+
+import argparse
+import os
+import glob
+import re
+import time
+import subprocess
+
+def parse_args(argv=None):
+"""Parse command line arguments"""
+parser = argparse.ArgumentParser(
+description="Generate separate ptest logs from any
testresults.json files",
+formatter_class=argparse.ArgumentDefaultsHelpFormatter)
+
+parser.add_argument('path', help='path to directory to index')
+
+return parser.parse_args(argv)
+
+args = parse_args()
+path = os.path.abspath(args.path)
+
+# Pad so 20190601-1 becomes 20190601-01 and sorts correctly
+def keygen(k):
+m = re.match(r"(\d+)-(\d+)", k)
+if m:
+k1, k2 = m.groups()
+return k1 + "-" + k2.rjust(6, '0')
+else:
+return k
+
+current_time = time.time()
+
+# number of seconds in 7 days
+days = 7 * 24 * 60 * 60
+
+for build in sorted(os.listdir(path), key=keygen, reverse=True):
+buildpath = os.path.join(path, build, "testresults")
+if not os.path.exists(buildpath):
+# No test results
+continue
+reldir = "./" + build + "/"
+modified_time = os.stat(buildpath).st_mtime
+
+# Only consider things in the last X days
+if (modified_time < (current_time - days):
+continue
+
+btype = "other"
+files = os.listdir(buildpath)
+if os.path.exists(buildpath + "/a-full-posttrigger") or \
+os.path.exists(buildpath + "/a-full"):
+btype = "full"
+elif os.path.exists(buildpath + "/a-quick-posttrigger") or \
+os.path.exists(buildpath + "/a-quick"):
+btype = "quick"
+elif len(files) == 1:
+btype = files[0]
+
+# Ensure we have saved out log data for ptest runs to aid debugging
+if "ptest" in btype or btype in ["full", "quick"]:
+for root, dirs, files in os.walk(buildpath):
+for name in dirs:
+if "ptest" in name:
+f = os.path.join(root, name)
+logs = glob.glob(f + "/*.log")
+if logs:
+continue
+subprocess.check_call(["resulttool", "log", f,
"--dump-ptest", f])
+# Ensure we don't rerun every time with a dummy log
+with open(f + "/resulttool-done.log", "a+") as tf:
+tf.write("\n")
+
diff --git a/scripts/generate-testresult-index.py
b/scripts/generate-testresult-index.py
index 568154d..d6d0a8f 100755
--- a/scripts/generate-testresult-index.py
+++ b/scripts/generate-testresult-index.py
@@ -145,20 +145,6 @@ for build in sorted(os.listdir(path), key=keygen,
reverse=True):
entries.append(entry)
-# Also ensure we have saved out log data for ptest runs to aid debugging
-if "ptest" in btype or btype in ["full", "quick"]:
-for root, dirs, files in os.walk(buildpath):
-for name in dirs:
-if "ptest" in name:
-f = os.path.join(root, name)
-logs = glob.glob(f + "/*.log")
-
Re: [Yocto][QA] patches : QA Issue: Missing Upstream-Status in patch
On Wed, 2024-07-10 at 14:02 +, Peter Marko via lists.yoctoproject.org wrote: > Following distro config did the trick for me half a year ago when I > started our migration to scarthgap. > ERROR_QA:remove = "patch-status" > WARN_QA:append = " patch-status" > > But I would not recommend to use it as permanent solution. > Rather adapt your toolkit which backports patches to add proper > upstream status flag. Whilst I have the concerns others have raised about disabling this, I'd like to make more people aware you can do: ERROR_QA:remove:layer-mylayername = "patch-status" WARN_QA:append:layer-mylayername = " patch-status" so at least you isolate this change to your layer. If you do then change other layers, they then retain the sanity checks needed for acceptable submissions. Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#63480): https://lists.yoctoproject.org/g/yocto/message/63480 Mute This Topic: https://lists.yoctoproject.org/mt/107142644/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] [yocto-autobuilder-helper] [PATCH] config.json: OpenSUSE 15.5 and 15.6 have old python
We need buildtools on these distros as the python version is still 3.6.
Signed-off-by: Richard Purdie
---
config.json | 2 ++
1 file changed, 2 insertions(+)
diff --git a/config.json b/config.json
index 1acdbc9..07ac95b 100644
--- a/config.json
+++ b/config.json
@@ -1938,6 +1938,8 @@
"alma8*" : "${BUILDTOOLS_URL}",
"stream8*" : "${BUILDTOOLS_URL}",
"opensuse154*" : "${BUILDTOOLS_URL}",
+"opensuse155*" : "${BUILDTOOLS_URL}",
+"opensuse156*" : "${BUILDTOOLS_URL}",
"perf-alma8*" : "${BUILDTOOLS_URL}",
"perf-ubuntu1804-ty-*" : "${BUILDTOOLS_URL}"
}
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#63463): https://lists.yoctoproject.org/g/yocto/message/63463
Mute This Topic: https://lists.yoctoproject.org/mt/107146657/21656
Group Owner: yocto+ow...@lists.yoctoproject.org
Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] Class file priority between meta-rust and oe-core layers
On Thu, 2024-06-27 at 13:35 +0100, Mark Corbin via lists.yoctoproject.org wrote: > Hello > > I'm currently updating my project from Kirkstone to Scarthgap and > have run into an issue with bbclass file priorities. > > We are using our own modified 'meta-rust' layer, but rust recipes are > inheriting the 'cargo.bbclass' file from 'openembedded- > core/meta/classes-recipe' directory even though the 'meta-rust' layer > has a higher priority. This wasn't an issue under Kirkstone. > > Is there any way to ensure that my 'meta-rust' classes are used by > default? I'd just note that the behaviour hasn't changed and that inherits search BBPATH for the class. I suspect your BBPATH ordering has changed or needs tweaking to do what you expect. Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#63402): https://lists.yoctoproject.org/g/yocto/message/63402 Mute This Topic: https://lists.yoctoproject.org/mt/106907674/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] [yocto-autobuilder-helper] config-json: use master branch for meta-agl
On Wed, 2024-06-12 at 11:30 -0400, Scott Murray via
lists.yoctoproject.org wrote:
> On Wed, 12 Jun 2024, Steve Sakoman via lists.yoctoproject.org wrote:
>
> > scarthgap is no longer supported on next branch
> >
> > Signed-off-by: Steve Sakoman
> > ---
> > config.json | 2 +-
> > 1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/config.json b/config.json
> > index fbf2e6c..2089fa9 100644
> > --- a/config.json
> > +++ b/config.json
> > @@ -1853,7 +1853,7 @@
> > },
> > "meta-agl": {
> > "url" :
> > "https://git.automotivelinux.org/AGL/meta-agl;,
> > - "branch" : "next",
> > + "branch" : "master",
> > "revision" : "HEAD",
> > "no-layer-add" : true
> > },
>
> Is this patch missing a "[scarthgap]"? I ask because it makes sense
> for the yocto-autobuilder-helper scarthgap branch, but master AB will
> need to keep using meta-agl's next branch.
It is and I did check that with Steve on irc. It only merged to the
scarthgap branch.
Cheers,
Richard
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#63334): https://lists.yoctoproject.org/g/yocto/message/63334
Mute This Topic: https://lists.yoctoproject.org/mt/106633253/21656
Group Owner: yocto+ow...@lists.yoctoproject.org
Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] [qa-build-notification] QA notification for completed autobuilder build (yocto-5.1_M1.rc2)
On Wed, 2024-06-05 at 08:26 +, Pokybuild User via lists.yoctoproject.org wrote: > > A build flagged for QA (yocto-5.1_M1.rc2) was completed on the > autobuilder and is available at: > > > https://autobuilder.yocto.io/pub/releases/yocto-5.1_M1.rc2 > > > Build URL: > https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/7008 > > Build hash information: > > bitbake: 23681775e5941e54ebead469addf708fca1e6beb > meta-agl: f65060f370e8834de0229c6074c6e4326bdd04a6 > meta-arm: f5ebb36c59b6eb847f45713a1f9b08a88f9978e1 > meta-aws: 15bb71aff6bcdcbeeaa21a0e7ec915e0e90fae39 > meta-clang: ce7d354da7a498f90ba260c006d29a4d23e3b4f7 > meta-intel: 4f4e6d763cf2e7d04eaa8db65c0bd3e8c3b689a4 > meta-mingw: e4d377d5ddb62b265536bd33dbccfbb0904f8701 > meta-openembedded: 446560ab41de5e29ef5dfb556de1149e76a80367 > meta-virtualization: e96da98e4038f5388596b4294ac3d8425b2dacb2 > oecore: e068ba7d6b91237f9dabc2050c9473bed311690c > poky: 5fc59beaec099cd4a78b38fb0686f377bbe7dadd There was a failing bitbake-selftest which affected multiple builds in this release build but it should be ok to release since that issue is well understood and has since been resolved. The meta-arm and meta-agl-core layers are also failing but this is also a known issue related to recent changes in master which those layers are still catching up on. Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#63297): https://lists.yoctoproject.org/g/yocto/message/63297 Mute This Topic: https://lists.yoctoproject.org/mt/106542059/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] query on do_stage
On Fri, 2024-05-31 at 15:17 -0700, SIMON BABY via
lists.yoctoproject.org wrote:
> Hei Team,
>
> I have a recipe below. I did not see the staging directory under the
> tmp generated folder. Is there a hidden folder or where are the files
> going?
>
>
>
[...]
>
> do_stage () {
> mkdir -p ${STAGING_INCDIR}/marvell-umsd
> cp --dereference -R ${MSD_ROOT}/include/*
> ${STAGING_INCDIR}/marvell-umsd/
> cp ${MSD_ROOT}/library/msdDrv.o ${STAGING_LIBDIR}
> }
>
>
>
> I checked the path below and could not see the include files or the
> msdDrv.o copied anywhere. Please correct me with any other solution.
> Can I go with do_install () ?
> The requirement is to use this recipe as a dependency for other
> modules to use the include file and library.
do_stage was a task which was removed many many years ago:
https://git.yoctoproject.org/poky/commit/?id=dc807f54f858419f97e211cd62fd2d30db9a80de
i.e. around 2010.
We opted to have the output from do_install work for both sysroots and
target packages.
Cheers,
Richard
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#63264): https://lists.yoctoproject.org/g/yocto/message/63264
Mute This Topic: https://lists.yoctoproject.org/mt/106417703/21656
Group Owner: yocto+ow...@lists.yoctoproject.org
Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] GPL License Compliance - Automatically detect linking against GPL libraries
On Fri, 2024-05-31 at 06:40 -0700, Hanke Fabian (DC/PAR) via lists.yoctoproject.org wrote: > Hello, > > thank you for all the responses so far. I guess we will have a look > at fossology and fossas, but we would prefer a solution which does > not require an additional thirdparty service. We know that there are > different types of GPL licenses which bring different obligations. > > We are searching for an automatic mechanism to detect linking to a > shared library from a GPL package. We thought there might be a way to > utilize the build system’s shared library resolver which is used for > the automatic runtime added runtime dependencies [1]. > > For static libraries we found that they are disabled by default [2]. > > [1] > https://docs.yoctoproject.org/overview-manual/concepts.html#automatically-added-runtime-dependencies > [2] > https://docs.yoctoproject.org/dev/dev-manual/licenses.html#compliance-limitations-with-executables-built-from-static-libraries It can definitely be done and we have a lot of the information there. Our packaging code does already look at linking as you mention. Nobody has proposed a solution that could be merged to OE-Core though. I'd love to see one. Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#63248): https://lists.yoctoproject.org/g/yocto/message/63248 Mute This Topic: https://lists.yoctoproject.org/mt/106365537/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] [meta-mingw] [PATCH 0/4] GCC-14 related fixes
On Wed, 2024-05-29 at 22:10 -0700, Khem Raj via lists.yoctoproject.org wrote: > This series addresses the set of issues seen with mingw builds with GCC-14 > > Khem Raj (4): > mingw-w64: Upgrade to 12.0.0 release > curl: Disable incompatible-pointer-types warning as error on mingw > gettext: Disable incompatible-pointer-types warning as error on mingw > diffutils: Fix build with GCC-14 These let the build with gcc14 get further thanks. I did have to add some whitespace to the CFLAGS appends. There was a failure in qemu these patches exposed so I've sent a cflags fix for that which lets mingw build cleanly. I also realised my config was using the wrong list for patches, I've changed it to yocto-patches now (README was already changed). Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#63239): https://lists.yoctoproject.org/g/yocto/message/63239 Mute This Topic: https://lists.yoctoproject.org/mt/106383341/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] [meta-mingw] [PATCH] qemu: Disable incompatible-pointer-types warning as error on mingw
GCC-14 promoted this warning into error by default but code does not build cleanly and now this warning becomes a hard error. Work around it for now. Signed-off-by: Richard Purdie --- recipes-devtools/qemu/qemu_%.bbappend | 1 + 1 file changed, 1 insertion(+) create mode 100644 recipes-devtools/qemu/qemu_%.bbappend diff --git a/recipes-devtools/qemu/qemu_%.bbappend b/recipes-devtools/qemu/qemu_%.bbappend new file mode 100644 index 000..12c9247 --- /dev/null +++ b/recipes-devtools/qemu/qemu_%.bbappend @@ -0,0 +1 @@ +CFLAGS:append:mingw32 = " -Wno-error=incompatible-pointer-types" -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#63238): https://lists.yoctoproject.org/g/yocto/message/63238 Mute This Topic: https://lists.yoctoproject.org/mt/106384493/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] [qa-build-notification] QA notification for completed autobuilder build (yocto-5.0.1.rc1)
On Thu, 2024-05-09 at 13:21 -0400, Bill Mills wrote: > Richard / Steve / All, > > On 5/8/24 10:12 AM, Richard Purdie via lists.yoctoproject.org wrote: > > On Wed, 2024-05-08 at 14:06 +, Pokybuild User via > > lists.yoctoproject.org wrote: > > > > > > A build flagged for QA (yocto-5.0.1.rc1) was completed on > > > the > > > autobuilder and is available at: > > > > > > > > > > > > https://autobuilder.yocto.io/pub/releases/yocto-5.0.1.rc1 > > > > > > > > > Build URL: > > > https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/6885 > > > > > > > From discussions with Steve, I think due to failures in this build > > we're trying an rc2. > > We are looking for a pipe cleaner for automated machine tests. > If this is DOA we probibly don't want to use it correct? There are a few QA failures but they're intermittent issues and it was marginal whether to rebuild or not. For a pipe cleaner it would be fine. > Unfortunately the 5.0 rc4 release is no longer on the autobuilder. > (But rc1 rc2 and rc3 still are). [1] How long are they kept and when > are they deleted? When an rc is promoted to a release it is removed (or moved) as part of the transition so rc4 became the release. We clean up old obsolete ones when we remember or are short on space on the NAS. > Is the dir structure of the release [2] the exact same as the rcN? I'm not 100% sure. It used to vary and I've been trying to get us to match, I don't know where we currently stand. Copying Chee Yang and Jing Hui who would know along with Michael. > It appears so on first glance. Did rc4 get mv'ed instead of cp'ed? > Is this the expected behavior? It is what happens during the publishing process. Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#63067): https://lists.yoctoproject.org/g/yocto/message/63067 Mute This Topic: https://lists.yoctoproject.org/mt/105981534/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] Yocto recipe file://DIRECTORY fetcher recursive tracking?
On Wed, 2024-05-08 at 12:37 -0700, bibibobibo via lists.yoctoproject.org wrote: > I have a dummy question regarding the file://DIRECTORY/ in recipe > usage, we are using this mechanism to directly point at a source > repository with potentially lots of subfolders, etc. Does bitbake > track file://DIRECTORY tracks all contents accordingly (i.e something > will recursively walk through this directory and checksum each file, > then arrive at a final combined checksum?) and if we happen to repo- > sync or git-pull to pull new changes in DIRECTORY/, Yocto will be > able to detect any changes in this directory and automatically > rebuild this recipe. > > I was getting mixed answers from a few sources, some said this can > work, some said it works sometimes but not always. So I would like to > get a definitive answer from Yocto forum. It is meant to work. There have been some bugs in this area in the past, I believe they have been fixed. It also doesn't perform well with large trees of files from a speed perspective. Something like git allows for much faster operations as git is designed to do tracking of this kind of thing. So yes, it should work but isn't optimal with really large trees. Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#63060): https://lists.yoctoproject.org/g/yocto/message/63060 Mute This Topic: https://lists.yoctoproject.org/mt/105997003/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] [qa-build-notification] QA notification for completed autobuilder build (yocto-5.0.1.rc1)
On Wed, 2024-05-08 at 14:06 +, Pokybuild User via lists.yoctoproject.org wrote: > > A build flagged for QA (yocto-5.0.1.rc1) was completed on the > autobuilder and is available at: > > > https://autobuilder.yocto.io/pub/releases/yocto-5.0.1.rc1 > > > Build URL: > https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/6885 > >From discussions with Steve, I think due to failures in this build we're trying an rc2. Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#63056): https://lists.yoctoproject.org/g/yocto/message/63056 Mute This Topic: https://lists.yoctoproject.org/mt/105981534/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] Generating SPDX documents without actually building anything
On Wed, 2024-05-08 at 13:59 +0200, Sebastian Schuberth via lists.yoctoproject.org wrote: > Hi all, > > to start with a disclaimer, I'm a total noob when it comes to Open > Embedded / Embedded Linux / Yocto / BitBake stuff. That said, I'm in > need of creating SPDX documents for Yocto projects in the context of > [1]. I'm already quite far in that I work with Poky and > programmatically run something like > > bitbake -r /home/sebastian/Development/GitHub/doubleopen- > project/ort/plugins/package- > managers/bitbake/src/main/resources/spdx.conf -c create_spdx xmlto > > However, I continuously run into various build problems on my Fedora > 40 machine (one of which has been recently addressed by [2]), let > alone the time it takes to build something from scratch, even > something as small as e.g. the "xmlto" target. So, ideally, I would > be able to create SPDX documents without really building any binaries > at all. I probably want all the source download and patching to > happen, in order to archive the patched code that goes into the > binaries, but skip the actual compiler / linker invocations to save > time, and just create the SPDX documents. > > Is something like this already possible somehow, maybe by setting > from compiler / linker variables to fake scripts that just do > nothing? The SPDX generated information includes information about the generated binaries, the dependencies between them and also traces license headers used by a given binary through the binary debug information. Whilst it is true you could generate a simpler SPDX document without compiling anything, that isn't how our SPDX generation currently works, it includes a lot more information and some of that is from the binaries. You could create a different simpler spdx like class that could do what you describe but that isn't what we have today. We went for the version with all the information. Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#63054): https://lists.yoctoproject.org/g/yocto/message/63054 Mute This Topic: https://lists.yoctoproject.org/mt/105979197/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] Forbidden host packages?
On Mon, 2024-04-29 at 16:37 +, Danter, Richard via lists.yoctoproject.org wrote: > I know there are host packages which are required in order to build > Yocto, but are there any that are forbidden? > > For example, I have encountered a build issue on a RHEL host, but > only > if the daxctl-devel package is installed on that host. Without it, > the > build works fine. > > The failure is in building qemu, see below. qemu-system-native is a little special in that it needs to interface to the host graphics so it has wider search paths than most packages. We try to explicitly disable dependencies we don't use so this sounds like some configuration for this one is missing. Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#63011): https://lists.yoctoproject.org/g/yocto/message/63011 Mute This Topic: https://lists.yoctoproject.org/mt/105804577/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] [yocto-autobuilder-helper] [PATCH] cve-generate-chartdata: Handle unexpected files in the stats directory
A build of "stable/dunfell-nut" caused incorrect data to be added to the metrics
repo and resulted in script failures. Handle this situation better.
Signed-off-by: Richard Purdie
---
scripts/cve-generate-chartdata | 6 +-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/scripts/cve-generate-chartdata b/scripts/cve-generate-chartdata
index e187f8b..8b9df64 100755
--- a/scripts/cve-generate-chartdata
+++ b/scripts/cve-generate-chartdata
@@ -28,7 +28,11 @@ for branch in os.listdir(args.resultsdir):
branchdir = os.path.join(args.resultsdir, branch)
for f in os.listdir(branchdir):
ts = f.split(".")[0]
-rounded_ts = str(round_to_day(ts))
+try:
+rounded_ts = str(round_to_day(ts))
+except ValueError:
+# Couldn't parse a timestamp from filename
+continue
if rounded_ts not in counts:
counts[rounded_ts] = {}
if branch not in counts[rounded_ts]:
--
2.40.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#63004): https://lists.yoctoproject.org/g/yocto/message/63004
Mute This Topic: https://lists.yoctoproject.org/mt/105747379/21656
Group Owner: yocto+ow...@lists.yoctoproject.org
Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] [qa-build-notification] QA notification for completed autobuilder build (yocto-5.0.rc4)
On Fri, 2024-04-19 at 00:18 +, Pokybuild User via lists.yoctoproject.org wrote: > > A build flagged for QA (yocto-5.0.rc4) was completed on the > autobuilder and is available at: > > > https://autobuilder.yocto.io/pub/releases/yocto-5.0.rc4 > > > Build URL: > https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/6823 > This one was a green build, so we're good to replace rc2 with this one. Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#62967): https://lists.yoctoproject.org/g/yocto/message/62967 Mute This Topic: https://lists.yoctoproject.org/mt/105614807/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] [tsc] [qa-build-notification] QA notification for completed autobuilder build (yocto-5.0.rc2)
On Thu, 2024-04-18 at 11:45 -0400, Denys Dmytriyenko wrote: > On Thu, Apr 18, 2024 at 11:49:26AM +0100, Richard Purdie wrote: > > On Thu, 2024-04-18 at 10:45 +, Ross Burton wrote: > > > On 18 Apr 2024, at 09:11, Richard Purdie > > > wrote: > > > > For the purposes of history, the issue is in curl 8.7.0 and > > > > 8.7.1 was > > > > just a single bug fix. There is a fix in git master branch for > > > > the curl > > > > issue and curl themselves also have a fix in master as yet > > > > unreleased. > > > > I've backported the patch to git for now, we can drop it when > > > > we > > > > upgrade things. > > > > > > As the underlying issue is in curl and has been fixed in master: > > > > > > https://github.com/curl/curl/commit/721941aadf4adf4f6aeb3f4c0ab489bb89610c36 > > > > > > I think we should apply this instead. The issue is a curl > > > regression, we just saw it first in git. > > > > > > I lean towards this being a release blocker and think we should > > > respin. > > > > Much as I would prefer not to, I agree. > > > > We should abandon rc2 and build an rc3. I'd like to keep the > > dunfell/kirkstone builds for after rc3 goes through QA though. We > > should be able to build rc3 quite quickly. > > I believe 5.0 is still "within" the schedule, right? > > What about dunfell and kirkstone minor releases? I know their builds > are being delayed by 5.0 taking up AB. But are they already behind > the schedule? If not, then 5.0 rc3 should definitely get prioritized. The dunfell build is actually built and ready for QA. The schedule is ours to run though and I think the correct thing to do is to get 5.0 rc4 into QA, then the others. I've stopped the rc3 build as there was still a buildtools issue. It was infrastructure usage related rather than anything in the tarball this time. rc4 is therefore building. I cancelled a load of master stuff to ensure it has priority. Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#62964): https://lists.yoctoproject.org/g/yocto/message/62964 Mute This Topic: https://lists.yoctoproject.org/mt/105575131/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] [tsc] [qa-build-notification] QA notification for completed autobuilder build (yocto-5.0.rc2)
On Thu, 2024-04-18 at 10:45 +, Ross Burton wrote: > On 18 Apr 2024, at 09:11, Richard Purdie > wrote: > > For the purposes of history, the issue is in curl 8.7.0 and 8.7.1 was > > just a single bug fix. There is a fix in git master branch for the curl > > issue and curl themselves also have a fix in master as yet unreleased. > > I've backported the patch to git for now, we can drop it when we > > upgrade things. > > As the underlying issue is in curl and has been fixed in master: > > https://github.com/curl/curl/commit/721941aadf4adf4f6aeb3f4c0ab489bb89610c36 > > I think we should apply this instead. The issue is a curl regression, we > just saw it first in git. > > I lean towards this being a release blocker and think we should respin. Much as I would prefer not to, I agree. We should abandon rc2 and build an rc3. I'd like to keep the dunfell/kirkstone builds for after rc3 goes through QA though. We should be able to build rc3 quite quickly. Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#62962): https://lists.yoctoproject.org/g/yocto/message/62962 Mute This Topic: https://lists.yoctoproject.org/mt/105575131/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] [tsc] [qa-build-notification] QA notification for completed autobuilder build (yocto-5.0.rc2)
On Wed, 2024-04-17 at 22:23 +0100, Richard Purdie via lists.yoctoproject.org wrote: > On Wed, 2024-04-17 at 11:36 +, Ross Burton via > lists.yoctoproject.org wrote: > > On 17 Apr 2024, at 09:50, Richard Purdie via lists.yoctoproject.org > > wrote: > > > For b), updating buildtools should resolve it, I thought I had done > > > that but there were a lot of moving pieces so I must not have got > > > the > > > latest. > > > > > > I'm proposing we do but this into QA as I'd not block release on > > > these > > > issues. I've done my best to get this into as good a shape as we > > > can. > > > > Agreed. Can we do a build in the meantime of the specific job that > > failed so we can verify the buildtools/pip issue is resolved? > > I dug into the "github failure" and it turns out that if you have git > 2.44.0 compiled/linked against curl 8.7.1 then the command: > > git-clone --bare --mirror https://github.com/grpc/grpc.git /tmp/somewhere > > exits silently with an error exit code of 141 (SIGPIPE). > > With curl 8.6.0 this doesn't happen and the new buildtools has the new > curl+git version. > > You can demo this in a build quite simply with: > > $ bitbake git-replacement-native -c compile > > then: > > $ PATH=./tmp/work/aarch64-linux/git-native/2.44.0/git-2.44.0/:$PATH git-clone > --bare --mirror https://github.com/grpc/grpc.git /tmp/bitbake-fetch-test > > More digging to do to debug it but that should be enough to let others > reproduce. I suspect it will happen on x86 too, I just used the arm > worker to narrow it down since it was the most frequent occurrence. > > You have to manipulate PATH so git's internal exec calls find the right > git binaries. We'll not mention how it took me to work that out. For the purposes of history, the issue is in curl 8.7.0 and 8.7.1 was just a single bug fix. There is a fix in git master branch for the curl issue and curl themselves also have a fix in master as yet unreleased. I've backported the patch to git for now, we can drop it when we upgrade things. Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#62958): https://lists.yoctoproject.org/g/yocto/message/62958 Mute This Topic: https://lists.yoctoproject.org/mt/105575131/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] [tsc] [qa-build-notification] QA notification for completed autobuilder build (yocto-5.0.rc2)
On Wed, 2024-04-17 at 11:36 +, Ross Burton via lists.yoctoproject.org wrote: > On 17 Apr 2024, at 09:50, Richard Purdie via lists.yoctoproject.org > wrote: > > For b), updating buildtools should resolve it, I thought I had done > > that but there were a lot of moving pieces so I must not have got > > the > > latest. > > > > I'm proposing we do but this into QA as I'd not block release on > > these > > issues. I've done my best to get this into as good a shape as we > > can. > > Agreed. Can we do a build in the meantime of the specific job that > failed so we can verify the buildtools/pip issue is resolved? I dug into the "github failure" and it turns out that if you have git 2.44.0 compiled/linked against curl 8.7.1 then the command: git-clone --bare --mirror https://github.com/grpc/grpc.git /tmp/somewhere exits silently with an error exit code of 141 (SIGPIPE). With curl 8.6.0 this doesn't happen and the new buildtools has the new curl+git version. You can demo this in a build quite simply with: $ bitbake git-replacement-native -c compile then: $ PATH=./tmp/work/aarch64-linux/git-native/2.44.0/git-2.44.0/:$PATH git-clone --bare --mirror https://github.com/grpc/grpc.git /tmp/bitbake-fetch-test More digging to do to debug it but that should be enough to let others reproduce. I suspect it will happen on x86 too, I just used the arm worker to narrow it down since it was the most frequent occurrence. You have to manipulate PATH so git's internal exec calls find the right git binaries. We'll not mention how it took me to work that out. Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#62954): https://lists.yoctoproject.org/g/yocto/message/62954 Mute This Topic: https://lists.yoctoproject.org/mt/105575131/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] [qa-build-notification] QA notification for completed autobuilder build (yocto-5.0.rc2)
On Wed, 2024-04-17 at 08:37 +, Pokybuild User via lists.yoctoproject.org wrote: > > A build flagged for QA (yocto-5.0.rc2) was completed on the autobuilder > and is available at: > > > https://autobuilder.yocto.io/pub/releases/yocto-5.0.rc2 > > > Build URL: > https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/6813 This does have two failures in it and two known ptest warnings (curl and gstreamer). The failures are from: a) a github network issue (rate limiting?) b) pip being missing in the buildtools tarball used to run the build for patchtest-selftest For b), updating buildtools should resolve it, I thought I had done that but there were a lot of moving pieces so I must not have got the latest. I'm proposing we do but this into QA as I'd not block release on these issues. I've done my best to get this into as good a shape as we can. Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#62944): https://lists.yoctoproject.org/g/yocto/message/62944 Mute This Topic: https://lists.yoctoproject.org/mt/105573383/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] Any recommendation to make software layer Yocto Compatible?
On Mon, 2024-04-08 at 05:36 -0700, Duy wrote: > Hi Richard, > > Thanks for your response. > Here is one of the recipe bbappend files I'm working on: > meta-agl-ic-container/recipes-extended/dlt-daemon/dlt- > daemon_%.bbappend · master · Automotive Grade Linux / AGL / meta-agl- > devel · GitLab > meta-agl-ic-container/recipes-extended/dlt-daemon/dlt- > daemon_%.bbappend · master · Automotive Grade Linux / AGL / meta-agl- > devel · GitLab > > I think some configurations are better to be upstreaming, e.g: Adding > new PACKAGES, adding new PACKAGECONFIG. It avoids adding too much to > bbappend. If you have any idea, please share it. Where possible please do share changes with upstream recipes, particularly for things like PACKAGECONFIG entries. There is no good reason to have them separate in most cases. Other parts of this look to be adding configuration files. Are these generally useful? That might help to know if they should be upstream. If not, should they be in a separate recipe? This isn't just a software layer, it has quite a bit of configuration/distro policy in there so the layer checks are right to flag it as problematic in my view. Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#62928): https://lists.yoctoproject.org/g/yocto/message/62928 Mute This Topic: https://lists.yoctoproject.org/mt/105397924/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] Any recommendation to make software layer Yocto Compatible?
On Mon, 2024-04-08 at 02:25 -0700, Duy wrote: > I'm currently trying to make my software layer Yocto compatible. > Running yocto-check-layer script is a convenient way. > But in practice, I found that for software layer, it is difficult to > make it pass test_signatures case. It requires to put the condition > check to every variable or function appending in bbappend file. With > a deep customizing software layer, it reduces code readability. > Can anyone share your experience in making a software layer Yocto > compatible while keeping it easy to read? The question to ask is probably why are you needing to bbappend and change the behaviour of so many recipes? Software layers adding software wouldn't need to do that, so it sounds like you're changing configuration. If you're doing that, it is a distro configuration change and that isn't a software layer and the config changes should be conditional on the distro. Can you share some example bbappends you're needing? Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#62901): https://lists.yoctoproject.org/g/yocto/message/62901 Mute This Topic: https://lists.yoctoproject.org/mt/105397924/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] make a-quick quick again (and light on the AB, particularly arm)
On Wed, 2024-03-20 at 12:27 +0100, Alexander Kanavin wrote: > On Wed, 20 Mar 2024 at 11:55, Ross Burton > wrote: > > I’d be against removing _all_ arm hosts from the quick build. With > > the in-progress changes to the autobuilder hosting we should be > > able to retire the slower Arm workers and a build running on the > > Arm hosts shouldn’t be a problem. It would be a waste of effort to > > attempt to optimise the build for the current cluster when there > > are plans to relocate shortly anyway. > > Right, then let's not make this about arm builders, and simply drop > selftest of any kind and reproducible from a-quick. How about that? > > Particularly selftest keeps getting bigger and slower and that isn't > going to change. It used to take under an hour, nowadays it's 2+ > hours on x86 and 3+ on arm, with full sstate ready from the start. It does depend what you're testing btw. You tend to test package upgrades which don't really affect selftest as much as some of the other changes I'm often dealing with. I don't disagree with adding a separate target but I do want to make it clear we should improve the speed of selftest too, I'm not ready to give up on that yet! Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#62804): https://lists.yoctoproject.org/g/yocto/message/62804 Mute This Topic: https://lists.yoctoproject.org/mt/105041413/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] make a-quick quick again (and light on the AB, particularly arm)
On Wed, 2024-03-20 at 10:33 +0100, Alexander Kanavin wrote: > I feel that a-quick builds as they are have ceased to be quick or > lightweight, and I'd like to trim heavy items from them. The goal is > to: > > - have a-quick complete in under an hour with fully populated sstate > - lessen the load on arm workers in particular; there is not enough of > them, and they should be used only in a-full > - make a-quick useful as a quick smoke check for patchsets that > reveals obvious issues rapidly, completes quickly, and doesn't compete > with a-full for resources too much. I would then be willing to use > them for that purpose. > > The definitions of a-quick and a-full are here: > https://git.yoctoproject.org/yocto-autobuilder2/tree/config.py#n75 > > I'd like to make the following changes to reduce the load on arm workers: > - oe-selftest-armhost moves from shared to full. This is the biggest offender. > > - qemuarm64-armhost does the same > > There are also other heavy builds in a-quick that I think shouldn't be > a part of it: > > - reproducible should move from shared to full. There's no way to make > it quicker. > > - oe-selftest should be dropped from quick, or we could introduce > oe-selftest-quick that runs only a subset of sefltests that completes > under an hour (existing logs can be used to make that selection, e.g. > by tagging known slow test classes). > > Thoughts? a-quick was added with the idea of covering the key things which catch errors without the duplication that "full" has. In that sense, "quick" was less of the intent than allowing relaxing of some of the tests where it was possible to do so without breaking coverage too much. For various reasons people haven't really used "quick". Also, in general people do want to know about all the issues, including reproducibility and selftest (on arm). Host specific issues are comparatively rare on selftest now (gl passthrough being an exception) and we already stopped per-distro reproducibility testing. So I'm torn. It would be fine and easy to maintain another target alongside quick and full though so there isn't really a good reason not to add something if it would be useful. Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#62803): https://lists.yoctoproject.org/g/yocto/message/62803 Mute This Topic: https://lists.yoctoproject.org/mt/105041413/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] [qa-build-notification] QA notification for completed autobuilder build (yocto-5.0_M3.rc1)
On Fri, 2024-03-08 at 07:24 -0800, Pokybuild User wrote: > > A build flagged for QA (yocto-5.0_M3.rc1) was completed on the > autobuilder and is available at: > > > https://autobuilder.yocto.io/pub/releases/yocto-5.0_M3.rc1 > Somewhat surprisingly, this did build cleanly. I think we've therefore reached feature freeze and this can go into QA. Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#62710): https://lists.yoctoproject.org/g/yocto/message/62710 Mute This Topic: https://lists.yoctoproject.org/mt/104812280/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] Yocto Project Quick Build - qemu
On Thu, 2024-03-07 at 15:06 -0800, Worik Stanton wrote: > Friends > > I am starting from first principles and following Yocto Project Quick Build. > > I am at the step: Simulate Your Image Using QEMU: > > I am unfamiliar with qemu. But it has started up (I watched it spin its > wheels in top) but there is nothing to show for it. > > I am running in a Debian Bullseye LXC container as I have had trouble > building Yocto builds in Bookworm. Was that a bad decision? > > I am unsure if I have failed or succeeded! > > What I see in my terminal is: > > runqemu - INFO - Running MACHINE=qemux86-64 bitbake -e ... > runqemu - INFO - Continuing with the following parameters: > KERNEL: [/home/yocto/poky/build/tmp/deploy/images/qemux86-64/bzImage] > MACHINE: [qemux86-64] > FSTYPE: [ext4] > ROOTFS: > [/home/yocto/poky/build/tmp/deploy/images/qemux86-64/core-image-sato-qemux86-64.rootfs-20240307220730.ext4] > CONFFILE: > [/home/yocto/poky/build/tmp/deploy/images/qemux86-64/core-image-sato-qemux86-64.rootfs-20240307220730.qemuboot.conf] > > runqemu - INFO - Using preconfigured tap device tap0 > runqemu - INFO - If this is not intended, touch /tmp/qemu-tap-locks/tap0.skip > to make runqemu skip tap0. > runqemu - INFO - Network configuration: > ip=192.168.7.2::192.168.7.1:255.255.255.0::eth0:off:8.8.8.8 net.ifnames=0 > runqemu - INFO - Running > /home/yocto/poky/build/tmp/work/x86_64-linux/qemu-helper-native/1.0/recipe-sysroot-native/usr/bin/qemu-system-x86_64 > -device virtio-net-pci,netdev=net0,mac=52:54:00:12:34:02 -netdev > tap,id=net0,ifname=tap0,script=no,downscript=no -object > rng-random,filename=/dev/urandom,id=rng0 -device virtio-rng-pci,rng=rng0 > -drive > file=/home/yocto/poky/build/tmp/deploy/images/qemux86-64/core-image-sato-qemux86-64.rootfs-20240307220730.ext4,if=virtio,format=raw > -usb -device usb-tablet -usb -device usb-kbd -cpu IvyBridge -machine > q35,i8042=off -smp 4 -m 512 -serial mon:vc -serial null -device virtio-vga > -display sdl,show-cursor=on -kernel > /home/yocto/poky/build/tmp/deploy/images/qemux86-64/bzImage -append > 'root=/dev/vda rw > ip=192.168.7.2::192.168.7.1:255.255.255.0::eth0:off:8.8.8.8 net.ifnames=0 > oprofile.timer=1 tsc=reliable no_timer_check rcupdate.rcu_expedited=1 > swiotlb=0 ' > > runqemu - INFO - Host uptime: 3820.75 You should be able to ssh to root@192.168.7.2 Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#62701): https://lists.yoctoproject.org/g/yocto/message/62701 Mute This Topic: https://lists.yoctoproject.org/mt/104799142/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] [yocto-autobuilder-helper] [PATCH] config.json: Switch poky-altcfg to use ipk as the package backend
On Mon, 2024-03-04 at 16:34 +0100, Michael Opdenacker wrote:
> Hi Richard
>
> Many thanks for the patch!
>
> On 3/4/24 at 16:14, Richard Purdie wrote:
> > Since we plan to have better "binary distro" testing on the autobuilder
> > and we plan to primarily use ipk for this, having ipk based images available
> > as part of our builds is desireable.
> >
> > Since we want systemd based images with ipk, it makes sense to leave
> > standard
> > poky using rpm images but switch poky-altcfg to use ipk.
> >
> > Whilst late in the release cycle, doing this now for scarthgap gives us a
> > good
> > platform to build off in the long term too.
> >
> > Signed-off-by: Richard Purdie
> > ---
> > config.json | 5 +
> > 1 file changed, 5 insertions(+)
> >
> > diff --git a/config.json b/config.json
> > index fd8335f..4d0f258 100644
> > --- a/config.json
> > +++ b/config.json
> > @@ -127,6 +127,7 @@
> > "BUILDINFO" : true,
> > "BUILDHISTORY" : true,
> > "extravars" : [
> > + "PACKAGE_CLASSES = 'package_ipk package_rpm package_deb'",
> > "IMAGE_INSTALL:append = ' ssh-pregen-hostkeys'"
> > ],
> > "extratools" : "${EXTRATOOLS_URL}",
> > @@ -140,6 +141,7 @@
> > "BUILDINFO" : true,
> > "BUILDHISTORY" : true,
> > "extravars" : [
> > + "PACKAGE_CLASSES = 'package_ipk package_rpm package_deb'",
> > "IMAGE_INSTALL:append = ' ssh-pregen-hostkeys'"
> > ],
> > "extratools" : "${EXTRATOOLS_URL}",
> > @@ -198,6 +200,9 @@
> > "altcfg-hw" : {
> > "DISTRO" : "poky-altcfg",
> > "BUILDINFO" : true,
> > + "extravars" : [
> > + "PACKAGE_CLASSES = 'package_ipk package_rpm package_deb'"
> > + ],
> > "step1" : {
> > "BBTARGETS" : "core-image-full-cmdline core-image-sato
> > core-image-sato-sdk"
> > }
> >
> Actually, to use these images for testing package upgrades through
> feeds, we also need enough free space in the images.
> Could we also add something like IMAGE_ROOTFS_EXTRA_SPACE = "262144"?
Setting that "globally" for those templates will potentially break other
images the usual way to do it would be:
IMAGE_ROOTFS_EXTRA_SPACE:append = "+ 102400"
but I still worry about side effects, e.g. for minimal or initramfs images.
Which images do we plan to test? core-image-full-cmdline and what else?
We probably need a set of definitions like:
IMAGE_ROOTFS_EXTRA_SPACE:append:pn-core-image-full-cmdline = " + 262144"
but something like:
IMAGE_ROOTFS_EXTRA_SPACE:append = "${@bb.utils.contains("IMAGE_FEATURES,
"package-management", " + 262144", "", d)}"
might work.
Cheers,
Richard
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#62672): https://lists.yoctoproject.org/g/yocto/message/62672
Mute This Topic: https://lists.yoctoproject.org/mt/104723845/21656
Group Owner: yocto+ow...@lists.yoctoproject.org
Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[yocto] [yocto-autobuilder-helper] [PATCH] config.json: Switch poky-altcfg to use ipk as the package backend
Since we plan to have better "binary distro" testing on the autobuilder
and we plan to primarily use ipk for this, having ipk based images available
as part of our builds is desireable.
Since we want systemd based images with ipk, it makes sense to leave standard
poky using rpm images but switch poky-altcfg to use ipk.
Whilst late in the release cycle, doing this now for scarthgap gives us a good
platform to build off in the long term too.
Signed-off-by: Richard Purdie
---
config.json | 5 +
1 file changed, 5 insertions(+)
diff --git a/config.json b/config.json
index fd8335f..4d0f258 100644
--- a/config.json
+++ b/config.json
@@ -127,6 +127,7 @@
"BUILDINFO" : true,
"BUILDHISTORY" : true,
"extravars" : [
+"PACKAGE_CLASSES = 'package_ipk package_rpm package_deb'",
"IMAGE_INSTALL:append = ' ssh-pregen-hostkeys'"
],
"extratools" : "${EXTRATOOLS_URL}",
@@ -140,6 +141,7 @@
"BUILDINFO" : true,
"BUILDHISTORY" : true,
"extravars" : [
+"PACKAGE_CLASSES = 'package_ipk package_rpm package_deb'",
"IMAGE_INSTALL:append = ' ssh-pregen-hostkeys'"
],
"extratools" : "${EXTRATOOLS_URL}",
@@ -198,6 +200,9 @@
"altcfg-hw" : {
"DISTRO" : "poky-altcfg",
"BUILDINFO" : true,
+"extravars" : [
+"PACKAGE_CLASSES = 'package_ipk package_rpm package_deb'"
+],
"step1" : {
"BBTARGETS" : "core-image-full-cmdline core-image-sato
core-image-sato-sdk"
}
--
2.40.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#62665): https://lists.yoctoproject.org/g/yocto/message/62665
Mute This Topic: https://lists.yoctoproject.org/mt/104723845/21656
Group Owner: yocto+ow...@lists.yoctoproject.org
Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[yocto] [yocto-autobuilder-helper] [PATCH] config.json: Drop obsolete distros from master config
Signed-off-by: Richard Purdie
---
config.json | 11 +--
1 file changed, 1 insertion(+), 10 deletions(-)
diff --git a/config.json b/config.json
index 0ef4d54..a1fb16a 100644
--- a/config.json
+++ b/config.json
@@ -1868,20 +1868,11 @@
}
},
"buildtools" : {
-"debian8*" : "${BUILDTOOLS_URL}",
-"debian9*" : "${BUILDTOOLS_URL}",
-"centos7*" : "${BUILDTOOLS_URL}",
-"ubuntu1604*" : "${BUILDTOOLS_URL}",
-"ubuntu1804-ty-*" : "${BUILDTOOLS_URL}",
"ubuntu1804-arm-*" : "${BUILDTOOLS_ARM_URL}",
"alma8*" : "${BUILDTOOLS_URL}",
"stream8*" : "${BUILDTOOLS_URL}",
-"opensuse153*" : "${BUILDTOOLS_URL}",
"opensuse154*" : "${BUILDTOOLS_URL}",
"perf-alma8*" : "${BUILDTOOLS_URL}",
-"perf-centos7*" : "${BUILDTOOLS_URL}",
-"perf-ubuntu1604*" : "${BUILDTOOLS_URL}",
-"perf-ubuntu1804-ty-*" : "${BUILDTOOLS_URL}",
-"perf-ubuntu1804-arm-*" : "${BUILDTOOLS_ARM_URL}"
+"perf-ubuntu1804-ty-*" : "${BUILDTOOLS_URL}"
}
}
--
2.40.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#62663): https://lists.yoctoproject.org/g/yocto/message/62663
Mute This Topic: https://lists.yoctoproject.org/mt/104722179/21656
Group Owner: yocto+ow...@lists.yoctoproject.org
Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[yocto] [meta-mingw] [PATCH] gettext: Resolve locale alias file issue
Adapt to changes in OE-Core locale generation and match alias handling
with the change in glibc packaging. The file is only used for legacy
situations.
Signed-off-by: Richard Purdie
---
recipes-core/gettext/gettext_%.bbappend | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/recipes-core/gettext/gettext_%.bbappend
b/recipes-core/gettext/gettext_%.bbappend
index 4e18798..7bb9a4f 100644
--- a/recipes-core/gettext/gettext_%.bbappend
+++ b/recipes-core/gettext/gettext_%.bbappend
@@ -7,6 +7,7 @@ SRC_URI:append:mingw32 = " \
FILES:libgettextlib:mingw32 = "${bindir}/libgettextlib-*.dll"
FILES:libgettextsrc:mingw32 = "${bindir}/libgettextsrc-*.dll"
-PACKAGES:prepend:mingw32 = "libintl "
+PACKAGES:prepend:mingw32 = "libintl ${LOCALEBASEPN}-locale-alias "
FILES:libintl:mingw32 = "${bindir}/libintl*.dll"
+FILES:${LOCALEBASEPN}-locale-alias = "${datadir}/locale/locale.alias"
--
2.40.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#62632): https://lists.yoctoproject.org/g/yocto/message/62632
Mute This Topic: https://lists.yoctoproject.org/mt/104621902/21656
Group Owner: yocto+ow...@lists.yoctoproject.org
Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] 'yocto-kernel-tools' requires git version much greater than general Yocto requirement
On Fri, 2024-02-23 at 13:45 -0500, Bruce Ashfield wrote: > On Fri, Feb 23, 2024 at 12:15 PM Konstantin Aladyshev > wrote: > > > > Should I see the new git version in the linux-yocto devshell? > > """ > > bitbake linux-yocto -c devshell > > """ > > > > I've tried to check it in the current poky master, and it gives me: > > """ > > $/data/poky/build/workspace/sources/linux-yocto# git --version > > git version 2.30.2 > > $/data/poky/build/workspace/sources/linux-yocto# which git > > /data/poky/scripts/git > > """ > > So in this case host git is used. "bitbake kern-tools-native -c > > devshell" gives the same result. > > Can this be considered as a proof that git-native is not used? > > > > It is likely ASSUME_PROVIDED that is causing the issue, git-native is > in there, which means it is going to use the host version and not > build our -native recipe for it. > > We'll need some guidance from Richard on this, because the use of > that > flag in the git operations is in fact required for security reasons, > so it isn't going to be optional. If that means we need a minimum git > version or to use our own built one (and possibly the buildtools > tarball, etc), then those are the hoops we'll have to jump through. > > When I pulled git-native out of ASSUME_PROVIDED locally, I ended up > in > a dependency loop. > > I can have a closer look at this again on Monday, but i'm out of time > for it today. We do have a way to handle this. DEPENDS += "git-replacement-native" should do it. Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#62589): https://lists.yoctoproject.org/g/yocto/message/62589 Mute This Topic: https://lists.yoctoproject.org/mt/104509876/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] [PATCH yocto-autobuilder2] config.py: add meta-clang to a-full
On Fri, 2024-02-23 at 12:35 +, Ross Burton wrote: > From: Ross Burton > > Now that meta-clang builds successfully on the autobuilder, we can ensure > that it doesn't regress by enabling it in a-full builds. > > Signed-off-by: Ross Burton > --- > config.py | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/config.py b/config.py > index 652897e..89158f2 100644 > --- a/config.py > +++ b/config.py > @@ -106,7 +106,7 @@ trigger_builders_wait_quick = > trigger_builders_wait_shared + [ > trigger_builders_wait_full = trigger_builders_wait_shared + [ > "qemumips-alt", "edgerouter-alt", "qemuppc-alt", "qemux86-world-alt", > "oe-selftest-ubuntu", "oe-selftest-debian", "oe-selftest-fedora", > "oe-selftest-centos", > - "qemux86-64-ptest", "qemux86-64-ltp", "qemuarm64-ptest", "qemuarm64-ltp", > + "qemux86-64-ptest", "qemux86-64-ltp", "qemuarm64-ptest", > "qemuarm64-ltp", "meta-clang", > "meta-intel", "meta-arm", "meta-aws", "meta-agl-core", "meta-virt", > "qemuarmv5" > ] > This will break older releases. I think we'll have to do something like: https://git.yoctoproject.org/yocto-autobuilder2/commit/?id=c1591c59dac5a3e60550332ef757e2a75ac89b30 https://git.yoctoproject.org/yocto-autobuilder2/commit/?id=488f6f2b810e2587a9fcc780d6477a5f6739ae07 unless we want to entirely drop support for testing against the older releases in helper. Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#62581): https://lists.yoctoproject.org/g/yocto/message/62581 Mute This Topic: https://lists.yoctoproject.org/mt/104527175/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] [yocto-autobuilder-helper] [PATCH] Add bitbake-extra-tests to cover taskexp ncurses UI.
Signed-off-by: Richard Purdie
---
config.json | 4
scripts/bitbake-extra-tests | 14 ++
2 files changed, 18 insertions(+)
create mode 100755 scripts/bitbake-extra-tests
diff --git a/config.json b/config.json
index 3032fd7..3a2ab49 100644
--- a/config.json
+++ b/config.json
@@ -276,6 +276,10 @@
"step4" : {
"shortname" : "OEQA core tests",
"EXTRACMDS" : ["oe-test core"]
+},
+"step5" : {
+"shortname" : "Bitbake extra tests",
+"EXTRACMDS" : ["${SCRIPTSDIR}/bitbake-extra-tests"]
}
},
"reproducible" : {
diff --git a/scripts/bitbake-extra-tests b/scripts/bitbake-extra-tests
new file mode 100755
index 000..39f248d
--- /dev/null
+++ b/scripts/bitbake-extra-tests
@@ -0,0 +1,14 @@
+#!/bin/bash
+TASK_EXP_UNIT_TEST=1 bitbake -g -u taskexp_ncurses zlib acl
+if [ $? -ne 0 ]; then
+exit 1
+fi
+TASK_EXP_UNIT_TEST=1 bitbake -g -u taskexp_ncurses zlib acl foo
+if [ $? -ne 1 ]; then
+exit 1
+fi
+TASK_EXP_UNIT_TEST_NOTERM=1 bitbake -g -u taskexp_ncurses quilt
+if [ $? -ne 0 ]; then
+exit 1
+fi
+
--
2.40.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#62531): https://lists.yoctoproject.org/g/yocto/message/62531
Mute This Topic: https://lists.yoctoproject.org/mt/104426629/21656
Group Owner: yocto+ow...@lists.yoctoproject.org
Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [linux-yocto] [yocto-kernel-cache][PATCH] qemuriscv32/qemuriscv64: Enable Goldfish RTC
On Fri, 2024-02-16 at 01:05 -0800, Khem Raj wrote: > This is required for the qemu based riscv system to set system time > correctly. Otherwise, it falls back to defaults in /etc/timestamp > which > is set at image build time and is not current. > > Fixes > hwclock: can't open '/dev/misc/rtc': No such file or directory > Fri Mar 9 12:34:56 UTC 2018 > hwclock: can't open '/dev/misc/rtc': No such file or directory > hwclock: can't open '/dev/misc/rtc': No such file or directory > > Signed-off-by: Khem Raj > --- > bsp/qemuriscv32/qemuriscv32.cfg | 6 ++ > bsp/qemuriscv64/qemuriscv64.cfg | 6 ++ > 2 files changed, 12 insertions(+) Acked-by: Richard Purdie This solves a number of problems we're having on the autobuilder. Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#13572): https://lists.yoctoproject.org/g/linux-yocto/message/13572 Mute This Topic: https://lists.yoctoproject.org/mt/104389680/21656 Group Owner: linux-yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/linux-yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] bmap-tools repository closes
On Fri, 2024-01-26 at 08:26 -0700, Joshua Watt wrote: > I'd rather it stay on GitHub; I think it makes more sense _for this > project_. First of all, it already is on GitHub, so moving it is > likely more disruptive to the workflow > > Secondly, it already has CI setup through GitHub actions, which we > would need to spend effort to replicate elsewhere. > > Thirdly, I've always wanted this to be published as a python module on > PyPi to make it easier for users to keep up to date; GitHub and PyPi > have streamlined this process to the point that it's surprisingly > trivial. > > Finally, if we are worried about the maintenance, it's pretty easy to > delegate the maintenance of the repo to the people who are > volunteering to maintain it here, which is likely what would happen if > it went somewhere other than the Yocto GitHub account anyway. I hadn't realised there was CI in there, that does change things quite a bit. For bmap-tools it therefore makes sense. >From the wider Yocto Project perspective, this will cause problems as you can't just enable some bits of github on a per repo basis very easily but disable them for others. Obviously those are issues others will have to worry about, probably Michael and I. I do feel it makes sense to join bmap-tools to the wider Yocto Project umbrella of projects rather than letting it sit in isolation on a personal account. Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#62301): https://lists.yoctoproject.org/g/yocto/message/62301 Mute This Topic: https://lists.yoctoproject.org/mt/103930262/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] bmap-tools repository closes
On Thu, 2024-01-25 at 17:58 -0500, Philip Balister wrote: > On 1/25/24 11:58, Trevor Woerner wrote: > > On Wed 2024-01-24 @ 03:16:04 PM, Tim Orling wrote: > > > On Wed, Jan 24, 2024 at 2:02 PM Joshua Watt wrote: > > > > > > > On Wed, Jan 24, 2024 at 10:59 AM Trevor Woerner > > > > wrote: > > > > > > > > > > Hi Artem, > > > > > > > > > > On Wed 2024-01-24 @ 02:01:08 PM, Artem Bityutskiy wrote: > > > > > > Hello Yocto community, > > > > > > > > > > > > some parts of Yocto software (MIC?) use the 'bmap-tools' project to > > > > speed up > > > > > > image flashing. > > > > > > > > > > > > https://github.com/intel/bmap-tools > > > > > > > > > > > > I am the original author of the software, and I created it many > > > > > > years > > > > ago to > > > > > > speed up Tizen image flashing, and it helped a lot back at the time. > > > > It was also > > > > > > my first python project, so it was a lot of fun learning python > > > > > > while > > > > also > > > > > > creating something useful. > > > > > > > > > > Thank you for your contribution, this tool has been quite useful for > > > > > me > > > > over > > > > > the years. > > > > > > > > > > > But after that, I stopped working on it and it was mostly Yocto > > > > > > folks > > > > who > > > > > > contributed changes here and there. I never had time and enough > > > > motivation to > > > > > > maintain the project further, but other folks helped. > > > > > > > > > > > > Simon McVitte was active maintainer, but he said he does not have > > > > > > time > > > > for it > > > > > > now as well. > > > > > > > > > > > > The project is in "github.com/intel" space, and Intel is going to > > > > archive the > > > > > > git repository soon. This basically means the repository becomes > > > > read-only soon. > > > > > > > > > > Thank you for this update and letting us know ahead of time. > > > > > > > > > > > Would Yocto community have enthusiasts to fork it and maintain the > > > > fork? > > > > > > > > > > Yes, I'll volunteer to maintain it going forward. My non-stackoverflow > > > > python > > > > > knowledge is minimal, but I recently dug deeply into bmaptools to > > > > > solve > > > > an > > > > > issue I had noticed. So I'm confident enough to take it over if nobody > > > > else is > > > > > interested. > > > > > > > > Ya, it's an awesome tool and a huge time saver for us, so can also > > > > help maintain it > > > > > > > > > > We can probably move it under the https://github.com/yoctoproject > > > umbrella? > > > I can also help maintain this tremendous time saver. > > > > https://github.com/yoctoproject and the github workflow is fine with me as > > long as I can commit to it from my @gmail identity. > > > > @tim do you want to import the current repository from intel? Then we can > > update the recipe in oe-core and go from there. > > > > Michael, if you need any support from the Yocto Project advisory board > to move bmaptool to the yoctoproject github, I can help with that. This falls to the YP TSC, not the board. I have no problem with the project hosting that git repo and maintaining it. I do worry quite a bit about moving it to github. Would moiving it to git.yoctoproject.org be ok? The reason I say this is that I already get a lot of comments/randon questions against commits in github repos. I feel quite sad that people are effectively talking into a void. If we start supporting different workflows for different repos under yoctoproject, we're going to really confuse people. You might argue that we should host issues for bmap-tools. The challenge is that we have bugzilla, we have a triage team and we have a load of processes. Telling those people to just cover both would likely result in a flat refusal. Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#62298): https://lists.yoctoproject.org/g/yocto/message/62298 Mute This Topic: https://lists.yoctoproject.org/mt/103930262/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] [yocto-autobuilder2] [PATCH] schedulers: Run toaster tests daily
Signed-off-by: Richard Purdie
---
schedulers.py | 4
1 file changed, 4 insertions(+)
diff --git a/schedulers.py b/schedulers.py
index edd77b51..9ffa88f5 100644
--- a/schedulers.py
+++ b/schedulers.py
@@ -509,6 +509,10 @@
schedulers.append(sched.Nightly(name='nightly-meta-oe-mirror-dunfell', propertie
schedulers.append(sched.Nightly(name='nightly-buildperf', branch='master',
properties=parent_default_props('buildperf-debian11'),
builderNames=['buildperf-debian11', 'buildperf-alma8'],
hour=[3,9,15,21], minute=0))
+# Run the toaster tests daily
+schedulers.append(sched.Nightly(name='nightly-toaster', branch='master',
properties=parent_default_props('toaster'),
+ builderNames=['toaster'], hour=[3], minute=0))
+
# Run the AUH twice a month on 1st and 15th
schedulers.append(sched.Nightly(name='nightly-auh', branch='master',
properties=parent_default_props('auh'),
builderNames=['auh'], dayOfMonth=[1, 15], hour=5, minute=0))
--
2.40.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#62250): https://lists.yoctoproject.org/g/yocto/message/62250
Mute This Topic: https://lists.yoctoproject.org/mt/103920750/21656
Group Owner: yocto+ow...@lists.yoctoproject.org
Unsubscribe:
https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] Remove Python dependency from target package only
On Tue, 2024-01-23 at 21:58 +, Sebert, Holger.ext wrote:
> Hi Quentin,
>
>
> Von: Quentin Schulz
>
> > What does
> > bitbake-getvar -r libxml2-native PACKAGECONFIG
> > and
> > bitbake-getvar -r libxml2 PACKAGECONFIG
> >
> > return?
>
> Here is the output of the respective commands:
>
> $ bitbake-getvar -r libxml2-native PACKAGECONFIG
> #
> # $PACKAGECONFIG [6 operations]
> # :append[pn-qemu-system-native]
> /home/hosebert.ext/data/poky-python-getvar/build/conf/local.conf:251
> # " sdl"
> # :remove[class-target:pn-libxml2]
> /home/hosebert.ext/data/poky-python-getvar/build/conf/local.conf:292
> # "python"
> # set
> /home/hosebert.ext/data/poky-python-getvar/meta/conf/documentation.conf:320
> # [doc] "This variable provides a means of enabling or disabling features
> of a recipe on a per-recipe basis."
> # set
> /home/hosebert.ext/data/poky-python-getvar/meta/recipes-core/libxml/libxml2_2.11.5.bb:32
> # [_defaultval] "python ${@bb.utils.filter('DISTRO_FEATURES', 'ipv6',
> d)} "
> # set
> /home/hosebert.ext/data/poky-python-getvar/meta/recipes-core/libxml/libxml2_2.11.5.bb:33
> # [python] "--with-python=${PYTHON},--without-python,python3"
> # set
> /home/hosebert.ext/data/poky-python-getvar/meta/recipes-core/libxml/libxml2_2.11.5.bb:34
> # [ipv6] "--enable-ipv6,--disable-ipv6,"
> # pre-expansion value:
> # "python ${@bb.utils.filter('DISTRO_FEATURES', 'ipv6', d)} "
> PACKAGECONFIG="python ipv6 "
>
> $ bitbake-getvar -r libxml2 PACKAGECONFIG
> #
> # $PACKAGECONFIG [6 operations]
> # :append[pn-qemu-system-native]
> /home/hosebert.ext/data/poky-python-getvar/build/conf/local.conf:251
> # " sdl"
> # :remove[class-target:pn-libxml2]
> /home/hosebert.ext/data/poky-python-getvar/build/conf/local.conf:292
> # "python"
> # set
> /home/hosebert.ext/data/poky-python-getvar/meta/conf/documentation.conf:320
> # [doc] "This variable provides a means of enabling or disabling features
> of a recipe on a per-recipe basis."
> # set
> /home/hosebert.ext/data/poky-python-getvar/meta/recipes-core/libxml/libxml2_2.11.5.bb:32
> # [_defaultval] "python ${@bb.utils.filter('DISTRO_FEATURES', 'ipv6',
> d)} "
> # set
> /home/hosebert.ext/data/poky-python-getvar/meta/recipes-core/libxml/libxml2_2.11.5.bb:33
> # [python] "--with-python=${PYTHON},--without-python,python3"
> # set
> /home/hosebert.ext/data/poky-python-getvar/meta/recipes-core/libxml/libxml2_2.11.5.bb:34
> # [ipv6] "--enable-ipv6,--disable-ipv6,"
> # pre-expansion value:
> # "python ${@bb.utils.filter('DISTRO_FEATURES', 'ipv6', d)} "
> PACKAGECONFIG=" ipv6 "
>
> So ... PACKAGECONFIG seems o.k. Notice, however, that the value of the PYTHON
> variable is incorrect:
>
> $ bitbake-getvar -r libxml2-native PYTHON
> #
> # $PYTHON
> # set /home/hosebert.ext/data/poky-python-getvar/meta/conf/bitbake.conf:575
> # "${@sys.executable}"
> PYTHON="/usr/bin/python3"
>
> It points to the host's Python installation and not the one in Yocto!
I suspect this is related to:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=15361
and there is an explanation of what is happening there. I'm not
entirely happy about the behaviour of class- overrides and
inherit_defer but I'm not sure what to do about it.
Cheers,
Richard
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#62249): https://lists.yoctoproject.org/g/yocto/message/62249
Mute This Topic: https://lists.yoctoproject.org/mt/103871920/21656
Group Owner: yocto+ow...@lists.yoctoproject.org
Unsubscribe:
https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[yocto] [yocto-autobuilder-helper] [PATCH] cve-report: Add summary counts of CVEs by recipe
It is often helpful to know how many CVEs are open against a given recipe.
Add a summary table of this to the end of the CVE listing.
Signed-off-by: Richard Purdie
---
scripts/cve-report.py | 14 ++
1 file changed, 14 insertions(+)
diff --git a/scripts/cve-report.py b/scripts/cve-report.py
index 7a95668..203ea6d 100755
--- a/scripts/cve-report.py
+++ b/scripts/cve-report.py
@@ -12,6 +12,7 @@ with open(jsonfile) as f:
cvedata = json.load(f)
cves = dict()
+recipe_counts = {}
for recipe in cvedata['package']:
if recipe['name'] in ignored_recipes:
@@ -28,3 +29,16 @@ for recipe in cvedata['package']:
print("Found %d unpatched CVEs" % len(cves))
for cve in sorted(cves.keys()):
print("%s: %s https://web.nvd.nist.gov/view/vuln/detail?vulnId=%s *" %
(cve, cves[cve], cve))
+
+for cve in cves:
+recipename = cves[cve]
+if recipename in recipe_counts:
+recipe_counts[recipename] += 1
+else:
+recipe_counts[recipename] = 1
+
+
+print("\n")
+print("Summary of CVE counts by recipes:\n")
+for recipe, count in sorted(recipe_counts.items(), key=lambda x: x[1],
reverse=True):
+print(" %s: %s" % (recipe, count))
--
2.40.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#62243): https://lists.yoctoproject.org/g/yocto/message/62243
Mute This Topic: https://lists.yoctoproject.org/mt/103910282/21656
Group Owner: yocto+ow...@lists.yoctoproject.org
Unsubscribe:
https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] Remove Python dependency from target package only
On Sun, 2024-01-21 at 18:57 +, Sebert, Holger.ext wrote: > Hi, > > I am trying to remove the Python dependency in libxml2 for the target. > For the libxml2-native package it shall still be available, however. > > I have put this into my local.conf: > > PACKAGECONFIG:remove:pn-libxml2 = "python" Try: PACKAGECONFIG:remove:class-target:pn-libxml2 = "python" Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#62215): https://lists.yoctoproject.org/g/yocto/message/62215 Mute This Topic: https://lists.yoctoproject.org/mt/103871920/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] [PATCH yocto-autobuilder-helper] config.json: meta-agl now needs meta-oe
On Thu, 2024-01-18 at 13:32 +0100, Alexandre Belloni via
lists.yoctoproject.org wrote:
> From: Alexandre Belloni
>
> rng-tools has been moved to meta-oe and is a dependency of
> packagegroup-agl-core-boot
>
> Signed-off-by: Alexandre Belloni
> ---
> config.json | 5 +++--
> 1 file changed, 3 insertions(+), 2 deletions(-)
>
> diff --git a/config.json b/config.json
> index 4686068dc5b6..b92a4f237c96 100644
> --- a/config.json
> +++ b/config.json
> @@ -646,9 +646,10 @@
> }
> },
> "meta-agl-core" : {
> -"NEEDREPOS" : ["poky", "meta-agl"],
> +"NEEDREPOS" : ["poky", "meta-openembedded", "meta-agl"],
> "ADDLAYER" : [
> -"${BUILDDIR}/../meta-agl/meta-agl-core"
> +"${BUILDDIR}/../meta-agl/meta-agl-core",
> +"${BUILDDIR}/../meta-openembedded/meta-oe"
> ],
> "DISTRO" : "poky-agl",
> "BUILDINFO" : true,
We've decided not to do this for now so I'm not taking this.
Cheers,
Richard
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#62204): https://lists.yoctoproject.org/g/yocto/message/62204
Mute This Topic: https://lists.yoctoproject.org/mt/103806903/21656
Group Owner: yocto+ow...@lists.yoctoproject.org
Unsubscribe:
https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[yocto] [yocto-autobuilder2] [PATCH] schedulers: Drop mickledore for meta-oe/layer-checks
Signed-off-by: Richard Purdie
---
schedulers.py | 6 --
1 file changed, 6 deletions(-)
diff --git a/schedulers.py b/schedulers.py
index 4d7522a9..b8ff44b5 100644
--- a/schedulers.py
+++ b/schedulers.py
@@ -487,12 +487,6 @@
schedulers.append(sched.Nightly(name='nightly-check-layer-nanbield', properties=
schedulers.append(sched.Nightly(name='nightly-meta-oe-mirror-nanbield',
properties=parent_default_props('meta-oe-mirror', 'nanbield'),
builderNames=['meta-oe-mirror'], dayOfWeek=[3, 7], hour=2,
minute=0, codebases = {'' : {'branch' : 'nanbield'}}))
-# Run check-layer-nightly amd meta-oe-mirror twice a week for mickledore
-schedulers.append(sched.Nightly(name='nightly-check-layer-mickledore',
properties=parent_default_props('check-layer-nightly', 'mickledore'),
- builderNames=['check-layer-nightly'], dayOfWeek=[2, 5],
hour=2, minute=0, codebases = {'' : {'branch' : 'mickledore'}}))
-schedulers.append(sched.Nightly(name='nightly-meta-oe-mirror-mickledore',
properties=parent_default_props('meta-oe-mirror', 'mickledore'),
- builderNames=['meta-oe-mirror'], dayOfWeek=[2, 5], hour=2,
minute=0, codebases = {'' : {'branch' : 'mickledore'}}))
-
# Run check-layer-nightly and meta-oe-mirror twice a week for kirkstone
schedulers.append(sched.Nightly(name='nightly-check-layer-kirkstone',
properties=parent_default_props('check-layer-nightly', 'kirkstone'),
builderNames=['check-layer-nightly'], dayOfWeek=[0, 3],
hour=2, minute=0, codebases = {'' : {'branch' : 'kirkstone'}}))
--
2.39.2
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#62101): https://lists.yoctoproject.org/g/yocto/message/62101
Mute This Topic: https://lists.yoctoproject.org/mt/103596671/21656
Group Owner: yocto+ow...@lists.yoctoproject.org
Unsubscribe:
https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] CVE Scanners and Package Version
On Sat, 2023-12-23 at 02:47 -0800, fabian.hanke via lists.yoctoproject.org wrote: > we must provide a SBOM for our Yocto based product which will then be > used for (internal) CVE scanning by the security department. > Generating the base document in cycloneDX format is fairly easy > (thanks to the nature of Yocto). > But we do not know how to include information about CVE patches for > each package in the document. Not providing these, will cause a lot > of “false” feedback on CVEs for specific versions which are already > patched (but version number did not change). https://docs.yoctoproject.org/dev/dev-manual/vulnerabilities.html#vulnerability-check-at-build-time The cve-check tooling can check which issues are present and which are fixed in some way so that information is there. > This problem was also mentioned a few days ago in the presentation > from David Reyna: https://youtu.be/PegU1G1bA80?t=1127 . I like the > proposed solution of adding a vendor specific string to the package > version. But I'm still wondering: How would the CVE scanner vendor > know which CVEs are included in a yocto specific version and which > are not? The data could also be written into our SPDX SBoM information, offhand I'm not sure if it is already or not. > I hope this is the correct place to start a discussion (if not please > point me to the correct place): > Does anyone else also have the same problem with false feedback from > CVE scanners? How do you deal with it? The project has been focused around the cve-check tooling and SPDX SBoM generation. If you want to use that data we'd suggest you extract it from those sources as the proejct iself doesn't want to try and generate multiple different output formats. Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#62032): https://lists.yoctoproject.org/g/yocto/message/62032 Mute This Topic: https://lists.yoctoproject.org/mt/103332846/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] FW: Yocto and Red Hat 8
On Mon, 2023-12-18 at 12:28 +, Monsees, Steven C (US) wrote: > Is there any documentation or guidance you can provide on porting > Yocto for centos7 to 8 ? > I am currently working centos7 ZEUS… According to: https://git.yoctoproject.org/yocto-autobuilder2/tree/config.py#n177 we can build dunfell on Centos 8 and are testing this on our autobuilder. Dunfell is our oldest LTS and expires in April. Zeus is EoL and no longer maintained or tested by us. If you need to stick with Zeus, you'd have to look at the fixes in Dunfell and backport as needed. Upgrading to Dunfell or preferably kirkstone should mean things work on centos 8 out the box for OE-Core. Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#61970): https://lists.yoctoproject.org/g/yocto/message/61970 Mute This Topic: https://lists.yoctoproject.org/mt/103241390/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] [qa-build-notification] QA notification for completed autobuilder build (yocto-5.0_M1.rc3)
On Mon, 2023-12-18 at 14:43 +, Pokybuild User wrote: > A build flagged for QA (yocto-5.0_M1.rc3) was completed on the > autobuilder and is available at: > > > https://autobuilder.yocto.io/pub/releases/yocto-5.0_M1.rc3 > > > Build URL: > https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/6345 > > Build hash information: > > bitbake: 8295ac1b6672c25bee595cff6e000b2af817f904 > meta-agl: 266ac5373b4458d7fcb78b1e8948eb2b33f18b5d > meta-arm: e6e7ac4d99f103d26a0ffefd6d0a5d2deef5b17c > meta-aws: 7915e6bafd876ab3d654a26629dbf501479caae0 > meta-intel: 5cfefd9a8ff1f5a3534c1ba9d7d7f6971ed5d56f > meta-mingw: e01d47d183945caeaf3c5c086539af0a925ddc32 > meta-openembedded: 0a0ea87b8dda01a2887a525cef78eb6c3f4c2c32 > meta-virtualization: caa14c63f158fdd13382ccf1ff4e20a8ba6ad667 > oecore: 6b729088dce302eb3a967cb6839f00488025be0e > poky: 33112178d164ddd9ef0b1115c254ad4341ec3ad1 > > > > This is an automated message from the Yocto Project Autobuilder > Git: git://git.yoctoproject.org/yocto-autobuilder2 > Email: richard.pur...@linuxfoundation.org > There was one autobuilder failure against this build in qemux86 but it is a known issue where we're waiting for upstream to merge a kernel fix for alternatives handling. I believe we can proceed for QA and it does not block release. I'd also note the regression report generation was successful :) Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#61968): https://lists.yoctoproject.org/g/yocto/message/61968 Mute This Topic: https://lists.yoctoproject.org/mt/103246711/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] [qa-build-notification] QA notification for completed autobuilder build (yocto-5.0_M2.rc2)
On Mon, 2023-12-18 at 02:03 +, Pokybuild User wrote: > A build flagged for QA (yocto-5.0_M2.rc2) was completed on the > autobuilder and is available at: > > > https://autobuilder.yocto.io/pub/releases/yocto-5.0_M2.rc2 > > > Build URL: > https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/6344 > > Build hash information: > > bitbake: 834e452243ff2eea6e8e2e7f4935b5233ffb4b00 > meta-agl: 266ac5373b4458d7fcb78b1e8948eb2b33f18b5d > meta-arm: e6e7ac4d99f103d26a0ffefd6d0a5d2deef5b17c > meta-aws: 7915e6bafd876ab3d654a26629dbf501479caae0 > meta-intel: 5cfefd9a8ff1f5a3534c1ba9d7d7f6971ed5d56f > meta-mingw: e01d47d183945caeaf3c5c086539af0a925ddc32 > meta-openembedded: 0a0ea87b8dda01a2887a525cef78eb6c3f4c2c32 > meta-virtualization: caa14c63f158fdd13382ccf1ff4e20a8ba6ad667 > oecore: bb9f2a9c0ff5dcdeaf1a0beb6a614d0d022a2481 > poky: a49de3ccc11faedf5a98963c39d67f8dd5bdf6e1 > > > > This is an automated message from the Yocto Project Autobuilder > Git: git://git.yoctoproject.org/yocto-autobuilder2 > Email: richard.pur...@linuxfoundation.org I somehow managed to select M2 instead of M1 for rc2 so this one should be ignored. The build has successful CDN testing but had two 2 outright qemuarm64 ptest failure issues and 1 failed test warning. We could use rc1 or try an rc3, I'm open to input from QA on the timings at this point. Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#61955): https://lists.yoctoproject.org/g/yocto/message/61955 Mute This Topic: https://lists.yoctoproject.org/mt/103239452/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] [yocto-autobuilder-helper] [PATCH] config.json: Add rpm builds to pkgman-non-rpm
The CDN tests fail since do_package is run without rpm's being generated.
With hash equivalence, this can lead to missing sstate artefacts from the
CDN.
Signed-off-by: Richard Purdie
---
config.json | 12
1 file changed, 12 insertions(+)
diff --git a/config.json b/config.json
index 6d9a4ed..d504d07 100644
--- a/config.json
+++ b/config.json
@@ -966,6 +966,18 @@
"shortname" : "arm deb",
"MACHINE" : "qemuarm64",
"PACKAGE_CLASSES" : "package_deb"
+},
+"step5" : {
+"shortname" : "x86 rpm (for CDN)",
+"MACHINE" : "qemux86-64",
+"PACKAGE_CLASSES" : "package_rpm",
+"SANITYTARGETS" : ""
+},
+"step6" : {
+"shortname" : "arm rpm (for CDN)",
+"MACHINE" : "qemuarm64",
+"PACKAGE_CLASSES" : "package_rpm",
+"SANITYTARGETS" : ""
}
},
"poky-tiny" : {
--
2.39.2
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#61944): https://lists.yoctoproject.org/g/yocto/message/61944
Mute This Topic: https://lists.yoctoproject.org/mt/103224247/21656
Group Owner: yocto+ow...@lists.yoctoproject.org
Unsubscribe:
https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[yocto] QA notification for completed autobuilder build (yocto-5.0_M1.rc1)
A build flagged for QA (yocto-5.0_M1.rc1) was completed on the autobuilder and is available at: https://autobuilder.yocto.io/pub/releases/yocto-5.0_M1.rc1 Build URL: https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/6338 Build hash information: bitbake: eb417e27be5717a259f27e98dbd73255b1a42fc9 meta-agl: 0e41f2f4671866c110fc8c8b5c10a38b4aeb7b96 meta-arm: e6e7ac4d99f103d26a0ffefd6d0a5d2deef5b17c meta-aws: 7915e6bafd876ab3d654a26629dbf501479caae0 meta-intel: 5cfefd9a8ff1f5a3534c1ba9d7d7f6971ed5d56f meta-mingw: e01d47d183945caeaf3c5c086539af0a925ddc32 meta-openembedded: 0a0ea87b8dda01a2887a525cef78eb6c3f4c2c32 meta-virtualization: caa14c63f158fdd13382ccf1ff4e20a8ba6ad667 oecore: 7ebd5247f6e040234ca4c8b3b1239843386bb79b poky: 5815ccc1f6fd93690a684f7ff0f8a0a19758fdaa This is an automated message from the Yocto Project Autobuilder Git: git://git.yoctoproject.org/yocto-autobuilder2 Email: richard.pur...@linuxfoundation.org [RP: I'm sending the QA manually as the final step of the build failed with a reporting problem. The regression report was not generated. There was also a single build failure with the CDN tests. Given this is a known issue being worked upon, I believe we should proceed with QA. Both issues will be investigated. Sorry the build has been as delayed, I had hoped to have several fixes for intermittent issues but things are proving tricky to resolve.] -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#61943): https://lists.yoctoproject.org/g/yocto/message/61943 Mute This Topic: https://lists.yoctoproject.org/mt/103222904/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] [yocto-autobuilder-helper] [PATCH] config.json: Update buildtools to 4.3 release tarballs
The older buildtools has a bug that zoneinfo is missing. Update to a newer
release version to fix this.
Signed-off-by: Richard Purdie
---
config.json | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/config.json b/config.json
index c5fca45..6d9a4ed 100644
--- a/config.json
+++ b/config.json
@@ -12,9 +12,9 @@
"BUILDTOOLS_URL_TEMPLOCAL3" :
"https://downloads.yoctoproject.org/releases/yocto/yocto-4.1/buildtools/x86_64-buildtools-extended-nativesdk-standalone-4.1.sh;,
"BUILDTOOLS_ARM_URL_TEMPLOCAL" :
"https://downloads.yoctoproject.org/releases/yocto/yocto-4.1/buildtools/aarch64-buildtools-extended-nativesdk-standalone-4.1.sh;,
-"BUILDTOOLS_URL" :
"/srv/autobuilder/autobuilder.yocto.io/pub/non-release/20230223-18/buildtools/x86_64-buildtools-extended-nativesdk-standalone-4.1+snapshot-9c07de0f20042c81496185293857284048d7912c.sh",
-"BUILDTOOLS_ARM_URL" :
"/srv/autobuilder/autobuilder.yocto.io/pub/non-release/20230223-18/buildtools/aarch64-buildtools-extended-nativesdk-standalone-4.1+snapshot-9c07de0f20042c81496185293857284048d7912c.sh",
-"BUILDTOOLS_MAKE_URL" :
"https://downloads.yoctoproject.org/releases/yocto/yocto-4.1/buildtools/x86_64-buildtools-make-nativesdk-standalone-4.1.sh;,
+"BUILDTOOLS_URL" :
"https://downloads.yoctoproject.org/releases/yocto/yocto-4.3/buildtools/x86_64-buildtools-extended-nativesdk-standalone-4.3.sh;,
+"BUILDTOOLS_ARM_URL" :
"https://downloads.yoctoproject.org/releases/yocto/yocto-4.3/buildtools/aarch64-buildtools-extended-nativesdk-standalone-4.3.sh;,
+"BUILDTOOLS_MAKE_URL" :
"https://downloads.yoctoproject.org/releases/yocto/yocto-4.3/buildtools/x86_64-buildtools-make-nativesdk-standalone-4.3.sh;,
"REPO_STASH_DIR" : "${BASE_HOMEDIR}/git/mirror",
"TRASH_DIR" : "${BASE_HOMEDIR}/git/trash",
--
2.39.2
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#61820): https://lists.yoctoproject.org/g/yocto/message/61820
Mute This Topic: https://lists.yoctoproject.org/mt/103023736/21656
Group Owner: yocto+ow...@lists.yoctoproject.org
Unsubscribe:
https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] [RFC] devtool: support multiple sources
On Wed, 2023-11-22 at 13:53 +0100, Julien Stephan wrote: > Le mar. 14 nov. 2023 à 12:21, Alexander Kanavin > a écrit : > > > > I think no one actually anticipated all these corner cases when > > devtool was written. In general it's simply not designed to handle > > multiple sources in SRC_URI, it can do (tarball or git)+patches, and > > not any of the other options. No one is expecting a fix for all of > > them, perhaps it's best to keep the scope to plain gitsm://. > > > > Hi Alex, > > I sent a series to add submodule support. As you suggested I focused > myself on gitsm recipes, but I had to handle the situation of a git > tree extracted into S because it was throwing an error now with my > submodule implementation (while it was not working but not throwing > any error on master). > > > I don't have specific advice for the broader issue; if you have ideas, > > it would be good to at least document them. > > > > Alex > > > For the other cases, I was thinking of using the newly merged API > "unpack tracer" > (https://git.yoctoproject.org/poky/commit/?id=ef3e46afd910d4b7727d42c4c18b501525c65695) > (suggested by RP in this thread > https://git.yoctoproject.org/poky/commit/?id=ef3e46afd910d4b7727d42c4c18b501525c65695). > I really think that we could leverage this new api to create a devtool > unpack tracer, so we can have a better idea of what is unpacked and > where. This could help to solve the corner cases I described > previously but also the bug 15162 > (https://bugzilla.yoctoproject.org/show_bug.cgi?id=15162). I think it > may also help to simplify devtool code in various places. > Where should I document this? > Should I open a new ticket? Yes please, it would be good to at least document the idea as I do think it could help devtool with a number of the corner cases. A bugzilla bug for it is the place to do it. Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#61732): https://lists.yoctoproject.org/g/yocto/message/61732 Mute This Topic: https://lists.yoctoproject.org/mt/102509834/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] [yocto-autobuilder-helper] Add toaster test builder
On Mon, 2023-11-13 at 22:30 +, Richard Purdie via
lists.yoctoproject.org wrote:
> On Mon, 2023-11-13 at 13:21 -0500,
> alexander.lussier-cul...@savoirfairelinux.com wrote:
> > From: Alexander Lussier-Cullen
> >
> >
> > Add a toaster test builder that runs the tox test suite using a new
> > run-toaster-tests script.
> >
> > Signed-off-by: Alexander Lussier-Cullen
> >
> > CC: richard.pur...@linuxfoundation.org
> > ---
> > config.json | 3 +++
> > scripts/run-toaster-tests | 22 ++
> > 2 files changed, 25 insertions(+)
> > create mode 100644 scripts/run-toaster-tests
> >
> > diff --git a/config.json b/config.json
> > index d10a3a8..0c581d1 100644
> > --- a/config.json
> > +++ b/config.json
> > @@ -1675,6 +1675,9 @@
> > "step1" : {
> > "BBTARGETS" : "core-image-patchtest"
> > }
> > +},
> > +"toaster" : {
> > +"EXTRACMDS" : ["${SCRIPTSDIR}/run-toaster-tests
> > ${HELPERBUILDDIR} ${HELPERBUILDDIR}/.."]
> > }
> > },
> > "repo-defaults" : {
> > diff --git a/scripts/run-toaster-tests b/scripts/run-toaster-tests
> > new file mode 100644
> > index 000..a785c36
> > --- /dev/null
> > +++ b/scripts/run-toaster-tests
> > @@ -0,0 +1,22 @@
> > +#!/bin/bash
> > +#
> > +# SPDX-License-Identifier: GPL-2.0-only
> > +#
> > +# Called with $1 as the build directory
> > +# $2 as the path to poky
> > +
> > +set -e
> > +set -u
> > +set -o pipefail
> > +set -x
> > +
> > +builddir=$(realpath "$1")
> > +pokydir=$(realpath "$2")
> > +
> > +cd $builddir
> > +mkdir -p toaster_logs
> > +python3 -m venv venv --without-pip --system-site-packages
> > +source venv/bin/activate
> > +python3 -m pip install tox
> > +
> > +tox -c $pokydir/bitbake/lib/toaster/tox.ini
>
> Thanks! I tested this and the first issue was:
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/161/builds/1
>
> so I added a chmod a+x onto the commit which is on master-next of
> yocto-autobuilder-helper.
>
> This lead to:
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/161/builds/2
> and
> https://autobuilder.yoctoproject.org/typhoon/#/builders/161/builds/3
>
> which were because a worker with python 3.8/3.9/3.10 wasn't available
> at the time of testing and I'm not restricting the worker set the
> target is using.
>
> I put a commit onto bitbake/poky master-next to add 3.11 and 3.12 to
> tox.ini which leads to:
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/161/builds/4
>
> which does appear to be running some testing with some errors.
>
> So we're making progress :)
https://autobuilder.yoctoproject.org/typhoon/#/builders/161/builds/4
(on ubuntu 23.04 with python3.11) ran to completion in 45 minutes,
failures=2, errors=18.
It'd note that it didn't use SSTATE_DIR or DL_DIR from the parent
environment which is partly why it was so slow, we need to fix that.
I tried adding some of the pending toaster patches to master-next and
rebuilding. There were more workers available so it tried some
different combinations which failed:
https://autobuilder.yoctoproject.org/typhoon/#/builders/161/builds/5
opensuse 15.4 python 3.11
File "/home/pokybuild/yocto-
worker/toaster/build/build/.tox/py311/lib/python3.11/site-
packages/django/utils/timezone.py", line 12, in
from backports import zoneinfo
ModuleNotFoundError: No module named 'backports'
https://autobuilder.yoctoproject.org/typhoon/#/builders/161/builds/6
ubuntu 20.04 python 3.8
File "/usr/lib/python3.8/importlib/metadata.py", line 79, in load
return functools.reduce(getattr, attrs, module)
AttributeError: module
'virtualenv.create.via_global_ref.builtin.cpython.mac_os' has no attribute
'CPython2macOsFramework'
I've set https://autobuilder.yoctoproject.org/typhoon/#/builders/161/builds/7
on ubuntu 23.04 away so we can see if things improve or not with the patches.
Cheers,
Richard
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#61675): https://lists.yoctoproject.org/g/yocto/message/61675
Mute This Topic: https://lists.yoctoproject.org/mt/102567727/21656
Group Owner: yocto+ow...@lists.yoctoproject.org
Unsubscribe:
https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] [yocto-autobuilder-helper] Add toaster test builder
On Mon, 2023-11-13 at 13:21 -0500,
alexander.lussier-cul...@savoirfairelinux.com wrote:
> From: Alexander Lussier-Cullen
>
> Add a toaster test builder that runs the tox test suite using a new
> run-toaster-tests script.
>
> Signed-off-by: Alexander Lussier-Cullen
>
> CC: richard.pur...@linuxfoundation.org
> ---
> config.json | 3 +++
> scripts/run-toaster-tests | 22 ++
> 2 files changed, 25 insertions(+)
> create mode 100644 scripts/run-toaster-tests
>
> diff --git a/config.json b/config.json
> index d10a3a8..0c581d1 100644
> --- a/config.json
> +++ b/config.json
> @@ -1675,6 +1675,9 @@
> "step1" : {
> "BBTARGETS" : "core-image-patchtest"
> }
> +},
> +"toaster" : {
> +"EXTRACMDS" : ["${SCRIPTSDIR}/run-toaster-tests
> ${HELPERBUILDDIR} ${HELPERBUILDDIR}/.."]
> }
> },
> "repo-defaults" : {
> diff --git a/scripts/run-toaster-tests b/scripts/run-toaster-tests
> new file mode 100644
> index 000..a785c36
> --- /dev/null
> +++ b/scripts/run-toaster-tests
> @@ -0,0 +1,22 @@
> +#!/bin/bash
> +#
> +# SPDX-License-Identifier: GPL-2.0-only
> +#
> +# Called with $1 as the build directory
> +# $2 as the path to poky
> +
> +set -e
> +set -u
> +set -o pipefail
> +set -x
> +
> +builddir=$(realpath "$1")
> +pokydir=$(realpath "$2")
> +
> +cd $builddir
> +mkdir -p toaster_logs
> +python3 -m venv venv --without-pip --system-site-packages
> +source venv/bin/activate
> +python3 -m pip install tox
> +
> +tox -c $pokydir/bitbake/lib/toaster/tox.ini
Thanks! I tested this and the first issue was:
https://autobuilder.yoctoproject.org/typhoon/#/builders/161/builds/1
so I added a chmod a+x onto the commit which is on master-next of
yocto-autobuilder-helper.
This lead to:
https://autobuilder.yoctoproject.org/typhoon/#/builders/161/builds/2
and
https://autobuilder.yoctoproject.org/typhoon/#/builders/161/builds/3
which were because a worker with python 3.8/3.9/3.10 wasn't available
at the time of testing and I'm not restricting the worker set the
target is using.
I put a commit onto bitbake/poky master-next to add 3.11 and 3.12 to
tox.ini which leads to:
https://autobuilder.yoctoproject.org/typhoon/#/builders/161/builds/4
which does appear to be running some testing with some errors.
So we're making progress :)
Cheers,
Richard
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#61670): https://lists.yoctoproject.org/g/yocto/message/61670
Mute This Topic: https://lists.yoctoproject.org/mt/102567727/21656
Group Owner: yocto+ow...@lists.yoctoproject.org
Unsubscribe:
https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] [ANNOUNCEMENT] Yocto Project 4.3 is Released
On Fri, 2023-11-10 at 15:57 +0100, Michael Opdenacker via lists.yoctoproject.org wrote: > Hi Lee > > On 09.11.23 at 16:29, Lee Chee Yang wrote: > > > > Hi > > > > We are pleased to announce the Yocto Project 4.3 Release is now > > available for download. > > > > > > https://downloads.yoctoproject.org/releases/yocto/yocto-4.3/poky-15b576c4101231d248fda7ae0824e1780e1a8901.tar.bz2 > > > > http://mirrors.kernel.org/yocto/yocto/yocto-4.3/poky-15b576c4101231d248fda7ae0824e1780e1a8901.tar.bz2 > > > > A gpg signed version of these release notes is available at: > > > > http://downloads.yoctoproject.org/releases/yocto/yocto-4.3/RELEASENOTES > > > > Many thanks for the release. Maybe fore the next major release you could > mention the HTML version of the release notes, as well as the migration > notes, as they were available when the release was made: > https://docs.yoctoproject.org/dev/migration-guides/release-4.3.html > > That's a little better than the raw rst sources :) > > For release updates, I'm not sure that's possible, as the release notes > are usually available in HTML after the release is made. There is always the option of tweaking the process to change that? Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#61652): https://lists.yoctoproject.org/g/yocto/message/61652 Mute This Topic: https://lists.yoctoproject.org/mt/102487580/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] [OE-core] Core workflow: sstate for all, bblock/bbunlock, tools for why is sstate not being reused?
On Sat, 2023-11-04 at 11:29 +0100, adrian.freiho...@gmail.com wrote:
> Hi Alex, hi Richard
>
> After some internal discussions, I would like to clarify my previous
> answers on this topic.
>
> * Usually there are two different workflows
> - application developers: could use an SDK with a locked sstate-cache.
> - Yocto/BSP developers: need an unlocked SDK. They change the recipes.
> * A locked SDK
> - can work with setscene from SSTATE_MIRRORS
> - setscene does caching in the SSTATE_DIR (no issue about that)
> - But network problems can occur during the initial build because
> bitbake executes many independent setscene tasks. Opening so many
> independent connections slows down the build, especially if the
> server treats them as a denial of service attack.
> - The denial of service problem is difficult to solve because each
> setscene task runs in its own bibtake task. Reusing a connection to
> download multiple sstate artifacts seems almost impossible.
> This is much easier to solve with separate sstate download script.
FWIW, we did have a similar issue with do_fetch overloading
servers/proxies/ISPs and added:
do_fetch[number_threads] = "4"
Finding the right place to put a thread limit on overall setscene tasks
is harder but in theory possible. Or perhaps a "network capable tasks"
thread limit?
Is the overload caused by the initial query of sstate presence, or,
does it happen when the setscene tasks themselves run?
> * An unlocked SDK
> - Tries to download the sstate cache for changed recipes and their
> dependencies, which obviously can't work.
> - The useless download requests slow down the build considerably and
> cause a high load on the servers without any benefit.
Is this sstate over http(s) or something else? I seem to remember you
mentioning sftp. If this were using sftp, it would be horribly slow as
it was designed for a light overhead "does this exist?" check which
http(s) can manage well.
Recently we've been wondering about teaching the hashequiv server about
"presence", which would then mean the build would only query things
that stood a good chance of existing.
> - A script which gets a list of sstate artifacts from bitbake and then
> does a upfront download works much better
>+ The script runs only when the user calls it or the SDK gets boot-
> strapped
>+ The script uses a reasonable amount of parallel connections which
> are re-used for more then one artifact download
Explaining to users they need to do X before Y quickly gets tiring,
both for people explaining it and the people doing it trying to
remember. I'd really like to get to a point where the system "does the
right thing" if we can.
I don't believe the problems you describe are insurmountable. If you
are using sftp, that is going to be a big chunk of the problem as the
system assumes something faster is available. Yes, I've taken patches
to make sftp work but it isn't recommended at all. I appreciate there
would be reasons why you use sftp but if it is possible to get a list
of "available sstate" via other means, it would improve things.
> * Idea for a smart lock/unlock implementation
> - Form a user's perspective a locked vs. an unlocked SDK does not make
> much sense. It makes more sense if the SDK would automatically
> download the sstate-cache if it is expected to be available.
> Lets think about an implementation (which allows to override the
> logic) to switch from automatic to manual mode:
>
> SSTATE_MIRRORS_ENABLED ?= "${is_sstate_mirror_available()}"
What determines this availability? I worry that is something very
fragile and specific to your use case. It is also not an all or nothing
binary thing.
> In our case the sstate mirror is expected to provide all artifacts
> for tagged commits and for some git branches of the layer
> repositories.
> The sstate is obviousely not usable for a "dirty" git layer
> repository.
That isn't correct and isn't going to work. If I make a single change
locally, there is a good chance that 99.9% of the sstate could still be
valid in some cases. Forcing the user through 10 hours of rebuild when
potentially that much was available is a really really bad user
experience.
> That's what the is_sstate_mirror_available function
> could check to automatically enable and disable lazy downloads.
>
> - If is_sstate_mirror_available() returns false, it should still be
> possible to initiate a sstate-cache download manually.
>
> * Terminology
> - Older Yocto Releases:
>+ eSDK means an installer which provides a different environment with
> different tools
>+ The eSDK was static, with a locked sstate cache
>+ Was for one MACHINE, for one image...
> - Newer Yocto Releases:
>+ The bitbake environment offers all
[yocto] [yocto-autobuilder-helper] [PATCH] [dunfell] config.json/scripts: Update to handle CVE checks for meta-oe
Backport the recent CVE script changes from the master branch and add in
cvelayer.bbclass
as a way to exclude specific paths from the CVE check. Master can use the layer
overrides
to do this but these are not present on kirkstone.
Signed-off-by: Richard Purdie
---
config.json | 33 --
scripts/cve-generate-chartdata| 65 ++
scripts/cve-report.py | 30 +
scripts/cvecheck/cvelayer.bbclass | 9 +++
scripts/run-cvecheck | 106 ++
5 files changed, 212 insertions(+), 31 deletions(-)
create mode 100755 scripts/cve-generate-chartdata
create mode 100755 scripts/cve-report.py
create mode 100644 scripts/cvecheck/cvelayer.bbclass
diff --git a/config.json b/config.json
index c5d478d..3520e9a 100644
--- a/config.json
+++ b/config.json
@@ -835,7 +835,7 @@
}
},
"metrics" : {
-"NEEDREPOS" : ["poky"],
+"NEEDREPOS" : ["poky", "meta-openembedded"],
"extravars" : [
"INHERIT += 'cve-check'",
"BB_DISKMON_DIRS = ''",
@@ -845,12 +845,35 @@
"BB_SERVER_TIMEOUT = '0'"
],
"step1" : {
-"shortname" : "Generating patch metrics",
-"EXTRACMDS" :
["../../yocto-autobuilder-helper/scripts/run-patchmetrics ../ ../meta/
${HELPERRESULTSDIR}/../../patchmetrics . ${HELPERBRANCHNAME}"]
+"shortname" : "Fetching metrics repositories",
+"EXTRAPLAINCMDS" : [
+"git clone ssh://g...@push.yoctoproject.org/yocto-metrics
&& git clone ssh://g...@push.yoctoproject.org/yocto-metrics-meta-oe",
+"install -D ${SCRIPTSDIR}/cvecheck/cvelayer.bbclass
./build/classes/cvelayer.bbclass"
+]
},
"step2" : {
-"shortname" : "Running CVE checks",
-"EXTRACMDS" :
["../../yocto-autobuilder-helper/scripts/run-cvecheck ../ ../meta/
${HELPERRESULTSDIR}/../../patchmetrics . ${HELPERBRANCHNAME}"]
+"shortname" : "CVE checks for meta",
+"EXTRACMDS" : ["${SCRIPTSDIR}/run-cvecheck --metrics
../yocto-metrics --branch ${HELPERBRANCHNAME} --results
${HELPERRESULTSDIR}/../../patchmetrics --push"]
+},
+"step3" : {
+"shortname" : "CVE checks for meta-oe",
+"ADDLAYER" : [
+"${BUILDDIR}/../meta-openembedded/meta-oe",
+"${BUILDDIR}/../meta-openembedded/meta-python",
+"${BUILDDIR}/../meta-openembedded/meta-perl",
+"${BUILDDIR}/../meta-openembedded/meta-networking",
+"${BUILDDIR}/../meta-openembedded/meta-multimedia",
+"${BUILDDIR}/../meta-openembedded/meta-gnome",
+"${BUILDDIR}/../meta-openembedded/meta-xfce",
+"${BUILDDIR}/../meta-openembedded/meta-filesystems",
+"${BUILDDIR}/../meta-openembedded/meta-initramfs",
+"${BUILDDIR}/../meta-openembedded/meta-webserver"
+],
+"extravars" : [
+"INHERIT += 'cvelayer'",
+"SKIP_PATHS += '${TOPDIR}/../meta'"
+],
+"EXTRACMDS" : ["${SCRIPTSDIR}/run-cvecheck --metrics
../yocto-metrics-meta-oe --branch ${HELPERBRANCHNAME} --results
${HELPERRESULTSDIR}/../../patchmetrics-meta-oe --push"]
}
},
"meta-mingw" : {
diff --git a/scripts/cve-generate-chartdata b/scripts/cve-generate-chartdata
new file mode 100755
index 000..dbbbe82
--- /dev/null
+++ b/scripts/cve-generate-chartdata
@@ -0,0 +1,65 @@
+#!/usr/bin/env python3
+import json, os.path, collections
+import sys
+import argparse
+import subprocess
+import tempfile
+from datetime import datetime, date, timedelta
+
+args = argparse.ArgumentParser(description="Generate CVE count data files")
+args.add_argument("-j", "--json", help="JSON data file to use")
+args.add_argument("-r", "--resultsdir", help="results directory to parse")
+args = args.parse_args()
+
+try:
+with open(args.json) as f:
+counts = json.load(f)
+except FileNotFoundError:
+# if the file does not exist, start with an empty database.
+counts = {}
+
+lastyear = {}
+
+#
+# Write CVE counts by day
+#
+
[yocto] [yocto-autobuilder-helper] [PATCH] [kirkstone] config.json/scripts: Update to handle CVE checks for meta-oe
Backport the recent CVE script changes from the master branch and add in
cvelayer.bbclass
as a way to exclude specific paths from the CVE check. Master can use the layer
overrides
to do this but these are not present on kirkstone.
Signed-off-by: Richard Purdie
---
config.json | 33 --
scripts/cve-generate-chartdata| 65 ++
scripts/cve-report.py | 30 +
scripts/cvecheck/cvelayer.bbclass | 9 +++
scripts/run-cvecheck | 106 ++
5 files changed, 212 insertions(+), 31 deletions(-)
create mode 100755 scripts/cve-generate-chartdata
create mode 100755 scripts/cve-report.py
create mode 100644 scripts/cvecheck/cvelayer.bbclass
diff --git a/config.json b/config.json
index 86ba2f4..7828462 100644
--- a/config.json
+++ b/config.json
@@ -910,7 +910,7 @@
}
},
"metrics" : {
-"NEEDREPOS" : ["poky"],
+"NEEDREPOS" : ["poky", "meta-openembedded"],
"extravars" : [
"INHERIT += 'cve-check'",
"BB_DISKMON_DIRS = ''",
@@ -920,12 +920,35 @@
"BB_SERVER_TIMEOUT = '0'"
],
"step1" : {
-"shortname" : "Generating patch metrics",
-"EXTRACMDS" :
["../../yocto-autobuilder-helper/scripts/run-patchmetrics ../ ../meta/
${HELPERRESULTSDIR}/../../patchmetrics . ${HELPERBRANCHNAME}"]
+"shortname" : "Fetching metrics repositories",
+"EXTRAPLAINCMDS" : [
+"git clone ssh://g...@push.yoctoproject.org/yocto-metrics
&& git clone ssh://g...@push.yoctoproject.org/yocto-metrics-meta-oe",
+"install -D ${SCRIPTSDIR}/cvecheck/cvelayer.bbclass
./build/classes/cvelayer.bbclass"
+]
},
"step2" : {
-"shortname" : "Running CVE checks",
-"EXTRACMDS" :
["../../yocto-autobuilder-helper/scripts/run-cvecheck ../ ../meta/
${HELPERRESULTSDIR}/../../patchmetrics . ${HELPERBRANCHNAME}"]
+"shortname" : "CVE checks for meta",
+"EXTRACMDS" : ["${SCRIPTSDIR}/run-cvecheck --metrics
../yocto-metrics --branch ${HELPERBRANCHNAME} --results
${HELPERRESULTSDIR}/../../patchmetrics --push"]
+},
+"step3" : {
+"shortname" : "CVE checks for meta-oe",
+"ADDLAYER" : [
+"${BUILDDIR}/../meta-openembedded/meta-oe",
+"${BUILDDIR}/../meta-openembedded/meta-python",
+"${BUILDDIR}/../meta-openembedded/meta-perl",
+"${BUILDDIR}/../meta-openembedded/meta-networking",
+"${BUILDDIR}/../meta-openembedded/meta-multimedia",
+"${BUILDDIR}/../meta-openembedded/meta-gnome",
+"${BUILDDIR}/../meta-openembedded/meta-xfce",
+"${BUILDDIR}/../meta-openembedded/meta-filesystems",
+"${BUILDDIR}/../meta-openembedded/meta-initramfs",
+"${BUILDDIR}/../meta-openembedded/meta-webserver"
+],
+"extravars" : [
+"INHERIT += 'cvelayer'",
+"SKIP_PATHS += '${TOPDIR}/../meta'"
+],
+"EXTRACMDS" : ["${SCRIPTSDIR}/run-cvecheck --metrics
../yocto-metrics-meta-oe --branch ${HELPERBRANCHNAME} --results
${HELPERRESULTSDIR}/../../patchmetrics-meta-oe --push"]
}
},
"meta-mingw" : {
diff --git a/scripts/cve-generate-chartdata b/scripts/cve-generate-chartdata
new file mode 100755
index 000..dbbbe82
--- /dev/null
+++ b/scripts/cve-generate-chartdata
@@ -0,0 +1,65 @@
+#!/usr/bin/env python3
+import json, os.path, collections
+import sys
+import argparse
+import subprocess
+import tempfile
+from datetime import datetime, date, timedelta
+
+args = argparse.ArgumentParser(description="Generate CVE count data files")
+args.add_argument("-j", "--json", help="JSON data file to use")
+args.add_argument("-r", "--resultsdir", help="results directory to parse")
+args = args.parse_args()
+
+try:
+with open(args.json) as f:
+counts = json.load(f)
+except FileNotFoundError:
+# if the file does not exist, start with an empty database.
+counts = {}
+
+lastyear = {}
+
+#
+# Write CVE counts by day
+#
+
[yocto] [yocto-autobuilder-helper] [PATCH] config.json: Fix upgrade helper conf path
Signed-off-by: Richard Purdie
---
config.json | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/config.json b/config.json
index 7505e59..ee13bcf 100644
--- a/config.json
+++ b/config.json
@@ -1467,7 +1467,7 @@
"git config user.email a...@yoctoproject.org",
"git config user.name 'Auto Upgrade Helper'",
"git switch -C tmp-auh-upgrades HEAD",
-"install -D ${SCRIPTSDIR}/auh-config/upgrade-helper.conf
./upgrade-helper/upgrade-helper.conf"
+"install -D ${SCRIPTSDIR}/auh-config/upgrade-helper.conf
./build/upgrade-helper/upgrade-helper.conf"
]
},
"step2" : {
@@ -1504,7 +1504,7 @@
"git -C ./meta-openembedded config user.email
a...@yoctoproject.org",
"git -C ./meta-openembedded config user.name 'Auto Upgrade
Helper'",
"git -C ./meta-openembedded switch -C tmp-auh-upgrades
HEAD",
-"install -D
${SCRIPTSDIR}/auh-config/upgrade-helper-meta-oe.conf
./upgrade-helper/upgrade-helper.conf"
+"install -D
${SCRIPTSDIR}/auh-config/upgrade-helper-meta-oe.conf
./build/upgrade-helper/upgrade-helper.conf"
]
},
"step2": {
--
2.39.2
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#61593): https://lists.yoctoproject.org/g/yocto/message/61593
Mute This Topic: https://lists.yoctoproject.org/mt/102361508/21656
Group Owner: yocto+ow...@lists.yoctoproject.org
Unsubscribe:
https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[yocto] [yocto-autobuilder-helper] [PATCH] auh-meta-oe: Send to the openembedded-devel mailing list, not core
To do that we need a separate config file.
Signed-off-by: Richard Purdie
---
config.json | 2 +-
.../auh-config/upgrade-helper-meta-oe.conf| 42 +++
2 files changed, 43 insertions(+), 1 deletion(-)
create mode 100644 scripts/auh-config/upgrade-helper-meta-oe.conf
diff --git a/config.json b/config.json
index dd45578..7505e59 100644
--- a/config.json
+++ b/config.json
@@ -1504,7 +1504,7 @@
"git -C ./meta-openembedded config user.email
a...@yoctoproject.org",
"git -C ./meta-openembedded config user.name 'Auto Upgrade
Helper'",
"git -C ./meta-openembedded switch -C tmp-auh-upgrades
HEAD",
-"install -D ${SCRIPTSDIR}/auh-config/upgrade-helper.conf
./upgrade-helper/upgrade-helper.conf"
+"install -D
${SCRIPTSDIR}/auh-config/upgrade-helper-meta-oe.conf
./upgrade-helper/upgrade-helper.conf"
]
},
"step2": {
diff --git a/scripts/auh-config/upgrade-helper-meta-oe.conf
b/scripts/auh-config/upgrade-helper-meta-oe.conf
new file mode 100644
index 000..14c6957
--- /dev/null
+++ b/scripts/auh-config/upgrade-helper-meta-oe.conf
@@ -0,0 +1,42 @@
+[maintainer_override]
+# mails for recipe upgrades will go to john.doe instead of jane.doe, etc
+#ross.bur...@intel.com=anibal.li...@linux.intel.com
+
+[settings]
+# recipes in blacklist will be skipped
+blacklist=linux-libc-headers linux-yocto alsa-utils-scripts
build-appliance-image
+
+#maintainers_whitelist=anibal.li...@linux.intel.com
+
+# SMTP server
+smtp=localhost:25
+
+# from whom should the mails arrive
+from=a...@yoctoproject.org
+
+# who should get the status mail with statistics, at the end
+status_recipients=openembedded-de...@lists.openembedded.org
+
+# who should be CCd with upgrade emails
+cc_recipients=openembedded-de...@lists.openembedded.org
+
+# clean sstate directory before upgrading
+#clean_sstate=yes
+
+# clean tmp directory before upgrading
+#clean_tmp=yes
+
+# machines to test build with
+#machines=qemux86 qemux86-64 qemuarm qemumips qemuppc
+#machines=qemux86
+
+buildhistory=yes
+
+#testimage=yes
+#testimage_name=core-image-minimal
+
+#workdir=/home/auh/work/
+publish_work_url=https://autobuilder.yocto.io/pub/auh
+
+commit_revert_policy=all
+
--
2.39.2
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#61592): https://lists.yoctoproject.org/g/yocto/message/61592
Mute This Topic: https://lists.yoctoproject.org/mt/102361360/21656
Group Owner: yocto+ow...@lists.yoctoproject.org
Unsubscribe:
https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[yocto] [yocto-autobuilder-helper] [PATCH] config.json: Add auto-upgrade-helper repo definition
Signed-off-by: Richard Purdie
---
config.json | 6 ++
1 file changed, 6 insertions(+)
diff --git a/config.json b/config.json
index 1b3c2b0..b2d6fb8 100644
--- a/config.json
+++ b/config.json
@@ -1631,6 +1631,12 @@
"branch" : "master",
"revision" : "HEAD",
"no-layer-add" : true
+},
+"auto-upgrade-helper" : {
+"url" : "git://git.yoctoproject.org/auto-upgrade-helper",
+"branch" : "master",
+"revision" : "HEAD",
+"no-layer-add" : true
}
},
"buildtools" : {
--
2.39.2
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#61591): https://lists.yoctoproject.org/g/yocto/message/61591
Mute This Topic: https://lists.yoctoproject.org/mt/102361268/21656
Group Owner: yocto+ow...@lists.yoctoproject.org
Unsubscribe:
https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] [yocto-autobuilder-helper][PATCH v4 1/2] config.json: Move AUH from custom scripts to config.json
On Fri, 2023-11-03 at 10:44 +0100, Alexander Kanavin wrote: > On Fri, 3 Nov 2023 at 10:37, Alexander Kanavin via > lists.yoctoproject.org > wrote: > > > > Ka-boom: > > https://autobuilder.yoctoproject.org/typhoon/#/builders/130/builds/59 > > " > ERROR: Command . ./oe-init-build-env; bitbake-layers add-layer > /home/pokybuild/yocto-worker/auh/build/auto-upgrade-helper failed with > exit code 1, see errors above. (1699003823.8) > " > > I'm not exactly sure, but I think this should be addressed by adding > an entry in config.json with "no-layer-add" : true (there are examples > there). My changes have a piece at the end of config.json which adds the repo and fix that issue. I'll separate out that change and add it that way. Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#61590): https://lists.yoctoproject.org/g/yocto/message/61590 Mute This Topic: https://lists.yoctoproject.org/mt/102344544/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] [auto-upgrade-helper] [PATCH 2/2] weeklyjob.sh: Drop
We run AUH from the autobuilder and we don't need the cron script any more, we'd point people at the scripts there instead. Signed-off-by: Richard Purdie --- weeklyjob.sh | 30 -- 1 file changed, 30 deletions(-) delete mode 100755 weeklyjob.sh diff --git a/weeklyjob.sh b/weeklyjob.sh deleted file mode 100755 index 94598a2..000 --- a/weeklyjob.sh +++ /dev/null @@ -1,30 +0,0 @@ -#!/bin/bash - -# Cronjob which can be run weekly to run the upgrade helper script. -# Add the job in /etc/crontab like below. -# -# It will execute weekly at the same hour (8 AM). -# -# 00 8 * * 6 auh /home/auh/bin/weeklyjob.sh - -# Re-assign these to match your setup! -auh_dir=~/auto-upgrade-helper -poky_dir=~/poky -build_dir=~/build-tmp-auh-upgrades -sstate_dir=~/sstate-cache - -pushd $poky_dir - -# Base the upgrades on poky master -git fetch origin -git checkout -B tmp-auh-upgrades origin/master - -source $poky_dir/oe-init-build-env $build_dir -$auh_dir/upgrade-helper.py -e all - -# clean up to avoid the disk filling up -rm -rf $build_dir/tmp/ -rm -rf $build_dir/workspace/sources/* -find $sstate_dir -atime +10 -delete - -popd -- 2.39.2 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#61554): https://lists.yoctoproject.org/g/yocto/message/61554 Mute This Topic: https://lists.yoctoproject.org/mt/102316509/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] [auto-upgrade-helper] [PATCH 1/2] upgrade-helper: Drop layer_machines
We don't need the ability to change this per layer, drop the option
and simplify the code.
Signed-off-by: Richard Purdie
---
upgrade-helper.py | 9 ++---
1 file changed, 2 insertions(+), 7 deletions(-)
diff --git a/upgrade-helper.py b/upgrade-helper.py
index 3be6d68..869dc3f 100755
--- a/upgrade-helper.py
+++ b/upgrade-helper.py
@@ -109,8 +109,6 @@ def parse_cmdline():
help="layers to include in the upgrade research")
parser.add_argument("--layer-dir", action="store", default='',
help="the layers root directory")
-parser.add_argument("--layer-machines", nargs='*', action="store",
default='',
-help="machine to build for the layers")
return parser.parse_args()
def parse_config_file(config_file):
@@ -180,7 +178,7 @@ class Updater(object):
E(" In layer mode enable you need to specify %s.\n" % setting)
exit(1)
-layer_settings = ('layer_name', 'layer_dir', 'layer_machines')
+layer_settings = ('layer_name', 'layer_dir')
for s in layer_settings:
self.opts[s] = settings.get(s, '')
if not self.opts[s]:
@@ -188,13 +186,11 @@ class Updater(object):
self.git = Git(self.opts['layer_dir'])
self.poky_git = Git(os.path.dirname(os.getenv('PATH',
False).split(':')[0]))
-self.opts['machines'] = self.opts['layer_machines'].split()
else:
# XXX: assume that the poky directory is the first entry in the
PATH
self.git = Git(os.path.dirname(os.getenv('PATH',
False).split(':')[0]))
self.poky_git = None
-self.opts['machines'] = settings.get('machines',
-'qemux86-64 qemuarm_musl').split()
+self.opts['machines'] = settings.get('machines', 'qemux86-64
qemuarm_musl').split()
self.opts['send_email'] = self.args.send_emails
self.opts['author'] = "Upgrade Helper <%s>" % \
@@ -773,7 +769,6 @@ if __name__ == "__main__":
settings['layer_mode'] = 'yes'
settings['layer_dir'] = args.layer_dir
settings['layer_name'] = ' '.join(args.layer_names)
-settings['layer_machines'] = ' '.join(args.layer_machines)
updater = UniverseUpdater(args)
updater.run()
--
2.39.2
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#61553): https://lists.yoctoproject.org/g/yocto/message/61553
Mute This Topic: https://lists.yoctoproject.org/mt/102316508/21656
Group Owner: yocto+ow...@lists.yoctoproject.org
Unsubscribe:
https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] [OE-core] Core workflow: sstate for all, bblock/bbunlock, tools for why is sstate not being reused?
On Tue, 2023-10-31 at 13:08 +0100, Alexander Kanavin wrote: > On Mon, 30 Oct 2023 at 16:02, Alexander Kanavin via > lists.openembedded.org > wrote: > > So here's what could be done: > > > > - esdk tools become symlinks in poky/scripts/esdk-tools/. esdk > > environment script puts that in PATH, rather than some custom > > esdk-specific location (the code to generate that can then be > > dropped). > > This is now implemented (needs to be tested on AB). > > > - esdk tweaks to local.conf move into a dedicated include file, which > > can be static and under version control, except for perhaps > > METADATA_REVISION:poky = "4a1e0b9625729e422fcf24e632ee2a3c79f986d5" - > > I need to check why is it there and how that is used. > > It's actually more complicated. The code to generate esdk-specific > local.conf with all the tweaks has too much dynamic stuff in it which > is subject to what various variables are set to. So I'm thinking of > extracting that to a dedicated function, then attaching a bitbake task > to that function. > > Then we can pull all of it together into 'devtool esdk ' > command (or similar), which would enter the esdk environment directly > via: > - running 'bitbake meta-ide-support' > - running the above mentioned bitbake local.conf task to generate the > esdk-specific local.conf > - sourcing the environment script produced by meta-ide-support > - rewriting PATH to provide only the curated esdk tools and not > everything plus bitbake. > - writing a custom devtool.conf similar to that of standalone esdk so > that devtool can find bitbake and bitbake can use the esdk-specific > local.conf > > And it would be tested in the same way standalone esdks are. > > Thoughts? Anything missing from the above list? That sounds like a good way to handle this to me! Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#61525): https://lists.yoctoproject.org/g/yocto/message/61525 Mute This Topic: https://lists.yoctoproject.org/mt/102295494/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] Core workflow: sstate for all, bblock/bbunlock, tools for why is sstate not being reused?
On Mon, 2023-10-30 at 14:50 +0100, Alexander Kanavin wrote: > On Thu, 14 Sept 2023 at 14:56, Richard Purdie > wrote: > > There are design elements to this work. We need to work out how we can > > make eSDK and "normal" builds more similar and less of an overhead to > > switch between one and the other. A "bblock all" command does partly > > get you to an eSDK effectively so part of this may be switching eSDK to > > use the new lock command. What other differences are there? What other > > differences are necessary or make sense for the use cases eSDK was > > designed for? How would you turn an existing build into an eSDK like > > one? Could you provide a copy of a local build to someone else easily > > using something like eSDK's tooling? What does the eSDK look like at > > the end of this. One section we don't have good answers to yet is setup > > and configuration although I know you've started on some of that. > > So I see the following differences between esdk and normal modes: > > 1. Environment and tooling availability. > > a) esdk sets a number of variables from its initialization script that > aid with cross-compiling components directly (e.g. the core use case > of SDKs). Normal mode doesn't do that, but recently added > meta-ide-support will generate a similar initialization script that > will set up the same environment from the normal mode. There are tests > and documentation for it. In that case, this one is something we can document as how to make the functionality available in the normal build. > b) PATH. eSDK has a number of items in PATH that point to various > locations inside tmp/sysroots/, collectively they provide the > cross-toolchain. > > eSDK also puts a selection of yocto tools into path - wic, devtool but > not bitbake: > > > alex@Zen2:~/poky_sdk$ ls -l sysroots/x86_64-pokysdk-linux/usr/bin/ > total 48 > lrwxrwxrwx 1 alex alex 39 Oct 30 12:52 devtool -> > ../../../../layers/poky/scripts/devtool > lrwxrwxrwx 1 alex alex 54 Oct 30 12:52 oe-find-native-sysroot -> > ../../../../layers/poky/scripts/oe-find-native-sysroot > lrwxrwxrwx 1 alex alex 42 Oct 30 12:52 recipetool -> > ../../../../layers/poky/scripts/recipetool > lrwxrwxrwx 1 alex alex 39 Oct 30 12:52 runqemu -> > ../../../../layers/poky/scripts/runqemu > lrwxrwxrwx 1 alex alex 55 Oct 30 12:52 runqemu-addptable2image -> > ../../../../layers/poky/scripts/runqemu-addptable2image > lrwxrwxrwx 1 alex alex 53 Oct 30 12:52 runqemu-export-rootfs -> > ../../../../layers/poky/scripts/runqemu-export-rootfs > lrwxrwxrwx 1 alex alex 51 Oct 30 12:52 runqemu-extract-sdk -> > ../../../../layers/poky/scripts/runqemu-extract-sdk > lrwxrwxrwx 1 alex alex 51 Oct 30 12:52 runqemu-gen-tapdevs -> > ../../../../layers/poky/scripts/runqemu-gen-tapdevs > lrwxrwxrwx 1 alex alex 46 Oct 30 12:52 runqemu-ifdown -> > ../../../../layers/poky/scripts/runqemu-ifdown > lrwxrwxrwx 1 alex alex 44 Oct 30 12:52 runqemu-ifup -> > ../../../../layers/poky/scripts/runqemu-ifup > lrwxrwxrwx 1 alex alex 100 Oct 30 12:52 unfsd -> > ../../../../tmp/work/qemuarm64-poky-linux/core-image-minimal/1.0/recipe-sysroot-native/usr/bin/unfsd > lrwxrwxrwx 1 alex alex 35 Oct 30 12:52 wic -> > ../../../../layers/poky/scripts/wic > == > > 'normal mode' puts bitbake/bin/ and oe-core/scripts in PATH. > Cross-toolchain can be added by the same environment script made by > meta-ide-support as mentioned in 1a. Right, so in theory we can change PATH and change this which can also easily be documented. > 2. Configuration (e.g. local.conf). > > eSDK local.conf is local.conf from the normal mode that was used to > produce eSDK, stripped of all comments, and with a bunch of extra > settings: > > > INHERIT:remove = "buildhistory icecc" > CONNECTIVITY_CHECK_URIS = "" > > SIGGEN_LOCKEDSIGS_SSTATE_EXISTS_CHECK = "none" > > SIGGEN_LOCKEDSIGS_TASKSIG_CHECK = "warn" > > BB_HASHCONFIG_IGNORE_VARS:append = " SIGGEN_UNLOCKED_RECIPES" > > BB_SETSCENE_ENFORCE_IGNORE_TASKS = "%:* *:do_shared_workdir > *:do_rm_work wic-tools:* *:do_addto_recipe_sysroot" > > BUILDCFG_HEADER = "" > > METADATA_REVISION:poky = "4a1e0b9625729e422fcf24e632ee2a3c79f986d5" > > # Provide a flag to indicate we are in the EXT_SDK Context > WITHIN_EXT_SDK = "1" > > SSTATE_MIRRORS += " file://universal/(.*) file://universal-4.9/\1 > file://universal-4.9/(.*) file://universal-4.8/\1" > Perhaps some of this should become a generic include file which is included w
Re: [yocto] [yocto-autobuilder-helper] [PATCH] config.json: Fix check-layer for meta-poky/meta-yocto-bsp
On Sat, 2023-10-28 at 11:10 -0700, Tim Orling wrote: > Steve, > > This needs to be applied to dunfell, kirkstone, mickledore and > nanbield (none of those are performing the correct check now). I've gone ahead and backported this since I've been working on this repo a lot recently. If there are any issues we'll resolve them... Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#61512): https://lists.yoctoproject.org/g/yocto/message/61512 Mute This Topic: https://lists.yoctoproject.org/mt/102097822/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] [qa-build-notification] QA notification for completed autobuilder build (yocto-3.1.29.rc1)
On Fri, 2023-10-27 at 12:38 -1000, Steve Sakoman wrote: > There is currently a known reproducibility issue with the vim-common > package. In my opinion nothing that should hold up the release. > > Details: > > It seems that the last sstate build of vim was done on an Ubuntu 22.04 > worker and recent changes there seem to result in an occurance of > "charset=cp1251" whereas builds done with the buildtools tarball yield > "charset=CP1251" > > See: > https://autobuilder.yocto.io/pub/repro-fail/oe-reproducible-20231024-kys9f0xn/packages/diff-html/ > > The reproducibility build for 3.1.29 was done on alma8, so this > difference resulted in a build error. > > I reran the reproducibility test on an Ubuntu 22.04 worker and as > expected it passed: > > https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/3827 > > I will be sending an autobuilder helper patch to use buildtools > tarball with Ubuntu 22.04 and this should prevent future occurrences > of this issue (once the version in sstate is no longer valid) So we understand why master doesn't see this? I'm still a bit worried about why this difference is occurring. Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#61508): https://lists.yoctoproject.org/g/yocto/message/61508 Mute This Topic: https://lists.yoctoproject.org/mt/102231778/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] [yocto-autobuilder-helper][PATCH v3] AUH: Add Openembedded auto-update-helper with list of layer to test
Hi David,
On Thu, 2023-10-26 at 15:03 +0200, David Pierret wrote:
> - Modify the setup-auh to use repositories cloned by the auto-builder.
> Manage the case where meta-openembedded is not used.
>
> - Modify the run-auh to manage the optional additional layer.
> Only 1 layer can be provided.
>
> - Add auh-meta-openembedded job with 1 step per layer.
>
> Signed-off-by: David Pierret
> ---
> config.json | 102 --
> scripts/run-auh | 39 --
> scripts/setup-auh | 34 ++--
> 3 files changed, 146 insertions(+), 29 deletions(-)
This took a while to review. Firstly, this really should be written as
a set of commits with incremental changes.
For example, changing the script to use poky from the autobuilder
checkout instead of the local clone is one logical change. We could
take and review/merge that independently of the other changes.
Another logical independent change is using auto-upgrade-helper from
the checkout of the autobuilder. That did highlight an issue that the
autobuilder does not provide a checkout of that repository.
With that and this work in mind I added auh-meta-oe here:
https://git.yoctoproject.org/yocto-autobuilder2/commit/?id=47c0119b6e87f920566d2176793ef8d982cf31e2
and then added the auto-upgrade-helper repo here:
https://git.yoctoproject.org/yocto-autobuilder2/commit/?id=854a60ce8bf8ce0feec370bc6782cc12aa1e29a5
which then means this patch is closer to working.
I had to make some tweaks on top of the patch to make it work:
https://git.yoctoproject.org/yocto-autobuilder-helper/commit/?h=master-next=673745f473024020ed9efece26462e8c884dc819
where I found that:
* poky in the AB context is $1/.. since it is the top level
container of the layers
* meta-openembedded is located at ${HELPERBUILDDIR}/../meta-openembedded
* we still need to source the build env script. The AB could be
configured to do that for us but currently isn't.
* we need to trim the machine_list down as mentioned
* the naming we're using is auh-meta-oe, not auh-meta-openembedded
* I've temporarily configured to use the test-list so we don't
send lots of mail to all users
* layer dependencies were missing in ADDLAYER. I hate having to do
that but it is is what it is right now.
With that, we get this "successful" build:
https://autobuilder.yoctoproject.org/typhoon/#/builders/159/builds/5
which shows the exit code handling isn't great but probably deliberate
to avoid autobuilder errors.
This gets us to the point where the code in the upgrade helper itself
needs the layer support changing.
Since I've now had to look into that code, it seems the layer options
are entirely missing from the commandline options parser. I'd strongly
suggest dropping layer_machines variable and just use the machines
value in all cases.
"layer-mode" seems pointless as you can determine if we're in "layer
mode" if a layer is specified on the commandline (which needs adding).
I'm hoping you could use the layer overrides to be able to identify
which recipes to upgrade.
Cheers,
Richard
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#61507): https://lists.yoctoproject.org/g/yocto/message/61507
Mute This Topic: https://lists.yoctoproject.org/mt/102199062/21656
Group Owner: yocto+ow...@lists.yoctoproject.org
Unsubscribe:
https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] [PATCH yocto-autobuilder-helper] config.json: add a standalone builder for running yocto mirror tests
On Fri, 2023-10-27 at 09:21 +0200, Alexander Kanavin wrote:
> Those tests are currently run as the last step in a-full/a-quick,
> but it's useful to also trigger them separately, after perhaps
> executing other builders that populate the mirrors (such as
> qemux86-64 and such).
>
> Signed-off-by: Alexander Kanavin
> ---
> config.json | 8
> 1 file changed, 8 insertions(+)
>
> diff --git a/config.json b/config.json
> index 9987514..918cf08 100644
> --- a/config.json
> +++ b/config.json
> @@ -1433,6 +1433,14 @@
> "${SCRIPTSDIR}/setup-auh ${HELPERBUILDDIR};
> ${SCRIPTSDIR}/run-auh ${HELPERBUILDDIR} ${WEBPUBLISH_DIR}/pub/auh/"
> ]
> },
> +"yocto-mirror" : {
> +"MACHINE" : "qemux86-64",
> +"step1" : {
> +"shortname" : "Yocto Mirror Selftest",
> +"EXTRACMDS" : ["${SCRIPTSDIR}/checkvnc; DISPLAY=:1
> oe-selftest -a -t yocto-mirrors -j 15"],
> +"ADDLAYER" : ["${BUILDDIR}/../meta-selftest"]
> +}
> +},
> "meta-oe-mirror" : {
> "SDKMACHINE" : "x86_64",
> "MACHINE" : "qemux86-64",
This and the controller patch should be live.
Cheers,
Richard
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#61500): https://lists.yoctoproject.org/g/yocto/message/61500
Mute This Topic: https://lists.yoctoproject.org/mt/102216408/21656
Group Owner: yocto+ow...@lists.yoctoproject.org
Unsubscribe:
https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] [yocto-autobuilder-helper][PATCH v3] AUH: Add Openembedded auto-update-helper with list of layer to test
On Thu, 2023-10-26 at 15:11 +0200, David Pierret wrote:
> Hi,
>
> I've made separate steps for setup and run
> I didn't yet modify the python script in upgrade-helper to include
> Kevin reviews.
>
> follow some questions I have about the autobuilder.
>
> On Thu, Oct 26, 2023 at 3:04 PM David Pierret via
> lists.yoctoproject.org
> wrote:
> >
> > - Modify the setup-auh to use repositories cloned by the auto-builder.
> > Manage the case where meta-openembedded is not used.
> >
> > - Modify the run-auh to manage the optional additional layer.
> > Only 1 layer can be provided.
> >
> > - Add auh-meta-openembedded job with 1 step per layer.
> >
> > Signed-off-by: David Pierret
> > ---
> > config.json | 102 --
> > scripts/run-auh | 39 --
> > scripts/setup-auh | 34 ++--
> > 3 files changed, 146 insertions(+), 29 deletions(-)
> >
> > diff --git a/config.json b/config.json
> > index 3acb710..a168e48 100644
> > --- a/config.json
> > +++ b/config.json
> > @@ -1416,9 +1416,99 @@
> > "TEMPLATE" : "buildperf"
> > },
> > "auh" : {
> > -"EXTRAPLAINCMDS" : [
> > -"${SCRIPTSDIR}/setup-auh ${HELPERBUILDDIR};
> > ${SCRIPTSDIR}/run-auh ${HELPERBUILDDIR} ${WEBPUBLISH_DIR}/pub/auh/"
> > -]
> > +"NEEDREPOS" : ["poky", "auto-upgrade-helper"],
> > +"step1" : {
> > +"shortname" : "setup AUH",
> > +"EXTRAPLAINCMDS" : [
> > +"${SCRIPTSDIR}/setup-auh ${HELPERBUILDDIR}"
> > +]
> > +},
> > +"step2" : {
> > +"shortname" : "run AUH poky",
> > +"EXTRAPLAINCMDS" : [
> > +"${SCRIPTSDIR}/run-auh ${HELPERBUILDDIR}
> > ${WEBPUBLISH_DIR}/pub/auh/"
> > +]
> > +}
> > +},
> > +"auh-meta-openembedded" : {
> > +"NEEDREPOS" : ["poky", "meta-openembedded",
> > "auto-upgrade-helper"],
> > +"step1" : {
> > +"shortname" : "setup AUH",
> > +"EXTRAPLAINCMDS" : [
> > +"${SCRIPTSDIR}/setup-auh ${HELPERBUILDDIR}"
> > +]
> > +},
> > +"step2" : {
> > +"shortname" : "run AUH for layer meta-oe",
> > +
> > +"ADDLAYER" : ["${BUILDDIR}/../meta-openembedded/meta-oe"],
> > +"EXTRAPLAINCMDS" : [
> > +"${SCRIPTSDIR}/run-auh ${HELPERBUILDDIR}
> > ${WEBPUBLISH_DIR}/pub/auh/ ${HELPERBUILDDIR}/meta-openembedded meta-oe"
> > +]
> > +},
> > +"step3" : {
> > +"shortname" : "run AUH for layer meta-python",
> > +"ADDLAYER" :
> > ["${BUILDDIR}/../meta-openembedded/meta-python"],
> > +"EXTRAPLAINCMDS" : [
> > +"${SCRIPTSDIR}/run-auh ${HELPERBUILDDIR}
> > ${WEBPUBLISH_DIR}/pub/auh/ ${HELPERBUILDDIR}/meta-openembedded meta-python"
> > +]
> > +},
> > +"step4" : {
> > +"shortname" : "run AUH for layer meta-perl",
> > +"ADDLAYER" :
> > ["${BUILDDIR}/../meta-openembedded/meta-perl"],
> > +"EXTRAPLAINCMDS" : [
> > +"${SCRIPTSDIR}/run-auh ${HELPERBUILDDIR}
> > ${WEBPUBLISH_DIR}/pub/auh/ ${HELPERBUILDDIR}/meta-openembedded meta-perl"
> > +]
> > +},
> > +"step5" : {
> > +"shortname" : "run AUH for layer meta-networking",
> > +"ADDLAYER" :
> > ["${BUILDDIR}/../meta-openembedded/meta-networking"],
> > +"EXTRAPLAINCMDS" : [
> > +"${SCRIPTSDIR}/run-auh ${HELPERBUILDDIR}
> > ${WEBPUBLISH_DIR}/pub/auh/ ${HELPERBUILDDIR}/meta-openembedded
> > meta-networking"
> > +]
> > +},
> > +"step6" : {
> > +"shortname" : "run AUH for layer meta-multimedia",
> > +"ADDLAYER" :
> > ["${BUILDDIR}/../meta-openembedded/meta-multimedia"],
> > +"EXTRAPLAINCMDS" : [
> > +"${SCRIPTSDIR}/run-auh ${HELPERBUILDDIR}
> > ${WEBPUBLISH_DIR}/pub/auh/ ${HELPERBUILDDIR}/meta-openembedded
> > meta-multimedia"
> > +]
> > +},
> > +"step7" : {
> > +"shortname" : "run AUH for layer meta-gnome",
> > +"ADDLAYER" :
> > ["${BUILDDIR}/../meta-openembedded/meta-gnome"],
> > +"EXTRAPLAINCMDS" : [
> > +"${SCRIPTSDIR}/run-auh ${HELPERBUILDDIR}
> > ${WEBPUBLISH_DIR}/pub/auh/ ${HELPERBUILDDIR}/meta-openembedded meta-gnome"
> > +]
> > +},
> > +"step8" : {
> > +"shortname" : "run AUH for layer meta-xfce",
> > +"ADDLAYER" :
> > ["${BUILDDIR}/../meta-openembedded/meta-xfce"],
>
Re: [yocto] [yocto-autobuilder-helper][PATCH v3] AUH: Add Openembedded auto-update-helper with list of layer to test
On Thu, 2023-10-26 at 15:53 +0200, Alexander Kanavin wrote:
> This looks basically ok, but how can this be tested if needed changes
> to AUH are still in review?
>
> Running auh over meta-oe layers shouldn't be defined as steps that are
> run one after the other, but as separate parallelizable jobs, e.g.
> "auh-meta-oe" : {
> ...
> "auh-meta-python" : {
> ...
> etc. They could use a job template so you only need to set the varying
> parameters (config.json has examples).
In general you're right, but for AUH and meta-oe, I'm ok with this
being done serially given what AUH does. We don't really want 11
workers working on this at the same time?
Cheers,
Richard
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#61494): https://lists.yoctoproject.org/g/yocto/message/61494
Mute This Topic: https://lists.yoctoproject.org/mt/102199062/21656
Group Owner: yocto+ow...@lists.yoctoproject.org
Unsubscribe:
https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] [yocto-autobuilder-helper] [PATCH] config.json: Make meta-oe source mirror config wider coverage
On Tue, 2023-10-24 at 10:15 -0700, Khem Raj wrote:
> On 10/24/23 4:03 AM, Richard Purdie wrote:
> > Some recipes depend on DISTRO_FEATURES, commercial licensing or compiler
> > config like fortran. Set these things so that we get wider soruce mirror
> > coverage and fewer warnings.
> >
> > 'commercial' license usage is ok here since we're not building and then
> > shipping any binaries or using it, only mirroring the source code.
> >
> > Signed-off-by: Richard Purdie
> > ---
> > config.json | 6 ++
> > 1 file changed, 6 insertions(+)
> >
> > diff --git a/config.json b/config.json
> > index 0c35632..61cf374 100644
> > --- a/config.json
> > +++ b/config.json
> > @@ -1451,6 +1451,12 @@
> > "${BUILDDIR}/../meta-openembedded/meta-initramfs",
> > "${BUILDDIR}/../meta-openembedded/meta-webserver"
> > ],
> > +"extravars" : [
> > +"LICENSE_FLAGS_ACCEPTED = 'commercial'",
> > +"DISTRO_FEATURES:append = ' pam systemd usrmerge'",
> > +"FORTRAN:forcevariable = ',fortran'",
> > +"RUNTIMETARGET:append:pn-gcc-runtime = ' libquadmath'"
>
> I think RUNTIMETARGET should automatically be appended with libquadmath
> if FORTRAN is set.
Perhaps but that is a change to work on in OE-Core and isn't going to
happen overnight or in previous releases.
Improvements welcome there or perhaps a bug to track it at least.
Cheers,
Richard
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#61472): https://lists.yoctoproject.org/g/yocto/message/61472
Mute This Topic: https://lists.yoctoproject.org/mt/102155468/21656
Group Owner: yocto+ow...@lists.yoctoproject.org
Unsubscribe:
https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[yocto] [meta-mingw] [PATCH] SECURITY.md: add file
Add a SECURITY.md file with hints for security researchers and other parties who might report potential security vulnerabilities. Signed-off-by: Richard Purdie --- SECURITY.md | 24 1 file changed, 24 insertions(+) create mode 100644 SECURITY.md diff --git a/SECURITY.md b/SECURITY.md new file mode 100644 index 000..7d2ce1f --- /dev/null +++ b/SECURITY.md @@ -0,0 +1,24 @@ +How to Report a Potential Vulnerability? + + +If you would like to report a public issue (for example, one with a released +CVE number), please report it using the +[https://bugzilla.yoctoproject.org/enter_bug.cgi?product=Security Security Bugzilla]. +If you have a patch ready, submit it following the same procedure as any other +patch as described in README.md. + +If you are dealing with a not-yet released or urgent issue, please send a +message to security AT yoctoproject DOT org, including as many details as +possible: the layer or software module affected, the recipe and its version, +and any example code, if available. + +Branches maintained with security fixes +--- + +See [https://wiki.yoctoproject.org/wiki/Stable_Release_and_LTS Stable release and LTS] +for detailed info regarding the policies and maintenance of Stable branches. + +The [https://wiki.yoctoproject.org/wiki/Releases Release page] contains a list of all +releases of the Yocto Project. Versions in grey are no longer actively maintained with +security patches, but well-tested patches may still be accepted for them for +significant issues. -- 2.39.2 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#61467): https://lists.yoctoproject.org/g/yocto/message/61467 Mute This Topic: https://lists.yoctoproject.org/mt/102156138/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] [yocto-autobuilder2] [PATCH] SECURITY.md: Add file
Add a SECURITY.md file with hints for security researchers and other parties who might report potential security vulnerabilities. Signed-off-by: Richard Purdie --- SECURITY.md | 13 + 1 file changed, 13 insertions(+) create mode 100644 SECURITY.md diff --git a/SECURITY.md b/SECURITY.md new file mode 100644 index ..7ccecc1f --- /dev/null +++ b/SECURITY.md @@ -0,0 +1,13 @@ +How to Report a Potential Vulnerability? + + +If you would like to report a public issue (for example, one with a released +CVE number), please report it using the +[https://bugzilla.yoctoproject.org/enter_bug.cgi?product=Security Security Bugzilla]. +If you have a patch ready, submit it following the same procedure as any other +patch as described in README.md. + +If you are dealing with a not-yet released or urgent issue, please send a +message to security AT yoctoproject DOT org, including as many details as +possible: the layer or software module affected, the recipe and its version, +and any example code, if available. -- 2.39.2 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#61466): https://lists.yoctoproject.org/g/yocto/message/61466 Mute This Topic: https://lists.yoctoproject.org/mt/102156136/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] [yocto-autobuilder-helper] [PATCH] SECURITY.md: Add file
Add a SECURITY.md file with hints for security researchers and other parties who might report potential security vulnerabilities. Signed-off-by: Richard Purdie --- SECURITY.md | 24 1 file changed, 24 insertions(+) create mode 100644 SECURITY.md diff --git a/SECURITY.md b/SECURITY.md new file mode 100644 index 000..7d2ce1f --- /dev/null +++ b/SECURITY.md @@ -0,0 +1,24 @@ +How to Report a Potential Vulnerability? + + +If you would like to report a public issue (for example, one with a released +CVE number), please report it using the +[https://bugzilla.yoctoproject.org/enter_bug.cgi?product=Security Security Bugzilla]. +If you have a patch ready, submit it following the same procedure as any other +patch as described in README.md. + +If you are dealing with a not-yet released or urgent issue, please send a +message to security AT yoctoproject DOT org, including as many details as +possible: the layer or software module affected, the recipe and its version, +and any example code, if available. + +Branches maintained with security fixes +--- + +See [https://wiki.yoctoproject.org/wiki/Stable_Release_and_LTS Stable release and LTS] +for detailed info regarding the policies and maintenance of Stable branches. + +The [https://wiki.yoctoproject.org/wiki/Releases Release page] contains a list of all +releases of the Yocto Project. Versions in grey are no longer actively maintained with +security patches, but well-tested patches may still be accepted for them for +significant issues. -- 2.39.2 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#61465): https://lists.yoctoproject.org/g/yocto/message/61465 Mute This Topic: https://lists.yoctoproject.org/mt/102156134/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] [yocto-autobuilder-helper] [PATCH] config.json: Make meta-oe source mirror config wider coverage
Some recipes depend on DISTRO_FEATURES, commercial licensing or compiler
config like fortran. Set these things so that we get wider soruce mirror
coverage and fewer warnings.
'commercial' license usage is ok here since we're not building and then
shipping any binaries or using it, only mirroring the source code.
Signed-off-by: Richard Purdie
---
config.json | 6 ++
1 file changed, 6 insertions(+)
diff --git a/config.json b/config.json
index 0c35632..61cf374 100644
--- a/config.json
+++ b/config.json
@@ -1451,6 +1451,12 @@
"${BUILDDIR}/../meta-openembedded/meta-initramfs",
"${BUILDDIR}/../meta-openembedded/meta-webserver"
],
+"extravars" : [
+"LICENSE_FLAGS_ACCEPTED = 'commercial'",
+"DISTRO_FEATURES:append = ' pam systemd usrmerge'",
+"FORTRAN:forcevariable = ',fortran'",
+"RUNTIMETARGET:append:pn-gcc-runtime = ' libquadmath'"
+],
"step1" : {
"shortname" : "Sources pre-fetching",
"BBTARGETS" : "universe -c fetch -k",
--
2.39.2
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#61464): https://lists.yoctoproject.org/g/yocto/message/61464
Mute This Topic: https://lists.yoctoproject.org/mt/102155468/21656
Group Owner: yocto+ow...@lists.yoctoproject.org
Unsubscribe:
https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] [yocto-autobuilder-helper][PATCH v2] AUH: Add Openembedded auto-update-helper with list of layer to test
Hi David,
Comments below. I did try and sync up with Alex Kanavin so we have a
consistent direction.
On Tue, 2023-10-24 at 10:16 +0200, David PIERRET wrote:
> On Fri, Oct 20, 2023 at 4:57 PM Richard Purdie
> wrote:
> >
> > Hi David,
> >
> > Thanks for revising this, it is heading in the right direction.
> >
> > On Fri, 2023-10-20 at 16:05 +0200, David Pierret wrote:
> > > - Add setup and run script openembedded specific
> > > - Add job config
> > >
> > > Signed-off-by: David Pierret
> > > ---
> > > config.json | 6 ++
> > > scripts/run-auh-oe | 45
> > > scripts/setup-auh-oe | 34 +
> > > 3 files changed, 85 insertions(+)
> > > create mode 100755 scripts/run-auh-oe
> > > create mode 100755 scripts/setup-auh-oe
> > >
> > > diff --git a/config.json b/config.json
> > > index 3acb710..bbd0aaf 100644
> > > --- a/config.json
> > > +++ b/config.json
> > > @@ -1420,6 +1420,12 @@
> > > "${SCRIPTSDIR}/setup-auh ${HELPERBUILDDIR};
> > > ${SCRIPTSDIR}/run-auh ${HELPERBUILDDIR} ${WEBPUBLISH_DIR}/pub/auh/"
> > > ]
> > > },
> > > +"auh-openembedded" : {
> >
> > This needs to be "auh-meta-oe" since "auh" is for openembedded-core and
> > this is otherwise going to be confusing.
> >
> > > +"NEEDREPOS" : ["poky", "meta-openembedded"],
> > > +"EXTRAPLAINCMDS" : [
> > > +"${SCRIPTSDIR}/setup-auh-oe ${HELPERBUILDDIR};
> > > ${SCRIPTSDIR}/run-auh-oe ${HELPERBUILDDIR} ${WEBPUBLISH_DIR}/pub/auh/
> > > ${HELPERBUILDDIR}/meta-openembedded
> > > ${HELPERBUILDDIR}/meta-openembedded/meta-oe
> > > ${HELPERBUILDDIR}/meta-openembedded/meta-python
> > > ${HELPERBUILDDIR}/meta-openembedded/meta-perl
> > > ${HELPERBUILDDIR}/meta-openembedded/meta-networking
> > > ${HELPERBUILDDIR}/meta-openembedded/meta-multimedia
> > > ${HELPERBUILDDIR}/meta-openembedded/meta-gnome
> > > ${HELPERBUILDDIR}/meta-openembedded/meta-xfce
> > > ${HELPERBUILDDIR}/meta-openembedded/meta-filesystems
> > > ${HELPERBUILDDIR}/meta-openembedded/meta-initramfs
> > > ${HELPERBUILDDIR}/meta-openembedded/meta-webserver "
> > > +]
> > > +},
> > > "a-quick" : {
> > > "TEMPLATE" : "trigger-build"
> > > },
> >
> > Would it be better to have one setup-auh/run-auh script and add
> > ${HELPERTARGET} as a parameter?
> >
> > This should translate to "auh" or "auh-meta-oe" and the script can then
> > have conditionals in to handle both cases rather than duplicating the
> > script?
>
> The setup-auh script is cloning its own repository instead of using
> the one initialized by CI.
> Any reason to not use them ? We could use the NEEDREPOS instead of the
> setup script,
> and add the auto-upgrade-helper repository in the config.
I think it does it's own clone for historical reasons from when it was
a standalone tool. I'm fine with letting the autobuilder handle this.
> > > diff --git a/scripts/run-auh-oe b/scripts/run-auh-oe
> > > new file mode 100755
> > > index 000..588755a
> > > --- /dev/null
> > > +++ b/scripts/run-auh-oe
> >
> > Similarly, this should be run-auh-meta-oe.
> >
> > > @@ -0,0 +1,45 @@
> > > +#!/bin/bash
> > > +#
> > > +# SPDX-License-Identifier: GPL-2.0-only
> > > +#
> > > +# Run Auto Upgrade Helper in a directory set up by setup_auh.
> > > +#
> > > +# Called with $1 - the directory where the setup was created
> > > +
> > > +if [ -z $1 ]; then
> > > + echo "Use: $0 [auh_setup_dir] [publish_dir] [meta_dir] [meta_list]"
> > > + exit 1
> > > +fi
> > > +
> > > +full_dir=$(readlink -e $1)
> > > +
> > > +auh_dir=$full_dir/auto-upgrade-helper
> > > +poky_dir=$full_dir/poky
> > > +
> > > +build_dir=$full_dir/build
> > > +sstate_dir=$full_dir/build/sstate-cache
> > > +meta_dir=$3
> > > +meta_list=${@:4}
> > > +machine_list="qemux86 qemux86-64 qemuarm qemumips qemuppc qemux86_musl"
> >
> > Do we need to vary the machine_list per layer? At present it only seems
> > t
[yocto] [yocto-autobuilder-helper] [PATCH 3/5] metrics: Pass branchname to scripts
To prepapre to run the scripts per branch, pass the branchname to the scripts.
Signed-off-by: Richard Purdie
---
config.json | 4 ++--
scripts/run-cvecheck | 1 +
scripts/run-patchmetrics | 1 +
3 files changed, 4 insertions(+), 2 deletions(-)
diff --git a/config.json b/config.json
index f225148..0c35632 100644
--- a/config.json
+++ b/config.json
@@ -1209,11 +1209,11 @@
],
"step1" : {
"shortname" : "Generating patch metrics",
-"EXTRACMDS" :
["../../yocto-autobuilder-helper/scripts/run-patchmetrics ../ ../meta/
${HELPERRESULTSDIR}/../../patchmetrics ."]
+"EXTRACMDS" :
["../../yocto-autobuilder-helper/scripts/run-patchmetrics ../ ../meta/
${HELPERRESULTSDIR}/../../patchmetrics . ${HELPERBRANCHNAME}"]
},
"step2" : {
"shortname" : "Running CVE checks",
-"EXTRACMDS" :
["../../yocto-autobuilder-helper/scripts/run-cvecheck ../ ../meta/
${HELPERRESULTSDIR}/../../patchmetrics ."]
+"EXTRACMDS" :
["../../yocto-autobuilder-helper/scripts/run-cvecheck ../ ../meta/
${HELPERRESULTSDIR}/../../patchmetrics . ${HELPERBRANCHNAME}"]
}
},
diff --git a/scripts/run-cvecheck b/scripts/run-cvecheck
index 35c796b..d48fd68 100755
--- a/scripts/run-cvecheck
+++ b/scripts/run-cvecheck
@@ -6,6 +6,7 @@ PARENTDIR=`realpath $1`
TARGETDIR=`realpath $2`
RESULTSDIR=`realpath -m $3`
BUILDDIR=`realpath $4`
+BRANCH=$5
OURDIR=`dirname $0`
TIMESTAMP=`date +"%s"`
diff --git a/scripts/run-patchmetrics b/scripts/run-patchmetrics
index e45d463..20e6268 100755
--- a/scripts/run-patchmetrics
+++ b/scripts/run-patchmetrics
@@ -6,6 +6,7 @@ PARENTDIR=`realpath $1`
TARGETDIR=`realpath $2`
RESULTSDIR=`realpath -m $3`
BUILDDIR=`realpath $4`
+BRANCH=$5
OURDIR=`dirname $0`
TIMESTAMP=`date +"%s"`
--
2.39.2
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#61456): https://lists.yoctoproject.org/g/yocto/message/61456
Mute This Topic: https://lists.yoctoproject.org/mt/102138010/21656
Group Owner: yocto+ow...@lists.yoctoproject.org
Unsubscribe:
https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[yocto] [yocto-autobuilder-helper] [PATCH 5/5] scripts/run-cvecheck: Remove branch iteration
Rather than running multiple checkouts, lets move this to the autobuilder to handle and have it trigger the builds with the right checkouts. Signed-off-by: Richard Purdie --- scripts/run-cvecheck | 52 1 file changed, 23 insertions(+), 29 deletions(-) diff --git a/scripts/run-cvecheck b/scripts/run-cvecheck index d48fd68..6294fe6 100755 --- a/scripts/run-cvecheck +++ b/scripts/run-cvecheck @@ -22,34 +22,28 @@ if [ ! -d $RESULTSDIR ]; then mkdir $RESULTSDIR fi -for branch in master mickledore langdale kirkstone dunfell; do -mkdir -p $PARENTDIR/yocto-metrics/cve-check/$branch/ -git -C $PARENTDIR reset origin/$branch --hard -rm conf/local.conf -rm conf/bblayers.conf -rm -f conf/templateconf.cfg -rm tmp/ -rf -unset BB_ENV_PASSTHROUGH_ADDITIONS -unset BB_ENV_EXTRAWHITE -cd .. -. oe-init-build-env build -bitbake world --runall cve_check -R conf/distro/include/cve-extra-exclusions.inc -if [ -e tmp/log/cve/cve-summary.json ]; then -git -C $PARENTDIR/yocto-metrics rm cve-check/$branch/*.json -mkdir -p $PARENTDIR/yocto-metrics/cve-check/$branch -cp tmp/log/cve/cve-summary.json $PARENTDIR/yocto-metrics/cve-check/$branch/$TIMESTAMP.json -git -C $PARENTDIR/yocto-metrics add cve-check/$branch/$TIMESTAMP.json -git -C $PARENTDIR/yocto-metrics commit -asm "Autobuilder adding new CVE data for branch $branch" -git -C $PARENTDIR/yocto-metrics push -$OURDIR/cve-report.py tmp/log/cve/cve-summary.json > $RESULTSDIR/cve-status-$branch.txt -fi -done +mkdir -p $PARENTDIR/yocto-metrics/cve-check/$BRANCH/ +cd .. +. oe-init-build-env build +bitbake world --runall cve_check -R conf/distro/include/cve-extra-exclusions.inc +if [ -e tmp/log/cve/cve-summary.json ]; then +git -C $PARENTDIR/yocto-metrics rm cve-check/$BRANCH/*.json +mkdir -p $PARENTDIR/yocto-metrics/cve-check/$BRANCH +cp tmp/log/cve/cve-summary.json $PARENTDIR/yocto-metrics/cve-check/$BRANCH/$TIMESTAMP.json +git -C $PARENTDIR/yocto-metrics add cve-check/$BRANCH/$TIMESTAMP.json +git -C $PARENTDIR/yocto-metrics commit -asm "Autobuilder adding new CVE data for branch $BRANCH" +git -C $PARENTDIR/yocto-metrics push +$OURDIR/cve-report.py tmp/log/cve/cve-summary.json > $RESULTSDIR/cve-status-$BRANCH.txt +fi + +if [ "$BRANCH" = "master" ]; then +mkdir -p $PARENTDIR/yocto-metrics/cve-check/ +$OURDIR/cve-generate-chartdata --json $PARENTDIR/yocto-metrics/cve-count-byday.json --resultsdir $PARENTDIR/yocto-metrics/cve-check/ +git -C $PARENTDIR/yocto-metrics add cve-count-byday.json +git -C $PARENTDIR/yocto-metrics commit -asm "Autobuilder updating CVE counts" +git -C $PARENTDIR/yocto-metrics push -mkdir -p $PARENTDIR/yocto-metrics/cve-check/ -$OURDIR/cve-generate-chartdata --json $PARENTDIR/yocto-metrics/cve-count-byday.json --resultsdir $PARENTDIR/yocto-metrics/cve-check/ -git -C $PARENTDIR/yocto-metrics add cve-count-byday.json -git -C $PARENTDIR/yocto-metrics commit -asm "Autobuilder updating CVE counts" -git -C $PARENTDIR/yocto-metrics push +cp $PARENTDIR/yocto-metrics/cve-count-byday.json $RESULTSDIR +cp $PARENTDIR/yocto-metrics/cve-count-byday-lastyear.json $RESULTSDIR +fi -cp $PARENTDIR/yocto-metrics/cve-count-byday.json $RESULTSDIR -cp $PARENTDIR/yocto-metrics/cve-count-byday-lastyear.json $RESULTSDIR -- 2.39.2 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#61458): https://lists.yoctoproject.org/g/yocto/message/61458 Mute This Topic: https://lists.yoctoproject.org/mt/102138012/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] [yocto-autobuilder-helper] [PATCH 4/5] scripts/run-patchmetrics: Only monitor master branch
We only monitor the master branch for patch metrics as we can't really make improvements to release branches. Signed-off-by: Richard Purdie --- scripts/run-patchmetrics | 5 + 1 file changed, 5 insertions(+) diff --git a/scripts/run-patchmetrics b/scripts/run-patchmetrics index 20e6268..391ac45 100755 --- a/scripts/run-patchmetrics +++ b/scripts/run-patchmetrics @@ -11,6 +11,11 @@ OURDIR=`dirname $0` TIMESTAMP=`date +"%s"` +# We only monitor patch metrics on the master branch +if [ "$BRANCH" != "master" ]; then +exit 0 +fi + # # Patch Metrics # -- 2.39.2 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#61457): https://lists.yoctoproject.org/g/yocto/message/61457 Mute This Topic: https://lists.yoctoproject.org/mt/102138011/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] [yocto-autobuilder-helper] [PATCH 2/5] scripts/run-patchmetrics: Split out CVE checks
Split the CVE checks from the patch metrics script
Signed-off-by: Richard Purdie
---
config.json | 6 +
scripts/run-cvecheck | 54
scripts/run-patchmetrics | 36 ---
3 files changed, 60 insertions(+), 36 deletions(-)
create mode 100755 scripts/run-cvecheck
diff --git a/config.json b/config.json
index bebd999..f225148 100644
--- a/config.json
+++ b/config.json
@@ -1208,8 +1208,14 @@
"BB_SERVER_TIMEOUT = '0'"
],
"step1" : {
+"shortname" : "Generating patch metrics",
"EXTRACMDS" :
["../../yocto-autobuilder-helper/scripts/run-patchmetrics ../ ../meta/
${HELPERRESULTSDIR}/../../patchmetrics ."]
+},
+"step2" : {
+"shortname" : "Running CVE checks",
+"EXTRACMDS" :
["../../yocto-autobuilder-helper/scripts/run-cvecheck ../ ../meta/
${HELPERRESULTSDIR}/../../patchmetrics ."]
}
+
},
"meta-mingw" : {
"NEEDREPOS" : ["poky", "meta-mingw"],
diff --git a/scripts/run-cvecheck b/scripts/run-cvecheck
new file mode 100755
index 000..35c796b
--- /dev/null
+++ b/scripts/run-cvecheck
@@ -0,0 +1,54 @@
+#!/bin/bash
+#
+# SPDX-License-Identifier: GPL-2.0-only
+#
+PARENTDIR=`realpath $1`
+TARGETDIR=`realpath $2`
+RESULTSDIR=`realpath -m $3`
+BUILDDIR=`realpath $4`
+OURDIR=`dirname $0`
+
+TIMESTAMP=`date +"%s"`
+
+#
+# CVE Checks
+#
+if [ ! -e $PARENTDIR/yocto-metrics ]; then
+git clone ssh://g...@push.yoctoproject.org/yocto-metrics
$PARENTDIR/yocto-metrics
+fi
+
+if [ ! -d $RESULTSDIR ]; then
+mkdir $RESULTSDIR
+fi
+
+for branch in master mickledore langdale kirkstone dunfell; do
+mkdir -p $PARENTDIR/yocto-metrics/cve-check/$branch/
+git -C $PARENTDIR reset origin/$branch --hard
+rm conf/local.conf
+rm conf/bblayers.conf
+rm -f conf/templateconf.cfg
+rm tmp/ -rf
+unset BB_ENV_PASSTHROUGH_ADDITIONS
+unset BB_ENV_EXTRAWHITE
+cd ..
+. oe-init-build-env build
+bitbake world --runall cve_check -R
conf/distro/include/cve-extra-exclusions.inc
+if [ -e tmp/log/cve/cve-summary.json ]; then
+git -C $PARENTDIR/yocto-metrics rm cve-check/$branch/*.json
+mkdir -p $PARENTDIR/yocto-metrics/cve-check/$branch
+cp tmp/log/cve/cve-summary.json
$PARENTDIR/yocto-metrics/cve-check/$branch/$TIMESTAMP.json
+git -C $PARENTDIR/yocto-metrics add cve-check/$branch/$TIMESTAMP.json
+git -C $PARENTDIR/yocto-metrics commit -asm "Autobuilder adding new
CVE data for branch $branch"
+git -C $PARENTDIR/yocto-metrics push
+$OURDIR/cve-report.py tmp/log/cve/cve-summary.json >
$RESULTSDIR/cve-status-$branch.txt
+fi
+done
+
+mkdir -p $PARENTDIR/yocto-metrics/cve-check/
+$OURDIR/cve-generate-chartdata --json
$PARENTDIR/yocto-metrics/cve-count-byday.json --resultsdir
$PARENTDIR/yocto-metrics/cve-check/
+git -C $PARENTDIR/yocto-metrics add cve-count-byday.json
+git -C $PARENTDIR/yocto-metrics commit -asm "Autobuilder updating CVE counts"
+git -C $PARENTDIR/yocto-metrics push
+
+cp $PARENTDIR/yocto-metrics/cve-count-byday.json $RESULTSDIR
+cp $PARENTDIR/yocto-metrics/cve-count-byday-lastyear.json $RESULTSDIR
diff --git a/scripts/run-patchmetrics b/scripts/run-patchmetrics
index abe58c7..e45d463 100755
--- a/scripts/run-patchmetrics
+++ b/scripts/run-patchmetrics
@@ -27,39 +27,3 @@ fi
$OURDIR/patchmetrics-generate-chartdata --json
$PARENTDIR/yocto-metrics/patch-status.json --outputdir $RESULTSDIR
cp $PARENTDIR/yocto-metrics/patch-status.json $RESULTSDIR
cp $PARENTDIR/yocto-metrics/patch-status/* $RESULTSDIR
-
-#
-# CVE Checks
-#
-for branch in master mickledore langdale kirkstone dunfell; do
-mkdir -p $PARENTDIR/yocto-metrics/cve-check/$branch/
-git -C $PARENTDIR reset origin/$branch --hard
-rm conf/local.conf
-rm conf/bblayers.conf
-rm -f conf/templateconf.cfg
-rm tmp/ -rf
-unset BB_ENV_PASSTHROUGH_ADDITIONS
-unset BB_ENV_EXTRAWHITE
-cd ..
-. oe-init-build-env build
-bitbake world --runall cve_check -R
conf/distro/include/cve-extra-exclusions.inc
-if [ -e tmp/log/cve/cve-summary.json ]; then
-git -C $PARENTDIR/yocto-metrics rm cve-check/$branch/*.json
-mkdir -p $PARENTDIR/yocto-metrics/cve-check/$branch
-cp tmp/log/cve/cve-summary.json
$PARENTDIR/yocto-metrics/cve-check/$branch/$TIMESTAMP.json
-git -C $PARENTDIR/yocto-metrics add cve-check/$branch/$TIMESTAMP.json
-git -C $PARENTDIR/yocto-metrics commit -asm "Autobuilder adding new
CVE data for branch $branch"
-git -C $PARENTDIR/yocto-metrics push
-$OURDIR/cve-report.py tmp/log/cve/cve-summary.json >
$RESULTSDIR/cve-status-$branc
[yocto] [yocto-autobuilder-helper] [PATCH 1/5] scripts/run-patchmetrics: Only clone metrics if it isn't present
To prepare for splitting things up, only clone the metrics repo if it isn't present. Signed-off-by: Richard Purdie --- scripts/run-patchmetrics | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/scripts/run-patchmetrics b/scripts/run-patchmetrics index fc3f214..abe58c7 100755 --- a/scripts/run-patchmetrics +++ b/scripts/run-patchmetrics @@ -13,7 +13,9 @@ TIMESTAMP=`date +"%s"` # # Patch Metrics # -git clone ssh://g...@push.yoctoproject.org/yocto-metrics $PARENTDIR/yocto-metrics +if [ ! -e $PARENTDIR/yocto-metrics ]; then +git clone ssh://g...@push.yoctoproject.org/yocto-metrics $PARENTDIR/yocto-metrics +fi $OURDIR/patchmetrics-update --repo $PARENTDIR --patchscript $PARENTDIR/scripts/contrib/patchreview.py --metadata $TARGETDIR --json $PARENTDIR/yocto-metrics/patch-status.json git -C $PARENTDIR/yocto-metrics commit -asm "Autobuilder adding new patch stats" git -C $PARENTDIR/yocto-metrics push -- 2.39.2 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#61454): https://lists.yoctoproject.org/g/yocto/message/61454 Mute This Topic: https://lists.yoctoproject.org/mt/102138008/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[yocto] [yocto-autobuilder-helper] [PATCH] config.json: Fix check-layer for meta-poky/meta-yocto-bsp
Currently, the tests for meta-poky and meta-yocto-bsp were malfunctioning.
We need to a) set a distro other than poky and b) avoid the isolation wrapper
since it will use the default environment and reset to the poky distro and
default bblayers.conf which will disable the layer test.
With those two fixes, the layer checks work correctly.
Signed-off-by: Richard Purdie
---
config.json | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/config.json b/config.json
index 27e56c5..55fde54 100644
--- a/config.json
+++ b/config.json
@@ -1150,11 +1150,12 @@
},
"check-layer" : {
"NEEDREPOS" : ["poky", "meta-mingw"],
+"DISTRO" : "nodistro",
"step1" : {
-"EXTRACMDS" : ["yocto-check-layer-wrapper ../meta-poky"]
+"EXTRACMDS" : ["bitbake-layers remove-layer ../meta-poky;
yocto-check-layer ../meta-poky"]
},
"step2" : {
-"EXTRACMDS" : ["yocto-check-layer-wrapper ../meta-yocto-bsp"]
+"EXTRACMDS" : ["bitbake-layers remove-layer ../meta-yocto-bsp;
yocto-check-layer ../meta-yocto-bsp"]
},
"step3" : {
"EXTRACMDS" : ["yocto-check-layer-wrapper ../meta-mingw"]
--
2.39.2
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#61444): https://lists.yoctoproject.org/g/yocto/message/61444
Mute This Topic: https://lists.yoctoproject.org/mt/102097822/21656
Group Owner: yocto+ow...@lists.yoctoproject.org
Unsubscribe:
https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] [yocto-autobuilder-helper][PATCH v2] AUH: Add Openembedded auto-update-helper with list of layer to test
Hi David,
Thanks for revising this, it is heading in the right direction.
On Fri, 2023-10-20 at 16:05 +0200, David Pierret wrote:
> - Add setup and run script openembedded specific
> - Add job config
>
> Signed-off-by: David Pierret
> ---
> config.json | 6 ++
> scripts/run-auh-oe | 45
> scripts/setup-auh-oe | 34 +
> 3 files changed, 85 insertions(+)
> create mode 100755 scripts/run-auh-oe
> create mode 100755 scripts/setup-auh-oe
>
> diff --git a/config.json b/config.json
> index 3acb710..bbd0aaf 100644
> --- a/config.json
> +++ b/config.json
> @@ -1420,6 +1420,12 @@
> "${SCRIPTSDIR}/setup-auh ${HELPERBUILDDIR};
> ${SCRIPTSDIR}/run-auh ${HELPERBUILDDIR} ${WEBPUBLISH_DIR}/pub/auh/"
> ]
> },
> +"auh-openembedded" : {
This needs to be "auh-meta-oe" since "auh" is for openembedded-core and
this is otherwise going to be confusing.
> +"NEEDREPOS" : ["poky", "meta-openembedded"],
> +"EXTRAPLAINCMDS" : [
> +"${SCRIPTSDIR}/setup-auh-oe ${HELPERBUILDDIR};
> ${SCRIPTSDIR}/run-auh-oe ${HELPERBUILDDIR} ${WEBPUBLISH_DIR}/pub/auh/
> ${HELPERBUILDDIR}/meta-openembedded
> ${HELPERBUILDDIR}/meta-openembedded/meta-oe
> ${HELPERBUILDDIR}/meta-openembedded/meta-python
> ${HELPERBUILDDIR}/meta-openembedded/meta-perl
> ${HELPERBUILDDIR}/meta-openembedded/meta-networking
> ${HELPERBUILDDIR}/meta-openembedded/meta-multimedia
> ${HELPERBUILDDIR}/meta-openembedded/meta-gnome
> ${HELPERBUILDDIR}/meta-openembedded/meta-xfce
> ${HELPERBUILDDIR}/meta-openembedded/meta-filesystems
> ${HELPERBUILDDIR}/meta-openembedded/meta-initramfs
> ${HELPERBUILDDIR}/meta-openembedded/meta-webserver "
> +]
> +},
> "a-quick" : {
> "TEMPLATE" : "trigger-build"
> },
Would it be better to have one setup-auh/run-auh script and add
${HELPERTARGET} as a parameter?
This should translate to "auh" or "auh-meta-oe" and the script can then
have conditionals in to handle both cases rather than duplicating the
script?
> diff --git a/scripts/run-auh-oe b/scripts/run-auh-oe
> new file mode 100755
> index 000..588755a
> --- /dev/null
> +++ b/scripts/run-auh-oe
Similarly, this should be run-auh-meta-oe.
> @@ -0,0 +1,45 @@
> +#!/bin/bash
> +#
> +# SPDX-License-Identifier: GPL-2.0-only
> +#
> +# Run Auto Upgrade Helper in a directory set up by setup_auh.
> +#
> +# Called with $1 - the directory where the setup was created
> +
> +if [ -z $1 ]; then
> + echo "Use: $0 [auh_setup_dir] [publish_dir] [meta_dir] [meta_list]"
> + exit 1
> +fi
> +
> +full_dir=$(readlink -e $1)
> +
> +auh_dir=$full_dir/auto-upgrade-helper
> +poky_dir=$full_dir/poky
> +
> +build_dir=$full_dir/build
> +sstate_dir=$full_dir/build/sstate-cache
> +meta_dir=$3
> +meta_list=${@:4}
> +machine_list="qemux86 qemux86-64 qemuarm qemumips qemuppc qemux86_musl"
Do we need to vary the machine_list per layer? At present it only seems
to support one list anyway so I'm not sure this does anything useful?
> +pushd $meta_dir || exit 1
> +
> +# Base the upgrades on meta_openembedded master
> +git fetch origin
> +git checkout -B tmp-auh-upgrades origin/main
> +
> +source $poky_dir/oe-init-build-env $build_dir
> +
> +# build the layer_names variable to me used in the command line
> +layer_names=""
> +for d in $meta_list; do
> + layer_names+=" $(basename ${d})"
> +done
> +
> +$auh_dir/upgrade-helper.py -e --layer-dir ${meta_dir} --layer-names
> ${layer_names} --layer-machines ${machine_list} -- all
Would it be simpler just to iterate, calling upgrade-helper once per
sub-layer or meta-openembedded?
You may have considered that in which case I'd just like to understand
why you ended up with this as the preferred way of doing things?
In some ways a separate report/run may be useful for the way meta-oe
maintainers might handle this?
> +
> +if [ -n $2 ]; then
> + cp -rf $build_dir/upgrade-helper/* $2
> +fi
> +
> +popd
> diff --git a/scripts/setup-auh-oe b/scripts/setup-auh-oe
> new file mode 100755
> index 000..ba1a7fb
> --- /dev/null
> +++ b/scripts/setup-auh-oe
This naming also needs tweaking.
> @@ -0,0 +1,34 @@
> +#!/bin/bash
> +#
> +# SPDX-License-Identifier: GPL-2.0-only
> +#
> +# Initialize Auto Upgrade Helper in a directory.
> +#
> +# Called with $1 - the directory to place the setup
> +CONFIG_DIR=`dirname $0`/auh-config
> +
> +if [ -z $1 ]; then
> + echo "Use: $0 target_dir"
> + exit 1
> +fi
> +
> +mkdir -p $1
> +pushd $1
> +
> +git clone git://git.yoctoproject.org/poky
> +pushd poky
> +git config user.email a...@yoctoproject.org
> +git config user.name "Auto Upgrade Helper"
> +popd
> +git clone git://git.openembedded.org/meta-openembedded
> +pushd meta-openembedded
> +git config user.email a...@yoctoproject.org
> +git config user.name "Auto Upgrade Helper"
> +popd
> +git clone git://git.yoctoproject.org/auto-upgrade-helper
> +source
Re: [yocto] [qa-build-notification] QA notification for completed autobuilder build (yocto-4.3.rc1)
On Thu, 2023-10-19 at 14:06 +, Jing Hui Tham wrote: > Hi all, > > Intel and WR YP QA is planning for QA execution for YP build yocto-4.3.rc1. > We are planning to execute following tests for this cycle: > > OEQA-manual tests for following module: > 1. OE-Core > 2. BSP-hw > > Runtime auto test for following platforms: > 1. MinnowBoard Turbot - 32bit > 2. Kaby Lake (7th Generation Intel(r) Core(tm) Processors) > 3. Tiger Lake (11th Generation Intel(r) Core(tm) Processors) > 4. Alder Lake-S (12th Generation Intel(r) Core(tm) Processors) > 5. Raptor Lake-P (13th Generation Intel(r) Core(tm) Processors) > 6. Beaglebone > > > ETA for completion next Tuesday, October 24. We've realised there is a nasty bug in rc1 related to patchtest deleting local changes. Given the build failure in the rc1 build and other security issues we're thinking we should make an rc2 build and abandon rc1. Sorry for the churn. The new build should be ready soon with any luck. Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#61422): https://lists.yoctoproject.org/g/yocto/message/61422 Mute This Topic: https://lists.yoctoproject.org/mt/102060627/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] [yocto-security] SRTool usage for CVE management at YP
On Wed, 2023-10-18 at 07:43 +0200, Marta Rybczynska wrote: > Hello all, > There' a discussion pending on the usage of SRTool and CVE management > for the Yocto project in general. It is related to the need of having > a list of CVEs the project is affected by, those fixed and those that > we know we are not affected. > > In the previous episode, we have had a demo of SRTool by David Reyna. > This weekend he has updated the code base. The next call is tomorrow > (Thursday, 19th October, half an hour after the end of the Bug Triage > meeting) to discuss conclusions of first tests and the next steps. If > you are interested to join, let us know! I'd be happy to listen in. Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#61420): https://lists.yoctoproject.org/g/yocto/message/61420 Mute This Topic: https://lists.yoctoproject.org/mt/102062524/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] [yocto-autobuilder-helper][PATCH v2] meta-oe-mirror: Use a 2 step job to fetch and verify meta-openembedded mirror.
On Thu, 2023-10-19 at 12:52 +0100, Richard Purdie via lists.yoctoproject.org wrote: > On Thu, 2023-10-19 at 11:58 +0100, Richard Purdie via > lists.yoctoproject.org wrote: > > On Wed, 2023-10-18 at 15:46 +0200, David Pierret wrote: > > > Inspired from trigger-build and trigger-build-post-trigger > > > The branch must be selected on build configuration. > > > > > > Signed-off-by: David Pierret > > > Reviewed-by: Yoann Congal > > > --- > > > config.json | 30 ++ > > > 1 file changed, 30 insertions(+) > > > > I fixed a json parsing issue and merged. I also added the autobuilder2 > > config to support it. A test build is now running here: > > > > https://autobuilder.yoctoproject.org/typhoon/#/builders/156/builds/1 > > That failed with a single fetch failure which isn't related to this > patch. > > I've therefore pulled the patch into the dunfell, kirkstone and > mickledore branches. > > Autobuilder enabling is here: > > https://git.yoctoproject.org/yocto-autobuilder2/commit/?id=93c034d436cbe631db2354a945850c3e97dd46bd > > with scheduling for the all branches here: > > https://git.yoctoproject.org/yocto-autobuilder2/commit/?id=94f2878f64f11993e8de8295bb1b46202b741019 Master reran and succeeded this time but is showing a lot of warnings: https://autobuilder.yoctoproject.org/typhoon/#/builders/156/builds/2/steps/12/logs/warnings Kirkstone has an issue with grubby-git and a missing SRCREV: https://autobuilder.yoctoproject.org/typhoon/#/builders/156/builds/3/steps/13/logs/stdio (plus warnings) Mickledore has an issue with uutils-coreutils: https://autobuilder.yoctoproject.org/typhoon/#/builders/156/builds/5 (plus warnings) Dunfell broke with space issues so I reran it here: https://autobuilder.yoctoproject.org/typhoon/#/builders/156/builds/6 So we're making progress but the meta-oe layer maintainers may need some tweaks on the same branches or we need to see some missing files into the mirror for those branches. We also need to find a way to address the warnings. Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#61416): https://lists.yoctoproject.org/g/yocto/message/61416 Mute This Topic: https://lists.yoctoproject.org/mt/102039136/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] QA notification for completed autobuilder build (yocto-4.3.rc1)
On Thu, 2023-10-19 at 14:28 +0200, Alexis Lothoré wrote: > Hi Ross, > On 10/19/23 13:04, Ross Burton wrote: > > > On 19 Oct 2023, at 09:36, Alexis Lothoré via lists.yoctoproject.org > > > wrote: > > I just skimmed your report and have some feedback to hopefully make it > > easier to read in the future. > > > > I’d suggest sorting the output in order of importance. For example, this > > is a section that I really don’t care about: > > > > Match: sdk_core-image-sato_x86_64_fvp-base_20230910083055 > > sdk_core-image-sato_x86_64_fvp-base_20231017222150 > > > > Put those at the bottom, or even better collate them into a single section > > where there have been no changes. > > > > Similarly: > > > > Match: runtime_core-image-sato_qemux86_20230911011430 > > runtime_core-image-sato_qemux86_20231017223736 > > Additionally, 1 new test(s) is/are present > > > > I guess marginally more important than identical results, but “there are > > new tests that passed” isn’t very interesting. > > Indeed, not so useful and hiding the real content at the bottom > > > Regression: oeselftest_ubuntu-22.04_qemux86-64_20230911011940 > > oeselftest_almalinux-9.2_qemux86-64_20231017221342 > > > > Should they have matched? The host distro doesn’t match and this matters > > for some of the tests, as some distros don’t support some of the selftests. > > In this case specifically, there are seven regressions and six of them are > > specific to the host changing, which has the side-effect of hiding the one > > actual regression. > > Yeah, that's a point I have been struggling with when starting to update those > tools. The initial assumption I have started working with, after discussing > the > matter with Richard (see [1]), is the following: > 1. MACHINE _must_ match between base and target > 2. Different HOSTS _can_ be cross-checked > > But the issue you are pointing tends to show it does not work well in some > cases. I will have to do some tests to see if dropping 2. reduce this noise > without loosing valuable data, or if we need to find something smarter What the autobuilder does is either runs one selftest on a random host for q-quick, or for a-full it will run five selftests, one for "centos", one for "arm", one for "ubuntu", one for "fedora" and one for "debian". Ideally we'd therefore compare debian to debian if we have any choice. It is a question of finding the closest matches. That is hard in code though whilst keeping it generic. Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#61415): https://lists.yoctoproject.org/g/yocto/message/61415 Mute This Topic: https://lists.yoctoproject.org/mt/102034502/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] QA notification for completed autobuilder build (yocto-4.3.rc1)
On Thu, 2023-10-19 at 10:36 +0200, Alexis Lothoré wrote: > Hello, > > On 10/18/23 08:16, Pokybuild User wrote: > > > > A build flagged for QA (yocto-4.3.rc1) was completed on the autobuilder > > and is available at: > > > > > > https://autobuilder.yocto.io/pub/releases/yocto-4.3.rc1 > > The regression report looks worryingly empty. It has been generated with > yocto-4.2 as comparison base. When taking a look at yocto-testresults content > for it, I only find a few test results (11, while it should be something > around > 300). I am not sure yet if we did some mistakes while re-uploading manually > missing tests results during the 4.2 cycle, or if tests results are indeed > suffering some issues for release builds (this is repeatable on different > 4.2.x > releases). I'd noticed that too, not sure what happened but it wasn't what I'd expected. It would be good to understand what went wrong. > So in the mean time, here is a manual regression report between 4.3_M3 and > yocto-4.3.rc1: > > https://pastebin.com/6QbfGstR > > The report has a "rate limit" applied for noisy regression (patch incoming) I was thinking about ideas like printing the regressions first, then the matches. I think Ross has some good feedback about how we can improve things. Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#61413): https://lists.yoctoproject.org/g/yocto/message/61413 Mute This Topic: https://lists.yoctoproject.org/mt/102034502/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] [yocto-autobuilder-helper][PATCH v2] meta-oe-mirror: Use a 2 step job to fetch and verify meta-openembedded mirror.
On Thu, 2023-10-19 at 11:58 +0100, Richard Purdie via lists.yoctoproject.org wrote: > On Wed, 2023-10-18 at 15:46 +0200, David Pierret wrote: > > Inspired from trigger-build and trigger-build-post-trigger > > The branch must be selected on build configuration. > > > > Signed-off-by: David Pierret > > Reviewed-by: Yoann Congal > > --- > > config.json | 30 ++ > > 1 file changed, 30 insertions(+) > > I fixed a json parsing issue and merged. I also added the autobuilder2 > config to support it. A test build is now running here: > > https://autobuilder.yoctoproject.org/typhoon/#/builders/156/builds/1 That failed with a single fetch failure which isn't related to this patch. I've therefore pulled the patch into the dunfell, kirkstone and mickledore branches. Autobuilder enabling is here: https://git.yoctoproject.org/yocto-autobuilder2/commit/?id=93c034d436cbe631db2354a945850c3e97dd46bd with scheduling for the all branches here: https://git.yoctoproject.org/yocto-autobuilder2/commit/?id=94f2878f64f11993e8de8295bb1b46202b741019 Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#61412): https://lists.yoctoproject.org/g/yocto/message/61412 Mute This Topic: https://lists.yoctoproject.org/mt/102039136/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] [yocto-autobuilder-helper][PATCH v2] meta-oe-mirror: Use a 2 step job to fetch and verify meta-openembedded mirror.
On Wed, 2023-10-18 at 15:46 +0200, David Pierret wrote: > Inspired from trigger-build and trigger-build-post-trigger > The branch must be selected on build configuration. > > Signed-off-by: David Pierret > Reviewed-by: Yoann Congal > --- > config.json | 30 ++ > 1 file changed, 30 insertions(+) I fixed a json parsing issue and merged. I also added the autobuilder2 config to support it. A test build is now running here: https://autobuilder.yoctoproject.org/typhoon/#/builders/156/builds/1 Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#61410): https://lists.yoctoproject.org/g/yocto/message/61410 Mute This Topic: https://lists.yoctoproject.org/mt/102039136/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] [tsc] [qa-build-notification] QA notification for completed autobuilder build (yocto-4.3.rc1)
On Wed, 2023-10-18 at 15:32 +, Ross Burton wrote: > On 18 Oct 2023, at 07:29, Richard Purdie via lists.yoctoproject.org > wrote: > > > > On Wed, 2023-10-18 at 06:16 +, Pokybuild User wrote: > > >A build flagged for QA (yocto-4.3.rc1) was completed on the > > > autobuilder and is available at: > > > > > > > > >https://autobuilder.yocto.io/pub/releases/yocto-4.3.rc1 > > > > > > > > >Build URL: > > > https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/6062 > > > > There was one failure in the build, a serial login issue on ttyS1, so > > an occurrence of our annoying 6.5 issue. This is the first time we've > > seen it with the workaround applied. > > > > The question is to proceed with rc1 in testing, or apply the upstream > > fixes and try an rc2 with that? I'm torn… > > I’d say we should proceed with RC1 whilst testing upstream fixes. > > If they manage to shake out all of the failures then we can decide > whether to spin another RC with the fixes in, or just release with > what with have - with copious release notes obviously. Lets run with this approach, I agree. I am running test builds in the background and we'll see how reliable the upstream fix is. I think moving forward with rc1 as is makes sense. Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#61404): https://lists.yoctoproject.org/g/yocto/message/61404 Mute This Topic: https://lists.yoctoproject.org/mt/102041450/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [yocto] CDN for sstate.yoctoproject.org
On Wed, 2023-10-18 at 14:02 +0200, Alexander Kanavin wrote: > On Wed, 18 Oct 2023 at 08:45, Richard Purdie > wrote: > > I'm torn on the targets to test as sato-sdk is a large image and world > > is a lot of work. I'd be tempted to test sato, weston and full-cmdline? > > World is a good test I guess and if from sstate, shouldn't have that > > much of an issue. It does also prove things are working. > > I ran '-S printdiff world' on a blank build directory. First, > scalability isn't great: > > Initialising tasks: 100% > > ##| > Time: 0:24:19 > Checking sstate mirror object availability: 100% > > ##| > Time: 0:12:14 > > So it's taking 36 minutes just preparing to fetch the objects, and 2/3 > of that time goes into communicating with hash equivalence server > (e.g. BB_HASHSERVE_UPSTREAM = "hashserv.yocto.io:8687"). FWIW the second time an artefact is pulled from the CDN, it will be much faster. I don't know how much that is a factor here. We know hashequiv is slow, particularly from europe. We've ideas on that we're looking into but for now it is what it is. Part of this work was to find out what the issues are so we write that one up. > Second, there are significant misses. I don't have a clear theory > where they come from, just want to list them: > > The differences between the current build and any cached tasks start > at the following tasks: > /srv/work/alex/poky/meta/recipes-core/meta/meta-ide-support.bb:do_populate_ide_support > /srv/work/alex/poky/meta/recipes-graphics/xorg-driver/xf86-input-synaptics_1.9.2.bb:do_collect_spdx_deps > /srv/work/alex/poky/meta/recipes-devtools/bootchart2/bootchart2_0.14.9.bb:do_create_runtime_spdx > /srv/work/alex/poky/meta/recipes-graphics/igt-gpu-tools/igt-gpu-tools_git.bb:do_collect_spdx_deps > /srv/work/alex/poky/meta/recipes-devtools/python/python3-dbusmock_0.29.1.bb:do_create_runtime_spdx > /srv/work/alex/poky/meta/recipes-devtools/dpkg/dpkg_1.22.0.bb:do_configure > /srv/work/alex/poky/meta/recipes-graphics/xorg-driver/xf86-input-vmmouse_13.2.0.bb:do_collect_spdx_deps > /srv/work/alex/poky/meta/recipes-graphics/glew/glew_2.2.0.bb:do_collect_spdx_deps > /srv/work/alex/poky/meta/recipes-graphics/xorg-driver/xf86-input-evdev_2.10.6.bb:do_collect_spdx_deps > /srv/work/alex/poky/meta/recipes-graphics/libva/libva_2.19.0.bb:do_collect_spdx_deps > /srv/work/alex/poky/meta/recipes-sato/webkit/libwpe_1.14.1.bb:do_collect_spdx_deps > /srv/work/alex/poky/meta/recipes-multimedia/gstreamer/gstreamer1.0-meta-base.bb:do_collect_spdx_deps > /srv/work/alex/poky/meta/recipes-multimedia/gstreamer/gstreamer1.0-python_1.22.6.bb:do_collect_spdx_deps > /srv/work/alex/poky/meta/recipes-extended/cups/cups_2.4.6.bb:do_collect_spdx_deps > /srv/work/alex/poky/meta/recipes-gnome/libdazzle/libdazzle_3.44.0.bb:do_collect_spdx_deps > /srv/work/alex/poky/meta/recipes-graphics/igt-gpu-tools/igt-gpu-tools_git.bb:do_package > /srv/work/alex/poky/meta/recipes-multimedia/gstreamer/gst-devtools_1.22.6.bb:do_collect_spdx_deps > /srv/work/alex/poky/meta/recipes-core/packagegroups/packagegroup-core-sdk.bb:do_package > /srv/work/alex/poky/meta/recipes-graphics/xorg-driver/xf86-input-mouse_1.9.5.bb:do_collect_spdx_deps > /srv/work/alex/poky/meta/recipes-graphics/waffle/waffle_1.7.2.bb:do_collect_spdx_deps > /srv/work/alex/poky/meta/recipes-multimedia/alsa/alsa-tools_1.2.5.bb:do_collect_spdx_deps > /srv/work/alex/poky/meta/recipes-multimedia/gstreamer/gstreamer1.0-rtsp-server_1.22.6.bb:do_collect_spdx_deps > /srv/work/alex/poky/meta/recipes-devtools/apt/apt_2.6.1.bb:do_install > /srv/work/alex/poky/meta/recipes-gnome/gtk+/gtk4_4.12.3.bb:do_collect_spdx_deps > /srv/work/alex/poky/meta/recipes-devtools/devel-config/distcc-config.bb:do_create_runtime_spdx > /srv/work/alex/poky/meta/recipes-gnome/libhandy/libhandy_1.8.2.bb:do_collect_spdx_deps > /srv/work/alex/poky/meta/recipes-support/vim/vim_9.0.bb:do_collect_spdx_deps We'll have to dig into this. > So I think we should start with specific images first - minimal, > full-cmdline, weston, sato and sato-sdk are all much faster to check. Agreed, lets start with simpler images, full-cmdline, minimal, sato and weston. We can increase coverage as we move forward (world, sato-sdk). > On qemux86_64 none of them show misses, but on qemuarm64 there are > problems with sato, sato-sdk and weston, i.e. sato-sdk shows: > > The differences between the current build and any cached tasks start > at the following tasks: > /srv/work/alex/poky/meta/rec
Re: [yocto] CDN for sstate.yoctoproject.org
On Tue, 2023-10-17 at 17:33 +0200, Alexander Kanavin wrote: > Thanks for working on this! I finally got to playing with CDN mirror a > bit, and it seems to basically work ok, so next I'm going to write AB > tests that check that it remains useful. Specifically: > > 1. What should the test be? > > I tried 'bitbake -S printdiff core-image-sato' on a blank build > directory, and the result against current master is: > > === > Checking sstate mirror object availability: 100% > > ##| > Time: 0:05:35 > The differences between the current build and any cached tasks start > at the following tasks: > /srv/work/alex/poky/meta/recipes-sato/images/core-image-sato.bb:do_deploy_source_date_epoch > /srv/work/alex/poky/meta/recipes-core/base-files/base-files_3.0.14.bb:do_install > === > > I think this is pretty good! The two missing objects depend on the > current date, and so should go to the exception list in the test, and > otherwise there is a 100% match rate. The bitbake targets could be > 'world core-image-sato-sdk', and target machines could be qemux86_64 > and qemuarm64. That match does look good. As you say, the revision of the metadata will change base-files so that is expected. I'm torn on the targets to test as sato-sdk is a large image and world is a lot of work. I'd be tempted to test sato, weston and full-cmdline? World is a good test I guess and if from sstate, shouldn't have that much of an issue. It does also prove things are working. > Just to be sure that mismatches elsewhere will be reported as CDN > misses, adding my shadow 4.14 update and re-running the command shows: > > The differences between the current build and any cached tasks start > at the following tasks: > /srv/work/alex/poky/meta/recipes-gnome/hicolor-icon-theme/hicolor-icon-theme_0.17.bb:do_package > /srv/work/alex/poky/meta/recipes-sato/pulseaudio-sato/pulseaudio-client-conf-sato_1.bb:do_package > /srv/work/alex/poky/meta/recipes-core/initscripts/initscripts_1.0.bb:do_package > /srv/work/alex/poky/meta/recipes-core/update-rc.d/update-rc.d_0.8.bb:do_package > /srv/work/alex/poky/meta/recipes-multimedia/alsa/alsa-ucm-conf_1.2.10.bb:do_package > /srv/work/alex/poky/meta/recipes-kernel/wireless-regdb/wireless-regdb_2023.09.01.bb:do_package > /srv/work/alex/poky/meta/recipes-sato/packagegroups/packagegroup-core-x11-sato.bb:do_package > /srv/work/alex/poky/meta/recipes-kernel/linux-libc-headers/linux-libc-headers_6.5.bb:do_package > /srv/work/alex/poky/meta/recipes-core/packagegroups/packagegroup-core-ssh-dropbear.bb:do_package > /srv/work/alex/poky/meta/recipes-graphics/packagegroups/packagegroup-core-x11.bb:do_package > /srv/work/alex/poky/meta/recipes-sato/shutdown-desktop/shutdown-desktop.bb:do_package > virtual:native:/srv/work/alex/poky/meta/recipes-support/libbsd/libbsd_0.11.7.bb:do_prepare_recipe_sysroot > /srv/work/alex/poky/meta/recipes-support/ca-certificates/ca-certificates_20211016.bb:do_package > /srv/work/alex/poky/meta/recipes-core/sysvinit/sysvinit-inittab_2.88dsf.bb:do_package > virtual:native:/srv/work/alex/poky/meta/recipes-extended/shadow/shadow_4.14.0.bb:do_recipe_qa > ... (a few more missing do_package objects) > === > > 2. When to run the test, and against which commit in poky? > > RP suggested that the test could run in an antiphase with the nightly > a-quick (i.e. 12 hours after). We can do that for a start, but I'm > (perhaps prematurely) worrying that this will be unstable: either > because the objects from the nightly haven't yet propagated to CDN, or > because master has meanwhile (e.g. in the 12 hours that have passed) > gained new commits without corresponding cache objects. Are those real > concerns? > > Thoughts? You're right, we need to run the test against a commit we know has been built on the autobuilder. If we run with a newer commit we could easily see mismatch since it won't have been built yet. I'm less worried about CDN propagation, that should happen quickly and if something isn't present, it might just be slow to obtain/lookup as it ripples through the system. Either we start logging what we've built so we get the last known revisions, or we run the test as part of a-quick/a-full at the end of the build? I don't really want to extend the build but I'm not sure we may have much choice. Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#61387): https://lists.yoctoproject.org/g/yocto/message/61387 Mute This Topic: https://lists.yoctoproject.org/mt/101525879/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
