[OE-core] RDEPENDS:${PN}-lic and multilib
We've been successfully using the "lic-pkgs" image feature in order to
install licence information in our images for some time and we ensure that
the licences for header-only or static libraries are correctly included too
by adding RDEPENDS like:
RDEPENDS:${PN}-lic += "boost-lic"
to the affected recipes in the absence of anything better[1].
I've just tried to build a multilib image with both Scarthgap and Dunfell
and discovered that this doesn't work. I get warnings:
WARNING: lib32-lictest-1.0-r0 do_package: QA Issue: lib32-lictest package
lib32-lictest-lic - suspicious values 'boost-lic' in RDEPENDS [multilib]
WARNING: lib32-lictest-1.0-r0 do_package: QA Issue: lib32-lictest package
lib32-lictest-dev - suspicious values 'boost-lic-dev' in RRECOMMENDS [multilib]
WARNING: lib32-lictest-1.0-r0 do_package_qa: QA Issue: lib32-lictest-lic
rdepends on boost-lic, but it isn't a build dependency? [build-deps]
The solver then complains that nothing provides boost-lic (because I'm not
building "boost", just "lib32-boost".)
If I change the recipe to instead say:
RDEPENDS:${PN}-lic += "${MLPREFIX}boost-lic"
then the warnings go away and the image is generated correctly containing
lib32-boost-lic.
(I found that I needed to cleansstate both my test recipe and the image in
order to make the behaviour described above reproducible when switching
back and forth. Changes to RDEPENDS:${PN}-lic in the recipe alone didn't
seem to cause everything to be rebuilt correctly.)
RDEPENDS don't normally need to use ${MLPREFIX}. It looks like
ClassExtender's rename_packages automatically adds ${MLPREFIX} when
necessary to all the RDEPENDS overrides that are listed in PACKAGES. But
${PN}-lic isn't in PACKAGES so it doesn't get that treatment.
If I try to add ${PN}-lic to PACKAGES then I get:
ERROR: Nothing RPROVIDES 'lib32-boost-lic'
Whilst I can add ${MLPREFIX} to each entry in DEPENDS:${PN}-lic it feels as
if I might be fixing this problem in the wrong way. Is there something else
I should be doing instead?
Thanks.
Mike.
[1] https://lists.openembedded.org/g/openembedded-core/message/156909
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#200599):
https://lists.openembedded.org/g/openembedded-core/message/200599
Mute This Topic: https://lists.openembedded.org/mt/106650861/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [dunfell][PATCH] glibc: Fix CVE-2023-4911 "Looney Tunables"
From: Mike Crowe
Take the patch from the source for Debian's glibc 2.31-13+deb11u7
package, the changelog for which starts with:
glibc (2.31-13+deb11u7) bullseye-security; urgency=medium
* debian/patches/any/local-CVE-2023-4911.patch: Fix a buffer overflow in the
dynamic loader's processing of the GLIBC_TUNABLES environment variable
(CVE-2023-4911).
This addresses the "Looney Tunables" vulnerability described at
https://www.qualys.com/2023/10/03/cve-2023-4911/looney-tunables-local-privilege-escalation-glibc-ld-so.txt
Signed-off-by: Mike Crowe
---
.../glibc/glibc/CVE-2023-4911.patch | 63 +++
meta/recipes-core/glibc/glibc_2.31.bb | 1 +
2 files changed, 64 insertions(+)
create mode 100644 meta/recipes-core/glibc/glibc/CVE-2023-4911.patch
diff --git a/meta/recipes-core/glibc/glibc/CVE-2023-4911.patch
b/meta/recipes-core/glibc/glibc/CVE-2023-4911.patch
new file mode 100644
index 00..4d3146509a
--- /dev/null
+++ b/meta/recipes-core/glibc/glibc/CVE-2023-4911.patch
@@ -0,0 +1,63 @@
+From d2b77337f734fcacdfc8e0ddec14cf31a746c7be Mon Sep 17 00:00:00 2001
+From: Siddhesh Poyarekar
+Date: Mon, 11 Sep 2023 18:53:15 -0400
+Subject: [PATCH v2] tunables: Terminate immediately if end of input is reached
+
+The string parsing routine may end up writing beyond bounds of tunestr
+if the input tunable string is malformed, of the form name=name=val.
+This gets processed twice, first as name=name=val and next as name=val,
+resulting in tunestr being name=name=val:name=val, thus overflowing
+tunestr.
+
+Terminate the parsing loop at the first instance itself so that tunestr
+does not overflow.
+---
+Changes from v1:
+
+- Also null-terminate tunestr before exiting.
+
+ elf/dl-tunables.c | 17 ++---
+ 1 file changed, 10 insertions(+), 7 deletions(-)
+
+Upstream-Status: Backport [git://sourceware.org/git/glibc.git]
+CVE: CVE-2023-4911
+
+diff --git a/elf/dl-tunables.c b/elf/dl-tunables.c
+index 8e7ee9df10..76cf8b9da3 100644
+--- a/elf/dl-tunables.c
b/elf/dl-tunables.c
+@@ -187,11 +187,7 @@ parse_tunables (char *tunestr, char *valstring)
+ /* If we reach the end of the string before getting a valid name-value
+pair, bail out. */
+ if (p[len] == '\0')
+- {
+-if (__libc_enable_secure)
+- tunestr[off] = '\0';
+-return;
+- }
++ break;
+
+ /* We did not find a valid name-value pair before encountering the
+colon. */
+@@ -251,9 +247,16 @@ parse_tunables (char *tunestr, char *valstring)
+ }
+ }
+
+- if (p[len] != '\0')
+- p += len + 1;
++ /* We reached the end while processing the tunable string. */
++ if (p[len] == '\0')
++ break;
++
++ p += len + 1;
+ }
++
++ /* Terminate tunestr before we leave. */
++ if (__libc_enable_secure)
++tunestr[off] = '\0';
+ }
+ #endif
+
+--
+2.41.0
+
diff --git a/meta/recipes-core/glibc/glibc_2.31.bb
b/meta/recipes-core/glibc/glibc_2.31.bb
index 8d216f6ed1..1862586749 100644
--- a/meta/recipes-core/glibc/glibc_2.31.bb
+++ b/meta/recipes-core/glibc/glibc_2.31.bb
@@ -80,6 +80,7 @@ SRC_URI = "${GLIBC_GIT_URI};branch=${SRCBRANCH};name=glibc \
file://0036-i386-Avoid-lazy-relocation-of-tlsdesc-BZ-27137.patch \
file://0037-Avoid-deadlock-between-pthread_create-and-ctors.patch \
file://CVE-2023-0687.patch \
+ file://CVE-2023-4911.patch \
"
S = "${WORKDIR}/git"
B = "${WORKDIR}/build-${TARGET_SYS}"
--
2.39.2
BrightSign considers your privacy to be very important. The emails you send to
us will be protected and secured. Furthermore, we will only use your email and
contact information for the reasons you sent them to us and for tracking how
effectively we respond to your requests.
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#188708):
https://lists.openembedded.org/g/openembedded-core/message/188708
Mute This Topic: https://lists.openembedded.org/mt/101773057/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [dunfell][PATCH] glibc: Fix CVE-2023-4911 "Looney Tunables"
On Thursday 05 October 2023 at 04:17:08 -1000, Steve Sakoman wrote: > Hmmm ... does this build for you? Yes, on top of 0111b5b152c1bcff0ab26cf8632ca9002237f070 (current HEAD of openembedded-core dunfell branch.) > I'm getting: > > ERROR: glibc-2.31+gitAUTOINC+2d4f26e5cf-r0 do_patch: Applying patch > 'CVE-2023-4911.patch' on target directory > '/home/steve/builds/poky-contrib/build/tmp/work/core2-64-poky-linux/glibc/2.31+gitAUTOINC+2d4f26e5cf-r0/git' > Command Error: 'quilt --quiltrc > /home/steve/builds/poky-contrib/build/tmp/work/core2-64-poky-linux/glibc/2.31+gitAUTOINC+2d4f26e5cf-r0/recipe-sysroot-native/etc/quiltrc > push' exited with 0 Output: > Applying patch CVE-2023-4911.patch > patching file elf/dl-tunables.c > Hunk #1 FAILED at 187. > Hunk #2 FAILED at 251. > 2 out of 2 hunks FAILED -- rejects in file elf/dl-tunables.c > Patch CVE-2023-4911.patch does not apply (enforce with -f) Well that's strange. I can see from your work directory name that you're using the same glibc commit as me: 2d4f26e5cfda682f9ce61444b81533b83f6381af, and I can apply the CVE-2023-4911.patch to that commit without conflicts. Do you have any other changes beyond 0111b5b152c1bcff0ab26cf8632ca9002237f070 in your tree that might be applying other patches? Mike. -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#188727): https://lists.openembedded.org/g/openembedded-core/message/188727 Mute This Topic: https://lists.openembedded.org/mt/101773057/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [dunfell][PATCH] glibc: Fix CVE-2023-4911 "Looney Tunables"
On Thursday 05 October 2023 at 11:16:29 -0400, Scott Murray wrote: > Debian's page at https://security-tracker.debian.org/tracker/CVE-2023-4911 > indicates at the bottom that they're only vulnerable on their 2.31 based > versions because they backported the change that introduced the > vulnerability, which I don't believe has been done in oe-core... It has. The openembedded-core dunfell branch is using glibc 2d4f26e5cfda682f9ce61444b81533b83f6381af. This commit is a successor of 8e88c0d8885f68d22f47b22969c273004c6e719f, which is the backport of 2ed18c5b534d9e92fc006202a5af0df6b72e7aca (as mentioned in the Qualsys advisory) that introduced the vulnerability. Mike. -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#188728): https://lists.openembedded.org/g/openembedded-core/message/188728 Mute This Topic: https://lists.openembedded.org/mt/101773057/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [dunfell][PATCH] glibc: Fix CVE-2023-4911 "Looney Tunables"
On Thursday 05 October 2023 at 09:25:44 -1000, Steve Sakoman wrote: > Strange! Nothing that should affect glibc. I'm applying to the head > of stable/dunfell-nut: > > https://git.openembedded.org/openembedded-core-contrib/log/?h=stable/dunfell-nut All my fault. I managed to lose my mail-sending wrapper in some recent configuration setting cleanups and the patch went out via Microsoft's mail server which seems to have corrupted the whitespace in the patch. I'll post v2 shortly via a less broken mail server. Sorry for the confusion. Mike. -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#188736): https://lists.openembedded.org/g/openembedded-core/message/188736 Mute This Topic: https://lists.openembedded.org/mt/101773057/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [dunfell][PATCH v2] glibc: Fix CVE-2023-4911 "Looney Tunables"
From: Mike Crowe
Take the patch from the source for Debian's glibc 2.31-13+deb11u7
package, the changelog for which starts with:
glibc (2.31-13+deb11u7) bullseye-security; urgency=medium
* debian/patches/any/local-CVE-2023-4911.patch: Fix a buffer overflow in the
dynamic loader's processing of the GLIBC_TUNABLES environment variable
(CVE-2023-4911).
This addresses the "Looney Tunables" vulnerability described at
https://www.qualys.com/2023/10/03/cve-2023-4911/looney-tunables-local-privilege-escalation-glibc-ld-so.txt
Signed-off-by: Mike Crowe
---
.../glibc/glibc/CVE-2023-4911.patch | 63 +++
meta/recipes-core/glibc/glibc_2.31.bb | 1 +
2 files changed, 64 insertions(+)
create mode 100644 meta/recipes-core/glibc/glibc/CVE-2023-4911.patch
Hopefully sent by a route that doesn't corrupt the whitespace this time.
diff --git a/meta/recipes-core/glibc/glibc/CVE-2023-4911.patch
b/meta/recipes-core/glibc/glibc/CVE-2023-4911.patch
new file mode 100644
index 00..4d3146509a
--- /dev/null
+++ b/meta/recipes-core/glibc/glibc/CVE-2023-4911.patch
@@ -0,0 +1,63 @@
+From d2b77337f734fcacdfc8e0ddec14cf31a746c7be Mon Sep 17 00:00:00 2001
+From: Siddhesh Poyarekar
+Date: Mon, 11 Sep 2023 18:53:15 -0400
+Subject: [PATCH v2] tunables: Terminate immediately if end of input is reached
+
+The string parsing routine may end up writing beyond bounds of tunestr
+if the input tunable string is malformed, of the form name=name=val.
+This gets processed twice, first as name=name=val and next as name=val,
+resulting in tunestr being name=name=val:name=val, thus overflowing
+tunestr.
+
+Terminate the parsing loop at the first instance itself so that tunestr
+does not overflow.
+---
+Changes from v1:
+
+- Also null-terminate tunestr before exiting.
+
+ elf/dl-tunables.c | 17 ++---
+ 1 file changed, 10 insertions(+), 7 deletions(-)
+
+Upstream-Status: Backport [git://sourceware.org/git/glibc.git]
+CVE: CVE-2023-4911
+
+diff --git a/elf/dl-tunables.c b/elf/dl-tunables.c
+index 8e7ee9df10..76cf8b9da3 100644
+--- a/elf/dl-tunables.c
b/elf/dl-tunables.c
+@@ -187,11 +187,7 @@ parse_tunables (char *tunestr, char *valstring)
+ /* If we reach the end of the string before getting a valid name-value
+pair, bail out. */
+ if (p[len] == '\0')
+- {
+-if (__libc_enable_secure)
+- tunestr[off] = '\0';
+-return;
+- }
++ break;
+
+ /* We did not find a valid name-value pair before encountering the
+colon. */
+@@ -251,9 +247,16 @@ parse_tunables (char *tunestr, char *valstring)
+ }
+ }
+
+- if (p[len] != '\0')
+- p += len + 1;
++ /* We reached the end while processing the tunable string. */
++ if (p[len] == '\0')
++ break;
++
++ p += len + 1;
+ }
++
++ /* Terminate tunestr before we leave. */
++ if (__libc_enable_secure)
++tunestr[off] = '\0';
+ }
+ #endif
+
+--
+2.41.0
+
diff --git a/meta/recipes-core/glibc/glibc_2.31.bb
b/meta/recipes-core/glibc/glibc_2.31.bb
index 8d216f6ed1..1862586749 100644
--- a/meta/recipes-core/glibc/glibc_2.31.bb
+++ b/meta/recipes-core/glibc/glibc_2.31.bb
@@ -80,6 +80,7 @@ SRC_URI = "${GLIBC_GIT_URI};branch=${SRCBRANCH};name=glibc \
file://0036-i386-Avoid-lazy-relocation-of-tlsdesc-BZ-27137.patch \
file://0037-Avoid-deadlock-between-pthread_create-and-ctors.patch \
file://CVE-2023-0687.patch \
+ file://CVE-2023-4911.patch \
"
S = "${WORKDIR}/git"
B = "${WORKDIR}/build-${TARGET_SYS}"
--
2.39.2
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#188738):
https://lists.openembedded.org/g/openembedded-core/message/188738
Mute This Topic: https://lists.openembedded.org/mt/101785197/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [dunfell][PATCH 1/2] curl: Backport fix for CVE-2023-38545
From: Mike Crowe
Backporting this change required tweaking the error value since the
two-level CURLE_PROXY error reporting was introduced after curl
7.69.1. The test required some tweaks to not rely on more-recent
improvements to the test infrastructure too.
Signed-off-by: Mike Crowe
---
.../curl/curl/CVE-2023-38545.patch| 146 ++
meta/recipes-support/curl/curl_7.69.1.bb | 1 +
2 files changed, 147 insertions(+)
create mode 100644 meta/recipes-support/curl/curl/CVE-2023-38545.patch
diff --git a/meta/recipes-support/curl/curl/CVE-2023-38545.patch
b/meta/recipes-support/curl/curl/CVE-2023-38545.patch
new file mode 100644
index 00..40a92ce3fb
--- /dev/null
+++ b/meta/recipes-support/curl/curl/CVE-2023-38545.patch
@@ -0,0 +1,146 @@
+From 600a1caeb2312fdee5ef1caf7d613c12a8b2424a Mon Sep 17 00:00:00 2001
+From: Mike Crowe
+Date: Wed, 11 Oct 2023 20:50:28 +0100
+Subject: [PATCH] socks: return error if hostname too long for remote resolve
+To: libcurl development
+
+Prior to this change the state machine attempted to change the remote
+resolve to a local resolve if the hostname was longer than 255
+characters. Unfortunately that did not work as intended and caused a
+security issue.
+
+Name resolvers cannot resolve hostnames longer than 255 characters.
+
+Bug: https://curl.se/docs/CVE-2023-38545.html
+
+Unfortunately CURLE_PROXY and CURLPX_LONG_HOSTNAME were introduced in
+7.73.0 so they can't be used in 7.69.1. Let's use
+CURLE_COULDNT_RESOLVE_HOST as the best available alternative and update
+the test appropriately.
+
+libcurl's test support has been improved considerably since 7.69.1 which
+means that the test must be modified to remove use of %VERSION and
+%TESTNUMBER and the stderr output can no longer be checked.
+
+Upstream-Status: Backport [fb4415d8aee6c1045be932a34fe6107c2f5ed147]
+---
+ lib/socks.c | 13 +
+ tests/data/Makefile.inc | 2 +-
+ tests/data/test728 | 60 +
+ 3 files changed, 69 insertions(+), 6 deletions(-)
+ create mode 100644 tests/data/test728
+
+diff --git a/lib/socks.c b/lib/socks.c
+index 37099130e..f3bf40533 100644
+--- a/lib/socks.c
b/lib/socks.c
+@@ -521,11 +521,14 @@ CURLcode Curl_SOCKS5(const char *proxy_user,
+ infof(conn->data, "SOCKS5: connecting to HTTP proxy %s port %d\n",
+ hostname, remote_port);
+
+-/* RFC1928 chapter 5 specifies max 255 chars for domain name in packet */
++/* RFC1928 chapter 5 specifies max 255 chars for domain name in packet. */
+ if(!socks5_resolve_local && hostname_len > 255) {
+- infof(conn->data, "SOCKS5: server resolving disabled for hostnames of "
+-"length > 255 [actual len=%zu]\n", hostname_len);
+- socks5_resolve_local = TRUE;
++ failf(data, "SOCKS5: the destination hostname is too long to be "
++"resolved remotely by the proxy.");
++ /* This version of libcurl doesn't have CURLE_PROXY and
++ * therefore CURLPX_LONG_HOSTNAME, so let's report the best we
++ * can. */
++ return CURLE_COULDNT_RESOLVE_HOST;
+ }
+
+ if(auth & ~(CURLAUTH_BASIC | CURLAUTH_GSSAPI))
+@@ -837,7 +840,7 @@ CURLcode Curl_SOCKS5(const char *proxy_user,
+
+ if(!socks5_resolve_local) {
+ socksreq[len++] = 3; /* ATYP: domain name = 3 */
+- socksreq[len++] = (char) hostname_len; /* one byte address length */
++ socksreq[len++] = (unsigned char) hostname_len; /* one byte length */
+ memcpy(&socksreq[len], hostname, hostname_len); /* address w/o NULL */
+ len += hostname_len;
+ infof(data, "SOCKS5 connect to %s:%d (remotely resolved)\n",
+diff --git a/tests/data/Makefile.inc b/tests/data/Makefile.inc
+index 3d8565c36..5ee2284ff 100644
+--- a/tests/data/Makefile.inc
b/tests/data/Makefile.inc
+@@ -89,7 +89,7 @@ test662 test663 test664 test665 test666 test667 test668 \
+ test670 test671 test672 test673 \
+ \
+ test700 test701 test702 test703 test704 test705 test706 test707 test708 \
+-test709 test710 test711 test712 test713 test714 test715 test716 test717 \
++test709 test710 test711 test712 test713 test714 test715 test716 test717
test728 \
+ \
+ test800 test801 test802 test803 test804 test805 test806 test807 test808 \
+ test809 test810 test811 test812 test813 test814 test815 test816 test817 \
+diff --git a/tests/data/test728 b/tests/data/test728
+new file mode 100644
+index 0..7b1d8b2f3
+--- /dev/null
b/tests/data/test728
+@@ -0,0 +1,60 @@
++
++
++
++HTTP
++HTTP GET
++SOCKS5
++SOCKS5h
++followlocation
++
++
++
++#
++# Server-side
++
++# The hostname in this redirect is 256 characters and too long (> 255) for
++# SOCKS5 remote resolve. curl must return error CURLE_PROXY in this case.
++
++HTTP/1.1 301 Moved Permanently
++Location:
http://
[OE-core] [dunfell][PATCH 2/2] curl: Backport fix for CVE-2023-38546
From: Mike Crowe
Take patch from Debian 7.64.0-4+deb10u7.
Signed-off-by: Mike Crowe
---
.../curl/curl/CVE-2023-38546.patch| 131 ++
meta/recipes-support/curl/curl_7.69.1.bb | 1 +
2 files changed, 132 insertions(+)
create mode 100644 meta/recipes-support/curl/curl/CVE-2023-38546.patch
diff --git a/meta/recipes-support/curl/curl/CVE-2023-38546.patch
b/meta/recipes-support/curl/curl/CVE-2023-38546.patch
new file mode 100644
index 00..21887d58a3
--- /dev/null
+++ b/meta/recipes-support/curl/curl/CVE-2023-38546.patch
@@ -0,0 +1,131 @@
+From 7b67721f12cbe6ed1a41e7332f3b5a7186a5e23f Mon Sep 17 00:00:00 2001
+From: Daniel Stenberg
+Date: Thu, 14 Sep 2023 23:28:32 +0200
+Subject: [PATCH] cookie: remove unnecessary struct fields
+To: libcurl development
+
+Plus: reduce the hash table size from 256 to 63. It seems unlikely to
+make much of a speed difference for most use cases but saves 1.5KB of
+data per instance.
+
+Closes #11862
+
+This patch taken from Debian's 7.64.0-4+deb10u7 package which applied with
+only a little fuzz.
+
+CVE: CVE-2023-38546
+Upstream-Status: Backport [61275672b46d9abb32857404]
+---
+ lib/cookie.c | 13 +
+ lib/cookie.h | 7 ++-
+ lib/easy.c | 4 +---
+ 3 files changed, 4 insertions(+), 20 deletions(-)
+
+diff --git a/lib/cookie.c b/lib/cookie.c
+index 68054e1c4..a378f28e1 100644
+--- a/lib/cookie.c
b/lib/cookie.c
+@@ -114,7 +114,6 @@ static void freecookie(struct Cookie *co)
+ free(co->name);
+ free(co->value);
+ free(co->maxage);
+- free(co->version);
+ free(co);
+ }
+
+@@ -641,11 +640,7 @@ Curl_cookie_add(struct Curl_easy *data,
+ }
+ }
+ else if(strcasecompare("version", name)) {
+- strstore(&co->version, whatptr);
+- if(!co->version) {
+-badcookie = TRUE;
+-break;
+- }
++ /* just ignore */
+ }
+ else if(strcasecompare("max-age", name)) {
+ /* Defined in RFC2109:
+@@ -1042,7 +1037,6 @@ Curl_cookie_add(struct Curl_easy *data,
+ free(clist->path);
+ free(clist->spath);
+ free(clist->expirestr);
+-free(clist->version);
+ free(clist->maxage);
+
+ *clist = *co; /* then store all the new data */
+@@ -,9 +1105,6 @@ struct CookieInfo *Curl_cookie_init(struct Curl_easy
*data,
+ c = calloc(1, sizeof(struct CookieInfo));
+ if(!c)
+ return NULL; /* failed to get memory */
+-c->filename = strdup(file?file:"none"); /* copy the name just in case */
+-if(!c->filename)
+- goto fail; /* failed to get memory */
+ }
+ else {
+ /* we got an already existing one, use that */
+@@ -1241,7 +1232,6 @@ static struct Cookie *dup_cookie(struct Cookie *src)
+ CLONE(name);
+ CLONE(value);
+ CLONE(maxage);
+-CLONE(version);
+ d->expires = src->expires;
+ d->tailmatch = src->tailmatch;
+ d->secure = src->secure;
+@@ -1457,7 +1447,6 @@ void Curl_cookie_cleanup(struct CookieInfo *c)
+ {
+ if(c) {
+ unsigned int i;
+-free(c->filename);
+ for(i = 0; i < COOKIE_HASH_SIZE; i++)
+ Curl_cookie_freelist(c->cookies[i]);
+ free(c); /* free the base struct as well */
+diff --git a/lib/cookie.h b/lib/cookie.h
+index b3865e601..2e667cda0 100644
+--- a/lib/cookie.h
b/lib/cookie.h
+@@ -36,8 +36,6 @@ struct Cookie {
+ char *expirestr; /* the plain text version */
+ bool tailmatch;/* whether we do tail-matching of the domain name */
+
+- /* RFC 2109 keywords. Version=1 means 2109-compliant cookie sending */
+- char *version; /* Version = */
+ char *maxage; /* Max-Age = */
+
+ bool secure; /* whether the 'secure' keyword was used */
+@@ -54,15 +52,14 @@ struct Cookie {
+ #define COOKIE_PREFIX__SECURE (1<<0)
+ #define COOKIE_PREFIX__HOST (1<<1)
+
+-#define COOKIE_HASH_SIZE 256
++#define COOKIE_HASH_SIZE 63
+
+ struct CookieInfo {
+ /* linked list of cookies we know of */
+ struct Cookie *cookies[COOKIE_HASH_SIZE];
+
+- char *filename; /* file we read from/write to */
+ bool running;/* state info, for cookie adding information */
+- long numcookies; /* number of cookies in the "jar" */
++ int numcookies; /* number of cookies in the "jar" */
+ bool newsession; /* new session, discard session cookies on load */
+ int lastct; /* last creation-time used in the jar */
+ };
+diff --git a/lib/easy.c b/lib/easy.c
+index b648e80c1..cdca0fb03 100644
+--- a/lib/easy.c
b/lib/easy.c
+@@ -840,9 +840,7 @@ struct Curl_easy *curl_easy_duphandle(struct Curl_easy
*data)
+ if(data->cookies) {
+ /* If cookies are enabled in the parent handle, we enable them
+in the clone as well! */
+-outcurl->cookies = Curl_cookie_init(data,
+-data->cookies->filename,
+-outcurl->cookies,
++outcurl->cookies = Curl_cookie_init(data, NULL, outcurl->cookies,
+
[OE-core] [dunfell][PATCH v2 1/2] curl: Backport fix for CVE-2023-38545
From: Mike Crowe
Backporting this change required tweaking the error value since the
two-level CURLE_PROXY error reporting was introduced after curl
7.69.1. The test required some tweaks to not rely on more-recent
improvements to the test infrastructure too.
Signed-off-by: Mike Crowe
CVE: CVE-2023-38545
---
.../curl/curl/CVE-2023-38545.patch| 148 ++
meta/recipes-support/curl/curl_7.69.1.bb | 1 +
2 files changed, 149 insertions(+)
create mode 100644 meta/recipes-support/curl/curl/CVE-2023-38545.patch
diff --git a/meta/recipes-support/curl/curl/CVE-2023-38545.patch
b/meta/recipes-support/curl/curl/CVE-2023-38545.patch
new file mode 100644
index 00..4d952de046
--- /dev/null
+++ b/meta/recipes-support/curl/curl/CVE-2023-38545.patch
@@ -0,0 +1,148 @@
+From 600a1caeb2312fdee5ef1caf7d613c12a8b2424a Mon Sep 17 00:00:00 2001
+From: Mike Crowe
+Date: Wed, 11 Oct 2023 20:50:28 +0100
+Subject: [PATCH] socks: return error if hostname too long for remote resolve
+To: libcurl development
+
+Prior to this change the state machine attempted to change the remote
+resolve to a local resolve if the hostname was longer than 255
+characters. Unfortunately that did not work as intended and caused a
+security issue.
+
+Name resolvers cannot resolve hostnames longer than 255 characters.
+
+Bug: https://curl.se/docs/CVE-2023-38545.html
+
+Unfortunately CURLE_PROXY and CURLPX_LONG_HOSTNAME were introduced in
+7.73.0 so they can't be used in 7.69.1. Let's use
+CURLE_COULDNT_RESOLVE_HOST as the best available alternative and update
+the test appropriately.
+
+libcurl's test support has been improved considerably since 7.69.1 which
+means that the test must be modified to remove use of %VERSION and
+%TESTNUMBER and the stderr output can no longer be checked.
+
+CVE: CVE-2023-38545
+Upstream-Status: Backport [fb4415d8aee6c1045be932a34fe6107c2f5ed147]
+Signed-off-by: Mike Crowe
+---
+ lib/socks.c | 13 +
+ tests/data/Makefile.inc | 2 +-
+ tests/data/test728 | 60 +
+ 3 files changed, 69 insertions(+), 6 deletions(-)
+ create mode 100644 tests/data/test728
+
+diff --git a/lib/socks.c b/lib/socks.c
+index 37099130e..f3bf40533 100644
+--- a/lib/socks.c
b/lib/socks.c
+@@ -521,11 +521,14 @@ CURLcode Curl_SOCKS5(const char *proxy_user,
+ infof(conn->data, "SOCKS5: connecting to HTTP proxy %s port %d\n",
+ hostname, remote_port);
+
+-/* RFC1928 chapter 5 specifies max 255 chars for domain name in packet */
++/* RFC1928 chapter 5 specifies max 255 chars for domain name in packet. */
+ if(!socks5_resolve_local && hostname_len > 255) {
+- infof(conn->data, "SOCKS5: server resolving disabled for hostnames of "
+-"length > 255 [actual len=%zu]\n", hostname_len);
+- socks5_resolve_local = TRUE;
++ failf(data, "SOCKS5: the destination hostname is too long to be "
++"resolved remotely by the proxy.");
++ /* This version of libcurl doesn't have CURLE_PROXY and
++ * therefore CURLPX_LONG_HOSTNAME, so let's report the best we
++ * can. */
++ return CURLE_COULDNT_RESOLVE_HOST;
+ }
+
+ if(auth & ~(CURLAUTH_BASIC | CURLAUTH_GSSAPI))
+@@ -837,7 +840,7 @@ CURLcode Curl_SOCKS5(const char *proxy_user,
+
+ if(!socks5_resolve_local) {
+ socksreq[len++] = 3; /* ATYP: domain name = 3 */
+- socksreq[len++] = (char) hostname_len; /* one byte address length */
++ socksreq[len++] = (unsigned char) hostname_len; /* one byte length */
+ memcpy(&socksreq[len], hostname, hostname_len); /* address w/o NULL */
+ len += hostname_len;
+ infof(data, "SOCKS5 connect to %s:%d (remotely resolved)\n",
+diff --git a/tests/data/Makefile.inc b/tests/data/Makefile.inc
+index 3d8565c36..5ee2284ff 100644
+--- a/tests/data/Makefile.inc
b/tests/data/Makefile.inc
+@@ -89,7 +89,7 @@ test662 test663 test664 test665 test666 test667 test668 \
+ test670 test671 test672 test673 \
+ \
+ test700 test701 test702 test703 test704 test705 test706 test707 test708 \
+-test709 test710 test711 test712 test713 test714 test715 test716 test717 \
++test709 test710 test711 test712 test713 test714 test715 test716 test717
test728 \
+ \
+ test800 test801 test802 test803 test804 test805 test806 test807 test808 \
+ test809 test810 test811 test812 test813 test814 test815 test816 test817 \
+diff --git a/tests/data/test728 b/tests/data/test728
+new file mode 100644
+index 0..7b1d8b2f3
+--- /dev/null
b/tests/data/test728
+@@ -0,0 +1,60 @@
++
++
++
++HTTP
++HTTP GET
++SOCKS5
++SOCKS5h
++followlocation
++
++
++
++#
++# Server-side
++
++# The hostname in this redirect is 256 characters and too long (> 255) for
++# SOCKS5 remote resolve. curl must return error CURLE_PROXY in this case.
++
++HTTP/1.1 301 Moved Permanently
++Location:
http://AAA
[OE-core] [dunfell][PATCH v2 2/2] curl: Backport fix for CVE-2023-38546
From: Mike Crowe
Take patch from Debian 7.64.0-4+deb10u7.
Signed-off-by: Mike Crowe
CVE: CVE-2023-38546
---
.../curl/curl/CVE-2023-38546.patch| 132 ++
meta/recipes-support/curl/curl_7.69.1.bb | 1 +
2 files changed, 133 insertions(+)
create mode 100644 meta/recipes-support/curl/curl/CVE-2023-38546.patch
diff --git a/meta/recipes-support/curl/curl/CVE-2023-38546.patch
b/meta/recipes-support/curl/curl/CVE-2023-38546.patch
new file mode 100644
index 00..30ef2fd038
--- /dev/null
+++ b/meta/recipes-support/curl/curl/CVE-2023-38546.patch
@@ -0,0 +1,132 @@
+From 7b67721f12cbe6ed1a41e7332f3b5a7186a5e23f Mon Sep 17 00:00:00 2001
+From: Daniel Stenberg
+Date: Thu, 14 Sep 2023 23:28:32 +0200
+Subject: [PATCH] cookie: remove unnecessary struct fields
+To: libcurl development
+
+Plus: reduce the hash table size from 256 to 63. It seems unlikely to
+make much of a speed difference for most use cases but saves 1.5KB of
+data per instance.
+
+Closes #11862
+
+This patch taken from Debian's 7.64.0-4+deb10u7 package which applied with
+only a little fuzz.
+
+CVE: CVE-2023-38546
+Upstream-Status: Backport [61275672b46d9abb32857404]
+Signed-off-by: Mike Crowe
+---
+ lib/cookie.c | 13 +
+ lib/cookie.h | 7 ++-
+ lib/easy.c | 4 +---
+ 3 files changed, 4 insertions(+), 20 deletions(-)
+
+diff --git a/lib/cookie.c b/lib/cookie.c
+index 68054e1c4..a378f28e1 100644
+--- a/lib/cookie.c
b/lib/cookie.c
+@@ -114,7 +114,6 @@ static void freecookie(struct Cookie *co)
+ free(co->name);
+ free(co->value);
+ free(co->maxage);
+- free(co->version);
+ free(co);
+ }
+
+@@ -641,11 +640,7 @@ Curl_cookie_add(struct Curl_easy *data,
+ }
+ }
+ else if(strcasecompare("version", name)) {
+- strstore(&co->version, whatptr);
+- if(!co->version) {
+-badcookie = TRUE;
+-break;
+- }
++ /* just ignore */
+ }
+ else if(strcasecompare("max-age", name)) {
+ /* Defined in RFC2109:
+@@ -1042,7 +1037,6 @@ Curl_cookie_add(struct Curl_easy *data,
+ free(clist->path);
+ free(clist->spath);
+ free(clist->expirestr);
+-free(clist->version);
+ free(clist->maxage);
+
+ *clist = *co; /* then store all the new data */
+@@ -,9 +1105,6 @@ struct CookieInfo *Curl_cookie_init(struct Curl_easy
*data,
+ c = calloc(1, sizeof(struct CookieInfo));
+ if(!c)
+ return NULL; /* failed to get memory */
+-c->filename = strdup(file?file:"none"); /* copy the name just in case */
+-if(!c->filename)
+- goto fail; /* failed to get memory */
+ }
+ else {
+ /* we got an already existing one, use that */
+@@ -1241,7 +1232,6 @@ static struct Cookie *dup_cookie(struct Cookie *src)
+ CLONE(name);
+ CLONE(value);
+ CLONE(maxage);
+-CLONE(version);
+ d->expires = src->expires;
+ d->tailmatch = src->tailmatch;
+ d->secure = src->secure;
+@@ -1457,7 +1447,6 @@ void Curl_cookie_cleanup(struct CookieInfo *c)
+ {
+ if(c) {
+ unsigned int i;
+-free(c->filename);
+ for(i = 0; i < COOKIE_HASH_SIZE; i++)
+ Curl_cookie_freelist(c->cookies[i]);
+ free(c); /* free the base struct as well */
+diff --git a/lib/cookie.h b/lib/cookie.h
+index b3865e601..2e667cda0 100644
+--- a/lib/cookie.h
b/lib/cookie.h
+@@ -36,8 +36,6 @@ struct Cookie {
+ char *expirestr; /* the plain text version */
+ bool tailmatch;/* whether we do tail-matching of the domain name */
+
+- /* RFC 2109 keywords. Version=1 means 2109-compliant cookie sending */
+- char *version; /* Version = */
+ char *maxage; /* Max-Age = */
+
+ bool secure; /* whether the 'secure' keyword was used */
+@@ -54,15 +52,14 @@ struct Cookie {
+ #define COOKIE_PREFIX__SECURE (1<<0)
+ #define COOKIE_PREFIX__HOST (1<<1)
+
+-#define COOKIE_HASH_SIZE 256
++#define COOKIE_HASH_SIZE 63
+
+ struct CookieInfo {
+ /* linked list of cookies we know of */
+ struct Cookie *cookies[COOKIE_HASH_SIZE];
+
+- char *filename; /* file we read from/write to */
+ bool running;/* state info, for cookie adding information */
+- long numcookies; /* number of cookies in the "jar" */
++ int numcookies; /* number of cookies in the "jar" */
+ bool newsession; /* new session, discard session cookies on load */
+ int lastct; /* last creation-time used in the jar */
+ };
+diff --git a/lib/easy.c b/lib/easy.c
+index b648e80c1..cdca0fb03 100644
+--- a/lib/easy.c
b/lib/easy.c
+@@ -840,9 +840,7 @@ struct Curl_easy *curl_easy_duphandle(struct Curl_easy
*data)
+ if(data->cookies) {
+ /* If cookies are enabled in the parent handle, we enable them
+in the clone as well! */
+-outcurl->cookies = Curl_cookie_init(data,
+-data->cookies->filename,
+-outcurl->cookies,
++outcurl->cookies = Curl_cookie_ini
[OE-core] Using devtool without an absolute path to the workspace in bblayers.conf
I'm trying to work out how we can make use of devtool to make our lives
easier during development. In general it seems to work very well, but the
way that it modifies bblayers.conf to add an absolute path to the workspace
directory to BBLAYERS is incompatible with that file being held in a Git
repository and shared by multiple users in multiple trees. There's a high
risk that the file will accidentally be committed containing a path that's
only meaningful for a single user in a single source tree. All of our other
paths in bblayers.conf are relative to a variable that contains the path to
the top of the source tree.
I can manually add ${TOPDIR}/workspace to BBLAYERS in bblayers.conf and
commit that, but unfortunately devtool insists on continuing to add the
absolute path since it doesn't know that's the same as ${TOPDIR}/workspace.
I can set "workspace_path = workspace" in devtool.conf to use a relative
path, and that superficially works until externalsrc.bbclass gets upset
that the EXTERNALSRC is not an absolute path.
I've tried teaching devtool to prepend ${TOPDIR} if the workspace directory
is not absolute when writing bblayers.conf, but it looks like I'd also need
to do so in many other places.
My current workaround is just to add ${TOPDIR}/workspace to the committed
bblayers.conf myself and nobble devtool's _enable_workspace_layer with an
early return. This is ugly and we'd have to carry that change around
forever.
Since I'm clearly swimming against the tide I'm left wondering whether I've
missed something. Is there a way to use devtool without having an absolute
path to the workspace in bblayers.conf?
(At the moment I'm still using dunfell, but I looked at current master and
didn't spot anything that looked like it changed this behaviour.)
Thanks.
Mike.
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189396):
https://lists.openembedded.org/g/openembedded-core/message/189396
Mute This Topic: https://lists.openembedded.org/mt/102039990/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] Using devtool without an absolute path to the workspace in bblayers.conf
On Wednesday 18 October 2023 at 09:19:32 -0700, Khem Raj wrote:
> On Wed, Oct 18, 2023 at 7:29 AM Mike Crowe via lists.openembedded.org
> wrote:
> >
> > I'm trying to work out how we can make use of devtool to make our lives
> > easier during development. In general it seems to work very well, but the
> > way that it modifies bblayers.conf to add an absolute path to the workspace
> > directory to BBLAYERS is incompatible with that file being held in a Git
> > repository and shared by multiple users in multiple trees. There's a high
> > risk that the file will accidentally be committed containing a path that's
> > only meaningful for a single user in a single source tree. All of our other
> > paths in bblayers.conf are relative to a variable that contains the path to
> > the top of the source tree.
> >
> > I can manually add ${TOPDIR}/workspace to BBLAYERS in bblayers.conf and
> > commit that, but unfortunately devtool insists on continuing to add the
> > absolute path since it doesn't know that's the same as ${TOPDIR}/workspace.
> >
> > I can set "workspace_path = workspace" in devtool.conf to use a relative
> > path, and that superficially works until externalsrc.bbclass gets upset
> > that the EXTERNALSRC is not an absolute path.
> >
> > I've tried teaching devtool to prepend ${TOPDIR} if the workspace directory
> > is not absolute when writing bblayers.conf, but it looks like I'd also need
> > to do so in many other places.
> >
> > My current workaround is just to add ${TOPDIR}/workspace to the committed
> > bblayers.conf myself and nobble devtool's _enable_workspace_layer with an
> > early return. This is ugly and we'd have to carry that change around
> > forever.
> >
> > Since I'm clearly swimming against the tide I'm left wondering whether I've
> > missed something. Is there a way to use devtool without having an absolute
> > path to the workspace in bblayers.conf?
> >
> > (At the moment I'm still using dunfell, but I looked at current master and
> > didn't spot anything that looked like it changed this behaviour.)
>
> Perhaps an enhancement to not add the path if its already part of BBLAYERS
> might do it. You might also print a diagnostics about its pre-existence to
> make
> sure it's not accidental.
devtool already tries to do that. It doesn't work in this case since the
code compares the absolute path against the paths in bblayers.conf and
isn't in a position to expand variables at that point.
Thanks.
Mike.
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#189412):
https://lists.openembedded.org/g/openembedded-core/message/189412
Mute This Topic: https://lists.openembedded.org/mt/102039990/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] Using devtool without an absolute path to the workspace in bblayers.conf
On Wednesday 18 October 2023 at 17:47:50 +0200, Julien Stephan wrote: > Le mer. 18 oct. 2023 à 16:29, Mike Crowe via lists.openembedded.org > a écrit : > > > > I'm trying to work out how we can make use of devtool to make our lives > > easier during development. In general it seems to work very well, but the > > way that it modifies bblayers.conf to add an absolute path to the workspace > > directory to BBLAYERS is incompatible with that file being held in a Git > > repository and shared by multiple users in multiple trees. There's a high > > risk that the file will accidentally be committed containing a path that's > > only meaningful for a single user in a single source tree. All of our other > > paths in bblayers.conf are relative to a variable that contains the path to > > the top of the source tree. > > Hi Mike, > > I think it is a bad idea to version files under the build directory. Hi Julien, We don't version all files under the build directory, just ones that control the configuration that we want to share. We've been using OE that way for about twelve years now and this is the first time we've run into a problem. That might at least partly be because we haven't taken advantage of newer features like devtool though! We mostly avoided devtool by putting the sources that we modify heavily in submodules and used externalsrc.bbclass (with some local progressively-less-hacky modifications that I should try to upstream sometime.) > Maybe you have a specific use case that you can address in a different way? > You need to share bblayers.conf, I can see 2 options here: > - either you want your co-workers to fetch it only once, when setting > up their source tree > - or you need to frequently modify bblayer.conf because you are adding > or removing layers and want your coworkers to be up-to date > > In the first case, TEMPLATECONF is the way to go! More information > here; > https://docs.yoctoproject.org/ref-manual/variables.html#term-TEMPLATECONF We do need to modify bblayers.conf from time to time to add and remove layers. Using templates might be possible, but it would appear that this would force developers to manually incorporate changes (or just wipe their bblayers.conf file) when the template changes. Just keeping bblayers.conf under version control doesn't suffer from that problem. > In the second scenario, if this is something you *really* need, maybe > you can switch to some other tools such as 'kas' > (https://kas.readthedocs.io/en/latest/) that should be useful here. I have looked at kas in the past (and listened to podcasts about it), but it seemed to solve a different set of problems. I will look at it again. Thanks for the suggestions. Mike -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#189452): https://lists.openembedded.org/g/openembedded-core/message/189452 Mute This Topic: https://lists.openembedded.org/mt/102039990/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] Using devtool without an absolute path to the workspace in bblayers.conf
On Thursday 19 October 2023 at 14:51:56 +0200, Alexander Kanavin wrote: > On Thu, 19 Oct 2023 at 14:33, Mike Crowe via lists.openembedded.org > wrote: > > We do need to modify bblayers.conf from time to time to add and remove > > layers. > > > > Using templates might be possible, but it would appear that this would > > force developers to manually incorporate changes (or just wipe their > > bblayers.conf file) when the template changes. Just keeping bblayers.conf > > under version control doesn't suffer from that problem. > > Generally content of build/conf/, or anything else in build/ is not > designed for being under version control. You may be able to get away > with it by coincidence, but it's neither documented nor tested, and so > any issues are your own. > > I don't have a quick solution to ensure that templates and > configurations coming from them are strictly in sync, but welcome to > suggestions. People modify local.conf and bblayers.conf all the time > as well for local experiments, you're simply not supposed to lock them > down. There's no problem with modifying them for local experiments, even if they are under version control, but it's clear when those experiments need to be committed that those files need to be committed too. However, it's clear that I'm pushing against the tide. I'll stick with the early-return hack in devtool for the short term to see if we come across any other reasons to make changes or any other breakage. Once everything is working I'll investigate our options for not keeping bblayers.conf under version control. Thanks for you help. Mike. -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#189466): https://lists.openembedded.org/g/openembedded-core/message/189466 Mute This Topic: https://lists.openembedded.org/mt/102039990/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] rm_work causes image tasks to re-run
When we have rm_work enabled all image tasks are built every time. This has been happening since at least warrior and is still happening as of master today (8fc19639f47b959a141dae231395bbababa644e1). Steps to reproduce: bitbake core-image-minimal bitbake core-image-minimal echo 'INHERIT += "rm_work"' >> conf/local.conf bitbake core-image-minimal bitbake core-image-minimal The second invocation of bitbake doesn't build anything at all (as would be expected.) The fourth invocation of bitbake always builds: Sstate summary: Wanted 43 Found 43 Missed 0 Current 436 (100% match, 100% complete) NOTE: Executing Tasks NOTE: Setscene tasks completed NOTE: Running noexec task 1750 of 2000 (/fast2/mac/git/oe-core/meta/recipes-core/images/core-image-minimal.bb:do_rm_work_all) NOTE: Running task 1998 of 2000 (/fast2/mac/git/oe-core/meta/recipes-core/images/core-image-minimal.bb:do_populate_lic_deploy) NOTE: recipe core-image-minimal-1.0-r0: task do_populate_lic_deploy: Started NOTE: recipe core-image-minimal-1.0-r0: task do_populate_lic_deploy: Succeeded NOTE: Running task 1999 of 2000 (/fast2/mac/git/oe-core/meta/recipes-core/images/core-image-minimal.bb:do_rm_work) NOTE: recipe core-image-minimal-1.0-r0: task do_rm_work: Started NOTE: recipe core-image-minimal-1.0-r0: task do_rm_work: Succeeded NOTE: Running noexec task 2000 of 2000 (/fast2/mac/git/oe-core/meta/recipes-core/images/core-image-minimal.bb:do_build) NOTE: Tasks Summary: Attempted 2000 tasks of which 1996 didn't need to be rerun and all succeeded. If I teach rm_work.bbclass to also skip files matching "*do_populate_lic_deploy*" then this problem goes away. However, I have form for breaking things in rm_work so I suspect that this isn't the correct fix. Is there a better one? Thanks. Mike. -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#138581): https://lists.openembedded.org/g/openembedded-core/message/138581 Mute This Topic: https://lists.openembedded.org/mt/74394858/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] rm_work causes image tasks to re-run
On Friday 22 May 2020 at 19:02:47 +0200, Jacob Kroon wrote: > Hi Mike, > > On 5/22/20 10:13 AM, Mike Crowe via lists.openembedded.org wrote: > > When we have rm_work enabled all image tasks are built every time. This has > > been happening since at least warrior and is still happening as of master > > today (8fc19639f47b959a141dae231395bbababa644e1). > > > > Steps to reproduce: > > > > bitbake core-image-minimal > > bitbake core-image-minimal > > echo 'INHERIT += "rm_work"' >> conf/local.conf > > bitbake core-image-minimal > > bitbake core-image-minimal > > > > The second invocation of bitbake doesn't build anything at all (as would be > > expected.) > > > > The fourth invocation of bitbake always builds: > > > > Sstate summary: Wanted 43 Found 43 Missed 0 Current 436 (100% match, 100% > > complete) > > NOTE: Executing Tasks > > NOTE: Setscene tasks completed > > NOTE: Running noexec task 1750 of 2000 > > (/fast2/mac/git/oe-core/meta/recipes-core/images/core-image-minimal.bb:do_rm_work_all) > > NOTE: Running task 1998 of 2000 > > (/fast2/mac/git/oe-core/meta/recipes-core/images/core-image-minimal.bb:do_populate_lic_deploy) > > NOTE: recipe core-image-minimal-1.0-r0: task do_populate_lic_deploy: > > Started > > NOTE: recipe core-image-minimal-1.0-r0: task do_populate_lic_deploy: > > Succeeded > > NOTE: Running task 1999 of 2000 > > (/fast2/mac/git/oe-core/meta/recipes-core/images/core-image-minimal.bb:do_rm_work) > > NOTE: recipe core-image-minimal-1.0-r0: task do_rm_work: Started > > NOTE: recipe core-image-minimal-1.0-r0: task do_rm_work: Succeeded > > NOTE: Running noexec task 2000 of 2000 > > (/fast2/mac/git/oe-core/meta/recipes-core/images/core-image-minimal.bb:do_build) > > NOTE: Tasks Summary: Attempted 2000 tasks of which 1996 didn't need to be > > rerun and all succeeded. > > > > If I teach rm_work.bbclass to also skip files matching > > "*do_populate_lic_deploy*" then this problem goes away. However, I have > > form for breaking things in rm_work so I suspect that this isn't the > > correct fix. Is there a better one? > > > > Thanks. > > > > Mike. > > > > Since I'm the last guy touching rm_work.bbclass ... > > I don't see the rebuilds in my images. Although I do see > > core-image-minimal.bb:do_populate_lic_deploy > > being run, there is no regeneration of the rootfs images. I was originally investigating on warrior where the do_rootfs task does run unnecessarily. It turns out that if I cherry-pick the three subsequent rm_work changes from master back to warrior then it behaves the same as master. I shall do that in our tree. Is it possible to avoid the do_populate_lic_deploy task running unnecessarily on master too? Thanks. Mike. -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#138609): https://lists.openembedded.org/g/openembedded-core/message/138609 Mute This Topic: https://lists.openembedded.org/mt/74394858/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH] npm.bbclass: Allow nodedir to be overridden by NPM_NODEDIR
Node modules may need to be built against multiple Node
versions. Setting nodedir in the NPM configuration stops older ways of
doing this, such as setting npm_config_target and npm_config_disturl,
from working.
Signed-off-by: Mike Crowe
---
meta/classes/npm.bbclass | 6 --
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/meta/classes/npm.bbclass b/meta/classes/npm.bbclass
index 55a6985fb0..8f8712a024 100644
--- a/meta/classes/npm.bbclass
+++ b/meta/classes/npm.bbclass
@@ -247,8 +247,10 @@ python npm_do_compile() {
# Add node-gyp configuration
configs.append(("arch", d.getVar("NPM_ARCH")))
configs.append(("release", "true"))
-sysroot = d.getVar("RECIPE_SYSROOT_NATIVE")
-nodedir = os.path.join(sysroot, d.getVar("prefix_native").strip("/"))
+nodedir = d.getVar("NPM_NODEDIR")
+if not nodedir:
+sysroot = d.getVar("RECIPE_SYSROOT_NATIVE")
+nodedir = os.path.join(sysroot,
d.getVar("prefix_native").strip("/"))
configs.append(("nodedir", nodedir))
configs.append(("python", d.getVar("PYTHON")))
--
2.30.2
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#151704):
https://lists.openembedded.org/g/openembedded-core/message/151704
Mute This Topic: https://lists.openembedded.org/mt/82800205/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH] libnotify: Make gtk+3 dependency optional
libnotify only requires gtk+3 for its tests. Let's disable them by
default.
Signed-off-by: Mike Crowe
---
meta/recipes-gnome/libnotify/libnotify_0.7.9.bb | 7 ---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/meta/recipes-gnome/libnotify/libnotify_0.7.9.bb
b/meta/recipes-gnome/libnotify/libnotify_0.7.9.bb
index bbbd72193e..819cb304f7 100644
--- a/meta/recipes-gnome/libnotify/libnotify_0.7.9.bb
+++ b/meta/recipes-gnome/libnotify/libnotify_0.7.9.bb
@@ -9,15 +9,16 @@ SECTION = "libs"
LICENSE = "LGPLv2.1"
LIC_FILES_CHKSUM = "file://COPYING;md5=7fbc338309ac38fefcd64b04bb903e34"
-DEPENDS = "dbus gtk+3 glib-2.0"
+DEPENDS = "dbus glib-2.0 gdk-pixbuf"
+
+PACKAGECONFIG ?= ""
+PACKAGECONFIG[gtk+] = "--enable-tests,--disable-tests,,gtk+3"
GNOMEBASEBUILDCLASS = "meson"
GTKDOC_MESON_OPTION = "gtk_doc"
GIR_MESON_ENABLE_FLAG = "enabled"
GIR_MESON_DISABLE_FLAG = "disabled"
inherit gnomebase gtk-doc features_check gobject-introspection
-# depends on gtk+3
-ANY_OF_DISTRO_FEATURES = "${GTK3DISTROFEATURES}"
SRC_URI[archive.md5sum] = "ccd9c53364174cc8d13e18a1988faa76"
SRC_URI[archive.sha256sum] =
"66c0517ed16df7af258e83208faaf5069727dfd66995c4bbc51c16954d674761"
--
2.30.2
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#151706):
https://lists.openembedded.org/g/openembedded-core/message/151706
Mute This Topic: https://lists.openembedded.org/mt/82800551/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH] libnotify: Make gtk+3 dependency optional
On Thursday 13 May 2021 at 16:43:15 +0200, Alexander Kanavin wrote: > On Thu, 13 May 2021 at 16:36, Mike Crowe via lists.openembedded.org mac.mcrowe@lists.openembedded.org> wrote: > > > +PACKAGECONFIG[gtk+] = "--enable-tests,--disable-tests,,gtk+3" > > > > Was the --enable-tests scenario tested? Specifying gtk+3 after two ,,s > seems weird. You're right. That does look odd. This is actually an old patch I hoovered up after upgrading and didn't retest it. I'll do some more testing... Mike. -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#151708): https://lists.openembedded.org/g/openembedded-core/message/151708 Mute This Topic: https://lists.openembedded.org/mt/82800551/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH] libnotify: Make gtk+3 dependency optional
On Thursday 13 May 2021 at 17:06:23 +0100, Richard Purdie wrote: > On Thu, 2021-05-13 at 16:43 +0200, Alexander Kanavin wrote: > > On Thu, 13 May 2021 at 16:36, Mike Crowe via lists.openembedded.org < > > yocto=mac.mcrowe@lists.openembedded.org> wrote: > > > +PACKAGECONFIG[gtk+] = "--enable-tests,--disable-tests,,gtk+3" > > > > > > > > > Was the --enable-tests scenario tested? Specifying gtk+3 after two ,,s > > seems weird. > > It means to add a runtime (RDEPENDS) rather than DEPENDS. There are > options for positions 5 and 6 too if I remember rightly. Yes, but it should have been a build dependency. :( Alexander was correct to draw attention to it. It turns out that the whole line is wrong because I failed to spot that master is now using Meson too. :( A better patch will be arriving shortly. Thanks. Mike. -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#151712): https://lists.openembedded.org/g/openembedded-core/message/151712 Mute This Topic: https://lists.openembedded.org/mt/82800551/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH v2] libnotify: Make gtk+3 dependency optional
libnotify only requires gtk+3 for its tests. Let's disable them by
default.
Signed-off-by: Mike Crowe
---
meta/recipes-gnome/libnotify/libnotify_0.7.9.bb | 7 ---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/meta/recipes-gnome/libnotify/libnotify_0.7.9.bb
b/meta/recipes-gnome/libnotify/libnotify_0.7.9.bb
index bbbd72193e..0dc3dc3a3c 100644
--- a/meta/recipes-gnome/libnotify/libnotify_0.7.9.bb
+++ b/meta/recipes-gnome/libnotify/libnotify_0.7.9.bb
@@ -9,15 +9,16 @@ SECTION = "libs"
LICENSE = "LGPLv2.1"
LIC_FILES_CHKSUM = "file://COPYING;md5=7fbc338309ac38fefcd64b04bb903e34"
-DEPENDS = "dbus gtk+3 glib-2.0"
+DEPENDS = "dbus glib-2.0 gdk-pixbuf"
+
+PACKAGECONFIG ?= ""
+PACKAGECONFIG[gtk+] = "-Dtests=true,-Dtests=false,gtk+3"
GNOMEBASEBUILDCLASS = "meson"
GTKDOC_MESON_OPTION = "gtk_doc"
GIR_MESON_ENABLE_FLAG = "enabled"
GIR_MESON_DISABLE_FLAG = "disabled"
inherit gnomebase gtk-doc features_check gobject-introspection
-# depends on gtk+3
-ANY_OF_DISTRO_FEATURES = "${GTK3DISTROFEATURES}"
SRC_URI[archive.md5sum] = "ccd9c53364174cc8d13e18a1988faa76"
SRC_URI[archive.sha256sum] =
"66c0517ed16df7af258e83208faaf5069727dfd66995c4bbc51c16954d674761"
--
2.30.2
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#151713):
https://lists.openembedded.org/g/openembedded-core/message/151713
Mute This Topic: https://lists.openembedded.org/mt/82803205/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH v3] libnotify: Make gtk+3 dependency optional
libnotify only requires gtk+3 for its tests. Let's disable them by
default and only enable them if "tests" is in PACKAGECONFIG. If gtk+3 is
not available then we need to declare the dependency on gdk-pixbuf
explicitly.
It looks like the tests genuinely do need some sort of desktop
environment to run, so let's maintain the ANY_OF_DISTRO_FEATURES check
added back in 3edf08b38b0af93cef0933b061349264dc86d54c.
Signed-off-by: Mike Crowe
Cc: Khem Raj
Cc: Alexander Kanavin
---
meta/recipes-gnome/libnotify/libnotify_0.7.9.bb | 9 ++---
1 file changed, 6 insertions(+), 3 deletions(-)
diff --git a/meta/recipes-gnome/libnotify/libnotify_0.7.9.bb
b/meta/recipes-gnome/libnotify/libnotify_0.7.9.bb
index bbbd72193e..d2be715ce5 100644
--- a/meta/recipes-gnome/libnotify/libnotify_0.7.9.bb
+++ b/meta/recipes-gnome/libnotify/libnotify_0.7.9.bb
@@ -9,15 +9,18 @@ SECTION = "libs"
LICENSE = "LGPLv2.1"
LIC_FILES_CHKSUM = "file://COPYING;md5=7fbc338309ac38fefcd64b04bb903e34"
-DEPENDS = "dbus gtk+3 glib-2.0"
+DEPENDS = "dbus glib-2.0 gdk-pixbuf"
+
+PACKAGECONFIG ?= ""
+PACKAGECONFIG[tests] = "-Dtests=true,-Dtests=false,gtk+3"
GNOMEBASEBUILDCLASS = "meson"
GTKDOC_MESON_OPTION = "gtk_doc"
GIR_MESON_ENABLE_FLAG = "enabled"
GIR_MESON_DISABLE_FLAG = "disabled"
inherit gnomebase gtk-doc features_check gobject-introspection
-# depends on gtk+3
-ANY_OF_DISTRO_FEATURES = "${GTK3DISTROFEATURES}"
+# depends on gtk+3 if tests are enabled
+ANY_OF_DISTRO_FEATURES = "${@bb.utils.contains('PACKAGECONFIG', 'tests',
'${GTK3DISTROFEATURES}', '', d)}"
SRC_URI[archive.md5sum] = "ccd9c53364174cc8d13e18a1988faa76"
SRC_URI[archive.sha256sum] =
"66c0517ed16df7af258e83208faaf5069727dfd66995c4bbc51c16954d674761"
--
2.30.2
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#151755):
https://lists.openembedded.org/g/openembedded-core/message/151755
Mute This Topic: https://lists.openembedded.org/mt/82821110/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH] externalsrc: Pass through npmsw URIs in SRC_URI
NPM shrinkwrap files need to stay in SRC_URI even when using
externalsrc so that npm_do_fetch can run to fetch the required
dependencies.
Signed-off-by: Mike Crowe
---
meta/classes/externalsrc.bbclass | 1 +
1 file changed, 1 insertion(+)
diff --git a/meta/classes/externalsrc.bbclass b/meta/classes/externalsrc.bbclass
index 64e94e3301..c7b2bf2f49 100644
--- a/meta/classes/externalsrc.bbclass
+++ b/meta/classes/externalsrc.bbclass
@@ -68,6 +68,7 @@ python () {
url_data = fetch.ud[url]
parm = url_data.parm
if (url_data.type == 'file' or
+url_data.type == 'npmsw' or
'type' in parm and parm['type'] == 'kmeta'):
local_srcuri.append(url)
--
2.20.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#148596):
https://lists.openembedded.org/g/openembedded-core/message/148596
Mute This Topic: https://lists.openembedded.org/mt/80899197/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH] gcc-sanitizers: Move content from gcclibdir into libdir
In e9e5744ba8b0d43c8b874d365f83071ce20bf0a1, Khem Raj wrote:
> OE does not use the traditional /usr/lib/gcc prefix to store
> gcc-runtime it basically is moved into libdir, however some newer
> files were installed by newer versions of gcc especially libgomp (
> omp.h openacc.h ) into gcclibdir, so we have content in both
> directories, this confuses other tools which are trying to guess the
> gcc installation and its runtime location, since now we have two
> directories, the tools either choose one or other and we get
> inconsistent behavior, e.g. clang for aarch64 uses /usr/lib but same
> clang for riscv64 chose /usr/lib/gcc
> This change ensures that OE ends up with single valid location for gcc
> runtime files
I think that the same thing needs to happen in gcc-sanitizers.inc,
otherwise I get errors like:
| .../recipe-sysroot/usr/include/gpg-error-64.h:884:11: fatal error:
sanitizer/lsan_interface.h: No such file or directory
when attempting to compile with sanitizers enabled.
Signed-off-by: Mike Crowe
Cc: Khem Raj
---
meta/recipes-devtools/gcc/gcc-sanitizers.inc | 5 +
1 file changed, 5 insertions(+)
diff --git a/meta/recipes-devtools/gcc/gcc-sanitizers.inc
b/meta/recipes-devtools/gcc/gcc-sanitizers.inc
index 668e14a59f..67b755edf8 100644
--- a/meta/recipes-devtools/gcc/gcc-sanitizers.inc
+++ b/meta/recipes-devtools/gcc/gcc-sanitizers.inc
@@ -35,6 +35,11 @@ do_compile () {
do_install () {
cd ${B}/${TARGET_SYS}/libsanitizer/
oe_runmake 'DESTDIR=${D}' MULTIBUILDTOP=${B}/${TARGET_SYS}/libsanitizer/
install
+if [ -d ${D}${libdir}/gcc/${TARGET_SYS}/${BINV}/include ]; then
+ install -d ${D}${libdir}/${TARGET_SYS}/${BINV}/include
+ mv ${D}${libdir}/gcc/${TARGET_SYS}/${BINV}/include/*
${D}${libdir}/${TARGET_SYS}/${BINV}/include
+ rmdir --ignore-fail-on-non-empty -p
${D}${libdir}/gcc/${TARGET_SYS}/${BINV}/include
+fi
if [ -d ${D}${infodir} ]; then
rmdir --ignore-fail-on-non-empty -p ${D}${infodir}
fi
--
2.20.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#148609):
https://lists.openembedded.org/g/openembedded-core/message/148609
Mute This Topic: https://lists.openembedded.org/mt/80907601/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH v2] gcc-sanitizers: Move content from gcclibdir into libdir
In e9e5744ba8b0d43c8b874d365f83071ce20bf0a1, Khem Raj wrote:
> OE does not use the traditional /usr/lib/gcc prefix to store
> gcc-runtime it basically is moved into libdir, however some newer
> files were installed by newer versions of gcc especially libgomp (
> omp.h openacc.h ) into gcclibdir, so we have content in both
> directories, this confuses other tools which are trying to guess the
> gcc installation and its runtime location, since now we have two
> directories, the tools either choose one or other and we get
> inconsistent behavior, e.g. clang for aarch64 uses /usr/lib but same
> clang for riscv64 chose /usr/lib/gcc
> This change ensures that OE ends up with single valid location for gcc
> runtime files
I think that the same thing needs to happen in gcc-sanitizers.inc,
otherwise I get errors like:
| .../recipe-sysroot/usr/include/gpg-error-64.h:884:11: fatal error:
sanitizer/lsan_interface.h: No such file or directory
when attempting to compile with sanitizers enabled.
FILES_${PN} needs updating to match too.
Signed-off-by: Mike Crowe
Cc: Khem Raj
Cc: Alexandre Belloni
---
meta/recipes-devtools/gcc/gcc-sanitizers.inc | 7 ++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/meta/recipes-devtools/gcc/gcc-sanitizers.inc
b/meta/recipes-devtools/gcc/gcc-sanitizers.inc
index 668e14a59f..9e643ee277 100644
--- a/meta/recipes-devtools/gcc/gcc-sanitizers.inc
+++ b/meta/recipes-devtools/gcc/gcc-sanitizers.inc
@@ -35,6 +35,11 @@ do_compile () {
do_install () {
cd ${B}/${TARGET_SYS}/libsanitizer/
oe_runmake 'DESTDIR=${D}' MULTIBUILDTOP=${B}/${TARGET_SYS}/libsanitizer/
install
+if [ -d ${D}${libdir}/gcc/${TARGET_SYS}/${BINV}/include ]; then
+ install -d ${D}${libdir}/${TARGET_SYS}/${BINV}/include
+ mv ${D}${libdir}/gcc/${TARGET_SYS}/${BINV}/include/*
${D}${libdir}/${TARGET_SYS}/${BINV}/include
+ rmdir --ignore-fail-on-non-empty -p
${D}${libdir}/gcc/${TARGET_SYS}/${BINV}/include
+fi
if [ -d ${D}${infodir} ]; then
rmdir --ignore-fail-on-non-empty -p ${D}${infodir}
fi
@@ -109,4 +114,4 @@ FILES_libtsan-dev += "\
"
FILES_libtsan-staticdev += "${libdir}/libtsan.a"
-FILES_${PN} = "${libdir}/*.spec
${libdir}/gcc/${TARGET_SYS}/${BINV}/include/sanitizer/*.h"
+FILES_${PN} = "${libdir}/*.spec
${libdir}/${TARGET_SYS}/${BINV}/include/sanitizer/*.h"
--
2.20.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#148735):
https://lists.openembedded.org/g/openembedded-core/message/148735
Mute This Topic: https://lists.openembedded.org/mt/80970998/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH] gcc-sanitizers: Move content from gcclibdir into libdir
On Saturday 27 February 2021 at 16:02:18 +0100, Alexandre Belloni wrote:
> Hello,
>
> On 25/02/2021 17:36:53+0000, Mike Crowe via lists.openembedded.org wrote:
> > In e9e5744ba8b0d43c8b874d365f83071ce20bf0a1, Khem Raj wrote:
> > > OE does not use the traditional /usr/lib/gcc prefix to store
> > > gcc-runtime it basically is moved into libdir, however some newer
> > > files were installed by newer versions of gcc especially libgomp (
> > > omp.h openacc.h ) into gcclibdir, so we have content in both
> > > directories, this confuses other tools which are trying to guess the
> > > gcc installation and its runtime location, since now we have two
> > > directories, the tools either choose one or other and we get
> > > inconsistent behavior, e.g. clang for aarch64 uses /usr/lib but same
> > > clang for riscv64 chose /usr/lib/gcc
> >
> > > This change ensures that OE ends up with single valid location for gcc
> > > runtime files
> >
> > I think that the same thing needs to happen in gcc-sanitizers.inc,
> > otherwise I get errors like:
> >
> > | .../recipe-sysroot/usr/include/gpg-error-64.h:884:11: fatal error:
> > sanitizer/lsan_interface.h: No such file or directory
> >
> > when attempting to compile with sanitizers enabled.
> >
> > Signed-off-by: Mike Crowe
> > Cc: Khem Raj
> > ---
> > meta/recipes-devtools/gcc/gcc-sanitizers.inc | 5 +
> > 1 file changed, 5 insertions(+)
> >
> > diff --git a/meta/recipes-devtools/gcc/gcc-sanitizers.inc
> > b/meta/recipes-devtools/gcc/gcc-sanitizers.inc
> > index 668e14a59f..67b755edf8 100644
> > --- a/meta/recipes-devtools/gcc/gcc-sanitizers.inc
> > +++ b/meta/recipes-devtools/gcc/gcc-sanitizers.inc
> > @@ -35,6 +35,11 @@ do_compile () {
> > do_install () {
> > cd ${B}/${TARGET_SYS}/libsanitizer/
> > oe_runmake 'DESTDIR=${D}'
> > MULTIBUILDTOP=${B}/${TARGET_SYS}/libsanitizer/ install
> > +if [ -d ${D}${libdir}/gcc/${TARGET_SYS}/${BINV}/include ]; then
> > + install -d ${D}${libdir}/${TARGET_SYS}/${BINV}/include
> > + mv ${D}${libdir}/gcc/${TARGET_SYS}/${BINV}/include/*
> > ${D}${libdir}/${TARGET_SYS}/${BINV}/include
> > + rmdir --ignore-fail-on-non-empty -p
> > ${D}${libdir}/gcc/${TARGET_SYS}/${BINV}/include
> > +fi
> > if [ -d ${D}${infodir} ]; then
> > rmdir --ignore-fail-on-non-empty -p ${D}${infodir}
> > fi
>
> This seems to result in the following erro on the autobuilders:
>
> ERROR: gcc-sanitizers-10.2.0-r0 do_package: QA Issue: gcc-sanitizers:
> Files/directories were installed but not shipped in any package:
> /usr/lib/i686-poky-linux
> /usr/lib/i686-poky-linux/10.2.0
> /usr/lib/i686-poky-linux/10.2.0/include
> /usr/lib/i686-poky-linux/10.2.0/include/sanitizer
> /usr/lib/i686-poky-linux/10.2.0/include/sanitizer/common_interface_defs.h
> /usr/lib/i686-poky-linux/10.2.0/include/sanitizer/lsan_interface.h
> /usr/lib/i686-poky-linux/10.2.0/include/sanitizer/asan_interface.h
> /usr/lib/i686-poky-linux/10.2.0/include/sanitizer/tsan_interface.h
> Please set FILES such that these items are packaged. Alternatively if they
> are unneeded, avoid installing them or delete them within do_install.
> gcc-sanitizers: 8 installed and not shipped files. [installed-vs-shipped]
> ERROR: gcc-sanitizers-10.2.0-r0 do_package: Fatal QA errors found, failing
> task.
>
> It seems to be an easy one to fix.
Removing the gcc/ part of the path added to FILES_${PN} seems to solve the
problem for me. I'm not sure why I didn't see that in the dunfell/usrmerge
tree that I originally wrote the fix for.
Updated patch at:
https://lists.openembedded.org/g/openembedded-core/topic/patch_v2_gcc_sanitizers/80970998?p=,,,20,0,0,0::recentpostdate%2Fsticky,,,20,2,0,80970998
Thanks.
Mike.
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#148736):
https://lists.openembedded.org/g/openembedded-core/message/148736
Mute This Topic: https://lists.openembedded.org/mt/80907601/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [dunfell][PATCH] curl: Patch CVE-2021-22876 & CVE-2021-22890
Take patches from Ubuntu 20.04 7.68.0-1ubuntu2.5, which is close enough
that they apply without conflicts.
Signed-off-by: Mike Crowe
---
.../curl/curl/CVE-2021-22876.patch| 59 +++
.../curl/curl/CVE-2021-22890.patch| 464 ++
meta/recipes-support/curl/curl_7.69.1.bb | 2 +
3 files changed, 525 insertions(+)
create mode 100644 meta/recipes-support/curl/curl/CVE-2021-22876.patch
create mode 100644 meta/recipes-support/curl/curl/CVE-2021-22890.patch
diff --git a/meta/recipes-support/curl/curl/CVE-2021-22876.patch
b/meta/recipes-support/curl/curl/CVE-2021-22876.patch
new file mode 100644
index 00..fc396aabef
--- /dev/null
+++ b/meta/recipes-support/curl/curl/CVE-2021-22876.patch
@@ -0,0 +1,59 @@
+transfer: strip credentials from the auto-referer header field
+
+CVE-2021-22876
+
+Patch taken from Ubuntu curl 7.68.0-1ubuntu2.5.
+
+Bug: https://curl.se/docs/CVE-2021-22876.html
+Upstream-Status: backport
+---
+ lib/transfer.c | 25 +++--
+ 1 file changed, 23 insertions(+), 2 deletions(-)
+
+diff --git a/lib/transfer.c b/lib/transfer.c
+index e76834eb3..744e1c00b 100644
+--- a/lib/transfer.c
b/lib/transfer.c
+@@ -1570,6 +1570,9 @@ CURLcode Curl_follow(struct Curl_easy *data,
+ data->set.followlocation++; /* count location-followers */
+
+ if(data->set.http_auto_referer) {
++CURLU *u;
++char *referer;
++
+ /* We are asked to automatically set the previous URL as the referer
+when we get the next URL. We pick the ->url field, which may or may
+not be 100% correct */
+@@ -1579,9 +1582,27 @@ CURLcode Curl_follow(struct Curl_easy *data,
+ data->change.referer_alloc = FALSE;
+ }
+
+-data->change.referer = strdup(data->change.url);
+-if(!data->change.referer)
++/* Make a copy of the URL without crenditals and fragment */
++u = curl_url();
++if(!u)
++ return CURLE_OUT_OF_MEMORY;
++
++uc = curl_url_set(u, CURLUPART_URL, data->change.url, 0);
++if(!uc)
++ uc = curl_url_set(u, CURLUPART_FRAGMENT, NULL, 0);
++if(!uc)
++ uc = curl_url_set(u, CURLUPART_USER, NULL, 0);
++if(!uc)
++ uc = curl_url_set(u, CURLUPART_PASSWORD, NULL, 0);
++if(!uc)
++ uc = curl_url_get(u, CURLUPART_URL, &referer, 0);
++
++curl_url_cleanup(u);
++
++if(uc || referer == NULL)
+ return CURLE_OUT_OF_MEMORY;
++
++data->change.referer = referer;
+ data->change.referer_alloc = TRUE; /* yes, free this later */
+ }
+ }
+--
+2.20.1
+
diff --git a/meta/recipes-support/curl/curl/CVE-2021-22890.patch
b/meta/recipes-support/curl/curl/CVE-2021-22890.patch
new file mode 100644
index 00..8c0ecbfe7f
--- /dev/null
+++ b/meta/recipes-support/curl/curl/CVE-2021-22890.patch
@@ -0,0 +1,464 @@
+vtls: add 'isproxy' argument to Curl_ssl_get/addsessionid()
+
+To make sure we set and extract the correct session.
+
+Patch taken from Ubuntu curl 7.68.0-1ubuntu2.5.
+
+CVE-2021-22890
+
+Reported-by: Mingtao Yang
+Bug: https://curl.se/docs/CVE-2021-22890.html
+Upstream-Status: backport
+---
+ lib/vtls/bearssl.c | 9 +---
+ lib/vtls/gtls.c | 9 +---
+ lib/vtls/mbedtls.c | 8 ---
+ lib/vtls/mesalink.c | 9 +---
+ lib/vtls/openssl.c | 52 ++--
+ lib/vtls/schannel.c | 10 +
+ lib/vtls/sectransp.c | 9
+ lib/vtls/vtls.c | 9 ++--
+ lib/vtls/vtls.h | 2 ++
+ lib/vtls/wolfssl.c | 8 ---
+ 10 files changed, 88 insertions(+), 37 deletions(-)
+
+diff --git a/lib/vtls/bearssl.c b/lib/vtls/bearssl.c
+index 67f945831..32cb0a4c2 100644
+--- a/lib/vtls/bearssl.c
b/lib/vtls/bearssl.c
+@@ -372,7 +372,8 @@ static CURLcode bearssl_connect_step1(struct connectdata
*conn, int sockindex)
+ void *session;
+
+ Curl_ssl_sessionid_lock(conn);
+-if(!Curl_ssl_getsessionid(conn, &session, NULL, sockindex)) {
++if(!Curl_ssl_getsessionid(conn, SSL_IS_PROXY() ? TRUE : FALSE,
++ &session, NULL, sockindex)) {
+ br_ssl_engine_set_session_parameters(&BACKEND->ctx.eng, session);
+ infof(data, "BearSSL: re-using session ID\n");
+ }
+@@ -560,10 +561,12 @@ static CURLcode bearssl_connect_step3(struct connectdata
*conn, int sockindex)
+ return CURLE_OUT_OF_MEMORY;
+ br_ssl_engine_get_session_parameters(&BACKEND->ctx.eng, session);
+ Curl_ssl_sessionid_lock(conn);
+-incache = !(Curl_ssl_getsessionid(conn, &oldsession, NULL, sockindex));
++incache = !(Curl_ssl_getsessionid(conn, SSL_IS_PROXY() ? TRUE : FALSE,
++ &oldsession, NULL, sockindex));
+ if(incache)
+ Curl_ssl_delsessionid(conn, oldsession);
+-ret = Curl_ssl_addsessionid(conn, session, 0, sockindex);
++ret = Curl_ssl_addsessionid(conn, SSL_IS_PROXY() ? TRUE : FALSE,
++
[OE-core] archiver.bbclass and fetch2/npmsw.py
We have a class that stores the files in ${DL_DIR} that are required for
each recipe so they can be made available in ${DL_DIR} later. This is a bit
like archiver.bbclass, but not quite the same.
I'm having trouble making our class work for recipes that use the post
refactor[1] npm.bbclass and the npmsw fetcher in Dunfell. The individual
dependency paths aren't returned by bb.fetch2.Fetch's localpaths method.
I hoped that I could see how archiver.bbclass solved this problem, so I
tried running do_ar_mirror for the recipe and got:
NOTE: Using original download: [...]/package.json
NOTE: Copying source mirror
NOTE: Archiving url: npmsw:///[...]/npm-shrinkwrap.json
NOTE: Using original download: /home/mac/src/oe/build/downloads/
NOTE: Copying source mirror
DEBUG: Python function do_ar_mirror finished
and the whole of my downloads directory ends up copied to
${WORKDIR}/archiver-sources . :(
It looks like archiver.bbclass doesn't know the dependency filenames (or
subdirectories) either because NpmShrinkWrap doesn't expose them via a
localpath method.
It didn't look like there was an easy fix for this since everyone else
expects the localpath method to return only one path. I'm hoping that I
might have missed something. Does anyone have any advice?
Thanks.
Mike.
[1] oe-core:fb2252ee0777c6d26dea94c7588c323a6b97e961
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#150390):
https://lists.openembedded.org/g/openembedded-core/message/150390
Mute This Topic: https://lists.openembedded.org/mt/82039737/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [dunfell][PATCH] curl: Patch CVE-2021-22876 & CVE-2021-22890
On Tuesday 06 April 2021 at 13:53:42 +0100, Mike Crowe via
lists.openembedded.org wrote:
> Take patches from Ubuntu 20.04 7.68.0-1ubuntu2.5, which is close enough
> that they apply without conflicts.
>
> Signed-off-by: Mike Crowe
> ---
> .../curl/curl/CVE-2021-22876.patch| 59 +++
> .../curl/curl/CVE-2021-22890.patch| 464 ++
> meta/recipes-support/curl/curl_7.69.1.bb | 2 +
> 3 files changed, 525 insertions(+)
> create mode 100644 meta/recipes-support/curl/curl/CVE-2021-22876.patch
> create mode 100644 meta/recipes-support/curl/curl/CVE-2021-22890.patch
>
> diff --git a/meta/recipes-support/curl/curl/CVE-2021-22876.patch
> b/meta/recipes-support/curl/curl/CVE-2021-22876.patch
> new file mode 100644
> index 00..fc396aabef
> --- /dev/null
> +++ b/meta/recipes-support/curl/curl/CVE-2021-22876.patch
> @@ -0,0 +1,59 @@
> +transfer: strip credentials from the auto-referer header field
> +
> +CVE-2021-22876
> +
> +Patch taken from Ubuntu curl 7.68.0-1ubuntu2.5.
> +
> +Bug: https://curl.se/docs/CVE-2021-22876.html
> +Upstream-Status: backport
> +---
> + lib/transfer.c | 25 +++--
> + 1 file changed, 23 insertions(+), 2 deletions(-)
> +
> +diff --git a/lib/transfer.c b/lib/transfer.c
> +index e76834eb3..744e1c00b 100644
> +--- a/lib/transfer.c
> b/lib/transfer.c
> +@@ -1570,6 +1570,9 @@ CURLcode Curl_follow(struct Curl_easy *data,
> + data->set.followlocation++; /* count location-followers */
> +
> + if(data->set.http_auto_referer) {
> ++CURLU *u;
> ++char *referer;
> ++
> + /* We are asked to automatically set the previous URL as the referer
> +when we get the next URL. We pick the ->url field, which may or
> may
> +not be 100% correct */
> +@@ -1579,9 +1582,27 @@ CURLcode Curl_follow(struct Curl_easy *data,
> + data->change.referer_alloc = FALSE;
> + }
> +
> +-data->change.referer = strdup(data->change.url);
> +-if(!data->change.referer)
> ++/* Make a copy of the URL without crenditals and fragment */
> ++u = curl_url();
> ++if(!u)
> ++ return CURLE_OUT_OF_MEMORY;
> ++
> ++uc = curl_url_set(u, CURLUPART_URL, data->change.url, 0);
> ++if(!uc)
> ++ uc = curl_url_set(u, CURLUPART_FRAGMENT, NULL, 0);
> ++if(!uc)
> ++ uc = curl_url_set(u, CURLUPART_USER, NULL, 0);
> ++if(!uc)
> ++ uc = curl_url_set(u, CURLUPART_PASSWORD, NULL, 0);
> ++if(!uc)
> ++ uc = curl_url_get(u, CURLUPART_URL, &referer, 0);
> ++
> ++curl_url_cleanup(u);
> ++
> ++if(uc || referer == NULL)
> + return CURLE_OUT_OF_MEMORY;
> ++
> ++data->change.referer = referer;
> + data->change.referer_alloc = TRUE; /* yes, free this later */
> + }
> + }
> +--
> +2.20.1
> +
> diff --git a/meta/recipes-support/curl/curl/CVE-2021-22890.patch
> b/meta/recipes-support/curl/curl/CVE-2021-22890.patch
> new file mode 100644
> index 00..8c0ecbfe7f
> --- /dev/null
> +++ b/meta/recipes-support/curl/curl/CVE-2021-22890.patch
> @@ -0,0 +1,464 @@
> +vtls: add 'isproxy' argument to Curl_ssl_get/addsessionid()
> +
> +To make sure we set and extract the correct session.
> +
> +Patch taken from Ubuntu curl 7.68.0-1ubuntu2.5.
> +
> +CVE-2021-22890
> +
> +Reported-by: Mingtao Yang
> +Bug: https://curl.se/docs/CVE-2021-22890.html
> +Upstream-Status: backport
> +---
> + lib/vtls/bearssl.c | 9 +---
> + lib/vtls/gtls.c | 9 +---
> + lib/vtls/mbedtls.c | 8 ---
> + lib/vtls/mesalink.c | 9 +---
> + lib/vtls/openssl.c | 52 ++--
> + lib/vtls/schannel.c | 10 +
> + lib/vtls/sectransp.c | 9
> + lib/vtls/vtls.c | 9 ++--
> + lib/vtls/vtls.h | 2 ++
> + lib/vtls/wolfssl.c | 8 ---
> + 10 files changed, 88 insertions(+), 37 deletions(-)
> +
> +diff --git a/lib/vtls/bearssl.c b/lib/vtls/bearssl.c
> +index 67f945831..32cb0a4c2 100644
> +--- a/lib/vtls/bearssl.c
> b/lib/vtls/bearssl.c
> +@@ -372,7 +372,8 @@ static CURLcode bearssl_connect_step1(struct connectdata
> *conn, int sockindex)
> + void *session;
> +
> + Curl_ssl_sessionid_lock(conn);
> +-if(!Curl_ssl_getsessionid(conn, &session, NULL, sockindex)) {
> ++if(!Curl_ssl_getsessionid(conn, SSL_IS_PROXY() ? TRUE : FALSE,
> ++ &session, NULL, sockindex)) {
> + br_ssl_engine_set_session_parameters(&BACK
[OE-core] [PATCH] [RFC] kernel: improve transformation from KERNEL_IMAGETYPE_FOR_MAKE
In 526bdd88ccd758204452579333ba188e29270bde the imageType loop in
kernel_do_deploy was changed to use KERNEL_IMAGETYPE_FOR_MAKE rather
than KERNEL_IMAGETYPES. This broke the special handling for fitImage
immediately below because KERNEL_IMAGETYPE_FOR_MAKE never contains
fitImage.
It has always been my understanding that KERNEL_IMAGETYPE_FOR_MAKE
controlled what was passed to make, but KERNEL_IMAGETYPE controlled what
was installed/deployed. When the two are different then it's the
responsibility of whoever set KERNEL_IMAGETYPE_FOR_MAKE to ensure that
whatever comes out of the kernel build system has been transformed in to
the requested form by the time of installation. This is what happens for
kernel.bbclass's own support for vmlinux.gz.
I think this means that for KERNEL_IMAGETYPE vmlinux.gz, kernel.bbclass
is responsible for generating vmlinux.gz.initramfs[1] so that
kernel_do_deploy can deploy it. This means that the change in
526bdd88ccd758204452579333ba188e29270bde can be reverted, fixing
KERNEL_IMAGETYPE = "fitImage".
In addition, it ought to be possible for recipes and other classes that
use kernel.bbclass to hook into this mechanism by setting
KERNEL_IMAGETYPE_FOR_MAKE and performing their own transformations.
do_bundle_initramfs calls kernel_do_compile and we don't want it to
transform vmlinux to vmlinux.gz at that point, since it will fight
against the careful renaming and preserving that do_bundle_initramfs
does. Let's separate the transformation out of kernel_do_compile to a
new do_transform_kernel task that can be run at the right time. This
means that it's also logical to perform the equivalent translation for
the kernel with the initramfs in a separate
do_transform_bundled_initramfs task too.
This leaves two clear customisation points for recipes and other classes
to hook into the process and perform their transformations:
do_transform_kernel and do_transform_bundled_initramfs.
(I care about this because our recipes that use kernel.bbclass also set
KERNEL_IMAGETYPE_FOR_MAKE and transform vmlinux into a form suitable for
our bootloader after do_compile and do_bundle_initramfs into the format
matching KERNEL_IMAGETYPE. I'm unable to successfully bundle an
initramfs after 526bdd88ccd758204452579333ba188e29270bde, but I didn't
want to just revert that change to reintroduce the bug that it was
fixing.)
I can't say that I'm entirely happy with this change, but I'm unsure
what to do to improve it. I find the way that both the bare kernel and
the one with the initramfs both get deployed to be confusing, and a
waste of build time. I would like to not actually generate a publishable
kernel image at all during do_compile when an initramfs is in use, but I
suspect that this would affect valid use cases that I'm not aware of.
Signed-off-by: Mike Crowe
[1] It could be argued that this should be vmlinux.initramfs.gz, but
that would require another special case in kernel_do_deploy and the
filename is only visible within this class and the recipes that use it
anyway.
---
meta/classes/kernel.bbclass | 21 ++---
1 file changed, 18 insertions(+), 3 deletions(-)
diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index 4acec1877e..a91d248801 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -77,7 +77,7 @@ python __anonymous () {
# KERNEL_IMAGETYPES may contain a mixture of image types supported directly
# by the kernel build system and types which are created by post-processing
# the output of the kernel build system (e.g. compressing vmlinux ->
-# vmlinux.gz in kernel_do_compile()).
+# vmlinux.gz in kernel_do_transform_kernel()).
# KERNEL_IMAGETYPE_FOR_MAKE should contain only image types supported
# directly by the kernel build system.
if not d.getVar('KERNEL_IMAGETYPE_FOR_MAKE'):
@@ -134,6 +134,8 @@ set -e
# standalone for use by wic and other tools.
if image:
d.appendVarFlag('do_bundle_initramfs', 'depends', '
${INITRAMFS_IMAGE}:do_image_complete')
+if image and bb.utils.to_boolean(d.getVar('INITRAMFS_IMAGE_BUNDLE')):
+bb.build.addtask('do_transform_bundled_initramfs', 'do_deploy',
'do_bundle_initramfs', d)
# NOTE: setting INITRAMFS_TASK is for backward compatibility
# The preferred method is to set INITRAMFS_IMAGE, because
@@ -316,6 +318,14 @@ do_bundle_initramfs () {
}
do_bundle_initramfs[dirs] = "${B}"
+kernel_do_transform_bundled_initramfs() {
+# vmlinux.gz is not built by kernel
+ if (echo "${KERNEL_IMAGETYPES}" | grep -wq "vmlinux\.gz"); then
+ gzip -9cn < ${KERNEL_OUTPUT_DIR}/vmlinux.initramfs >
${KERNEL_OUTPUT_DIR}/vmlinux.gz.initramfs
+fi
+}
+do_transform_bundled_initramfs[dirs] = "${B}"
+
python do_devshell:prepend () {
os.environ["LDFLAGS"] = ''
}
@@ -355,12 +365,17 @@ kernel_do_compile() {
for typeformake in ${KERNEL_IMAGETYPE_FOR_MAKE} ; do
oe_runmake ${typeformak
[OE-core] [PATCH] license: Allow treating missing license as error
Use mechanism inspired by insane.bbclass to allow individual recipes or
other configuration to determine whether a missing licence should be
treated as a warning (as it is now) or as an error. This is controlled
by whether the error class is in WARN_LICENSE or ERROR_LICENSE.
Use bb.fatal in the error case to ensure that the task really fails. If
only bb.error is used then do_populate_lic isn't re-run on subsequent
builds which could lead to the error being missed.
Signed-off-by: Mike Crowe
---
meta/classes/license.bbclass | 19 ++-
1 file changed, 18 insertions(+), 1 deletion(-)
diff --git a/meta/classes/license.bbclass b/meta/classes/license.bbclass
index 45d912741d..1805f18076 100644
--- a/meta/classes/license.bbclass
+++ b/meta/classes/license.bbclass
@@ -12,6 +12,23 @@ LICENSE_CREATE_PACKAGE ??= "0"
LICENSE_PACKAGE_SUFFIX ??= "-lic"
LICENSE_FILES_DIRECTORY ??= "${datadir}/licenses/"
+# Elect whether a given type of error is a warning or error, they may
+# have been set by other files.
+WARN_LICENSE ?= "no-license"
+ERROR_LICENSE ?= ""
+WARN_LICENSE[doc] = "Space-separated list of license problems that should be
reported only as warnings"
+ERROR_LICENSE[doc] = "Space-separated list of license problems that should be
reported as errors"
+
+def package_license_handle_error(error_class, error_msg, d):
+if error_class in (d.getVar("ERROR_LICENSE") or "").split():
+package_qa_write_error(error_class, error_msg, d)
+bb.fatal("License Issue: %s [%s]" % (error_msg, error_class))
+elif error_class in (d.getVar("WARN_LICENSE") or "").split():
+package_qa_write_error(error_class, error_msg, d)
+bb.warn("License Issue: %s [%s]" % (error_msg, error_class))
+else:
+bb.note("QA Issue: %s [%s]" % (error_msg, error_class))
+
addtask populate_lic after do_patch before do_build
do_populate_lic[dirs] = "${LICSSTATEDIR}/${PN}"
do_populate_lic[cleandirs] = "${LICSSTATEDIR}"
@@ -190,7 +207,7 @@ def find_license_files(d):
# Add explicity avoid of CLOSED license because this isn't generic
if license_type != 'CLOSED':
# And here is where we warn people that their licenses are
lousy
-bb.warn("%s: No generic license file exists for: %s in any
provider" % (pn, license_type))
+package_license_handle_error("no-license", "%s: No generic
license file exists for: %s in any provider" % (pn, license_type), d)
pass
if not generic_directory:
--
2.30.2
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#156773):
https://lists.openembedded.org/g/openembedded-core/message/156773
Mute This Topic: https://lists.openembedded.org/mt/86165990/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH v2] license: Allow treating missing license as error
Use mechanism inspired by insane.bbclass to allow individual recipes or
other configuration to determine whether a missing licence should be
treated as a warning (as it is now) or as an error. This is controlled
by whether the error class is in WARN_LICENSE or ERROR_LICENSE.
Use bb.fatal in the error case to ensure that the task really fails. If
only bb.error is used then do_populate_lic isn't re-run on subsequent
builds which could lead to the error being missed.
Signed-off-by: Mike Crowe
---
meta/classes/license.bbclass | 19 ++-
1 file changed, 18 insertions(+), 1 deletion(-)
diff --git a/meta/classes/license.bbclass b/meta/classes/license.bbclass
index 45d912741d..f0a6c0c20e 100644
--- a/meta/classes/license.bbclass
+++ b/meta/classes/license.bbclass
@@ -12,6 +12,23 @@ LICENSE_CREATE_PACKAGE ??= "0"
LICENSE_PACKAGE_SUFFIX ??= "-lic"
LICENSE_FILES_DIRECTORY ??= "${datadir}/licenses/"
+# Elect whether a given type of error is a warning or error, they may
+# have been set by other files.
+WARN_LICENSE ?= "no-license"
+ERROR_LICENSE ?= ""
+WARN_LICENSE[doc] = "Space-separated list of license problems that should be
reported only as warnings"
+ERROR_LICENSE[doc] = "Space-separated list of license problems that should be
reported as errors"
+
+def package_license_handle_error(error_class, error_msg, d):
+if error_class in (d.getVar("ERROR_LICENSE") or "").split():
+package_qa_write_error(error_class, error_msg, d)
+bb.fatal("License Issue: %s [%s]" % (error_msg, error_class))
+elif error_class in (d.getVar("WARN_LICENSE") or "").split():
+package_qa_write_error(error_class, error_msg, d)
+bb.warn("License Issue: %s [%s]" % (error_msg, error_class))
+else:
+bb.note("License Issue: %s [%s]" % (error_msg, error_class))
+
addtask populate_lic after do_patch before do_build
do_populate_lic[dirs] = "${LICSSTATEDIR}/${PN}"
do_populate_lic[cleandirs] = "${LICSSTATEDIR}"
@@ -190,7 +207,7 @@ def find_license_files(d):
# Add explicity avoid of CLOSED license because this isn't generic
if license_type != 'CLOSED':
# And here is where we warn people that their licenses are
lousy
-bb.warn("%s: No generic license file exists for: %s in any
provider" % (pn, license_type))
+package_license_handle_error("no-license", "%s: No generic
license file exists for: %s in any provider" % (pn, license_type), d)
pass
if not generic_directory:
--
2.30.2
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#156793):
https://lists.openembedded.org/g/openembedded-core/message/156793
Mute This Topic: https://lists.openembedded.org/mt/86218525/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH] license: Allow treating missing license as error
On Saturday 09 October 2021 at 15:51:35 +, Peter Kjellerstedt wrote:
> > -Original Message-
> > From: openembedded-core@lists.openembedded.org > c...@lists.openembedded.org> On Behalf Of Mike Crowe via
> > lists.openembedded.org
> > Sent: den 8 oktober 2021 10:54
> > To: openembedded-core@lists.openembedded.org
> > Cc: Mike Crowe
> > Subject: [OE-core] [PATCH] license: Allow treating missing license as
> > error
> >
> > Use mechanism inspired by insane.bbclass to allow individual recipes or
> > other configuration to determine whether a missing licence should be
> > treated as a warning (as it is now) or as an error. This is controlled
> > by whether the error class is in WARN_LICENSE or ERROR_LICENSE.
> >
> > Use bb.fatal in the error case to ensure that the task really fails. If
> > only bb.error is used then do_populate_lic isn't re-run on subsequent
> > builds which could lead to the error being missed.
> >
> > Signed-off-by: Mike Crowe
> > ---
> > meta/classes/license.bbclass | 19 ++-
> > 1 file changed, 18 insertions(+), 1 deletion(-)
> >
> > diff --git a/meta/classes/license.bbclass b/meta/classes/license.bbclass
> > index 45d912741d..1805f18076 100644
> > --- a/meta/classes/license.bbclass
> > +++ b/meta/classes/license.bbclass
> > @@ -12,6 +12,23 @@ LICENSE_CREATE_PACKAGE ??= "0"
> > LICENSE_PACKAGE_SUFFIX ??= "-lic"
> > LICENSE_FILES_DIRECTORY ??= "${datadir}/licenses/"
> >
> > +# Elect whether a given type of error is a warning or error, they may
> > +# have been set by other files.
> > +WARN_LICENSE ?= "no-license"
> > +ERROR_LICENSE ?= ""
> > +WARN_LICENSE[doc] = "Space-separated list of license problems that should
> > be reported only as warnings"
> > +ERROR_LICENSE[doc] = "Space-separated list of license problems that should
> > be reported as errors"
> > +
> > +def package_license_handle_error(error_class, error_msg, d):
> > +if error_class in (d.getVar("ERROR_LICENSE") or "").split():
> > +package_qa_write_error(error_class, error_msg, d)
> > +bb.fatal("License Issue: %s [%s]" % (error_msg, error_class))
> > +elif error_class in (d.getVar("WARN_LICENSE") or "").split():
> > +package_qa_write_error(error_class, error_msg, d)
> > +bb.warn("License Issue: %s [%s]" % (error_msg, error_class))
> > +else:
> > +bb.note("QA Issue: %s [%s]" % (error_msg, error_class))
>
> Change "QA Issue" to "License Issue" for consistency.
Whoops. Thanks for spotting that. I've posted v2 with it fixed.
Mike.
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#156792):
https://lists.openembedded.org/g/openembedded-core/message/156792
Mute This Topic: https://lists.openembedded.org/mt/86165990/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH v2] license: Allow treating missing license as error
On Tuesday 12 October 2021 at 14:21:05 +0100, Richard Purdie wrote:
> On Sun, 2021-10-10 at 18:20 +0100, Mike Crowe via lists.openembedded.org
> wrote:
> > Use mechanism inspired by insane.bbclass to allow individual recipes or
> > other configuration to determine whether a missing licence should be
> > treated as a warning (as it is now) or as an error. This is controlled
> > by whether the error class is in WARN_LICENSE or ERROR_LICENSE.
> >
> > Use bb.fatal in the error case to ensure that the task really fails. If
> > only bb.error is used then do_populate_lic isn't re-run on subsequent
> > builds which could lead to the error being missed.
> >
> > Signed-off-by: Mike Crowe
> > ---
> > meta/classes/license.bbclass | 19 ++-
> > 1 file changed, 18 insertions(+), 1 deletion(-)
> >
> > diff --git a/meta/classes/license.bbclass b/meta/classes/license.bbclass
> > index 45d912741d..f0a6c0c20e 100644
> > --- a/meta/classes/license.bbclass
> > +++ b/meta/classes/license.bbclass
> > @@ -12,6 +12,23 @@ LICENSE_CREATE_PACKAGE ??= "0"
> > LICENSE_PACKAGE_SUFFIX ??= "-lic"
> > LICENSE_FILES_DIRECTORY ??= "${datadir}/licenses/"
> >
> > +# Elect whether a given type of error is a warning or error, they may
> > +# have been set by other files.
> > +WARN_LICENSE ?= "no-license"
> > +ERROR_LICENSE ?= ""
> > +WARN_LICENSE[doc] = "Space-separated list of license problems that should
> > be reported only as warnings"
> > +ERROR_LICENSE[doc] = "Space-separated list of license problems that should
> > be reported as errors"
> > +
> > +def package_license_handle_error(error_class, error_msg, d):
> > +if error_class in (d.getVar("ERROR_LICENSE") or "").split():
> > +package_qa_write_error(error_class, error_msg, d)
> > +bb.fatal("License Issue: %s [%s]" % (error_msg, error_class))
> > +elif error_class in (d.getVar("WARN_LICENSE") or "").split():
> > +package_qa_write_error(error_class, error_msg, d)
> > +bb.warn("License Issue: %s [%s]" % (error_msg, error_class))
> > +else:
> > +bb.note("License Issue: %s [%s]" % (error_msg, error_class))
> > +
> > addtask populate_lic after do_patch before do_build
> > do_populate_lic[dirs] = "${LICSSTATEDIR}/${PN}"
> > do_populate_lic[cleandirs] = "${LICSSTATEDIR}"
> > @@ -190,7 +207,7 @@ def find_license_files(d):
> > # Add explicity avoid of CLOSED license because this isn't
> > generic
> > if license_type != 'CLOSED':
> > # And here is where we warn people that their licenses are
> > lousy
> > -bb.warn("%s: No generic license file exists for: %s in any
> > provider" % (pn, license_type))
> > +package_license_handle_error("no-license", "%s: No generic
> > license file exists for: %s in any provider" % (pn, license_type), d)
> > pass
> >
> > if not generic_directory:
>
> I'm a little torn on this and whether we should make it use the same variables
> as the other QA checks? Is there a reason the user would want to configure
> this
> sanity check separately from the other sanity checks?
I modelled this on the QA checks in insane.bbclass because that appeared to
be the most likely to be an acceptable way to do it. I'd be happy to use
the same variables, although that does raise the question of whether
license.bbclass needs to inherit from insane.bbclass or those variables
need moving somewhere else (see below).
> I'm not sure I can see a long list of different license checks we'd want to
> add
> here?
There are many other warnings reported by license.bbclass. Many of them
feel like errors to me. I'd be happy to have a single switch that converted
them all to errors, but I haven't tried to do so to see what problems we'd
run into.
> The current sanity checks in insane.bbclass could do with some cleanup and
> refactoring so perhaps this could be come a common function (and common
> variable
> to control all the QA checks)?
Where would be the best place to put this function? A new qa.bbclass that
can be inherited by both license.bbclass and insane.bbclass?
Did you have any particular cleanups and refactorings in mind? I must admit
that I was surprised by the long list in a single variable assigned with
?=. It means that anyone overriding the variable won't benefit from
newly-adde
[OE-core] [PATCH v3 2/2] license: Allow treating missing license as error
Use the same WARN_WA and ERROR_QA variables as insane.bbclass to allow
individual recipes, the distro or other configuration to determine
whether a missing licence should be treated as a warning (as it is now)
or as an error.
oe.qa.handle_error isn't immediately fatal, so track the overall sanity
state and call bb.fatal if required at the end to ensure that the task
really fails. If only bb.error is used then do_populate_lic isn't re-run
on subsequent builds which could lead to the error being missed.
It seems odd for the license- error classes to be listed in
insane.bbclass but implemented in license.bbclass. All recommendations
for somewhere else to put them gratefully received.
Signed-off-by: Mike Crowe
---
meta/classes/insane.bbclass | 1 +
meta/classes/license.bbclass | 27 ---
2 files changed, 21 insertions(+), 7 deletions(-)
diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass
index 895216d3e8..57456c99ad 100644
--- a/meta/classes/insane.bbclass
+++ b/meta/classes/insane.bbclass
@@ -28,6 +28,7 @@ WARN_QA ?= " libdir xorg-driver-abi \
invalid-packageconfig host-user-contaminated uppercase-pn
patch-fuzz \
mime mime-xdg unlisted-pkg-lics unhandled-features-check \
missing-update-alternatives native-last missing-ptest \
+license-exists license-no-generic license-syntax license-format \
"
ERROR_QA ?= "dev-so debug-deps dev-deps debug-files arch pkgconfig la \
perms dep-cmp pkgvarcheck perm-config perm-line perm-link \
diff --git a/meta/classes/license.bbclass b/meta/classes/license.bbclass
index 45d912741d..6bbb71392e 100644
--- a/meta/classes/license.bbclass
+++ b/meta/classes/license.bbclass
@@ -112,6 +112,7 @@ def find_license_files(d):
import oe.license
from collections import defaultdict, OrderedDict
+sane = True
# All the license files for the package
lic_files = d.getVar('LIC_FILES_CHKSUM') or ""
pn = d.getVar('PN')
@@ -146,6 +147,7 @@ def find_license_files(d):
self.generic_visit(node)
def find_license(license_type):
+import oe.qa
try:
bb.utils.mkdirhier(gen_lic_dest)
except:
@@ -178,7 +180,8 @@ def find_license_files(d):
# The user may attempt to use NO_GENERIC_LICENSE for a generic
license which doesn't make sense
# and should not be allowed, warn the user in this case.
if d.getVarFlag('NO_GENERIC_LICENSE', license_type):
-bb.warn("%s: %s is a generic license, please don't use
NO_GENERIC_LICENSE for it." % (pn, license_type))
+sane &= oe.qa.handle_error("license-no-generic",
+"%s: %s is a generic license, please don't use
NO_GENERIC_LICENSE for it." % (pn, license_type), d)
elif non_generic_lic and non_generic_lic in lic_chksums:
# if NO_GENERIC_LICENSE is set, we copy the license files from the
fetched source
@@ -190,7 +193,8 @@ def find_license_files(d):
# Add explicity avoid of CLOSED license because this isn't generic
if license_type != 'CLOSED':
# And here is where we warn people that their licenses are
lousy
-bb.warn("%s: No generic license file exists for: %s in any
provider" % (pn, license_type))
+sane &= oe.qa.handle_error("license-exists",
+"%s: No generic license file exists for: %s in any
provider" % (pn, license_type), d)
pass
if not generic_directory:
@@ -215,7 +219,8 @@ def find_license_files(d):
except oe.license.InvalidLicense as exc:
bb.fatal('%s: %s' % (d.getVar('PF'), exc))
except SyntaxError:
-bb.warn("%s: Failed to parse it's LICENSE field." % (d.getVar('PF')))
+sane &= oe.qa.handle_error("license-syntax",
+"%s: Failed to parse it's LICENSE field." % (d.getVar('PF')), d)
# Add files from LIC_FILES_CHKSUM to list of license files
lic_chksum_paths = defaultdict(OrderedDict)
for path, data in sorted(lic_chksums.items()):
@@ -233,6 +238,8 @@ def find_license_files(d):
for i, data in enumerate(files.values()):
lic_files_paths.append(tuple(["%s.%d" % (basename, i)] +
list(data)))
+if not sane:
+bb.fatal("Fatal QA errors found, failing task.")
return lic_files_paths
def return_spdx(d, license):
@@ -398,6 +405,8 @@ def check_license_format(d):
Validate operators in LICENSES.
No spaces are allowed between LICENSES.
"""
+import oe.qa
+sane = True
pn = d.getVar('PN')
licenses = d.getVar('LICENSE')
from oe.license import license_operator, license_operator_chars,
license_pattern
@@ -406,14 +415,18 @@ def check_license_format(d):
for pos, element in enumerate(elements):
if license_pattern.match(element):
if pos > 0 and license_pattern.match(elements[pos
[OE-core] [PATCH v3 1/2] lib/oe/qa,insane: Extra error handling functions to library
Extract package_qa_write_error, package_qa_handle_error and
package_qa_add_message functions from insane.bbclass to lib/oe/qa.py and
drop the package_qa_ prefixes. Add import statements and convert callers
to use oe.qa. prefix.
Inspired by discussion resulting from
https://lists.openembedded.org/g/openembedded-core/message/156793
Signed-off-by: Mike Crowe
---
meta/classes/insane.bbclass | 209 +++-
meta/lib/oe/qa.py | 26 +
2 files changed, 135 insertions(+), 100 deletions(-)
diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass
index f2d2ca3689..895216d3e8 100644
--- a/meta/classes/insane.bbclass
+++ b/meta/classes/insane.bbclass
@@ -59,35 +59,9 @@ def package_qa_clean_path(path, d, pkg=None):
path = path.replace(os.path.join(d.getVar("PKGDEST"), pkg), "/")
return path.replace(d.getVar("TMPDIR"), "/").replace("//", "/")
-def package_qa_write_error(type, error, d):
-logfile = d.getVar('QA_LOGFILE')
-if logfile:
-p = d.getVar('P')
-with open(logfile, "a+") as f:
-f.write("%s: %s [%s]\n" % (p, error, type))
-
-def package_qa_handle_error(error_class, error_msg, d):
-if error_class in (d.getVar("ERROR_QA") or "").split():
-package_qa_write_error(error_class, error_msg, d)
-bb.error("QA Issue: %s [%s]" % (error_msg, error_class))
-d.setVar("QA_SANE", False)
-return False
-elif error_class in (d.getVar("WARN_QA") or "").split():
-package_qa_write_error(error_class, error_msg, d)
-bb.warn("QA Issue: %s [%s]" % (error_msg, error_class))
-else:
-bb.note("QA Issue: %s [%s]" % (error_msg, error_class))
-return True
-
-def package_qa_add_message(messages, section, new_msg):
-if section not in messages:
-messages[section] = new_msg
-else:
-messages[section] = messages[section] + "\n" + new_msg
-
QAPATHTEST[shebang-size] = "package_qa_check_shebang_size"
def package_qa_check_shebang_size(path, name, d, elf, messages):
-import stat
+import stat, oe.qa
if os.path.islink(path) or stat.S_ISFIFO(os.stat(path).st_mode) or elf:
return
@@ -106,11 +80,12 @@ def package_qa_check_shebang_size(path, name, d, elf,
messages):
return
if len(stanza) > 129:
-package_qa_add_message(messages, "shebang-size", "%s: %s maximum
shebang size exceeded, the maximum size is 128." % (name,
package_qa_clean_path(path, d)))
+oe.qa.add_message(messages, "shebang-size", "%s: %s maximum
shebang size exceeded, the maximum size is 128." % (name,
package_qa_clean_path(path, d)))
return
QAPATHTEST[libexec] = "package_qa_check_libexec"
def package_qa_check_libexec(path,name, d, elf, messages):
+import oe.qa
# Skip the case where the default is explicitly /usr/libexec
libexec = d.getVar('libexecdir')
@@ -118,7 +93,7 @@ def package_qa_check_libexec(path,name, d, elf, messages):
return True
if 'libexec' in path.split(os.path.sep):
-package_qa_add_message(messages, "libexec", "%s: %s is using libexec
please relocate to %s" % (name, package_qa_clean_path(path, d), libexec))
+oe.qa.add_message(messages, "libexec", "%s: %s is using libexec please
relocate to %s" % (name, package_qa_clean_path(path, d), libexec))
return False
return True
@@ -138,7 +113,7 @@ def package_qa_check_rpath(file,name, d, elf, messages):
phdrs = elf.run_objdump("-p", d)
-import re
+import re, oe.qa
rpath_re = re.compile(r"\s+RPATH\s+(.*)")
for line in phdrs.split("\n"):
m = rpath_re.match(line)
@@ -146,7 +121,7 @@ def package_qa_check_rpath(file,name, d, elf, messages):
rpath = m.group(1)
for dir in bad_dirs:
if dir in rpath:
-package_qa_add_message(messages, "rpaths", "package %s
contains bad RPATH %s in file %s" % (name, rpath, file))
+oe.qa.add_message(messages, "rpaths", "package %s contains
bad RPATH %s in file %s" % (name, rpath, file))
QAPATHTEST[useless-rpaths] = "package_qa_check_useless_rpaths"
def package_qa_check_useless_rpaths(file, name, d, elf, messages):
@@ -167,7 +142,7 @@ def package_qa_check_useless_rpaths(file, name, d, elf,
messages):
phdrs = elf.run_objdump("-p", d)
-import re
+import re, oe.qa
rpath_re = re.compile(r"\s+RPATH\s+(.*)")
for line in phdrs.split("\n"):
m = rpath_re.match(line)
@@ -176,7 +151,7 @@ def package_qa_check_useless_rpaths(file, name, d, elf,
messages):
if rpath_eq(rpath, libdir) or rpath_eq(rpath, base_libdir):
# The dynamic linker searches both these places anyway. There
is no point in
# looking there again.
-package_qa_add_message(messages, "useless-rpaths", "%s: %s
contains probably-redundant RPATH %s" % (name, package_qa_clean_path(file, d,
n
[OE-core] Installing licence files for static/header-only libraries into images
We're using:
EXTRA_IMAGE_FEATURES += "lic-pkgs"
to install the corresponding licence packages for all the packages
installed in our image. This works very well for binaries and dynamic
libraries. However, I've recently noticed that it doesn't install licence
files for any static or header-only libraries used during the build.
The corresponding -lic packages are built for such packages, but nothing
causes them to be installed since they are not runtime dependencies of
anything that is installed.
I thought that I could solve this problem by something like:
def lic_deps(d):
deps = []
for f in d.getVar('DEPENDS').split():
if not f.endswith("-native") and not f.startswith("virtual/"):
deps.append(f + "-lic")
return ' '.join(deps)
RDEPENDS_${PN}-lic += "${@lic_deps(d)}"
but this fails for recipes that use PROVIDES or other shenanigans which
means that just appending -lic to whatever is in DEPENDS is insufficient.
For the time being I can add the required packages by hand, but this is
brittle in the longer term.
Is there a better way to solve this problem?
Thanks.
Mike.
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#156909):
https://lists.openembedded.org/g/openembedded-core/message/156909
Mute This Topic: https://lists.openembedded.org/mt/86288556/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH v3 2/2] license: Allow treating missing license as error
On Wednesday 13 October 2021 at 11:48:05 +0100, Mike Crowe wrote:
> Use the same WARN_WA and ERROR_QA variables as insane.bbclass to allow
> individual recipes, the distro or other configuration to determine
> whether a missing licence should be treated as a warning (as it is now)
> or as an error.
>
> oe.qa.handle_error isn't immediately fatal, so track the overall sanity
> state and call bb.fatal if required at the end to ensure that the task
> really fails. If only bb.error is used then do_populate_lic isn't re-run
> on subsequent builds which could lead to the error being missed.
>
> It seems odd for the license- error classes to be listed in
> insane.bbclass but implemented in license.bbclass. All recommendations
> for somewhere else to put them gratefully received.
>
> Signed-off-by: Mike Crowe
> ---
> meta/classes/insane.bbclass | 1 +
> meta/classes/license.bbclass | 27 ---
> 2 files changed, 21 insertions(+), 7 deletions(-)
>
> diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass
> index 895216d3e8..57456c99ad 100644
> --- a/meta/classes/insane.bbclass
> +++ b/meta/classes/insane.bbclass
> @@ -28,6 +28,7 @@ WARN_QA ?= " libdir xorg-driver-abi \
> invalid-packageconfig host-user-contaminated uppercase-pn
> patch-fuzz \
> mime mime-xdg unlisted-pkg-lics unhandled-features-check \
> missing-update-alternatives native-last missing-ptest \
> +license-exists license-no-generic license-syntax license-format \
> "
> ERROR_QA ?= "dev-so debug-deps dev-deps debug-files arch pkgconfig la \
> perms dep-cmp pkgvarcheck perm-config perm-line perm-link \
> diff --git a/meta/classes/license.bbclass b/meta/classes/license.bbclass
> index 45d912741d..6bbb71392e 100644
> --- a/meta/classes/license.bbclass
> +++ b/meta/classes/license.bbclass
> @@ -112,6 +112,7 @@ def find_license_files(d):
> import oe.license
> from collections import defaultdict, OrderedDict
>
> +sane = True
> # All the license files for the package
> lic_files = d.getVar('LIC_FILES_CHKSUM') or ""
> pn = d.getVar('PN')
> @@ -146,6 +147,7 @@ def find_license_files(d):
> self.generic_visit(node)
>
> def find_license(license_type):
> +import oe.qa
There's a "sane = True" missing here.
> try:
> bb.utils.mkdirhier(gen_lic_dest)
> except:
> @@ -178,7 +180,8 @@ def find_license_files(d):
> # The user may attempt to use NO_GENERIC_LICENSE for a generic
> license which doesn't make sense
> # and should not be allowed, warn the user in this case.
> if d.getVarFlag('NO_GENERIC_LICENSE', license_type):
> -bb.warn("%s: %s is a generic license, please don't use
> NO_GENERIC_LICENSE for it." % (pn, license_type))
> +sane &= oe.qa.handle_error("license-no-generic",
> +"%s: %s is a generic license, please don't use
> NO_GENERIC_LICENSE for it." % (pn, license_type), d)
>
> elif non_generic_lic and non_generic_lic in lic_chksums:
> # if NO_GENERIC_LICENSE is set, we copy the license files from
> the fetched source
> @@ -190,7 +193,8 @@ def find_license_files(d):
> # Add explicity avoid of CLOSED license because this isn't
> generic
> if license_type != 'CLOSED':
> # And here is where we warn people that their licenses are
> lousy
> -bb.warn("%s: No generic license file exists for: %s in any
> provider" % (pn, license_type))
> +sane &= oe.qa.handle_error("license-exists",
> +"%s: No generic license file exists for: %s in any
> provider" % (pn, license_type), d)
> pass
and a check for not sane missing here.
>
> if not generic_directory:
> @@ -215,7 +219,8 @@ def find_license_files(d):
> except oe.license.InvalidLicense as exc:
> bb.fatal('%s: %s' % (d.getVar('PF'), exc))
> except SyntaxError:
> -bb.warn("%s: Failed to parse it's LICENSE field." % (d.getVar('PF')))
> +sane &= oe.qa.handle_error("license-syntax",
> +"%s: Failed to parse it's LICENSE field." % (d.getVar('PF')), d)
> # Add files from LIC_FILES_CHKSUM to list of license files
> lic_chksum_paths = defaultdict(OrderedDict)
> for path, data in sorted(lic_chksums.items()):
> @@ -233,6 +238,8 @@ def find_license_files(d):
> for i, data in enumerate(files.values()):
> lic_files_paths.append(tuple(["%s.%d" % (basename, i)] +
> list(data)))
>
> +if not sane:
> +bb.fatal("Fatal QA errors found, failing task.")
> return lic_files_paths
>
> def return_spdx(d, license):
> @@ -398,6 +405,8 @@ def check_license_format(d):
> Validate operators in LICENSES.
> No spaces are allowed between LICENSES.
> """
> +import oe.qa
> +sa
Re: [OE-core] Installing licence files for static/header-only libraries into images
On Wednesday 13 October 2021 at 13:32:03 -0700, Khem Raj wrote:
> On Wed, Oct 13, 2021 at 7:06 AM Mike Crowe via lists.openembedded.org mcrowe@lists.openembedded.org> wrote:
>
> > We're using:
> >
> > EXTRA_IMAGE_FEATURES += "lic-pkgs"
> >
> > to install the corresponding licence packages for all the packages
> > installed in our image. This works very well for binaries and dynamic
> > libraries. However, I've recently noticed that it doesn't install licence
> > files for any static or header-only libraries used during the build.
> >
> > The corresponding -lic packages are built for such packages, but nothing
> > causes them to be installed since they are not runtime dependencies of
> > anything that is installed.
> >
> > I thought that I could solve this problem by something like:
> >
> > def lic_deps(d):
> > deps = []
> > for f in d.getVar('DEPENDS').split():
> > if not f.endswith("-native") and not f.startswith("virtual/"):
> > deps.append(f + "-lic")
> > return ' '.join(deps)
>
>
> It will work but in some cases it will be do more than what’s needed
> Build time dependencies are not necessarily also runtime dependencies eg it
> could be using a tool from that package during build
Yes. I was attempting to avoid that to a certain extent by excluding
-native packages.
> > RDEPENDS_${PN}-lic += "${@lic_deps(d)}"
> >
> > but this fails for recipes that use PROVIDES or other shenanigans which
> > means that just appending -lic to whatever is in DEPENDS is insufficient.
> >
> > For the time being I can add the required packages by hand, but this is
> > brittle in the longer term.
> >
> > Is there a better way to solve this problem?
The deficiencies in this solution made me hope that there was a better one.
It doesn't sound like there is. :(
Thanks.
Mike.
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#156966):
https://lists.openembedded.org/g/openembedded-core/message/156966
Mute This Topic: https://lists.openembedded.org/mt/86288556/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH v3 2/2] license: Allow treating missing license as error
On Thursday 14 October 2021 at 17:54:18 +0100, Richard Purdie wrote:
> On Thu, 2021-10-14 at 17:42 +0100, Mike Crowe via lists.openembedded.org
> wrote:
> > On Wednesday 13 October 2021 at 11:48:05 +0100, Mike Crowe wrote:
> > > Use the same WARN_WA and ERROR_QA variables as insane.bbclass to allow
> > > individual recipes, the distro or other configuration to determine
> > > whether a missing licence should be treated as a warning (as it is now)
> > > or as an error.
> > >
> > > oe.qa.handle_error isn't immediately fatal, so track the overall sanity
> > > state and call bb.fatal if required at the end to ensure that the task
> > > really fails. If only bb.error is used then do_populate_lic isn't re-run
> > > on subsequent builds which could lead to the error being missed.
> > >
> > > It seems odd for the license- error classes to be listed in
> > > insane.bbclass but implemented in license.bbclass. All recommendations
> > > for somewhere else to put them gratefully received.
[snip]
> > Patch series v4 coming soon.
>
> I did have to rework it a bit as I also ran into a few issues and was just
> trying to get around to writing them up.
>
> I added:
>
> http://git.yoctoproject.org/cgit.cgi/poky/commit/?h=master-next&id=9e980f1a71e8b6af5ff6da9b02fac0f8bfd9d4fb
>
> which means you can drop the imports.
>
> You can use the nonlocal option to help the "sane" problem:
>
> http://git.yoctoproject.org/cgit.cgi/poky/diff/meta/classes/license.bbclass?h=master-next&id=110fa545ac84e560691c7d9e0d1e6e8f70c66980
>
> The autobuilder also shows issues as some of the functions you moved are
> referenced in other classes. The patch became:
>
> http://git.yoctoproject.org/cgit.cgi/poky/commit/?h=master-next&id=110fa545ac84e560691c7d9e0d1e6e8f70c66980
Thanks for doing that. I thought I'd searched for other occurrences, but I
must have not done so properly. :(
> I was also just thinking we should probably add a new function and add calls
> at
> the end of the tasks like:
>
> oe.qa.exit_if_errors(d)
>
> with a definition similar to:
>
> def exit_if_errors(d):
> qa_sane = d.getVar("QA_SANE")
> if not qa_sane:
> bb.fatal("Fatal QA errors were found.")
>
> and then renaming QA_SANE to something like QA_FATAL_ERRORS and exitting if
> set.
That sounds like it would neaten things up greatly and avoid the risk of
accidentally failing to make the error fatal. It ought to mean that many of the
sane local
variables can be removed too.
Would you like me to provide new versions of the existing patches or add
new patches on top of the ones already in master-next?
Thanks.
Mike.
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#156968):
https://lists.openembedded.org/g/openembedded-core/message/156968
Mute This Topic: https://lists.openembedded.org/mt/86284884/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH v3 2/2] license: Allow treating missing license as error
On Thursday 14 October 2021 at 22:48:07 +0100, Richard Purdie wrote:
> On Thu, 2021-10-14 at 18:09 +0100, Mike Crowe wrote:
> > On Thursday 14 October 2021 at 17:54:18 +0100, Richard Purdie wrote:
> > > On Thu, 2021-10-14 at 17:42 +0100, Mike Crowe via lists.openembedded.org
> > > wrote:
> > > > On Wednesday 13 October 2021 at 11:48:05 +0100, Mike Crowe wrote:
> > > > > Use the same WARN_WA and ERROR_QA variables as insane.bbclass to allow
> > > > > individual recipes, the distro or other configuration to determine
> > > > > whether a missing licence should be treated as a warning (as it is
> > > > > now)
> > > > > or as an error.
> > > > >
> > > > > oe.qa.handle_error isn't immediately fatal, so track the overall
> > > > > sanity
> > > > > state and call bb.fatal if required at the end to ensure that the task
> > > > > really fails. If only bb.error is used then do_populate_lic isn't
> > > > > re-run
> > > > > on subsequent builds which could lead to the error being missed.
> > > > >
> > > > > It seems odd for the license- error classes to be listed in
> > > > > insane.bbclass but implemented in license.bbclass. All recommendations
> > > > > for somewhere else to put them gratefully received.
> >
> > [snip]
> >
> > > > Patch series v4 coming soon.
> > >
> > > I did have to rework it a bit as I also ran into a few issues and was just
> > > trying to get around to writing them up.
> > >
> > > I added:
> > >
> > > http://git.yoctoproject.org/cgit.cgi/poky/commit/?h=master-next&id=9e980f1a71e8b6af5ff6da9b02fac0f8bfd9d4fb
> > >
> > > which means you can drop the imports.
> > >
> > > You can use the nonlocal option to help the "sane" problem:
> > >
> > > http://git.yoctoproject.org/cgit.cgi/poky/diff/meta/classes/license.bbclass?h=master-next&id=110fa545ac84e560691c7d9e0d1e6e8f70c66980
> > >
> > > The autobuilder also shows issues as some of the functions you moved are
> > > referenced in other classes. The patch became:
> > >
> > > http://git.yoctoproject.org/cgit.cgi/poky/commit/?h=master-next&id=110fa545ac84e560691c7d9e0d1e6e8f70c66980
> >
> > Thanks for doing that. I thought I'd searched for other occurrences, but I
> > must have not done so properly. :(
> >
> > > I was also just thinking we should probably add a new function and add
> > > calls at
> > > the end of the tasks like:
> > >
> > > oe.qa.exit_if_errors(d)
> > >
> > > with a definition similar to:
> > >
> > > def exit_if_errors(d):
> > > qa_sane = d.getVar("QA_SANE")
> > > if not qa_sane:
> > > bb.fatal("Fatal QA errors were found.")
> > >
> > > and then renaming QA_SANE to something like QA_FATAL_ERRORS and exitting
> > > if set.
> >
> > That sounds like it would neaten things up greatly and avoid the risk of
> > accidentally failing to make the error fatal. It ought to mean that many of
> > the sane local
> > variables can be removed too.
> >
> > Would you like me to provide new versions of the existing patches or add
> > new patches on top of the ones already in master-next?
>
> I had half a patch for this last bit so I pushed it into master-next. I also
> merged by oe.qa import change.
I think that I already had equivalent changes to everything in
master-next:a53f5758bb2e501dfd34c6a6369303e91b382792. It was slow testing
over night! I've gone with QA_ERRORS_FOUND for the variable name since I
thought that more accurately captured what it meant. I also have only a
single call to oe.qa.exit_if_errors at the end of each task.
> I'm happy for you to take over from here and rework the patches into whatever
> makes the best sense for a series now we know what we're aiming for? I think
> there is a more cleanup of the "sane" variables that can be done beyond my
> quick
> patch.
Thanks. I'll hopefully send a new series for review today.
I'm finding myself wondering whether having oe.qa.add_messages collect
together all the messages and reporting them all in one go is worth the
complexity. The code that works out in advance which checks generate
warnings and which generate errors appears to be quite old and perhaps it
predates the current mechanism for classifying checks when they fail.
However, any changes here would be rather large, so it would probably make
sense to tackle this separately.
Mike.
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#156990):
https://lists.openembedded.org/g/openembedded-core/message/156990
Mute This Topic: https://lists.openembedded.org/mt/86284884/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH v4 1/2] lib/oe/qa,insane: Move extra error handling functions to library
Extract package_qa_write_error, package_qa_handle_error and
package_qa_add_message functions from insane.bbclass to lib/oe/qa.py and
drop the package_qa_ prefixes.
Update various bbclasses to use the new functions. No import is required
since base.bbclass puts oe.qa in OE_IMPORTS.
Stop requiring callers to manually track whether a fatal error has been
encountered via a "sane" flag. Instead replace the QA_SANE variable with
QA_ERRORS_FOUND and call oe.qa.exit_if_errors or
oe.qa.exit_with_message_if_errors at the end of each task.
Inspired by discussion resulting from
https://lists.openembedded.org/g/openembedded-core/message/156793 and
https://lists.openembedded.org/g/openembedded-core/message/156900
Signed-off-by: Mike Crowe
---
meta/classes/buildhistory.bbclass | 3 +-
meta/classes/insane.bbclass | 180 --
meta/classes/multilib.bbclass | 3 +-
meta/classes/package.bbclass | 26 ++---
meta/classes/ptest.bbclass| 2 +-
meta/lib/oe/qa.py | 34 ++
6 files changed, 121 insertions(+), 127 deletions(-)
diff --git a/meta/classes/buildhistory.bbclass
b/meta/classes/buildhistory.bbclass
index 7c44fec2d1..62d0d781a1 100644
--- a/meta/classes/buildhistory.bbclass
+++ b/meta/classes/buildhistory.bbclass
@@ -287,7 +287,7 @@ python buildhistory_emit_pkghistory() {
r = bb.utils.vercmp((pkge, pkgv, pkgr), (last_pkge, last_pkgv,
last_pkgr))
if r < 0:
msg = "Package version for package %s went backwards which
would break package feeds (from %s:%s-%s to %s:%s-%s)" % (pkg, last_pkge,
last_pkgv, last_pkgr, pkge, pkgv, pkgr)
-package_qa_handle_error("version-going-backwards", msg, d)
+oe.qa.handle_error("version-going-backwards", msg, d)
pkginfo = PackageInfo(pkg)
# Apparently the version can be different on a per-package basis (see
Python)
@@ -321,6 +321,7 @@ python buildhistory_emit_pkghistory() {
# Create files-in-.txt files containing a list of files of
each recipe's package
bb.build.exec_func("buildhistory_list_pkg_files", d)
+oe.qa.exit_if_errors(d)
}
python buildhistory_emit_outputsigs() {
diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass
index 433e4dfa33..2df5edf138 100644
--- a/meta/classes/insane.bbclass
+++ b/meta/classes/insane.bbclass
@@ -18,7 +18,7 @@
# files under exec_prefix
# -Check if the package name is upper case
-QA_SANE = "True"
+QA_ERRORS_FOUND = "False"
# Elect whether a given type of error is a warning or error, they may
# have been set by other files.
@@ -59,32 +59,6 @@ def package_qa_clean_path(path, d, pkg=None):
path = path.replace(os.path.join(d.getVar("PKGDEST"), pkg), "/")
return path.replace(d.getVar("TMPDIR"), "/").replace("//", "/")
-def package_qa_write_error(type, error, d):
-logfile = d.getVar('QA_LOGFILE')
-if logfile:
-p = d.getVar('P')
-with open(logfile, "a+") as f:
-f.write("%s: %s [%s]\n" % (p, error, type))
-
-def package_qa_handle_error(error_class, error_msg, d):
-if error_class in (d.getVar("ERROR_QA") or "").split():
-package_qa_write_error(error_class, error_msg, d)
-bb.error("QA Issue: %s [%s]" % (error_msg, error_class))
-d.setVar("QA_SANE", False)
-return False
-elif error_class in (d.getVar("WARN_QA") or "").split():
-package_qa_write_error(error_class, error_msg, d)
-bb.warn("QA Issue: %s [%s]" % (error_msg, error_class))
-else:
-bb.note("QA Issue: %s [%s]" % (error_msg, error_class))
-return True
-
-def package_qa_add_message(messages, section, new_msg):
-if section not in messages:
-messages[section] = new_msg
-else:
-messages[section] = messages[section] + "\n" + new_msg
-
QAPATHTEST[shebang-size] = "package_qa_check_shebang_size"
def package_qa_check_shebang_size(path, name, d, elf, messages):
import stat
@@ -106,7 +80,7 @@ def package_qa_check_shebang_size(path, name, d, elf,
messages):
return
if len(stanza) > 129:
-package_qa_add_message(messages, "shebang-size", "%s: %s maximum
shebang size exceeded, the maximum size is 128." % (name,
package_qa_clean_path(path, d)))
+oe.qa.add_message(messages, "shebang-size", "%s: %s maximum
shebang size exceeded, the maximum size is 128." % (name,
package_qa_clean_path(path, d)))
return
QAPATHTEST[libexec] = "package_qa_check_libexec"
@@ -118,7 +92,7 @@ def package_qa_check_libexec(path,name, d, elf, messages):
return True
if 'libexec' in path.split(os.path.sep):
-package_qa_add_message(messages, "libexec", "%s: %s is using libexec
please relocate to %s" % (name, package_qa_clean_path(path, d), libexec))
+oe.qa.add_message(messages, "libexec", "%s: %s is using libexec please
relocate to %s" % (name, package_qa_clean_path(path, d), libexec))
[OE-core] [PATCH v4 2/2] insane,license,license_image: Allow treating license problems as errors
Use the same WARN_WA and ERROR_QA variables as insane.bbclass to allow
individual recipes, the distro or other configuration to determine
whether the various detected license errors should be treated as a
warning (as now) or as an error.
oe.qa.handle_error isn't immediately fatal, so oe.qa.exit_if_errors must
be called at the end of do_populate_lic to fail the task.
Signed-off-by: Mike Crowe
---
meta/classes/insane.bbclass| 2 ++
meta/classes/license.bbclass | 20 +---
meta/classes/license_image.bbclass | 11 ++-
3 files changed, 21 insertions(+), 12 deletions(-)
diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass
index 2df5edf138..fa58511e87 100644
--- a/meta/classes/insane.bbclass
+++ b/meta/classes/insane.bbclass
@@ -28,6 +28,8 @@ WARN_QA ?= " libdir xorg-driver-abi \
invalid-packageconfig host-user-contaminated uppercase-pn
patch-fuzz \
mime mime-xdg unlisted-pkg-lics unhandled-features-check \
missing-update-alternatives native-last missing-ptest \
+license-exists license-no-generic license-syntax license-format \
+license-incompatible license-file-missing \
"
ERROR_QA ?= "dev-so debug-deps dev-deps debug-files arch pkgconfig la \
perms dep-cmp pkgvarcheck perm-config perm-line perm-link \
diff --git a/meta/classes/license.bbclass b/meta/classes/license.bbclass
index 7a34e185c7..d5480d87e2 100644
--- a/meta/classes/license.bbclass
+++ b/meta/classes/license.bbclass
@@ -29,6 +29,7 @@ python do_populate_lic() {
with open(os.path.join(destdir, "recipeinfo"), "w") as f:
for key in sorted(info.keys()):
f.write("%s: %s\n" % (key, info[key]))
+oe.qa.exit_if_errors(d)
}
PSEUDO_IGNORE_PATHS .= ",${@','.join(((d.getVar('COMMON_LICENSE_DIR') or '') +
' ' + (d.getVar('LICENSE_PATH') or '') + ' ' + d.getVar('COREBASE') +
'/meta/COPYING').split())}"
@@ -182,7 +183,8 @@ def find_license_files(d):
# The user may attempt to use NO_GENERIC_LICENSE for a generic
license which doesn't make sense
# and should not be allowed, warn the user in this case.
if d.getVarFlag('NO_GENERIC_LICENSE', license_type):
-bb.warn("%s: %s is a generic license, please don't use
NO_GENERIC_LICENSE for it." % (pn, license_type))
+oe.qa.handle_error("license-no-generic",
+"%s: %s is a generic license, please don't use
NO_GENERIC_LICENSE for it." % (pn, license_type), d)
elif non_generic_lic and non_generic_lic in lic_chksums:
# if NO_GENERIC_LICENSE is set, we copy the license files from the
fetched source
@@ -194,7 +196,8 @@ def find_license_files(d):
# Add explicity avoid of CLOSED license because this isn't generic
if license_type != 'CLOSED':
# And here is where we warn people that their licenses are
lousy
-bb.warn("%s: No generic license file exists for: %s in any
provider" % (pn, license_type))
+oe.qa.handle_error("license-exists",
+"%s: No generic license file exists for: %s in any
provider" % (pn, license_type), d)
pass
if not generic_directory:
@@ -219,7 +222,8 @@ def find_license_files(d):
except oe.license.InvalidLicense as exc:
bb.fatal('%s: %s' % (d.getVar('PF'), exc))
except SyntaxError:
-bb.warn("%s: Failed to parse it's LICENSE field." % (d.getVar('PF')))
+oe.qa.handle_error("license-syntax",
+"%s: Failed to parse it's LICENSE field." % (d.getVar('PF')), d)
# Add files from LIC_FILES_CHKSUM to list of license files
lic_chksum_paths = defaultdict(OrderedDict)
for path, data in sorted(lic_chksums.items()):
@@ -410,14 +414,16 @@ def check_license_format(d):
for pos, element in enumerate(elements):
if license_pattern.match(element):
if pos > 0 and license_pattern.match(elements[pos - 1]):
-bb.warn('%s: LICENSE value "%s" has an invalid format -
license names ' \
+oe.qa.handle_error('license-format',
+'%s: LICENSE value "%s" has an invalid format -
license names ' \
'must be separated by the following characters to
indicate ' \
'the license selection: %s' %
-(pn, licenses, license_operator_chars))
+(pn, licenses, license_operator_chars), d)
elif not license_operator.match(element):
-bb.warn('%s: LICENSE value "%s" has an invalid separator "%s" that
is not ' \
+oe.qa.handle_error('license-format',
+'%s: LICENSE value "%s" has an invalid separator "%s" that
is not ' \
'in the valid list of separators (%s)' %
-(pn, licenses, element, license_operator_chars))
+(
Re: [OE-core] [PATCH v4 1/2] lib/oe/qa,insane: Move extra error handling functions to library
On Friday 15 October 2021 at 15:09:08 +0100, Richard Purdie wrote:
> On Fri, 2021-10-15 at 14:59 +0100, Mike Crowe via lists.openembedded.org
> wrote:
> > Extract package_qa_write_error, package_qa_handle_error and
> > package_qa_add_message functions from insane.bbclass to lib/oe/qa.py and
> > drop the package_qa_ prefixes.
> >
> > Update various bbclasses to use the new functions. No import is required
> > since base.bbclass puts oe.qa in OE_IMPORTS.
> >
> > Stop requiring callers to manually track whether a fatal error has been
> > encountered via a "sane" flag. Instead replace the QA_SANE variable with
> > QA_ERRORS_FOUND and call oe.qa.exit_if_errors or
> > oe.qa.exit_with_message_if_errors at the end of each task.
> >
> > Inspired by discussion resulting from
> > https://lists.openembedded.org/g/openembedded-core/message/156793 and
> > https://lists.openembedded.org/g/openembedded-core/message/156900
> >
> > Signed-off-by: Mike Crowe
> > ---
> > meta/classes/buildhistory.bbclass | 3 +-
> > meta/classes/insane.bbclass | 180 --
> > meta/classes/multilib.bbclass | 3 +-
> > meta/classes/package.bbclass | 26 ++---
> > meta/classes/ptest.bbclass| 2 +-
> > meta/lib/oe/qa.py | 34 ++
> > 6 files changed, 121 insertions(+), 127 deletions(-)
> >
> > diff --git a/meta/classes/buildhistory.bbclass
> > b/meta/classes/buildhistory.bbclass
> > index 7c44fec2d1..62d0d781a1 100644
> > --- a/meta/classes/buildhistory.bbclass
> > +++ b/meta/classes/buildhistory.bbclass
> > @@ -287,7 +287,7 @@ python buildhistory_emit_pkghistory() {
> > r = bb.utils.vercmp((pkge, pkgv, pkgr), (last_pkge, last_pkgv,
> > last_pkgr))
> > if r < 0:
> > msg = "Package version for package %s went backwards which
> > would break package feeds (from %s:%s-%s to %s:%s-%s)" % (pkg, last_pkge,
> > last_pkgv, last_pkgr, pkge, pkgv, pkgr)
> > -package_qa_handle_error("version-going-backwards", msg, d)
> > +oe.qa.handle_error("version-going-backwards", msg, d)
> >
> > pkginfo = PackageInfo(pkg)
> > # Apparently the version can be different on a per-package basis
> > (see Python)
> > @@ -321,6 +321,7 @@ python buildhistory_emit_pkghistory() {
> >
> > # Create files-in-.txt files containing a list of files
> > of each recipe's package
> > bb.build.exec_func("buildhistory_list_pkg_files", d)
> > +oe.qa.exit_if_errors(d)
>
> This is a change in behaviour since currently buildhistory doesn't do that and
> I'm not sure it should.
How about moving version-going-backwards from ERROR_QA to WARN_QA and
keeping the call to oe.qa.exit_if_errors?
It's somewhat confusing having version-going-backwards in ERROR_QA now if
it's not going to be fatal when all the other checks in that variable are
fatal. It also means that any future callers of oe.qa.handle_error from
this task would behave the same.
>
> > }
> >
> > python buildhistory_emit_outputsigs() {
> > diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass
> > index 433e4dfa33..2df5edf138 100644
> > --- a/meta/classes/insane.bbclass
> > +++ b/meta/classes/insane.bbclass
> > @@ -18,7 +18,7 @@
> > # files under exec_prefix
> > # -Check if the package name is upper case
> >
> > -QA_SANE = "True"
> > +QA_ERRORS_FOUND = "False"
> >
>
> Lets just delete this please, we don't need a default value and it is
> confusing
> being isolated from the other code.
OK. I'll await your response to the above question and then send v5.
Thanks.
Mike.
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#156998):
https://lists.openembedded.org/g/openembedded-core/message/156998
Mute This Topic: https://lists.openembedded.org/mt/86339420/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH v5 1/2] lib/oe/qa,insane: Move extra error handling functions to library
Extract package_qa_write_error, package_qa_handle_error and
package_qa_add_message functions from insane.bbclass to lib/oe/qa.py and
drop the package_qa_ prefixes.
Update various bbclasses to use the new functions. No import is required
since base.bbclass puts oe.qa in OE_IMPORTS.
Stop requiring callers to manually track whether a fatal error has been
encountered via a "sane" flag. Instead replace the QA_SANE variable with
QA_ERRORS_FOUND and call oe.qa.exit_if_errors or
oe.qa.exit_with_message_if_errors at the end of each task.
Inspired by discussion resulting from
https://lists.openembedded.org/g/openembedded-core/message/156793 and
https://lists.openembedded.org/g/openembedded-core/message/156900
Signed-off-by: Mike Crowe
---
meta/classes/buildhistory.bbclass | 3 +-
meta/classes/insane.bbclass | 180 --
meta/classes/multilib.bbclass | 3 +-
meta/classes/package.bbclass | 26 ++---
meta/classes/ptest.bbclass| 2 +-
meta/lib/oe/qa.py | 34 ++
6 files changed, 120 insertions(+), 128 deletions(-)
diff --git a/meta/classes/buildhistory.bbclass
b/meta/classes/buildhistory.bbclass
index 7c44fec2d1..62d0d781a1 100644
--- a/meta/classes/buildhistory.bbclass
+++ b/meta/classes/buildhistory.bbclass
@@ -287,7 +287,7 @@ python buildhistory_emit_pkghistory() {
r = bb.utils.vercmp((pkge, pkgv, pkgr), (last_pkge, last_pkgv,
last_pkgr))
if r < 0:
msg = "Package version for package %s went backwards which
would break package feeds (from %s:%s-%s to %s:%s-%s)" % (pkg, last_pkge,
last_pkgv, last_pkgr, pkge, pkgv, pkgr)
-package_qa_handle_error("version-going-backwards", msg, d)
+oe.qa.handle_error("version-going-backwards", msg, d)
pkginfo = PackageInfo(pkg)
# Apparently the version can be different on a per-package basis (see
Python)
@@ -321,6 +321,7 @@ python buildhistory_emit_pkghistory() {
# Create files-in-.txt files containing a list of files of
each recipe's package
bb.build.exec_func("buildhistory_list_pkg_files", d)
+oe.qa.exit_if_errors(d)
}
python buildhistory_emit_outputsigs() {
diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass
index 433e4dfa33..9ad9771dfa 100644
--- a/meta/classes/insane.bbclass
+++ b/meta/classes/insane.bbclass
@@ -18,8 +18,6 @@
# files under exec_prefix
# -Check if the package name is upper case
-QA_SANE = "True"
-
# Elect whether a given type of error is a warning or error, they may
# have been set by other files.
WARN_QA ?= " libdir xorg-driver-abi \
@@ -59,32 +57,6 @@ def package_qa_clean_path(path, d, pkg=None):
path = path.replace(os.path.join(d.getVar("PKGDEST"), pkg), "/")
return path.replace(d.getVar("TMPDIR"), "/").replace("//", "/")
-def package_qa_write_error(type, error, d):
-logfile = d.getVar('QA_LOGFILE')
-if logfile:
-p = d.getVar('P')
-with open(logfile, "a+") as f:
-f.write("%s: %s [%s]\n" % (p, error, type))
-
-def package_qa_handle_error(error_class, error_msg, d):
-if error_class in (d.getVar("ERROR_QA") or "").split():
-package_qa_write_error(error_class, error_msg, d)
-bb.error("QA Issue: %s [%s]" % (error_msg, error_class))
-d.setVar("QA_SANE", False)
-return False
-elif error_class in (d.getVar("WARN_QA") or "").split():
-package_qa_write_error(error_class, error_msg, d)
-bb.warn("QA Issue: %s [%s]" % (error_msg, error_class))
-else:
-bb.note("QA Issue: %s [%s]" % (error_msg, error_class))
-return True
-
-def package_qa_add_message(messages, section, new_msg):
-if section not in messages:
-messages[section] = new_msg
-else:
-messages[section] = messages[section] + "\n" + new_msg
-
QAPATHTEST[shebang-size] = "package_qa_check_shebang_size"
def package_qa_check_shebang_size(path, name, d, elf, messages):
import stat
@@ -106,7 +78,7 @@ def package_qa_check_shebang_size(path, name, d, elf,
messages):
return
if len(stanza) > 129:
-package_qa_add_message(messages, "shebang-size", "%s: %s maximum
shebang size exceeded, the maximum size is 128." % (name,
package_qa_clean_path(path, d)))
+oe.qa.add_message(messages, "shebang-size", "%s: %s maximum
shebang size exceeded, the maximum size is 128." % (name,
package_qa_clean_path(path, d)))
return
QAPATHTEST[libexec] = "package_qa_check_libexec"
@@ -118,7 +90,7 @@ def package_qa_check_libexec(path,name, d, elf, messages):
return True
if 'libexec' in path.split(os.path.sep):
-package_qa_add_message(messages, "libexec", "%s: %s is using libexec
please relocate to %s" % (name, package_qa_clean_path(path, d), libexec))
+oe.qa.add_message(messages, "libexec", "%s: %s is using libexec please
relocate to %s" % (name, package_qa_clean_path(path, d),
[OE-core] [PATCH v5 2/2] insane,license,license_image: Allow treating license problems as errors
Use the same WARN_WA and ERROR_QA variables as insane.bbclass to allow
individual recipes, the distro or other configuration to determine
whether the various detected license errors should be treated as a
warning (as now) or as an error.
oe.qa.handle_error isn't immediately fatal, so oe.qa.exit_if_errors must
be called at the end of do_populate_lic to fail the task.
Signed-off-by: Mike Crowe
---
meta/classes/insane.bbclass| 2 ++
meta/classes/license.bbclass | 20 +---
meta/classes/license_image.bbclass | 11 ++-
3 files changed, 21 insertions(+), 12 deletions(-)
diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass
index 9ad9771dfa..1e2f1b768a 100644
--- a/meta/classes/insane.bbclass
+++ b/meta/classes/insane.bbclass
@@ -26,6 +26,8 @@ WARN_QA ?= " libdir xorg-driver-abi \
invalid-packageconfig host-user-contaminated uppercase-pn
patch-fuzz \
mime mime-xdg unlisted-pkg-lics unhandled-features-check \
missing-update-alternatives native-last missing-ptest \
+license-exists license-no-generic license-syntax license-format \
+license-incompatible license-file-missing \
"
ERROR_QA ?= "dev-so debug-deps dev-deps debug-files arch pkgconfig la \
perms dep-cmp pkgvarcheck perm-config perm-line perm-link \
diff --git a/meta/classes/license.bbclass b/meta/classes/license.bbclass
index 7a34e185c7..d5480d87e2 100644
--- a/meta/classes/license.bbclass
+++ b/meta/classes/license.bbclass
@@ -29,6 +29,7 @@ python do_populate_lic() {
with open(os.path.join(destdir, "recipeinfo"), "w") as f:
for key in sorted(info.keys()):
f.write("%s: %s\n" % (key, info[key]))
+oe.qa.exit_if_errors(d)
}
PSEUDO_IGNORE_PATHS .= ",${@','.join(((d.getVar('COMMON_LICENSE_DIR') or '') +
' ' + (d.getVar('LICENSE_PATH') or '') + ' ' + d.getVar('COREBASE') +
'/meta/COPYING').split())}"
@@ -182,7 +183,8 @@ def find_license_files(d):
# The user may attempt to use NO_GENERIC_LICENSE for a generic
license which doesn't make sense
# and should not be allowed, warn the user in this case.
if d.getVarFlag('NO_GENERIC_LICENSE', license_type):
-bb.warn("%s: %s is a generic license, please don't use
NO_GENERIC_LICENSE for it." % (pn, license_type))
+oe.qa.handle_error("license-no-generic",
+"%s: %s is a generic license, please don't use
NO_GENERIC_LICENSE for it." % (pn, license_type), d)
elif non_generic_lic and non_generic_lic in lic_chksums:
# if NO_GENERIC_LICENSE is set, we copy the license files from the
fetched source
@@ -194,7 +196,8 @@ def find_license_files(d):
# Add explicity avoid of CLOSED license because this isn't generic
if license_type != 'CLOSED':
# And here is where we warn people that their licenses are
lousy
-bb.warn("%s: No generic license file exists for: %s in any
provider" % (pn, license_type))
+oe.qa.handle_error("license-exists",
+"%s: No generic license file exists for: %s in any
provider" % (pn, license_type), d)
pass
if not generic_directory:
@@ -219,7 +222,8 @@ def find_license_files(d):
except oe.license.InvalidLicense as exc:
bb.fatal('%s: %s' % (d.getVar('PF'), exc))
except SyntaxError:
-bb.warn("%s: Failed to parse it's LICENSE field." % (d.getVar('PF')))
+oe.qa.handle_error("license-syntax",
+"%s: Failed to parse it's LICENSE field." % (d.getVar('PF')), d)
# Add files from LIC_FILES_CHKSUM to list of license files
lic_chksum_paths = defaultdict(OrderedDict)
for path, data in sorted(lic_chksums.items()):
@@ -410,14 +414,16 @@ def check_license_format(d):
for pos, element in enumerate(elements):
if license_pattern.match(element):
if pos > 0 and license_pattern.match(elements[pos - 1]):
-bb.warn('%s: LICENSE value "%s" has an invalid format -
license names ' \
+oe.qa.handle_error('license-format',
+'%s: LICENSE value "%s" has an invalid format -
license names ' \
'must be separated by the following characters to
indicate ' \
'the license selection: %s' %
-(pn, licenses, license_operator_chars))
+(pn, licenses, license_operator_chars), d)
elif not license_operator.match(element):
-bb.warn('%s: LICENSE value "%s" has an invalid separator "%s" that
is not ' \
+oe.qa.handle_error('license-format',
+'%s: LICENSE value "%s" has an invalid separator "%s" that
is not ' \
'in the valid list of separators (%s)' %
-(pn, licenses, element, license_operator_chars))
+(
[OE-core] Correct way to set DRIDRIVERS and GALLIUMDRIVERS from mesa bbappend
I'm building for a specific chip and therefore don't wish to waste time and
electricity building and disk space on the target installing unwanted mesa
drivers. However, mesa.inc contains:
GALLIUMDRIVERS = "swrast"
GALLIUMDRIVERS:x86-x32 = ""
GALLIUMDRIVERS:append:x86:class-target = ",i915,iris,crocus"
GALLIUMDRIVERS:append:x86-64:class-target = ",i915,iris,crocus"
and mesa_21.3.0.bb contains:
DRIDRIVERS ??= ""
DRIDRIVERS:append:x86:class-target = ",r100,r200,nouveau,i965"
DRIDRIVERS:append:x86-64:class-target = ",r100,r200,nouveau,i965"
I'm unable to find a way to override these values. Using (for example):
DRIDRIVERS:forcevariable = ""
GALLIUMDRIVERS:forcevariable = "swrast"
doesn't work because the append still happens after the forcevariable
override takes effect. :(
Is there a way that I can override GALLIUMDRIVERS and DRIDRIVERS with my
own values for x86 and x86-64 without modifying oe-core itself?
If not, should the oe-core recipe being using something like:
GALLIUMDRIVERS_DEFAULT = "swrast"
GALLIUMDRIVERS_DEFAULT:x86-x32 = ""
GALLIUMDRIVERS_DEFAULT:append:x86:class-target = ",i915,iris,crocus"
GALLIUMDRIVERS_DEFAULT:append:x86-64:class-target = ",i915,iris,crocus"
GALLIUMDRIVERS ?= "${GALLIUMDRIVERS_DEFAULT}"
and similar for DRIDRIVERS to support this?
Thanks.
Mike.
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#159048):
https://lists.openembedded.org/g/openembedded-core/message/159048
Mute This Topic: https://lists.openembedded.org/mt/87432435/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] Correct way to set DRIDRIVERS and GALLIUMDRIVERS from mesa bbappend
Hi Alex,
Thanks for responding.
Just to make sure I understand, you think that I should add something like:
PACKAGECONFIG[i915] = ""
PACKAGECONFIG[iris] = ""
PACKAGECONFIG[crocus] = ""
GALLIUMDRIVERS:append = "${@bb.utils.contains('PACKAGECONFIG', 'i915',
',i915', '', d)}"
GALLIUMDRIVERS:append = "${@bb.utils.contains('PACKAGECONFIG', 'iris',
',iris', '', d)}"
GALLIUMDRIVERS:append = "${@bb.utils.contains('PACKAGECONFIG', 'crocus',
',crocus', '', d)}"
But then I need to find a way to get "i915 iris crocus" into PACKAGECONFIG
for x86 and x86-64 whilst still letting others override that. The best I
can come up with is to modify the existing default PACKAGECONFIG to be
something like:
PACKAGECONFIG_DEFAULT_FOR_TARGET:x86 = "i915 iris crocus"
PACKAGECONFIG_DEFAULT_FOR_TARGET:x86-64 = "i915 iris crocus"
PACKAGECONFIG:class-target ??= "${@bb.utils.filter('DISTRO_FEATURES', 'wayland
vulkan', d)} \
${@bb.utils.contains('DISTRO_FEATURES', 'opengl', 'opengl
egl gles gbm dri gallium virgl', '', d)} \
${@bb.utils.contains('DISTRO_FEATURES', 'x11 opengl', 'x11
dri3', '', d)} \
${@bb.utils.contains('DISTRO_FEATURES', 'x11 vulkan',
'dri3', '', d)} \
elf-tls \
${PACKAGECONFIG_DEFAULT_FOR_TARGET} \
"
This will probably break anyone who currently sets their own PACKAGECONFIG
for x86 and x86-64 until they add the drivers they require. This would seem
to be worse than my GALLIUMDRIVERS_DEFAULT suggestion. :(
Have I misunderstood you, or is there a better way?
Thanks.
Mike.
On Wednesday 01 December 2021 at 18:21:44 +0100, Alexander Kanavin wrote:
> I think you do need to modify oe-core unfortunately, like is done for other
> drivers:
>
> PACKAGECONFIG[etnaviv] = ""
> GALLIUMDRIVERS:append ="${@bb.utils.contains('PACKAGECONFIG', 'etnaviv',
> ',etnaviv', '', d)}"
>
> Alex
>
> On Wed, 1 Dec 2021 at 17:44, Mike Crowe via lists.openembedded.org mcrowe@lists.openembedded.org> wrote:
>
> > I'm building for a specific chip and therefore don't wish to waste time and
> > electricity building and disk space on the target installing unwanted mesa
> > drivers. However, mesa.inc contains:
> >
> > GALLIUMDRIVERS = "swrast"
> > GALLIUMDRIVERS:x86-x32 = ""
> > GALLIUMDRIVERS:append:x86:class-target = ",i915,iris,crocus"
> > GALLIUMDRIVERS:append:x86-64:class-target = ",i915,iris,crocus"
> >
> > and mesa_21.3.0.bb contains:
> >
> > DRIDRIVERS ??= ""
> > DRIDRIVERS:append:x86:class-target = ",r100,r200,nouveau,i965"
> > DRIDRIVERS:append:x86-64:class-target = ",r100,r200,nouveau,i965"
> >
> > I'm unable to find a way to override these values. Using (for example):
> >
> > DRIDRIVERS:forcevariable = ""
> > GALLIUMDRIVERS:forcevariable = "swrast"
> >
> > doesn't work because the append still happens after the forcevariable
> > override takes effect. :(
> >
> > Is there a way that I can override GALLIUMDRIVERS and DRIDRIVERS with my
> > own values for x86 and x86-64 without modifying oe-core itself?
> >
> > If not, should the oe-core recipe being using something like:
> >
> > GALLIUMDRIVERS_DEFAULT = "swrast"
> > GALLIUMDRIVERS_DEFAULT:x86-x32 = ""
> > GALLIUMDRIVERS_DEFAULT:append:x86:class-target = ",i915,iris,crocus"
> > GALLIUMDRIVERS_DEFAULT:append:x86-64:class-target = ",i915,iris,crocus"
> > GALLIUMDRIVERS ?= "${GALLIUMDRIVERS_DEFAULT}"
> >
> > and similar for DRIDRIVERS to support this?
> >
> > Thanks.
> >
> > Mike.
> >
> >
> >
> >
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#159055):
https://lists.openembedded.org/g/openembedded-core/message/159055
Mute This Topic: https://lists.openembedded.org/mt/87432435/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] Correct way to set DRIDRIVERS and GALLIUMDRIVERS from mesa bbappend
On Wednesday 01 December 2021 at 14:18:19 -0500, Justin Bronder wrote:
> On 01/12/21 16:43 +0000, Mike Crowe via lists.openembedded.org wrote:
> > I'm building for a specific chip and therefore don't wish to waste time and
> > electricity building and disk space on the target installing unwanted mesa
> > drivers. However, mesa.inc contains:
> >
> > GALLIUMDRIVERS = "swrast"
> > GALLIUMDRIVERS:x86-x32 = ""
> > GALLIUMDRIVERS:append:x86:class-target = ",i915,iris,crocus"
> > GALLIUMDRIVERS:append:x86-64:class-target = ",i915,iris,crocus"
> >
> > and mesa_21.3.0.bb contains:
> >
> > DRIDRIVERS ??= ""
> > DRIDRIVERS:append:x86:class-target = ",r100,r200,nouveau,i965"
> > DRIDRIVERS:append:x86-64:class-target = ",r100,r200,nouveau,i965"
> >
> > I'm unable to find a way to override these values. Using (for example):
>
> You can use an anonymous python function which runs after parsing.
> https://www.yoctoproject.org/docs/current/mega-manual/mega-manual.html#anonymous-python-functions
>
> python __anonymous () {
> d.setVar("DRIDRIVERS", "i965")
> }
Thanks for the suggestion. It solves my problem. However, it seems like
quite a subversive way to do the sort of external customisation that
recipes normally support.
Thanks.
Mike.
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#159058):
https://lists.openembedded.org/g/openembedded-core/message/159058
Mute This Topic: https://lists.openembedded.org/mt/87432435/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] Correct way to set DRIDRIVERS and GALLIUMDRIVERS from mesa bbappend
ote:
>
> > Hi Alex,
> >
> > Thanks for responding.
> >
> > Just to make sure I understand, you think that I should add something like:
> >
> > PACKAGECONFIG[i915] = ""
> > PACKAGECONFIG[iris] = ""
> > PACKAGECONFIG[crocus] = ""
> > GALLIUMDRIVERS:append = "${@bb.utils.contains('PACKAGECONFIG', 'i915',
> > ',i915', '', d)}"
> > GALLIUMDRIVERS:append = "${@bb.utils.contains('PACKAGECONFIG', 'iris',
> > ',iris', '', d)}"
> > GALLIUMDRIVERS:append = "${@bb.utils.contains('PACKAGECONFIG', 'crocus',
> > ',crocus', '', d)}"
> >
> > But then I need to find a way to get "i915 iris crocus" into PACKAGECONFIG
> > for x86 and x86-64 whilst still letting others override that. The best I
> > can come up with is to modify the existing default PACKAGECONFIG to be
> > something like:
> >
> > PACKAGECONFIG_DEFAULT_FOR_TARGET:x86 = "i915 iris crocus"
> > PACKAGECONFIG_DEFAULT_FOR_TARGET:x86-64 = "i915 iris crocus"
> > PACKAGECONFIG:class-target ??= "${@bb.utils.filter('DISTRO_FEATURES',
> > 'wayland vulkan', d)} \
> > ${@bb.utils.contains('DISTRO_FEATURES', 'opengl',
> > 'opengl egl gles gbm dri gallium virgl', '', d)} \
> > ${@bb.utils.contains('DISTRO_FEATURES', 'x11 opengl',
> > 'x11 dri3', '', d)} \
> > ${@bb.utils.contains('DISTRO_FEATURES', 'x11 vulkan',
> > 'dri3', '', d)} \
> > elf-tls \
> > ${PACKAGECONFIG_DEFAULT_FOR_TARGET} \
> > "
> >
> > This will probably break anyone who currently sets their own PACKAGECONFIG
> > for x86 and x86-64 until they add the drivers they require. This would seem
> > to be worse than my GALLIUMDRIVERS_DEFAULT suggestion. :(
> >
> > Have I misunderstood you, or is there a better way?
> >
> > Thanks.
> >
> > Mike.
> >
> > On Wednesday 01 December 2021 at 18:21:44 +0100, Alexander Kanavin wrote:
> > > I think you do need to modify oe-core unfortunately, like is done for
> > other
> > > drivers:
> > >
> > > PACKAGECONFIG[etnaviv] = ""
> > > GALLIUMDRIVERS:append ="${@bb.utils.contains('PACKAGECONFIG', 'etnaviv',
> > > ',etnaviv', '', d)}"
> > >
> > > Alex
> > >
> > > On Wed, 1 Dec 2021 at 17:44, Mike Crowe via lists.openembedded.org > > mcrowe@lists.openembedded.org> wrote:
> > >
> > > > I'm building for a specific chip and therefore don't wish to waste
> > time and
> > > > electricity building and disk space on the target installing unwanted
> > mesa
> > > > drivers. However, mesa.inc contains:
> > > >
> > > > GALLIUMDRIVERS = "swrast"
> > > > GALLIUMDRIVERS:x86-x32 = ""
> > > > GALLIUMDRIVERS:append:x86:class-target = ",i915,iris,crocus"
> > > > GALLIUMDRIVERS:append:x86-64:class-target = ",i915,iris,crocus"
> > > >
> > > > and mesa_21.3.0.bb contains:
> > > >
> > > > DRIDRIVERS ??= ""
> > > > DRIDRIVERS:append:x86:class-target = ",r100,r200,nouveau,i965"
> > > > DRIDRIVERS:append:x86-64:class-target = ",r100,r200,nouveau,i965"
> > > >
> > > > I'm unable to find a way to override these values. Using (for example):
> > > >
> > > > DRIDRIVERS:forcevariable = ""
> > > > GALLIUMDRIVERS:forcevariable = "swrast"
> > > >
> > > > doesn't work because the append still happens after the forcevariable
> > > > override takes effect. :(
> > > >
> > > > Is there a way that I can override GALLIUMDRIVERS and DRIDRIVERS with
> > my
> > > > own values for x86 and x86-64 without modifying oe-core itself?
> > > >
> > > > If not, should the oe-core recipe being using something like:
> > > >
> > > > GALLIUMDRIVERS_DEFAULT = "swrast"
> > > > GALLIUMDRIVERS_DEFAULT:x86-x32 = ""
> > > > GALLIUMDRIVERS_DEFAULT:append:x86:class-target = ",i915,iris,crocus"
> > > > GALLIUMDRIVERS_DEFAULT:append:x86-64:class-target =
> > ",i915,iris,crocus"
> > > > GALLIUMDRIVERS ?= "${GALLIUMDRIVERS_DEFAULT}"
> > > >
> > > > and similar for DRIDRIVERS to support this?
> > > >
> > > > Thanks.
> > > >
> > > > Mike.
> > > >
> > > >
> > > >
> > > >
> >
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#159092):
https://lists.openembedded.org/g/openembedded-core/message/159092
Mute This Topic: https://lists.openembedded.org/mt/87432435/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH 1/2] package: Only snap libraries if they would be processed by ldconfig OS-12840
PACKAGE_SNAP_LIB_SYMLINKS renames libraries based on their SONAME so
that they can be found directly rather than going via symlinks that
would be created by ldconfig. For example, without
PACKAGE_SNAP_LIB_SYMLINKS in ${libdir} we have:
libharfbuzz.so.0 -> libharfbuzz.so.0.20600.4
libharfbuzz.so.0.20600.4
but with PACKAGE_SNAP_LIB_SYMLINKS="1" we have just:
libharfbuzz.so.0
Unfortunately, this renaming is done based on the SONAME which breaks
packages like mesa which install a single library with multiple hard
links:
-rwxr-xr-x root/root 13593488 2021-12-07 12:26 ./usr/lib/dri/i915_dri.so
-rwxr-xr-x root/root 13137328 2021-12-07 12:26 ./usr/lib/dri/i965_dri.so
hrwxr-xr-x root/root 0 2021-12-07 12:26 ./usr/lib/dri/iris_dri.so link
to ./usr/lib/dri/i915_dri.so
hrwxr-xr-x root/root 0 2021-12-07 12:26 ./usr/lib/dri/kms_swrast_dri.so
link to ./usr/lib/dri/i915_dri.so
hrwxr-xr-x root/root 0 2021-12-07 12:26
./usr/lib/dri/nouveau_vieux_dri.so link to ./usr/lib/dri/i965_dri.so
hrwxr-xr-x root/root 0 2021-12-07 12:26 ./usr/lib/dri/r200_dri.so link
to ./usr/lib/dri/i965_dri.so
hrwxr-xr-x root/root 0 2021-12-07 12:26 ./usr/lib/dri/radeon_dri.so
link to ./usr/lib/dri/i965_dri.so
hrwxr-xr-x root/root 0 2021-12-07 12:26 ./usr/lib/dri/swrast_dri.so
link to ./usr/lib/dri/i915_dri.so
The SONAME for i915_dri.so (and therefore all the other names that link
to the same file) is libgallium_dri.so. This means that
PACKAGE_SNAP_LIB_SYMLINKS causes do_package to successfully rename the
first name found to libgallium_dri.so. A similar thing happens to
i965_dri.so with its SONAME of libmesa_dri_drivers.so. The order is not
deterministic, so this means that although every build will be missing
one name, it's not always the same one.
-rwxr-xr-x root/root 13593488 2021-11-30 15:17 ./usr/lib/dri/i915_dri.so
hrwxr-xr-x root/root 0 2021-11-30 15:17 ./usr/lib/dri/kms_swrast_dri.so
link to ./usr/lib/dri/i915_dri.so
hrwxr-xr-x root/root 0 2021-11-30 15:17 ./usr/lib/dri/libgallium_dri.so
link to ./usr/lib/dri/i915_dri.so
-rwxr-xr-x root/root 13137328 2021-11-30 15:17
./usr/lib/dri/libmesa_dri_drivers.so
hrwxr-xr-x root/root 0 2021-11-30 15:17
./usr/lib/dri/nouveau_vieux_dri.so link to ./usr/lib/dri/libmesa_dri_drivers.so
hrwxr-xr-x root/root 0 2021-11-30 15:17 ./usr/lib/dri/r200_dri.so link
to ./usr/lib/dri/libmesa_dri_drivers.so
hrwxr-xr-x root/root 0 2021-11-30 15:17 ./usr/lib/dri/radeon_dri.so
link to ./usr/lib/dri/libmesa_dri_drivers.so
hrwxr-xr-x root/root 0 2021-11-30 15:17 ./usr/lib/dri/swrast_dri.so
link to ./usr/lib/dri/i915_dri.so
This renaming means that the library cannot be found at runtime.
The simplest way to avoid this renaming is to only snap libraries that
would be processed by ldconfig.
Signed-off-by: Mike Crowe
Signed-off-by: Phil Blundell
---
meta/classes/package.bbclass | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/meta/classes/package.bbclass b/meta/classes/package.bbclass
index 84eafbd529..09cd376f4a 100644
--- a/meta/classes/package.bbclass
+++ b/meta/classes/package.bbclass
@@ -1878,7 +1878,7 @@ python package_do_shlibs() {
sonames.add(prov)
if libdir_re.match(os.path.dirname(file)):
needs_ldconfig = True
-if snap_symlinks and (os.path.basename(file) != this_soname):
+if needs_ldconfig and snap_symlinks and
(os.path.basename(file) != this_soname):
renames.append((file, os.path.join(os.path.dirname(file),
this_soname)))
return (needs_ldconfig, needed, sonames, renames)
--
2.30.2
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#159306):
https://lists.openembedded.org/g/openembedded-core/message/159306
Mute This Topic: https://lists.openembedded.org/mt/87566846/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH 2/2] documentation: Document PACKAGE_SNAP_LIB_SYMLINKS
PACKAGE_SNAP_LIB_SYMLINKS was added[1] originally to OpenEmbedded in 2008 and then to oe-core in 2011[2] but appears to have evaded documentation throughout that time. Let's add something that at least gives some clue as to what it does. Signed-off-by: Mike Crowe Signed-off-by: Phil Blundell [1] https://git.openembedded.org/openembedded/commit/?id=cf7114179ead8ddff8f66e84d630811920ac9add [2] https://git.openembedded.org/openembedded-core/commit/?id=600dbb7cb384c2290af38b993a9bea3a4dfc4494 --- meta/conf/documentation.conf | 1 + 1 file changed, 1 insertion(+) diff --git a/meta/conf/documentation.conf b/meta/conf/documentation.conf index 45cd01374a..f63f4b223a 100644 --- a/meta/conf/documentation.conf +++ b/meta/conf/documentation.conf @@ -315,6 +315,7 @@ PACKAGE_EXCLUDE[doc] = "Packages to exclude from the installation. If a listed p PACKAGE_EXTRA_ARCHS[doc] = "Specifies the list of architectures compatible with the device CPU. This variable is useful when you build for several different devices that use miscellaneous processors." PACKAGE_INSTALL[doc] = "List of the packages to be installed into the image. The variable is generally not user-defined and uses IMAGE_INSTALL as part of the list." PACKAGE_INSTALL_ATTEMPTONLY[doc] = "List of packages attempted to be installed. If a listed package fails to install, the build system does not generate an error. This variable is generally not user-defined." +PACKAGE_SNAP_LIB_SYMLINKS[doc] = "Rename library files based on their SONAME to avoid an extra layer of indirection through a symlink. Only suitable for a read-only rootfs where libraries are not upgraded in place." PACKAGECONFIG[doc] = "This variable provides a means of enabling or disabling features of a recipe on a per-recipe basis." PACKAGES[doc] = "The list of packages to be created from the recipe." PACKAGES_DYNAMIC[doc] = "A promise that your recipe satisfies runtime dependencies for optional modules that are found in other recipes." -- 2.30.2 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#159307): https://lists.openembedded.org/g/openembedded-core/message/159307 Mute This Topic: https://lists.openembedded.org/mt/87566850/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] license.bbclass does not add recommends to dynamic packages
We're using LICENSE_CREATE_PACKAGE to create ${PN}-lic package files and
relying on the automatically generated recommends to cause them to be
installed in the image. This works well for most packages, but fails for
packages where we only install package created using PACKAGES_DYNAMIC.
For example, liborc is being installed in our image but that package lacks
a recommends for orc-lic, so the licences that apply to it are not being
installed. This is because license.bbclass:add_package_and_files iterates
only over the packages listed in PACKAGES.
Steps to reproduce on current master:
$ echo 'LICENSE_CREATE_PACKAGE = "1"' >> conf/local.conf
$ bitbake orc
$ dpkg-deb -I
tmp-glibc/deploy/ipk/armv7vet2hf-neon/orc_0.4.32-r0_armv7vet2hf-neon.ipk|grep
Recommends
Recommends: orc-lic
$ dpkg-deb -I
tmp-glibc/deploy/ipk/armv7vet2hf-neon/liborc-0.4-0_0.4.32-r0_armv7vet2hf-neon.ipk|grep
Recommends
$
(I would have expected the last command to produce the same output as the
penultimate one.)
Even if I could fathom out how to fix orc and any other recipes so that they
did add the ${PN}-lic dependency, I'd be worried about not noticing that
the problem affected other recipes in the future.
Is there a way to teach license.bbclass:add_package_and_files to add the
${PN}-lic recommends for dynamic packages, or would it be necessary to
teach package.bbclass to do so?
Thanks.
Mike.
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#153640):
https://lists.openembedded.org/g/openembedded-core/message/153640
Mute This Topic: https://lists.openembedded.org/mt/84042415/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] license.bbclass does not add recommends to dynamic packages
On Wednesday 07 July 2021 at 13:25:17 +0100, Richard Purdie wrote:
> On Wed, 2021-07-07 at 12:53 +0100, Mike Crowe via lists.openembedded.org
> wrote:
> > We're using LICENSE_CREATE_PACKAGE to create ${PN}-lic package files and
> > relying on the automatically generated recommends to cause them to be
> > installed in the image. This works well for most packages, but fails for
> > packages where we only install package created using PACKAGES_DYNAMIC.
> >
> > For example, liborc is being installed in our image but that package lacks
> > a recommends for orc-lic, so the licences that apply to it are not being
> > installed. This is because license.bbclass:add_package_and_files iterates
> > only over the packages listed in PACKAGES.
> >
> > Steps to reproduce on current master:
> >
> > $ echo 'LICENSE_CREATE_PACKAGE = "1"' >> conf/local.conf
> > $ bitbake orc
> > $ dpkg-deb -I
> > tmp-glibc/deploy/ipk/armv7vet2hf-neon/orc_0.4.32-r0_armv7vet2hf-neon.ipk|grep
> > Recommends
> > Recommends: orc-lic
> > $ dpkg-deb -I
> > tmp-glibc/deploy/ipk/armv7vet2hf-neon/liborc-0.4-0_0.4.32-r0_armv7vet2hf-neon.ipk|grep
> > Recommends
> > $
> >
> > (I would have expected the last command to produce the same output as the
> > penultimate one.)
> >
> > Even if I could fathom out how to fix orc and any other recipes so that they
> > did add the ${PN}-lic dependency, I'd be worried about not noticing that
> > the problem affected other recipes in the future.
> >
> > Is there a way to teach license.bbclass:add_package_and_files to add the
> > ${PN}-lic recommends for dynamic packages, or would it be necessary to
> > teach package.bbclass to do so?
>
> That all sounds rather horrible :/.
>
> Would IMAGE_INSTALL_COMPLEMENTARY += "*-lic" work instead?
That seems to have worked well.
I wonder whether this means that it would be better to stop adding the
recommends automatically and tell users that need this to use
IMAGE_INSTALL_COMPLEMENTARY instead (either directly, or by teaching
license_image.bbclass to modify it based on another variable.)
Losing the recommends would also meaan I wouldn't need to add
--no-recommends to the image recipes that don't need the licence files.
Thanks for the speedy response!
Mike.
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#153643):
https://lists.openembedded.org/g/openembedded-core/message/153643
Mute This Topic: https://lists.openembedded.org/mt/84042415/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] license.bbclass does not add recommends to dynamic packages
On Wednesday 07 July 2021 at 17:48:20 +0100, Richard Purdie wrote:
> On Wed, 2021-07-07 at 17:43 +0100, Richard Purdie via lists.openembedded.org
> wrote:
> > On Wed, 2021-07-07 at 14:05 +0100, Mike Crowe wrote:
> > > On Wednesday 07 July 2021 at 13:25:17 +0100, Richard Purdie wrote:
> > > > On Wed, 2021-07-07 at 12:53 +0100, Mike Crowe via
> > > > lists.openembedded.org wrote:
> > > > > We're using LICENSE_CREATE_PACKAGE to create ${PN}-lic package files
> > > > > and
> > > > > relying on the automatically generated recommends to cause them to be
> > > > > installed in the image. This works well for most packages, but fails
> > > > > for
> > > > > packages where we only install package created using PACKAGES_DYNAMIC.
> > > > >
> > > > > For example, liborc is being installed in our image but that package
> > > > > lacks
> > > > > a recommends for orc-lic, so the licences that apply to it are not
> > > > > being
> > > > > installed. This is because license.bbclass:add_package_and_files
> > > > > iterates
> > > > > only over the packages listed in PACKAGES.
> > > > >
> > > > > Steps to reproduce on current master:
> > > > >
> > > > > $ echo 'LICENSE_CREATE_PACKAGE = "1"' >> conf/local.conf
> > > > > $ bitbake orc
> > > > > $ dpkg-deb -I
> > > > > tmp-glibc/deploy/ipk/armv7vet2hf-neon/orc_0.4.32-r0_armv7vet2hf-neon.ipk|grep
> > > > > Recommends
> > > > > Recommends: orc-lic
> > > > > $ dpkg-deb -I
> > > > > tmp-glibc/deploy/ipk/armv7vet2hf-neon/liborc-0.4-0_0.4.32-r0_armv7vet2hf-neon.ipk|grep
> > > > > Recommends
> > > > > $
> > > > >
> > > > > (I would have expected the last command to produce the same output as
> > > > > the
> > > > > penultimate one.)
> > > > >
> > > > > Even if I could fathom out how to fix orc and any other recipes so
> > > > > that they
> > > > > did add the ${PN}-lic dependency, I'd be worried about not noticing
> > > > > that
> > > > > the problem affected other recipes in the future.
> > > > >
> > > > > Is there a way to teach license.bbclass:add_package_and_files to add
> > > > > the
> > > > > ${PN}-lic recommends for dynamic packages, or would it be necessary to
> > > > > teach package.bbclass to do so?
> > > >
> > > > That all sounds rather horrible :/.
> > > >
> > > > Would IMAGE_INSTALL_COMPLEMENTARY += "*-lic" work instead?
> > >
> > > That seems to have worked well.
> > >
> > > I wonder whether this means that it would be better to stop adding the
> > > recommends automatically and tell users that need this to use
> > > IMAGE_INSTALL_COMPLEMENTARY instead (either directly, or by teaching
> > > license_image.bbclass to modify it based on another variable.)
> >
> > That would seem the better option to me at least...
>
> To be clear, I'd definitely support dropping that existing RRECOMMENDS code,
> I think there are better ways to handle this now. I may even write that patch
> :)
I was thinking of something like the following untested patch:
diff --git a/meta/classes/license.bbclass b/meta/classes/license.bbclass
index f7978e266b..c87473cbb8 100644
--- a/meta/classes/license.bbclass
+++ b/meta/classes/license.bbclass
@@ -63,14 +63,6 @@ def add_package_and_files(d):
# first in PACKAGES to be sure that nothing else gets
LICENSE_FILES_DIRECTORY
d.setVar('PACKAGES', "%s %s" % (pn_lic, packages))
d.setVar('FILES_' + pn_lic, files)
-for pn in packages.split():
-if pn == pn_lic:
-continue
-rrecommends_pn = d.getVar('RRECOMMENDS_' + pn)
-if rrecommends_pn:
-d.setVar('RRECOMMENDS_' + pn, "%s %s" % (pn_lic, rrecommends_pn))
-else:
-d.setVar('RRECOMMENDS_' + pn, "%s" % (pn_lic))
def copy_license_files(lic_files_paths, destdir):
import shutil
diff --git a/meta/classes/license_image.bbclass
b/meta/classes/license_image.bbclass
index 73cebb4d55..293a033855 100644
--- a/meta/classes/license_image.bbclass
+++ b/meta/classes/license_image.bbclass
@@ -279,3 +279,6 @@ python license_qa_dead_symlink() {
bb.error("broken symlink: " + full_path)
}
IMAGE_QA_COMMANDS += "license_qa_dead_symlink"
+
+IMAGE_INSTALL_LICENSES ??= "${LICENSE_CREATE_PACKAGE}"
+IMAGE_INSTALL_COMPLEMENTARY +=
"${oe.utils.conditional("IMAGE_INSTALL_LICENSES", "0", "", "*-lic", d)}"
I'm not sure whether it's safe to += an empty string onto the end of
IMAGE_INSTALL_COMPLEMENTARY, but I'm trying to avoid adding yet more
anonymous Python. :-)
Is that the sort of thing you meant?
Thanks.
Mike.
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#153657):
https://lists.openembedded.org/g/openembedded-core/message/153657
Mute This Topic: https://lists.openembedded.org/mt/84042415/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH] license: Drop adding RRECOMMENDS for license packages
On Wednesday 07 July 2021 at 18:06:07 +0100, Richard Purdie wrote: > This changes behaviour when LICENSE_CREATE_PACKAGE is in use. Packages > no longer have RRECOMMENDS adding to them. > > It was highlighted that this doesn't apply to PACKAGES_DYNAMIC, nor can > it easily be made to do so. There is also a much easier way to handle this > which is: > > IMAGE_INSTALL_COMPLEMENTARY += "*-lic" > > which works on a per image basis and doesn't change the underlying > package dependencies. I propose we switch to this instead. The downside to this is that IMAGE_INSTALL_COMPLEMENTARY isn't documented at https://www.yoctoproject.org/docs/latest/ref-manual/ref-manual.html so I'd assumed it to be an implementation detail. (Adding it to the documentation is presumably not difficult though.) Since the equivalent for "*-dbg" is to add dbg-pkgs to IMAGE_FEATURES, I think that adding: COMPLEMENTARY_GLOB[lic-pkgs] = "*-lic" to license_image.bbclass is a cleaner solution. We could make the feature imply LICENSE_CREATE_PACKAGE="1" but it feels strange for an image feature to affect the packages when they are built so perhaps that isn't a good idea. Thanks. Mike. -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#153684): https://lists.openembedded.org/g/openembedded-core/message/153684 Mute This Topic: https://lists.openembedded.org/mt/84049073/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH] license: Drop adding RRECOMMENDS for license packages
On Thursday 08 July 2021 at 09:33:12 +0100, Richard Purdie wrote:
> On Thu, 2021-07-08 at 09:26 +0100, Mike Crowe wrote:
> > On Wednesday 07 July 2021 at 18:06:07 +0100, Richard Purdie wrote:
> > > This changes behaviour when LICENSE_CREATE_PACKAGE is in use. Packages
> > > no longer have RRECOMMENDS adding to them.
> > >
> > > It was highlighted that this doesn't apply to PACKAGES_DYNAMIC, nor can
> > > it easily be made to do so. There is also a much easier way to handle this
> > > which is:
> > >
> > > IMAGE_INSTALL_COMPLEMENTARY += "*-lic"
> > >
> > > which works on a per image basis and doesn't change the underlying
> > > package dependencies. I propose we switch to this instead.
[snip]
> > Since the equivalent for "*-dbg" is to add dbg-pkgs to IMAGE_FEATURES, I
> > think that adding:
> >
> > COMPLEMENTARY_GLOB[lic-pkgs] = "*-lic"
> >
> > to license_image.bbclass is a cleaner solution.
>
> Yes, I like that.
It seems to work well for me.
> > We could make the feature
> > imply LICENSE_CREATE_PACKAGE="1" but it feels strange for an image feature
> > to affect the packages when they are built so perhaps that isn't a good
> > idea.
>
> I agree, I'm not sure that would be too helpful. Maybe a comment next to
> the line above mentioning it needs to be set? We might also be able to
> add a warning if it wasn't set but lic-pkgs was used?
The comment is easy, as is documenting the image feature. If I add
"lic-pkgs" to EXTRA_IMAGE_FEATURES but don't set LICENSE_CREATE_PACKAGE="1"
then there are no errors when building the image. I added:
--8<--
python() {
if not oe.data.typed_value('LICENSE_CREATE_PACKAGE', d):
features = set(oe.data.typed_value('IMAGE_FEATURES', d))
if 'lic-pkgs' in features:
bb.error("'lic-pkgs' in IMAGE_FEATURES but LICENSE_CREATE_PACKAGE
not enabled to generate -lic packages")
}
-->8--
to license_image.bbclass which reports the problem, but does so for every
image even if it isn't being built. An alternative would be to add a new
function to ROOTFS_PREPROCESS_COMMAND which does the check at image
creation time.
Do you have a preference, or is there a better way that I've missed?
Thanks.
Mike.
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#153689):
https://lists.openembedded.org/g/openembedded-core/message/153689
Mute This Topic: https://lists.openembedded.org/mt/84049073/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH] licence_image: Add lic-pkgs IMAGE_FEATURE
Installing license packages is similar to installing -dev or -dbg
packages, so let's invent a "lic-pkgs" IMAGE_FEATURE that does so and
document it in core-image.bbclass.
This image feature only works if LICENSE_CREATE_PACKAGE is set, so
refuse to generate an image if the lic-pkgs feature is enabled without
LICENSE_CREATE_PACKAGE.
Signed-off-by: Mike Crowe
---
meta/classes/core-image.bbclass| 2 ++
meta/classes/license_image.bbclass | 10 ++
2 files changed, 12 insertions(+)
diff --git a/meta/classes/core-image.bbclass b/meta/classes/core-image.bbclass
index d81f68bd2e..84fd3eeb38 100644
--- a/meta/classes/core-image.bbclass
+++ b/meta/classes/core-image.bbclass
@@ -31,6 +31,8 @@
# - post-install-logging
# - dev-pkgs- development packages (headers, etc.) for all
installed packages in the rootfs
# - dbg-pkgs- debug symbol packages for all installed packages in
the rootfs
+# - lic-pkgs- license packages for all installed pacakges in the
rootfs, requires
+# LICENSE_CREATE_PACKAGE="1" to be set when building
packages too
# - doc-pkgs- documentation packages for all installed packages in
the rootfs
# - bash-completion-pkgs - bash-completion packages for recipes using
bash-completion bbclass
# - ptest-pkgs - ptest packages for all ptest-enabled recipes
diff --git a/meta/classes/license_image.bbclass
b/meta/classes/license_image.bbclass
index 73cebb4d55..5dbec288a4 100644
--- a/meta/classes/license_image.bbclass
+++ b/meta/classes/license_image.bbclass
@@ -1,5 +1,15 @@
ROOTFS_LICENSE_DIR = "${IMAGE_ROOTFS}/usr/share/common-licenses"
+# This requires LICENSE_CREATE_PACKAGE=1 to work too
+COMPLEMENTARY_GLOB[lic-pkgs] = "*-lic"
+
+python() {
+if not oe.data.typed_value('LICENSE_CREATE_PACKAGE', d):
+features = set(oe.data.typed_value('IMAGE_FEATURES', d))
+if 'lic-pkgs' in features:
+bb.error("'lic-pkgs' in IMAGE_FEATURES but LICENSE_CREATE_PACKAGE
not enabled to generate -lic packages")
+}
+
python write_package_manifest() {
# Get list of installed packages
license_image_dir = d.expand('${LICENSE_DIRECTORY}/${IMAGE_NAME}')
--
2.30.2
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#153793):
https://lists.openembedded.org/g/openembedded-core/message/153793
Mute This Topic: https://lists.openembedded.org/mt/84156163/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH] license: Drop adding RRECOMMENDS for license packages
On Wednesday 07 July 2021 at 18:06:07 +0100, Richard Purdie wrote:
> This changes behaviour when LICENSE_CREATE_PACKAGE is in use. Packages
> no longer have RRECOMMENDS adding to them.
>
> It was highlighted that this doesn't apply to PACKAGES_DYNAMIC, nor can
> it easily be made to do so. There is also a much easier way to handle this
> which is:
>
> IMAGE_INSTALL_COMPLEMENTARY += "*-lic"
>
> which works on a per image basis and doesn't change the underlying
> package dependencies. I propose we switch to this instead.
>
> Signed-off-by: Richard Purdie
> ---
> meta/classes/license.bbclass | 8
> 1 file changed, 8 deletions(-)
>
> diff --git a/meta/classes/license.bbclass b/meta/classes/license.bbclass
> index f7978e266b6..c87473cbb85 100644
> --- a/meta/classes/license.bbclass
> +++ b/meta/classes/license.bbclass
> @@ -63,14 +63,6 @@ def add_package_and_files(d):
> # first in PACKAGES to be sure that nothing else gets
> LICENSE_FILES_DIRECTORY
> d.setVar('PACKAGES', "%s %s" % (pn_lic, packages))
> d.setVar('FILES_' + pn_lic, files)
> -for pn in packages.split():
> -if pn == pn_lic:
> -continue
> -rrecommends_pn = d.getVar('RRECOMMENDS_' + pn)
> -if rrecommends_pn:
> -d.setVar('RRECOMMENDS_' + pn, "%s %s" % (pn_lic, rrecommends_pn))
> -else:
> -d.setVar('RRECOMMENDS_' + pn, "%s" % (pn_lic))
>
> def copy_license_files(lic_files_paths, destdir):
> import shutil
> --
> 2.30.2
>
It turns out that this has an interesting side effect when
IMAGE_FEATURES also contains "dbg-pkgs".
If python3-core is installed in my image then dbg-pkgs causes python3-dbg
to be installed. python3-dbg recommends gdbm-dbg, so libgdbm-dbg also gets
installed.
Prior to this change, libgdbm-dbg recommends gdbm-lic, so libgdbm-lic gets
installed too.
After this change, libgdbm-dbg no longer recommends gdbm-lic and since
libgdbm itself isn't installed and the complementary globs have already
been matched libgdbm-lic is not installed.
(Other recipes that have multiple binary packages with dependencies that
aren't all installed are affected similarly.)
I don't think this really matters in general, and don't propose to change
anything, but it is a surprise if you have a check that goes through the
results of image_list_installed_packages(d) confirming that all the licence
files have been installed and it gets upset that libgdbm-lic isn't there.
Thanks.
Mike.
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#153936):
https://lists.openembedded.org/g/openembedded-core/message/153936
Mute This Topic: https://lists.openembedded.org/mt/84049073/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [dunfell][PATCH] curl: Fix CVE-2021-22924 and CVE-2021-22925
curl v7.78 contained fixes for five CVEs:
CVE-2021-22922 and CVE-2021-22923 are only present when support for
metalink is enabled. EXTRA_OECONF contains "--without-libmetalink" so
these fixes are unnecessary.
CVE-2021-22926 only affects builds for MacOS.
CVE-2021-22924 and CVE-2021-22925 are both applicable. Take the patches
from Ubuntu 20.04 curl_7.68.0-1ubuntu2.6 package which is close enough
that the patch for CVE-2021-22924 applies without conflicts. The
CVE-2021-22925 patch required only a small tweak to apply.
Signed-off-by: Mike Crowe
---
.../curl/curl/CVE-2021-22924.patch| 224 ++
.../curl/curl/CVE-2021-22925.patch| 41
meta/recipes-support/curl/curl_7.69.1.bb | 2 +
3 files changed, 267 insertions(+)
create mode 100644 meta/recipes-support/curl/curl/CVE-2021-22924.patch
create mode 100644 meta/recipes-support/curl/curl/CVE-2021-22925.patch
diff --git a/meta/recipes-support/curl/curl/CVE-2021-22924.patch
b/meta/recipes-support/curl/curl/CVE-2021-22924.patch
new file mode 100644
index 00..7f0b53842f
--- /dev/null
+++ b/meta/recipes-support/curl/curl/CVE-2021-22924.patch
@@ -0,0 +1,224 @@
+Subject: [PATCH] vtls: fix connection reuse checks for issuer cert and
+ case sensitivity CVE-2021-22924
+
+Reported-by: Harry Sintonen
+Bug: https://curl.se/docs/CVE-2021-22924.html
+Upstream-Status: backport from Ubuntu curl_7.68.0-1ubuntu2.6
+---
+ lib/url.c | 5 +++--
+ lib/urldata.h | 2 +-
+ lib/vtls/gtls.c| 10 +-
+ lib/vtls/nss.c | 4 ++--
+ lib/vtls/openssl.c | 12 ++--
+ lib/vtls/vtls.c| 23 ++-
+ 6 files changed, 35 insertions(+), 21 deletions(-)
+
+diff --git a/lib/url.c b/lib/url.c
+index 47fc66aed..eebad8d32 100644
+--- a/lib/url.c
b/lib/url.c
+@@ -3555,6 +3555,9 @@ static CURLcode create_conn(struct Curl_easy *data,
+ data->set.proxy_ssl.primary.CApath = data->set.str[STRING_SSL_CAPATH_PROXY];
+ data->set.ssl.primary.CAfile = data->set.str[STRING_SSL_CAFILE_ORIG];
+ data->set.proxy_ssl.primary.CAfile = data->set.str[STRING_SSL_CAFILE_PROXY];
++ data->set.ssl.primary.issuercert =
data->set.str[STRING_SSL_ISSUERCERT_ORIG];
++ data->set.proxy_ssl.primary.issuercert =
++data->set.str[STRING_SSL_ISSUERCERT_PROXY];
+ data->set.ssl.primary.random_file = data->set.str[STRING_SSL_RANDOM_FILE];
+ data->set.proxy_ssl.primary.random_file =
+ data->set.str[STRING_SSL_RANDOM_FILE];
+@@ -3575,8 +3578,6 @@ static CURLcode create_conn(struct Curl_easy *data,
+
+ data->set.ssl.CRLfile = data->set.str[STRING_SSL_CRLFILE_ORIG];
+ data->set.proxy_ssl.CRLfile = data->set.str[STRING_SSL_CRLFILE_PROXY];
+- data->set.ssl.issuercert = data->set.str[STRING_SSL_ISSUERCERT_ORIG];
+- data->set.proxy_ssl.issuercert = data->set.str[STRING_SSL_ISSUERCERT_PROXY];
+ data->set.ssl.cert = data->set.str[STRING_CERT_ORIG];
+ data->set.proxy_ssl.cert = data->set.str[STRING_CERT_PROXY];
+ data->set.ssl.cert_type = data->set.str[STRING_CERT_TYPE_ORIG];
+diff --git a/lib/urldata.h b/lib/urldata.h
+index fbb8b645e..615fbf369 100644
+--- a/lib/urldata.h
b/lib/urldata.h
+@@ -224,6 +224,7 @@ struct ssl_primary_config {
+ long version_max; /* max supported version the client wants to use*/
+ char *CApath; /* certificate dir (doesn't work on windows) */
+ char *CAfile; /* certificate to verify peer against */
++ char *issuercert; /* optional issuer certificate filename */
+ char *clientcert;
+ char *random_file; /* path to file containing "random" data */
+ char *egdsocket; /* path to file containing the EGD daemon socket */
+@@ -240,7 +241,6 @@ struct ssl_config_data {
+ struct ssl_primary_config primary;
+ long certverifyresult; /* result from the certificate verification */
+ char *CRLfile; /* CRL to check certificate revocation */
+- char *issuercert;/* optional issuer certificate filename */
+ curl_ssl_ctx_callback fsslctx; /* function to initialize ssl ctx */
+ void *fsslctxp;/* parameter for call back */
+ char *cert; /* client certificate file name */
+diff --git a/lib/vtls/gtls.c b/lib/vtls/gtls.c
+index 46e149c7d..8c051024f 100644
+--- a/lib/vtls/gtls.c
b/lib/vtls/gtls.c
+@@ -1059,7 +1059,7 @@ gtls_connect_step3(struct connectdata *conn,
+ if(!chainp) {
+ if(SSL_CONN_CONFIG(verifypeer) ||
+SSL_CONN_CONFIG(verifyhost) ||
+- SSL_SET_OPTION(issuercert)) {
++ SSL_CONN_CONFIG(issuercert)) {
+ #ifdef USE_TLS_SRP
+ if(SSL_SET_OPTION(authtype) == CURL_TLSAUTH_SRP
+ && SSL_SET_OPTION(username) != NULL
+@@ -1241,21 +1241,21 @@ gtls_connect_step3(struct connectdata *conn,
+gnutls_x509_crt_t format */
+ gnutls_x509_crt_import(x509_cert, chainp, GNUTLS_X509_FMT_DER);
+
+- if(SSL_SET_OPTION(issuercert)) {
++ if(SSL_CONN_CONFIG(issuercert)) {
+ gnutls_x509_crt_init(&x509_issuer);
+-issuerp = load_file(SSL_SET_OPTION(issuercert));
++issuer
[OE-core] [dunfell][PATCH v2] curl: Fix CVE-2021-22924 and CVE-2021-22925
curl v7.78 contained fixes for five CVEs:
CVE-2021-22922[1] and CVE-2021-22923[2] are only present when support
for metalink is enabled. EXTRA_OECONF contains "--without-libmetalink"
so these fixes are unnecessary.
CVE-2021-22926[3] only affects builds for MacOS.
CVE-2021-22924[4] and CVE-2021-22925[5] are both applicable. Take the
patches from Ubuntu 20.04 curl_7.68.0-1ubuntu2.6 package which is close
enough that the patch for CVE-2021-22924 applies without conflicts. The
CVE-2021-22925 patch required only a small tweak to apply.
[1] https://curl.se/docs/CVE-2021-22922.html
[2] https://curl.se/docs/CVE-2021-22923.html
[3] https://curl.se/docs/CVE-2021-22926.html
[4] https://curl.se/docs/CVE-2021-22924.html
[5] https://curl.se/docs/CVE-2021-22925.html
Signed-off-by: Mike Crowe
---
.../curl/curl/CVE-2021-22924.patch| 226 ++
.../curl/curl/CVE-2021-22925.patch| 43
meta/recipes-support/curl/curl_7.69.1.bb | 3 +
3 files changed, 272 insertions(+)
create mode 100644 meta/recipes-support/curl/curl/CVE-2021-22924.patch
create mode 100644 meta/recipes-support/curl/curl/CVE-2021-22925.patch
diff --git a/meta/recipes-support/curl/curl/CVE-2021-22924.patch
b/meta/recipes-support/curl/curl/CVE-2021-22924.patch
new file mode 100644
index 00..68fde45ddf
--- /dev/null
+++ b/meta/recipes-support/curl/curl/CVE-2021-22924.patch
@@ -0,0 +1,226 @@
+Subject: [PATCH] vtls: fix connection reuse checks for issuer cert and
+ case sensitivity CVE-2021-22924
+
+Reported-by: Harry Sintonen
+Bug: https://curl.se/docs/CVE-2021-22924.html
+CVE: CVE-2021-22924
+Upstream-Status: backport from Ubuntu curl_7.68.0-1ubuntu2.6
+Signed-off-by: Mike Crowe
+---
+ lib/url.c | 5 +++--
+ lib/urldata.h | 2 +-
+ lib/vtls/gtls.c| 10 +-
+ lib/vtls/nss.c | 4 ++--
+ lib/vtls/openssl.c | 12 ++--
+ lib/vtls/vtls.c| 23 ++-
+ 6 files changed, 35 insertions(+), 21 deletions(-)
+
+diff --git a/lib/url.c b/lib/url.c
+index 47fc66aed..eebad8d32 100644
+--- a/lib/url.c
b/lib/url.c
+@@ -3555,6 +3555,9 @@ static CURLcode create_conn(struct Curl_easy *data,
+ data->set.proxy_ssl.primary.CApath = data->set.str[STRING_SSL_CAPATH_PROXY];
+ data->set.ssl.primary.CAfile = data->set.str[STRING_SSL_CAFILE_ORIG];
+ data->set.proxy_ssl.primary.CAfile = data->set.str[STRING_SSL_CAFILE_PROXY];
++ data->set.ssl.primary.issuercert =
data->set.str[STRING_SSL_ISSUERCERT_ORIG];
++ data->set.proxy_ssl.primary.issuercert =
++data->set.str[STRING_SSL_ISSUERCERT_PROXY];
+ data->set.ssl.primary.random_file = data->set.str[STRING_SSL_RANDOM_FILE];
+ data->set.proxy_ssl.primary.random_file =
+ data->set.str[STRING_SSL_RANDOM_FILE];
+@@ -3575,8 +3578,6 @@ static CURLcode create_conn(struct Curl_easy *data,
+
+ data->set.ssl.CRLfile = data->set.str[STRING_SSL_CRLFILE_ORIG];
+ data->set.proxy_ssl.CRLfile = data->set.str[STRING_SSL_CRLFILE_PROXY];
+- data->set.ssl.issuercert = data->set.str[STRING_SSL_ISSUERCERT_ORIG];
+- data->set.proxy_ssl.issuercert = data->set.str[STRING_SSL_ISSUERCERT_PROXY];
+ data->set.ssl.cert = data->set.str[STRING_CERT_ORIG];
+ data->set.proxy_ssl.cert = data->set.str[STRING_CERT_PROXY];
+ data->set.ssl.cert_type = data->set.str[STRING_CERT_TYPE_ORIG];
+diff --git a/lib/urldata.h b/lib/urldata.h
+index fbb8b645e..615fbf369 100644
+--- a/lib/urldata.h
b/lib/urldata.h
+@@ -224,6 +224,7 @@ struct ssl_primary_config {
+ long version_max; /* max supported version the client wants to use*/
+ char *CApath; /* certificate dir (doesn't work on windows) */
+ char *CAfile; /* certificate to verify peer against */
++ char *issuercert; /* optional issuer certificate filename */
+ char *clientcert;
+ char *random_file; /* path to file containing "random" data */
+ char *egdsocket; /* path to file containing the EGD daemon socket */
+@@ -240,7 +241,6 @@ struct ssl_config_data {
+ struct ssl_primary_config primary;
+ long certverifyresult; /* result from the certificate verification */
+ char *CRLfile; /* CRL to check certificate revocation */
+- char *issuercert;/* optional issuer certificate filename */
+ curl_ssl_ctx_callback fsslctx; /* function to initialize ssl ctx */
+ void *fsslctxp;/* parameter for call back */
+ char *cert; /* client certificate file name */
+diff --git a/lib/vtls/gtls.c b/lib/vtls/gtls.c
+index 46e149c7d..8c051024f 100644
+--- a/lib/vtls/gtls.c
b/lib/vtls/gtls.c
+@@ -1059,7 +1059,7 @@ gtls_connect_step3(struct connectdata *conn,
+ if(!chainp) {
+ if(SSL_CONN_CONFIG(verifypeer) ||
+SSL_CONN_CONFIG(verifyhost) ||
+- SSL_SET_OPTION(issuercert)) {
++ SSL_CONN_CONFIG(issuercert)) {
+ #ifdef USE_TLS_SRP
+ if(SSL_SET_OPTION(authtype) == CURL_TLSAUTH_SRP
+ && SSL_SET_OPTION(username) != NULL
+@@ -1241,21 +1241,21 @@ gtls_connect_step3(struct connectdata *conn,
+
Re: [OE-core] [dunfell][PATCH] curl: Fix CVE-2021-22924 and CVE-2021-22925
On Wednesday 04 August 2021 at 06:44:51 -1000, Steve Sakoman wrote: > On Tue, Aug 3, 2021 at 10:11 PM Mike Crowe via lists.openembedded.org > wrote: > > > > curl v7.78 contained fixes for five CVEs: > > > > CVE-2021-22922 and CVE-2021-22923 are only present when support for > > metalink is enabled. EXTRA_OECONF contains "--without-libmetalink" so > > these fixes are unnecessary. > > > > CVE-2021-22926 only affects builds for MacOS. > > > > CVE-2021-22924 and CVE-2021-22925 are both applicable. Take the patches > > from Ubuntu 20.04 curl_7.68.0-1ubuntu2.6 package which is close enough > > that the patch for CVE-2021-22924 applies without conflicts. The > > CVE-2021-22925 patch required only a small tweak to apply. > > Being curious why none of these are showing up in the reports I > checked the CPE database and it seems none of them are present! So > that explains why. > > Do you know why they are missing? Perhaps a status of RESERVED? See: > > https://nvd.nist.gov/vuln/detail/CVE-2021-22923 I'm afraid that I have no idea. :( I just watch curl release announcements to assess the security impact on our products and spotted these. > Since they seem to be real issues though I can take the patch once you > send a V2 with the issue below fixed. > [ Need to have a CVE tag and your signed-off-by in both patch files. ] v2 should have arrived. I must have sneaked my previous CVE fixes through without them somehow. :) > It might make sense to whitelist the CVE's that don't apply to us so > that once the entries hit the database we will already have dealt with > them. Hopefully done. Thanks. Mike. -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#154446): https://lists.openembedded.org/g/openembedded-core/message/154446 Mute This Topic: https://lists.openembedded.org/mt/84657570/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [dunfell][PATCH v2] curl: Fix CVE-2021-22924 and CVE-2021-22925
On Wednesday 04 August 2021 at 08:05:27 -1000, Steve Sakoman wrote: > On Wed, Aug 4, 2021 at 7:27 AM Steve Sakoman via > lists.openembedded.org > wrote: > > > > On Wed, Aug 4, 2021 at 7:06 AM Mike Crowe via lists.openembedded.org > > wrote: > > > > > > curl v7.78 contained fixes for five CVEs: > > > > > > CVE-2021-22922[1] and CVE-2021-22923[2] are only present when support > > > for metalink is enabled. EXTRA_OECONF contains "--without-libmetalink" > > > so these fixes are unnecessary. > > > > > > CVE-2021-22926[3] only affects builds for MacOS. > > > > > > CVE-2021-22924[4] and CVE-2021-22925[5] are both applicable. Take the > > > patches from Ubuntu 20.04 curl_7.68.0-1ubuntu2.6 package which is close > > > enough that the patch for CVE-2021-22924 applies without conflicts. The > > > CVE-2021-22925 patch required only a small tweak to apply. > > > > > > [1] https://curl.se/docs/CVE-2021-22922.html > > > [2] https://curl.se/docs/CVE-2021-22923.html > > > [3] https://curl.se/docs/CVE-2021-22926.html > > > [4] https://curl.se/docs/CVE-2021-22924.html > > > [5] https://curl.se/docs/CVE-2021-22925.html > > > > This patch wouldn't apply because there's another curl CVE fix in my > > testing queue (curl: Fix for CVE-2021-22898): > > > > https://lists.openembedded.org/g/openembedded-core/message/154145 > > > > I went ahead and did the required fixup so no need for you to do anything. > > Sigh. I spoke too soon. Your CVE-2021-22925 patch and the previous > CVE-2021-22898 patch both touch lib/telnet.c so your patch won't apply > now. > > You mentioned that you had to tweak the CVE-2021-22925 patch, might > this be related to the CVE-2021-22898 fix (which is a one-liner)? Ah, yes. That's the change I had to accommodate. You can either tweak my patch (just adding the "== 2" to the patch should work - that's the opposite of what I did) or just drop your CVE-2021-22898 patch since the CVE-2021-22925 patch supersedes it.) Alternatively, I can do whichever of those you prefer tomorrow if you wish. Thanks. Mike. -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#154461): https://lists.openembedded.org/g/openembedded-core/message/154461 Mute This Topic: https://lists.openembedded.org/mt/84666902/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core][dunfell 07/33] curl: Fix CVE-2021-22924 and CVE-2021-22925
On Thursday 05 August 2021 at 05:33:44 -1000, Steve Sakoman wrote: > From: Mike Crowe > > curl v7.78 contained fixes for five CVEs: > > CVE-2021-22922[1] and CVE-2021-22923[2] are only present when support > for metalink is enabled. EXTRA_OECONF contains "--without-libmetalink" > so these fixes are unnecessary. > > CVE-2021-22926[3] only affects builds for MacOS. > > CVE-2021-22924[4] and CVE-2021-22925[5] are both applicable. Take the > patches from Ubuntu 20.04 curl_7.68.0-1ubuntu2.6 package which is close > enough that the patch for CVE-2021-22924 applies without conflicts.. Now that you've added back the "== 2", I believe the final sentence is now true for both patches. That may not be worth worrying about. > > [1] https://curl.se/docs/CVE-2021-22922.html > [2] https://curl.se/docs/CVE-2021-22923.html > [3] https://curl.se/docs/CVE-2021-22926.html > [4] https://curl.se/docs/CVE-2021-22924.html > [5] https://curl.se/docs/CVE-2021-22925.html > > Signed-off-by: Mike Crowe > Signed-off-by: Steve Sakoman Mike. -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#154567): https://lists.openembedded.org/g/openembedded-core/message/154567 Mute This Topic: https://lists.openembedded.org/mt/84687225/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] pseudo: Outdated records for newly-ignored paths in database cause mismatches
Our CI Dunfell builds started failing during image creation with pseudo
aborts like:
path mismatch [2 links]: ino 123107550 db
'/.../build/tmp-glibc/work/mymachine-oe-linux/myimage/1.0-r2/oe-rootfs-repo/mymachine/mypackage-dbg_1.0-r7_mymachine.ipk'
req '/.../build/mymachine-root/usr/bin'.
Inode 123107550 is the second of the two paths.
We're using the latest pseudo (b988b0a6b8afd8d459bc9a2528e834f63a3d59b2)
because we ran into problems sharing sstate cache between different build
OS versions prior to oe-core:d7e87a5851d717da047f552be394d5712efa0402.
The mismatches started happening just after we took
oe-core:9463be2292b942a1072eea1b9644e55aadb9 (as
b04d7a7aed5b05e8561029c5e570206ac9b9fa4e for Dunfell):
index 459d872b4a..244f5bb8ff 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -180,6 +180,8 @@ LINGUAS_INSTALL ?= "${@" ".join(map(lambda s:
"locale-base-%s" % s, d.getVar('IM
# aren't yet available.
PSEUDO_PASSWD = "${IMAGE_ROOTFS}:${STAGING_DIR_NATIVE}"
+PSEUDO_IGNORE_PATHS .=
",${WORKDIR}/intercept_scripts,${WORKDIR}/oe-rootfs-repo"
+
I was able to reproduce a similar problem by commenting out the above
PSEUDO_IGNORE_PATHS line, building and image, putting it back and forcing
do_rootfs for the image to run again without any intervening cleaning. It
didn't happen every time though.
I believe that the pseudo database was populated with many paths in
oe-rootfs-repo before this change. After the change, the files in
oe-rootfs-repo were replaced which freed up their inodes, but because the
paths were ignored the database wasn't updated. Those inodes were
then used for files and directories in during rootfs creation. Pseudo
incorrectly believed that these inodes were already associated with files
it knew about based on the out-of-date database records.
Cleaning the work directory makes the problem go away because that deletes
the pseudo databases.
Does the above make sense as an explanation for these errors? If so, is
there a good way to avoid these errors?
Could pseudo check whether mismatched paths are now ignored and if so not
treat the mismatch as fatal?
Should changing PSEUDO_IGNORE_PATHS cause all tasks for the recipe to be
re-run so that the out-of-date database is removed?
Even if it's not worth employing some technical measure, perhaps this is
worth mentioning as a potential false alarm at
https://wiki.yoctoproject.org/wiki/Pseudo_Abort ?
Thanks.
Mike.
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154617):
https://lists.openembedded.org/g/openembedded-core/message/154617
Mute This Topic: https://lists.openembedded.org/mt/84766871/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] pseudo: Outdated records for newly-ignored paths in database cause mismatches
On Monday 09 August 2021 at 09:09:16 -0500, Seebs wrote:
> On Mon, 9 Aug 2021 13:19:51 +0100
> "Mike Crowe via lists.openembedded.org"
> wrote:
>
> > Cleaning the work directory makes the problem go away because that
> > deletes the pseudo databases.
> >
> > Does the above make sense as an explanation for these errors? If so,
> > is there a good way to avoid these errors?
>
> Good diagnostic work, makes sense to me. It would make some sense for
> pseudo to ignore mismatches involving ignored paths, but it wasn't
> originally designed with the ignored paths concept, so it currently
> doesn't.
Thanks for the review.
I have a test case and patch for pseudo (attached) to detect newly-ignored
paths and warn rather than abort on them, but I'm not really convinced that
it is the right solution. Ideally the errant entry would be removed from
the database too in order to avoid having to continue to consult the ignore
list.
It's not even clear to me that oe-core continuing to use an existing pseudo
database after the value of PSEUDO_CLIENT_IGNORE_PATH changes is a sane
thing to expect to work. Perhaps we could just arrange to force a whole new
work directory in that case?
Thanks.
Mike.
>From e81aeff391148280d76609e5782bf7f0a115f72e Mon Sep 17 00:00:00 2001
From: Mike Crowe
Date: Wed, 11 Aug 2021 15:55:55 +0100
Subject: [PATCH] pseudo: Path mismatch on now-ignored path should not be fatal
If a database survives from before a change to PSEUDO_CLIENT_IGNORE_PATH
then there's a risk that the now-ignored files have been deleted and
their inodes re-used without pseudo noticing. Such files are reported as
path mismatches which provoke aborts.
Let's check to see whether the database filename would now be ignored,
and if so just warn about the mismatch rather than aborting.
Unfortunately the test case for this doesn't fit into the existing
infrastructure since the server must be restarted during the test.
Signed-off-by: Mike Crowe
---
pseudo.c | 2 +
run_tests.sh | 16
.../standalone-test-newly-ignored-mismatch.sh | 41 +++
3 files changed, 59 insertions(+)
create mode 100755 test/standalone-test-newly-ignored-mismatch.sh
diff --git a/pseudo.c b/pseudo.c
index 528fe1b..30b0a36 100644
--- a/pseudo.c
+++ b/pseudo.c
@@ -695,6 +695,8 @@ pseudo_op(pseudo_msg_t *msg, const char *program, const char *tag, char **respon
*/
pseudo_debug(PDBGF_FILE, "inode mismatch for '%s' -- old one was marked for deletion.\n",
msg->path);
+} else if (path_by_ino && pseudo_client_ignore_path(path_by_ino)) {
+ pseudo_diag("path mismatch on now-ignored '%s'", path_by_ino);
} else {
pseudo_diag("path mismatch [%d link%s]: ino %llu db '%s' req '%s'.\n",
msg->nlink,
diff --git a/run_tests.sh b/run_tests.sh
index c637c27..a0b8675 100755
--- a/run_tests.sh
+++ b/run_tests.sh
@@ -48,5 +48,21 @@ do
fi
rm -rf var/pseudo/*
done
+for file in test/standalone-test*.sh
+do
+filename=${file#test/}
+let num_tests++
+mkdir -p var/pseudo
+$file ${opt_verbose}
+if [ "$?" -eq "0" ]; then
+let num_passed_tests++
+if [ "${opt_verbose}" == "-v" ]; then
+echo "${filename%.sh}: Passed."
+fi
+else
+echo "${filename/%.sh}: Failed."
+fi
+rm -rf var/pseudo/*
+done
echo "${num_passed_tests}/${num_tests} test(s) passed."
diff --git a/test/standalone-test-newly-ignored-mismatch.sh b/test/standalone-test-newly-ignored-mismatch.sh
new file mode 100755
index 000..bf7d5f7
--- /dev/null
+++ b/test/standalone-test-newly-ignored-mismatch.sh
@@ -0,0 +1,41 @@
+#!/bin/bash
+#
+# SPDX-License-Identifier: LGPL-2.1-only
+#
+export PSEUDO_PREFIX=${PWD}
+pseudo=bin/pseudo
+
+trap "rm -rf testdir" EXIT
+rm -rf testdir
+mkdir testdir || exit 1
+
+mkdir -p testdir/to-be-ignored
+mkdir -p testdir/not-ignored
+
+create_files() {
+for i in a b c d e f g h i j; do
+ for j in 0 1 2 3 4 5 6 7 8 9; do
+ touch testdir/to-be-ignored/$i$j
+ done
+done
+}
+
+test_results() {
+for i in a b c d e f g h i j; do
+ for j in 0 1 2 3 4 5 6 7 8 9; do
+ touch testdir/not-ignored/$i$j
+ done
+done
+}
+
+export -f create_files
+export -f test_results
+
+export PSEUDO_IGNORE_PATHS=/initial
+
+$pseudo /bin/bash -c "create_files"
+rm testdir/to-be-ignored/*
+
+# Kill server so that we can change the value of PSEUDO_IGNORE_PATHS
+$pseudo -S
+PSEUDO_IGNORE_PATHS=${PWD}/testdir/to-be-ignored $pseudo /bin/bash -c "test_results"
--
2.30.2
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154721):
https://lists.openembedded.org/g/openembedded-core/message/154721
Mute This Topic: https://lists.openembedded.org/mt/84766871/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [pseudo][PATCH] fcntl: Add support for F_GETPIPE_SZ
When running the test suite on my Debian 11 box I see many occurrences of: unknown fcntl argument 1032, assuming long argument. (for example from test-execl.sh.) It appears that this is F_GETPIPE_SZ and it takes no arguments. Let's add it to avoid the warning messages. I could add F_SETPIPE_SZ too, but that apparently takes an int argument which would mean moving the va_arg call into the switch statement. :( Signed-off-by: Mike Crowe --- ports/linux/guts/fcntl.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/ports/linux/guts/fcntl.c b/ports/linux/guts/fcntl.c index 434c7f3..e9c6140 100644 --- a/ports/linux/guts/fcntl.c +++ b/ports/linux/guts/fcntl.c @@ -37,6 +37,9 @@ case F_GETOWN: case F_GETSIG: case F_GETLEASE: +#if defined(F_GETPIPE_SZ) +case F_GETPIPE_SZ: +#endif rc = real_fcntl(fd, cmd); break; /* long argument */ -- 2.30.2 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#154723): https://lists.openembedded.org/g/openembedded-core/message/154723 Mute This Topic: https://lists.openembedded.org/mt/84819589/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH] python3: use monotonic clock for condvar if possible
On Wednesday 11 August 2021 at 13:36:23 +0200, Alexander Kanavin wrote: > I too do not think this is sufficiently explained. All of python ptests > pass, so there needs to be a demonstrator of incorrect behavior, or let's > just revert it. I agree regarding the lack of explanation. However, even if the problem is real (which it looks like it is based on https://bugs.python.org/issue41710) then it wouldn't be expected to cause test failures unless the system clock was being warped at inconvenient times during the tests. Thanks. Mike. -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#154724): https://lists.openembedded.org/g/openembedded-core/message/154724 Mute This Topic: https://lists.openembedded.org/mt/84632397/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [pseudo][PATCH] makewrappers: Handle parameters marked as nonnull
On Monday 17 May 2021 at 21:25:06 +0200, Philip Lorenz wrote: > Commit 60e25a36558f1f07dcce1a044fe976b475bec42b started dereferencing > the "path" parameter which for some functions is annotated with the > "nonnull" attribute. While the commit explicitly checks for NULL > pointers before dereferencing it, GCC (at optimization level 1 and > above) removes the check due to the "nonnull" attribute being set for > some parameters in the glibc headers (e.g. statx()). > > However, the statx() man page explicitly allows calling with NULL > pointers (in which case the EFAULT is returned) and this behaviour is > used in the wild (e.g. in Rust) to determine whether the statx() system > call is supported. > > Disabling the optimization is not possible ([1]) so prevent the compiler > optimization by referencing the parameter in a noop inline assembly > instruction instead. > > [1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100404 > > Signed-off-by: Philip Lorenz > --- > Makefile.in| 2 ++ > makewrappers | 6 +- > test/test-fstat.c | 29 + > test/test-fstat.sh | 8 > 4 files changed, 44 insertions(+), 1 deletion(-) > create mode 100644 test/test-fstat.c > create mode 100755 test/test-fstat.sh > > diff --git a/Makefile.in b/Makefile.in > index cf13010..10441ef 100644 > --- a/Makefile.in > +++ b/Makefile.in > @@ -77,6 +77,8 @@ all: $(LIBPSEUDO) $(PSEUDO) $(PSEUDODB) $(PSEUDOLOG) > $(PSEUDO_PROFILE) > test: all | $(BIN) $(LIB) $(LOCALSTATE) > $(CC) $(CFLAGS) $(CFLAGS_PSEUDO) -o test/test-rename-fstat > test/test-rename-fstat.c > $(CC) $(CFLAGS) $(CFLAGS_PSEUDO) -o test/test-openat test/test-openat.c > + $(CC) $(CFLAGS) $(CFLAGS_PSEUDO) -o test/test-statx test/test-statx.c The test/test-stax.c file wasn't included in the patch, which means that "make test" no longer works on the current pseudo oe-core branch. :( Perhaps this went unnoticed since it's necessary to run configure again for the Makefile to be regenerated to see the problem. Do you still have the file, or should I just submit a change to remove this line? Thanks. Mike. -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#154725): https://lists.openembedded.org/g/openembedded-core/message/154725 Mute This Topic: https://lists.openembedded.org/mt/82895298/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH] python3: use monotonic clock for condvar if possible
On Wednesday 11 August 2021 at 18:32:31 +0200, Alexander Kanavin wrote: > On Wed, 11 Aug 2021 at 18:14, Mike Crowe wrote: > > I agree regarding the lack of explanation. However, even if the problem is > > real (which it looks like it is based on > > https://bugs.python.org/issue41710) > > then it wouldn't be expected to cause test failures unless the system clock > > was being warped at inconvenient times during the tests. > > So can we just patch python to use the right glibc function with the right > clock type, and send the patch upstream then? Much better than the > workaround that was merged. In theory, yes. I've no idea how hard that is though. It probably requires adding build-time detection of whether the new functions are available to avoid breaking other/older target operating systems. Mike. -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#154729): https://lists.openembedded.org/g/openembedded-core/message/154729 Mute This Topic: https://lists.openembedded.org/mt/84632397/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [pseudo][PATCH] test-openat: Consider device as well as inode number
It just so happens that my /home/mac and /home directories have the same
inode number but on different filesystems.
This means that test-openat fails with "Recursion failed!" even when run
without pseudo.
Let's consider both the device number and the inode number before
assuming that we've found the same directory again.
Signed-off-by: Mike Crowe
---
test/test-openat.c | 5 -
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/test/test-openat.c b/test/test-openat.c
index b710285..df6655a 100644
--- a/test/test-openat.c
+++ b/test/test-openat.c
@@ -25,11 +25,13 @@ int main () {
int fd, dir_fd;
struct stat st;
ino_t ino;
+dev_t dev;
char *path;
fd = openat(AT_FDCWD, ".", O_DIRECTORY, 0);
fstat(fd, &st);
ino = st.st_ino;
+dev = st.st_dev;
while (1) {
path = path_of(fd);
@@ -37,7 +39,7 @@ int main () {
dir_fd = openat(fd, "../", O_DIRECTORY, 0);
fstat(dir_fd, &st);
-if (st.st_ino == ino) {
+if (st.st_ino == ino && st.st_dev == dev) {
if (strcmp(path, "/") == 0) {
//puts("Reached top of tree");
return 0;
@@ -49,6 +51,7 @@ int main () {
free (path);
ino = st.st_ino;
+dev = st.st_dev;
fd = dir_fd;
}
return 0;
--
2.30.2
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154730):
https://lists.openembedded.org/g/openembedded-core/message/154730
Mute This Topic: https://lists.openembedded.org/mt/84821456/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [pseudo][PATCH] test: Add missing test-statx test case
On Wednesday 11 August 2021 at 18:41:32 +0200, Philip Lorenz wrote:
> Adding this test case was erroneously omitted in
> 7c722296879906fe093e1e7c4b7537e150d492cd.
>
> Signed-off-by: Philip Lorenz
> ---
> test/test-statx.c | 20
> test/test-statx.sh | 6 ++
> 2 files changed, 26 insertions(+)
> create mode 100644 test/test-statx.c
> create mode 100755 test/test-statx.sh
>
> diff --git a/test/test-statx.c b/test/test-statx.c
> new file mode 100644
> index 000..06d86af
> --- /dev/null
> +++ b/test/test-statx.c
> @@ -0,0 +1,20 @@
> +/*
> + * Test that passing NULL to a parameter marked as nonnull works correctly
> + * SPDX-License-Identifier: LGPL-2.1-only
> + *
> + */
> +#define _GNU_SOURCE
> +
> +#include
> +#include
> +#include
> +
> +// Passing a null pointer is the test scenario
> +#pragma GCC diagnostic ignored "-Wnonnull"
> +
> +int main(void) {
> +if (statx(0, NULL, 0, 0, NULL) != -1) {
> +return 1;
> +}
> +return 0;
> +}
> diff --git a/test/test-statx.sh b/test/test-statx.sh
> new file mode 100755
> index 000..77d0302
> --- /dev/null
> +++ b/test/test-statx.sh
> @@ -0,0 +1,6 @@
> +#!/bin/bash
> +#
> +# SPDX-License-Identifier: LGPL-2.1-only
> +#
> +
> +exec ./test/test-statx
> --
> 2.32.0
>
>
>
>
Applying this patch (along with <169a4fe272bb166c.28...@lists.openembedded.org>)
makes all the tests pass for me. However, the run of test-statx.sh causes:
couldn't allocate absolute path for '(null)'.
to be printed.
Thanks.
Mike.
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154731):
https://lists.openembedded.org/g/openembedded-core/message/154731
Mute This Topic: https://lists.openembedded.org/mt/84820658/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [pseudo][PATCH] fcntl: Add support for F_GETPIPE_SZ
On Wednesday 11 August 2021 at 22:38:23 +0100, Richard Purdie wrote: > On Wed, 2021-08-11 at 16:58 +0100, Mike Crowe via lists.openembedded.org > wrote: > > When running the test suite on my Debian 11 box I see many occurrences > > of: > > > > unknown fcntl argument 1032, assuming long argument. > > > > (for example from test-execl.sh.) > > > > It appears that this is F_GETPIPE_SZ and it takes no arguments. Let's > > add it to avoid the warning messages. > > > > I could add F_SETPIPE_SZ too, but that apparently takes an int argument > > which would mean moving the va_arg call into the switch statement. :( > > > > Signed-off-by: Mike Crowe > > --- > > ports/linux/guts/fcntl.c | 3 +++ > > 1 file changed, 3 insertions(+) > > > > diff --git a/ports/linux/guts/fcntl.c b/ports/linux/guts/fcntl.c > > index 434c7f3..e9c6140 100644 > > --- a/ports/linux/guts/fcntl.c > > +++ b/ports/linux/guts/fcntl.c > > @@ -37,6 +37,9 @@ > > case F_GETOWN: > > case F_GETSIG: > > case F_GETLEASE: > > +#if defined(F_GETPIPE_SZ) > > +case F_GETPIPE_SZ: > > +#endif > > rc = real_fcntl(fd, cmd); > > break; > > /* long argument */ > > This goes make to making pseudo host specific which will break > uninative/sstate. > We'll probably have to add a define if it isn't defined to get the behaviour > we > need. Good point. I should have realised that. > It is probably only a question of time before some program is using > F_SETPIPE_SZ > too :( OK. I'll work on a new patch that supports F_GETPIPE_SZ and F_SETPIPE_SZ without relying on the constants being available. Thanks! Mike. -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#154735): https://lists.openembedded.org/g/openembedded-core/message/154735 Mute This Topic: https://lists.openembedded.org/mt/84819589/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [pseudo][PATCH] fcntl: Add support for F_GETPIPE_SZ
On Thursday 12 August 2021 at 08:46:08 +0100, Mike Crowe via lists.openembedded.org wrote: > On Wednesday 11 August 2021 at 22:38:23 +0100, Richard Purdie wrote: > > On Wed, 2021-08-11 at 16:58 +0100, Mike Crowe via lists.openembedded.org > > wrote: > > > When running the test suite on my Debian 11 box I see many occurrences > > > of: > > > > > > unknown fcntl argument 1032, assuming long argument. > > > > > > (for example from test-execl.sh.) > > > > > > It appears that this is F_GETPIPE_SZ and it takes no arguments. Let's > > > add it to avoid the warning messages. > > > > > > I could add F_SETPIPE_SZ too, but that apparently takes an int argument > > > which would mean moving the va_arg call into the switch statement. :( > > > > > > Signed-off-by: Mike Crowe > > > --- > > > ports/linux/guts/fcntl.c | 3 +++ > > > 1 file changed, 3 insertions(+) > > > > > > diff --git a/ports/linux/guts/fcntl.c b/ports/linux/guts/fcntl.c > > > index 434c7f3..e9c6140 100644 > > > --- a/ports/linux/guts/fcntl.c > > > +++ b/ports/linux/guts/fcntl.c > > > @@ -37,6 +37,9 @@ > > > case F_GETOWN: > > > case F_GETSIG: > > > case F_GETLEASE: > > > +#if defined(F_GETPIPE_SZ) > > > +case F_GETPIPE_SZ: > > > +#endif > > > rc = real_fcntl(fd, cmd); > > > break; > > > /* long argument */ > > > > This goes make to making pseudo host specific which will break > > uninative/sstate. > > We'll probably have to add a define if it isn't defined to get the > > behaviour we > > need. > > Good point. I should have realised that. Having said that, why doesn't that concern apply to F_DUPFD_CLOEXEC too? Do we not care about systems that are so old that they don't have F_DUPFD_CLOEXEC? In which case, why other with the #ifdef at all? Similarly for F_OFD_GETLK and friends. Mike. -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#154761): https://lists.openembedded.org/g/openembedded-core/message/154761 Mute This Topic: https://lists.openembedded.org/mt/84819589/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [pseudo][PATCH v2] fcntl: Add support for fcntl F_GETPIPE_SZ and F_SETPIPE_SZ
When running the test suite on my Debian 11 box I see many occurrences
of:
unknown fcntl argument 1032, assuming long argument.
(for example from test-execl.sh.)
It appears that this is F_GETPIPE_SZ and it takes no arguments. Let's
add it and the corresponding F_SETPIPE_SZ too to avoid the warning
messages.
F_SETPIPE_SZ accepts an int argument, which strictly speaking isn't the
same as the long that the wrapper expects. However, this is also true
for F_DUPFD which seems to be working correctly on all the targets that
people care about.
We need to define the command constants if the system headers don't
provide them to ensure that a binary built on an old system works
without the new commands works correctly only a newer one that tries to
use them. If the system values differ from the expected ones then such a
binary would also be incompatible, so fail the build in that case too.
Signed-off-by: Mike Crowe
---
Makefile.in | 1 +
ports/linux/guts/fcntl.c | 21 +++
test/test-fcntl.c| 58
test/test-fcntl.sh | 5
4 files changed, 85 insertions(+)
create mode 100644 test/test-fcntl.c
create mode 100755 test/test-fcntl.sh
diff --git a/Makefile.in b/Makefile.in
index 10441ef..4ebe5da 100644
--- a/Makefile.in
+++ b/Makefile.in
@@ -79,6 +79,7 @@ test: all | $(BIN) $(LIB) $(LOCALSTATE)
$(CC) $(CFLAGS) $(CFLAGS_PSEUDO) -o test/test-openat test/test-openat.c
$(CC) $(CFLAGS) $(CFLAGS_PSEUDO) -o test/test-statx test/test-statx.c
$(CC) $(CFLAGS) $(CFLAGS_PSEUDO) -o test/test-fstat test/test-fstat.c
+ $(CC) $(CFLAGS) $(CFLAGS_PSEUDO) -o test/test-fcntl test/test-fcntl.c
@./run_tests.sh -v
install-lib: $(LIBPSEUDO)
diff --git a/ports/linux/guts/fcntl.c b/ports/linux/guts/fcntl.c
index 434c7f3..ffb50be 100644
--- a/ports/linux/guts/fcntl.c
+++ b/ports/linux/guts/fcntl.c
@@ -8,6 +8,22 @@
* wrap_fcntl(int fd, int cmd, ...struct flock *lock) {
* int rc = -1;
*/
+#if !defined(F_GETPIPE_SZ)
+#define F_GETPIPE_SZ (1032)
+#endif
+
+#if F_GETPIPE_SZ != 1032
+#error System F_GETPIPE_SZ has unexpected value
+#endif
+
+#if !defined(F_SETPIPE_SZ)
+#define F_SETPIPE_SZ (1031)
+#endif
+
+#if F_SETPIPE_SZ != 1031
+#error System F_SETPIPE_SZ has unexpected value
+#endif
+
long arg;
int save_errno;
@@ -31,12 +47,17 @@
}
errno = save_errno;
break;
+ case F_SETPIPE_SZ:
+ /* actually do something */
+ rc = real_fcntl(fd, cmd, arg);
+ break;
/* no argument: */
case F_GETFD:
case F_GETFL:
case F_GETOWN:
case F_GETSIG:
case F_GETLEASE:
+ case F_GETPIPE_SZ:
rc = real_fcntl(fd, cmd);
break;
/* long argument */
diff --git a/test/test-fcntl.c b/test/test-fcntl.c
new file mode 100644
index 000..b593d50
--- /dev/null
+++ b/test/test-fcntl.c
@@ -0,0 +1,58 @@
+/* fcntl-linux.h doesn't define F_GETPIPE_SZ and F_SETPIPE_SZ without
+ * this */
+#define _GNU_SOURCE
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+int test_pipe_sz()
+{
+#if defined(F_GETPIPE_SZ) && defined(F_SETPIPE_SZ)
+ int pipefd[2];
+
+ if (pipe(pipefd) < 0) {
+ perror("pipe");
+ return 1;
+ }
+
+ const int orig_size = fcntl(pipefd[0], F_GETPIPE_SZ);
+ if (orig_size < 0) {
+ perror("F_GETPIPE_SZ");
+ return 1;
+ }
+
+ const int new_size = orig_size * 2;
+
+ if (fcntl(pipefd[0], F_SETPIPE_SZ, new_size) < 0) {
+ perror("F_SETPIPE_SZ");
+ return 1;
+ }
+
+ const int final_size = fcntl(pipefd[0], F_GETPIPE_SZ);
+ if (final_size < 0) {
+ perror("Second F_GETPIPE_SZ");
+ return 1;
+ }
+
+ if (final_size < new_size) {
+ fprintf(stderr, "Unexpected final pipe size: %d\n", final_size);
+ return 1;
+ }
+#else
+ printf("Host too old for F_GETPIPE_SZ and F_SETPIPE_SZ tests\n");
+#endif
+ return 0;
+}
+
+int main()
+{
+ int result = 0;
+ result += test_pipe_sz();
+ return result;
+}
diff --git a/test/test-fcntl.sh b/test/test-fcntl.sh
new file mode 100755
index 000..7112620
--- /dev/null
+++ b/test/test-fcntl.sh
@@ -0,0 +1,5 @@
+#!/bin/bash
+#
+# SPDX-License-Identifier: LGPL-2.1-only
+#
+./test/test-fcntl
--
2.30.2
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#154762):
https://lists.openembedded.org/g/openembedded-core/message/154762
Mute This Topic: https://lists.openembedded.org/mt/84860960/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [RFC][PATCH] default-distrovars.inc: Set BBINCLUDELOGS to empty to disable printing failed task output multiple times
On Friday 03 September 2021 at 13:55:21 +0200, Martin Jansa wrote: > * the output is shown 3 times with default configuration and 5 times when > --verbose > is being used with knotty, there might be other use-cases where we actually > need > this, but until the logging is resolved better, setting this to empty looks > like > more reasonable option (considering that e.g. log.do_compile from > chromium-x11 > can be over 50MB long, generating 150MB+ cooker log) > > * more details in: > https://bugzilla.yoctoproject.org/show_bug.cgi?id=14542 > > Signed-off-by: Martin Jansa > --- > meta/conf/distro/include/default-distrovars.inc | 6 -- > meta/conf/documentation.conf| 2 +- > 2 files changed, 5 insertions(+), 3 deletions(-) > > diff --git a/meta/conf/distro/include/default-distrovars.inc > b/meta/conf/distro/include/default-distrovars.inc > index f91df632d5..5035508b98 100644 > --- a/meta/conf/distro/include/default-distrovars.inc > +++ b/meta/conf/distro/include/default-distrovars.inc > @@ -33,8 +33,10 @@ BB_GENERATE_MIRROR_TARBALLS ??= "0" > > NO32LIBS ??= "1" > > -# Default to emitting logfiles if a build fails. > -BBINCLUDELOGS ??= "yes" > +# Default logger already emits logfiles if a build fails, setting this to > any non-empty value would just include more copies (prefixed with "|") in the > output > +# https://bugzilla.yoctoproject.org/show_bug.cgi?id=14542 > +BBINCLUDELOGS ??= "" > + > SDK_VERSION ??= "nodistro.0" > DISTRO_VERSION ??= "nodistro.0" > > diff --git a/meta/conf/documentation.conf b/meta/conf/documentation.conf > index c5a38b0764..f4697e28d0 100644 > --- a/meta/conf/documentation.conf > +++ b/meta/conf/documentation.conf > @@ -88,7 +88,7 @@ BBFILE_COLLECTIONS[doc] = "Lists the names of configured > layers. These names are > BBFILE_PATTERN[doc] = "Variable that expands to match files from BBFILES in > a particular layer. This variable is used in the layer.conf file and must be > suffixed with the name of a layer." > BBFILE_PRIORITY[doc] = "Assigns the priority for recipe files in each layer. > Setting this variable allows you to prioritize a layer against other layers > that contain the same recipe." > BBFILES[doc] = "List of recipe files used by BitBake to build software." > -BBINCLUDELOGS[doc] = "Variable that controls how BitBake displays logs on > build failure." > +BBINCLUDELOGS[doc] = "Variable that controls how BitBake displays logs on > build failure. Set to empty if you don't want to have 2nd copy of failed task > output (prefixed with '|') in the cooker log." > BBINCLUDELOGS_LINES[doc] = "Amount of log lines printed on failure." > BBLAYERS[doc] = "Lists the layers to enable during the build. This variable > is defined in the bblayers.conf configuration file." > BBMASK[doc] = "Prevents BitBake from processing specific recipes or recipe > append files." > -- > 2.32.0 This patch (when applied to Dunfell) appears to solve the problem for me. I now only get a single copy of the log output in both the output from bitbake and the cooker log. Whether it's the right solution or not I cannot say, though. :) Thanks! Mike. -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#155651): https://lists.openembedded.org/g/openembedded-core/message/155651 Mute This Topic: https://lists.openembedded.org/mt/85349515/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [pseudo][PATCH] test-openat: Consider device as well as inode number
On Wednesday 11 August 2021 at 18:13:54 +0100, Mike Crowe via
lists.openembedded.org wrote:
> It just so happens that my /home/mac and /home directories have the same
> inode number but on different filesystems.
>
> This means that test-openat fails with "Recursion failed!" even when run
> without pseudo.
>
> Let's consider both the device number and the inode number before
> assuming that we've found the same directory again.
>
> Signed-off-by: Mike Crowe
> ---
> test/test-openat.c | 5 -
> 1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/test/test-openat.c b/test/test-openat.c
> index b710285..df6655a 100644
> --- a/test/test-openat.c
> +++ b/test/test-openat.c
> @@ -25,11 +25,13 @@ int main () {
> int fd, dir_fd;
> struct stat st;
> ino_t ino;
> +dev_t dev;
> char *path;
>
> fd = openat(AT_FDCWD, ".", O_DIRECTORY, 0);
> fstat(fd, &st);
> ino = st.st_ino;
> +dev = st.st_dev;
>
> while (1) {
> path = path_of(fd);
> @@ -37,7 +39,7 @@ int main () {
>
> dir_fd = openat(fd, "../", O_DIRECTORY, 0);
> fstat(dir_fd, &st);
> -if (st.st_ino == ino) {
> +if (st.st_ino == ino && st.st_dev == dev) {
> if (strcmp(path, "/") == 0) {
> //puts("Reached top of tree");
> return 0;
> @@ -49,6 +51,7 @@ int main () {
>
> free (path);
> ino = st.st_ino;
> +dev = st.st_dev;
> fd = dir_fd;
> }
> return 0;
> --
> 2.30.2
>
Ping!
Mike.
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#155721):
https://lists.openembedded.org/g/openembedded-core/message/155721
Mute This Topic: https://lists.openembedded.org/mt/84821456/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [pseudo][PATCH v2] fcntl: Add support for fcntl F_GETPIPE_SZ and F_SETPIPE_SZ
On Friday 13 August 2021 at 12:05:09 +0100, Mike Crowe via
lists.openembedded.org wrote:
> When running the test suite on my Debian 11 box I see many occurrences
> of:
>
> unknown fcntl argument 1032, assuming long argument.
>
> (for example from test-execl.sh.)
>
> It appears that this is F_GETPIPE_SZ and it takes no arguments. Let's
> add it and the corresponding F_SETPIPE_SZ too to avoid the warning
> messages.
>
> F_SETPIPE_SZ accepts an int argument, which strictly speaking isn't the
> same as the long that the wrapper expects. However, this is also true
> for F_DUPFD which seems to be working correctly on all the targets that
> people care about.
>
> We need to define the command constants if the system headers don't
> provide them to ensure that a binary built on an old system works
> without the new commands works correctly only a newer one that tries to
> use them. If the system values differ from the expected ones then such a
> binary would also be incompatible, so fail the build in that case too.
>
> Signed-off-by: Mike Crowe
> ---
> Makefile.in | 1 +
> ports/linux/guts/fcntl.c | 21 +++
> test/test-fcntl.c| 58
> test/test-fcntl.sh | 5
> 4 files changed, 85 insertions(+)
> create mode 100644 test/test-fcntl.c
> create mode 100755 test/test-fcntl.sh
>
> diff --git a/Makefile.in b/Makefile.in
> index 10441ef..4ebe5da 100644
> --- a/Makefile.in
> +++ b/Makefile.in
> @@ -79,6 +79,7 @@ test: all | $(BIN) $(LIB) $(LOCALSTATE)
> $(CC) $(CFLAGS) $(CFLAGS_PSEUDO) -o test/test-openat test/test-openat.c
> $(CC) $(CFLAGS) $(CFLAGS_PSEUDO) -o test/test-statx test/test-statx.c
> $(CC) $(CFLAGS) $(CFLAGS_PSEUDO) -o test/test-fstat test/test-fstat.c
> + $(CC) $(CFLAGS) $(CFLAGS_PSEUDO) -o test/test-fcntl test/test-fcntl.c
> @./run_tests.sh -v
>
> install-lib: $(LIBPSEUDO)
> diff --git a/ports/linux/guts/fcntl.c b/ports/linux/guts/fcntl.c
> index 434c7f3..ffb50be 100644
> --- a/ports/linux/guts/fcntl.c
> +++ b/ports/linux/guts/fcntl.c
> @@ -8,6 +8,22 @@
> * wrap_fcntl(int fd, int cmd, ...struct flock *lock) {
> * int rc = -1;
> */
> +#if !defined(F_GETPIPE_SZ)
> +#define F_GETPIPE_SZ (1032)
> +#endif
> +
> +#if F_GETPIPE_SZ != 1032
> +#error System F_GETPIPE_SZ has unexpected value
> +#endif
> +
> +#if !defined(F_SETPIPE_SZ)
> +#define F_SETPIPE_SZ (1031)
> +#endif
> +
> +#if F_SETPIPE_SZ != 1031
> +#error System F_SETPIPE_SZ has unexpected value
> +#endif
> +
> long arg;
> int save_errno;
>
> @@ -31,12 +47,17 @@
> }
> errno = save_errno;
> break;
> + case F_SETPIPE_SZ:
> + /* actually do something */
> + rc = real_fcntl(fd, cmd, arg);
> + break;
> /* no argument: */
> case F_GETFD:
> case F_GETFL:
> case F_GETOWN:
> case F_GETSIG:
> case F_GETLEASE:
> + case F_GETPIPE_SZ:
> rc = real_fcntl(fd, cmd);
> break;
> /* long argument */
> diff --git a/test/test-fcntl.c b/test/test-fcntl.c
> new file mode 100644
> index 000..b593d50
> --- /dev/null
> +++ b/test/test-fcntl.c
> @@ -0,0 +1,58 @@
> +/* fcntl-linux.h doesn't define F_GETPIPE_SZ and F_SETPIPE_SZ without
> + * this */
> +#define _GNU_SOURCE
> +
> +#include
> +#include
> +#include
> +#include
> +#include
> +#include
> +#include
> +#include
> +
> +int test_pipe_sz()
> +{
> +#if defined(F_GETPIPE_SZ) && defined(F_SETPIPE_SZ)
> + int pipefd[2];
> +
> + if (pipe(pipefd) < 0) {
> + perror("pipe");
> + return 1;
> + }
> +
> + const int orig_size = fcntl(pipefd[0], F_GETPIPE_SZ);
> + if (orig_size < 0) {
> + perror("F_GETPIPE_SZ");
> + return 1;
> + }
> +
> + const int new_size = orig_size * 2;
> +
> + if (fcntl(pipefd[0], F_SETPIPE_SZ, new_size) < 0) {
> + perror("F_SETPIPE_SZ");
> + return 1;
> + }
> +
> + const int final_size = fcntl(pipefd[0], F_GETPIPE_SZ);
> + if (final_size < 0) {
> + perror("Second F_GETPIPE_SZ");
> + return 1;
> + }
> +
> + if (final_size < new_size) {
> + fprintf(stderr, "Unexpected final pipe size: %d\n", final_size);
> + return 1;
> + }
> +#else
> + printf("Host too old for F_GETPIPE_SZ and F_SETPIPE_SZ tests\n");
Re: [OE-core] [pseudo][PATCH v2] fcntl: Add support for fcntl F_GETPIPE_SZ and F_SETPIPE_SZ
On Monday 06 September 2021 at 17:37:14 +0100, Mike Crowe via lists.openembedded.org wrote: > On Friday 13 August 2021 at 12:05:09 +0100, Mike Crowe via > lists.openembedded.org wrote: > > When running the test suite on my Debian 11 box I see many occurrences > > of: > > > > unknown fcntl argument 1032, assuming long argument. > > > > (for example from test-execl.sh.) > > > > It appears that this is F_GETPIPE_SZ and it takes no arguments. Let's > > add it and the corresponding F_SETPIPE_SZ too to avoid the warning > > messages. > > > > F_SETPIPE_SZ accepts an int argument, which strictly speaking isn't the > > same as the long that the wrapper expects. However, this is also true > > for F_DUPFD which seems to be working correctly on all the targets that > > people care about. > > > > We need to define the command constants if the system headers don't > > provide them to ensure that a binary built on an old system works > > without the new commands works correctly only a newer one that tries to > > use them. If the system values differ from the expected ones then such a > > binary would also be incompatible, so fail the build in that case too. > > > > Signed-off-by: Mike Crowe > > --- > > Makefile.in | 1 + > > ports/linux/guts/fcntl.c | 21 +++ > > test/test-fcntl.c| 58 > > test/test-fcntl.sh | 5 > > 4 files changed, 85 insertions(+) > > create mode 100644 test/test-fcntl.c > > create mode 100755 test/test-fcntl.sh Hi Richard & Seebs, It looks like only part of this change landed as 328452d74917ce9314c8c4afe2bd277473a4c076: | Makefile.in | 1 + | ports/linux/guts/fcntl.c | 21 + This means that the tests no longer pass: cc: error: test/test-fcntl.c: No such file or directory It looks like something similar happened with test-statx leading to https://lists.openembedded.org/g/openembedded-core/message/154797 so I wonder whether there's some sort of tooling fault somewhere? Thanks. Mike. -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#155931): https://lists.openembedded.org/g/openembedded-core/message/155931 Mute This Topic: https://lists.openembedded.org/mt/84860960/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [dunfell][PATCH] curl: Fix CVE-2021-22946 and CVE-22947, whitelist CVE-2021-22945
curl v7.79.0 contained fixes for three CVEs:
The description of CVE-2021-22945[1] contains:
> This flaw was introduced in commit 2522903b79 but since MQTT support
> was marked 'experimental' then and not enabled in the build by default
> until curl 7.73.0 (October 14, 2020) we count that as the first flawed
> version.
which I believe means that curl v7.69.1 is not vulnerable.
curl v7.69.1 is vulnerable to both CVE-2021-22946[2] and CVE-22947[3].
These patches are from Ubuntu 20.04's curl 7.68.0 package. The patches
applied without conflicts, but I used devtool to regenerate them to
avoid fuzz warnings.
[1] https://curl.se/docs/CVE-2021-22945.html
[2] https://curl.se/docs/CVE-2021-22946.html
[3] https://curl.se/docs/CVE-2021-22947.html
Signed-off-by: Mike Crowe
---
.../curl/curl/CVE-2021-22946-pre1.patch | 86 +
.../curl/curl/CVE-2021-22946.patch| 328
.../curl/curl/CVE-2021-22947.patch| 352 ++
meta/recipes-support/curl/curl_7.69.1.bb | 5 +-
4 files changed, 770 insertions(+), 1 deletion(-)
create mode 100644 meta/recipes-support/curl/curl/CVE-2021-22946-pre1.patch
create mode 100644 meta/recipes-support/curl/curl/CVE-2021-22946.patch
create mode 100644 meta/recipes-support/curl/curl/CVE-2021-22947.patch
I kept the fix for 22946 as two separate patches because that's what
Ubuntu had. I can roll them together into a single patch if it is
preferred.
diff --git a/meta/recipes-support/curl/curl/CVE-2021-22946-pre1.patch
b/meta/recipes-support/curl/curl/CVE-2021-22946-pre1.patch
new file mode 100644
index 00..4afd755149
--- /dev/null
+++ b/meta/recipes-support/curl/curl/CVE-2021-22946-pre1.patch
@@ -0,0 +1,86 @@
+Backport of:
+
+From 1397a7de6e312e019a3b339f855ba0a5cafa9127 Mon Sep 17 00:00:00 2001
+From: Daniel Stenberg
+Date: Mon, 21 Sep 2020 09:15:51 +0200
+Subject: [PATCH] ftp: separate FTPS from FTP over "HTTPS proxy"
+
+When using HTTPS proxy, SSL is used but not in the view of the FTP
+protocol handler itself so separate the connection's use of SSL from the
+FTP control connection's sue.
+
+Reported-by: Mingtao Yang
+Fixes #5523
+Closes #6006
+
+Upstream-Status: backport from 7.68.0-1ubuntu2.7
+Signed-off-by: Mike Crowe
+---
+ lib/ftp.c | 13 ++---
+ lib/urldata.h | 1 +
+ 2 files changed, 7 insertions(+), 7 deletions(-)
+
+diff --git a/lib/ftp.c b/lib/ftp.c
+index 3382772..677527f 100644
+--- a/lib/ftp.c
b/lib/ftp.c
+@@ -2488,7 +2488,7 @@ static CURLcode ftp_state_loggedin(struct connectdata
*conn)
+ {
+ CURLcode result = CURLE_OK;
+
+- if(conn->ssl[FIRSTSOCKET].use) {
++ if(conn->bits.ftp_use_control_ssl) {
+ /* PBSZ = PROTECTION BUFFER SIZE.
+
+ The 'draft-murray-auth-ftp-ssl' (draft 12, page 7) says:
+@@ -2633,11 +2633,8 @@ static CURLcode ftp_statemach_act(struct connectdata
*conn)
+ }
+ #endif
+
+- if(data->set.use_ssl &&
+- (!conn->ssl[FIRSTSOCKET].use ||
+- (conn->bits.proxy_ssl_connected[FIRSTSOCKET] &&
+- !conn->proxy_ssl[FIRSTSOCKET].use))) {
+-/* We don't have a SSL/TLS connection yet, but FTPS is
++ if(data->set.use_ssl && !conn->bits.ftp_use_control_ssl) {
++/* We don't have a SSL/TLS control connection yet, but FTPS is
+requested. Try a FTPS connection now */
+
+ ftpc->count3 = 0;
+@@ -2682,6 +2679,7 @@ static CURLcode ftp_statemach_act(struct connectdata
*conn)
+ result = Curl_ssl_connect(conn, FIRSTSOCKET);
+ if(!result) {
+ conn->bits.ftp_use_data_ssl = FALSE; /* clear-text data */
++ conn->bits.ftp_use_control_ssl = TRUE; /* SSL on control */
+ result = ftp_state_user(conn);
+ }
+ }
+@@ -3072,7 +3070,7 @@ static CURLcode ftp_block_statemach(struct connectdata
*conn)
+ *
+ */
+ static CURLcode ftp_connect(struct connectdata *conn,
+- bool *done) /* see description above */
++bool *done) /* see description above */
+ {
+ CURLcode result;
+ struct ftp_conn *ftpc = &conn->proto.ftpc;
+@@ -3093,6 +3091,7 @@ static CURLcode ftp_connect(struct connectdata *conn,
+ result = Curl_ssl_connect(conn, FIRSTSOCKET);
+ if(result)
+ return result;
++conn->bits.ftp_use_control_ssl = TRUE;
+ }
+
+ Curl_pp_init(pp); /* init the generic pingpong data */
+diff --git a/lib/urldata.h b/lib/urldata.h
+index ff2d686..d1fb4a9 100644
+--- a/lib/urldata.h
b/lib/urldata.h
+@@ -461,6 +461,7 @@ struct ConnectBits {
+ EPRT doesn't work we disable it for the forthcoming
+ requests */
+ BIT(ftp_use_data_ssl); /* Enabled SSL for the data connection */
++ BIT(ftp_use_control_ssl); /* Enabled SSL for the control connection */
+ #endif
+ BIT(netrc); /* name+password provided by netrc */
+ BIT(userpwd_in_url); /* name+password found in url */
diff --git a/meta/recipes-support/curl/curl/CVE-2021-22946
Re: [OE-core] [dunfell][PATCH] curl: Fix CVE-2021-22946 and CVE-22947, whitelist CVE-2021-22945
Of course, the subject line ought to say CVE-2021-22947 rather than
CVE-22947. :(
Mike.
On Friday 17 September 2021 at 17:14:33 +0100, Mike Crowe via
lists.openembedded.org wrote:
> curl v7.79.0 contained fixes for three CVEs:
>
> The description of CVE-2021-22945[1] contains:
> > This flaw was introduced in commit 2522903b79 but since MQTT support
> > was marked 'experimental' then and not enabled in the build by default
> > until curl 7.73.0 (October 14, 2020) we count that as the first flawed
> > version.
>
> which I believe means that curl v7.69.1 is not vulnerable.
>
> curl v7.69.1 is vulnerable to both CVE-2021-22946[2] and CVE-22947[3].
> These patches are from Ubuntu 20.04's curl 7.68.0 package. The patches
> applied without conflicts, but I used devtool to regenerate them to
> avoid fuzz warnings.
>
> [1] https://curl.se/docs/CVE-2021-22945.html
> [2] https://curl.se/docs/CVE-2021-22946.html
> [3] https://curl.se/docs/CVE-2021-22947.html
>
> Signed-off-by: Mike Crowe
> ---
> .../curl/curl/CVE-2021-22946-pre1.patch | 86 +
> .../curl/curl/CVE-2021-22946.patch| 328
> .../curl/curl/CVE-2021-22947.patch| 352 ++
> meta/recipes-support/curl/curl_7.69.1.bb | 5 +-
> 4 files changed, 770 insertions(+), 1 deletion(-)
> create mode 100644 meta/recipes-support/curl/curl/CVE-2021-22946-pre1.patch
> create mode 100644 meta/recipes-support/curl/curl/CVE-2021-22946.patch
> create mode 100644 meta/recipes-support/curl/curl/CVE-2021-22947.patch
>
> I kept the fix for 22946 as two separate patches because that's what
> Ubuntu had. I can roll them together into a single patch if it is
> preferred.
>
> diff --git a/meta/recipes-support/curl/curl/CVE-2021-22946-pre1.patch
> b/meta/recipes-support/curl/curl/CVE-2021-22946-pre1.patch
> new file mode 100644
> index 00..4afd755149
> --- /dev/null
> +++ b/meta/recipes-support/curl/curl/CVE-2021-22946-pre1.patch
> @@ -0,0 +1,86 @@
> +Backport of:
> +
> +From 1397a7de6e312e019a3b339f855ba0a5cafa9127 Mon Sep 17 00:00:00 2001
> +From: Daniel Stenberg
> +Date: Mon, 21 Sep 2020 09:15:51 +0200
> +Subject: [PATCH] ftp: separate FTPS from FTP over "HTTPS proxy"
> +
> +When using HTTPS proxy, SSL is used but not in the view of the FTP
> +protocol handler itself so separate the connection's use of SSL from the
> +FTP control connection's sue.
> +
> +Reported-by: Mingtao Yang
> +Fixes #5523
> +Closes #6006
> +
> +Upstream-Status: backport from 7.68.0-1ubuntu2.7
> +Signed-off-by: Mike Crowe
> +---
> + lib/ftp.c | 13 ++---
> + lib/urldata.h | 1 +
> + 2 files changed, 7 insertions(+), 7 deletions(-)
> +
> +diff --git a/lib/ftp.c b/lib/ftp.c
> +index 3382772..677527f 100644
> +--- a/lib/ftp.c
> b/lib/ftp.c
> +@@ -2488,7 +2488,7 @@ static CURLcode ftp_state_loggedin(struct connectdata
> *conn)
> + {
> + CURLcode result = CURLE_OK;
> +
> +- if(conn->ssl[FIRSTSOCKET].use) {
> ++ if(conn->bits.ftp_use_control_ssl) {
> + /* PBSZ = PROTECTION BUFFER SIZE.
> +
> + The 'draft-murray-auth-ftp-ssl' (draft 12, page 7) says:
> +@@ -2633,11 +2633,8 @@ static CURLcode ftp_statemach_act(struct connectdata
> *conn)
> + }
> + #endif
> +
> +- if(data->set.use_ssl &&
> +- (!conn->ssl[FIRSTSOCKET].use ||
> +- (conn->bits.proxy_ssl_connected[FIRSTSOCKET] &&
> +- !conn->proxy_ssl[FIRSTSOCKET].use))) {
> +-/* We don't have a SSL/TLS connection yet, but FTPS is
> ++ if(data->set.use_ssl && !conn->bits.ftp_use_control_ssl) {
> ++/* We don't have a SSL/TLS control connection yet, but FTPS is
> +requested. Try a FTPS connection now */
> +
> + ftpc->count3 = 0;
> +@@ -2682,6 +2679,7 @@ static CURLcode ftp_statemach_act(struct connectdata
> *conn)
> + result = Curl_ssl_connect(conn, FIRSTSOCKET);
> + if(!result) {
> + conn->bits.ftp_use_data_ssl = FALSE; /* clear-text data */
> ++ conn->bits.ftp_use_control_ssl = TRUE; /* SSL on control */
> + result = ftp_state_user(conn);
> + }
> + }
> +@@ -3072,7 +3070,7 @@ static CURLcode ftp_block_statemach(struct connectdata
> *conn)
> + *
> + */
> + static CURLcode ftp_connect(struct connectdata *conn,
> +- bool *done) /* see description above */
> ++bool *done) /* see description above */
> + {
> + CURLcode result;
> + struct ftp_conn *ftpc = &conn->proto.f
Re: [OE-core] [PATCH][dunfell] zlib: backport the fix for CVE-2018-25032
On Wednesday 13 April 2022 at 06:02:22 -1000, Steve Sakoman wrote: > Both runs completed and I'm still seeing success without the zlib patch: > > https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/5069 > > and failure with the patch: > > https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/5070 I'm certainly no expert with the autobuilder, but it looks like nothing was actually compiled for both of those builds - everything came from the sstate cache. I believe that Ralph's reproduction of the test failure without the zlib patch was from a complete rebuild without anything coming from the sstate cache. I suspect that if a PR bump or something similar that causes zlib and all its reverse dependencies to be built were tested on top of the commit used for build 5069 then the test failure would occur then as well and exonerate the zlib patch. Mike. -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#164342): https://lists.openembedded.org/g/openembedded-core/message/164342 Mute This Topic: https://lists.openembedded.org/mt/90107518/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH] sanity: skip make 4.2.1 warning for debian
On Wednesday 27 April 2022 at 11:40:49 +0100, Richard Purdie wrote:
> As far as I know, we don't use LSB_DISTRO_ADJUST in core at all. I suspect it
> should really probably be added to the lsb.py function in most cases. Is there
> any documentation or other info about when it should be applied and when it
> should not?
We used to use LSB_DISTRO_ADJUST to stop Debian minor release upgrades
causing sstate paths to change up until about five years ago when we
upgraded to an oe-core version that only considered the major version to be
important.
> I did have a look at
> https://git.yoctoproject.org/poky/commit/meta/classes/base.bbclass?id=096306ecd1bb80fe5e732584caca0172305628a2
> where it was introduced 10 years ago but there isn't much more info.
I believe that we use the mapping in SSTATE_MIRRORS to do the equivalent of
what is described in that commit message:
SSTATE_MIRRORS ?= "\
file://debian ${OUR_SSTATE_DIR}/debian-10 \n \
file://debian-11 ${OUR_SSTATE_DIR}/debian-10 \n \
file://debian-10 ${OUR_SSTATE_DIR}/debian-10 \n \
\
file://debian-9 ${OUR_SSTATE_DIR}/debian-9 \n \
file://.* ${OUR_SSTATE_DIR}/PATH \n \
"
(Our autobuilders are running Debian 10 at the moment, so anyone running
Debian 11 can make use of sstate files they wrote to debian-10, but anyone
running Debian 9 cannot.)
I have no idea whether this works for the Red Hat world though.
HTH.
Mike.
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#164922):
https://lists.openembedded.org/g/openembedded-core/message/164922
Mute This Topic: https://lists.openembedded.org/mt/90716488/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
