[OE-core] [PATCH v2] sstate.bbclass: fix errors about read-only sstate mirrors
From: Michael Ho
If a read-only sstate mirror is used in conjunction with hash equiv,
then OSError will be raised when an sstate-cache hit is achieved.
This is because sstate_task_postfunc will try to "touch" the symlinks
that point to the read-only sstate mirror when sstate_report_unihash
has changed SSTATE_PKG.
This commit adds an additional exception handler to silently mask read
only rootfs errors thrown during the touch.
The fix is also duplicated to sstate_eventhandler as the code is very
similar but it may not be needed there.
Example of the error:
File: 'exec_python_func() autogenerated', lineno: 2, function:
0001:
*** 0002:sstate_task_postfunc(d)
0003:
File: '/poky/meta/classes/sstate.bbclass', lineno: 774, function:
sstate_task_postfunc
0770:
0771:omask = os.umask(0o002)
0772:if omask != 0o002:
0773: bb.note("Using umask 0o002 (not %0o) for sstate packaging" %
omask)
*** 0774:sstate_package(shared_state, d)
0775:os.umask(omask)
0776:
0777:sstateinst = d.getVar("SSTATE_INSTDIR")
0778:d.setVar('SSTATE_FIXMEDIR', shared_state['fixmedir'])
File: '/poky/meta/classes/sstate.bbclass', lineno: 703, function: sstate_package
0699:if not os.path.exists(siginfo):
0700:bb.siggen.dump_this_task(siginfo, d)
0701:else:
0702:try:
*** 0703:os.utime(siginfo, None)
0704:except PermissionError:
0705:pass
0706:
0707:return
Exception: OSError: [Errno 30] Read-only file system
Signed-off-by: Michael Ho
---
meta/classes/sstate.bbclass | 8
1 file changed, 8 insertions(+)
diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
index 3a3f7cc24b..72eebd70c9 100644
--- a/meta/classes/sstate.bbclass
+++ b/meta/classes/sstate.bbclass
@@ -703,6 +703,10 @@ def sstate_package(ss, d):
os.utime(siginfo, None)
except PermissionError:
pass
+except OSError as e:
+# Handle read-only file systems gracefully
+if e.errno != errno.EROFS:
+raise e
return
@@ -1145,6 +1149,10 @@ python sstate_eventhandler() {
os.utime(siginfo, None)
except PermissionError:
pass
+except OSError as e:
+# Handle read-only file systems gracefully
+if e.errno != errno.EROFS:
+raise e
}
--
2.25.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#153351):
https://lists.openembedded.org/g/openembedded-core/message/153351
Mute This Topic: https://lists.openembedded.org/mt/83840814/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH] sstate.bbclass: fix errors about read-only sstate mirrors
From: Michael Ho
If a read-only sstate mirror is used in conjunction with hash equiv,
then OSError will be raised when an sstate-cache hit is achieved.
This is because sstate_task_postfunc will try to "touch" the symlinks
that point to the read-only sstate mirror when sstate_report_unihash
has changed SSTATE_PKG.
This commit adds an additional exception handler to silently mask read
only rootfs errors thrown during the touch.
The fix is also duplicated to sstate_eventhandler as the code is very
similar but it may not be needed there.
Example of the error:
File: 'exec_python_func() autogenerated', lineno: 2, function:
0001:
*** 0002:sstate_task_postfunc(d)
0003:
File: '/poky/meta/classes/sstate.bbclass', lineno: 774, function:
sstate_task_postfunc
0770:
0771:omask = os.umask(0o002)
0772:if omask != 0o002:
0773: bb.note("Using umask 0o002 (not %0o) for sstate packaging" %
omask)
*** 0774:sstate_package(shared_state, d)
0775:os.umask(omask)
0776:
0777:sstateinst = d.getVar("SSTATE_INSTDIR")
0778:d.setVar('SSTATE_FIXMEDIR', shared_state['fixmedir'])
File: '/poky/meta/classes/sstate.bbclass', lineno: 703, function: sstate_package
0699:if not os.path.exists(siginfo):
0700:bb.siggen.dump_this_task(siginfo, d)
0701:else:
0702:try:
*** 0703:os.utime(siginfo, None)
0704:except PermissionError:
0705:pass
0706:
0707:return
Exception: OSError: [Errno 30] Read-only file system
---
meta/classes/sstate.bbclass | 8
1 file changed, 8 insertions(+)
diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass
index 3a3f7cc24b..163f97495c 100644
--- a/meta/classes/sstate.bbclass
+++ b/meta/classes/sstate.bbclass
@@ -703,6 +703,10 @@ def sstate_package(ss, d):
os.utime(siginfo, None)
except PermissionError:
pass
+except OSError as e:
+# Handle read-only file systems gracefully
+if not e.errno == 30:
+raise e
return
@@ -1145,6 +1149,10 @@ python sstate_eventhandler() {
os.utime(siginfo, None)
except PermissionError:
pass
+except OSError as e:
+# Handle read-only file systems gracefully
+if not e.errno == 30:
+raise e
}
--
2.25.1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#153256):
https://lists.openembedded.org/g/openembedded-core/message/153256
Mute This Topic: https://lists.openembedded.org/mt/83757817/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH] rootfs.py: expose prepared_index and ipk_repo_workdir for OpkgPM
Hi RP,
Thanks for the comments. Sounds good to me. I'll send another patchset.
I'll see if I can also set up a renaming patch for the other IPKG variables for
review comments.
Kind regards,
Michael
--
BMW Car IT GmbH
Michael Ho
Spezialist Entwicklung – Build and Release Engineering
Lise-Meitner-Straße 14
89081 Ulm
Tel.: +49-731-37804-071
Mobil: +49-152-54980-471
Fax: +49-731-37804-001
Mail: michael...@bmw-carit.de
Web: http://www.bmw-carit.de <http://www.bmw-carit.de/>
-
BMW Car IT GmbH
Geschäftsführer: Kai-Uwe Balszuweit und Michael Böttrich
Sitz und Registergericht: München HRB 134810
-
On 05.02.21, 13:33, "Richard Purdie"
wrote:
On Fri, 2021-02-05 at 09:51 +0100, Michael Ho wrote:
> From: Michael Ho
>
> The OpkgPM class has several constructor options that allows for skipping
> the ipk repository set up and controlling the ipk repository directory.
This
> commit exposes these option via the OpkgRootfs class so others can make
use
> of them if they wish to.
>
> Adds the bitbake variables "IPKGREPO_TARGET_PREPARED" and
"IPKGREPO_TARGET"
> to act as the interfaces.
>
> Signed-off-by: Michael Ho
> ---
> meta/lib/oe/package_manager/ipk/rootfs.py | 14 --
> 1 file changed, 12 insertions(+), 2 deletions(-)
>
> diff --git a/meta/lib/oe/package_manager/ipk/rootfs.py
b/meta/lib/oe/package_manager/ipk/rootfs.py
> index 26dbee6..6313b72 100644
> --- a/meta/lib/oe/package_manager/ipk/rootfs.py
> +++ b/meta/lib/oe/package_manager/ipk/rootfs.py
> @@ -129,6 +129,14 @@ class PkgRootfs(DpkgOpkgRootfs):
> self.manifest = PkgManifest(d, manifest_dir)
> self.opkg_conf = self.d.getVar("IPKGCONF_TARGET")
> self.pkg_archs = self.d.getVar("ALL_MULTILIB_PACKAGE_ARCHS")
> +self.opkg_repo = self.d.getVar("IPKGREPO_TARGET")
> +self.opkg_repo_prepared =
self.d.getVar("IPKGREPO_TARGET_PREPARED")
> +
> +kwargs = {}
> +if self.opkg_repo:
> +kwargs["ipk_repo_workdir"] = self.opkg_repo
> +if self.opkg_repo_prepared:
> +kwargs["prepare_index"] = False
>
>
It seems reasonable to expose them, I'd just suggest that we don't put
them in self here but do something like:
+if self.d.getVar("IPKGREPO_TARGET"):
+kwargs["ipk_repo_workdir"] = self.opkg_repo
+if self.d.getVar("IPKGREPO_TARGET_PREPARED"):
+kwargs["prepare_index"] = False
and my only other concern is the naming. Somehow we need to get from
the mess we're currently in to some kind of more consistent and
understandable naming of variables.
OPKG_ROOTFS_FEED_WORKDIR
OPKG_ROOTFS_REBUILDINDEX
maybe?
I'd also yes, we should document them.
Cheers,
Richard
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#147700):
https://lists.openembedded.org/g/openembedded-core/message/147700
Mute This Topic: https://lists.openembedded.org/mt/80401750/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH] rootfs.py: expose prepared_index and ipk_repo_workdir for OpkgPM
From: Michael Ho
The OpkgPM class has several constructor options that allows for skipping
the ipk repository set up and controlling the ipk repository directory. This
commit exposes these option via the OpkgRootfs class so others can make use
of them if they wish to.
Adds the bitbake variables "IPKGREPO_TARGET_PREPARED" and "IPKGREPO_TARGET"
to act as the interfaces.
Signed-off-by: Michael Ho
---
meta/lib/oe/package_manager/ipk/rootfs.py | 14 --
1 file changed, 12 insertions(+), 2 deletions(-)
diff --git a/meta/lib/oe/package_manager/ipk/rootfs.py
b/meta/lib/oe/package_manager/ipk/rootfs.py
index 26dbee6..6313b72 100644
--- a/meta/lib/oe/package_manager/ipk/rootfs.py
+++ b/meta/lib/oe/package_manager/ipk/rootfs.py
@@ -129,6 +129,14 @@ class PkgRootfs(DpkgOpkgRootfs):
self.manifest = PkgManifest(d, manifest_dir)
self.opkg_conf = self.d.getVar("IPKGCONF_TARGET")
self.pkg_archs = self.d.getVar("ALL_MULTILIB_PACKAGE_ARCHS")
+self.opkg_repo = self.d.getVar("IPKGREPO_TARGET")
+self.opkg_repo_prepared = self.d.getVar("IPKGREPO_TARGET_PREPARED")
+
+kwargs = {}
+if self.opkg_repo:
+kwargs["ipk_repo_workdir"] = self.opkg_repo
+if self.opkg_repo_prepared:
+kwargs["prepare_index"] = False
self.inc_opkg_image_gen = self.d.getVar('INC_IPK_IMAGE_GEN') or ""
if self._remove_old_rootfs():
@@ -136,12 +144,14 @@ class PkgRootfs(DpkgOpkgRootfs):
self.pm = OpkgPM(d,
self.image_rootfs,
self.opkg_conf,
- self.pkg_archs)
+ self.pkg_archs,
+ **kwargs)
else:
self.pm = OpkgPM(d,
self.image_rootfs,
self.opkg_conf,
- self.pkg_archs)
+ self.pkg_archs,
+ **kwargs)
self.pm.recover_packaging_data()
bb.utils.remove(self.d.getVar('MULTILIB_TEMP_ROOTFS'), True)
--
2.7.4
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#147685):
https://lists.openembedded.org/g/openembedded-core/message/147685
Mute This Topic: https://lists.openembedded.org/mt/80401750/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH] RFC: rootfs.py: expose prepared_index and ipk_repo_workdir for OpkgPM
From: Michael Ho Hi, The OpkgPM class has two arguments for advanced control over the ipk repository creation/use that I would like to make use of from outside of OpkgRootfs (which wraps OpkgPM). This is for purposes beyond just standard image generation so it's more of an advanced interface that is not intended for normal use. In this commit, I've added the variables IPKGREPO_TARGET and IPKGREPO_TARGET_PREPARED as interfaces to forward on to as ipk_repo_workdir and prepare_index when calling the OpkgPM constructor. Is this something that could be suitable for upstreaming or not likely? And if so, should I send a patch to document these variables or leave them undocumented like other IPKG variables (eg. IPKGCONF_TARGET). Thanks! Michael Ho (1): rootfs.py: expose prepared_index and ipk_repo_workdir for OpkgPM meta/lib/oe/package_manager/ipk/rootfs.py | 14 -- 1 file changed, 12 insertions(+), 2 deletions(-) -- 2.7.4 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#147684): https://lists.openembedded.org/g/openembedded-core/message/147684 Mute This Topic: https://lists.openembedded.org/mt/80401746/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH] license_image: Introduce SKIP_LICENSE_MANIFEST_RECIPES variable to prevent from trying to use license recipeinfo file from image recipes
(Sorry, didn't read your patch correctly, it already included my change).
I guess I mean should my change (not dep[0] == pn) be removed and yours
take its place as yours should cover that case.
Kind regards,
Michael Ho
--
BMW Car IT GmbH
Michael Ho
Spezialist Entwicklung – Build and Release Engineering
Lise-Meitner-Straße 14
89081 Ulm
Tel.: +49-731-37804-071
Mobil: +49-152-54980-471
Fax: +49-731-37804-001
Mail: michael...@bmw-carit.de
Web: http://www.bmw-carit.de <http://www.bmw-carit.de/>
-
BMW Car IT GmbH
Geschäftsführer: Kai-Uwe Balszuweit und Michael Böttrich
Sitz und Registergericht: München HRB 134810
-
On 04.02.21, 14:45, "openembedded-core@lists.openembedded.org on behalf of
bounce+67612+147659+4454854+8786...@lists.openembedded.org"
wrote:
Hi,
I submitted something similar awhile back to this mailing list.
https://lists.openembedded.org/g/openembedded-core/message/146607?p=,,,20,0,0,0::relevance,,Michael.Ho%40bmw.de,20,2,0,79619287
If yours makes more sense (I didn't think there's ever a case where an image
recipe actually has a license) then my patch should probably be skipped over
for yours.
Kind regards,
Michael Ho
--
BMW Car IT GmbH
Michael Ho
Spezialist Entwicklung – Build and Release Engineering
Lise-Meitner-Straße 14
89081 Ulm
Tel.: +49-731-37804-071
Mobil: +49-152-54980-471
Fax: +49-731-37804-001
Mail: michael...@bmw-carit.de
Web: http://www.bmw-carit.de <http://www.bmw-carit.de/>
-
BMW Car IT GmbH
Geschäftsführer: Kai-Uwe Balszuweit und Michael Böttrich
Sitz und Registergericht: München HRB 134810
-
On 04.02.21, 13:11, "openembedded-core@lists.openembedded.org on behalf of
Tomasz Dziendzielski" wrote:
If image recipe depends on another image recipe it will try to use
license recipeinfo that is not deployed. It will result in:
| Exception: FileNotFoundError: [Errno 2] No such file or directory:
| 'TMPDIR/deploy/licenses/foo-image/recipeinfo'
We can't determine in a simple and efficient way if dependency recipe
inherits image.bbclass, so let's introduce SKIP_LICENSE_MANIFEST_RECIPES
variable that will contain list of recipes that should be skipped.
Signed-off-by: Tomasz Dziendzielski
---
meta/classes/license_image.bbclass | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/meta/classes/license_image.bbclass
b/meta/classes/license_image.bbclass
index 8fd88cfb2d..e2f3178306 100644
--- a/meta/classes/license_image.bbclass
+++ b/meta/classes/license_image.bbclass
@@ -223,7 +223,8 @@ def get_deployed_dependencies(d):
pn = d.getVar("PN", True)
depends = list(set([dep[0] for dep
in list(taskdata.values())
-if not dep[0].endswith("-native") and not dep[0]
== pn]))
+if not dep[0].endswith("-native") and not dep[0]
== pn
+and dep[0] not in
(d.getVar("SKIP_LICENSE_MANIFEST_RECIPES", True) or "").split()]))
# To verify what was deployed it checks the rootfs dependencies
against
# the SSTATE_MANIFESTS for "deploy" task.
--
2.30.0
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#147660):
https://lists.openembedded.org/g/openembedded-core/message/147660
Mute This Topic: https://lists.openembedded.org/mt/80377785/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH] license_image: Introduce SKIP_LICENSE_MANIFEST_RECIPES variable to prevent from trying to use license recipeinfo file from image recipes
Hi,
I submitted something similar awhile back to this mailing list.
https://lists.openembedded.org/g/openembedded-core/message/146607?p=,,,20,0,0,0::relevance,,Michael.Ho%40bmw.de,20,2,0,79619287
If yours makes more sense (I didn't think there's ever a case where an image
recipe actually has a license) then my patch should probably be skipped over
for yours.
Kind regards,
Michael Ho
--
BMW Car IT GmbH
Michael Ho
Spezialist Entwicklung – Build and Release Engineering
Lise-Meitner-Straße 14
89081 Ulm
Tel.: +49-731-37804-071
Mobil: +49-152-54980-471
Fax: +49-731-37804-001
Mail: michael...@bmw-carit.de
Web: http://www.bmw-carit.de <http://www.bmw-carit.de/>
-
BMW Car IT GmbH
Geschäftsführer: Kai-Uwe Balszuweit und Michael Böttrich
Sitz und Registergericht: München HRB 134810
-
On 04.02.21, 13:11, "openembedded-core@lists.openembedded.org on behalf of
Tomasz Dziendzielski" wrote:
If image recipe depends on another image recipe it will try to use
license recipeinfo that is not deployed. It will result in:
| Exception: FileNotFoundError: [Errno 2] No such file or directory:
| 'TMPDIR/deploy/licenses/foo-image/recipeinfo'
We can't determine in a simple and efficient way if dependency recipe
inherits image.bbclass, so let's introduce SKIP_LICENSE_MANIFEST_RECIPES
variable that will contain list of recipes that should be skipped.
Signed-off-by: Tomasz Dziendzielski
---
meta/classes/license_image.bbclass | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/meta/classes/license_image.bbclass
b/meta/classes/license_image.bbclass
index 8fd88cfb2d..e2f3178306 100644
--- a/meta/classes/license_image.bbclass
+++ b/meta/classes/license_image.bbclass
@@ -223,7 +223,8 @@ def get_deployed_dependencies(d):
pn = d.getVar("PN", True)
depends = list(set([dep[0] for dep
in list(taskdata.values())
-if not dep[0].endswith("-native") and not dep[0] ==
pn]))
+if not dep[0].endswith("-native") and not dep[0] == pn
+and dep[0] not in
(d.getVar("SKIP_LICENSE_MANIFEST_RECIPES", True) or "").split()]))
# To verify what was deployed it checks the rootfs dependencies against
# the SSTATE_MANIFESTS for "deploy" task.
--
2.30.0
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#147659):
https://lists.openembedded.org/g/openembedded-core/message/147659
Mute This Topic: https://lists.openembedded.org/mt/80377785/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [pseudo][PATCH] pseudo_client.c: Rebuild passwd paths after chroot
From: Michael Ho
Pseudo will calculate the search paths used for passwd operations such
as lckpwdf and ulckpwdf using build_passwd_paths when it initiates. This
takes into account the chroot at the time.
The problem is that after a chroot is performed, pseudo continues to use
the search paths calculated from the start for lckpwdf and ulckpwdf.
This makes it write the pwd.lock files to a different sysroot if a chroot
is called during runtime.
This commit resolves that by calling build_passwd_paths again after
intercepting chroot calls so the search paths are up to date.
This bug manifests in Yocto when shadow is installed into an SDK target
rootfs. The postinst triggered will call shadow-native with -R to point
to the SDK target rootfs which in turn makes shadow call chroot to the
SDK target rootfs before it perform its actions including lckpwdf() and
ulckpwdf(). The lock files however will write instead to the normal image
target rootfs because it was specified in PSEUDO_PASSWD and was the first
path set when the pseudo environment was initiated. By rebuilding the
search path after chroot is applied, the lockfiles appear in the correct
rootfs.
Signed-off-by: Michael Ho
---
pseudo_client.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/pseudo_client.c b/pseudo_client.c
index 478e450..60cd0d1 100644
--- a/pseudo_client.c
+++ b/pseudo_client.c
@@ -822,6 +822,8 @@ pseudo_client_chroot(const char *path) {
}
memcpy(pseudo_chroot, path, pseudo_chroot_len + 1);
pseudo_set_value("PSEUDO_CHROOT", pseudo_chroot);
+ /* Rebuild passwd paths since we've done a chroot */
+ build_passwd_paths();
return 0;
}
--
2.7.4
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#147245):
https://lists.openembedded.org/g/openembedded-core/message/147245
Mute This Topic: https://lists.openembedded.org/mt/80126389/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] Question about security_flags.inc and CC_ARCH
Hi, Thanks for the info/comments. It's clear that it wouldn't be wise to differ the SDK compiler flags from the bitbake flags. Maybe coming back to the original point, what would the recommended way be for developers in an SDK to create debug builds of their software? Should they attempt to filter out FORTIFY_SOURCE from the compiler flags on their own? Could it make sense to leave things like the stack-protector flags in CC_ARCH but move just FORTIFY_SOURCE to CFLAGS as FORTIFY_SOURCE does not do anything without optimisations (which should not be part of CC/CXX)? Thanks for any advice! Kind regards, Michael Ho -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#147220): https://lists.openembedded.org/g/openembedded-core/message/147220 Mute This Topic: https://lists.openembedded.org/mt/79701669/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] Question about security_flags.inc and CC_ARCH
Hi all, I wanted to get a bit more understanding of why security_flags.inc tweaks CC_ARCH instead of CFLAGS. Some developers who consume an SDK we produce using Yocto noticed that CC and CXX has FORTIFY_SOURCE embedded in the variables. These developers sometimes want to compile software in the SDK with compiler optimisations turned off in order to run code coverage tools like gcov. Typically they drop CFLAGS/CXXFLAGS in order to do this but they noted that with the SDK they also have to manually tweak CC/CXX to remove the FORTIFY_SOURCE references (because compilation fails without optimisation flags when using FORTIFY_SOURCE). This comes from: https://patchwork.openembedded.org/patch/167198/ and http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=6733a7873ca121295a2e309a6915b9816e1ae36b I would’ve expected actually that FORTIFY_SOURCE bundles itself with CFLAGS/CXXFLAGS as it is dependent on being with the compiler optimisations. This is also how the Debian hardening wiki seems to describe it used [1]. I am guessing that this is moved to CC_ARCH to ensure FORTIFY_SOURCE is being enforced around the build system in case components are skipping out on CFLAGS and CXXFLAGS. Is that right? Would there be some objection to moving the security flags to CFLAGS/CXXFLAGS for the cross-canadian target (sdk)? Thanks for any insights people can share! Kind regards, Michael Ho [1] https://wiki.debian.org/Hardening#Using_Hardening_Options -- BMW Car IT GmbH Michael Ho Spezialist Entwicklung – Build and Release Engineering Lise-Meitner-Straße 14 89081 Ulm Tel.: +49-731-37804-071 Mobil: +49-152-54980-471 Fax: +49-731-37804-001 Mail: michael...@bmw-carit.de<mailto:michael...@bmw-carit.de> Web: http://www.bmw-carit.de<http://www.bmw-carit.de/> - BMW Car IT GmbH Geschäftsführer: Kai-Uwe Balszuweit und Michael Böttrich Sitz und Registergericht: München HRB 134810 - -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#146824): https://lists.openembedded.org/g/openembedded-core/message/146824 Mute This Topic: https://lists.openembedded.org/mt/79701669/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH] rootfs_ipk: allow do_populate_sdk in parallel to do_rootfs
Hi RP,
I would trust your judgement call here, I understand not taking this type of
patch in
to avoid complicating the ipk package management.
We will soon start running in our CI essentially the dunfell patch soon so we
can
test for race conditions quite heavily there. But given the refactoring of the
package
managers in Yocto, it won't be the same as the master branch patch.
I'm not so deep into the oe.selftest stuff. Would it help if I try to add a
test that calls
core-image-minimal:do_rootfs and core-image-minimal:do_populate_sdk at the
same time and check the build doesn't crash and try running that a few hundred
times?
Kind regards,
Michael
--
BMW Car IT GmbH
Michael Ho
Spezialist Entwicklung – Build and Release Engineering
Lise-Meitner-Straße 14
89081 Ulm
Tel.: +49-731-37804-071
Mobil: +49-152-54980-471
Fax: +49-731-37804-001
Mail: michael...@bmw-carit.de
Web: http://www.bmw-carit.de <http://www.bmw-carit.de/>
-
BMW Car IT GmbH
Geschäftsführer: Kai-Uwe Balszuweit und Michael Böttrich
Sitz und Registergericht: München HRB 134810
-
On 12.01.21, 12:58, "Richard Purdie"
wrote:
On Tue, 2021-01-12 at 09:21 +, Richard Purdie via
lists.openembedded.org wrote:
> On Mon, 2021-01-11 at 13:45 +0100, Michael Ho wrote:
> > From: Michael Ho
> >
> > Switch do_populate_sdk for the ipk package manager to use a separate
target
> > opkg config file and separate the lockfiles restricting do_rootfs and
> > do_populate_sdk from running in parallel.
> >
> > This way if an image recipe includes a dependency to do_populate_sdk by
> > default then it will run in parallel to do_rootfs saving time compared
to the
> > sequential execution.
> >
> > Signed-off-by: Michael Ho
> > ---
> > meta/classes/package_ipk.bbclass | 1 +
> > meta/classes/rootfs_ipk.bbclass| 4 ++--
> > meta/lib/oe/package_manager/ipk/sdk.py | 6 ++
> > 3 files changed, 9 insertions(+), 2 deletions(-)
>
> I have to admit I'm very nervous about this change. The races we've
> seen betweem rootfs and sdk can be quite unusual.
>
> I did put this in for testing and we saw:
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/65/builds/2938
>
> which in the pseudo.log shows:
>
> path mismatch [2 links]: ino 372706913 db
'/home/pokybuild/yocto-worker/beaglebone/build/build/tmp/work/beaglebone_yocto-poky-linux-gnueabi/core-image-sato/1.0-r0/tmp-wic'
req
'/home/pokybuild/yocto-worker/beaglebone/build/build/tmp/work/beaglebone_yocto-poky-linux-gnueabi/core-image-sato/1.0-r0/sdk/image/opt/poky/3.2+snapshot/sysroots/cortexa8hf-neon-poky-linux-gnueabi/usr/src/debug/puzzles'.
> path mismatch [2 links]: ino 372706913 db
'/home/pokybuild/yocto-worker/beaglebone/build/build/tmp/work/beaglebone_yocto-poky-linux-gnueabi/core-image-sato/1.0-r0/tmp-wic'
req
'/home/pokybuild/yocto-worker/beaglebone/build/build/tmp/work/beaglebone_yocto-poky-linux-gnueabi/core-image-sato/1.0-r0/sdk/image/opt/poky/3.2+snapshot/sysroots/cortexa8hf-neon-poky-linux-gnueabi/usr/src/debug/puzzles'.
>
> its hard to know if this is due to this patch or possibly Paul's wic
> changes as both were in this test series. I suspect it won't reproduce
> every time since its a race.
Just to confirm, this is Paul's patch as it was seen with rpm packaging
and therefore couldn't be the ipk change.
I am worried we don't test ipk extensively enough to spot the races in
the ipk code though, I know I've had to fix quite a few over the years
:(.
Cheers,
Richard
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#146621):
https://lists.openembedded.org/g/openembedded-core/message/146621
Mute This Topic: https://lists.openembedded.org/mt/79594382/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH] license_image.bbclass: fix missing recipeinfo on self
From: Michael Ho
Resolve a build bug where image recipes with a do_deploy task will fail.
If the image recipe inheriting license_image.bbclass has a deploy task, then
the function get_deployed_dependencies will add itself to the list of recipes
to get license information for.
However, image recipes don't generally deploy license info so this results in
an error.
File: '/nvme/poky/meta/classes/license_image.bbclass', lineno: 192, function:
license_deployed_manifest
...
Exception: FileNotFoundError: [Errno 2] No such file or directory:
'/nvme/poky/build/tmp/deploy/licenses/core-image-minimal/recipeinfo'
Add a corner case to exclude the originating image recipe from the list of
dependencies to check.
Signed-off-by: Michael Ho
---
meta/classes/license_image.bbclass | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/meta/classes/license_image.bbclass
b/meta/classes/license_image.bbclass
index 119c8df..8fd88cf 100644
--- a/meta/classes/license_image.bbclass
+++ b/meta/classes/license_image.bbclass
@@ -220,9 +220,10 @@ def get_deployed_dependencies(d):
deploy = {}
# Get all the dependencies for the current task (rootfs).
taskdata = d.getVar("BB_TASKDEPDATA", False)
+pn = d.getVar("PN", True)
depends = list(set([dep[0] for dep
in list(taskdata.values())
-if not dep[0].endswith("-native")]))
+if not dep[0].endswith("-native") and not dep[0] == pn]))
# To verify what was deployed it checks the rootfs dependencies against
# the SSTATE_MANIFESTS for "deploy" task.
--
2.7.4
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#146607):
https://lists.openembedded.org/g/openembedded-core/message/146607
Mute This Topic: https://lists.openembedded.org/mt/79619287/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH] rootfs_ipk: allow do_populate_sdk in parallel to do_rootfs
Hello Alex,
Seems to not be so quick to change for the deb packages. First I had issues
building the sdk on
the master branch as it runs some postinst scripts that leak out of the yocto
sandbox and into
the host system [1]. Not sure if that’s a recent regression as I don’t normally
use deb packages
in Yocto.
Then I found that apt will fight over a lock file written into the
recipe-sysroots-native directory [2].
I am not so familiar with apt-get to resolve that trivially. I’ll try to
re-visit it another day.
Kind regards,
Michael
[1]
Setting up nativesdk-shadow (4.8.1-r0) ...
mkdir: cannot create directory ‘/usr/lib/opkg’: Permission denied
dpkg: error processing package nativesdk-shadow (--configure):
installed nativesdk-shadow package post-installation script subprocess
returned error exit status 1
[2]
E: Could not get lock
/nvme/poky/build-deb/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/recipe-sysroot-native/var/cache/apt/archives/lock
- open (11: Resource temporarily unavailable)
--
BMW Car IT GmbH
Michael Ho
Spezialist Entwicklung – Build and Release Engineering
Lise-Meitner-Straße 14
89081 Ulm
Tel.: +49-731-37804-071
Mobil: +49-152-54980-471
Fax: +49-731-37804-001
Mail: michael...@bmw-carit.de<mailto:michael...@bmw-carit.de>
Web: http://www.bmw-carit.de<http://www.bmw-carit.de/>
-
BMW Car IT GmbH
Geschäftsführer: Kai-Uwe Balszuweit und Michael Böttrich
Sitz und Registergericht: München HRB 134810
-
From: Alexander Kanavin
Date: Monday, 11. January 2021 at 15:45
To: "Ho Michael, JC-3UL"
Cc: OE-core
Subject: Re: [OE-core] [PATCH] rootfs_ipk: allow do_populate_sdk in parallel to
do_rootfs
If you can fix the deb packaging easily, then please do so, but it is not
mandatory, and don't spend too much time if it turns out to be a time sink.
Alex
On Mon, 11 Jan 2021 at 15:01, mailto:michael...@bmw.de>>
wrote:
Hello Alexander,
Did a quick test and it seems the rpm packaging classes don’t have this issue.
The rootfs_deb bbclass however has the same lock files like with rootfs_ipk.
Should I try to patch this too to keep them aligned?
Kind regards,
Michael
--
BMW Car IT GmbH
Michael Ho
Spezialist Entwicklung – Build and Release Engineering
Lise-Meitner-Straße 14
89081 Ulm
Tel.: +49-731-37804-071
Mobil: +49-152-54980-471
Fax: +49-731-37804-001
Mail: michael...@bmw-carit.de<mailto:michael...@bmw-carit.de>
Web: http://www.bmw-carit.de<http://www.bmw-carit.de/>
-
BMW Car IT GmbH
Geschäftsführer: Kai-Uwe Balszuweit und Michael Böttrich
Sitz und Registergericht: München HRB 134810
-
From: Alexander Kanavin mailto:alex.kana...@gmail.com>>
Date: Monday, 11. January 2021 at 13:46
To: "Ho Michael, JC-3UL" mailto:michael...@bmw.de>>
Cc: OE-core
mailto:openembedded-core@lists.openembedded.org>>
Subject: Re: [OE-core] [PATCH] rootfs_ipk: allow do_populate_sdk in parallel to
do_rootfs
Does deb/rpm packaging already have this feature?
Alex
On Mon, 11 Jan 2021 at 13:45, Michael Ho
mailto:michael...@bmw.de>> wrote:
From: Michael Ho mailto:michael...@bmw.de>>
Switch do_populate_sdk for the ipk package manager to use a separate target
opkg config file and separate the lockfiles restricting do_rootfs and
do_populate_sdk from running in parallel.
This way if an image recipe includes a dependency to do_populate_sdk by
default then it will run in parallel to do_rootfs saving time compared to the
sequential execution.
Signed-off-by: Michael Ho mailto:michael...@bmw.de>>
---
meta/classes/package_ipk.bbclass | 1 +
meta/classes/rootfs_ipk.bbclass| 4 ++--
meta/lib/oe/package_manager/ipk/sdk.py | 6 ++
3 files changed, 9 insertions(+), 2 deletions(-)
diff --git a/meta/classes/package_ipk.bbclass b/meta/classes/package_ipk.bbclass
index 79cb36c..d31dba0 100644
--- a/meta/classes/package_ipk.bbclass
+++ b/meta/classes/package_ipk.bbclass
@@ -4,6 +4,7 @@ IMAGE_PKGTYPE ?= "ipk"
IPKGCONF_TARGET = "${WORKDIR}/opkg.conf"
IPKGCONF_SDK = "${WORKDIR}/opkg-sdk.conf"
+IPKGCONF_SDK_TARGET = "${WORKDIR}/opkg-sdk-target.conf"
PKGWRITEDIRIPK = "${WORKDIR}/deploy-ipks"
diff --git a/meta/classes/rootfs_ipk.bbclass b/meta/classes/rootfs_ipk.bbclass
index f1e0219..245c256 100644
--- a/meta/classes/rootfs_ipk.bbclass
+++ b/meta/classes/rootfs_ipk.bbclass
@@ -14,8 +14,8 @@ do_rootfs[recrdeptask] += "do_package_write_ipk do_package_qa"
do_rootfs[vardeps] += "PACKAGE_FEED_URIS PACKAGE_FEED_BASE_PATHS
PACKAGE_FEED_ARCHS"
do_rootfs[lockfiles] += "${WORKDIR}/ipk.lock"
-do_populate_sdk[lockfiles] += "${WORKDIR}/ipk.lock&quo
Re: [OE-core] [PATCH] rootfs_ipk: allow do_populate_sdk in parallel to do_rootfs
Hello Alexander,
Did a quick test and it seems the rpm packaging classes don’t have this issue.
The rootfs_deb bbclass however has the same lock files like with rootfs_ipk.
Should I try to patch this too to keep them aligned?
Kind regards,
Michael
--
BMW Car IT GmbH
Michael Ho
Spezialist Entwicklung – Build and Release Engineering
Lise-Meitner-Straße 14
89081 Ulm
Tel.: +49-731-37804-071
Mobil: +49-152-54980-471
Fax: +49-731-37804-001
Mail: michael...@bmw-carit.de<mailto:michael...@bmw-carit.de>
Web: http://www.bmw-carit.de<http://www.bmw-carit.de/>
-
BMW Car IT GmbH
Geschäftsführer: Kai-Uwe Balszuweit und Michael Böttrich
Sitz und Registergericht: München HRB 134810
-
From: Alexander Kanavin
Date: Monday, 11. January 2021 at 13:46
To: "Ho Michael, JC-3UL"
Cc: OE-core
Subject: Re: [OE-core] [PATCH] rootfs_ipk: allow do_populate_sdk in parallel to
do_rootfs
Does deb/rpm packaging already have this feature?
Alex
On Mon, 11 Jan 2021 at 13:45, Michael Ho
mailto:michael...@bmw.de>> wrote:
From: Michael Ho mailto:michael...@bmw.de>>
Switch do_populate_sdk for the ipk package manager to use a separate target
opkg config file and separate the lockfiles restricting do_rootfs and
do_populate_sdk from running in parallel.
This way if an image recipe includes a dependency to do_populate_sdk by
default then it will run in parallel to do_rootfs saving time compared to the
sequential execution.
Signed-off-by: Michael Ho mailto:michael...@bmw.de>>
---
meta/classes/package_ipk.bbclass | 1 +
meta/classes/rootfs_ipk.bbclass| 4 ++--
meta/lib/oe/package_manager/ipk/sdk.py | 6 ++
3 files changed, 9 insertions(+), 2 deletions(-)
diff --git a/meta/classes/package_ipk.bbclass b/meta/classes/package_ipk.bbclass
index 79cb36c..d31dba0 100644
--- a/meta/classes/package_ipk.bbclass
+++ b/meta/classes/package_ipk.bbclass
@@ -4,6 +4,7 @@ IMAGE_PKGTYPE ?= "ipk"
IPKGCONF_TARGET = "${WORKDIR}/opkg.conf"
IPKGCONF_SDK = "${WORKDIR}/opkg-sdk.conf"
+IPKGCONF_SDK_TARGET = "${WORKDIR}/opkg-sdk-target.conf"
PKGWRITEDIRIPK = "${WORKDIR}/deploy-ipks"
diff --git a/meta/classes/rootfs_ipk.bbclass b/meta/classes/rootfs_ipk.bbclass
index f1e0219..245c256 100644
--- a/meta/classes/rootfs_ipk.bbclass
+++ b/meta/classes/rootfs_ipk.bbclass
@@ -14,8 +14,8 @@ do_rootfs[recrdeptask] += "do_package_write_ipk do_package_qa"
do_rootfs[vardeps] += "PACKAGE_FEED_URIS PACKAGE_FEED_BASE_PATHS
PACKAGE_FEED_ARCHS"
do_rootfs[lockfiles] += "${WORKDIR}/ipk.lock"
-do_populate_sdk[lockfiles] += "${WORKDIR}/ipk.lock"
-do_populate_sdk_ext[lockfiles] += "${WORKDIR}/ipk.lock"
+do_populate_sdk[lockfiles] += "${WORKDIR}/sdk-ipk.lock"
+do_populate_sdk_ext[lockfiles] += "${WORKDIR}/sdk-ipk.lock"
OPKG_PREPROCESS_COMMANDS = ""
diff --git a/meta/lib/oe/package_manager/ipk/sdk.py
b/meta/lib/oe/package_manager/ipk/sdk.py
index 37af034..e2ca415 100644
--- a/meta/lib/oe/package_manager/ipk/sdk.py
+++ b/meta/lib/oe/package_manager/ipk/sdk.py
@@ -14,6 +14,12 @@ class PkgSdk(Sdk):
def __init__(self, d, manifest_dir=None):
super(PkgSdk, self).__init__(d, manifest_dir)
+# In sdk_list_installed_packages the call to opkg is hardcoded to
+# always use IPKGCONF_TARGET and there's no exposed API to change this
+# so simply override IPKGCONF_TARGET to use this separated config file.
+ipkgconf_sdk_target = d.getVar("IPKGCONF_SDK_TARGET")
+d.setVar("IPKGCONF_TARGET", ipkgconf_sdk_target)
+
self.target_conf = self.d.getVar("IPKGCONF_TARGET")
self.host_conf = self.d.getVar("IPKGCONF_SDK")
--
2.7.4
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#146591):
https://lists.openembedded.org/g/openembedded-core/message/146591
Mute This Topic: https://lists.openembedded.org/mt/79594382/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [dunfell][PATCH] rootfs_ipk: allow do_populate_sdk in parallel to do_rootfs
From: Michael Ho
Switch do_populate_sdk for the ipk package manager to use a separate target
opkg config file and separate the lockfiles restricting do_rootfs and
do_populate_sdk from running in parallel.
This way if an image recipe includes a dependency to do_populate_sdk by
default then it will run in parallel to do_rootfs saving time compared to the
sequential execution.
Signed-off-by: Michael Ho
---
meta/classes/package_ipk.bbclass | 1 +
meta/classes/rootfs_ipk.bbclass | 4 ++--
meta/lib/oe/sdk.py | 4 ++--
3 files changed, 5 insertions(+), 4 deletions(-)
diff --git a/meta/classes/package_ipk.bbclass b/meta/classes/package_ipk.bbclass
index c008559..54da32e 100644
--- a/meta/classes/package_ipk.bbclass
+++ b/meta/classes/package_ipk.bbclass
@@ -4,6 +4,7 @@ IMAGE_PKGTYPE ?= "ipk"
IPKGCONF_TARGET = "${WORKDIR}/opkg.conf"
IPKGCONF_SDK = "${WORKDIR}/opkg-sdk.conf"
+IPKGCONF_SDK_TARGET = "${WORKDIR}/opkg-sdk-target.conf"
PKGWRITEDIRIPK = "${WORKDIR}/deploy-ipks"
diff --git a/meta/classes/rootfs_ipk.bbclass b/meta/classes/rootfs_ipk.bbclass
index f1e0219..245c256 100644
--- a/meta/classes/rootfs_ipk.bbclass
+++ b/meta/classes/rootfs_ipk.bbclass
@@ -14,8 +14,8 @@ do_rootfs[recrdeptask] += "do_package_write_ipk do_package_qa"
do_rootfs[vardeps] += "PACKAGE_FEED_URIS PACKAGE_FEED_BASE_PATHS
PACKAGE_FEED_ARCHS"
do_rootfs[lockfiles] += "${WORKDIR}/ipk.lock"
-do_populate_sdk[lockfiles] += "${WORKDIR}/ipk.lock"
-do_populate_sdk_ext[lockfiles] += "${WORKDIR}/ipk.lock"
+do_populate_sdk[lockfiles] += "${WORKDIR}/sdk-ipk.lock"
+do_populate_sdk_ext[lockfiles] += "${WORKDIR}/sdk-ipk.lock"
OPKG_PREPROCESS_COMMANDS = ""
diff --git a/meta/lib/oe/sdk.py b/meta/lib/oe/sdk.py
index d02a274..3d6141b 100644
--- a/meta/lib/oe/sdk.py
+++ b/meta/lib/oe/sdk.py
@@ -219,7 +219,7 @@ class OpkgSdk(Sdk):
def __init__(self, d, manifest_dir=None):
super(OpkgSdk, self).__init__(d, manifest_dir)
-self.target_conf = self.d.getVar("IPKGCONF_TARGET")
+self.target_conf = self.d.getVar("IPKGCONF_SDK_TARGET")
self.host_conf = self.d.getVar("IPKGCONF_SDK")
self.target_manifest = OpkgManifest(d, self.manifest_dir,
@@ -400,7 +400,7 @@ def sdk_list_installed_packages(d, target, rootfs_dir=None):
os_var = ["SDK_OS", None][target is True]
return RpmPkgsList(d, rootfs_dir).list_pkgs()
elif img_type == "ipk":
-conf_file_var = ["IPKGCONF_SDK", "IPKGCONF_TARGET"][target is True]
+conf_file_var = ["IPKGCONF_SDK", "IPKGCONF_SDK_TARGET"][target is True]
return OpkgPkgsList(d, rootfs_dir, d.getVar(conf_file_var)).list_pkgs()
elif img_type == "deb":
return DpkgPkgsList(d, rootfs_dir).list_pkgs()
--
2.7.4
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#146589):
https://lists.openembedded.org/g/openembedded-core/message/146589
Mute This Topic: https://lists.openembedded.org/mt/79594400/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH] rootfs_ipk: allow do_populate_sdk in parallel to do_rootfs
From: Michael Ho
Switch do_populate_sdk for the ipk package manager to use a separate target
opkg config file and separate the lockfiles restricting do_rootfs and
do_populate_sdk from running in parallel.
This way if an image recipe includes a dependency to do_populate_sdk by
default then it will run in parallel to do_rootfs saving time compared to the
sequential execution.
Signed-off-by: Michael Ho
---
meta/classes/package_ipk.bbclass | 1 +
meta/classes/rootfs_ipk.bbclass| 4 ++--
meta/lib/oe/package_manager/ipk/sdk.py | 6 ++
3 files changed, 9 insertions(+), 2 deletions(-)
diff --git a/meta/classes/package_ipk.bbclass b/meta/classes/package_ipk.bbclass
index 79cb36c..d31dba0 100644
--- a/meta/classes/package_ipk.bbclass
+++ b/meta/classes/package_ipk.bbclass
@@ -4,6 +4,7 @@ IMAGE_PKGTYPE ?= "ipk"
IPKGCONF_TARGET = "${WORKDIR}/opkg.conf"
IPKGCONF_SDK = "${WORKDIR}/opkg-sdk.conf"
+IPKGCONF_SDK_TARGET = "${WORKDIR}/opkg-sdk-target.conf"
PKGWRITEDIRIPK = "${WORKDIR}/deploy-ipks"
diff --git a/meta/classes/rootfs_ipk.bbclass b/meta/classes/rootfs_ipk.bbclass
index f1e0219..245c256 100644
--- a/meta/classes/rootfs_ipk.bbclass
+++ b/meta/classes/rootfs_ipk.bbclass
@@ -14,8 +14,8 @@ do_rootfs[recrdeptask] += "do_package_write_ipk do_package_qa"
do_rootfs[vardeps] += "PACKAGE_FEED_URIS PACKAGE_FEED_BASE_PATHS
PACKAGE_FEED_ARCHS"
do_rootfs[lockfiles] += "${WORKDIR}/ipk.lock"
-do_populate_sdk[lockfiles] += "${WORKDIR}/ipk.lock"
-do_populate_sdk_ext[lockfiles] += "${WORKDIR}/ipk.lock"
+do_populate_sdk[lockfiles] += "${WORKDIR}/sdk-ipk.lock"
+do_populate_sdk_ext[lockfiles] += "${WORKDIR}/sdk-ipk.lock"
OPKG_PREPROCESS_COMMANDS = ""
diff --git a/meta/lib/oe/package_manager/ipk/sdk.py
b/meta/lib/oe/package_manager/ipk/sdk.py
index 37af034..e2ca415 100644
--- a/meta/lib/oe/package_manager/ipk/sdk.py
+++ b/meta/lib/oe/package_manager/ipk/sdk.py
@@ -14,6 +14,12 @@ class PkgSdk(Sdk):
def __init__(self, d, manifest_dir=None):
super(PkgSdk, self).__init__(d, manifest_dir)
+# In sdk_list_installed_packages the call to opkg is hardcoded to
+# always use IPKGCONF_TARGET and there's no exposed API to change this
+# so simply override IPKGCONF_TARGET to use this separated config file.
+ipkgconf_sdk_target = d.getVar("IPKGCONF_SDK_TARGET")
+d.setVar("IPKGCONF_TARGET", ipkgconf_sdk_target)
+
self.target_conf = self.d.getVar("IPKGCONF_TARGET")
self.host_conf = self.d.getVar("IPKGCONF_SDK")
--
2.7.4
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#146588):
https://lists.openembedded.org/g/openembedded-core/message/146588
Mute This Topic: https://lists.openembedded.org/mt/79594382/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH] package_rpm.bbclass: respect package overrides for the main package
Hi Steve,
Not sure if that question was for me or the general community. I think the
follow-on series fixing the sstate-caching might be nice in dunfell but I'd
defer judgement to the community about this one as it's about changing a
behavior in the recipes and I don't know what LTS guidelines there are in place
in the project.
Should I in the future address patches to the dunfell branch? (Where might I
find the guidelines for submitting to dunfell vs master).
Thanks.
Kind regards,
Michael
--
BMW Car IT GmbH
Michael Ho
Spezialist Entwicklung – Build and Release Engineering
Lise-Meitner-Straße 14
89081 Ulm
Tel.: +49-731-37804-071
Mobil: +49-152-54980-471
Fax: +49-731-37804-001
Mail: michael...@bmw-carit.de
Web: http://www.bmw-carit.de <http://www.bmw-carit.de/>
-
BMW Car IT GmbH
Geschäftsführer: Kai-Uwe Balszuweit und Michael Böttrich
Sitz und Registergericht: München HRB 134810
-
On 26.05.20, 19:02, "Steve Sakoman" wrote:
Does this patch (and the follow-on series) make sense for inclusion in
dunfell?
Steve
On Sun, May 24, 2020 at 9:46 PM Michael Ho wrote:
>
> From: Michael Ho
>
> Apply ${PN} to OVERRIDES when determining the base package spec variables.
> Without this, there is a mismatch in behaviour where overrides for the
base
> package have no effect while overrides for subpackages do. The subpackages
> should not be affected by this override as they apply their own package
> overrides and will by default fetch the base variables and apply them in
> the spec file.
>
> This makes the behaviour for package overrides match the implementation
> in package_deb.bbclass and package_ipk.bbclass.
>
> Signed-off-by: Michael Ho
> ---
> meta/classes/package_rpm.bbclass | 22 --
> 1 file changed, 12 insertions(+), 10 deletions(-)
>
> diff --git a/meta/classes/package_rpm.bbclass
b/meta/classes/package_rpm.bbclass
> index 9a4be2e..519c22b 100644
> --- a/meta/classes/package_rpm.bbclass
> +++ b/meta/classes/package_rpm.bbclass
> @@ -286,16 +286,18 @@ python write_specfile () {
>
> # Construct the SPEC file...
> srcname= d.getVar('PN')
> -srcsummary = (d.getVar('SUMMARY') or d.getVar('DESCRIPTION') or ".")
> -srcversion = d.getVar('PKGV').replace('-', '+')
> -srcrelease = d.getVar('PKGR')
> -srcepoch = (d.getVar('PKGE') or "")
> -srclicense = d.getVar('LICENSE')
> -srcsection = d.getVar('SECTION')
> -srcmaintainer = d.getVar('MAINTAINER')
> -srchomepage= d.getVar('HOMEPAGE')
> -srcdescription = d.getVar('DESCRIPTION') or "."
> -srccustomtagschunk = get_package_additional_metadata("rpm", d)
> +localdata = bb.data.createCopy(d)
> +localdata.setVar('OVERRIDES', d.getVar("OVERRIDES", False) + ":" +
srcname)
> +srcsummary = (localdata.getVar('SUMMARY') or
localdata.getVar('DESCRIPTION') or ".")
> +srcversion = localdata.getVar('PKGV').replace('-', '+')
> +srcrelease = localdata.getVar('PKGR')
> +srcepoch = (localdata.getVar('PKGE') or "")
> +srclicense = localdata.getVar('LICENSE')
> +srcsection = localdata.getVar('SECTION')
> +srcmaintainer = localdata.getVar('MAINTAINER')
> +srchomepage= localdata.getVar('HOMEPAGE')
> +srcdescription = localdata.getVar('DESCRIPTION') or "."
> +srccustomtagschunk = get_package_additional_metadata("rpm",
localdata)
>
> srcdepends = d.getVar('DEPENDS')
> srcrdepends= []
> --
> 2.7.4
>
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#138841):
https://lists.openembedded.org/g/openembedded-core/message/138841
Mute This Topic: https://lists.openembedded.org/mt/74452506/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH v2 2/4] package_ipk.bbclass: add PACKAGE_ADD_METADATA_IPK to sstate variables
From: Michael Ho
Add PACKAGE_ADD_METADATA_IPK to IPKEXTRAVARS so it affects the sstate hash
of do_package_write_ipk.
Signed-off-by: Michael Ho
---
meta/classes/package_ipk.bbclass | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/meta/classes/package_ipk.bbclass b/meta/classes/package_ipk.bbclass
index c008559..79cb36c 100644
--- a/meta/classes/package_ipk.bbclass
+++ b/meta/classes/package_ipk.bbclass
@@ -238,7 +238,7 @@ def ipk_write_pkg(pkg, d):
bb.utils.unlockfile(lf)
# Have to list any variables referenced as X_ that aren't in pkgdata here
-IPKEXTRAVARS = "PRIORITY MAINTAINER PACKAGE_ARCH HOMEPAGE"
+IPKEXTRAVARS = "PRIORITY MAINTAINER PACKAGE_ARCH HOMEPAGE
PACKAGE_ADD_METADATA_IPK"
ipk_write_pkg[vardeps] += "${@gen_packagevar(d, 'IPKEXTRAVARS')}"
# Otherwise allarch packages may change depending on override configuration
--
2.7.4
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#138662):
https://lists.openembedded.org/g/openembedded-core/message/138662
Mute This Topic: https://lists.openembedded.org/mt/74452820/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH v2 4/4] package_rpm.bbclass: add PACKAGE_ADD_METADATA_RPM to sstate variables
From: Michael Ho
Refactor the addition of vardeps for this package class to use RPMEXTRAVARS
and pass the items through gen_packagevar so package name specific overrides
are accounted for. This matches the style used in package_ipk.bbclass. Since
do_package_write_rpm does not have any "extra vars" that affect the rpm
packaging yet, this commit has to add the variable.
This refactoring is needed to add PACKAGE_ADD_METADATA_RPM to RPMEXTRAVARS
so it affects the sstate hash of do_package_write_rpm.
Signed-off-by: Michael Ho
---
meta/classes/package_rpm.bbclass | 4
1 file changed, 4 insertions(+)
diff --git a/meta/classes/package_rpm.bbclass b/meta/classes/package_rpm.bbclass
index 9145717..9a4be2e 100644
--- a/meta/classes/package_rpm.bbclass
+++ b/meta/classes/package_rpm.bbclass
@@ -621,6 +621,10 @@ python write_specfile () {
# Otherwise allarch packages may change depending on override configuration
write_specfile[vardepsexclude] = "OVERRIDES"
+# Have to list any variables referenced as X_ that aren't in pkgdata here
+RPMEXTRAVARS = "PACKAGE_ADD_METADATA_RPM"
+write_specfile[vardeps] += "${@gen_packagevar(d, 'RPMEXTRAVARS')}"
+
python do_package_rpm () {
workdir = d.getVar('WORKDIR')
tmpdir = d.getVar('TMPDIR')
--
2.7.4
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#138663):
https://lists.openembedded.org/g/openembedded-core/message/138663
Mute This Topic: https://lists.openembedded.org/mt/74452821/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH v2 3/4] package_deb.bbclass: add PACKAGE_ADD_METADATA_DEB to sstate variables
From: Michael Ho
Refactor the addition of vardeps for this package class to use DEBEXTRAVARS
and pass the items through gen_packagevar so package name specific overrides
are accounted for. This matches the style used in package_ipk.bbclass.
Additionally add PACKAGE_ADD_METADATA_DEB to the DEBEXTRAVARS list so the
packaging is redone if the variable changes value.
Signed-off-by: Michael Ho
---
meta/classes/package_deb.bbclass | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/meta/classes/package_deb.bbclass b/meta/classes/package_deb.bbclass
index 790b26a..cb723fc 100644
--- a/meta/classes/package_deb.bbclass
+++ b/meta/classes/package_deb.bbclass
@@ -283,8 +283,9 @@ def deb_write_pkg(pkg, d):
# Otherwise allarch packages may change depending on override configuration
deb_write_pkg[vardepsexclude] = "OVERRIDES"
-# Indirect references to these vars
-do_package_write_deb[vardeps] += "PKGV PKGR PKGV DESCRIPTION SECTION PRIORITY
MAINTAINER DPKG_ARCH PN HOMEPAGE"
+# Have to list any variables referenced as X_ that aren't in pkgdata here
+DEBEXTRAVARS = "PKGV PKGR PKGV DESCRIPTION SECTION PRIORITY MAINTAINER
DPKG_ARCH PN HOMEPAGE PACKAGE_ADD_METADATA_DEB"
+do_package_write_deb[vardeps] += "${@gen_packagevar(d, 'DEBEXTRAVARS')}"
SSTATETASKS += "do_package_write_deb"
do_package_write_deb[sstate-inputdirs] = "${PKGWRITEDIRDEB}"
--
2.7.4
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#138661):
https://lists.openembedded.org/g/openembedded-core/message/138661
Mute This Topic: https://lists.openembedded.org/mt/74452818/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH v2 1/4] package.bbclass: add PACKAGE_ADD_METADATA to sstate variables
From: Michael Ho
The PACKAGE_ADD_METADATA can be used to define additional metadata for
packages. Changes to this variable should affect the packaging sstate hash
so packages are re-generated when this variable changes.
This variable is added to both PKGDATA_VARS and PACKAGEVARS. It is needed
in PACKAGEVARS to ensure changes invoke do_package to re-run. It is needed
in PKGDATA_VARS to ensure changes are written into the pkgdata so that the
hash equiv reports a change and does not skip over the package_write tasks.
Signed-off-by: Michael Ho
---
meta/classes/package.bbclass | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/meta/classes/package.bbclass b/meta/classes/package.bbclass
index 0b5cf47..0af5f66 100644
--- a/meta/classes/package.bbclass
+++ b/meta/classes/package.bbclass
@@ -1508,7 +1508,7 @@ EXPORT_FUNCTIONS package_name_hook
PKGDESTWORK = "${WORKDIR}/pkgdata"
-PKGDATA_VARS = "PN PE PV PR PKGE PKGV PKGR LICENSE DESCRIPTION SUMMARY
RDEPENDS RPROVIDES RRECOMMENDS RSUGGESTS RREPLACES RCONFLICTS SECTION PKG
ALLOW_EMPTY FILES CONFFILES FILES_INFO pkg_postinst pkg_postrm pkg_preinst
pkg_prerm"
+PKGDATA_VARS = "PN PE PV PR PKGE PKGV PKGR LICENSE DESCRIPTION SUMMARY
RDEPENDS RPROVIDES RRECOMMENDS RSUGGESTS RREPLACES RCONFLICTS SECTION PKG
ALLOW_EMPTY FILES CONFFILES FILES_INFO PACKAGE_ADD_METADATA pkg_postinst
pkg_postrm pkg_preinst pkg_prerm"
python emit_pkgdata() {
from glob import glob
@@ -2263,7 +2263,7 @@ python package_depchains() {
# Since bitbake can't determine which variables are accessed during package
# iteration, we need to list them here:
-PACKAGEVARS = "FILES RDEPENDS RRECOMMENDS SUMMARY DESCRIPTION RSUGGESTS
RPROVIDES RCONFLICTS PKG ALLOW_EMPTY pkg_postinst pkg_postrm
pkg_postinst_ontarget INITSCRIPT_NAME INITSCRIPT_PARAMS DEBIAN_NOAUTONAME
ALTERNATIVE PKGE PKGV PKGR USERADD_PARAM GROUPADD_PARAM CONFFILES
SYSTEMD_SERVICE LICENSE SECTION pkg_preinst pkg_prerm RREPLACES GROUPMEMS_PARAM
SYSTEMD_AUTO_ENABLE SKIP_FILEDEPS PRIVATE_LIBS"
+PACKAGEVARS = "FILES RDEPENDS RRECOMMENDS SUMMARY DESCRIPTION RSUGGESTS
RPROVIDES RCONFLICTS PKG ALLOW_EMPTY pkg_postinst pkg_postrm
pkg_postinst_ontarget INITSCRIPT_NAME INITSCRIPT_PARAMS DEBIAN_NOAUTONAME
ALTERNATIVE PKGE PKGV PKGR USERADD_PARAM GROUPADD_PARAM CONFFILES
SYSTEMD_SERVICE LICENSE SECTION pkg_preinst pkg_prerm RREPLACES GROUPMEMS_PARAM
SYSTEMD_AUTO_ENABLE SKIP_FILEDEPS PRIVATE_LIBS PACKAGE_ADD_METADATA"
def gen_packagevar(d, pkgvars="PACKAGEVARS"):
ret = []
--
2.7.4
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#138660):
https://lists.openembedded.org/g/openembedded-core/message/138660
Mute This Topic: https://lists.openembedded.org/mt/74452817/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH] package_rpm.bbclass: respect package overrides for the main package
From: Michael Ho
Apply ${PN} to OVERRIDES when determining the base package spec variables.
Without this, there is a mismatch in behaviour where overrides for the base
package have no effect while overrides for subpackages do. The subpackages
should not be affected by this override as they apply their own package
overrides and will by default fetch the base variables and apply them in
the spec file.
This makes the behaviour for package overrides match the implementation
in package_deb.bbclass and package_ipk.bbclass.
Signed-off-by: Michael Ho
---
meta/classes/package_rpm.bbclass | 22 --
1 file changed, 12 insertions(+), 10 deletions(-)
diff --git a/meta/classes/package_rpm.bbclass b/meta/classes/package_rpm.bbclass
index 9a4be2e..519c22b 100644
--- a/meta/classes/package_rpm.bbclass
+++ b/meta/classes/package_rpm.bbclass
@@ -286,16 +286,18 @@ python write_specfile () {
# Construct the SPEC file...
srcname= d.getVar('PN')
-srcsummary = (d.getVar('SUMMARY') or d.getVar('DESCRIPTION') or ".")
-srcversion = d.getVar('PKGV').replace('-', '+')
-srcrelease = d.getVar('PKGR')
-srcepoch = (d.getVar('PKGE') or "")
-srclicense = d.getVar('LICENSE')
-srcsection = d.getVar('SECTION')
-srcmaintainer = d.getVar('MAINTAINER')
-srchomepage= d.getVar('HOMEPAGE')
-srcdescription = d.getVar('DESCRIPTION') or "."
-srccustomtagschunk = get_package_additional_metadata("rpm", d)
+localdata = bb.data.createCopy(d)
+localdata.setVar('OVERRIDES', d.getVar("OVERRIDES", False) + ":" + srcname)
+srcsummary = (localdata.getVar('SUMMARY') or
localdata.getVar('DESCRIPTION') or ".")
+srcversion = localdata.getVar('PKGV').replace('-', '+')
+srcrelease = localdata.getVar('PKGR')
+srcepoch = (localdata.getVar('PKGE') or "")
+srclicense = localdata.getVar('LICENSE')
+srcsection = localdata.getVar('SECTION')
+srcmaintainer = localdata.getVar('MAINTAINER')
+srchomepage= localdata.getVar('HOMEPAGE')
+srcdescription = localdata.getVar('DESCRIPTION') or "."
+srccustomtagschunk = get_package_additional_metadata("rpm", localdata)
srcdepends = d.getVar('DEPENDS')
srcrdepends= []
--
2.7.4
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#138658):
https://lists.openembedded.org/g/openembedded-core/message/138658
Mute This Topic: https://lists.openembedded.org/mt/74452506/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH 4/5] package_rpm.bbclass: add PACKAGE_ADD_METADATA_RPM to sstate variables
From: Michael Ho
Refactor the addition of vardeps for this package class to use RPMEXTRAVARS
and pass the items through gen_packagevar so package name specific overrides
are accounted for. This matches the style used in package_ipk.bbclass. Since
do_package_write_rpm does not have any "extra vars" that affect the rpm
packaging yet, this commit has to add the variable.
This refactoring is needed to add PACKAGE_ADD_METADATA_RPM to RPMEXTRAVARS
so it affects the sstate hash of do_package_write_rpm.
Signed-off-by: Michael Ho
---
meta/classes/package_rpm.bbclass | 4
1 file changed, 4 insertions(+)
diff --git a/meta/classes/package_rpm.bbclass b/meta/classes/package_rpm.bbclass
index 9145717..9a4be2e 100644
--- a/meta/classes/package_rpm.bbclass
+++ b/meta/classes/package_rpm.bbclass
@@ -621,6 +621,10 @@ python write_specfile () {
# Otherwise allarch packages may change depending on override configuration
write_specfile[vardepsexclude] = "OVERRIDES"
+# Have to list any variables referenced as X_ that aren't in pkgdata here
+RPMEXTRAVARS = "PACKAGE_ADD_METADATA_RPM"
+write_specfile[vardeps] += "${@gen_packagevar(d, 'RPMEXTRAVARS')}"
+
python do_package_rpm () {
workdir = d.getVar('WORKDIR')
tmpdir = d.getVar('TMPDIR')
--
2.7.4
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#138656):
https://lists.openembedded.org/g/openembedded-core/message/138656
Mute This Topic: https://lists.openembedded.org/mt/74452488/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH 3/5] package_deb.bbclass: add PACKAGE_ADD_METADATA_DEB to sstate variables
From: Michael Ho
Refactor the addition of vardeps for this package class to use DEBEXTRAVARS
and pass the items through gen_packagevar so package name specific overrides
are accounted for. This matches the style used in package_ipk.bbclass.
Additionally add PACKAGE_ADD_METADATA_DEB to the DEBEXTRAVARS list so the
packaging is redone if the variable changes value.
Signed-off-by: Michael Ho
---
meta/classes/package_deb.bbclass | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/meta/classes/package_deb.bbclass b/meta/classes/package_deb.bbclass
index 790b26a..cb723fc 100644
--- a/meta/classes/package_deb.bbclass
+++ b/meta/classes/package_deb.bbclass
@@ -283,8 +283,9 @@ def deb_write_pkg(pkg, d):
# Otherwise allarch packages may change depending on override configuration
deb_write_pkg[vardepsexclude] = "OVERRIDES"
-# Indirect references to these vars
-do_package_write_deb[vardeps] += "PKGV PKGR PKGV DESCRIPTION SECTION PRIORITY
MAINTAINER DPKG_ARCH PN HOMEPAGE"
+# Have to list any variables referenced as X_ that aren't in pkgdata here
+DEBEXTRAVARS = "PKGV PKGR PKGV DESCRIPTION SECTION PRIORITY MAINTAINER
DPKG_ARCH PN HOMEPAGE PACKAGE_ADD_METADATA_DEB"
+do_package_write_deb[vardeps] += "${@gen_packagevar(d, 'DEBEXTRAVARS')}"
SSTATETASKS += "do_package_write_deb"
do_package_write_deb[sstate-inputdirs] = "${PKGWRITEDIRDEB}"
--
2.7.4
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#138655):
https://lists.openembedded.org/g/openembedded-core/message/138655
Mute This Topic: https://lists.openembedded.org/mt/74452487/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH 1/5] package.bbclass: add PACKAGE_ADD_METADATA to sstate variables
From: Michael Ho
The PACKAGE_ADD_METADATA can be used to define additional metadata for
packages. Changes to this variable should affect the packaging sstate hash
so packages are re-generated when this variable changes.
This variable is added to both PKGDATA_VARS and PACKAGEVARS. It is needed
in PACKAGEVARS to ensure changes invoke do_package to re-run. It is needed
in PKGDATA_VARS to ensure changes are written into the pkgdata so that the
hash equiv reports a change and does not skip over the package_write tasks.
Signed-off-by: Michael Ho
---
meta/classes/package.bbclass | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/meta/classes/package.bbclass b/meta/classes/package.bbclass
index 0b5cf47..0af5f66 100644
--- a/meta/classes/package.bbclass
+++ b/meta/classes/package.bbclass
@@ -1508,7 +1508,7 @@ EXPORT_FUNCTIONS package_name_hook
PKGDESTWORK = "${WORKDIR}/pkgdata"
-PKGDATA_VARS = "PN PE PV PR PKGE PKGV PKGR LICENSE DESCRIPTION SUMMARY
RDEPENDS RPROVIDES RRECOMMENDS RSUGGESTS RREPLACES RCONFLICTS SECTION PKG
ALLOW_EMPTY FILES CONFFILES FILES_INFO pkg_postinst pkg_postrm pkg_preinst
pkg_prerm"
+PKGDATA_VARS = "PN PE PV PR PKGE PKGV PKGR LICENSE DESCRIPTION SUMMARY
RDEPENDS RPROVIDES RRECOMMENDS RSUGGESTS RREPLACES RCONFLICTS SECTION PKG
ALLOW_EMPTY FILES CONFFILES FILES_INFO PACKAGE_ADD_METADATA pkg_postinst
pkg_postrm pkg_preinst pkg_prerm"
python emit_pkgdata() {
from glob import glob
@@ -2263,7 +2263,7 @@ python package_depchains() {
# Since bitbake can't determine which variables are accessed during package
# iteration, we need to list them here:
-PACKAGEVARS = "FILES RDEPENDS RRECOMMENDS SUMMARY DESCRIPTION RSUGGESTS
RPROVIDES RCONFLICTS PKG ALLOW_EMPTY pkg_postinst pkg_postrm
pkg_postinst_ontarget INITSCRIPT_NAME INITSCRIPT_PARAMS DEBIAN_NOAUTONAME
ALTERNATIVE PKGE PKGV PKGR USERADD_PARAM GROUPADD_PARAM CONFFILES
SYSTEMD_SERVICE LICENSE SECTION pkg_preinst pkg_prerm RREPLACES GROUPMEMS_PARAM
SYSTEMD_AUTO_ENABLE SKIP_FILEDEPS PRIVATE_LIBS"
+PACKAGEVARS = "FILES RDEPENDS RRECOMMENDS SUMMARY DESCRIPTION RSUGGESTS
RPROVIDES RCONFLICTS PKG ALLOW_EMPTY pkg_postinst pkg_postrm
pkg_postinst_ontarget INITSCRIPT_NAME INITSCRIPT_PARAMS DEBIAN_NOAUTONAME
ALTERNATIVE PKGE PKGV PKGR USERADD_PARAM GROUPADD_PARAM CONFFILES
SYSTEMD_SERVICE LICENSE SECTION pkg_preinst pkg_prerm RREPLACES GROUPMEMS_PARAM
SYSTEMD_AUTO_ENABLE SKIP_FILEDEPS PRIVATE_LIBS PACKAGE_ADD_METADATA"
def gen_packagevar(d, pkgvars="PACKAGEVARS"):
ret = []
--
2.7.4
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#138653):
https://lists.openembedded.org/g/openembedded-core/message/138653
Mute This Topic: https://lists.openembedded.org/mt/74452485/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH 2/5] package_ipk.bbclass: add PACKAGE_ADD_METADATA_IPK to sstate variables
From: Michael Ho
Add PACKAGE_ADD_METADATA_IPK to IPKEXTRAVARS so it affects the sstate hash
of do_package_write_ipk.
Signed-off-by: Michael Ho
---
meta/classes/package_ipk.bbclass | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/meta/classes/package_ipk.bbclass b/meta/classes/package_ipk.bbclass
index c008559..79cb36c 100644
--- a/meta/classes/package_ipk.bbclass
+++ b/meta/classes/package_ipk.bbclass
@@ -238,7 +238,7 @@ def ipk_write_pkg(pkg, d):
bb.utils.unlockfile(lf)
# Have to list any variables referenced as X_ that aren't in pkgdata here
-IPKEXTRAVARS = "PRIORITY MAINTAINER PACKAGE_ARCH HOMEPAGE"
+IPKEXTRAVARS = "PRIORITY MAINTAINER PACKAGE_ARCH HOMEPAGE
PACKAGE_ADD_METADATA_IPK"
ipk_write_pkg[vardeps] += "${@gen_packagevar(d, 'IPKEXTRAVARS')}"
# Otherwise allarch packages may change depending on override configuration
--
2.7.4
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#138654):
https://lists.openembedded.org/g/openembedded-core/message/138654
Mute This Topic: https://lists.openembedded.org/mt/74452486/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH 5/5] ref-manual: add PACKAGE_ADD_METADATA documentation
From: Michael Ho Add a basic variable definition and a small section to the development tasks manual for using PACKAGE_ADD_METADATA to add custom metadata to packages. Signed-off-by: Michael Ho --- .../dev-manual/dev-manual-common-tasks.xml | 58 ++ documentation/ref-manual/ref-variables.xml | 34 + 2 files changed, 92 insertions(+) diff --git a/documentation/dev-manual/dev-manual-common-tasks.xml b/documentation/dev-manual/dev-manual-common-tasks.xml index e9ce182..605d1ad 100644 --- a/documentation/dev-manual/dev-manual-common-tasks.xml +++ b/documentation/dev-manual/dev-manual-common-tasks.xml @@ -9057,6 +9057,9 @@ Creating node package manager (NPM) packages + +Adding custom metadata to packages + @@ -10761,6 +10764,61 @@ + + +Adding custom metadata to packages + + +The variable PACKAGE_ADD_METADATA +can be used to add additional metadata to packages. This is +reflected in the package control/spec file. To take the ipk +format for example, the CONTROL file stored inside would +contain the additional metadata as additional lines. + + + +The variable can be used in multiple ways, including using +suffixes to set it for a specific package type and/or package. +Note that the order of precedence is the same as this list: + + + PACKAGE_ADD_METADATA_PKGTYPE_PN + + + PACKAGE_ADD_METADATA_PKGTYPE + + +PACKAGE_ADD_METADATA_PN + + +PACKAGE_ADD_METADATA + + +PKGTYPE is a parameter and expected to be a +distinct name of specific package type: + +IPK for .ipk packages +DEB for .deb packages +RPM for .rpm packages + +PN is a parameter and expected to be a package name. + + + +The variable can contain multiple [one-line] metadata fields +separated by the literal sequence '\n'. The separator can be +redefined using the variable flag separator. + + + +The following is an example that adds two custom fields for +ipk packages: + +PACKAGE_ADD_METADATA_IPK = "Vendor: CustomIpk\nGroup: Applications/Spreadsheets" + + + + diff --git a/documentation/ref-manual/ref-variables.xml b/documentation/ref-manual/ref-variables.xml index 364cd09..657f6cf 100644 --- a/documentation/ref-manual/ref-variables.xml +++ b/documentation/ref-manual/ref-variables.xml @@ -9539,6 +9539,40 @@ +PACKAGE_ADD_METADATA + +PACKAGE_ADD_METADATA[doc] = "This variable defines additional metadata to add to packages." + + + + +This variable defines additional metdata to add to packages. + + + +You may find you need to inject additional metadata into +packages. This variable allows you to do that by setting +the injected data as the value. Multiple fields can be +added by splitting the content with the literal separator +"\n". + + + +The suffixes '_IPK', '_DEB', or '_RPM' can be applied to +the variable to do package type specific settings. It can +also be made package specific by using the package name as +a suffix. + + + +You can find out more about applying this variable in +the +"Adding custom metadata to packages" +section in the Yocto Project Development Tasks Manual. + + + + PACKAGE_ARCH PACKAGE_ARCH[doc] = "The architecture of the resulting package or packages." -- 2.7.4 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#138657): https://lists.openembedded.org/g/openembedded-core/message/138657 Mute This Topic: https://lists.openembedded.org/mt/74452489/21656 Group Own
Re: [OE-core] [meta-oe][PATCH] 2/2] package_{ipk, deb, rpm}.bbclass: support per package user-defined metadata.
Hi Paulo,
Yes, this is handled by the bitbake data handlers automatically. You can
check this by trying the following:
python do_test() {
d.setVar("HELLO", "world!")
d.setVar("HELLO_space", "space!")
bb.warn("HELLO: %s" % d.getVar("HELLO", True))
d.setVar("OVERRIDES", "space")
bb.warn("HELLO: %s" % d.getVar("HELLO", True))
}
addtask test
Then calling this task with bitbake:
$ bitbake zip -c test | tee
...
WARNING: zip-3.0-r2 do_test: HELLO: world!
WARNING: zip-3.0-r2 do_test: HELLO: space!
I don't know the code or tests in oe that implement this functionality off
the top of my head so I can't point you to the oe.selftest coverage.
Kind regards,
Michael
On Fri, May 22, 2020 at 8:09 PM Paulo Neves wrote:
> Hello Michael,
>
> The behavior you describe is indeed what i wanted. If you say this is
> already working then my patches are redundant. Even so i am surprised
> it is working. Is it done automatically by the getVar function?
>
> def get_package_additional_metadata (pkg_type, d):
> base_key = "PACKAGE_ADD_METADATA"
> for key in ("%s_%s" % (base_key, pkg_type.upper()), base_key):
> if d.getVar(key, False) is None:
> continue
> d.setVarFlag(key, "type", "list")
> if d.getVarFlag(key, "separator") is None:
> d.setVarFlag(key, "separator", "\\n")
> metadata_fields = [field.strip() for field in
> oe.data.typed_value(key, d)]
> return "\n".join(metadata_fields).strip()
>
> From my understanding, i do not see how the per package override
> happens. Do you have the test for this functionality in oe-selftest?
>
> Grateful for the feedback
> Paulo Neves
>
> On Fri, May 22, 2020 at 8:00 PM Michael Ho wrote:
> >
> > Hi Paulo,
> >
> > I actually have some patches in my "to send upstream list" to address
> the documentation and some sstate-caching bugs with these variables so it's
> fresh to me at the moment.
> > Correct me if I'm wrong but I think this is already available in the
> code (applying metadata specific to a certain package in a recipe, that is).
> >
> > eg.
> http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/classes/package_ipk.bbclass
> at line 72 sets the local overrides so if PACKAGE_ADD_METADATA_IPK_xxx is
> set, it overrides PACKAGE_ADD_METADATA_IPK.
> >
> > For rpm, see
> >
> http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/classes/package_rpm.bbclass#n298
> where the recipe wide variable is fetched
> >
> http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/classes/package_rpm.bbclass#n338
> where the local override is also set
> >
> http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/classes/package_rpm.bbclass#n354
> where the package specific variables are fetched
> >
> http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/classes/package_rpm.bbclass#n436
> where the package specific variables are applied to rpm packages
> >
> > I tested the zip recipe by applying these two lines:
> > PACKAGE_ADD_METADATA_RPM = "Vendor: None"
> > PACKAGE_ADD_METADATA_RPM_zip-dbg = "Vendor: Not you!"
> > and could see differing results in the two rpm files produced.
> >
> > I think this covers the use case you're showing in your patches already,
> what do you think?
> >
> > Kind regards,
> > Michael
> >
> > (sorry if this shows up as a triple email, my mail client wasn't
> cooperating).
> >
> > On Fri, May 22, 2020 at 5:29 PM Paulo Neve wrote:
> >>
> >> Is ./meta/lib/oeqa/selftest/cases/package.py the right place?
> >> If yes i can work on it. Note that this functionality is not used
> >> anywhere in poky and is not documented at all. It just happened to
> >> exist for a long time and be useful for my case.
> >>
> >> On Fri, May 22, 2020 at 5:22 PM Alexander Kanavin
> >> wrote:
> >> >
> >> > Should there be a test for this functionality? Somewhere in
> oe-selftest perhaps where various package manager features are tested?
> >> >
> >> > Alex
> >> >
> >> > On Fri, 22 May 2020 at 17:13, Paulo Neve wrote:
> >> >>
> >> >> Up to now the user defined metadata was set recipe wide,
> >> >> meaning all the packages generated in a recipe where the
> >> >> variable was set would get the same metadata. That is not
> >> >> always ideal and is counter to the per package control that
> &g
Re: [OE-core] [meta-oe][PATCH] 2/2] package_{ipk, deb, rpm}.bbclass: support per package user-defined metadata.
Hi Paulo,
I actually have some patches in my "to send upstream list" to address the
documentation and some sstate-caching bugs with these variables so it's
fresh to me at the moment.
Correct me if I'm wrong but I think this is already available in the code
(applying metadata specific to a certain package in a recipe, that is).
eg.
http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/classes/package_ipk.bbclass
at line 72 sets the local overrides so if PACKAGE_ADD_METADATA_IPK_xxx is
set, it overrides PACKAGE_ADD_METADATA_IPK.
For rpm, see
http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/classes/package_rpm.bbclass#n298
where the recipe wide variable is fetched
http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/classes/package_rpm.bbclass#n338
where the local override is also set
http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/classes/package_rpm.bbclass#n354
where the package specific variables are fetched
http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/classes/package_rpm.bbclass#n436
where the package specific variables are applied to rpm packages
I tested the zip recipe by applying these two lines:
PACKAGE_ADD_METADATA_RPM = "Vendor: None"
PACKAGE_ADD_METADATA_RPM_zip-dbg = "Vendor: Not you!"
and could see differing results in the two rpm files produced.
I think this covers the use case you're showing in your patches already,
what do you think?
Kind regards,
Michael
(sorry if this shows up as a triple email, my mail client wasn't
cooperating).
On Fri, May 22, 2020 at 5:29 PM Paulo Neve wrote:
> Is ./meta/lib/oeqa/selftest/cases/package.py the right place?
> If yes i can work on it. Note that this functionality is not used
> anywhere in poky and is not documented at all. It just happened to
> exist for a long time and be useful for my case.
>
> On Fri, May 22, 2020 at 5:22 PM Alexander Kanavin
> wrote:
> >
> > Should there be a test for this functionality? Somewhere in oe-selftest
> perhaps where various package manager features are tested?
> >
> > Alex
> >
> > On Fri, 22 May 2020 at 17:13, Paulo Neve wrote:
> >>
> >> Up to now the user defined metadata was set recipe wide,
> >> meaning all the packages generated in a recipe where the
> >> variable was set would get the same metadata. That is not
> >> always ideal and is counter to the per package control that
> >> is usually available.
> >>
> >> Keep support for package agnostic metadata but also add option
> >> to add it to a specific option like:
> >> PACKAGE_ADD_METADATA_${PN}_IPK = "mymeta: myvalue"
> >>
> >> Signed-off-by: Paulo Neves
> >> ---
> >> meta/classes/package.bbclass | 4 ++--
> >> meta/classes/package_deb.bbclass | 2 +-
> >> meta/classes/package_ipk.bbclass | 2 +-
> >> meta/classes/package_rpm.bbclass | 5 +++--
> >> 4 files changed, 7 insertions(+), 6 deletions(-)
> >>
> >> diff --git a/meta/classes/package.bbclass b/meta/classes/package.bbclass
> >> index 0b5cf47749..1678e0d794 100644
> >> --- a/meta/classes/package.bbclass
> >> +++ b/meta/classes/package.bbclass
> >> @@ -637,9 +637,9 @@ def get_package_mapping (pkg, basepkg, d,
> depversions=None):
> >>
> >> return pkg
> >>
> >> -def get_package_additional_metadata (pkg_type, d):
> >> +def get_package_additional_metadata (pkg_type, pkg_name, d):
> >> base_key = "PACKAGE_ADD_METADATA"
> >> -for key in ("%s_%s" % (base_key, pkg_type.upper()), base_key):
> >> +for key in ("%s_%s" % (base_key, pkg_type.upper()), "%s_%s_%s" %
> (base_key, pkg_name, pkg_type.upper()), base_key):
> >> if d.getVar(key, False) is None:
> >> continue
> >> d.setVarFlag(key, "type", "list")
> >> diff --git a/meta/classes/package_deb.bbclass
> b/meta/classes/package_deb.bbclass
> >> index 790b26aef2..6ef9c8cb3a 100644
> >> --- a/meta/classes/package_deb.bbclass
> >> +++ b/meta/classes/package_deb.bbclass
> >> @@ -167,7 +167,7 @@ def deb_write_pkg(pkg, d):
> >>
> >> # more fields
> >>
> >> -custom_fields_chunk = get_package_additional_metadata("deb",
> localdata)
> >> +custom_fields_chunk = get_package_additional_metadata("deb",
> pkgname, localdata)
> >> if custom_fields_chunk:
> >> ctrlfile.write(custom_fields_chunk)
> >> ctrlfile.write("\n")
> >> diff --git a/meta/classes/package_ipk.bbclass
> b/meta/classes/package_ipk.bbclass
> >> index c008559e4a..f78cec7a70 100644
> >> --- a/meta/classes/package_ipk.bbclass
> >> +++ b/meta/classes/package_ipk.bbclass
> >> @@ -155,7 +155,7 @@ def ipk_write_pkg(pkg, d):
> >> else:
> >> ctrlfile.write(c % tuple(pullData(fs, localdata)))
> >>
> >> -custom_fields_chunk = get_package_additional_metadata("ipk",
> localdata)
> >> +custom_fields_chunk = get_package_additional_metadata("ipk",
> pkgname, localdata)
> >> if custom_fields_chunk is not None:
> >> ctrlfile.write(custom_fields_chunk)
> >> ctrlfile.write("\n")
> >> diff --git
[OE-core] [PATCH] cmake.bbclass: add HOSTTOOLS_DIR to CMAKE_FIND_ROOT_PATH
The find_program command will fail if it is used on a tool that is listed in
ASSUME_PROVIDED. This is because these tools are in the hosttools directory
which is not listed in CMAKE_FIND_ROOT_PATH so cmake will not find them.
Adding the directory HOSTTOOLS_DIR to the CMAKE_FIND_ROOT_PATH variable fixes
the initial issue of needing to search for tools in ASSUME_PROVIDED.
Note that this change alone does not fix the issue because find_program will
by default only look into the subdirectories bin and usr/bin under the paths
in CMAKE_FIND_ROOT_PATH to find the programs and the hosttools directory has
instead the symlinks directly present without these subdirectories.
Set CMAKE_PROGRAM_PATH to by default include the root directory so
find_program can search the hosttools directory without needing the prefix
directories.
---
meta/classes/cmake.bbclass | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/meta/classes/cmake.bbclass b/meta/classes/cmake.bbclass
index d3f0d70..f41eb8d 100644
--- a/meta/classes/cmake.bbclass
+++ b/meta/classes/cmake.bbclass
@@ -99,11 +99,12 @@ set( CMAKE_CXX_LINK_FLAGS "${OECMAKE_CXX_LINK_FLAGS}" CACHE
STRING "LDFLAGS" )
# only search in the paths provided so cmake doesnt pick
# up libraries and tools from the native build machine
-set( CMAKE_FIND_ROOT_PATH ${STAGING_DIR_HOST} ${STAGING_DIR_NATIVE}
${CROSS_DIR} ${OECMAKE_PERLNATIVE_DIR} ${OECMAKE_EXTRA_ROOT_PATH}
${EXTERNAL_TOOLCHAIN})
+set( CMAKE_FIND_ROOT_PATH ${STAGING_DIR_HOST} ${STAGING_DIR_NATIVE}
${CROSS_DIR} ${OECMAKE_PERLNATIVE_DIR} ${OECMAKE_EXTRA_ROOT_PATH}
${EXTERNAL_TOOLCHAIN} ${HOSTTOOLS_DIR})
set( CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY )
set( CMAKE_FIND_ROOT_PATH_MODE_PROGRAM ${OECMAKE_FIND_ROOT_PATH_MODE_PROGRAM} )
set( CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY )
set( CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY )
+set( CMAKE_PROGRAM_PATH "/" )
# Use qt.conf settings
set( ENV{QT_CONF_PATH} ${WORKDIR}/qt.conf )
--
2.7.4
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] cmake find_program and hosttools_dir
Hi all,
I wanted to ask what would be a good way to fix an issue with cmake
find_program and the tools listed in ASSUME_PROVIDED. I checked against the
Warrior branch and it seems that currently find_program does not search the
hosttools directory for programs because HOSTTOOLS_DIR is not listed in the
CMAKE_FIND_ROOT_PATH variable (in cmake.bbclass). This means if you call
find_program to get the path to the git binary, it will fail to find the
program. I've attached a minimal recipe and CMakeLists.txt to demonstrate.
If you attempt to compile the recipe, you'll see:
| CMake Error at CMakeLists.txt:6 (message):
| No git binary found
If I modify the cmake.bbclass and add this directory to CMAKE_FIND_ROOT_PATH,
it still fails because by default it wants to only search in the
subdirectories bin and usr/bin. If I then modify base.bbclass to generate a
"bin" symlink in the hosttools directory that points back to the hosttools
directory, it is enough to trick cmake. I also found a second solution where
if I instead add a default definition to CMAKE_PROGRAM_PATH to include the
root directory, this works also. I've included this second fix in the patch
I've sent.
Is this a good solution or is there something I'm missing here?
Thanks.
Kind regards,
Michael Ho
$ cat files/CMakeLists.txt
cmake_minimum_required (VERSION 2.6)
project (Tutorial)
find_program(HOST_GIT git)
if(NOT HOST_GIT)
message(FATAL_ERROR "No git binary found")
endif()
$ cat find-git.bb
SUMMARY = "Example recipe to test find_program and git-native"
LICENSE = "CLOSED"
DEPENDS = "git-native"
SRC_URI = "file://CMakeLists.txt"
S = "${WORKDIR}"
inherit cmake
do_install() {
:
}
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v2] base.bbclass: add named SRCREVs to the sstate hash
Several fetchers support named sources that require setting a SRCREV with
the source name as a suffix. These named SRCREV variables are not captured
in the sstate hash calculation because they're only referenced within the
bitbake fetcher function. Several recipes avoid this bug by adding the
different SRCREVs to their packaging versioning but this is not enforced so
it is very trivial to trip this bug case.
Add a snippet to the base.bbclass anonymous python to add all named SRCREV
variables to the vardeps of do_fetch to capture them in the sstate hash
calculation.
Testing of the bug can be shown by running the following bitbake commands
with this patch set not applied:
bitbake vulkan-demos | tee
sed -i 's/SRCREV_gli = ".*"/SRCREV_gli = "xxx"/' \
../meta/recipes-graphics/vulkan/vulkan-demos_git.bb
bitbake vulkan-demos | tee;
The above results in no errors despite a broken SRCREV because the recipe is
considered unchanged by the bitbake sstate hash.
After applying this patch the above commands instead result in a fetcher
error which is correct.
---
meta/classes/base.bbclass | 12
1 file changed, 12 insertions(+)
diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
index 1636c6e..25d5fff 100644
--- a/meta/classes/base.bbclass
+++ b/meta/classes/base.bbclass
@@ -639,6 +639,18 @@ python () {
if needsrcrev:
d.setVar("SRCPV", "${@bb.fetch2.get_srcrev(d)}")
+# Gather all named SRCREVs to add to the sstate hash calculation
+# This anonymous python snippet is called multiple times so we
+# need to be careful to not double up the appends here and cause
+# the base hash to mismatch the task hash
+for uri in srcuri.split():
+parm = bb.fetch.decodeurl(uri)[5]
+uri_names = parm.get("name", "").split(",")
+for uri_name in filter(None, uri_names):
+srcrev_name = "SRCREV_{}".format(uri_name)
+if srcrev_name not in (d.getVarFlag("do_fetch", "vardeps") or
"").split():
+d.appendVarFlag("do_fetch", "vardeps", "
{}".format(srcrev_name))
+
set_packagetriplet(d)
# 'multimachine' handling
--
2.7.4
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH] base.bbclass: add named SRCREVs to the sstate hash
Several fetchers support named sources that require setting a SRCREV with
the source name as a suffix. These named SRCREV variables are not captured
in the sstate hash calculation because they're only referenced within the
bitbake fetcher function.
Add a snippet to the base.bbclass anonymous python to add all named SRCREV
variables to the vardeps of do_fetch to capture them in the sstate hash
calculation.
Testing of the bug can be shown by running the following bitbake commands
with this patch set not applied:
bitbake vulkan-demos | tee
sed -i 's/SRCREV_gli = ".*"/SRCREV_gli = "xxx"/' \
../meta/recipes-graphics/vulkan/vulkan-demos_git.bb
bitbake vulkan-demos | tee;
Results in no errors despite a broken SRCREV because the vulkan-demos is
considered unchanged.
After applying this patch the above commands instead result in a fetcher
error which is correct.
---
meta/classes/base.bbclass | 10 ++
1 file changed, 10 insertions(+)
diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
index 1636c6e..84a27f5 100644
--- a/meta/classes/base.bbclass
+++ b/meta/classes/base.bbclass
@@ -638,6 +638,16 @@ python () {
if needsrcrev:
d.setVar("SRCPV", "${@bb.fetch2.get_srcrev(d)}")
+# Gather all SRCREV_* references to add to the sstate hash calculation
+# This may capture SRCREVs not used but it's difficult to try to
restrict it
+# to only what is needed
+for dkey in d.keys():
+if dkey.startswith("SRCREV_"):
+# This anonymous python snippet is called multiple times so we
+# need to be careful to not double up the appends here and
cause
+# the base hash to mismatch the task hash
+if dkey not in (d.getVarFlag("do_fetch", "vardeps") or
'').split():
+d.appendVarFlag("do_fetch", "vardeps", " {}".format(dkey))
set_packagetriplet(d)
--
2.7.4
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
