Re: [OE-core] GO runtime crashes
Hi, On Thu, 28 Feb 2019 at 09:04, Vincent Prince wrote: > > Hi Khem, > > GO_DYNLINK_x86-64 = "" solution works as well, > > Best regards, > Vincent > > Le jeu. 28 févr. 2019 à 14:14, Khem Raj a écrit : >> >> On Thu, Feb 28, 2019 at 1:35 AM Vincent Prince >> wrote: >> > >> > Hi Damien, >> > >> > With GO_LINKSHARED = "" it work fine. >> > >> >> Please set >> GO_DYNLINK_x86-64 = "1" >> to >> GO_DYNLINK_x86-64 = "" >> in goarch.bbclass Well, making a change in the goarch.bbclass is way more intrusive than changing the value in a single recipe, so it would be nice to (at least) give some justification about this solution. >> >> > I'm very new to Go, does it mean that there is a mixing issue in go >> > modules from Yocto and from node_exporter vendor folder, so it links >> > wrongly at runtime? As to why the GO_LINKSHARED ="" works, I cannot tell. I just noticed that our application was linked differently on my machine (where it worked) and in Yocto (where it didn't work), so I tweaked link flags until I got it to work. Cheers, Damien -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] GO runtime crashes
On Wed, 27 Feb 2019 at 12:10, Vincent Prince wrote: > > Hello everyone, > > I'm trying to add node_exporter from Prometheus project to an Intel x86-64 > machine. > I made the following recipe: > > https://github.com/nefethael/meta-random/blob/master/recipes-connectivity/prometheus/go-nodeexporter_0.18.0.bb > > node_exporter is crashing with multiple different runtime errors, so I raised > an issue on github: > https://github.com/prometheus/node_exporter/issues/1244 I've faced the same kind of issue with a custom application of ours. The fix was to add the following line to the recipe: GO_LINKSHARED = "" Let me know if that helps. Cheers, Damien -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] [RFC] go.bbclass: install dependencies under $GOPATH instead of $GOROOT
On Thu, 20 Dec 2018 at 11:00, Matt Madison wrote:
>
> On Thu, Dec 20, 2018 at 6:02 AM Damien Riegel wrote:
> >
> > On Thu, 20 Dec 2018 at 07:53, Matt Madison wrote:
> > >
> > > On Wed, Dec 19, 2018 at 5:52 PM Damien Riegel
> > > wrote:
> > > >
> > > > On Wed, 19 Dec 2018 at 16:41, Matt Madison wrote:
> > > > >
> > > > > On Wed, Dec 19, 2018 at 9:51 AM Damien Riegel
> > > > > wrote:
> > > > > >
> > > > > > To package a go application in Yocto, one needs to also package its
> > > > > > dependencies to keep the build reproducible and under control. The
> > > > > > default install task of the go class puts the source in
> > > > > > ${libdir}/go/src.
> > > > > >
> > > > > > ${libdir}/go/src is where the standard go packages resides, aka
> > > > > > $GOROOT.
> > > > > > When a go package has dependencies on other go packages, they must
> > > > > > be
> > > > > > located under $GOPATH, otherwise errors like this one may happen:
> > > > > >
> > > > > > Non-standard package in standard package
> > > > > >
> > > > > >
> > > > > > This point of this patch is to trigger a discussion on how this
> > > > > > issue
> > > > > > can be tackled in Yocto. Following code is working but should not be
> > > > > > considered for inclusion.
> > > > >
> > > > > Can you describe the context of this a bit more? The way go.bbclass
> > > > > is set up, all of your recipe's dependencies (and their dependencies)
> > > > > should already have been installed in the per-recipe sysroot (where
> > > > > GOROOT lives for the current recipe's build), and should get resolved
> > > > > from there. GOPATH gets set to ${B} for the current recipe's build.
> > > > > Nothing in the GOROOT should be importing packages that reside in
> > > > > GOPATH, which is what that particular error message is indicating. If
> > > > > you're getting that error, then there might be something wrong with
> > > > > recipe dependencies.
> > > >
> > > > Sure. I think the issue is because I have the following chain of
> > > > dependencies:
> > > >
> > > > A --> B --> C
> > > > `-> C
> > > >
> > > > Building C is fine, building B is fine, but building A triggers the
> > > > error I mentioned, because the Go compiler complains about B using a
> > > > non-standard package C. It thinks B is a standard package because it's
> > > > located in GOROOT.
> > >
> > > Are the three packages in separate source repositories? If so, the go
> > > tool should not be finding any C files in GOPATH.
> > >
> > > If A and C are part of the same source repository, but B is separate,
> > > that would make the situation a bit trickier, but still solvable.
> >
> > They are three separate repositories. The issue is precisely that C
> > files are not in GOPATH but in GOROOT. C files _should be_ in GOPATH or
> > that will break Go tools.
>
> Go does allow for non-"standard" packages to be in GOROOT as well.
> The way the go tool marks a package as "standard" is if it resides in
> GOROOT _and_ starts with a prefix that does not look like a domain
> name (specifically, does not contain a dot '.'). The Go convention,
> though, is to use a domain name as the first element of the import
> path for anything that isn't part of the base runtime (std, cmd).
Damn, I think that's the issue, some packages don't use a domain name
as the first element of the import path.
> If you do have a situation where you have multiple interdependent
> packages that for some reason have import paths that do not start with
> a domain name, you could build them all using a single recipe, rather
> than splitting them up into separate recipes. I've had to do that for
> cases where there were circular dependencies between two (or more)
> packages; the same technique could work this, though, too.
Okay, got it. There are no circular dependencies here, it's really a
straightforward situation "A needs B", so fixing the package names
should solve the issue.
On a side note, it would be nice to have documentation somewhere on
how to use the go class, especially for GO_IMPORT and GO_INSTALL.
Does that belong in the class file itself? Or should that be put in the doc
repository?
Thank your very much for your help!
--
Damien
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] [RFC] go.bbclass: install dependencies under $GOPATH instead of $GOROOT
On Thu, 20 Dec 2018 at 07:53, Matt Madison wrote:
>
> On Wed, Dec 19, 2018 at 5:52 PM Damien Riegel wrote:
> >
> > On Wed, 19 Dec 2018 at 16:41, Matt Madison wrote:
> > >
> > > On Wed, Dec 19, 2018 at 9:51 AM Damien Riegel
> > > wrote:
> > > >
> > > > To package a go application in Yocto, one needs to also package its
> > > > dependencies to keep the build reproducible and under control. The
> > > > default install task of the go class puts the source in
> > > > ${libdir}/go/src.
> > > >
> > > > ${libdir}/go/src is where the standard go packages resides, aka $GOROOT.
> > > > When a go package has dependencies on other go packages, they must be
> > > > located under $GOPATH, otherwise errors like this one may happen:
> > > >
> > > > Non-standard package in standard package
> > > >
> > > > This point of this patch is to trigger a discussion on how this issue
> > > > can be tackled in Yocto. Following code is working but should not be
> > > > considered for inclusion.
> > >
> > > Can you describe the context of this a bit more? The way go.bbclass
> > > is set up, all of your recipe's dependencies (and their dependencies)
> > > should already have been installed in the per-recipe sysroot (where
> > > GOROOT lives for the current recipe's build), and should get resolved
> > > from there. GOPATH gets set to ${B} for the current recipe's build.
> > > Nothing in the GOROOT should be importing packages that reside in
> > > GOPATH, which is what that particular error message is indicating. If
> > > you're getting that error, then there might be something wrong with
> > > recipe dependencies.
> >
> > Sure. I think the issue is because I have the following chain of
> > dependencies:
> >
> > A --> B --> C
> > `-> C
> >
> > Building C is fine, building B is fine, but building A triggers the
> > error I mentioned, because the Go compiler complains about B using a
> > non-standard package C. It thinks B is a standard package because it's
> > located in GOROOT.
>
> Are the three packages in separate source repositories? If so, the go
> tool should not be finding any C files in GOPATH.
>
> If A and C are part of the same source repository, but B is separate,
> that would make the situation a bit trickier, but still solvable.
They are three separate repositories. The issue is precisely that C
files are not in GOPATH but in GOROOT. C files _should be_ in GOPATH or
that will break Go tools.
> > Regarding dependencies, I created recipes named foocorp.com-{A,B,C}.bb
> > and used DEPENDS_append = " foocorp.com-A" for instance.
>
> If A imports from B and C, you should have DEPENDS = "foocorp.com-B
> foocorp.com-C" in the recipe for A, and
> DEPENDS = "foocorp.com-C" in the recipe for B.
Okay, that part I got right then!
> > I understand it's convenient to install dependencies in the GOROOT
> > directory of recipes' sysroot, but we should keep in mind that when one
> > calls `go get`, dependencies are installed in GOPATH, not GOROOT. I
> > think that if we want to avoid (a bit obscure) build failures and cover
> > all use cases, we should try to mimic that behaviour as close as we can.
>
> go.bbclass does not use `go get`. It only uses `go install`.
Sorry, that was not clear, I was talking about Go development outside of
Yocto. So outside of Yocto, I would use `go fetch` to get the
dependencies and they would end up somewhere in GOPATH. When packaging
these dependencies in Yocto, they will end up in the GOROOT of recipes'
sysroots, and that breaks builds.
My point was that if we want to be able to write recipe for any valid Go
package, dependencies should be found under GOPATH, not GOROOT.
> -Matt
>
> >
> > >
> > > -Matt
> > > >
> > > > Signed-off-by: Damien Riegel
> > > > ---
> > > > meta/classes/go.bbclass | 18 +++---
> > > > 1 file changed, 15 insertions(+), 3 deletions(-)
> > > >
> > > > diff --git a/meta/classes/go.bbclass b/meta/classes/go.bbclass
> > > > index 167d02e3fa..c0ed3211cd 100644
> > > > --- a/meta/classes/go.bbclass
> > > > +++ b/meta/classes/go.bbclass
> > > > @@ -111,9 +111,9 @@ do_compile_ptest_base() {
> > > > do_compile_ptest_base[dirs] =+ "${GOTMPDIR}"
> > > >
> > > > go_do_install() {
> > > > - install -d ${D}${lib
Re: [OE-core] [PATCH] [RFC] go.bbclass: install dependencies under $GOPATH instead of $GOROOT
On Wed, 19 Dec 2018 at 16:41, Matt Madison wrote:
>
> On Wed, Dec 19, 2018 at 9:51 AM Damien Riegel wrote:
> >
> > To package a go application in Yocto, one needs to also package its
> > dependencies to keep the build reproducible and under control. The
> > default install task of the go class puts the source in ${libdir}/go/src.
> >
> > ${libdir}/go/src is where the standard go packages resides, aka $GOROOT.
> > When a go package has dependencies on other go packages, they must be
> > located under $GOPATH, otherwise errors like this one may happen:
> >
> > Non-standard package in standard package
> >
> > This point of this patch is to trigger a discussion on how this issue
> > can be tackled in Yocto. Following code is working but should not be
> > considered for inclusion.
>
> Can you describe the context of this a bit more? The way go.bbclass
> is set up, all of your recipe's dependencies (and their dependencies)
> should already have been installed in the per-recipe sysroot (where
> GOROOT lives for the current recipe's build), and should get resolved
> from there. GOPATH gets set to ${B} for the current recipe's build.
> Nothing in the GOROOT should be importing packages that reside in
> GOPATH, which is what that particular error message is indicating. If
> you're getting that error, then there might be something wrong with
> recipe dependencies.
Sure. I think the issue is because I have the following chain of
dependencies:
A --> B --> C
`-> C
Building C is fine, building B is fine, but building A triggers the
error I mentioned, because the Go compiler complains about B using a
non-standard package C. It thinks B is a standard package because it's
located in GOROOT.
Regarding dependencies, I created recipes named foocorp.com-{A,B,C}.bb
and used DEPENDS_append = " foocorp.com-A" for instance.
I understand it's convenient to install dependencies in the GOROOT
directory of recipes' sysroot, but we should keep in mind that when one
calls `go get`, dependencies are installed in GOPATH, not GOROOT. I
think that if we want to avoid (a bit obscure) build failures and cover
all use cases, we should try to mimic that behaviour as close as we can.
>
> -Matt
> >
> > Signed-off-by: Damien Riegel
> > ---
> > meta/classes/go.bbclass | 18 +++---
> > 1 file changed, 15 insertions(+), 3 deletions(-)
> >
> > diff --git a/meta/classes/go.bbclass b/meta/classes/go.bbclass
> > index 167d02e3fa..c0ed3211cd 100644
> > --- a/meta/classes/go.bbclass
> > +++ b/meta/classes/go.bbclass
> > @@ -111,9 +111,9 @@ do_compile_ptest_base() {
> > do_compile_ptest_base[dirs] =+ "${GOTMPDIR}"
> >
> > go_do_install() {
> > - install -d ${D}${libdir}/go/src/${GO_IMPORT}
> > + install -d ${D}/gopath/src/${GO_IMPORT} ${D}${libdir}/go
> > tar -C ${S}/src/${GO_IMPORT} -cf - --exclude-vcs --exclude '*.test'
> > --exclude 'testdata' . | \
> > - tar -C ${D}${libdir}/go/src/${GO_IMPORT} --no-same-owner
> > -xf -
> > + tar -C ${D}/gopath/src/${GO_IMPORT} --no-same-owner -xf -
> > tar -C ${B} -cf - pkg | tar -C ${D}${libdir}/go --no-same-owner -xf
> > -
> >
> > if [ -n "`ls ${B}/${GO_BUILD_BINDIR}/`" ]; then
> > @@ -172,9 +172,21 @@ do_install_ptest_base() {
> > chown -R root:root ${D}${PTEST_PATH}
> > }
> >
> > +do_fixupdeps() {
> > + gopath="${WORKDIR}/recipe-sysroot/gopath"
> > + if [ -d ${gopath} ]; then
> > + tar -C ${gopath} -cf - src | tar -C ${B} --no-same-owner -k
> > -xf -
> > + else
> > + echo "no dependencies to fixup"
> > + fi
> > +}
> > +
> > EXPORT_FUNCTIONS do_unpack do_configure do_compile do_install
> >
> > -FILES_${PN}-dev = "${libdir}/go/src"
> > +addtask do_fixupdeps after do_configure before do_compile
> > +
> > +SYSROOT_DIRS_append = "/gopath"
> > +FILES_${PN}-dev = "/gopath/src"
> > FILES_${PN}-staticdev = "${libdir}/go/pkg"
> >
> > INSANE_SKIP_${PN} += "ldflags"
> > --
> > 2.19.2
> >
> > --
> > ___
> > Openembedded-core mailing list
> > Openembedded-core@lists.openembedded.org
> > http://lists.openembedded.org/mailman/listinfo/openembedded-core
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH] [RFC] go.bbclass: install dependencies under $GOPATH instead of $GOROOT
To package a go application in Yocto, one needs to also package its
dependencies to keep the build reproducible and under control. The
default install task of the go class puts the source in ${libdir}/go/src.
${libdir}/go/src is where the standard go packages resides, aka $GOROOT.
When a go package has dependencies on other go packages, they must be
located under $GOPATH, otherwise errors like this one may happen:
Non-standard package in standard package
This point of this patch is to trigger a discussion on how this issue
can be tackled in Yocto. Following code is working but should not be
considered for inclusion.
Signed-off-by: Damien Riegel
---
meta/classes/go.bbclass | 18 +++---
1 file changed, 15 insertions(+), 3 deletions(-)
diff --git a/meta/classes/go.bbclass b/meta/classes/go.bbclass
index 167d02e3fa..c0ed3211cd 100644
--- a/meta/classes/go.bbclass
+++ b/meta/classes/go.bbclass
@@ -111,9 +111,9 @@ do_compile_ptest_base() {
do_compile_ptest_base[dirs] =+ "${GOTMPDIR}"
go_do_install() {
- install -d ${D}${libdir}/go/src/${GO_IMPORT}
+ install -d ${D}/gopath/src/${GO_IMPORT} ${D}${libdir}/go
tar -C ${S}/src/${GO_IMPORT} -cf - --exclude-vcs --exclude '*.test'
--exclude 'testdata' . | \
- tar -C ${D}${libdir}/go/src/${GO_IMPORT} --no-same-owner -xf -
+ tar -C ${D}/gopath/src/${GO_IMPORT} --no-same-owner -xf -
tar -C ${B} -cf - pkg | tar -C ${D}${libdir}/go --no-same-owner -xf -
if [ -n "`ls ${B}/${GO_BUILD_BINDIR}/`" ]; then
@@ -172,9 +172,21 @@ do_install_ptest_base() {
chown -R root:root ${D}${PTEST_PATH}
}
+do_fixupdeps() {
+ gopath="${WORKDIR}/recipe-sysroot/gopath"
+ if [ -d ${gopath} ]; then
+ tar -C ${gopath} -cf - src | tar -C ${B} --no-same-owner -k -xf
-
+ else
+ echo "no dependencies to fixup"
+ fi
+}
+
EXPORT_FUNCTIONS do_unpack do_configure do_compile do_install
-FILES_${PN}-dev = "${libdir}/go/src"
+addtask do_fixupdeps after do_configure before do_compile
+
+SYSROOT_DIRS_append = "/gopath"
+FILES_${PN}-dev = "/gopath/src"
FILES_${PN}-staticdev = "${libdir}/go/pkg"
INSANE_SKIP_${PN} += "ldflags"
--
2.19.2
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v2 1/1] systemd: escape paths passed to shell
Systemd mount configuration file must have a name that match the mount
point directory they control. So for instance, if a mount file contains
[Mount]
...
Where=/mnt/my-data
The file must be named `mnt-my\x2ddata.mount`, or systemd will refuse to
honour it.
If this config file contains an [Install] section, it will silently fail
because the unit file is not escaped properly when systemctl is called.
To fix that, make sure paths are escaped through `shlex.quote`.
Signed-off-by: Damien Riegel
---
Changes in v2:
- v1 didn't handle multiple files in SYSTEMD_SERVICE_${PN}, this v2
fixes using a solution suggested by Christopher Larson
meta/classes/systemd.bbclass | 14 +-
1 file changed, 9 insertions(+), 5 deletions(-)
diff --git a/meta/classes/systemd.bbclass b/meta/classes/systemd.bbclass
index 1b134322fb..c7b784dea8 100644
--- a/meta/classes/systemd.bbclass
+++ b/meta/classes/systemd.bbclass
@@ -34,10 +34,10 @@ if type systemctl >/dev/null 2>/dev/null; then
systemctl daemon-reload
fi
- systemctl $OPTS ${SYSTEMD_AUTO_ENABLE} ${SYSTEMD_SERVICE}
+ systemctl $OPTS ${SYSTEMD_AUTO_ENABLE} ${SYSTEMD_SERVICE_ESCAPED}
if [ -z "$D" -a "${SYSTEMD_AUTO_ENABLE}" = "enable" ]; then
- systemctl --no-block restart ${SYSTEMD_SERVICE}
+ systemctl --no-block restart ${SYSTEMD_SERVICE_ESCAPED}
fi
fi
}
@@ -51,10 +51,10 @@ fi
if type systemctl >/dev/null 2>/dev/null; then
if [ -z "$D" ]; then
- systemctl stop ${SYSTEMD_SERVICE}
+ systemctl stop ${SYSTEMD_SERVICE_ESCAPED}
fi
- systemctl $OPTS disable ${SYSTEMD_SERVICE}
+ systemctl $OPTS disable ${SYSTEMD_SERVICE_ESCAPED}
fi
}
@@ -65,6 +65,7 @@ systemd_populate_packages[vardepsexclude] += "OVERRIDES"
python systemd_populate_packages() {
import re
+import shlex
if not bb.utils.contains('DISTRO_FEATURES', 'systemd', True, False, d):
return
@@ -85,6 +86,9 @@ python systemd_populate_packages() {
def systemd_generate_package_scripts(pkg):
bb.debug(1, 'adding systemd calls to postinst/postrm for %s' % pkg)
+paths_escaped = ' '.join(shlex.quote(s) for s in
d.getVar('SYSTEMD_SERVICE_' + pkg, True).split())
+d.setVar('SYSTEMD_SERVICE_ESCAPED_' + pkg, paths_escaped)
+
# Add pkg to the overrides so that it finds the SYSTEMD_SERVICE_pkg
# variable.
localdata = d.createCopy()
@@ -130,7 +134,7 @@ python systemd_populate_packages() {
systemd_add_files_and_parse(pkg_systemd, path, service_base +
'@.service', keys)
for key in keys.split():
# recurse all dependencies found in keys
('Also';'Conflicts';..) and add to files
-cmd = "grep %s %s | sed 's,%s=,,g' | tr ',' '\\n'" % (key,
fullpath, key)
+cmd = "grep %s %s | sed 's,%s=,,g' | tr ',' '\\n'" % (key,
shlex.quote(fullpath), key)
pipe = os.popen(cmd, 'r')
line = pipe.readline()
while line:
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v2 0/1] systemd: escape paths
This is the second attempt at escaping properly the paths passed to
various shell scripts in the systemd class.
I tried various approch to this fix but I'm still not sure I have the
cleanest solution:
- using inline ${@' '.join(shlex.quote(s) for s in
d.getVar('SYSTEMD_SERVICE').split()}
doesn't work as shlex is not imported
- creating a `def systemd_escape_paths(d)` function and calling it
inline through ${@systemd_escape_paths(d)) has issues as well:
- d.getVar('SYSTEMD_SERVICE') returned None (remember the variable
is actually SYSTEMD_SERVICE_${PN})
- d.expand('SYSTEMD_SERVICE') returns 'SYSTEMD_SERVICE'
The solution I came up with and that actually works is to inject
SYSTEMD_SERVICE_ESCAPED in the data store, and use that in the postinst
and prerm scripts.
Damien Riegel (1):
systemd: escape paths passed to shell
meta/classes/systemd.bbclass | 14 +-
1 file changed, 9 insertions(+), 5 deletions(-)
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] systemd: quote SYSTEMD_SERVICE in prerm and postinst
On Fri, Jun 22, 2018 at 08:32:14AM -0700, Christopher Larson wrote:
> Have you tried something like this? Might need to shift it to a def'd python
> function to
> ensure shlex gets imported, though.
>
> systemctl $OPTS disable ${@' '.join(shlex.quote(s) for s in d.getVar
> ('SYSTEMD_SERVICE').split()}
Cool, looks way cleaner. I'll give it a try and send a v2 with that if I
can make it work.
Thanks,
--
Damien
> On Thu, Jun 21, 2018 at 1:36 PM Damien Riegel
> <[1]damien.rie...@savoirfairelinux.com>
> wrote:
>
> Systemd mount configuration file must have a name that match the mount
> point directory they control. So for instance, if a mount file contains
>
> [Mount]
> ...
> Where=/mnt/my-data
>
> The file must be named `mnt-my\x2ddata.mount`, or systemd will refuse to
> honour it.
>
> If this config file contains an [Install] section, it will silently fail
> because the unit file is not quoted when systemctl is called. To fix
> that, quote `${SYSTEMD_SERVICE}` in prerm and postinst hooks.
>
> Signed-off-by: Damien Riegel <[2]damien.rie...@savoirfairelinux.com>
> ---
> meta/classes/systemd.bbclass | 8
> 1 file changed, 4 insertions(+), 4 deletions(-)
>
> diff --git a/meta/classes/systemd.bbclass b/meta/classes/systemd.bbclass
> index 1b134322fb..589e73c855 100644
> --- a/meta/classes/systemd.bbclass
> +++ b/meta/classes/systemd.bbclass
> @@ -34,10 +34,10 @@ if type systemctl >/dev/null 2>/dev/null; then
> systemctl daemon-reload
> fi
>
> - systemctl $OPTS ${SYSTEMD_AUTO_ENABLE} ${SYSTEMD_SERVICE}
> + systemctl $OPTS ${SYSTEMD_AUTO_ENABLE} "${SYSTEMD_SERVICE}"
>
> if [ -z "$D" -a "${SYSTEMD_AUTO_ENABLE}" = "enable" ]; then
> - systemctl --no-block restart ${SYSTEMD_SERVICE}
> + systemctl --no-block restart "${SYSTEMD_SERVICE}"
> fi
> fi
> }
> @@ -51,10 +51,10 @@ fi
>
> if type systemctl >/dev/null 2>/dev/null; then
> if [ -z "$D" ]; then
> - systemctl stop ${SYSTEMD_SERVICE}
> + systemctl stop "${SYSTEMD_SERVICE}"
> fi
>
> - systemctl $OPTS disable ${SYSTEMD_SERVICE}
> + systemctl $OPTS disable "${SYSTEMD_SERVICE}"
> fi
> }
>
> --
> 2.17.1
>
> --
> ___
> Openembedded-core mailing list
> [3]Openembedded-core@lists.openembedded.org
> [4]http://lists.openembedded.org/mailman/listinfo/openembedded-core
>
>
>
> --
> Christopher Larson
> kergoth at gmail dot com
> Founder - BitBake, OpenEmbedded, OpenZaurus
> Senior Software Engineer, Mentor Graphics
>
> References:
>
> [1] mailto:damien.rie...@savoirfairelinux.com
> [2] mailto:damien.rie...@savoirfairelinux.com
> [3] mailto:Openembedded-core@lists.openembedded.org
> [4] http://lists.openembedded.org/mailman/listinfo/openembedded-core
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] systemd: quote SYSTEMD_SERVICE in prerm and postinst
v2 looks like the snippet below, but I'm not very fond of these path
manipulations. Any suggestion for a better solution?
Regards,
--
Damien
diff --git a/meta/classes/systemd.bbclass b/meta/classes/systemd.bbclass
index 589e73c855..9bacdecb9b 100644
--- a/meta/classes/systemd.bbclass
+++ b/meta/classes/systemd.bbclass
@@ -81,10 +81,27 @@ python systemd_populate_packages() {
if not pkg_systemd in packages.split():
bb.error('%s does not appear in package list, please add it' %
pkg_systemd)
+def systemd_path_replace(pkg, old, new):
+ services = d.getVar('SYSTEMD_SERVICE_' + pkg, True)
+ if services:
+ services = services.replace(old, new)
+ d.setVar('SYSTEMD_SERVICE_' + pkg, services
+
+def systemd_path_unescape(pkg):
+systemd_path_replace(pkg, r'\\x2d', r'\x2d')
+
+def systemd_path_escape(pkg):
+systemd_path_replace(pkg, r'\x2d', r'\\x2d')
def systemd_generate_package_scripts(pkg):
bb.debug(1, 'adding systemd calls to postinst/postrm for %s' % pkg)
+# Some systemd config files may have paths that contain escape
+# characters, but systemctl will fail to install these units if
+# paths are not escaped, so tweak these paths during generation
+# of prerm and postinst scripts
+systemd_path_escape(pkg)
+
# Add pkg to the overrides so that it finds the SYSTEMD_SERVICE_pkg
# variable.
localdata = d.createCopy()
@@ -102,6 +119,8 @@ python systemd_populate_packages() {
prerm += localdata.getVar('systemd_prerm')
d.setVar('pkg_prerm_%s' % pkg, prerm)
+systemd_path_unescape(pkg)
+
# Add files to FILES_*-systemd if existent and not already done
def systemd_append_file(pkg_systemd, file_append):
@@ -130,7 +149,7 @@ python systemd_populate_packages() {
systemd_add_files_and_parse(pkg_systemd, path, service_base +
'@.service', keys)
for key in keys.split():
# recurse all dependencies found in keys
('Also';'Conflicts';..) and add to files
-cmd = "grep %s %s | sed 's,%s=,,g' | tr ',' '\\n'" % (key,
fullpath, key)
+cmd = "grep %s '%s' | sed 's,%s=,,g' | tr ',' '\\n'" % (key,
fullpath, key)
pipe = os.popen(cmd, 'r')
line = pipe.readline()
while line:
On Thu, Jun 21, 2018 at 10:35:56PM -0400, Damien Riegel wrote:
> Hi,
>
> On Fri, Jun 22, 2018 at 09:29:37AM +0800, ChenQi wrote:
> > Hi Damien,
> >
> > If some recipe has more than one unit file, this patch will cause problem
> > for it.
>
> That's a good point, let me see if I can come up with a solution that
> addresses that.
>
> Regards,
> --
> Damien
>
> >
> > Best Regards,
> > Chen Qi
> >
> >
> > On 06/22/2018 04:35 AM, Damien Riegel wrote:
> > > Systemd mount configuration file must have a name that match the mount
> > > point directory they control. So for instance, if a mount file contains
> > >
> > > [Mount]
> > > ...
> > > Where=/mnt/my-data
> > >
> > > The file must be named `mnt-my\x2ddata.mount`, or systemd will refuse to
> > > honour it.
> > >
> > > If this config file contains an [Install] section, it will silently fail
> > > because the unit file is not quoted when systemctl is called. To fix
> > > that, quote `${SYSTEMD_SERVICE}` in prerm and postinst hooks.
> > >
> > > Signed-off-by: Damien Riegel
> > > ---
> > > meta/classes/systemd.bbclass | 8
> > > 1 file changed, 4 insertions(+), 4 deletions(-)
> > >
> > > diff --git a/meta/classes/systemd.bbclass b/meta/classes/systemd.bbclass
> > > index 1b134322fb..589e73c855 100644
> > > --- a/meta/classes/systemd.bbclass
> > > +++ b/meta/classes/systemd.bbclass
> > > @@ -34,10 +34,10 @@ if type systemctl >/dev/null 2>/dev/null; then
> > > systemctl daemon-reload
> > > fi
> > > - systemctl $OPTS ${SYSTEMD_AUTO_ENABLE} ${SYSTEMD_SERVICE}
> > > + systemctl $OPTS ${SYSTEMD_AUTO_ENABLE} "${SYSTEMD_SERVICE}"
> > > if [ -z "$D" -a "${SYSTEMD_AUTO_ENABLE}" = "enable" ]; then
> > > - systemctl --no-block restart ${SYSTEMD_SERVICE}
> > > + systemctl --no-block restart "${SYSTEMD_SERVICE}"
> > > fi
> > >
Re: [OE-core] [PATCH] systemd: quote SYSTEMD_SERVICE in prerm and postinst
Hi,
On Fri, Jun 22, 2018 at 09:29:37AM +0800, ChenQi wrote:
> Hi Damien,
>
> If some recipe has more than one unit file, this patch will cause problem
> for it.
That's a good point, let me see if I can come up with a solution that
addresses that.
Regards,
--
Damien
>
> Best Regards,
> Chen Qi
>
>
> On 06/22/2018 04:35 AM, Damien Riegel wrote:
> > Systemd mount configuration file must have a name that match the mount
> > point directory they control. So for instance, if a mount file contains
> >
> > [Mount]
> > ...
> > Where=/mnt/my-data
> >
> > The file must be named `mnt-my\x2ddata.mount`, or systemd will refuse to
> > honour it.
> >
> > If this config file contains an [Install] section, it will silently fail
> > because the unit file is not quoted when systemctl is called. To fix
> > that, quote `${SYSTEMD_SERVICE}` in prerm and postinst hooks.
> >
> > Signed-off-by: Damien Riegel
> > ---
> > meta/classes/systemd.bbclass | 8
> > 1 file changed, 4 insertions(+), 4 deletions(-)
> >
> > diff --git a/meta/classes/systemd.bbclass b/meta/classes/systemd.bbclass
> > index 1b134322fb..589e73c855 100644
> > --- a/meta/classes/systemd.bbclass
> > +++ b/meta/classes/systemd.bbclass
> > @@ -34,10 +34,10 @@ if type systemctl >/dev/null 2>/dev/null; then
> > systemctl daemon-reload
> > fi
> > - systemctl $OPTS ${SYSTEMD_AUTO_ENABLE} ${SYSTEMD_SERVICE}
> > + systemctl $OPTS ${SYSTEMD_AUTO_ENABLE} "${SYSTEMD_SERVICE}"
> > if [ -z "$D" -a "${SYSTEMD_AUTO_ENABLE}" = "enable" ]; then
> > - systemctl --no-block restart ${SYSTEMD_SERVICE}
> > + systemctl --no-block restart "${SYSTEMD_SERVICE}"
> > fi
> > fi
> > }
> > @@ -51,10 +51,10 @@ fi
> > if type systemctl >/dev/null 2>/dev/null; then
> > if [ -z "$D" ]; then
> > - systemctl stop ${SYSTEMD_SERVICE}
> > + systemctl stop "${SYSTEMD_SERVICE}"
> > fi
> > - systemctl $OPTS disable ${SYSTEMD_SERVICE}
> > + systemctl $OPTS disable "${SYSTEMD_SERVICE}"
> > fi
> > }
>
>
> --
> ___
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH] systemd: quote SYSTEMD_SERVICE in prerm and postinst
Systemd mount configuration file must have a name that match the mount
point directory they control. So for instance, if a mount file contains
[Mount]
...
Where=/mnt/my-data
The file must be named `mnt-my\x2ddata.mount`, or systemd will refuse to
honour it.
If this config file contains an [Install] section, it will silently fail
because the unit file is not quoted when systemctl is called. To fix
that, quote `${SYSTEMD_SERVICE}` in prerm and postinst hooks.
Signed-off-by: Damien Riegel
---
meta/classes/systemd.bbclass | 8
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/meta/classes/systemd.bbclass b/meta/classes/systemd.bbclass
index 1b134322fb..589e73c855 100644
--- a/meta/classes/systemd.bbclass
+++ b/meta/classes/systemd.bbclass
@@ -34,10 +34,10 @@ if type systemctl >/dev/null 2>/dev/null; then
systemctl daemon-reload
fi
- systemctl $OPTS ${SYSTEMD_AUTO_ENABLE} ${SYSTEMD_SERVICE}
+ systemctl $OPTS ${SYSTEMD_AUTO_ENABLE} "${SYSTEMD_SERVICE}"
if [ -z "$D" -a "${SYSTEMD_AUTO_ENABLE}" = "enable" ]; then
- systemctl --no-block restart ${SYSTEMD_SERVICE}
+ systemctl --no-block restart "${SYSTEMD_SERVICE}"
fi
fi
}
@@ -51,10 +51,10 @@ fi
if type systemctl >/dev/null 2>/dev/null; then
if [ -z "$D" ]; then
- systemctl stop ${SYSTEMD_SERVICE}
+ systemctl stop "${SYSTEMD_SERVICE}"
fi
- systemctl $OPTS disable ${SYSTEMD_SERVICE}
+ systemctl $OPTS disable "${SYSTEMD_SERVICE}"
fi
}
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v3] populate_sdk_ext.bbclass: fix corebase identification
When generating the extended SDK, there is a copy step where this class
goes through the layers and other stuff that have been copied to
generate the SDK. The corebase; ie. the folder that contains the core
layer 'meta' is treated in a special way. Unfortunately in our tree, we
have:
sources/meta/meta
| `- core layer
`--- corebase
In populate_sdk_ext's copy_buildsystem, the heuristic to determine which
element of the list returned by copy_bitbake_and_layers is corebase is
fooled by such layout.
In copy_bitbake_and_layers, corebase is already handled specifically and
reliably, so we should let that function tell us which folder is
corebase instead of trying to determine it.
To do so, change the return type of copy_bitbake_and_layers to a tuple
that contains (corebase, copied_layers). It also simplifies the code on
the caller side.
Signed-off-by: Damien Riegel
---
Changes in v3:
- rebased on master to apply cleanly
Changes in v2:
- move the logic in copy_bitbake_and_layers
meta/classes/populate_sdk_ext.bbclass | 12 +++-
meta/lib/oe/copy_buildsystem.py | 8 +---
2 files changed, 8 insertions(+), 12 deletions(-)
diff --git a/meta/classes/populate_sdk_ext.bbclass
b/meta/classes/populate_sdk_ext.bbclass
index e1bba49eaf..f0c8709c5b 100644
--- a/meta/classes/populate_sdk_ext.bbclass
+++ b/meta/classes/populate_sdk_ext.bbclass
@@ -200,15 +200,9 @@ python copy_buildsystem () {
workspace_name = 'orig-workspace'
else:
workspace_name = None
-layers_copied = buildsystem.copy_bitbake_and_layers(baseoutpath +
'/layers', workspace_name)
-
-sdkbblayers = []
-corebase = os.path.basename(d.getVar('COREBASE'))
-for layer in layers_copied:
-if corebase == os.path.basename(layer):
-conf_bbpath = os.path.join('layers', layer, 'bitbake')
-else:
-sdkbblayers.append(layer)
+
+corebase, sdkbblayers = buildsystem.copy_bitbake_and_layers(baseoutpath +
'/layers', workspace_name)
+conf_bbpath = os.path.join('layers', corebase, 'bitbake')
for path in os.listdir(baseoutpath + '/layers'):
relpath = os.path.join('layers', path, oe_init_env_script)
diff --git a/meta/lib/oe/copy_buildsystem.py b/meta/lib/oe/copy_buildsystem.py
index 4b94806c73..4abec4666f 100644
--- a/meta/lib/oe/copy_buildsystem.py
+++ b/meta/lib/oe/copy_buildsystem.py
@@ -26,6 +26,7 @@ class BuildSystem(object):
def copy_bitbake_and_layers(self, destdir, workspace_name=None):
# Copy in all metadata layers + bitbake (as repositories)
+copied_corebase = None
layers_copied = []
bb.utils.mkdirhier(destdir)
layers = list(self.layerdirs)
@@ -84,17 +85,18 @@ class BuildSystem(object):
layer_relative = os.path.relpath(layerdestpath,
destdir)
-layers_copied.append(layer_relative)
-
# Treat corebase as special since it typically will contain
# build directories or other custom items.
if corebase == layer:
+copied_corebase = layer_relative
bb.utils.mkdirhier(layerdestpath)
for f in corebase_files:
f_basename = os.path.basename(f)
destname = os.path.join(layerdestpath, f_basename)
_smart_copy(f, destname)
else:
+layers_copied.append(layer_relative)
+
if os.path.exists(os.path.join(layerdestpath,
'conf/layer.conf')):
bb.note("Skipping layer %s, already handled" % layer)
else:
@@ -140,7 +142,7 @@ class BuildSystem(object):
layers_copied.remove(layer)
break
-return layers_copied
+return copied_corebase, layers_copied
def generate_locked_sigs(sigfile, d):
bb.utils.mkdirhier(os.path.dirname(sigfile))
--
2.17.1
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH v2] populate_sdk_ext.bbclass: fix corebase identification
Richard, what's your opinion on this v2? It fixes the issue I have while
addressing the concern you had about the v1.
--
Damien
On Thu, Jun 07, 2018 at 10:48:14AM -0400, Damien Riegel wrote:
> When generating the extended SDK, there is a copy step where this class
> goes through the layers and other stuff that have been copied to
> generate the SDK. The corebase; ie. the folder that contains the core
> layer 'meta' is treated in a special way. Unfortunately in our tree, we
> have:
>
> sources/meta/meta
>| `- core layer
>`--- corebase
>
> In populate_sdk_ext's copy_buildsystem, the heuristic to determine which
> element of the list returned by copy_bitbake_and_layers is corebase is
> fooled by such layout.
>
> In copy_bitbake_and_layers, corebase is already handled specifically and
> reliably, so we should let that function tell us which folder is
> corebase instead of trying to determine it.
>
> To do so, change the return type of copy_bitbake_and_layers to a tuple
> that contains (corebase, copied_layers). It also simplifies the code on
> the caller side.
>
> Signed-off-by: Damien Riegel
> ---
> Changes in v2:
> - move the logic in copy_bitbake_and_layers
>
> meta/classes/populate_sdk_ext.bbclass | 12 +++-
> meta/lib/oe/copy_buildsystem.py | 8 +---
> 2 files changed, 8 insertions(+), 12 deletions(-)
>
> diff --git a/meta/classes/populate_sdk_ext.bbclass
> b/meta/classes/populate_sdk_ext.bbclass
> index 2dd21237e2..cc3f5d0e50 100644
> --- a/meta/classes/populate_sdk_ext.bbclass
> +++ b/meta/classes/populate_sdk_ext.bbclass
> @@ -202,15 +202,9 @@ python copy_buildsystem () {
> workspace_name = 'orig-workspace'
> else:
> workspace_name = None
> -layers_copied = buildsystem.copy_bitbake_and_layers(baseoutpath +
> '/layers', workspace_name)
> -
> -sdkbblayers = []
> -corebase = os.path.basename(d.getVar('COREBASE'))
> -for layer in layers_copied:
> -if corebase == os.path.basename(layer):
> -conf_bbpath = os.path.join('layers', layer, 'bitbake')
> -else:
> -sdkbblayers.append(layer)
> +
> +corebase, sdkbblayers = buildsystem.copy_bitbake_and_layers(baseoutpath
> + '/layers', workspace_name)
> +conf_bbpath = os.path.join('layers', corebase, 'bitbake')
>
> for path in os.listdir(baseoutpath + '/layers'):
> relpath = os.path.join('layers', path, oe_init_env_script)
> diff --git a/meta/lib/oe/copy_buildsystem.py b/meta/lib/oe/copy_buildsystem.py
> index ac2fae1ed1..05f65ac632 100644
> --- a/meta/lib/oe/copy_buildsystem.py
> +++ b/meta/lib/oe/copy_buildsystem.py
> @@ -26,6 +26,7 @@ class BuildSystem(object):
>
> def copy_bitbake_and_layers(self, destdir, workspace_name=None):
> # Copy in all metadata layers + bitbake (as repositories)
> +copied_corebase = None
> layers_copied = []
> bb.utils.mkdirhier(destdir)
> layers = list(self.layerdirs)
> @@ -84,17 +85,18 @@ class BuildSystem(object):
>
> layer_relative = os.path.relpath(layerdestpath,
> destdir)
> -layers_copied.append(layer_relative)
> -
> # Treat corebase as special since it typically will contain
> # build directories or other custom items.
> if corebase == layer:
> +copied_corebase = layer_relative
> bb.utils.mkdirhier(layerdestpath)
> for f in corebase_files:
> f_basename = os.path.basename(f)
> destname = os.path.join(layerdestpath, f_basename)
> _smart_copy(f, destname)
> else:
> +layers_copied.append(layer_relative)
> +
> if os.path.exists(layerdestpath):
> bb.note("Skipping layer %s, already handled" % layer)
> else:
> @@ -140,7 +142,7 @@ class BuildSystem(object):
> layers_copied.remove(layer)
> break
>
> -return layers_copied
> +return copied_corebase, layers_copied
>
> def generate_locked_sigs(sigfile, d):
> bb.utils.mkdirhier(os.path.dirname(sigfile))
> --
> 2.17.0
>
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH v2] populate_sdk_ext.bbclass: fix corebase identification
When generating the extended SDK, there is a copy step where this class
goes through the layers and other stuff that have been copied to
generate the SDK. The corebase; ie. the folder that contains the core
layer 'meta' is treated in a special way. Unfortunately in our tree, we
have:
sources/meta/meta
| `- core layer
`--- corebase
In populate_sdk_ext's copy_buildsystem, the heuristic to determine which
element of the list returned by copy_bitbake_and_layers is corebase is
fooled by such layout.
In copy_bitbake_and_layers, corebase is already handled specifically and
reliably, so we should let that function tell us which folder is
corebase instead of trying to determine it.
To do so, change the return type of copy_bitbake_and_layers to a tuple
that contains (corebase, copied_layers). It also simplifies the code on
the caller side.
Signed-off-by: Damien Riegel
---
Changes in v2:
- move the logic in copy_bitbake_and_layers
meta/classes/populate_sdk_ext.bbclass | 12 +++-
meta/lib/oe/copy_buildsystem.py | 8 +---
2 files changed, 8 insertions(+), 12 deletions(-)
diff --git a/meta/classes/populate_sdk_ext.bbclass
b/meta/classes/populate_sdk_ext.bbclass
index 2dd21237e2..cc3f5d0e50 100644
--- a/meta/classes/populate_sdk_ext.bbclass
+++ b/meta/classes/populate_sdk_ext.bbclass
@@ -202,15 +202,9 @@ python copy_buildsystem () {
workspace_name = 'orig-workspace'
else:
workspace_name = None
-layers_copied = buildsystem.copy_bitbake_and_layers(baseoutpath +
'/layers', workspace_name)
-
-sdkbblayers = []
-corebase = os.path.basename(d.getVar('COREBASE'))
-for layer in layers_copied:
-if corebase == os.path.basename(layer):
-conf_bbpath = os.path.join('layers', layer, 'bitbake')
-else:
-sdkbblayers.append(layer)
+
+corebase, sdkbblayers = buildsystem.copy_bitbake_and_layers(baseoutpath +
'/layers', workspace_name)
+conf_bbpath = os.path.join('layers', corebase, 'bitbake')
for path in os.listdir(baseoutpath + '/layers'):
relpath = os.path.join('layers', path, oe_init_env_script)
diff --git a/meta/lib/oe/copy_buildsystem.py b/meta/lib/oe/copy_buildsystem.py
index ac2fae1ed1..05f65ac632 100644
--- a/meta/lib/oe/copy_buildsystem.py
+++ b/meta/lib/oe/copy_buildsystem.py
@@ -26,6 +26,7 @@ class BuildSystem(object):
def copy_bitbake_and_layers(self, destdir, workspace_name=None):
# Copy in all metadata layers + bitbake (as repositories)
+copied_corebase = None
layers_copied = []
bb.utils.mkdirhier(destdir)
layers = list(self.layerdirs)
@@ -84,17 +85,18 @@ class BuildSystem(object):
layer_relative = os.path.relpath(layerdestpath,
destdir)
-layers_copied.append(layer_relative)
-
# Treat corebase as special since it typically will contain
# build directories or other custom items.
if corebase == layer:
+copied_corebase = layer_relative
bb.utils.mkdirhier(layerdestpath)
for f in corebase_files:
f_basename = os.path.basename(f)
destname = os.path.join(layerdestpath, f_basename)
_smart_copy(f, destname)
else:
+layers_copied.append(layer_relative)
+
if os.path.exists(layerdestpath):
bb.note("Skipping layer %s, already handled" % layer)
else:
@@ -140,7 +142,7 @@ class BuildSystem(object):
layers_copied.remove(layer)
break
-return layers_copied
+return copied_corebase, layers_copied
def generate_locked_sigs(sigfile, d):
bb.utils.mkdirhier(os.path.dirname(sigfile))
--
2.17.0
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [PATCH] populate_sdk_ext.bbclass: fix corebase identification
On Mon, Jun 04, 2018 at 04:54:08PM +0100, Richard Purdie wrote:
> On Mon, 2018-06-04 at 11:38 -0400, Damien Riegel wrote:
> > When generating the extended SDK, there is a copy step where this
> > class
> > goes through the layers and other stuff that have been copied to
> > generate the SDK.
> >
> > The corebase; ie. the folder that contains the core layer 'meta' is
> > treated in a special way. Unfortunately in our tree, we have:
> >
> > layers/meta/meta
> >| `- core layer
> >`--- corebase
> >
> > When populate_sdk_ext checks this layer, it thinks wrongly that it's
> > the
> > corebase folder, so it treats it differently and doesn't add it to
> > the
> > layer list. When bitbake tries to run in the generated SDK, it will
> > fail
> > because it doesn't find information stored in that layer.
> >
> > This patch makes sure only the corebase folder is identified as so.
> >
> > Signed-off-by: Damien Riegel
> > ---
> > meta/classes/populate_sdk_ext.bbclass | 2 +-
> > 1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/meta/classes/populate_sdk_ext.bbclass
> > b/meta/classes/populate_sdk_ext.bbclass
> > index 2dd21237e2..917fbfa1b4 100644
> > --- a/meta/classes/populate_sdk_ext.bbclass
> > +++ b/meta/classes/populate_sdk_ext.bbclass
> > @@ -207,7 +207,7 @@ python copy_buildsystem () {
> > sdkbblayers = []
> > corebase = os.path.basename(d.getVar('COREBASE'))
> > for layer in layers_copied:
> > -if corebase == os.path.basename(layer):
> > +if corebase == layer:
> > conf_bbpath = os.path.join('layers', layer, 'bitbake')
> >
>
> Whilst I appreciate the problem, the fix doesn't look like it would
> work for other use cases since its comparing a basename on one side and
> a non-basename on the other...
Well, the issue is precisely caused by the comparison of basenames.
>>> os.path.basename("meta/meta")
'meta'
>>> os.path.basename("meta")
'meta'
So it doesn't differentiate between a layer within corebase and corebase
itself. copy_bitbake_and_layers seems to do the comparison differently
[1]. Arguably there, it's comparison between absolute paths, but still.
Do you have suggestions on how I could fix this bug otherwise? Maybe
copy_bitbake_and_layers could return a tuple (corebase, layers) instead
of a list that contains copied_layers + corebase.
[1]
https://github.com/openembedded/openembedded-core/blob/master/meta/lib/oe/copy_buildsystem.py#L91
Thank for the feedback,
--
Damien
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [PATCH] populate_sdk_ext.bbclass: fix corebase identification
When generating the extended SDK, there is a copy step where this class
goes through the layers and other stuff that have been copied to
generate the SDK.
The corebase; ie. the folder that contains the core layer 'meta' is
treated in a special way. Unfortunately in our tree, we have:
layers/meta/meta
| `- core layer
`--- corebase
When populate_sdk_ext checks this layer, it thinks wrongly that it's the
corebase folder, so it treats it differently and doesn't add it to the
layer list. When bitbake tries to run in the generated SDK, it will fail
because it doesn't find information stored in that layer.
This patch makes sure only the corebase folder is identified as so.
Signed-off-by: Damien Riegel
---
meta/classes/populate_sdk_ext.bbclass | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/meta/classes/populate_sdk_ext.bbclass
b/meta/classes/populate_sdk_ext.bbclass
index 2dd21237e2..917fbfa1b4 100644
--- a/meta/classes/populate_sdk_ext.bbclass
+++ b/meta/classes/populate_sdk_ext.bbclass
@@ -207,7 +207,7 @@ python copy_buildsystem () {
sdkbblayers = []
corebase = os.path.basename(d.getVar('COREBASE'))
for layer in layers_copied:
-if corebase == os.path.basename(layer):
+if corebase == layer:
conf_bbpath = os.path.join('layers', layer, 'bitbake')
else:
sdkbblayers.append(layer)
--
2.17.0
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] [rocko][PATCH] npm.bbclass: Fix building node modules with npm@5
From: Böszörményi Zoltán
npm cache clear throws an error with npm@5 and suggests to
use npm cache verify instead. But our cache is actually empty,
so use npm cache clear --force.
npm install in the source directory creates symlinks with npm@5.
Use a combination of npm pack and npm install module-version.tgz
that works the same way with older and new npm versions and is
guaranteed to create actual copies instead of directory symlinks.
This change allows using nodejs 8.x LTS, tested with 8.9.4.
Signed-off-by: Zoltán Böszörményi
Signed-off-by: Ross Burton
---
I'm sending a request to include this patch in the rocko branch, as the
version of NodeJS included in rocko version of meta-oe is 8.4.0, so it
can definitely benefit from this patch.
meta/classes/npm.bbclass | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/meta/classes/npm.bbclass b/meta/classes/npm.bbclass
index a69bedbb28..898a54eb50 100644
--- a/meta/classes/npm.bbclass
+++ b/meta/classes/npm.bbclass
@@ -31,7 +31,7 @@ npm_do_compile() {
fi
npm set cache ${WORKDIR}/npm_cache
# clear cache before every build
- npm cache clear
+ npm cache clear --force
# Install pkg into ${S} without going to the registry
if [ "${NPM_INSTALL_DEV}" = "1" ]; then
npm --arch=${NPM_ARCH} --target_arch=${NPM_ARCH} --no-registry
install
@@ -45,7 +45,8 @@ npm_do_install() {
# be created in this directory
export HOME=${WORKDIR}
mkdir -p ${NPM_INSTALLDIR}/
- npm install --prefix ${D}${prefix} -g --arch=${NPM_ARCH}
--target_arch=${NPM_ARCH} --production --no-registry
+ npm pack .
+ npm install --prefix ${D}${prefix} -g --arch=${NPM_ARCH}
--target_arch=${NPM_ARCH} --production --no-registry ${PN}-${PV}.tgz
if [ -d ${D}${prefix}/etc ] ; then
# This will be empty
rmdir ${D}${prefix}/etc
--
2.17.0
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [OE-core] [bitbake-devel] Generating extensible SDK fails because of missing LAYERSERIES_CORENAMES
On Sat, May 26, 2018 at 08:22:39AM -0400, Damien Riegel wrote:
> Hi,
>
>
> When generating the extended SDK, my setup fails with the following
> error:
>
> ERROR: Failed to generate filtered task list for extensible SDK:
> ERROR: bitbake failed:
> ERROR: Unable to start bitbake server
>
> I digged into the issue and found out why bitbake doesn't
> start.
>
> bitbake thinks layers are incompatible because they define
> LAYERSERIES_COMPAT but LAYERSERIES_CORENAMES is not set. So this check
> in cookerdata.py fails [1].
>
> The reason this check fails is that the core `meta` layer is not listed
> in the generated BBLAYERS, because it's treated in a special way in
> populate_sdk_ext.bbclass [2]. So as this layer is not listed in
> BBLAYERS, the compatibility check fails and bitbake cannot start.
>
>
> Shouldn't `meta` simply be added to BBLAYERS as any other layer? If not,
> what is the proper way to solve my bug?
I think it got confused because the `meta` layer is contained in a
folder named `meta` as well.
I managed to go further in my build with the following snippet:
--- a/meta/classes/populate_sdk_ext.bbclass
+++ b/meta/classes/populate_sdk_ext.bbclass
@@ -207,7 +207,7 @@ python copy_buildsystem () {
sdkbblayers = []
corebase = os.path.basename(d.getVar('COREBASE'))
for layer in layers_copied:
-if corebase == os.path.basename(layer):
+if corebase == layer:
conf_bbpath = os.path.join('layers', layer, 'bitbake')
else:
sdkbblayers.append(layer)
>
> [1]
> https://github.com/openembedded/bitbake/blob/master/lib/bb/cookerdata.py#L397
> [2]
> https://github.com/openembedded/openembedded-core/blob/master/meta/classes/populate_sdk_ext.bbclass#L208
>
> Thank you,
> --
> Damien
> --
> ___________
> bitbake-devel mailing list
> bitbake-de...@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/bitbake-devel
--
Damien Riegel
Free Software Consultant
Tel: +1 514 276 5468 (ext: 355)
--
___
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
[OE-core] Generating extensible SDK fails because of missing LAYERSERIES_CORENAMES
Hi, When generating the extended SDK, my setup fails with the following error: ERROR: Failed to generate filtered task list for extensible SDK: ERROR: bitbake failed: ERROR: Unable to start bitbake server I digged into the issue and found out why bitbake doesn't start. bitbake thinks layers are incompatible because they define LAYERSERIES_COMPAT but LAYERSERIES_CORENAMES is not set. So this check in cookerdata.py fails [1]. The reason this check fails is that the core `meta` layer is not listed in the generated BBLAYERS, because it's treated in a special way in populate_sdk_ext.bbclass [2]. So as this layer is not listed in BBLAYERS, the compatibility check fails and bitbake cannot start. Shouldn't `meta` simply be added to BBLAYERS as any other layer? If not, what is the proper way to solve my bug? [1] https://github.com/openembedded/bitbake/blob/master/lib/bb/cookerdata.py#L397 [2] https://github.com/openembedded/openembedded-core/blob/master/meta/classes/populate_sdk_ext.bbclass#L208 Thank you, -- Damien -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
