Re: [yocto] sysroot not being populated
On 12.06.2018 00:48, Andre McCurdy wrote: > On Fri, Jun 8, 2018 at 2:42 AM, Patrick Vacek wrote: >> On 08.06.2018 10:26, Khem Raj wrote: >>> On 6/8/18 12:27 AM, Patrick Vacek wrote: On 07.06.2018 19:06, Khem Raj wrote: > On Mon, Jun 4, 2018 at 2:01 AM, Patrick Vacek > wrote: >> I have a recipe (aktualizr-hsm-prov) that depends on another >> (aktualizr) >> to provide an executable and a config file. The former recipe includes >> `DEPENDS = "aktualizr-native"`, and my do_install() for >> aktualizr-hsm-prov has a line something like this: >> >> aktualizr -i ${STAGING_DIR_NATIVE}${libdir}/sota.conf >> >> The binary executable (aktualizr) runs, which tells me that the recipe >> can find that. (Although to be honest, I'm not sure which version >> it is >> running.) However, it doesn't find the config file, and sure enough, >> ${STAGING_DIR_NATIVE}${libdir} does not have the config file I >> expect. I >> can see that aktualizr-native is populating its sysroot-destdir, >> but it >> isn't getting copied to the sysroot for aktualizr-hsm-prov. >> >> I see this problem in sumo and master, although previously this logic >> has worked just fine in morty/pyro/rocko. >> >> What am I doing wrong? What changed between rocko and sumo? >>> Can you check location of sota.conf in the build tree for >>> aktualizr-native in directory called package/ >> Oddly, I do not see a package subdirectory inside the aktualizr-native >> directory in the build tree. I do see it inside the aktualizr directory, >> though, and it contains everything that I would expect. Is there some >> sort of configuration of the packaging system for a native recipe that I >> haven't done correctly? > No, what you see is expected - there's no packaging for -native recipes. > > Back to the original problem, I think you should consider the > aktualizr executable and the sota.conf file as two separate things. > The host executable should always be provided by aktualizr-native. > It's less clear what should provide sota.conf - it depends whether > it's just required for building other recipes (ie like a header file) > or whether it needs to be present in the final rootfs? If it's only > required for building other packages, then it should be in the -dev > package for aktualizr. If it needs to be present in the final rootfs > then it should be in the default package for aktualizr. Either way, > recipes which need to find sota.conf in sysroot would then depend on > "aktualizr". Therefore recipes which need both the host executable and > sota.conf in sysroot should depend on both aktualizr and > aktualizr-native. Actually, after some further research, it appears that my problem is not entirely resolved. The explanation of packaging and native vs. target dependencies all makes sense to me and usually works. However, this is not the case for one specific condition which happens to be part of my test suite. One of my tests is to change the MACHINE in my local.conf, add some additional layers, change some other variables, and then bitbake. Then I undo that and bitbake again. For the most part, that works fine. In rocko and before, it worked consistently. But it seems like on sumo and master, when I do the test by changing the MACHINE and the layers, it somehow clears out everything in my recipe's work directory, despite that that recipe is not used by that test. In fact, it removes every single file in tmp/work/core2-64-poky-linux (but leaves most of the directories). When I switch back to how it was before and bitbake, most of the files get recreated or re-copied, but not all. I have to explicitly run `bitbake -c cleanall aktualizr aktualizr-hsm-prov` to get things to work again. (Perhaps cleanall isn't necessary, but some form of cleaning is, and it has to be both recipes.) Can someone help explain why I am seeing this behavior? I don't understand why the files get erased, why they don't get repopulated, and why this was different back in rocko. Thanks! Patrick -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] sysroot not being populated
On 12.06.2018 00:48, Andre McCurdy wrote: > On Fri, Jun 8, 2018 at 2:42 AM, Patrick Vacek wrote: >> On 08.06.2018 10:26, Khem Raj wrote: >>> On 6/8/18 12:27 AM, Patrick Vacek wrote: On 07.06.2018 19:06, Khem Raj wrote: > Hi Patrick > > On Mon, Jun 4, 2018 at 2:01 AM, Patrick Vacek > wrote: >> Hello all, >> >> I have a recipe (aktualizr-hsm-prov) that depends on another >> (aktualizr) >> to provide an executable and a config file. The former recipe includes >> `DEPENDS = "aktualizr-native"`, and my do_install() for >> aktualizr-hsm-prov has a line something like this: >> >> aktualizr -i ${STAGING_DIR_NATIVE}${libdir}/sota.conf >> >> The binary executable (aktualizr) runs, which tells me that the recipe >> can find that. (Although to be honest, I'm not sure which version >> it is >> running.) However, it doesn't find the config file, and sure enough, >> ${STAGING_DIR_NATIVE}${libdir} does not have the config file I >> expect. I >> can see that aktualizr-native is populating its sysroot-destdir, >> but it >> isn't getting copied to the sysroot for aktualizr-hsm-prov. >> >> I see this problem in sumo and master, although previously this logic >> has worked just fine in morty/pyro/rocko. >> >> Switching to depending on aktualizr (instead of aktualizr-native) and >> using STAGING_DIR_HOST or STAGING_DIR_TARGET does not help, and in >> fact >> makes it worse, as it can't even find the aktualizr executable in >> that case. >> >> What am I doing wrong? What changed between rocko and sumo? > is aktualizr recipe installing this .conf file when building native > version. If not may be > add it to do_install > > install -D -m 0644 ${D}${libdir}/sota.conf Hello Khem, I believe so. The aktualizr recipe includes the following (edited for simplicity): BBCLASSEXTEND =+ "native" do_install_append () { install -m 0644 ${S}/config/sota.conf ${D}/${libdir}/sota.conf } PACKAGES =+ " ${PN}-host-tools " FILES_${PN}-host-tools = " ${libdir}/sota.conf " >>> Can you check location of sota.conf in the build tree for >>> aktualizr-native in directory called package/ >> Oddly, I do not see a package subdirectory inside the aktualizr-native >> directory in the build tree. I do see it inside the aktualizr directory, >> though, and it contains everything that I would expect. Is there some >> sort of configuration of the packaging system for a native recipe that I >> haven't done correctly? > No, what you see is expected - there's no packaging for -native recipes. > > Back to the original problem, I think you should consider the > aktualizr executable and the sota.conf file as two separate things. > The host executable should always be provided by aktualizr-native. > It's less clear what should provide sota.conf - it depends whether > it's just required for building other recipes (ie like a header file) > or whether it needs to be present in the final rootfs? If it's only > required for building other packages, then it should be in the -dev > package for aktualizr. If it needs to be present in the final rootfs > then it should be in the default package for aktualizr. Either way, > recipes which need to find sota.conf in sysroot would then depend on > "aktualizr". Therefore recipes which need both the host executable and > sota.conf in sysroot should depend on both aktualizr and > aktualizr-native. Hello Andre, Thanks for your insight. I didn't realize that there was no packaging for native recipes (although that makes sense now that you've explained it). In my use case, the second recipe just needs the config file from the first for building a package, and it looks like your advice has solved my problem perfectly. Thank you! Patrick -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] sysroot not being populated
On Thu, Jun 14, 2018 at 5:45 AM Patrick Vacek wrote: > > On 12.06.2018 00:48, Andre McCurdy wrote: > > On Fri, Jun 8, 2018 at 2:42 AM, Patrick Vacek > > wrote: > >> On 08.06.2018 10:26, Khem Raj wrote: > >>> On 6/8/18 12:27 AM, Patrick Vacek wrote: > On 07.06.2018 19:06, Khem Raj wrote: > > On Mon, Jun 4, 2018 at 2:01 AM, Patrick Vacek > > wrote: > >> I have a recipe (aktualizr-hsm-prov) that depends on another > >> (aktualizr) > >> to provide an executable and a config file. The former recipe includes > >> `DEPENDS = "aktualizr-native"`, and my do_install() for > >> aktualizr-hsm-prov has a line something like this: > >> > >> aktualizr -i ${STAGING_DIR_NATIVE}${libdir}/sota.conf > >> > >> The binary executable (aktualizr) runs, which tells me that the recipe > >> can find that. (Although to be honest, I'm not sure which version > >> it is > >> running.) However, it doesn't find the config file, and sure enough, > >> ${STAGING_DIR_NATIVE}${libdir} does not have the config file I > >> expect. I > >> can see that aktualizr-native is populating its sysroot-destdir, > >> but it > >> isn't getting copied to the sysroot for aktualizr-hsm-prov. > >> > >> I see this problem in sumo and master, although previously this logic > >> has worked just fine in morty/pyro/rocko. > >> > >> What am I doing wrong? What changed between rocko and sumo? > >>> Can you check location of sota.conf in the build tree for > >>> aktualizr-native in directory called package/ > >> Oddly, I do not see a package subdirectory inside the aktualizr-native > >> directory in the build tree. I do see it inside the aktualizr directory, > >> though, and it contains everything that I would expect. Is there some > >> sort of configuration of the packaging system for a native recipe that I > >> haven't done correctly? > > No, what you see is expected - there's no packaging for -native recipes. > > > > Back to the original problem, I think you should consider the > > aktualizr executable and the sota.conf file as two separate things. > > The host executable should always be provided by aktualizr-native. > > It's less clear what should provide sota.conf - it depends whether > > it's just required for building other recipes (ie like a header file) > > or whether it needs to be present in the final rootfs? If it's only > > required for building other packages, then it should be in the -dev > > package for aktualizr. If it needs to be present in the final rootfs > > then it should be in the default package for aktualizr. Either way, > > recipes which need to find sota.conf in sysroot would then depend on > > "aktualizr". Therefore recipes which need both the host executable and > > sota.conf in sysroot should depend on both aktualizr and > > aktualizr-native. > Actually, after some further research, it appears that my problem is not > entirely resolved. The explanation of packaging and native vs. target > dependencies all makes sense to me and usually works. However, this is > not the case for one specific condition which happens to be part of my > test suite. > > One of my tests is to change the MACHINE in my local.conf, add some > additional layers, change some other variables, and then bitbake. Then I > undo that and bitbake again. For the most part, that works fine. In > rocko and before, it worked consistently. But it seems like on sumo and > master, when I do the test by changing the MACHINE and the layers, it > somehow clears out everything in my recipe's work directory, despite > that that recipe is not used by that test. In fact, it removes every > single file in tmp/work/core2-64-poky-linux (but leaves most of the > directories). When I switch back to how it was before and bitbake, most > of the files get recreated or re-copied, but not all. I have to > explicitly run `bitbake -c cleanall aktualizr aktualizr-hsm-prov` to get > things to work again. (Perhaps cleanall isn't necessary, but some form > of cleaning is, and it has to be both recipes.) > > Can someone help explain why I am seeing this behavior? I don't > understand why the files get erased, why they don't get repopulated, and > why this was different back in rocko. > when machine and other variables are changed then its mostly resulting in hash changes which is enforcing it to rebuild and when rebuild happens it will clear the recipe specific build area for that given recipe and repopulate it as it has to rebuild/repackage it since this time it will get most of build artifacts from shared state as it has been bullt with same hash once before. so it will most probably be re-using that. If doing clean build is helping then this might mean that they are not staging all the files that you depend on. When you do clean build they get staged into build area during compile phase and it silently uses it. So ensure that all files you need are accounted for in do_install > Thanks! >
Re: [yocto] sysroot not being populated
On Fri, Jun 8, 2018 at 2:42 AM, Patrick Vacek wrote: > On 08.06.2018 10:26, Khem Raj wrote: >> On 6/8/18 12:27 AM, Patrick Vacek wrote: >>> On 07.06.2018 19:06, Khem Raj wrote: Hi Patrick On Mon, Jun 4, 2018 at 2:01 AM, Patrick Vacek wrote: > Hello all, > > I have a recipe (aktualizr-hsm-prov) that depends on another > (aktualizr) > to provide an executable and a config file. The former recipe includes > `DEPENDS = "aktualizr-native"`, and my do_install() for > aktualizr-hsm-prov has a line something like this: > > aktualizr -i ${STAGING_DIR_NATIVE}${libdir}/sota.conf > > The binary executable (aktualizr) runs, which tells me that the recipe > can find that. (Although to be honest, I'm not sure which version > it is > running.) However, it doesn't find the config file, and sure enough, > ${STAGING_DIR_NATIVE}${libdir} does not have the config file I > expect. I > can see that aktualizr-native is populating its sysroot-destdir, > but it > isn't getting copied to the sysroot for aktualizr-hsm-prov. > > I see this problem in sumo and master, although previously this logic > has worked just fine in morty/pyro/rocko. > > Switching to depending on aktualizr (instead of aktualizr-native) and > using STAGING_DIR_HOST or STAGING_DIR_TARGET does not help, and in > fact > makes it worse, as it can't even find the aktualizr executable in > that case. > > What am I doing wrong? What changed between rocko and sumo? is aktualizr recipe installing this .conf file when building native version. If not may be add it to do_install install -D -m 0644 ${D}${libdir}/sota.conf >>> >>> Hello Khem, >>> >>> I believe so. The aktualizr recipe includes the following (edited for >>> simplicity): >>> >>> BBCLASSEXTEND =+ "native" >>> >>> do_install_append () { >>> install -m 0644 ${S}/config/sota.conf ${D}/${libdir}/sota.conf >>> } >>> >>> PACKAGES =+ " ${PN}-host-tools " >>> >>> FILES_${PN}-host-tools = " ${libdir}/sota.conf " >>> >> >> Can you check location of sota.conf in the build tree for >> aktualizr-native in directory called package/ > Oddly, I do not see a package subdirectory inside the aktualizr-native > directory in the build tree. I do see it inside the aktualizr directory, > though, and it contains everything that I would expect. Is there some > sort of configuration of the packaging system for a native recipe that I > haven't done correctly? No, what you see is expected - there's no packaging for -native recipes. Back to the original problem, I think you should consider the aktualizr executable and the sota.conf file as two separate things. The host executable should always be provided by aktualizr-native. It's less clear what should provide sota.conf - it depends whether it's just required for building other recipes (ie like a header file) or whether it needs to be present in the final rootfs? If it's only required for building other packages, then it should be in the -dev package for aktualizr. If it needs to be present in the final rootfs then it should be in the default package for aktualizr. Either way, recipes which need to find sota.conf in sysroot would then depend on "aktualizr". Therefore recipes which need both the host executable and sota.conf in sysroot should depend on both aktualizr and aktualizr-native. > For the record, here is the actual recipe: > https://github.com/advancedtelematic/meta-updater/blob/master/recipes-sota/aktualizr/aktualizr_git.bb -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] sysroot not being populated
On 08.06.2018 10:26, Khem Raj wrote: > > > On 6/8/18 12:27 AM, Patrick Vacek wrote: >> On 07.06.2018 19:06, Khem Raj wrote: >>> Hi Patrick >>> >>> On Mon, Jun 4, 2018 at 2:01 AM, Patrick Vacek >>> wrote: Hello all, I have a recipe (aktualizr-hsm-prov) that depends on another (aktualizr) to provide an executable and a config file. The former recipe includes `DEPENDS = "aktualizr-native"`, and my do_install() for aktualizr-hsm-prov has a line something like this: aktualizr -i ${STAGING_DIR_NATIVE}${libdir}/sota.conf The binary executable (aktualizr) runs, which tells me that the recipe can find that. (Although to be honest, I'm not sure which version it is running.) However, it doesn't find the config file, and sure enough, ${STAGING_DIR_NATIVE}${libdir} does not have the config file I expect. I can see that aktualizr-native is populating its sysroot-destdir, but it isn't getting copied to the sysroot for aktualizr-hsm-prov. I see this problem in sumo and master, although previously this logic has worked just fine in morty/pyro/rocko. Switching to depending on aktualizr (instead of aktualizr-native) and using STAGING_DIR_HOST or STAGING_DIR_TARGET does not help, and in fact makes it worse, as it can't even find the aktualizr executable in that case. What am I doing wrong? What changed between rocko and sumo? >>> is aktualizr recipe installing this .conf file when building native >>> version. If not may be >>> add it to do_install >>> >>> install -D -m 0644 ${D}${libdir}/sota.conf >> >> Hello Khem, >> >> I believe so. The aktualizr recipe includes the following (edited for >> simplicity): >> >> BBCLASSEXTEND =+ "native" >> >> do_install_append () { >> install -m 0644 ${S}/config/sota.conf ${D}/${libdir}/sota.conf >> } >> >> PACKAGES =+ " ${PN}-host-tools " >> >> FILES_${PN}-host-tools = " ${libdir}/sota.conf " >> > > Can you check location of sota.conf in the build tree for > aktualizr-native in directory called package/ Oddly, I do not see a package subdirectory inside the aktualizr-native directory in the build tree. I do see it inside the aktualizr directory, though, and it contains everything that I would expect. Is there some sort of configuration of the packaging system for a native recipe that I haven't done correctly? For the record, here is the actual recipe: https://github.com/advancedtelematic/meta-updater/blob/master/recipes-sota/aktualizr/aktualizr_git.bb -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] sysroot not being populated
On 07.06.2018 19:06, Khem Raj wrote: > Hi Patrick > > On Mon, Jun 4, 2018 at 2:01 AM, Patrick Vacek wrote: >> Hello all, >> >> I have a recipe (aktualizr-hsm-prov) that depends on another (aktualizr) >> to provide an executable and a config file. The former recipe includes >> `DEPENDS = "aktualizr-native"`, and my do_install() for >> aktualizr-hsm-prov has a line something like this: >> >> aktualizr -i ${STAGING_DIR_NATIVE}${libdir}/sota.conf >> >> The binary executable (aktualizr) runs, which tells me that the recipe >> can find that. (Although to be honest, I'm not sure which version it is >> running.) However, it doesn't find the config file, and sure enough, >> ${STAGING_DIR_NATIVE}${libdir} does not have the config file I expect. I >> can see that aktualizr-native is populating its sysroot-destdir, but it >> isn't getting copied to the sysroot for aktualizr-hsm-prov. >> >> I see this problem in sumo and master, although previously this logic >> has worked just fine in morty/pyro/rocko. >> >> Switching to depending on aktualizr (instead of aktualizr-native) and >> using STAGING_DIR_HOST or STAGING_DIR_TARGET does not help, and in fact >> makes it worse, as it can't even find the aktualizr executable in that case. >> >> What am I doing wrong? What changed between rocko and sumo? > is aktualizr recipe installing this .conf file when building native > version. If not may be > add it to do_install > > install -D -m 0644 ${D}${libdir}/sota.conf Hello Khem, I believe so. The aktualizr recipe includes the following (edited for simplicity): BBCLASSEXTEND =+ "native" do_install_append () { install -m 0644 ${S}/config/sota.conf ${D}/${libdir}/sota.conf } PACKAGES =+ " ${PN}-host-tools " FILES_${PN}-host-tools = " ${libdir}/sota.conf " Thanks, Patrick -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] sysroot not being populated
On 6/8/18 12:27 AM, Patrick Vacek wrote: On 07.06.2018 19:06, Khem Raj wrote: Hi Patrick On Mon, Jun 4, 2018 at 2:01 AM, Patrick Vacek wrote: Hello all, I have a recipe (aktualizr-hsm-prov) that depends on another (aktualizr) to provide an executable and a config file. The former recipe includes `DEPENDS = "aktualizr-native"`, and my do_install() for aktualizr-hsm-prov has a line something like this: aktualizr -i ${STAGING_DIR_NATIVE}${libdir}/sota.conf The binary executable (aktualizr) runs, which tells me that the recipe can find that. (Although to be honest, I'm not sure which version it is running.) However, it doesn't find the config file, and sure enough, ${STAGING_DIR_NATIVE}${libdir} does not have the config file I expect. I can see that aktualizr-native is populating its sysroot-destdir, but it isn't getting copied to the sysroot for aktualizr-hsm-prov. I see this problem in sumo and master, although previously this logic has worked just fine in morty/pyro/rocko. Switching to depending on aktualizr (instead of aktualizr-native) and using STAGING_DIR_HOST or STAGING_DIR_TARGET does not help, and in fact makes it worse, as it can't even find the aktualizr executable in that case. What am I doing wrong? What changed between rocko and sumo? is aktualizr recipe installing this .conf file when building native version. If not may be add it to do_install install -D -m 0644 ${D}${libdir}/sota.conf Hello Khem, I believe so. The aktualizr recipe includes the following (edited for simplicity): BBCLASSEXTEND =+ "native" do_install_append () { install -m 0644 ${S}/config/sota.conf ${D}/${libdir}/sota.conf } PACKAGES =+ " ${PN}-host-tools " FILES_${PN}-host-tools = " ${libdir}/sota.conf " Can you check location of sota.conf in the build tree for aktualizr-native in directory called package/ Thanks, Patrick -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] sysroot not being populated
Hi Patrick On Mon, Jun 4, 2018 at 2:01 AM, Patrick Vacek wrote: > Hello all, > > I have a recipe (aktualizr-hsm-prov) that depends on another (aktualizr) > to provide an executable and a config file. The former recipe includes > `DEPENDS = "aktualizr-native"`, and my do_install() for > aktualizr-hsm-prov has a line something like this: > > aktualizr -i ${STAGING_DIR_NATIVE}${libdir}/sota.conf > > The binary executable (aktualizr) runs, which tells me that the recipe > can find that. (Although to be honest, I'm not sure which version it is > running.) However, it doesn't find the config file, and sure enough, > ${STAGING_DIR_NATIVE}${libdir} does not have the config file I expect. I > can see that aktualizr-native is populating its sysroot-destdir, but it > isn't getting copied to the sysroot for aktualizr-hsm-prov. > > I see this problem in sumo and master, although previously this logic > has worked just fine in morty/pyro/rocko. > > Switching to depending on aktualizr (instead of aktualizr-native) and > using STAGING_DIR_HOST or STAGING_DIR_TARGET does not help, and in fact > makes it worse, as it can't even find the aktualizr executable in that case. > > What am I doing wrong? What changed between rocko and sumo? is aktualizr recipe installing this .conf file when building native version. If not may be add it to do_install install -D -m 0644 ${D}${libdir}/sota.conf -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
[yocto] sysroot not being populated
Hello all, I have a recipe (aktualizr-hsm-prov) that depends on another (aktualizr) to provide an executable and a config file. The former recipe includes `DEPENDS = "aktualizr-native"`, and my do_install() for aktualizr-hsm-prov has a line something like this: aktualizr -i ${STAGING_DIR_NATIVE}${libdir}/sota.conf The binary executable (aktualizr) runs, which tells me that the recipe can find that. (Although to be honest, I'm not sure which version it is running.) However, it doesn't find the config file, and sure enough, ${STAGING_DIR_NATIVE}${libdir} does not have the config file I expect. I can see that aktualizr-native is populating its sysroot-destdir, but it isn't getting copied to the sysroot for aktualizr-hsm-prov. I see this problem in sumo and master, although previously this logic has worked just fine in morty/pyro/rocko. Switching to depending on aktualizr (instead of aktualizr-native) and using STAGING_DIR_HOST or STAGING_DIR_TARGET does not help, and in fact makes it worse, as it can't even find the aktualizr executable in that case. What am I doing wrong? What changed between rocko and sumo? Thanks, Patrick Vacek HERE Technologies -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto