Re: [yocto] IMAGE_INSTALL_append workflow
Hi Trevor, Le Mon, 26 Nov 2012 21:20:30 -0500, Trevor Woerner twoer...@gmail.com a écrit : On Mon, Nov 26, 2012 at 6:52 PM, Trevor Woerner twoer...@gmail.com wrote: I haven't yet figured out if the PREFERRED_PROVIDER_virtual/kernel = linux-yocto-rt line might be the culprit. No, it is not related, this problem doesn't need any fancy configuration features. This problem is reproducible with just the plain qemux86 machine and the core-image-minimal image. Also, before my latest tests I pulled the latest git repo. To reproduce this problem: - use the latest git repositories - run source poky/oe-init-build-env qemux86 - make no modifications to conf/bblayers.conf - make no changes to conf/local.conf - run bitbake core-image-minimal After that completes successfully add the following line to your conf/local.conf: IMAGE_INSTALL_append = trace-cmd - re-run the build bitbake core-image-minimal This should complete successfully. When it does, modify the last line of conf/local.conf to be: IMAGE_INSTALL_append = trace-cmd bash - try re-running the build now bitbake core-image-minimal. It fails for me as I have detailed in my previous emails. If I manually do: $ bitbake bash $ bitbake core-image-minimal everything succeeds and my image contains 'bash'. But something tells me there's a bug in there somewhere. I confirm the problem when following this procedure step by step. with bitbake -e we see that bash is in RDEPENDS and in IMAGE_INSTALL with bitbake -g we see that bash is not present in any of the *depends.dot files Eric ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] IMAGE_INSTALL_append workflow
On Tue, Nov 27, 2012 at 5:39 AM, Eric Bénard e...@eukrea.com wrote: I confirm the problem when following this procedure step by step. Thank you, Eric, for the confirmation. Just to clarify for anyone following along: the problem isn't with bash or trace-cmd themselves. If I perform the same steps but use 'bash' first, then 'trace-cmd', adding 'bash' succeeds but later adding 'trace-cmd' causes 'trace-cmd' to fail in the same way 'bash' did when it was added second. ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] IMAGE_INSTALL_append workflow
I can also confirm this. A package (I've tried with diffutils bash) must be built separately before it's added to an image via IMAGE_INSTALL_append. /Jonas - fetched poky today -Original Message- From: yocto-boun...@yoctoproject.org [mailto:yocto-boun...@yoctoproject.org] On Behalf Of Trevor Woerner Sent: den 27 november 2012 03:21 To: Eren Türkay Cc: yocto@yoctoproject.org Subject: Re: [yocto] IMAGE_INSTALL_append workflow On Mon, Nov 26, 2012 at 6:52 PM, Trevor Woerner twoer...@gmail.com wrote: I haven't yet figured out if the PREFERRED_PROVIDER_virtual/kernel = linux-yocto-rt line might be the culprit. No, it is not related, this problem doesn't need any fancy configuration features. This problem is reproducible with just the plain qemux86 machine and the core-image-minimal image. Also, before my latest tests I pulled the latest git repo. To reproduce this problem: - use the latest git repositories - run source poky/oe-init-build-env qemux86 - make no modifications to conf/bblayers.conf - make no changes to conf/local.conf - run bitbake core-image-minimal After that completes successfully add the following line to your conf/local.conf: IMAGE_INSTALL_append = trace-cmd - re-run the build bitbake core-image-minimal This should complete successfully. When it does, modify the last line of conf/local.conf to be: IMAGE_INSTALL_append = trace-cmd bash - try re-running the build now bitbake core-image-minimal. It fails for me as I have detailed in my previous emails. If I manually do: $ bitbake bash $ bitbake core-image-minimal everything succeeds and my image contains 'bash'. But something tells me there's a bug in there somewhere. ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] IMAGE_INSTALL_append workflow
Le Tue, 27 Nov 2012 14:00:34 +, Jonas Jonsson L jonas.l.jons...@ericsson.com a écrit : I can also confirm this. A package (I've tried with diffutils bash) must be built separately before it's added to an image via IMAGE_INSTALL_append. that seems to work the first time you add something using IMAGE_INSTALL_append. Then it fails when you add packages later (bitbake wants to install the package in the image but doesn't trigger the build of the package before). Eric ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] IMAGE_INSTALL_append workflow
On Tue, Nov 27, 2012 at 08:40:26AM -0500, Trevor Woerner wrote: Thank you, Eric, for the confirmation. Just to clarify for anyone following along: the problem isn't with bash or trace-cmd themselves. If I perform the same steps but use 'bash' first, then 'trace-cmd', adding 'bash' succeeds but later adding 'trace-cmd' causes 'trace-cmd' to fail in the same way 'bash' did when it was added second. Since it's confirmed from different people, I think it would be better to report it using bugzilla. I guess that it's not an intended behaviour/feature of bitbake. Please inform the bug number so that anyone who is interested can follow. -- . 73! DE TA1AET http://erenturkay.com/ pgpIGycH7Ifpr.pgp Description: PGP signature ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] IMAGE_INSTALL_append workflow
On Tue, 27 Nov 2012, Eren Türkay wrote: On Tue, Nov 27, 2012 at 08:40:26AM -0500, Trevor Woerner wrote: Thank you, Eric, for the confirmation. Just to clarify for anyone following along: the problem isn't with bash or trace-cmd themselves. If I perform the same steps but use 'bash' first, then 'trace-cmd', adding 'bash' succeeds but later adding 'trace-cmd' causes 'trace-cmd' to fail in the same way 'bash' did when it was added second. Since it's confirmed from different people, I think it would be better to report it using bugzilla. I guess that it's not an intended behaviour/feature of bitbake. Please inform the bug number so that anyone who is interested can follow. here's a test i made based on nothing but fetching. using oe-core, i selected qemuarm, added this line at the bottom of my local.conf: IMAGE_INSTALL_append = sysfsutils strace and ran $ bitbake -c fetchall core-image-minimal when that was done, i checked my downloads directory for the results of fetching (and patching) those two packages: $ ls strace* strace-4.7.tar.xz strace-4.7.tar.xz.done $ ls sysfs* sysfs.sh.done sysfsutils-2.1.0.tar.gz sysfsutils-2.0.0-class-dup.patch.done sysfsutils-2.1.0.tar.gz.done $ so that seemed to work fine. so i started over with a clean build and did it in two stages. first, i reduced the line to: IMAGE_INSTALL_append = sysfsutils ran the same fetchall operation again, and verified that that content had been fetched: $ ls sysfs* sysfs.sh.done sysfsutils-2.1.0.tar.gz sysfsutils-2.0.0-class-dup.patch.done sysfsutils-2.1.0.tar.gz.done $ i then *changed* the line in local.conf to: IMAGE_INSTALL_append = strace sysfsutils and ran the fetchall again. this time, no strace source was fetched, so it looks like an issue with simple fetching. rday -- Robert P. J. Day Ottawa, Ontario, CANADA http://crashcourse.ca Twitter: http://twitter.com/rpjday LinkedIn: http://ca.linkedin.com/in/rpjday ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] IMAGE_INSTALL_append workflow
On Tue, Nov 27, 2012 at 12:24 PM, Robert P. J. Day rpj...@crashcourse.ca wrote: and ran the fetchall again. this time, no strace source was fetched, so it looks like an issue with simple fetching. Thank you for your testing and observations. But I'd like to point out that in my tests, all the relevant files have already been fetched. My DL_DIR already contains everything to do with bash, trace-cmd, and various others that I have been testing. The errors I am getting are not fetching-related errors, but are errors because do_rootfs can't find any packages (since packages relating to the recently-appended recipes have not been built). Somewhere in the guts of bitbake a modified IMAGE_INSTALL_append variable is not being noticed; packages aren't being created (as I've noticed) and sources aren't being downloaded (as you have just noticed). But somehow do_rootfs is noticing the change and complaining it can't find the necessary packages. ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] IMAGE_INSTALL_append workflow
Le Tue, 27 Nov 2012 12:24:06 -0500 (EST), Robert P. J. Day rpj...@crashcourse.ca a écrit : and ran the fetchall again. this time, no strace source was fetched, so it looks like an issue with simple fetching. It's not fetched because the build system fails to see the new package as a dependencie of the image so that's not a fetching issue but a more global one. Eric ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] IMAGE_INSTALL_append workflow
On Tue, 27 Nov 2012, Trevor Woerner wrote: On Tue, Nov 27, 2012 at 12:24 PM, Robert P. J. Day rpj...@crashcourse.ca wrote: and ran the fetchall again. this time, no strace source was fetched, so it looks like an issue with simple fetching. Thank you for your testing and observations. But I'd like to point out that in my tests, all the relevant files have already been fetched. My DL_DIR already contains everything to do with bash, trace-cmd, and various others that I have been testing. The errors I am getting are not fetching-related errors, but are errors because do_rootfs can't find any packages (since packages relating to the recently-appended recipes have not been built). Somewhere in the guts of bitbake a modified IMAGE_INSTALL_append variable is not being noticed; packages aren't being created (as I've noticed) and sources aren't being downloaded (as you have just noticed). But somehow do_rootfs is noticing the change and complaining it can't find the necessary packages. ah, quite so ... but it's still useful to note that the problem extends even to the act of fetching. weird. rday -- Robert P. J. Day Ottawa, Ontario, CANADA http://crashcourse.ca Twitter: http://twitter.com/rpjday LinkedIn: http://ca.linkedin.com/in/rpjday ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] IMAGE_INSTALL_append workflow
On Tue, 27 Nov 2012, Trevor Woerner wrote: On Tue, Nov 27, 2012 at 12:24 PM, Robert P. J. Day rpj...@crashcourse.ca wrote: and ran the fetchall again. this time, no strace source was fetched, so it looks like an issue with simple fetching. Thank you for your testing and observations. But I'd like to point out that in my tests, all the relevant files have already been fetched. My DL_DIR already contains everything to do with bash, trace-cmd, and various others that I have been testing. The errors I am getting are not fetching-related errors, but are errors because do_rootfs can't find any packages (since packages relating to the recently-appended recipes have not been built). Somewhere in the guts of bitbake a modified IMAGE_INSTALL_append variable is not being noticed; packages aren't being created (as I've noticed) and sources aren't being downloaded (as you have just noticed). But somehow do_rootfs is noticing the change and complaining it can't find the necessary packages. for what it's worth, if you're building an image that's a variation of a core image, an alternate way of adding packages after the fact is: CORE_IMAGE_EXTRA_INSTALL = sysfsutils strace bash i just tested it and, when i added a new entry in that list, at least the fetching understood and fetched that new source and patched it. so it doesn't solve the original problem with IMAGE_INSTALL_append, but feel free to see if that variation works for you. again, it's only valid if you're building a core image or some variation thereof. rday -- Robert P. J. Day Ottawa, Ontario, CANADA http://crashcourse.ca Twitter: http://twitter.com/rpjday LinkedIn: http://ca.linkedin.com/in/rpjday ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] IMAGE_INSTALL_append workflow
This issue has been entered into the yocto bugzilla as: https://bugzilla.yoctoproject.org/show_bug.cgi?id=3503 ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] IMAGE_INSTALL_append workflow
I'm kinda surprised nobody jumped in on this one; either my question is so silly it isn't worth anyone's time, or I'm the only one who wants to add packages to an image after a full build. I know my question has nothing to do with trace-cmd specifically, adding any package after a full build results in the same problem. It would certainly seem odd that the yocto workflow assumes you always know 100% ahead of time which packages you want in your final image. I know when I was playing around with Hob this wasn't an issue: I would simply add the new packages and Hob would build them and add them to the image. From what I have seen from skimming the documentation, the documentation talks at length about how to add a new package to the configuration, but it doesn't say what commands to issue after modifying the config files. Obviously running bitbake image alone doesn't work. On Fri, Nov 23, 2012 at 1:27 PM, Trevor Woerner twoer...@gmail.com wrote: There have been many emails and lots of documentation describing how to add a package to a build, and that's all great. Today I was playing around with an image I had already built and decided I wanted to add a new package to the mix: trace-cmd. I added trace-cmd (with the required leading space) to IMAGE_INSTALL_append and re-ran my bitbake image command. It failed: Unable to resolve package trace-cmd If I perform: $ bitbake trace-cmd and then $ bitbake image it works fine. What am I missing? As it turns out, if I knew before my first build that I wanted trace-cmd, I could have just added it as described above and everything would have worked as well. Best regards, Trevor ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] IMAGE_INSTALL_append workflow
On Mon, Nov 26, 2012 at 10:47:31AM -0500, Trevor Woerner wrote:
I'm kinda surprised nobody jumped in on this one; either my question
is so silly it isn't worth anyone's time, or I'm the only one who
wants to add packages to an image after a full build. I know my
question has nothing to do with trace-cmd specifically, adding any
package after a full build results in the same problem.
I don't think you are the first person who wants to add a package to image after
the image has been built :)
The obvious problem would have been leading spaces in IMAGE_INSTALL_append but
you explicitly stated its correct. Normally, the class that's responsible for
image creation (image.bbclass) depends on the packages defined in IMAGE_INSTALL
so that the packages are built before the image creation begins.
image.bbclass, line 9
-
RDEPENDS += ${IMAGE_INSTALL} ${LINGUAS_INSTALL} ${NORMAL_FEATURE_INSTALL}
${ROOTFS_BOOTSTRAP_INSTALL}
On irc (#oe), Eric Benard tried IMAGE_INSTALL_append = trace-cmd after a full
build and reported that it built fine. I'm CCing him.
I believe that it will be better to include your image file and the files that
you changed so that it will be easier to debug. Mentioning your layers,
configuration, version of Yocto you use will also be helpful.
Regards,
Eren
--
. 73! DE TA1AET
http://erenturkay.com/
pgpbu9aAsczjX.pgp
Description: PGP signature
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] IMAGE_INSTALL_append workflow
On 11/26/2012 02:04 PM, Eren Türkay wrote:
On Mon, Nov 26, 2012 at 10:47:31AM -0500, Trevor Woerner wrote:
I'm kinda surprised nobody jumped in on this one; either my question
is so silly it isn't worth anyone's time, or I'm the only one who
wants to add packages to an image after a full build. I know my
question has nothing to do with trace-cmd specifically, adding any
package after a full build results in the same problem.
I don't think you are the first person who wants to add a package to image after
the image has been built :)
The obvious problem would have been leading spaces in IMAGE_INSTALL_append but
you explicitly stated its correct. Normally, the class that's responsible for
image creation (image.bbclass) depends on the packages defined in IMAGE_INSTALL
so that the packages are built before the image creation begins.
image.bbclass, line 9
-
RDEPENDS += ${IMAGE_INSTALL} ${LINGUAS_INSTALL} ${NORMAL_FEATURE_INSTALL}
${ROOTFS_BOOTSTRAP_INSTALL}
On irc (#oe), Eric Benard tried IMAGE_INSTALL_append = trace-cmd after a full
build and reported that it built fine. I'm CCing him.
Have you also tried this?
IMAGE_INSTALL_append += foo
Aws\
I believe that it will be better to include your image file and the files that
you changed so that it will be easier to debug. Mentioning your layers,
configuration, version of Yocto you use will also be helpful.
Regards,
Eren
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto
--
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] IMAGE_INSTALL_append workflow
On Mon, 26 Nov 2012, Aws Ismail wrote: Have you also tried this? IMAGE_INSTALL_append += foo GAHH!!! :-) rday -- Robert P. J. Day Ottawa, Ontario, CANADA http://crashcourse.ca Twitter: http://twitter.com/rpjday LinkedIn: http://ca.linkedin.com/in/rpjday ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] IMAGE_INSTALL_append workflow
Thank you so much for having a crack at this. Your results have allowed me to dig a bit deeper into this issue. I'm still trying to narrow down the exact issue but here's what I have so far. I'm working with the git repository, but I've been playing around with this for about a week, so I'm not 100% up to date. Specifically I'm using: master:c206007235df3e00c21a1cbf30e8a1bccc07a9e5 Starting with that repository I follow the regular steps: $ poky/oe-init-build qemux86 then I edit the conf/local.conf to end up with the following (virtually no changes, just the lines at the end): PARALLEL_MAKE = -j 8 MACHINE ??= qemux86 DISTRO ?= poky PACKAGE_CLASSES ?= package_rpm EXTRA_IMAGE_FEATURES = debug-tweaks USER_CLASSES ?= buildstats image-mklibs image-prelink PATCHRESOLVE = noop CONF_VERSION = 1 BB_DANGLINGAPPENDS_WARNONLY = yes PREFERRED_PROVIDER_virtual/kernel = linux-yocto-rt then I perform $ bitbake core-image-minimal everything goes well. I then add the following line to the end of my conf/local.conf IMAGE_INSTALL_append = trace-cmd and run $ bitbake core-image-minimal everything builds just fine. Then I modify that last line as follows: IMAGE_INSTALL_append = trace-cmd bash now when I try to build I get: Build Configuration: BB_VERSION= 1.16.0 TARGET_ARCH = i586 TARGET_OS = linux MACHINE = qemux86 DISTRO= poky DISTRO_VERSION= 1.3+snapshot-20121126 TUNE_FEATURES = m32 i586 TARGET_FPU= meta meta-yocto meta-yocto-bsp= master:c206007235df3e00c21a1cbf30e8a1bccc07a9e5 NOTE: Resolving any missing task queue dependencies NOTE: Preparing runqueue NOTE: Executing SetScene Tasks NOTE: Executing RunQueue Tasks ERROR: Function failed: do_rootfs (see /home/trevor/devel/yocto/gitrepo-method/qemux86/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/temp/log.do_rootfs.19668 for further information) ERROR: Logfile of failure stored in: /home/trevor/devel/yocto/gitrepo-method/qemux86/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/temp/log.do_rootfs.19668 Log data follows: | DEBUG: Executing shell function do_rootfs | Generating solve db for /home/trevor/devel/yocto/gitrepo-method/qemux86/tmp/deploy/rpm/qemux86... | Generating solve db for /home/trevor/devel/yocto/gitrepo-method/qemux86/tmp/deploy/rpm/i586... | Generating solve db for /home/trevor/devel/yocto/gitrepo-method/qemux86/tmp/deploy/rpm/all... |total: 1 0.00 MB 2.930406 secs |fingerprint: 1104 0.032554 MB 0.063469 secs |install: 368 0.00 MB 0.878790 secs |digest:736 5.357010 MB 0.027508 secs |signature: 736 0.00 MB 1.443277 secs |dbadd: 368 0.00 MB 0.866237 secs |dbget: 12620 0.00 MB 0.018020 secs |dbput: 368 3.063072 MB 0.541058 secs |readhdr: 3681 6.087218 MB 0.010347 secs |hdrload: 1929 10.076294 MB 0.012106 secs |hdrget: 70812 0.00 MB 0.081749 secs | Generating solve db for /home/trevor/devel/yocto/gitrepo-method/qemux86/tmp/deploy/rpm/all... | Processing packagegroup-core-boot... | Processing trace-cmd... | Processing bash... | Unable to resolve package bash | ERROR: some packages were missing | ERROR: Function failed: do_rootfs (see /home/trevor/devel/yocto/gitrepo-method/qemux86/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/temp/log.do_rootfs.19668 for further information) ERROR: Task 7 (/home/trevor/devel/yocto/gitrepo-method/poky/meta/recipes-core/images/ core-image-minimal.bb, do_rootfs) failed with exit code '1' NOTE: Tasks Summary: Attempted 1443 tasks of which 1442 didn't need to be rerun and 1 failed. No currently running tasks (1442 of 1444) Summary: 1 task failed: /home/trevor/devel/yocto/gitrepo-method/poky/meta/recipes-core/images/ core-image-minimal.bb, do_rootfs Summary: There was 1 ERROR message shown, returning a non-zero exit code. Can anyone reproduce this? It doesn't make sense. Initially when I first saw this problem, I had been using building core-image-rt so I thought it had something to do with that image. But my tests from today seem to indicate that it doesn't since I was able to reproduce my problem with core-image-minimal too. I haven't yet figured out if the PREFERRED_PROVIDER_virtual/kernel = linux-yocto-rt line might be the culprit. ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] IMAGE_INSTALL_append workflow
On Mon, Nov 26, 2012 at 6:52 PM, Trevor Woerner twoer...@gmail.com wrote: I haven't yet figured out if the PREFERRED_PROVIDER_virtual/kernel = linux-yocto-rt line might be the culprit. No, it is not related, this problem doesn't need any fancy configuration features. This problem is reproducible with just the plain qemux86 machine and the core-image-minimal image. Also, before my latest tests I pulled the latest git repo. To reproduce this problem: - use the latest git repositories - run source poky/oe-init-build-env qemux86 - make no modifications to conf/bblayers.conf - make no changes to conf/local.conf - run bitbake core-image-minimal After that completes successfully add the following line to your conf/local.conf: IMAGE_INSTALL_append = trace-cmd - re-run the build bitbake core-image-minimal This should complete successfully. When it does, modify the last line of conf/local.conf to be: IMAGE_INSTALL_append = trace-cmd bash - try re-running the build now bitbake core-image-minimal. It fails for me as I have detailed in my previous emails. If I manually do: $ bitbake bash $ bitbake core-image-minimal everything succeeds and my image contains 'bash'. But something tells me there's a bug in there somewhere. ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
