Re: [yocto] sstate cache not used when adding custom task
Hi,
2017-07-13 11:30 GMT+02:00 Joshua Lock :
> On Thu, 2017-07-13 at 10:52 +0200, Andreas Fenkart wrote:
>> Hi,
>>
>> A custom task in a bbclass causes all recipes including it to be
>> rebuilt every time. Link to full bbclass attached below.
>>
>> I already stripped that task down to this
>>
>>
>> addtask check after do_compile before do_build
>> addons_do_check() {
>> :
>> }
>>
>> If I remove the addtask line, the recipe is not rerun:
>
> The shared state machinery doesn't know how to capture your task. There
> are some guidelines on adding shared state wrapping to a task in the
> Yocto Project reference manual Shared State Cache section:
>
> http://www.yoctoproject.org/docs/latest/ref-manual/ref-manual.html#shar
> ed-state-cache
> http://www.yoctoproject.org/docs/latest/ref-manual/ref-manual.html#shar
> ed-state
>
> Joshua
That did the job, thank you
>
>> $ bitbake recipe_name
>> NOTE: Tasks Summary: Attempted 2746 tasks of which 2746 didn't need
>> to
>> be rerun and all succeeded.
>>
>> With the 'addtask check' all task of the recipe from fetch till
>> rm_work_all is executed every time.
>>
>> From the cooker log, do_populate_sysroot is the last task running
>> before do_build was, hence I changed the addtask line to this:
>> addtask check after do_populate_sysroot before do_build
>>
>> Then the recipe will not start from do_fetch on rerun, but will
>> reuse the previous do_populate_sysroot run. Hence only the last 4
>> tasks are run (do_check, do_build, do_rm_work, do_rm_work_all)
>>
>> I experimented with bitbake-diffsig/bitbake-dumpsig, but the hash
>> of the stamp files (do_check) are identical on each run, of
>> course its content too. Only the timestamp of the stamp file
>> changes each run.
>> I'm confused what triggers the rebuild. I looked into bitbake
>> trying to understand how the task dependency is calculated, how
>> the sstate files are used. But it's quite a biest and will take
>> me quite some time to understand.
>>
>> Any hints or regarding the issue or simply where to put a print
>> in bitbake to get me started was welcome.
>>
>> This is still yocto 2.0.2 as we missed a couple of upgrades.
>>
>> The full bbclass with the do_check task
>> https://git.digitalstrom.org/dss-oe/dss-oe/blob/master/yocto/dS/meta-
>> digitalstrom-devel/classes/addons.bbclass#L46
>>
>> /Andreas
--
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] swupate.bblcass dependencies not triggering rebuild
Hi
2017-07-28 15:20 GMT+02:00 Andreas Fenkart :
> Hi Stefano,
>
> 2017-06-14 10:32 GMT+02:00 Stefano Babic :
>> Hi Andreas,
>>
>> On 13/06/2017 14:18, Andreas Fenkart wrote:
>>> Hi list,
>>>
>>> I want to create a swupdate file, which is a cpio file with rootfs and
>>> some metadata to control software update. My problem is, that it is not
>>> rebuilt everytime the rootfs changes. I checked in the cooker log.
>>>
>>> NOTE: recipe swupdate-image-1.0-r4: task do_populate_sysroot_setscene:
>>> Succeeded
>>> NOTE: Running noexec task 5475 of 5478 (ID: 27,
>>> meta-devel/recipes-core/images/swupdate-image.bb, do_build)
>>>
>>> I generated the dot files with bitbake -g swupdate, and while do_swuimage
>>> has to run before do_build, do_build does not depend directly on
>>> do_swuimage:
>>>
>>> swupdate-image.do_createlink -> swupdate-image.do_swuimage
>>> swupdate-image.do_install -> swupdate-image.do_createlink
>>> swupdate-image.do_populate_sysroot -> swupdate-image.do_install
>>> swupdate-image.do_build" -> swupdate-image.do_populate_sysroot
>>>
>>> $ bitbake-diffsigs -t swupdate-image do_build
>>> ERROR: No sigdata files found matching swupdate-image do_build
>>>
>>> $ bitbake-diffsigs -t swupdate-image do_populate_sysroot
>>> Unable to find matching sigdata for
>>> recipes-core/images/swupdate-image.bb.do_install with hashes
>>> c0bc8039e8dbe8d70939dd3db2ceb922 or e9d7b242613f323d4d6adff3d6d4881b
>>>
>>> $ bitbake-diffsigs -t swupdate-image do_install
>>> ERROR: No sigdata files found matching swupdate-image do_install
>>>
>>> $ bitbake-diffsigs -t swupdate-image do_createlink
>>> Hash for dependent task imagesswupdate-image.bb.do_swuimage changed
>>> from cec64d7e21b0375ff641427d5d631254 to
>>> 45510576a402839f4c4d4ac9b9eff578
>>> Hash for dependent task imagesrootfs.bb.do_build changed from
>>> 876eb436b444f84efd355c0294b871bf to fabbabdd000427cdf5e721b59784d5c4
>>> Unable to find matching sigdata for
>>> meta-devel/recipes-core/images/rootfs.bb.do_build with hashes
>>> 876eb436b444f84efd355c0294b871bf or fabbabdd000427cdf5e721b59784d5c4
>>>
>>> The dependency on rootfs is added here:
>>> https://github.com/sbabic/meta-swupdate/blob/master/classes/swupdate.bbclass#L64
>>> with IMAGE_DEPENDS="rootfs"
>>>
>>> Is it correct to 'depend' on :do_build? It seems rather uncommon.
>>
>> I think that the first attempt to set dependency here was with
>> do_rootfs, but this just works for rootfs, not for all other images. In
>> the history. commit b2f47aa9de53a98380b8677eb9e078db608b0ad3 changed
>> from do_populatesysroot to do_build because it was noted that it runs
>> too early.
>>
>> I am not sure if I have this issue, maybe it was sporadic. I wrote a
>> patch, really to fix an issue in the transition morty --> pyro, where I
>> changed the dependency as:
>> diff --git a/classes/swupdate.bbclass b/classes/swupdate.bbclass
>> index 44e4546..511f147 100644
>> --- a/classes/swupdate.bbclass
>> +++ b/classes/swupdate.bbclass
>> @@ -61,16 +61,23 @@ def swupdate_getdepends(d):
>>
>> depstr = ""
>> for dep in deps:
>> -depstr += " " + dep + ":do_build"
>> +if dep.find(":") != -1:
>> +deptask = ""
>> +else:
>> +if (dep == "virtual/kernel") or (dep == "virtual/bootloader"):
>> +deptask = ":do_deploy"
>> +else:
>> +deptask = ":do_image_complete"
>> +
>> +depstr += " " + dep + deptask
>> +
>> return depstr
>>
>>
>> So "do_image_complete" instead of "do_build", but I have to check
>> explicitely for kernel and bootloader because they have no
>> do_image_complete. But I do not know if this can be a correct solution.
>>
>>>
>>> I checked the latest swupdate-image.do_populate siginfo in sstate, which is
>>> pretty large, so adding just the task-dependencies:
>>> ..
>>> Tasks this task depends on:
>>> ['..-devel/recipes-core/images/swupdate-image.bb.do_install']
>>> This task depends on the checksums of files: []
>>> Hash for dependent task
>>> ./meta-devel/recipes-core/images/swupdate-image.bb.do_install is
>
Re: [yocto] swupate.bblcass dependencies not triggering rebuild
Hi Stefano,
2017-06-14 10:32 GMT+02:00 Stefano Babic :
> Hi Andreas,
>
> On 13/06/2017 14:18, Andreas Fenkart wrote:
>> Hi list,
>>
>> I want to create a swupdate file, which is a cpio file with rootfs and
>> some metadata to control software update. My problem is, that it is not
>> rebuilt everytime the rootfs changes. I checked in the cooker log.
>>
>> NOTE: recipe swupdate-image-1.0-r4: task do_populate_sysroot_setscene:
>> Succeeded
>> NOTE: Running noexec task 5475 of 5478 (ID: 27,
>> meta-devel/recipes-core/images/swupdate-image.bb, do_build)
>>
>> I generated the dot files with bitbake -g swupdate, and while do_swuimage
>> has to run before do_build, do_build does not depend directly on do_swuimage:
>>
>> swupdate-image.do_createlink -> swupdate-image.do_swuimage
>> swupdate-image.do_install -> swupdate-image.do_createlink
>> swupdate-image.do_populate_sysroot -> swupdate-image.do_install
>> swupdate-image.do_build" -> swupdate-image.do_populate_sysroot
>>
>> $ bitbake-diffsigs -t swupdate-image do_build
>> ERROR: No sigdata files found matching swupdate-image do_build
>>
>> $ bitbake-diffsigs -t swupdate-image do_populate_sysroot
>> Unable to find matching sigdata for
>> recipes-core/images/swupdate-image.bb.do_install with hashes
>> c0bc8039e8dbe8d70939dd3db2ceb922 or e9d7b242613f323d4d6adff3d6d4881b
>>
>> $ bitbake-diffsigs -t swupdate-image do_install
>> ERROR: No sigdata files found matching swupdate-image do_install
>>
>> $ bitbake-diffsigs -t swupdate-image do_createlink
>> Hash for dependent task imagesswupdate-image.bb.do_swuimage changed
>> from cec64d7e21b0375ff641427d5d631254 to
>> 45510576a402839f4c4d4ac9b9eff578
>> Hash for dependent task imagesrootfs.bb.do_build changed from
>> 876eb436b444f84efd355c0294b871bf to fabbabdd000427cdf5e721b59784d5c4
>> Unable to find matching sigdata for
>> meta-devel/recipes-core/images/rootfs.bb.do_build with hashes
>> 876eb436b444f84efd355c0294b871bf or fabbabdd000427cdf5e721b59784d5c4
>>
>> The dependency on rootfs is added here:
>> https://github.com/sbabic/meta-swupdate/blob/master/classes/swupdate.bbclass#L64
>> with IMAGE_DEPENDS="rootfs"
>>
>> Is it correct to 'depend' on :do_build? It seems rather uncommon.
>
> I think that the first attempt to set dependency here was with
> do_rootfs, but this just works for rootfs, not for all other images. In
> the history. commit b2f47aa9de53a98380b8677eb9e078db608b0ad3 changed
> from do_populatesysroot to do_build because it was noted that it runs
> too early.
>
> I am not sure if I have this issue, maybe it was sporadic. I wrote a
> patch, really to fix an issue in the transition morty --> pyro, where I
> changed the dependency as:
> diff --git a/classes/swupdate.bbclass b/classes/swupdate.bbclass
> index 44e4546..511f147 100644
> --- a/classes/swupdate.bbclass
> +++ b/classes/swupdate.bbclass
> @@ -61,16 +61,23 @@ def swupdate_getdepends(d):
>
> depstr = ""
> for dep in deps:
> -depstr += " " + dep + ":do_build"
> +if dep.find(":") != -1:
> +deptask = ""
> +else:
> +if (dep == "virtual/kernel") or (dep == "virtual/bootloader"):
> +deptask = ":do_deploy"
> +else:
> +deptask = ":do_image_complete"
> +
> +depstr += " " + dep + deptask
> +
> return depstr
>
>
> So "do_image_complete" instead of "do_build", but I have to check
> explicitely for kernel and bootloader because they have no
> do_image_complete. But I do not know if this can be a correct solution.
>
>>
>> I checked the latest swupdate-image.do_populate siginfo in sstate, which is
>> pretty large, so adding just the task-dependencies:
>> ..
>> Tasks this task depends on:
>> ['..-devel/recipes-core/images/swupdate-image.bb.do_install']
>> This task depends on the checksums of files: []
>> Hash for dependent task
>> ./meta-devel/recipes-core/images/swupdate-image.bb.do_install is
>> e9d7b242613f323d4d6adff3d6d4881b
>> Computed Hash is d8b1d389351c25e0c4a519fb8fe9ddda
>> ..
>>
>> Which makes me wonder, since bitbake-diffsig said do_install has no sigdata
>>
>
> We have do_install[noexec] = "1" in the class - is this the reason ?
I removed all the 'noexec' stamps from swupdate.bbclass, but still the
cooker.log says, that none of the swupdate-image tasks needs
execut
[yocto] sstate cache not used when adding custom task
Hi,
A custom task in a bbclass causes all recipes including it to be
rebuilt every time. Link to full bbclass attached below.
I already stripped that task down to this
addtask check after do_compile before do_build
addons_do_check() {
:
}
If I remove the addtask line, the recipe is not rerun:
$ bitbake recipe_name
NOTE: Tasks Summary: Attempted 2746 tasks of which 2746 didn't need to
be rerun and all succeeded.
With the 'addtask check' all task of the recipe from fetch till
rm_work_all is executed every time.
>From the cooker log, do_populate_sysroot is the last task running
before do_build was, hence I changed the addtask line to this:
addtask check after do_populate_sysroot before do_build
Then the recipe will not start from do_fetch on rerun, but will
reuse the previous do_populate_sysroot run. Hence only the last 4
tasks are run (do_check, do_build, do_rm_work, do_rm_work_all)
I experimented with bitbake-diffsig/bitbake-dumpsig, but the hash
of the stamp files (do_check) are identical on each run, of
course its content too. Only the timestamp of the stamp file
changes each run.
I'm confused what triggers the rebuild. I looked into bitbake
trying to understand how the task dependency is calculated, how
the sstate files are used. But it's quite a biest and will take
me quite some time to understand.
Any hints or regarding the issue or simply where to put a print
in bitbake to get me started was welcome.
This is still yocto 2.0.2 as we missed a couple of upgrades.
The full bbclass with the do_check task
https://git.digitalstrom.org/dss-oe/dss-oe/blob/master/yocto/dS/meta-digitalstrom-devel/classes/addons.bbclass#L46
/Andreas
--
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto
[yocto] swupate.bblcass dependencies not triggering rebuild
Hi list, I want to create a swupdate file, which is a cpio file with rootfs and some metadata to control software update. My problem is, that it is not rebuilt everytime the rootfs changes. I checked in the cooker log. NOTE: recipe swupdate-image-1.0-r4: task do_populate_sysroot_setscene: Succeeded NOTE: Running noexec task 5475 of 5478 (ID: 27, meta-devel/recipes-core/images/swupdate-image.bb, do_build) I generated the dot files with bitbake -g swupdate, and while do_swuimage has to run before do_build, do_build does not depend directly on do_swuimage: swupdate-image.do_createlink -> swupdate-image.do_swuimage swupdate-image.do_install -> swupdate-image.do_createlink swupdate-image.do_populate_sysroot -> swupdate-image.do_install swupdate-image.do_build" -> swupdate-image.do_populate_sysroot $ bitbake-diffsigs -t swupdate-image do_build ERROR: No sigdata files found matching swupdate-image do_build $ bitbake-diffsigs -t swupdate-image do_populate_sysroot Unable to find matching sigdata for recipes-core/images/swupdate-image.bb.do_install with hashes c0bc8039e8dbe8d70939dd3db2ceb922 or e9d7b242613f323d4d6adff3d6d4881b $ bitbake-diffsigs -t swupdate-image do_install ERROR: No sigdata files found matching swupdate-image do_install $ bitbake-diffsigs -t swupdate-image do_createlink Hash for dependent task imagesswupdate-image.bb.do_swuimage changed from cec64d7e21b0375ff641427d5d631254 to 45510576a402839f4c4d4ac9b9eff578 Hash for dependent task imagesrootfs.bb.do_build changed from 876eb436b444f84efd355c0294b871bf to fabbabdd000427cdf5e721b59784d5c4 Unable to find matching sigdata for meta-devel/recipes-core/images/rootfs.bb.do_build with hashes 876eb436b444f84efd355c0294b871bf or fabbabdd000427cdf5e721b59784d5c4 The dependency on rootfs is added here: https://github.com/sbabic/meta-swupdate/blob/master/classes/swupdate.bbclass#L64 with IMAGE_DEPENDS="rootfs" Is it correct to 'depend' on :do_build? It seems rather uncommon. I checked the latest swupdate-image.do_populate siginfo in sstate, which is pretty large, so adding just the task-dependencies: .. Tasks this task depends on: ['..-devel/recipes-core/images/swupdate-image.bb.do_install'] This task depends on the checksums of files: [] Hash for dependent task ./meta-devel/recipes-core/images/swupdate-image.bb.do_install is e9d7b242613f323d4d6adff3d6d4881b Computed Hash is d8b1d389351c25e0c4a519fb8fe9ddda .. Which makes me wonder, since bitbake-diffsig said do_install has no sigdata I must admit that I feel pretty lost here. /Andreas -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
[yocto] Problems using "In-Tree" defconfig File
I want to use a defconfig, that is provided by the linux git
repository, hence this seems to fit my need:
http://www.yoctoproject.org/docs/1.8/kernel-dev/kernel-dev.html#using-an-in-tree-defconfig-file
KBUILD_DEFCONFIG_KMACHINE ?= defconfig_file
Anybody used that feature recently? I couldn't get it to work, :-(
The build fails in kernel_config target, in merge_config.sh. It fails
to merge kernel config fragments. Probably because I don't have any.
[INFO] collecting configs in .meta/meta-series
ARCH=arm O=/linux-dss11e-standard-build merge_config.sh -d
mv: cannot stat '/linux-dss11e-standard-build/.tmp.config*': No
such file or directory
Grep'ing for KBUILD_DEFCONFIG in kernel-tools only get_defconfig seems
to use that variable
$ get_defconfig arm dss20_defconfig
linux/arch/arm/configs/dss20_defconfig ...so that works fine
I took that snipped from check_defconfig functoin, here:
http://git.yoctoproject.org/cgit/cgit.cgi/yocto-kernel-tools/tree/tools/createme#n201
but that function is not called anywhere in script createme. After all
that script only creates the meta folder but has nothing to do with
kernel configuration. Seems to be a dead end. Probably get_defconfig
call should be added to configme script
Anybody had the same problem, is there a patch. Or did I misconfigure something?
kind regards,
Andi
Here my recipe:
---
inherit kernel
require recipes-kernel/linux/linux-yocto.inc
SRCBRANCH = "dSS"
SRCREV="82d97d74bb072fe24be10b46d0782d3735130199"
SRC_URI =
"git://git.digitalstrom.org/bsp/linux.git;protocol=https;branch=${SRCBRANCH}"
LINUX_VERSION = "4.1"
LINUX_VERSION_EXTENSION = ""
KBUILD_DEFCONFIG_dss11e = "dss20_defconfig"
PR = "r1"
PV = "${LINUX_VERSION}+git${SRCPV}"
COMPATIBLE_MACHINE = "(dss11e)"
--
___
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.org/listinfo/yocto
