I haven't tried the patches, but I really like this idea (I was suggesting something like that since 2011 http://permalink.gmane.org/gmane.comp.handhelds.openembedded.core/10327) and I'm glad you weren't discouraged attempting to do this.
It also implements 3) b) idea from https://bugzilla.yoctoproject.org/show_bug.cgi?id=5970 , you might be interested read that ticket. Thanks On Fri, Jun 30, 2017 at 11:48 AM, Michael Ho <michael...@bmw-carit.de> wrote: > Hi, at BMW Car IT we are working on an experimental feature to improve > sstate > cache hits and we are looking for comments on the approach who might have > some > insights to the problem and seeing if anyone is interested in the feature > for > mainline. > > The sstate-cache of a recipe is tied closely to its build dependencies, as > the > signature generated for a task includes all parent task's signatures as > part of > the signature information. This means that when changes occur in the parent > recipes, even if insignificant, all children recipes that have valid sstate > cache must invalidate their sstate cache objects. > > What this patchset does is propose to add another optional variable to > recipes, > CDEPENDS, which acts like DEPENDS for all RunQueue purposes but for > signature > generation it excludes any parent tasks that come from dependencies listed > in > it. This is to break the signature change domino effect. > > This patchset also proposes modifying RunQueue to then be able to run a > compatibility checker during task execution phase for recipes and tasks > that use > CDEPENDS and allow for deciding to re-execute a task despite being covered > by > sstate-cache. > > The patchset is based on the jethro branch for the poky repository, as > this is > the branch that we are using. If the general idea sounds good, we can > consider > porting it to master. > > Included is an patch that adds an example recipe and compatibility checker, > where compatibility is based on the file checksums of the parent recipes > packages. An example recipe, cdepends-test1, generates a compatibility > report > containing the file checksums of all files that it packages and which file > paths > they are at. Another recipe, cdepends-test2, can then strip this > compatibility > report to the minimal files it needs to be consistent and can compare the > latest > checksums it used to configure/compile/install with and if they have > changed, > trigger a rebuild. If not, the previous version restored from sstate-cache > is > used. > > We are still experimenting with the usages of this, including the use of > having > abi-compliance-checker to compare the ABI of shared libraries as a > compatibility > checker during RunQueue and using the results to avoid rebuilding child > recipes > when the .so files they depend on are still compatible. Example use cases > of > this are allowing recipes which provide multiple shared libraries to > change a > single .so file without rebuilding all its children that depend on the > other > shared libraries but not the one that changed. > > We're aware of the SIGGEN_EXCLUDERECIPES_ABISAFE feature but feel it > didn't meet > the feature requirements of what this compatibility checker callback is > doing, > although maybe when porting to master we could refactor to make better use > of > the work already done there. The current implementation is a bit hacky but > comments would be appreciated in regards to if the concept is feasible and > if > people are interested in making use of it and their use cases. > > Kind regards, > Michael Ho > > -- > BMW Car IT GmbH > Michael Ho > Spezialist Entwicklung - Linux Software Integration > Lise-Meitner-Str. 14 > 89081 Ulm > > Tel.: +49 731 3780 4071 > Mobil: +49 152 5498 0471 > Fax: +49-731-37804-001 > Mail: michael...@bmw-carit.de > Web: http://www.bmw-carit.de > -------------------------------------------------------- > BMW Car IT GmbH > Gechäftsführer: Kai-Uwe Balszuweit und Alexis Trolin > Sitz und Registergericht: München HRB 134810 > -------------------------------------------------------- > > -- > _______________________________________________ > yocto mailing list > yocto@yoctoproject.org > https://lists.yoctoproject.org/listinfo/yocto > >
-- _______________________________________________ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto