Hi Richard, > FWIW I do want to see the system used to do things like this. I've kind > of envisaged you'd do it "up front" though. For example it should be > trivial for a script to use tinfoil, iterate through and generate an > include file of the revisions you want, using the fetcher calls. > > Yes, its slightly more annoying to generate the config, then build it > but if it stops the core becoming unmaintainable with corner cases, > that could be the right decision. > > If you do something like that it would be great to share it so others > can see what you did and be able to build off it...
This is actually something I've been working on and thinking about a lot for the past year and a half. Our application layer has probably 30 or so recipes that use AUTOREV. For the purposes of reproducible builds, we need to generate similar output to what buildhistory-collect-srcrevs generates, e.g. SRCREV_pn-my_daemon = "githash" I've gone back and forth and re-implemented the code to do so a half dozen times. Chronologically, these are the fundamental strategies I've attempted: 1. Simply in terms of buildhistory-collect-srcrevs. Pros: dead simple Cons: requires that you actually build stuff first; requires buildhistory to be enabled 2. As a bbclass, injecting a task between do_fetch and do_unpack ("do_capture_srcrevs") Pros: can be invoked without actually building stuff, e.g. bitbake my-image --runall=do_capture_srcrevs Cons: adds yet another task to the recipe; a little ugly since you either need to INHERIT the class, or individually inherit it in recipes 3. As a separate tinfoil script (as you suggest) which I run before the actual bitbake. Pros: decoupled from the actual build Cons: since it's decoupled, it's slower - each recipe needs to be parsed 4. Hacking the srcrev cache (inside fetch2) to log all information necessary to generate the SRCREV overrides, paired with a script that reads the cache and does it Pros: Equally suitable for the "config-then-build" workflow ("bitbake --parse-only && generate_overrides.py >> local.conf && bitbake image") or the "build-then-config" workflow ("bitbake image && generate_overrides.py > overrides.conf") Cons: Invasive, required large changes to fetch2/__init__.py and some smaller changes to fetch2/git.py In the current incarnation (#4), I also go a step further and, in cases where a SRC_URI has a parameterized branch, I generate BRANCH override lines as well. For example, if I see: MY_BRANCH ??= "master" SRC_URI = "git://git.oe.handhelds.org/git/vip.git;tag=version-1; branch=${MY_BRANCH}" I generate this line along with the corresponding SRCREV override: MY_BRANCH_pn-recipe = "master" The algorithm for detecting the branch variable in the SRC_URI entry is as a little complicated, and not super relevant at this moment, but I want to get it off my chest :). The code first tries to find a parameterized branch by using a regex to find stuff like "branch=${VAR}". If that fails, it could be because the branch is expressed as something more complicated like this: BRANCH_RELEASE_NUMBER ??= "1.0" SRC_URI = "git://git.oe.handhelds.org/git/vip.git;tag=version-1; branch=release-${BRANCH_RELEASE_NUMBER}" In that case, I fall back on a breadth-first search on all the variables that SRC_URI references (recursively) to find the "deepest" one that controls the branch. I wrote this in terms of d.expandWithRefs, and can detail it further if people are interested. Basically, I didn't want to enforce a particular convention on the name of the branch variable. You'd think BRANCH would cover it, but the kernel recipes also have KMETA and stuff like that. I realize the branch stuff as currently implemented is maybe too opinionated (and a bit dangerous) for inclusion in bitbake. But I digress. I'm very interested in discussing the SRCREV (and perhaps BRANCH) stuff further with anyone interested. I would love to see some strategy for doing this make its way into bitbake or poky, so I can stop reimplementing it and second guessing myself :). Thanks, Chris -- _______________________________________________ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core