Alan,
You asked for advice, so here I go. Might not be what you need/want but we will see. On Fri, 2018-08-31 at 12:49 +0200, Alan Martinovic wrote: > Hi, > am just in the process of forming a base for automated nightly builds. > The goal is to have the build process run every night and report what > the status was. > > I've opted for python for all the conditionals and checks OK, so you might already be planning to do what we're doing (conditionals and checks in script, but then calling that script from a full CI system)? Just to be sure that you're not also planning to do the scheduling of builds and bookkeeping of results in your script -- of course it's totally your choice, but my advise would be to take the time and set up a "real" CI server locally (choose any of them). Many can be found packaged in docker containers so there's little fuss in the setup. You said "nightly" but I'm just going to call it CI (continuous integration) from now on. I think you will be happy to have a nice front end, keeping a history of builds to go back to, get desktop/mobile notifications, dashboards, and so on. And once you start you'll probably want to add on more automation :-) Maybe this was your plan already! > and then have external bash scripts that actually sets the environment and > invokes bitbake. > > Did it like that because I didn't find a clean way so far to deal with > > source oe/oe-init-build-env > > from python. Already answered, but I agree - you must execute a bash script (which then in turn is free to "source" the oe-init script) > > Would like to hear how did you script bitbake and how does your setup look > like? Most of this might be way overkill for your needs but: init.sh [1], which wraps oe-init-build-env. The main purpose is to update all layers that we keep as submodules. Minor thing - it downloads only the required BSP layers for the selected MACHINE. That's also why it is tied into our call chain for bitbake, when the value of the MACHINE variable is known. init.sh handles also some details so users can avoid having to edit conf files. Such as, if required, the user is asked interactively to accept a EULA and if they do, the appropriate line is added to conf. It's just convenient to not have to know the exact conf-file syntax. Now, I know you asked for nightly-build setup, but encoding such manual editing steps is also needed for automation. Now we come to ci-build.sh [2] - another layer above init.sh, containing lots of examples of handling special cases. Again tweaking the conf files with lots of conditionals. That script is very complex, but on the other hand much of that complexity is now avoided in the CI instructions. We found it very useful to hide all the special cases in there. (You might prefer python for such a complex script). Another reason is that configuration that normally requires you to write things into the Yocto/OE .conf files is there transformed to variability through environment variables. This is significant, because it is normally very convenient to set environment variables in the CI system. Our ci-build script exposes some pretty advanced things to environment variables, like: "build the given version of the project, but for this particular layer I want to try this tag/commit/branch of the layer instead". And of course lots of Yocto/OE variation like generate licenses, source code packages, location of downloads/sstate cache, any project-unique steps you need to take before calling bitbake. Another general preference (but this might be personal thing for me) is: whenever you think "The user (or myself) just needs to add some flag to this file (local.conf)", then instead think: How can I have such a variant of the file already prepared?. Granted, we let our scripts modify the conf-files as I noted, but we also try to keep fully static and prepared files as much as possible under version control. And then select the file somehow. It removes the modification steps, so it is less brittle, and it's another method to avoid the user having to know lots of detailed syntax. Using hierarchical "include" statements can help organize such variability - see [3] Probably way more than you asked for, but hope some of it helps. Sincerely, - Gunnar [1] https://github.com/GENIVI/genivi-dev-platform/blob/master/init.sh [2] https://github.com/GENIVI/genivi-dev-platform/blob/master/scripts/ci-build.sh [3] https://github.com/GENIVI/genivi-dev-platform/tree/master/gdp-src-build/conf/templates -- Gunnar Andersson <ganders...@genivi.org> Development Lead GENIVI Alliance -- _______________________________________________ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto