On Fri, 2021-12-17 at 09:42 -0800, Rusty Howell wrote: > Related to this topic of setting up a cluster of build nodes: > https://lists.yoctoproject.org/g/yocto/topic/85515144 > > We have multiple build nodes right now configured to use a shared SSTATE cache > and shared PR server. We are building four MACHINE types. Build jobs are > randomly assigned, so any node can build the image for any MACHINE type. > In order to maintain a proper package feed for long term, we are backing up > the > PR server database regularly. > > I still have these questions though. > > * Do we need to also backup anything in BUILDHISTORY_DIR? > > * Do we need to share anything in BUILDHISTORY_DIR?
Buildhistory has an "auto commit" mode where it can commit each build into the history repository. On the autobuilder there is some support code which before starting a build, checks out a git repo and then at the end of building the appropriate targets, pushes it back. It understands master always moves forward and master-next is rebased. You can see some of it here: https://git.yoctoproject.org/yocto-autobuilder-helper/tree/scripts/utils.py#n200 configuration is in config.json in that repo. You don't have to do this, it just lets you track changes historically. I wish we made better use of that data. If ends up here: https://git.yoctoproject.org/poky-buildhistory/ > * I often see people recommend using SSTATE_MIRROR. What are the pros/cons to > using a SSTATE_MIRROR vs all nodes using a shared SSTATE_DIR? SSTATE_MIRROR works well as a remote you pull from but you usually can't write to them. SSTATE_DIR you can write to. > * Can/should DL_DIR be shared across build nodes? Yes and yes. We do on the autobuilder. > * Should the nodes use a remote/shared BBSERVER? No, BBSERVER is not useful in the context of a cluster. The "Server" in bitbake is the thing running the build and you want that where the work is being done. > * I have also seen someone mention a "hash equivalence server" that can also > accelerate builds. Is that an old term for the PR_server? No, it is a new service and you should look it up. If the output of a task is found to be the same, it allows build artefacts that were built previously to be reused instead of creating new ones. > * BB_SIGNATURE_HANDLER - I see there are some options to tweak in > BB_SIGNATIRE_WHITELIST. Are there common tweaks to these vars are generally > beneficial? I image the defaults are the best. I'd stick to the defaults. We'd change them if there were good things to do here, unless you want to make interesting experiments. Cheers, Richard
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#55617): https://lists.yoctoproject.org/g/yocto/message/55617 Mute This Topic: https://lists.yoctoproject.org/mt/87794307/21656 Group Owner: yocto+ow...@lists.yoctoproject.org Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
