Andreas Enge <andr...@enge.fr> skribis: > In the end, I think it would be interesting to not use the offload approach, > where the offloading machine sends all inputs to the build machine, which > returns the result, so that more or less the complete set of binary packages > flows along the tree of build machines to the root and the other way round. > (As I recently understood during discussions with Ludovic, currently we have > a tree of height one: hydra is the root and all build machines are the leaves; > but the offloading mechanism could transparently work with trees of bigger > height, where hydra offloads to a root of a smaller build cluster, which > itself offloads to one of its subordinate build machines. And so on. The same > thing would also work in the Debian setting, assuming that build requests > come from the leaves and are forwarded towards the root, while a work package > is handed down the other way.) Now it would be interesting if machines just > got build jobs (or made requests) and looked for the necessary inputs inside > a big magma/cloud/gnunet/ipfs/mirror network, compiled the package, and > dropped the result into the same magma. Already in our current set-up, it > would be useful if hydra need not send the build inputs, but build machines > could just fetch them from an arbitrary nginx mirror, where they are cached.
I agree. However, I think there are a lot of unknowns, which makes it unsuitable for a GSoC project. That’s why I was suggesting that Mathieu do not consider workload distribution (meaning we would keep using the existing offload mechanism) as part of this project. Ludo’.