On Fri, Apr 16, 2021 at 1:11 AM Jed Brown <j...@jedbrown.org> wrote: > > Wes McKinney <wesmck...@gmail.com> writes: > > > I think we should take a more serious look at Buildkite for some of our CI. > > > > * First of all, it's very easy to connect self-hosted workers and > > supports ephemeral cloud workers in a way that would be difficult or > > impossible with GHA. No need to have Infra fiddle with the admin > > dashboard. So we could spin up extra workers during peak hours, or use > > autoscaling to respond to demand. > > > > * We can set up more complex / dependent job pipelines rather than the > > current GHA monolithic "long list of independent jobs" setup. For > > example, we could have a fast gatekeeper job for C++ builds (which > > lints and makes sure that everything compiles) that must pass before > > more exhaustive longer-running jobs run. > > I don't have experience with Buildkite, but note that gitlab-runner is also > lightweight and well-featured as above. Here's an example with gatekeeping > stages across about 60 environments (mostly on-prem at multiple sites), > including explicit "pause-for-approval" to avoid unnecessary time-consuming > jobs. > > https://gitlab.com/petsc/petsc/-/pipelines/286655535 > > We also use it for on-prem GPU-equipped CI with repositories hosted on > GitHub, reporting status to PRs. The Kubernetes and docker-machine executors > are intended for autoscaling. > > https://docs.gitlab.com/runner/executors/README.html
The CI technology/service we choose is just one piece of the puzzle. We need to figure out a sustainable way of funding for the agents/runners. Sadly we don't have many CIs with free offerings for OSS left to try (and allowed by INFRA).
