Re: Improving CI throughput

2020-08-28 Thread Ludovic Courtès
Hello! Mathieu Othacehe skribis: >> Yeah, this is a ridiculous situation. We should do a hackathon to get >> better monitoring of useful metrics (machine load, >> time-of-push-to-time-to-build-completion, etc.), to clearly identify the >> bottlenecks (crashes? inefficient protocol? scheduling i

Re: Improving CI throughput

2020-08-25 Thread Ricardo Wurmus
Mathieu Othacehe writes: > As most of the issues are only observed on Berlin machines, which access is > restricted, we will also have to find a way to reproduce them locally. You can access all Berlin build nodes from the head node at ci.guix.gnu.org, either as “root” or “hydra” (both with ro

Re: Improving CI throughput

2020-08-25 Thread Mathieu Othacehe
Hey, > Yeah, this is a ridiculous situation. We should do a hackathon to get > better monitoring of useful metrics (machine load, > time-of-push-to-time-to-build-completion, etc.), to clearly identify the > bottlenecks (crashes? inefficient protocol? scheduling issues? Cuirass > or offload or g

Re: Improving CI throughput

2020-08-24 Thread John Soo
Hi Ludo and Guix, I am not sure how much I can devote to the problem, but bpf now works in guix. The bpftrace scripting language is there if it might help. Hope that helps a little, John

Improving CI throughput

2020-08-24 Thread Ludovic Courtès
Hi, Mathieu Othacehe skribis: > The current situation is that due to Cuirass/offloading issues such as > [1], our build farm is most of the time idle. Given our computation > power, we should be able to bake much more substitutes I think. > > Maybe we could also take advantage of the build-coord