Willy,
On 3/8/22 20:11, Willy Tarreau wrote:
Yes my point was about VTest. However you made me think about a very good
reason for caching haproxy builds as well :-) Very commonly, some VTest
randomly fails. Timing etc are involved. And at the moment, it's impossible
to restart the tests without rebuilding everything. And it happens to me to
click "restart all jobs" sometimes up to 2-3 times in a row in order to end
I've looked up that roadmap entry I was thinking about: A "restart this
job" button apparently is planned for Q1 2022.
see https://github.com/github/roadmap/issues/271 "any individual job"
Caching the HAProxy binary really is something I strongly advice against
based on my experience with GitHub Actions and CI in general.
I think the restart of the individual job sufficiently solves the issue
of flaky builds (until they are fixed properly).
If those are gone,
then I expect the vast majority of the commits to be green so that it only
catches mistakes in strange configurations that one cannot easily test
locally.
This is already the case for the vast majority of them. I think that if we
eliminate the random failures on vtest, we're around 95-98% of successful
pushes. The remaining ones are caused by occasional download failures to
install dependencies and variables referenced outside an obscure ifdef
combination that only strikes on one platform.
Yeah, exactly that's what I wanted to say: If the CI is expected to be
green, then there should not be a need to manually check it for *every*
push and thus it doesn't matter as much of it takes 65 seconds or 80
seconds.
Of course if you think that the 8 seconds will improve your workflow, then
by all means: Commit it. From my POV in this case the cost is larger than
the benefit. Caching is one of the hard things in computer science :-)
[…]
extremely useful. Translating this back to haproxy builds, for me it
means: "if at any point there is any doubt that something that might
affect the build result might have changed, better rebuild". And that's
Yes, that's why I explicitly recommend against caching the HAProxy
binary. Cache invalidation really *is* hard for that without increasing
complexity of the vtest.yml which already is longer than I'm comfortable
with. At least with matrix.py it's manageable.
Now rest assured that I won't roll over the floor crying for this, but
I think that this is heading in the right direction. And with more CI
time available, we can even hope to test more combinations and squash
more bugs before they reach users. That's another aspect to think about.
I'm really happy to see that the duration between versions .0 and .1
increases from version to version, despite the fact that we add a lot
more code and more complex one. For now the CI scales well with the
code, I'm interested in continuing to help it scale even better.
40 minutes ago I've already sent a proposal that should make both you as
the developers and me as one of the CI experts happy :-) My concerns
were primarily with regard the number of additional steps in William's
proposal, not the caching of VTest per se.
Best regards
Tim Düsterhus
