Willy,
On 3/8/22 16:24, Willy Tarreau wrote:
Hi Tim,
On Tue, Mar 08, 2022 at 04:17:00PM +0100, Tim Düsterhus wrote:
William
On 3/8/22 16:06, William Lallemand wrote:
Also, I'm wondering if we could also cache the build of HAProxy, you
could think that weird, but in fact it will help relaunch the tests when
one is failing, without rebuilding the whole thing.
Please don't. You always want to run a clean build, otherwise you're going
to get super-hard-to-debug failures if some object file is accidentally
taken from the cache instead of a clean build.
What risk are you having in mind for example ? I'm personally thinking
that vtest is sufficiently self-contained to represent almost zero risk.
We could possibly even pre-populate it and decide to rebuild if it's not
able to dump its help page.
This was a reply to "cache the build of HAProxy", so unrelated to VTest.
As the HAProxy build is what we want to test, it's important to always
perform a clean build.
I don't like it. As you say: It's ugly and introduces complexity for a mere
8 second gain. Sure, we should avoid burning the planet by wasting CPU
cycles in CI, but there's a point we're the additional complexity results in
a maintenance nightmare.
In fact it's not just the number of CPU cycles, it's also a matter of
interactivity. A long time ago I remember that we used to push and wait
for the build to complete. For a long time it has changed, we push and
switch to something else, and completely forget that we'd submitted a
build. For sure it's not vtest alone that will radically change this,
but my impression is that it could help, and near-zero risk. But that's
my perception and I could be mistaken.
I can only comment from my side: For me any patches are sent are
inherently asynchronously processed by the CI, because I don't know when
you are taking them. So I test them as good as I can locally and usually
the CI is green afterwards.
Looking at this:
https://github.com/haproxy/haproxy/actions/runs/1952139455, the fastest
build is the gcc no features build with 1:33. Even if we subtract the 8
seconds for VTest, then that's more than I'd be willing to synchronously
wait. The slowest are the QUICTLS builds with 6:21, because QUICTLS is
not currently cached.
FWIW: Even 6:21 is super fast compared to other projects. I've recently
contributed to PHP and CI results appear after 20 minutes or so.
In any case I believe that our goal should be that one does not need to
check the CI, because no bad commits make it into the repository.
Unfortunately we still have some flaky tests, that needlessly take up
attention, because one will need to check the red builds. If those are
gone, then I expect the vast majority of the commits to be green so that
it only catches mistakes in strange configurations that one cannot
easily test locally.
Of course if you think that the 8 seconds will improve your workflow,
then by all means: Commit it. From my POV in this case the cost is
larger than the benefit. Caching is one of the hard things in computer
science :-)
Best regards
Tim Düsterhus
