Hello,
Abiding by the Apache Software Foundation's guidelines, every Arrow
release is voted on and requires at least 3 "binding" votes to be approved.
Also, every Arrow release vote is accompanied by a little ceremonial
where contributors and core developers run a release verification script
on their machine, wait for long minutes (sometimes an hour) and report
the results.
This ceremonial has gone on for years, and it has not really been
questioned. Yet, it's not obvious to me what it is achieving exactly.
I've been here since 2018, but I don't really understand what the
verification script is testing for, or, more importantly, *why* it is
testing for what it is testing. I'm probably not the only one?
I would like to bring the following points:
* platform compatibility is (supposed to be) exercised on Continuous
Integration; there is no understandable reason why it should be
ceremoniously tested on each developer's machine before the release
* just before a release is probably the wrong time to be testing
platform compatibility, and fixing compatibility bugs (though, of
course, it might still be better than not noticing?)
* home environments are unstable, and not all developers run the
verification script for each release, so each release is actually
verified on different, uncontrolled, platforms
* as for sanity checks on binary packages, GPG signatures, etc., there
shouldn't be any need to run them on multiple different machines, as
they are (should be?) entirely deterministic and platform-agnostic
* maintaining the verification scripts is a thankless task, in part due
to their nature (they need to track and mirror changes made in each
implementation's build chain), in part due to implementation choices
* due to the existence of the verification scripts, the release vote is
focussed on getting the script to run successfully (a very contextual
and non-reproducible result), rather than the actual *contents* of the
release
The most positive thing I can personally say about the verification
scripts is that they *may* help us trust the release is not broken? But
that's a very unqualified statement, and is very close to cargo-culting.
Regards
Antoine.
