If the main impetus for the verification script is to comply with ASF
requirements, probably the script can be made much simpler, such as just
verify the GPG signatures are valid? Or perhaps this can be achieved
without a script at all.
The irony is that, however complex, our verification script doesn't seem
to check the actual ASF requirements on artifacts.
For example, we don't check that """a source release SHOULD not contain
compiled code""" (also, what does "compiled code" mean? does generated
code, e.g. by the Flatbuffers compiler, apply?)
Checking that the release """MUST be sufficient for a user to build and
test the release provided they have access to the appropriate platform
and tools""" is ill-defined and potentially tautologic, because the
"appropriate platform and tools" is too imprecise and contextual (can
the "appropriate platform and tools" contain a bunch of proprietary
software that gets linked with the binaries? Well, it can, otherwise you
can't build on Windows).
Regards
Antoine.
Le 22/08/2023 à 12:31, Raúl Cumplido a écrit :
Hi,
I do agree that currently verifying the release locally provides
little benefit for the effort we have to put in but I thought this was
required as per Apache policy:
https://www.apache.org/legal/release-policy.html#release-approval
Copying the important bit:
"""
Before casting +1 binding votes, individuals are REQUIRED to download
all signed source code packages onto their own hardware, verify that
they meet all requirements of ASF policy on releases as described
below, validate all cryptographic signatures, compile as provided, and
test the result on their own platform.
"""
I also think we should try and challenge those.
In the past we have identified some minor issues on the local
verification but I don't recall any of them being blockers for the
release.
Thanks,
Raúl
El mar, 22 ago 2023 a las 11:46, Andrew Lamb (<al...@influxdata.com>) escribió:
The Rust arrow implementation (arrow-rs) and DataFusion also use release
verification scripts, mostly inherited from when they were split from the
mono repo. They have found issues from time to time, for us, but those
issues are often not platform related and have not been release blockers.
Thankfully for Rust, the verification scripts don't need much maintenance
so we just continue the ceremony. However, I certainly don't think we would
lose much/any test coverage if we stopped their use.
Andrew
On Tue, Aug 22, 2023 at 4:54 AM Antoine Pitrou <anto...@python.org> wrote:
Hello,
Abiding by the Apache Software Foundation's guidelines, every Arrow
release is voted on and requires at least 3 "binding" votes to be approved.
Also, every Arrow release vote is accompanied by a little ceremonial
where contributors and core developers run a release verification script
on their machine, wait for long minutes (sometimes an hour) and report
the results.
This ceremonial has gone on for years, and it has not really been
questioned. Yet, it's not obvious to me what it is achieving exactly.
I've been here since 2018, but I don't really understand what the
verification script is testing for, or, more importantly, *why* it is
testing for what it is testing. I'm probably not the only one?
I would like to bring the following points:
* platform compatibility is (supposed to be) exercised on Continuous
Integration; there is no understandable reason why it should be
ceremoniously tested on each developer's machine before the release
* just before a release is probably the wrong time to be testing
platform compatibility, and fixing compatibility bugs (though, of
course, it might still be better than not noticing?)
* home environments are unstable, and not all developers run the
verification script for each release, so each release is actually
verified on different, uncontrolled, platforms
* as for sanity checks on binary packages, GPG signatures, etc., there
shouldn't be any need to run them on multiple different machines, as
they are (should be?) entirely deterministic and platform-agnostic
* maintaining the verification scripts is a thankless task, in part due
to their nature (they need to track and mirror changes made in each
implementation's build chain), in part due to implementation choices
* due to the existence of the verification scripts, the release vote is
focussed on getting the script to run successfully (a very contextual
and non-reproducible result), rather than the actual *contents* of the
release
The most positive thing I can personally say about the verification
scripts is that they *may* help us trust the release is not broken? But
that's a very unqualified statement, and is very close to cargo-culting.
Regards
Antoine.
