I would be curious how old that language is in the ASF policy. In the era of cloud infrastructure and public CI, "your own hardware" might mean something different than it once did. IMO, it is essential that we download and test the release artifacts in a separate context from where they are built in order to verify that they work as intended, which seems like the spirit of the policy. But if I can kick off a CI job that does that across a matrix of platforms and environments, that seems more meaningful than whether it works on a particular machine in my physical possession.
Neal On Tue, Aug 22, 2023 at 11:18 AM Antoine Pitrou <anto...@python.org> wrote: > > And of course this is a bit pedantic, and only important if we want to > comply with *the letter* of the ASF policies. My own personal opinion is > that complying in spirit is enough (but I'm also not sure I understand > the ASF's spirit :-)). > > Regards > > Antoine. > > > Le 22/08/2023 à 17:10, Antoine Pitrou a écrit : > > > > Hmm... perhaps Flatbuffers compilation is usually more deterministic > > than compiling C++ code into machine code, but that's mainly (AFAIK) > > because the transformation step is much simpler in the former case, than > > in the latter. The Flatbuffers compiler also has a range of options that > > influence code generation, certainly with less variation than a C++ > > compiler, but still. > > > > In other words, I don't think being deterministic is a good criterion to > > know what "compiled code" means. There is a growing movement towards > > making generation of machine code artifacts deterministic: > > https://reproducible-builds.org/ > > > > Regards > > > > Antoine. > > > > > > > > Le 22/08/2023 à 16:47, Adam Lippai a écrit : > >> Compiled code usually means binaries you can’t derive in a > deterministic, > >> verifiable way from the source code *shipped next to it*. So in this > case > >> any developer should be able to reproduce the flatbuffers output from > the > >> release package only. > >> > >> “Caches”, multi stage compilation etc should be ok. > >> > >> Best regards, > >> Adam Lippai > >> > >> On Tue, Aug 22, 2023 at 10:40 Antoine Pitrou <anto...@python.org> > wrote: > >> > >>> > >>> If the main impetus for the verification script is to comply with ASF > >>> requirements, probably the script can be made much simpler, such as > just > >>> verify the GPG signatures are valid? Or perhaps this can be achieved > >>> without a script at all. > >>> > >>> The irony is that, however complex, our verification script doesn't > seem > >>> to check the actual ASF requirements on artifacts. > >>> > >>> For example, we don't check that """a source release SHOULD not contain > >>> compiled code""" (also, what does "compiled code" mean? does generated > >>> code, e.g. by the Flatbuffers compiler, apply?) > >>> > >>> Checking that the release """MUST be sufficient for a user to build and > >>> test the release provided they have access to the appropriate platform > >>> and tools""" is ill-defined and potentially tautologic, because the > >>> "appropriate platform and tools" is too imprecise and contextual (can > >>> the "appropriate platform and tools" contain a bunch of proprietary > >>> software that gets linked with the binaries? Well, it can, otherwise > you > >>> can't build on Windows). > >>> > >>> Regards > >>> > >>> Antoine. > >>> > >>> > >>> > >>> Le 22/08/2023 à 12:31, Raúl Cumplido a écrit : > >>>> Hi, > >>>> > >>>> I do agree that currently verifying the release locally provides > >>>> little benefit for the effort we have to put in but I thought this was > >>>> required as per Apache policy: > >>>> https://www.apache.org/legal/release-policy.html#release-approval > >>>> > >>>> Copying the important bit: > >>>> """ > >>>> Before casting +1 binding votes, individuals are REQUIRED to download > >>>> all signed source code packages onto their own hardware, verify that > >>>> they meet all requirements of ASF policy on releases as described > >>>> below, validate all cryptographic signatures, compile as provided, and > >>>> test the result on their own platform. > >>>> """ > >>>> > >>>> I also think we should try and challenge those. > >>>> > >>>> In the past we have identified some minor issues on the local > >>>> verification but I don't recall any of them being blockers for the > >>>> release. > >>>> > >>>> Thanks, > >>>> Raúl > >>>> > >>>> El mar, 22 ago 2023 a las 11:46, Andrew Lamb (<al...@influxdata.com>) > >>> escribió: > >>>>> > >>>>> The Rust arrow implementation (arrow-rs) and DataFusion also use > release > >>>>> verification scripts, mostly inherited from when they were split from > >>> the > >>>>> mono repo. They have found issues from time to time, for us, but > those > >>>>> issues are often not platform related and have not been release > >>> blockers. > >>>>> > >>>>> Thankfully for Rust, the verification scripts don't need much > >>> maintenance > >>>>> so we just continue the ceremony. However, I certainly don't think we > >>> would > >>>>> lose much/any test coverage if we stopped their use. > >>>>> > >>>>> Andrew > >>>>> > >>>>> On Tue, Aug 22, 2023 at 4:54 AM Antoine Pitrou <anto...@python.org> > >>> wrote: > >>>>> > >>>>>> > >>>>>> Hello, > >>>>>> > >>>>>> Abiding by the Apache Software Foundation's guidelines, every Arrow > >>>>>> release is voted on and requires at least 3 "binding" votes to be > >>> approved. > >>>>>> > >>>>>> Also, every Arrow release vote is accompanied by a little ceremonial > >>>>>> where contributors and core developers run a release verification > >>> script > >>>>>> on their machine, wait for long minutes (sometimes an hour) and > report > >>>>>> the results. > >>>>>> > >>>>>> This ceremonial has gone on for years, and it has not really been > >>>>>> questioned. Yet, it's not obvious to me what it is achieving > exactly. > >>>>>> I've been here since 2018, but I don't really understand what the > >>>>>> verification script is testing for, or, more importantly, *why* it > is > >>>>>> testing for what it is testing. I'm probably not the only one? > >>>>>> > >>>>>> I would like to bring the following points: > >>>>>> > >>>>>> * platform compatibility is (supposed to be) exercised on Continuous > >>>>>> Integration; there is no understandable reason why it should be > >>>>>> ceremoniously tested on each developer's machine before the release > >>>>>> > >>>>>> * just before a release is probably the wrong time to be testing > >>>>>> platform compatibility, and fixing compatibility bugs (though, of > >>>>>> course, it might still be better than not noticing?) > >>>>>> > >>>>>> * home environments are unstable, and not all developers run the > >>>>>> verification script for each release, so each release is actually > >>>>>> verified on different, uncontrolled, platforms > >>>>>> > >>>>>> * as for sanity checks on binary packages, GPG signatures, etc., > there > >>>>>> shouldn't be any need to run them on multiple different machines, as > >>>>>> they are (should be?) entirely deterministic and platform-agnostic > >>>>>> > >>>>>> * maintaining the verification scripts is a thankless task, in part > due > >>>>>> to their nature (they need to track and mirror changes made in each > >>>>>> implementation's build chain), in part due to implementation choices > >>>>>> > >>>>>> * due to the existence of the verification scripts, the release > vote is > >>>>>> focussed on getting the script to run successfully (a very > contextual > >>>>>> and non-reproducible result), rather than the actual *contents* of > the > >>>>>> release > >>>>>> > >>>>>> The most positive thing I can personally say about the verification > >>>>>> scripts is that they *may* help us trust the release is not broken? > But > >>>>>> that's a very unqualified statement, and is very close to > >>> cargo-culting. > >>>>>> > >>>>>> Regards > >>>>>> > >>>>>> Antoine. > >>>>>> > >>> > >> >
