I agree that this is a great time to look at improving the verification process. One solution I've seen work fairly well is to convert large bash scripts into a lightweight ETL pipeline that caches the results/status of each node as it executes. That way, restarting a pipeline at the right checkpoint is trivial. Existing open source ETL platforms should be able to do this, although I don't know which would be best. Something like Apache Airflow would probably be overkill IMO.
On Fri, Jan 19, 2024 at 5:44 AM Raúl Cumplido <rau...@apache.org> wrote: > Hi, > > One of the challenges we have when doing a release is verification and > voting. > > Currently the Arrow verification process is quite long, tedious and error > prone. > > I would like to use this email to get feedback and user requests in > order to improve the process. > > Several things already on my mind: > > One thing that is quite annoying is that any flaky failure makes us > restart the process and possibly requires downloading everything > again. It would be great to have some kind of retry mechanism that > allows us to keep going from where it failed and doesn't have to redo > the previous successful jobs. > > We do have a bunch of flags to do specific parts but that requires > knowledge and time to go over the different flags, etcetera so the UX > could be improved. > > Based on the ASF release policy [1] in order to cast a +1 vote we have > to validate the source code packages but it is not required to > validate binaries locally. Several binaries are currently tested using > docker images and they are already tested and validated on CI. Our > documentation for release verification points to perform binary > validation. I plan to update the documentation and move it to the > official docs instead of the wiki [2]. > > I would appreciate input on the topic so we can improve the current > process. > > Thanks everyone, > Raúl > > [1] https://www.apache.org/legal/release-policy.html#release-approval > [2] > https://cwiki.apache.org/confluence/display/ARROW/How+to+Verify+Release+Candidates >
