I think one thing that would help would be improving the reproducibility of the source release process. The RM has to have their machine configured in a particular way for it to work.
Before anyone says "Docker" it isn't an easy solution because the release scripts need to be able to create git commits (created by the Maven release plugin) and sign artifacts using the RM's GPG keys. On Sat, Jul 27, 2019 at 10:04 PM Micah Kornfield <emkornfi...@gmail.com> wrote: > > I just wanted to bump this thread. Kou and Krisztián as the last two > release managers is there any specific infrastructure that you think might > have helped? > > Thanks, > Micah > > On Wed, Jul 17, 2019 at 11:29 PM Micah Kornfield <emkornfi...@gmail.com> > wrote: > > > I'd can help as well, but not exactly sure where to start. It seems like > > there are already some JIRAs opened [1] > > for improving the release? Could someone more familiar with the process > > pick out the highest priority ones? Do more need to be opened? > > > > Thanks, > > Micah > > > > [1] > > https://issues.apache.org/jira/browse/ARROW-2880?jql=project%20%3D%20ARROW%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%20AND%20component%20in%20(%22Developer%20Tools%22%2C%20Packaging)%20and%20summary%20~%20Release > > > > On Sat, Jul 13, 2019 at 7:17 AM Wes McKinney <wesmck...@gmail.com> wrote: > > > >> To be effective at improving the life of release managers, the nightly > >> release process really should use as close as possible to the same > >> scripts that the RM uses to produce the release. Otherwise we could > >> have a situation where the nightlies succeed but there is some problem > >> that either fails an RC or is unable to be produced at all. > >> > >> On Sat, Jul 13, 2019 at 9:12 AM Andy Grove <andygrov...@gmail.com> wrote: > >> > > >> > I would like to volunteer to help with Java and Rust release process > >> work, > >> > especially nightly releases. > >> > > >> > Although I'm not that familiar with the Java implementation of Arrow, I > >> > have been using Java and Maven for a very long time. > >> > > >> > Do we envisage a single nightly release process that releases all > >> languages > >> > simultaneously? or do we want separate process per language, with > >> different > >> > maintainers? > >> > > >> > > >> > > >> > On Wed, Jul 10, 2019 at 8:18 AM Wes McKinney <wesmck...@gmail.com> > >> wrote: > >> > > >> > > On Sun, Jul 7, 2019 at 7:40 PM Sutou Kouhei <k...@clear-code.com> > >> wrote: > >> > > > > >> > > > Hi, > >> > > > > >> > > > > in future releases we should > >> > > > > institute a minimum 24-hour "quiet period" after any community > >> > > > > feedback on a release candidate to allow issues to be examined > >> > > > > further. > >> > > > > >> > > > I agree with this. I'll do so when I do a release manager in > >> > > > the future. > >> > > > > >> > > > > To be able to release more often, two things have to happen: > >> > > > > > >> > > > > * More PMC members must engage with the release management role, > >> > > > > process, and tools > >> > > > > * Continued improvements to release tooling to make the process > >> less > >> > > > > painful for the release manager. For example, it seems we may > >> want to > >> > > > > find a different place than Bintray to host binary artifacts > >> > > > > temporarily during release votes > >> > > > > >> > > > My opinion that we need to build nightly release system. > >> > > > > >> > > > It uses dev/release/NN-*.sh to build .tar.gz and binary > >> > > > artifacts from the .tar.gz. > >> > > > It also uses dev/release/verify-release-candidate.* to > >> > > > verify build .tar.gz and binary artifacts. > >> > > > It also uses dev/release/post-NN-*.sh to do post release > >> > > > tasks. (Some tasks such as uploading a package to packaging > >> > > > system will be dry-run.) > >> > > > > >> > > > >> > > I agree that having a turn-key release system that's capable of > >> > > producing nightly packages is the way to do. That way any problems > >> > > that would block a release will come up as they happen rather than > >> > > piling up until the very end like they are now. > >> > > > >> > > > I needed 10 or more changes for dev/release/ to create > >> > > > 0.14.0 RC0. (Some of them are still in my local stashes. I > >> > > > don't have time to create pull requests for them > >> > > > yet. Because I postponed some tasks of my main > >> > > > business. I'll create pull requests after I finished the > >> > > > postponed tasks of my main business.) > >> > > > > >> > > > >> > > Thanks. I'll follow up on the 0.14.1/0.15.0 thread -- since we need to > >> > > release again soon because of problems with 0.14.0 please let us know > >> > > what patches will be needed to make another release. > >> > > > >> > > > If we fix problems related to dev/release/ in our normal > >> > > > development process, release process will be less painful. > >> > > > > >> > > > The biggest problem for 0.14.0 RC0 is java/pom.xml related: > >> > > > https://github.com/apache/arrow/pull/4717 > >> > > > > >> > > > It was difficult for me because I don't have Java > >> > > > knowledge. Release manager needs help from many developers > >> > > > because release manager may not have knowledge of all > >> > > > supported languages. Apache Arrow supports 10 over > >> > > > languages. > >> > > > > >> > > > > >> > > > For Bintray API limit problem, we'll be able to resolve it. > >> > > > I was added to https://bintray.com/apache/ members: > >> > > > > >> > > > https://issues.apache.org/jira/browse/INFRA-18698 > >> > > > > >> > > > I'll be able to use Bintray API without limitation in the > >> > > > future. Release managers should also request the same thing. > >> > > > > >> > > > >> > > This is good, I will add myself. Other PMC members should also add > >> > > themselves. > >> > > > >> > > > > >> > > > Thanks, > >> > > > -- > >> > > > kou > >> > > > > >> > > > In <CAJPUwMBRzYQ=hbVwFuPYAB-O= > >> lsowxqxidjapc_cofguksj...@mail.gmail.com> > >> > > > "[DISCUSS] Release cadence and release vote conventions" on Sat, > >> 6 Jul > >> > > 2019 16:28:50 -0500, > >> > > > Wes McKinney <wesmck...@gmail.com> wrote: > >> > > > > >> > > > > hi folks, > >> > > > > > >> > > > > As a reminder, particularly since we have many new community > >> members > >> > > > > (some of whom have never been involved with an ASF project > >> before), > >> > > > > releases are approved exclusively by the PMC and in general > >> releases > >> > > > > cannot be vetoed. In spite of that, we strive to make releases > >> that > >> > > > > have unanimous (either by explicit +1 or lazy consent) support of > >> the > >> > > > > PMC. So it is better to have unanimous 5 +1 votes than 6 +1 votes > >> with > >> > > > > a -1 dissenting vote. > >> > > > > > >> > > > > On the 0.14.0 vote, as with previous release votes, some issues > >> with > >> > > > > the release were raised by members of the community, whether > >> build or > >> > > > > test-related problems or other failures. Technically speaking, > >> such > >> > > > > issues have no _direct_ bearing on whether a release vote passes, > >> only > >> > > > > on whether PMC members vote +1, 0, or -1. A PMC member is allowed > >> to > >> > > > > change their vote based on new information -- for example, if I > >> voted > >> > > > > +1 on a release and then someone reported a serious licensing > >> issue, > >> > > > > then I would revise my vote to -1. > >> > > > > > >> > > > > On the RC0 vote thread, Jacques wrote [1] > >> > > > > > >> > > > > "A release vote should last until we arrive at consensus. When an > >> > > > > issue is potentially identified, those that have voted should be > >> given > >> > > > > ample time to change their vote and others that may have been lazy > >> > > > > consenters should be given time to chime in. There is no maximum > >> > > > > amount of time a vote can be open. Allowing at least 24 hours > >> after an > >> > > > > objection is raised is a pretty minimum expectation unless the > >> > > > > objector removes their objection. > >> > > > > > >> > > > > Note that Apache is more focused on consensus than timing (as > >> opposed > >> > > to > >> > > > > virtually other other organizations in the world)." > >> > > > > > >> > > > > I agree with this and my opinion is that in future releases we > >> should > >> > > > > institute a minimum 24-hour "quiet period" after any community > >> > > > > feedback on a release candidate to allow issues to be examined > >> > > > > further. If someone finds a potential problem, and no negative > >> votes > >> > > > > are cast or changed, then the vote can close. > >> > > > > > >> > > > > As a related matter, it seems clear to me that Apache Arrow should > >> > > > > have more frequent releases. I think this would decrease pressure > >> on > >> > > > > developers and users alike. While we've made strides to improve > >> the > >> > > > > tooling for release management (big thanks to Kou, Yosuke, > >> Krisztian, > >> > > > > and others), there is still quite some labor involved and > >> potential > >> > > > > for issues (e.g. API rate limiting for binary artifacts on > >> Bintray). > >> > > > > > >> > > > > To be able to release more often, two things have to happen: > >> > > > > > >> > > > > * More PMC members must engage with the release management role, > >> > > > > process, and tools > >> > > > > * Continued improvements to release tooling to make the process > >> less > >> > > > > painful for the release manager. For example, it seems we may > >> want to > >> > > > > find a different place than Bintray to host binary artifacts > >> > > > > temporarily during release votes > >> > > > > > >> > > > > Any other ideas for things we can do to improve the process and > >> > > > > cadence of releases? > >> > > > > > >> > > > > Thanks, > >> > > > > Wes > >> > > > > > >> > > > > [1]: > >> > > > >> https://lists.apache.org/thread.html/be6210e97b838494a5516dad6408f479efe4c98aff805000597c0196@%3Cdev.arrow.apache.org%3E > >> > > > >> > >
