Re: [DISCUSS] Release cadence and release vote conventions

2019-08-29 Thread Micah Kornfield
 > and use it but I don't like it. Because arrow-site increases
> > > a commit day by day.
> > > Can we prepare a Web site for this? (arrow-nightly.ursalabs.org?)
> > >
> > > dev/release/post-04-rubygems.sh:
> > >
> > > We may be able to use GitHub Package Registry[2] to upload
> > > RubyGems. We can use "pre-release" package feature of
> > > https://rubygems.org/ but it's not suitable for
> > > nightly. It's for RC or beta release.
> > >
> > > [2]
> https://github.blog/2019-05-10-introducing-github-package-registry/
> > >
> > > dev/release/post-05-js.sh:
> > >
> > > We may be able to use GitHub Package Registry[2] to upload
> > > npm packages.
> > >
> > > dev/release/post-06-csharp.sh:
> > >
> > > We may be able to use GitHub Package Registry[2] to upload
> > > NuGet packages.
> > >
> > > dev/release/post-07-rust.sh:
> > >
> > > I don't have any idea. But it must be ran
> > > automatically. It's always failed. I needed to run each
> > > command manually.
> > >
> > > dev/release/post-08-remove-rc.sh:
> > >
> > > We'll be able to skip this.
> > >
> > >
> > > Thanks,
> > > --
> > > kou
> > >
> > > In  >
> > >   "Re: [DISCUSS] Release cadence and release vote conventions" on Wed,
> 31
> > > Jul 2019 15:35:57 -0500,
> > >   Wes McKinney  wrote:
> > >
> > > > The PMC member and their GPG keys need to be in the loop at some
> > > > point. The release artifacts can be produced by some kind of CI/CD
> > > > system so long as the PMC member has confidence in the security of
> > > > those artifacts before signing them. For example, we build the
> > > > official binary packages on public CI services and then download and
> > > > sign them with Crossbow. I think the same could be done in theory
> with
> > > > the source release but we'd first need to figure out what to do about
> > > > the parts that create git commits.
> > > >
> > > > On Wed, Jul 31, 2019 at 11:23 AM Andy Grove 
> > > wrote:
> > > >>
> > > >> To what extent would it be possible to automate the release process
> via
> > > >> CICD?
> > > >>
> > > >> On Wed, Jul 31, 2019 at 9:19 AM Wes McKinney 
> > > wrote:
> > > >>
> > > >> > I think one thing that would help would be improving the
> > > >> > reproducibility of the source release process. The RM has to have
> > > >> > their machine configured in a particular way for it to work.
> > > >> >
> > > >> > Before anyone says "Docker" it isn't an easy solution because the
> > > >> > release scripts need to be able to create git commits (created by
> the
> > > >> > Maven release plugin) and sign artifacts using the RM's GPG keys.
> > > >> >
> > > >> > On Sat, Jul 27, 2019 at 10:04 PM Micah Kornfield <
> > > emkornfi...@gmail.com>
> > > >> > wrote:
> > > >> > >
> > > >> > > I just wanted to bump this thread.  Kou and KrisztiƔn as the
> last
> > > two
> > > >> > > release managers is there any specific infrastructure that you
> think
> > > >> > might
> > > >> > > have helped?
> > > >> > >
> > > >> > > Thanks,
> > > >> > > Micah
> > > >> > >
> > > >> > > On Wed, Jul 17, 2019 at 11:29 PM Micah Kornfield <
> > > emkornfi...@gmail.com>
> > > >> > > wrote:
> > > >> > >
> > > >> > > > I'd can help as well, but not exactly sure where to start.  It
> > > seems
> > > >> > like
> > > >> > > > there are already some JIRAs opened [1]
> > > >> > > > for improving the release?  Could someone more familiar with
> the
> > > >> > process
> > > >> > > > pick out the highest priority ones? Do more need to be opened?
> > > >> > > >
> > > >> > > > Thanks,
> > > >> > > > Micah
> > > >> > > >
> > > >> > > > [1]
> > > >> > > >
> > >

Re: [DISCUSS] Release cadence and release vote conventions

2019-08-01 Thread Wes McKinney
I agree. In my experiences as RM I have found the involvement of Maven
in the release process to be a nuisance. I think it makes more sense
in Java-only projects

On Thu, Aug 1, 2019 at 2:54 PM Andy Grove  wrote:
>
> I'll start taking a look at the maven issue. We might not want to use maven
> release plugin given that we control the version number already across this
> repository via other means.
>
> On Wed, Jul 31, 2019 at 4:26 PM Sutou Kouhei  wrote:
>
> > Hi,
> >
> > Sorry for not replying this thread.
> >
> > I think that the biggest problem is related to our Java
> > package.
> >
> >
> > We'll be able to resolve the GPG key problem by creating a
> > GPG key only for nightly release test. We can share the test
> > GPG key publicly because it's a just for testing.
> >
> > It'll work for our binary artifacts and APT/Yum repositories
> > but not work for our Java package. I don't know where GPG
> > key is used in our Java package...
> >
> >
> > We'll be able to resolve the Git commit problem by creating
> > a cloned Git repository for test. It's done in our
> > dev/release/00-prepare-test.rb[1].
> >
> > [1]
> > https://github.com/apache/arrow/blob/master/dev/release/00-prepare-test.rb#L30
> >
> > The biggest problem for the Git commit is our Java package
> > requires "apache-arrow-${VERSION}" tag on
> > https://github.com/apache/arrow . (Right?)
> > I think that "mvm release:perform" in
> > dev/release/01-perform.sh does so but I don't know the
> > details of "mvm release:perform"...
> >
> >
> > More details:
> >
> > dev/release/00-prepare.sh:
> >
> > We'll be able to run this automatically when we can resolve
> > the above GPG key problem in our Java package. We can
> > resolve the Git commit problem by creating a cloned Git
> > repository.
> >
> > dev/release/01-prepare.sh:
> >
> > We'll be able to run this automatically when we can resolve
> > the above Git commit ("apche-arrow-${VERSION}" tag) problem
> > in our Java package.
> >
> > dev/release/02-source.sh:
> >
> > We'll be able to run this automatically by creating a GPG
> > key for nightly release test. We'll use Bintray to upload RC
> > source archive instead of dist.apache.org. Ah, we need a
> > Bintray API key for this. It must be secret.
> >
> > dev/release/03-binary.sh:
> >
> > We'll be able to run this automatically by creating a GPG
> > key for nightly release test. We need a Bintray API key too.
> >
> > We need to improve this to support nightly release test. It
> > use "XXX-rc" such as "debian-rc" for Bintray "package" name.
> > It should use "XXX-nightly" such as "debian-nightly" for
> > nightly release test instead.
> >
> > dev/release/post-00-release.sh:
> >
> > We'll be able to skip this.
> >
> > dev/release/post-01-upload.sh:
> >
> > We'll be able to skip this.
> >
> > dev/release/post-02-binary.sh:
> >
> > We'll be able to run this automatically by creating Bintray
> > "packages" for nightly release and use them. We can create
> > "XXX-nightly-release" ("debian-nightly-release") Bintray
> > "packages" and use them instead of "XXX" ("debian") Bintray
> > "packages".
> >
> > "debian" Bintray "package": https://bintray.com/apache/debian/
> >
> > We need to improve this to support nightly release.
> >
> > dev/release/post-03-website.sh:
> >
> > We'll be able to run this automatically by creating a cloned
> > Git repository for test.
> >
> > It's better that we have a Web site to show generated pages.
> > We can create
> > https://github.com/apache/arrow-site/tree/asf-site/nightly
> > and use it but I don't like it. Because arrow-site increases
> > a commit day by day.
> > Can we prepare a Web site for this? (arrow-nightly.ursalabs.org?)
> >
> > dev/release/post-04-rubygems.sh:
> >
> > We may be able to use GitHub Package Registry[2] to upload
> > RubyGems. We can use "pre-release" package feature of
> > https://rubygems.org/ but it's not suitable for
> > nightly. It's for RC or beta release.
> >
> > [2] https://github.blog/2019-05-10-introducing-github-package-registry/
> >
> > dev/release/post-05-js.sh:
> >
> > We may be able to use GitHub Package Registry[2] to upload
> > npm pac

Re: [DISCUSS] Release cadence and release vote conventions

2019-08-01 Thread Andy Grove
I'll start taking a look at the maven issue. We might not want to use maven
release plugin given that we control the version number already across this
repository via other means.

On Wed, Jul 31, 2019 at 4:26 PM Sutou Kouhei  wrote:

> Hi,
>
> Sorry for not replying this thread.
>
> I think that the biggest problem is related to our Java
> package.
>
>
> We'll be able to resolve the GPG key problem by creating a
> GPG key only for nightly release test. We can share the test
> GPG key publicly because it's a just for testing.
>
> It'll work for our binary artifacts and APT/Yum repositories
> but not work for our Java package. I don't know where GPG
> key is used in our Java package...
>
>
> We'll be able to resolve the Git commit problem by creating
> a cloned Git repository for test. It's done in our
> dev/release/00-prepare-test.rb[1].
>
> [1]
> https://github.com/apache/arrow/blob/master/dev/release/00-prepare-test.rb#L30
>
> The biggest problem for the Git commit is our Java package
> requires "apache-arrow-${VERSION}" tag on
> https://github.com/apache/arrow . (Right?)
> I think that "mvm release:perform" in
> dev/release/01-perform.sh does so but I don't know the
> details of "mvm release:perform"...
>
>
> More details:
>
> dev/release/00-prepare.sh:
>
> We'll be able to run this automatically when we can resolve
> the above GPG key problem in our Java package. We can
> resolve the Git commit problem by creating a cloned Git
> repository.
>
> dev/release/01-prepare.sh:
>
> We'll be able to run this automatically when we can resolve
> the above Git commit ("apche-arrow-${VERSION}" tag) problem
> in our Java package.
>
> dev/release/02-source.sh:
>
> We'll be able to run this automatically by creating a GPG
> key for nightly release test. We'll use Bintray to upload RC
> source archive instead of dist.apache.org. Ah, we need a
> Bintray API key for this. It must be secret.
>
> dev/release/03-binary.sh:
>
> We'll be able to run this automatically by creating a GPG
> key for nightly release test. We need a Bintray API key too.
>
> We need to improve this to support nightly release test. It
> use "XXX-rc" such as "debian-rc" for Bintray "package" name.
> It should use "XXX-nightly" such as "debian-nightly" for
> nightly release test instead.
>
> dev/release/post-00-release.sh:
>
> We'll be able to skip this.
>
> dev/release/post-01-upload.sh:
>
> We'll be able to skip this.
>
> dev/release/post-02-binary.sh:
>
> We'll be able to run this automatically by creating Bintray
> "packages" for nightly release and use them. We can create
> "XXX-nightly-release" ("debian-nightly-release") Bintray
> "packages" and use them instead of "XXX" ("debian") Bintray
> "packages".
>
> "debian" Bintray "package": https://bintray.com/apache/debian/
>
> We need to improve this to support nightly release.
>
> dev/release/post-03-website.sh:
>
> We'll be able to run this automatically by creating a cloned
> Git repository for test.
>
> It's better that we have a Web site to show generated pages.
> We can create
> https://github.com/apache/arrow-site/tree/asf-site/nightly
> and use it but I don't like it. Because arrow-site increases
> a commit day by day.
> Can we prepare a Web site for this? (arrow-nightly.ursalabs.org?)
>
> dev/release/post-04-rubygems.sh:
>
> We may be able to use GitHub Package Registry[2] to upload
> RubyGems. We can use "pre-release" package feature of
> https://rubygems.org/ but it's not suitable for
> nightly. It's for RC or beta release.
>
> [2] https://github.blog/2019-05-10-introducing-github-package-registry/
>
> dev/release/post-05-js.sh:
>
> We may be able to use GitHub Package Registry[2] to upload
> npm packages.
>
> dev/release/post-06-csharp.sh:
>
> We may be able to use GitHub Package Registry[2] to upload
> NuGet packages.
>
> dev/release/post-07-rust.sh:
>
> I don't have any idea. But it must be ran
> automatically. It's always failed. I needed to run each
> command manually.
>
> dev/release/post-08-remove-rc.sh:
>
> We'll be able to skip this.
>
>
> Thanks,
> --
> kou
>
> In 
>   "Re: [DISCUSS] Release cadence and release vote conventions" on Wed, 31
> Jul 2019 15:35:57 -0500,
>   Wes McKinney  wrote:
>
> > The PMC member and their GPG keys need to be in the loop at some
> > point. The release artifacts can be produced by some kind of CI/CD
> > system so long as the PMC member

Re: [DISCUSS] Release cadence and release vote conventions

2019-07-31 Thread Sutou Kouhei
Hi,

Sorry for not replying this thread.

I think that the biggest problem is related to our Java
package.


We'll be able to resolve the GPG key problem by creating a
GPG key only for nightly release test. We can share the test
GPG key publicly because it's a just for testing.

It'll work for our binary artifacts and APT/Yum repositories
but not work for our Java package. I don't know where GPG
key is used in our Java package...


We'll be able to resolve the Git commit problem by creating
a cloned Git repository for test. It's done in our
dev/release/00-prepare-test.rb[1].

[1] 
https://github.com/apache/arrow/blob/master/dev/release/00-prepare-test.rb#L30 

The biggest problem for the Git commit is our Java package
requires "apache-arrow-${VERSION}" tag on
https://github.com/apache/arrow . (Right?)
I think that "mvm release:perform" in
dev/release/01-perform.sh does so but I don't know the
details of "mvm release:perform"...


More details:

dev/release/00-prepare.sh:

We'll be able to run this automatically when we can resolve
the above GPG key problem in our Java package. We can
resolve the Git commit problem by creating a cloned Git
repository.

dev/release/01-prepare.sh:

We'll be able to run this automatically when we can resolve
the above Git commit ("apche-arrow-${VERSION}" tag) problem
in our Java package.

dev/release/02-source.sh:

We'll be able to run this automatically by creating a GPG
key for nightly release test. We'll use Bintray to upload RC
source archive instead of dist.apache.org. Ah, we need a
Bintray API key for this. It must be secret.

dev/release/03-binary.sh:

We'll be able to run this automatically by creating a GPG
key for nightly release test. We need a Bintray API key too.

We need to improve this to support nightly release test. It
use "XXX-rc" such as "debian-rc" for Bintray "package" name.
It should use "XXX-nightly" such as "debian-nightly" for
nightly release test instead.

dev/release/post-00-release.sh:

We'll be able to skip this.

dev/release/post-01-upload.sh:

We'll be able to skip this.

dev/release/post-02-binary.sh:

We'll be able to run this automatically by creating Bintray
"packages" for nightly release and use them. We can create
"XXX-nightly-release" ("debian-nightly-release") Bintray
"packages" and use them instead of "XXX" ("debian") Bintray
"packages".

"debian" Bintray "package": https://bintray.com/apache/debian/

We need to improve this to support nightly release.

dev/release/post-03-website.sh:

We'll be able to run this automatically by creating a cloned
Git repository for test.

It's better that we have a Web site to show generated pages.
We can create
https://github.com/apache/arrow-site/tree/asf-site/nightly
and use it but I don't like it. Because arrow-site increases
a commit day by day.
Can we prepare a Web site for this? (arrow-nightly.ursalabs.org?)

dev/release/post-04-rubygems.sh:

We may be able to use GitHub Package Registry[2] to upload
RubyGems. We can use "pre-release" package feature of
https://rubygems.org/ but it's not suitable for
nightly. It's for RC or beta release.

[2] https://github.blog/2019-05-10-introducing-github-package-registry/

dev/release/post-05-js.sh:

We may be able to use GitHub Package Registry[2] to upload
npm packages.

dev/release/post-06-csharp.sh:

We may be able to use GitHub Package Registry[2] to upload
NuGet packages.

dev/release/post-07-rust.sh:

I don't have any idea. But it must be ran
automatically. It's always failed. I needed to run each
command manually.

dev/release/post-08-remove-rc.sh:

We'll be able to skip this.


Thanks,
--
kou

In 
  "Re: [DISCUSS] Release cadence and release vote conventions" on Wed, 31 Jul 
2019 15:35:57 -0500,
  Wes McKinney  wrote:

> The PMC member and their GPG keys need to be in the loop at some
> point. The release artifacts can be produced by some kind of CI/CD
> system so long as the PMC member has confidence in the security of
> those artifacts before signing them. For example, we build the
> official binary packages on public CI services and then download and
> sign them with Crossbow. I think the same could be done in theory with
> the source release but we'd first need to figure out what to do about
> the parts that create git commits.
> 
> On Wed, Jul 31, 2019 at 11:23 AM Andy Grove  wrote:
>>
>> To what extent would it be possible to automate the release process via
>> CICD?
>>
>> On Wed, Jul 31, 2019 at 9:19 AM Wes McKinney  wrote:
>>
>> > I think one thing that would help would be improving the
>> > reproducibility of the source release process. The RM has to have
>> > their machine configured in a particular way for it to work.
>> >
>> > 

Re: [DISCUSS] Release cadence and release vote conventions

2019-07-31 Thread Wes McKinney
ave to happen:
> > > >> > > > >
> > > >> > > > > * More PMC members must engage with the release management
> > role,
> > > >> > > > > process, and tools
> > > >> > > > > * Continued improvements to release tooling to make the
> > process
> > > >> less
> > > >> > > > > painful for the release manager. For example, it seems we may
> > > >> want to
> > > >> > > > > find a different place than Bintray to host binary artifacts
> > > >> > > > > temporarily during release votes
> > > >> > > >
> > > >> > > > My opinion that we need to build nightly release system.
> > > >> > > >
> > > >> > > > It uses dev/release/NN-*.sh to build .tar.gz and binary
> > > >> > > > artifacts from the .tar.gz.
> > > >> > > > It also uses dev/release/verify-release-candidate.* to
> > > >> > > > verify build .tar.gz and binary artifacts.
> > > >> > > > It also uses dev/release/post-NN-*.sh to do post release
> > > >> > > > tasks. (Some tasks such as uploading a package to packaging
> > > >> > > > system will be dry-run.)
> > > >> > > >
> > > >> > >
> > > >> > > I agree that having a turn-key release system that's capable of
> > > >> > > producing nightly packages is the way to do. That way any problems
> > > >> > > that would block a release will come up as they happen rather than
> > > >> > > piling up until the very end like they are now.
> > > >> > >
> > > >> > > > I needed 10 or more changes for dev/release/ to create
> > > >> > > > 0.14.0 RC0. (Some of them are still in my local stashes. I
> > > >> > > > don't have time to create pull requests for them
> > > >> > > > yet. Because I postponed some tasks of my main
> > > >> > > > business. I'll create pull requests after I finished the
> > > >> > > > postponed tasks of my main business.)
> > > >> > > >
> > > >> > >
> > > >> > > Thanks. I'll follow up on the 0.14.1/0.15.0 thread -- since we
> > need to
> > > >> > > release again soon because of problems with 0.14.0 please let us
> > know
> > > >> > > what patches will be needed to make another release.
> > > >> > >
> > > >> > > > If we fix problems related to dev/release/ in our normal
> > > >> > > > development process, release process will be less painful.
> > > >> > > >
> > > >> > > > The biggest problem for 0.14.0 RC0 is java/pom.xml related:
> > > >> > > >   https://github.com/apache/arrow/pull/4717
> > > >> > > >
> > > >> > > > It was difficult for me because I don't have Java
> > > >> > > > knowledge. Release manager needs help from many developers
> > > >> > > > because release manager may not have knowledge of all
> > > >> > > > supported languages. Apache Arrow supports 10 over
> > > >> > > > languages.
> > > >> > > >
> > > >> > > >
> > > >> > > > For Bintray API limit problem, we'll be able to resolve it.
> > > >> > > > I was added to https://bintray.com/apache/ members:
> > > >> > > >
> > > >> > > >   https://issues.apache.org/jira/browse/INFRA-18698
> > > >> > > >
> > > >> > > > I'll be able to use Bintray API without limitation in the
> > > >> > > > future. Release managers should also request the same thing.
> > > >> > > >
> > > >> > >
> > > >> > > This is good, I will add myself. Other PMC members should also add
> > > >> > > themselves.
> > > >> > >
> > > >> > > >
> > > >> > > > Thanks,
> > > >> > > > --
> > > >> > > > kou
> > > >> > > >
> > > >> > > > In  > > >> lsowxqxidjapc_cofguksj...@mail.gmail.com&g

Re: [DISCUSS] Release cadence and release vote conventions

2019-07-31 Thread Andy Grove
t; It also uses dev/release/verify-release-candidate.* to
> > >> > > > verify build .tar.gz and binary artifacts.
> > >> > > > It also uses dev/release/post-NN-*.sh to do post release
> > >> > > > tasks. (Some tasks such as uploading a package to packaging
> > >> > > > system will be dry-run.)
> > >> > > >
> > >> > >
> > >> > > I agree that having a turn-key release system that's capable of
> > >> > > producing nightly packages is the way to do. That way any problems
> > >> > > that would block a release will come up as they happen rather than
> > >> > > piling up until the very end like they are now.
> > >> > >
> > >> > > > I needed 10 or more changes for dev/release/ to create
> > >> > > > 0.14.0 RC0. (Some of them are still in my local stashes. I
> > >> > > > don't have time to create pull requests for them
> > >> > > > yet. Because I postponed some tasks of my main
> > >> > > > business. I'll create pull requests after I finished the
> > >> > > > postponed tasks of my main business.)
> > >> > > >
> > >> > >
> > >> > > Thanks. I'll follow up on the 0.14.1/0.15.0 thread -- since we
> need to
> > >> > > release again soon because of problems with 0.14.0 please let us
> know
> > >> > > what patches will be needed to make another release.
> > >> > >
> > >> > > > If we fix problems related to dev/release/ in our normal
> > >> > > > development process, release process will be less painful.
> > >> > > >
> > >> > > > The biggest problem for 0.14.0 RC0 is java/pom.xml related:
> > >> > > >   https://github.com/apache/arrow/pull/4717
> > >> > > >
> > >> > > > It was difficult for me because I don't have Java
> > >> > > > knowledge. Release manager needs help from many developers
> > >> > > > because release manager may not have knowledge of all
> > >> > > > supported languages. Apache Arrow supports 10 over
> > >> > > > languages.
> > >> > > >
> > >> > > >
> > >> > > > For Bintray API limit problem, we'll be able to resolve it.
> > >> > > > I was added to https://bintray.com/apache/ members:
> > >> > > >
> > >> > > >   https://issues.apache.org/jira/browse/INFRA-18698
> > >> > > >
> > >> > > > I'll be able to use Bintray API without limitation in the
> > >> > > > future. Release managers should also request the same thing.
> > >> > > >
> > >> > >
> > >> > > This is good, I will add myself. Other PMC members should also add
> > >> > > themselves.
> > >> > >
> > >> > > >
> > >> > > > Thanks,
> > >> > > > --
> > >> > > > kou
> > >> > > >
> > >> > > > In  > >> lsowxqxidjapc_cofguksj...@mail.gmail.com>
> > >> > > >   "[DISCUSS] Release cadence and release vote conventions" on
> Sat,
> > >> 6 Jul
> > >> > > 2019 16:28:50 -0500,
> > >> > > >   Wes McKinney  wrote:
> > >> > > >
> > >> > > > > hi folks,
> > >> > > > >
> > >> > > > > As a reminder, particularly since we have many new community
> > >> members
> > >> > > > > (some of whom have never been involved with an ASF project
> > >> before),
> > >> > > > > releases are approved exclusively by the PMC and in general
> > >> releases
> > >> > > > > cannot be vetoed. In spite of that, we strive to make releases
> > >> that
> > >> > > > > have unanimous (either by explicit +1 or lazy consent)
> support of
> > >> the
> > >> > > > > PMC. So it is better to have unanimous 5 +1 votes than 6 +1
> votes
> > >> with
> > >> > > > > a -1 dissenting vote.
> > >> > > > >
> > >> > > > > On the 0.14.0 vote, as with previous release votes, some
> issues

Re: [DISCUSS] Release cadence and release vote conventions

2019-07-31 Thread Wes McKinney
uld block a release will come up as they happen rather than
> >> > > piling up until the very end like they are now.
> >> > >
> >> > > > I needed 10 or more changes for dev/release/ to create
> >> > > > 0.14.0 RC0. (Some of them are still in my local stashes. I
> >> > > > don't have time to create pull requests for them
> >> > > > yet. Because I postponed some tasks of my main
> >> > > > business. I'll create pull requests after I finished the
> >> > > > postponed tasks of my main business.)
> >> > > >
> >> > >
> >> > > Thanks. I'll follow up on the 0.14.1/0.15.0 thread -- since we need to
> >> > > release again soon because of problems with 0.14.0 please let us know
> >> > > what patches will be needed to make another release.
> >> > >
> >> > > > If we fix problems related to dev/release/ in our normal
> >> > > > development process, release process will be less painful.
> >> > > >
> >> > > > The biggest problem for 0.14.0 RC0 is java/pom.xml related:
> >> > > >   https://github.com/apache/arrow/pull/4717
> >> > > >
> >> > > > It was difficult for me because I don't have Java
> >> > > > knowledge. Release manager needs help from many developers
> >> > > > because release manager may not have knowledge of all
> >> > > > supported languages. Apache Arrow supports 10 over
> >> > > > languages.
> >> > > >
> >> > > >
> >> > > > For Bintray API limit problem, we'll be able to resolve it.
> >> > > > I was added to https://bintray.com/apache/ members:
> >> > > >
> >> > > >   https://issues.apache.org/jira/browse/INFRA-18698
> >> > > >
> >> > > > I'll be able to use Bintray API without limitation in the
> >> > > > future. Release managers should also request the same thing.
> >> > > >
> >> > >
> >> > > This is good, I will add myself. Other PMC members should also add
> >> > > themselves.
> >> > >
> >> > > >
> >> > > > Thanks,
> >> > > > --
> >> > > > kou
> >> > > >
> >> > > > In  >> lsowxqxidjapc_cofguksj...@mail.gmail.com>
> >> > > >   "[DISCUSS] Release cadence and release vote conventions" on Sat,
> >> 6 Jul
> >> > > 2019 16:28:50 -0500,
> >> > > >   Wes McKinney  wrote:
> >> > > >
> >> > > > > hi folks,
> >> > > > >
> >> > > > > As a reminder, particularly since we have many new community
> >> members
> >> > > > > (some of whom have never been involved with an ASF project
> >> before),
> >> > > > > releases are approved exclusively by the PMC and in general
> >> releases
> >> > > > > cannot be vetoed. In spite of that, we strive to make releases
> >> that
> >> > > > > have unanimous (either by explicit +1 or lazy consent) support of
> >> the
> >> > > > > PMC. So it is better to have unanimous 5 +1 votes than 6 +1 votes
> >> with
> >> > > > > a -1 dissenting vote.
> >> > > > >
> >> > > > > On the 0.14.0 vote, as with previous release votes, some issues
> >> with
> >> > > > > the release were raised by members of the community, whether
> >> build or
> >> > > > > test-related problems or other failures. Technically speaking,
> >> such
> >> > > > > issues have no _direct_ bearing on whether a release vote passes,
> >> only
> >> > > > > on whether PMC members vote +1, 0, or -1. A PMC member is allowed
> >> to
> >> > > > > change their vote based on new information -- for example, if I
> >> voted
> >> > > > > +1 on a release and then someone reported a serious licensing
> >> issue,
> >> > > > > then I would revise my vote to -1.
> >> > > > >
> >> > > > > On the RC0 vote thread, Jacques wrote [1]
> >> > > > >
> >> > > > > "A release vote should last until we arrive at consensus. When an
> >> > > > > issue is potentially identified, those that have voted should be
> >> given
> >> > > > > ample time to change their vote and others that may have been lazy
> >> > > > > consenters should be given time to chime in. There is no maximum
> >> > > > > amount of time a vote can be open. Allowing at least 24 hours
> >> after an
> >> > > > > objection is raised is a pretty minimum expectation unless the
> >> > > > > objector removes their objection.
> >> > > > >
> >> > > > > Note that Apache is more focused on consensus than timing (as
> >> opposed
> >> > > to
> >> > > > > virtually other other organizations in the world)."
> >> > > > >
> >> > > > > I agree with this and my opinion is that in future releases we
> >> should
> >> > > > > institute a minimum 24-hour "quiet period" after any community
> >> > > > > feedback on a release candidate to allow issues to be examined
> >> > > > > further. If someone finds a potential problem, and no negative
> >> votes
> >> > > > > are cast or changed, then the vote can close.
> >> > > > >
> >> > > > > As a related matter, it seems clear to me that Apache Arrow should
> >> > > > > have more frequent releases. I think this would decrease pressure
> >> on
> >> > > > > developers and users alike. While we've made strides to improve
> >> the
> >> > > > > tooling for release management (big thanks to Kou, Yosuke,
> >> Krisztian,
> >> > > > > and others), there is still quite some labor involved and
> >> potential
> >> > > > > for issues (e.g. API rate limiting for binary artifacts on
> >> Bintray).
> >> > > > >
> >> > > > > To be able to release more often, two things have to happen:
> >> > > > >
> >> > > > > * More PMC members must engage with the release management role,
> >> > > > > process, and tools
> >> > > > > * Continued improvements to release tooling to make the process
> >> less
> >> > > > > painful for the release manager. For example, it seems we may
> >> want to
> >> > > > > find a different place than Bintray to host binary artifacts
> >> > > > > temporarily during release votes
> >> > > > >
> >> > > > > Any other ideas for things we can do to improve the process and
> >> > > > > cadence of releases?
> >> > > > >
> >> > > > > Thanks,
> >> > > > > Wes
> >> > > > >
> >> > > > > [1]:
> >> > >
> >> https://lists.apache.org/thread.html/be6210e97b838494a5516dad6408f479efe4c98aff805000597c0196@%3Cdev.arrow.apache.org%3E
> >> > >
> >>
> >


Re: [DISCUSS] Release cadence and release vote conventions

2019-07-27 Thread Micah Kornfield
> > >
>> > > > If we fix problems related to dev/release/ in our normal
>> > > > development process, release process will be less painful.
>> > > >
>> > > > The biggest problem for 0.14.0 RC0 is java/pom.xml related:
>> > > >   https://github.com/apache/arrow/pull/4717
>> > > >
>> > > > It was difficult for me because I don't have Java
>> > > > knowledge. Release manager needs help from many developers
>> > > > because release manager may not have knowledge of all
>> > > > supported languages. Apache Arrow supports 10 over
>> > > > languages.
>> > > >
>> > > >
>> > > > For Bintray API limit problem, we'll be able to resolve it.
>> > > > I was added to https://bintray.com/apache/ members:
>> > > >
>> > > >   https://issues.apache.org/jira/browse/INFRA-18698
>> > > >
>> > > > I'll be able to use Bintray API without limitation in the
>> > > > future. Release managers should also request the same thing.
>> > > >
>> > >
>> > > This is good, I will add myself. Other PMC members should also add
>> > > themselves.
>> > >
>> > > >
>> > > > Thanks,
>> > > > --
>> > > > kou
>> > > >
>> > > > In > lsowxqxidjapc_cofguksj...@mail.gmail.com>
>> > > >   "[DISCUSS] Release cadence and release vote conventions" on Sat,
>> 6 Jul
>> > > 2019 16:28:50 -0500,
>> > > >   Wes McKinney  wrote:
>> > > >
>> > > > > hi folks,
>> > > > >
>> > > > > As a reminder, particularly since we have many new community
>> members
>> > > > > (some of whom have never been involved with an ASF project
>> before),
>> > > > > releases are approved exclusively by the PMC and in general
>> releases
>> > > > > cannot be vetoed. In spite of that, we strive to make releases
>> that
>> > > > > have unanimous (either by explicit +1 or lazy consent) support of
>> the
>> > > > > PMC. So it is better to have unanimous 5 +1 votes than 6 +1 votes
>> with
>> > > > > a -1 dissenting vote.
>> > > > >
>> > > > > On the 0.14.0 vote, as with previous release votes, some issues
>> with
>> > > > > the release were raised by members of the community, whether
>> build or
>> > > > > test-related problems or other failures. Technically speaking,
>> such
>> > > > > issues have no _direct_ bearing on whether a release vote passes,
>> only
>> > > > > on whether PMC members vote +1, 0, or -1. A PMC member is allowed
>> to
>> > > > > change their vote based on new information -- for example, if I
>> voted
>> > > > > +1 on a release and then someone reported a serious licensing
>> issue,
>> > > > > then I would revise my vote to -1.
>> > > > >
>> > > > > On the RC0 vote thread, Jacques wrote [1]
>> > > > >
>> > > > > "A release vote should last until we arrive at consensus. When an
>> > > > > issue is potentially identified, those that have voted should be
>> given
>> > > > > ample time to change their vote and others that may have been lazy
>> > > > > consenters should be given time to chime in. There is no maximum
>> > > > > amount of time a vote can be open. Allowing at least 24 hours
>> after an
>> > > > > objection is raised is a pretty minimum expectation unless the
>> > > > > objector removes their objection.
>> > > > >
>> > > > > Note that Apache is more focused on consensus than timing (as
>> opposed
>> > > to
>> > > > > virtually other other organizations in the world)."
>> > > > >
>> > > > > I agree with this and my opinion is that in future releases we
>> should
>> > > > > institute a minimum 24-hour "quiet period" after any community
>> > > > > feedback on a release candidate to allow issues to be examined
>> > > > > further. If someone finds a potential problem, and no negative
>> votes
>> > > > > are cast or changed, then the vote can close.
>> > > > >
>> > > > > As a related matter, it seems clear to me that Apache Arrow should
>> > > > > have more frequent releases. I think this would decrease pressure
>> on
>> > > > > developers and users alike. While we've made strides to improve
>> the
>> > > > > tooling for release management (big thanks to Kou, Yosuke,
>> Krisztian,
>> > > > > and others), there is still quite some labor involved and
>> potential
>> > > > > for issues (e.g. API rate limiting for binary artifacts on
>> Bintray).
>> > > > >
>> > > > > To be able to release more often, two things have to happen:
>> > > > >
>> > > > > * More PMC members must engage with the release management role,
>> > > > > process, and tools
>> > > > > * Continued improvements to release tooling to make the process
>> less
>> > > > > painful for the release manager. For example, it seems we may
>> want to
>> > > > > find a different place than Bintray to host binary artifacts
>> > > > > temporarily during release votes
>> > > > >
>> > > > > Any other ideas for things we can do to improve the process and
>> > > > > cadence of releases?
>> > > > >
>> > > > > Thanks,
>> > > > > Wes
>> > > > >
>> > > > > [1]:
>> > >
>> https://lists.apache.org/thread.html/be6210e97b838494a5516dad6408f479efe4c98aff805000597c0196@%3Cdev.arrow.apache.org%3E
>> > >
>>
>


Re: [DISCUSS] Release cadence and release vote conventions

2019-07-18 Thread Micah Kornfield
I'd can help as well, but not exactly sure where to start.  It seems like
there are already some JIRAs opened [1]
for improving the release?  Could someone more familiar with the process
pick out the highest priority ones? Do more need to be opened?

Thanks,
Micah

[1]
https://issues.apache.org/jira/browse/ARROW-2880?jql=project%20%3D%20ARROW%20AND%20status%20in%20(Open%2C%20%22In%20Progress%22%2C%20Reopened)%20AND%20component%20in%20(%22Developer%20Tools%22%2C%20Packaging)%20and%20summary%20~%20Release

On Sat, Jul 13, 2019 at 7:17 AM Wes McKinney  wrote:

> To be effective at improving the life of release managers, the nightly
> release process really should use as close as possible to the same
> scripts that the RM uses to produce the release. Otherwise we could
> have a situation where the nightlies succeed but there is some problem
> that either fails an RC or is unable to be produced at all.
>
> On Sat, Jul 13, 2019 at 9:12 AM Andy Grove  wrote:
> >
> > I would like to volunteer to help with Java and Rust release process
> work,
> > especially nightly releases.
> >
> > Although I'm not that familiar with the Java implementation of Arrow, I
> > have been using Java and Maven for a very long time.
> >
> > Do we envisage a single nightly release process that releases all
> languages
> > simultaneously? or do we want separate process per language, with
> different
> > maintainers?
> >
> >
> >
> > On Wed, Jul 10, 2019 at 8:18 AM Wes McKinney 
> wrote:
> >
> > > On Sun, Jul 7, 2019 at 7:40 PM Sutou Kouhei 
> wrote:
> > > >
> > > > Hi,
> > > >
> > > > > in future releases we should
> > > > > institute a minimum 24-hour "quiet period" after any community
> > > > > feedback on a release candidate to allow issues to be examined
> > > > > further.
> > > >
> > > > I agree with this. I'll do so when I do a release manager in
> > > > the future.
> > > >
> > > > > To be able to release more often, two things have to happen:
> > > > >
> > > > > * More PMC members must engage with the release management role,
> > > > > process, and tools
> > > > > * Continued improvements to release tooling to make the process
> less
> > > > > painful for the release manager. For example, it seems we may want
> to
> > > > > find a different place than Bintray to host binary artifacts
> > > > > temporarily during release votes
> > > >
> > > > My opinion that we need to build nightly release system.
> > > >
> > > > It uses dev/release/NN-*.sh to build .tar.gz and binary
> > > > artifacts from the .tar.gz.
> > > > It also uses dev/release/verify-release-candidate.* to
> > > > verify build .tar.gz and binary artifacts.
> > > > It also uses dev/release/post-NN-*.sh to do post release
> > > > tasks. (Some tasks such as uploading a package to packaging
> > > > system will be dry-run.)
> > > >
> > >
> > > I agree that having a turn-key release system that's capable of
> > > producing nightly packages is the way to do. That way any problems
> > > that would block a release will come up as they happen rather than
> > > piling up until the very end like they are now.
> > >
> > > > I needed 10 or more changes for dev/release/ to create
> > > > 0.14.0 RC0. (Some of them are still in my local stashes. I
> > > > don't have time to create pull requests for them
> > > > yet. Because I postponed some tasks of my main
> > > > business. I'll create pull requests after I finished the
> > > > postponed tasks of my main business.)
> > > >
> > >
> > > Thanks. I'll follow up on the 0.14.1/0.15.0 thread -- since we need to
> > > release again soon because of problems with 0.14.0 please let us know
> > > what patches will be needed to make another release.
> > >
> > > > If we fix problems related to dev/release/ in our normal
> > > > development process, release process will be less painful.
> > > >
> > > > The biggest problem for 0.14.0 RC0 is java/pom.xml related:
> > > >   https://github.com/apache/arrow/pull/4717
> > > >
> > > > It was difficult for me because I don't have Java
> > > > knowledge. Release manager needs help from many developers
> > > > because release manager may not have knowledge of all
> > > &g

Re: [DISCUSS] Release cadence and release vote conventions

2019-07-13 Thread Andy Grove
I would like to volunteer to help with Java and Rust release process work,
especially nightly releases.

Although I'm not that familiar with the Java implementation of Arrow, I
have been using Java and Maven for a very long time.

Do we envisage a single nightly release process that releases all languages
simultaneously? or do we want separate process per language, with different
maintainers?



On Wed, Jul 10, 2019 at 8:18 AM Wes McKinney  wrote:

> On Sun, Jul 7, 2019 at 7:40 PM Sutou Kouhei  wrote:
> >
> > Hi,
> >
> > > in future releases we should
> > > institute a minimum 24-hour "quiet period" after any community
> > > feedback on a release candidate to allow issues to be examined
> > > further.
> >
> > I agree with this. I'll do so when I do a release manager in
> > the future.
> >
> > > To be able to release more often, two things have to happen:
> > >
> > > * More PMC members must engage with the release management role,
> > > process, and tools
> > > * Continued improvements to release tooling to make the process less
> > > painful for the release manager. For example, it seems we may want to
> > > find a different place than Bintray to host binary artifacts
> > > temporarily during release votes
> >
> > My opinion that we need to build nightly release system.
> >
> > It uses dev/release/NN-*.sh to build .tar.gz and binary
> > artifacts from the .tar.gz.
> > It also uses dev/release/verify-release-candidate.* to
> > verify build .tar.gz and binary artifacts.
> > It also uses dev/release/post-NN-*.sh to do post release
> > tasks. (Some tasks such as uploading a package to packaging
> > system will be dry-run.)
> >
>
> I agree that having a turn-key release system that's capable of
> producing nightly packages is the way to do. That way any problems
> that would block a release will come up as they happen rather than
> piling up until the very end like they are now.
>
> > I needed 10 or more changes for dev/release/ to create
> > 0.14.0 RC0. (Some of them are still in my local stashes. I
> > don't have time to create pull requests for them
> > yet. Because I postponed some tasks of my main
> > business. I'll create pull requests after I finished the
> > postponed tasks of my main business.)
> >
>
> Thanks. I'll follow up on the 0.14.1/0.15.0 thread -- since we need to
> release again soon because of problems with 0.14.0 please let us know
> what patches will be needed to make another release.
>
> > If we fix problems related to dev/release/ in our normal
> > development process, release process will be less painful.
> >
> > The biggest problem for 0.14.0 RC0 is java/pom.xml related:
> >   https://github.com/apache/arrow/pull/4717
> >
> > It was difficult for me because I don't have Java
> > knowledge. Release manager needs help from many developers
> > because release manager may not have knowledge of all
> > supported languages. Apache Arrow supports 10 over
> > languages.
> >
> >
> > For Bintray API limit problem, we'll be able to resolve it.
> > I was added to https://bintray.com/apache/ members:
> >
> >   https://issues.apache.org/jira/browse/INFRA-18698
> >
> > I'll be able to use Bintray API without limitation in the
> > future. Release managers should also request the same thing.
> >
>
> This is good, I will add myself. Other PMC members should also add
> themselves.
>
> >
> > Thanks,
> > --
> > kou
> >
> > In 
> >   "[DISCUSS] Release cadence and release vote conventions" on Sat, 6 Jul
> 2019 16:28:50 -0500,
> >   Wes McKinney  wrote:
> >
> > > hi folks,
> > >
> > > As a reminder, particularly since we have many new community members
> > > (some of whom have never been involved with an ASF project before),
> > > releases are approved exclusively by the PMC and in general releases
> > > cannot be vetoed. In spite of that, we strive to make releases that
> > > have unanimous (either by explicit +1 or lazy consent) support of the
> > > PMC. So it is better to have unanimous 5 +1 votes than 6 +1 votes with
> > > a -1 dissenting vote.
> > >
> > > On the 0.14.0 vote, as with previous release votes, some issues with
> > > the release were raised by members of the community, whether build or
> > > test-related problems or other failures. Technically speaking, such
> > > issues have no _direct_ bearing on whether a release vote passes, only
> > > on whether PMC memb

Re: [DISCUSS] Release cadence and release vote conventions

2019-07-10 Thread Wes McKinney
On Sun, Jul 7, 2019 at 7:40 PM Sutou Kouhei  wrote:
>
> Hi,
>
> > in future releases we should
> > institute a minimum 24-hour "quiet period" after any community
> > feedback on a release candidate to allow issues to be examined
> > further.
>
> I agree with this. I'll do so when I do a release manager in
> the future.
>
> > To be able to release more often, two things have to happen:
> >
> > * More PMC members must engage with the release management role,
> > process, and tools
> > * Continued improvements to release tooling to make the process less
> > painful for the release manager. For example, it seems we may want to
> > find a different place than Bintray to host binary artifacts
> > temporarily during release votes
>
> My opinion that we need to build nightly release system.
>
> It uses dev/release/NN-*.sh to build .tar.gz and binary
> artifacts from the .tar.gz.
> It also uses dev/release/verify-release-candidate.* to
> verify build .tar.gz and binary artifacts.
> It also uses dev/release/post-NN-*.sh to do post release
> tasks. (Some tasks such as uploading a package to packaging
> system will be dry-run.)
>

I agree that having a turn-key release system that's capable of
producing nightly packages is the way to do. That way any problems
that would block a release will come up as they happen rather than
piling up until the very end like they are now.

> I needed 10 or more changes for dev/release/ to create
> 0.14.0 RC0. (Some of them are still in my local stashes. I
> don't have time to create pull requests for them
> yet. Because I postponed some tasks of my main
> business. I'll create pull requests after I finished the
> postponed tasks of my main business.)
>

Thanks. I'll follow up on the 0.14.1/0.15.0 thread -- since we need to
release again soon because of problems with 0.14.0 please let us know
what patches will be needed to make another release.

> If we fix problems related to dev/release/ in our normal
> development process, release process will be less painful.
>
> The biggest problem for 0.14.0 RC0 is java/pom.xml related:
>   https://github.com/apache/arrow/pull/4717
>
> It was difficult for me because I don't have Java
> knowledge. Release manager needs help from many developers
> because release manager may not have knowledge of all
> supported languages. Apache Arrow supports 10 over
> languages.
>
>
> For Bintray API limit problem, we'll be able to resolve it.
> I was added to https://bintray.com/apache/ members:
>
>   https://issues.apache.org/jira/browse/INFRA-18698
>
> I'll be able to use Bintray API without limitation in the
> future. Release managers should also request the same thing.
>

This is good, I will add myself. Other PMC members should also add themselves.

>
> Thanks,
> --
> kou
>
> In 
>   "[DISCUSS] Release cadence and release vote conventions" on Sat, 6 Jul 2019 
> 16:28:50 -0500,
>   Wes McKinney  wrote:
>
> > hi folks,
> >
> > As a reminder, particularly since we have many new community members
> > (some of whom have never been involved with an ASF project before),
> > releases are approved exclusively by the PMC and in general releases
> > cannot be vetoed. In spite of that, we strive to make releases that
> > have unanimous (either by explicit +1 or lazy consent) support of the
> > PMC. So it is better to have unanimous 5 +1 votes than 6 +1 votes with
> > a -1 dissenting vote.
> >
> > On the 0.14.0 vote, as with previous release votes, some issues with
> > the release were raised by members of the community, whether build or
> > test-related problems or other failures. Technically speaking, such
> > issues have no _direct_ bearing on whether a release vote passes, only
> > on whether PMC members vote +1, 0, or -1. A PMC member is allowed to
> > change their vote based on new information -- for example, if I voted
> > +1 on a release and then someone reported a serious licensing issue,
> > then I would revise my vote to -1.
> >
> > On the RC0 vote thread, Jacques wrote [1]
> >
> > "A release vote should last until we arrive at consensus. When an
> > issue is potentially identified, those that have voted should be given
> > ample time to change their vote and others that may have been lazy
> > consenters should be given time to chime in. There is no maximum
> > amount of time a vote can be open. Allowing at least 24 hours after an
> > objection is raised is a pretty minimum expectation unless the
> > objector removes their objection.
> >
> > Note that Apache is more focused on consensus than timing (as oppose

Re: [DISCUSS] Release cadence and release vote conventions

2019-07-07 Thread Sutou Kouhei
Hi,

> in future releases we should
> institute a minimum 24-hour "quiet period" after any community
> feedback on a release candidate to allow issues to be examined
> further.

I agree with this. I'll do so when I do a release manager in
the future.

> To be able to release more often, two things have to happen:
> 
> * More PMC members must engage with the release management role,
> process, and tools
> * Continued improvements to release tooling to make the process less
> painful for the release manager. For example, it seems we may want to
> find a different place than Bintray to host binary artifacts
> temporarily during release votes

My opinion that we need to build nightly release system.

It uses dev/release/NN-*.sh to build .tar.gz and binary
artifacts from the .tar.gz.
It also uses dev/release/verify-release-candidate.* to
verify build .tar.gz and binary artifacts.
It also uses dev/release/post-NN-*.sh to do post release
tasks. (Some tasks such as uploading a package to packaging
system will be dry-run.)

I needed 10 or more changes for dev/release/ to create
0.14.0 RC0. (Some of them are still in my local stashes. I
don't have time to create pull requests for them
yet. Because I postponed some tasks of my main
business. I'll create pull requests after I finished the
postponed tasks of my main business.)

If we fix problems related to dev/release/ in our normal
development process, release process will be less painful.

The biggest problem for 0.14.0 RC0 is java/pom.xml related:
  https://github.com/apache/arrow/pull/4717

It was difficult for me because I don't have Java
knowledge. Release manager needs help from many developers
because release manager may not have knowledge of all
supported languages. Apache Arrow supports 10 over
languages.


For Bintray API limit problem, we'll be able to resolve it.
I was added to https://bintray.com/apache/ members:

  https://issues.apache.org/jira/browse/INFRA-18698

I'll be able to use Bintray API without limitation in the
future. Release managers should also request the same thing.


Thanks,
--
kou

In 
  "[DISCUSS] Release cadence and release vote conventions" on Sat, 6 Jul 2019 
16:28:50 -0500,
  Wes McKinney  wrote:

> hi folks,
> 
> As a reminder, particularly since we have many new community members
> (some of whom have never been involved with an ASF project before),
> releases are approved exclusively by the PMC and in general releases
> cannot be vetoed. In spite of that, we strive to make releases that
> have unanimous (either by explicit +1 or lazy consent) support of the
> PMC. So it is better to have unanimous 5 +1 votes than 6 +1 votes with
> a -1 dissenting vote.
> 
> On the 0.14.0 vote, as with previous release votes, some issues with
> the release were raised by members of the community, whether build or
> test-related problems or other failures. Technically speaking, such
> issues have no _direct_ bearing on whether a release vote passes, only
> on whether PMC members vote +1, 0, or -1. A PMC member is allowed to
> change their vote based on new information -- for example, if I voted
> +1 on a release and then someone reported a serious licensing issue,
> then I would revise my vote to -1.
> 
> On the RC0 vote thread, Jacques wrote [1]
> 
> "A release vote should last until we arrive at consensus. When an
> issue is potentially identified, those that have voted should be given
> ample time to change their vote and others that may have been lazy
> consenters should be given time to chime in. There is no maximum
> amount of time a vote can be open. Allowing at least 24 hours after an
> objection is raised is a pretty minimum expectation unless the
> objector removes their objection.
> 
> Note that Apache is more focused on consensus than timing (as opposed to
> virtually other other organizations in the world)."
> 
> I agree with this and my opinion is that in future releases we should
> institute a minimum 24-hour "quiet period" after any community
> feedback on a release candidate to allow issues to be examined
> further. If someone finds a potential problem, and no negative votes
> are cast or changed, then the vote can close.
> 
> As a related matter, it seems clear to me that Apache Arrow should
> have more frequent releases. I think this would decrease pressure on
> developers and users alike. While we've made strides to improve the
> tooling for release management (big thanks to Kou, Yosuke, Krisztian,
> and others), there is still quite some labor involved and potential
> for issues (e.g. API rate limiting for binary artifacts on Bintray).
> 
> To be able to release more often, two things have to happen:
> 
> * More PMC members must engage with the release management role,
> process, and tools
> * Continued 

[DISCUSS] Release cadence and release vote conventions

2019-07-06 Thread Wes McKinney
hi folks,

As a reminder, particularly since we have many new community members
(some of whom have never been involved with an ASF project before),
releases are approved exclusively by the PMC and in general releases
cannot be vetoed. In spite of that, we strive to make releases that
have unanimous (either by explicit +1 or lazy consent) support of the
PMC. So it is better to have unanimous 5 +1 votes than 6 +1 votes with
a -1 dissenting vote.

On the 0.14.0 vote, as with previous release votes, some issues with
the release were raised by members of the community, whether build or
test-related problems or other failures. Technically speaking, such
issues have no _direct_ bearing on whether a release vote passes, only
on whether PMC members vote +1, 0, or -1. A PMC member is allowed to
change their vote based on new information -- for example, if I voted
+1 on a release and then someone reported a serious licensing issue,
then I would revise my vote to -1.

On the RC0 vote thread, Jacques wrote [1]

"A release vote should last until we arrive at consensus. When an
issue is potentially identified, those that have voted should be given
ample time to change their vote and others that may have been lazy
consenters should be given time to chime in. There is no maximum
amount of time a vote can be open. Allowing at least 24 hours after an
objection is raised is a pretty minimum expectation unless the
objector removes their objection.

Note that Apache is more focused on consensus than timing (as opposed to
virtually other other organizations in the world)."

I agree with this and my opinion is that in future releases we should
institute a minimum 24-hour "quiet period" after any community
feedback on a release candidate to allow issues to be examined
further. If someone finds a potential problem, and no negative votes
are cast or changed, then the vote can close.

As a related matter, it seems clear to me that Apache Arrow should
have more frequent releases. I think this would decrease pressure on
developers and users alike. While we've made strides to improve the
tooling for release management (big thanks to Kou, Yosuke, Krisztian,
and others), there is still quite some labor involved and potential
for issues (e.g. API rate limiting for binary artifacts on Bintray).

To be able to release more often, two things have to happen:

* More PMC members must engage with the release management role,
process, and tools
* Continued improvements to release tooling to make the process less
painful for the release manager. For example, it seems we may want to
find a different place than Bintray to host binary artifacts
temporarily during release votes

Any other ideas for things we can do to improve the process and
cadence of releases?

Thanks,
Wes

[1]: 
https://lists.apache.org/thread.html/be6210e97b838494a5516dad6408f479efe4c98aff805000597c0196@%3Cdev.arrow.apache.org%3E