Re: [Governance] [Proposal] Stop force-pushing to PRs after release?

[Wes McKinney]

It sounds like releasing from branches going forward would solve our problems:

* No more force-pushing master
* No need to rebase after releases

If there are no objections, I would say we should go ahead and do
this. I don't think there is a need for a vote but if anyone wants one
please chime in.

There is a procedural matter around updating the "dev version"  (e.g.
from 3.0.0-dev to 4.0.0-dev) but that should be easy to sort out.

Thanks
Wes

On Thu, Nov 26, 2020 at 6:09 AM Joris Van den Bossche
 wrote:
>
> Note that we already released from a branch in the past, for the 1.0.1
> release, if I am not mistaken. In this case we cherry-picked the
> patches for the bugfix release on that release branch
> (https://github.com/apache/arrow/pull/7933).
> +1 to do that also for the main releases to avoid any need of force
> pushing master.
>
> Joris
>
> On Thu, 26 Nov 2020 at 12:05, Andrew Lamb  wrote:
> >
> > I have also used the pattern of releasing from a release branch and
> > cherry-picking (as opposed to merge) fixes from main to the release branch
> > as needed for the release (on commercial software products), and it worked
> > well.
> >
> > As long as you have a regular and frequent release schedule (as Arrow does
> > with quarterly releases), most cherry-picks  are likely to happen very soon
> > after the release branch is cut, when the divergence from main is the
> > smallest, and thus the chance of conflicts remains low.
> >
> >
> > On Thu, Nov 26, 2020 at 3:07 AM Antoine Pitrou  wrote:
> >
> > >
> > > Personally, I simply don't really understand the aversion for merge
> > > commits.  The need to recreate the "master" branch locally after a
> > > release has bitten me several times (git lets you screw that up very
> > > easily...), and it has always been a bit frustrating.
> > >
> > > Regards
> > >
> > > Antoine.
> > >
> > >
> > > Le 26/11/2020 à 02:02, Jacques Nadeau a écrit :
> > > >>
> > > >> I don’t have a problem with releasing out of branches. I think I (or
> > > >> someone) proposed this in the past and there was not consensus but it
> > > seems
> > > >> like a good time to revisit the issue.
> > > >>
> > > >
> > > > Thanks for the recap. I just couldn't remember where people were at on
> > > this.
> > > >
> > > > I'm a big +1 for releasing out of branches. The biggest downside from my
> > > > pov is that if the release takes a long time, you spend some time
> > > > cherry-picking. That being said, I think the other pros (development
> > > isn't
> > > > stalled, you don't have force pushes, etc) outweigh the cherry-picking
> > > > pain. Especially with a fast moving project like Arrow.
> > > >
> > > > What do others think?
> > > >
> > >

Re: [Governance] [Proposal] Stop force-pushing to PRs after release?

[Joris Van den Bossche]

Note that we already released from a branch in the past, for the 1.0.1
release, if I am not mistaken. In this case we cherry-picked the
patches for the bugfix release on that release branch
(https://github.com/apache/arrow/pull/7933).
+1 to do that also for the main releases to avoid any need of force
pushing master.

Joris

On Thu, 26 Nov 2020 at 12:05, Andrew Lamb  wrote:
>
> I have also used the pattern of releasing from a release branch and
> cherry-picking (as opposed to merge) fixes from main to the release branch
> as needed for the release (on commercial software products), and it worked
> well.
>
> As long as you have a regular and frequent release schedule (as Arrow does
> with quarterly releases), most cherry-picks  are likely to happen very soon
> after the release branch is cut, when the divergence from main is the
> smallest, and thus the chance of conflicts remains low.
>
>
> On Thu, Nov 26, 2020 at 3:07 AM Antoine Pitrou  wrote:
>
> >
> > Personally, I simply don't really understand the aversion for merge
> > commits.  The need to recreate the "master" branch locally after a
> > release has bitten me several times (git lets you screw that up very
> > easily...), and it has always been a bit frustrating.
> >
> > Regards
> >
> > Antoine.
> >
> >
> > Le 26/11/2020 à 02:02, Jacques Nadeau a écrit :
> > >>
> > >> I don’t have a problem with releasing out of branches. I think I (or
> > >> someone) proposed this in the past and there was not consensus but it
> > seems
> > >> like a good time to revisit the issue.
> > >>
> > >
> > > Thanks for the recap. I just couldn't remember where people were at on
> > this.
> > >
> > > I'm a big +1 for releasing out of branches. The biggest downside from my
> > > pov is that if the release takes a long time, you spend some time
> > > cherry-picking. That being said, I think the other pros (development
> > isn't
> > > stalled, you don't have force pushes, etc) outweigh the cherry-picking
> > > pain. Especially with a fast moving project like Arrow.
> > >
> > > What do others think?
> > >
> >

Re: [Governance] [Proposal] Stop force-pushing to PRs after release?

[Andrew Lamb]

I have also used the pattern of releasing from a release branch and
cherry-picking (as opposed to merge) fixes from main to the release branch
as needed for the release (on commercial software products), and it worked
well.

As long as you have a regular and frequent release schedule (as Arrow does
with quarterly releases), most cherry-picks  are likely to happen very soon
after the release branch is cut, when the divergence from main is the
smallest, and thus the chance of conflicts remains low.


On Thu, Nov 26, 2020 at 3:07 AM Antoine Pitrou  wrote:

>
> Personally, I simply don't really understand the aversion for merge
> commits.  The need to recreate the "master" branch locally after a
> release has bitten me several times (git lets you screw that up very
> easily...), and it has always been a bit frustrating.
>
> Regards
>
> Antoine.
>
>
> Le 26/11/2020 à 02:02, Jacques Nadeau a écrit :
> >>
> >> I don’t have a problem with releasing out of branches. I think I (or
> >> someone) proposed this in the past and there was not consensus but it
> seems
> >> like a good time to revisit the issue.
> >>
> >
> > Thanks for the recap. I just couldn't remember where people were at on
> this.
> >
> > I'm a big +1 for releasing out of branches. The biggest downside from my
> > pov is that if the release takes a long time, you spend some time
> > cherry-picking. That being said, I think the other pros (development
> isn't
> > stalled, you don't have force pushes, etc) outweigh the cherry-picking
> > pain. Especially with a fast moving project like Arrow.
> >
> > What do others think?
> >
>

Re: [Governance] [Proposal] Stop force-pushing to PRs after release?

[Antoine Pitrou]

Personally, I simply don't really understand the aversion for merge
commits.  The need to recreate the "master" branch locally after a
release has bitten me several times (git lets you screw that up very
easily...), and it has always been a bit frustrating.

Regards

Antoine.


Le 26/11/2020 à 02:02, Jacques Nadeau a écrit :
>>
>> I don’t have a problem with releasing out of branches. I think I (or
>> someone) proposed this in the past and there was not consensus but it seems
>> like a good time to revisit the issue.
>>
> 
> Thanks for the recap. I just couldn't remember where people were at on this.
> 
> I'm a big +1 for releasing out of branches. The biggest downside from my
> pov is that if the release takes a long time, you spend some time
> cherry-picking. That being said, I think the other pros (development isn't
> stalled, you don't have force pushes, etc) outweigh the cherry-picking
> pain. Especially with a fast moving project like Arrow.
> 
> What do others think?
> 

Re: [Governance] [Proposal] Stop force-pushing to PRs after release?

[Jacques Nadeau]

>
> I don’t have a problem with releasing out of branches. I think I (or
> someone) proposed this in the past and there was not consensus but it seems
> like a good time to revisit the issue.
>

Thanks for the recap. I just couldn't remember where people were at on this.

I'm a big +1 for releasing out of branches. The biggest downside from my
pov is that if the release takes a long time, you spend some time
cherry-picking. That being said, I think the other pros (development isn't
stalled, you don't have force pushes, etc) outweigh the cherry-picking
pain. Especially with a fast moving project like Arrow.

What do others think?

Re: [Governance] [Proposal] Stop force-pushing to PRs after release?

[Wes McKinney]

I don’t have a problem with releasing out of branches. I think I (or
someone) proposed this in the past and there was not consensus but it seems
like a good time to revisit the issue.

On Wed, Nov 25, 2020 at 7:41 PM Jacques Nadeau  wrote:

> I'm catching up here. A couple questions.
>
>- I don't think we should require the inclusion of the release commits
>in the main branch. Having leafs created right before release seems to
>simplify this and resolve any issues around force PRs, no? Or maybe I'm
>misunderstanding something? Maybe to put another way, why do we have to
>force push anything to main ever? That seems like the real problem.
>- I'm not sure I follow the comments about the java release process
>requiring any interactions with the Apache repo or a particular branch.
> In
>the past (note, it has been a while), I used mvn properties to override
> any
>of this behavior so that I might do any pushes to a personal repo rather
>than the Apache one (or an apache release branch rather than the main
>branch).
>
> - The local release branch has some unpushed commits such as bumping to the
> > next snapshot version.
>
>
> Most projects I've worked on work this way:
>
> We decide to do a release. At that point, we create a branch for the
> release. In the main branch, we bump the version up immediately.
> In the release branch, we do minimal commits as needed for hotfixes etc
> (and all cherry pick to main).
> We do release steps and push those commits (excluding the post-release
> version bump) to the release branch.
> We vote on the release commit and artifacts.
> If there is a release failure, we revert any release commits in the release
> branch and then apply addiitonal fixes.
> Then do the process again.
>
> This way main never has a force push.
>
> (Full disclosure, the one exception I've worked on is that Calcite uses
> force pushes as well as process-based branch locking. I think we should
> avoid both of those patterns).
>
> I'd also clarify, while I'm against merge commits, I think that a release
> branch that forks slightly off the main branch should be fine. Do people
> disagree with this?
>
>
>
> On Wed, Nov 25, 2020 at 5:52 AM Wes McKinney  wrote:
>
> > > The first two sound logical, but why couldn't those version bumps be a
> > merge commit into master?
> >
> > We've made the commitment to maintaining a linear commit history in
> > this project.
> >
> > Auto-rebasing the PRs at this point is best described as "harm
> > reduction". The root cause is GitHub's UI which shows a bunch of
> > unrelated commits in the UI when a PR is based on a commit that no
> > longer exists due to the rebase. This can confuse people and sometimes
> > cause them to engage in incorrect actions (trying to "git merge" in
> > various ways) that cause their branches to get very messed up. If we
> > were using another code review system (like Gerrit), this problem
> > wouldn't exist.
> >
> > We can stop rebasing PRs after releases, but it *is* overall going to
> > make life more difficult for the project's maintainers and
> > contributors -- we know this based on what has occurred in the past.
> > The current policy is the most pragmatic policy, even if it is not the
> > most technically pure.
> >
> > >2. stop pushing to others' PRs without their explicit consent
> >
> > Since I've edited people's branches to fix problems or address minor
> > issues countless times, this would create a lot of hardship for me and
> > other maintainers. This is another pragmatism vs. technical purity
> > question.
> >
> > One possible middle ground here is to create a "disallow-list" for
> > contributors who don't want their PRs edited by tools.
> >
> > On Wed, Nov 25, 2020 at 4:33 AM Jörn Horstmann
> >  wrote:
> > >
> > > Is there more documentation why a force push to master is necessary?
> > >
> > > I read
> > >
> >
> https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide#ReleaseManagementGuide-Mainsourcereleaseandvote
> > > which says
> > >
> > > - The local release branch has some unpushed commits such as bumping to
> > the
> > > next snapshot version.
> > > - So you need to add these commits to the master branch.
> > > - This needs force push.
> > >
> > > The first two sound logical, but why couldn't those version bumps be a
> > > merge commit into master?
> > >
> > > If this is somehow required only by the java/mvn setup, I'd be happy to
> > > help with that. I haven't worked on the java part of arrow yet, but
> have
> > > good experience with java and maven.
> > >
> > > Cheers,
> > > Jörn
> > >
> > >
> > > On Wed, Nov 25, 2020 at 10:11 AM Uwe L. Korn  wrote:
> > >
> > > > Hello Jorge,
> > > >
> > > > I know from the past on the Python/C++ side, we needed to do this
> for a
> > > > lot of contributors to enable them to work with their branches/PRs
> > again as
> > > > they were overwhelmed with the complexity of these rebases.
> > Personally, I
> > > > wouldn't 

Re: [Governance] [Proposal] Stop force-pushing to PRs after release?

[Jacques Nadeau]

I'm catching up here. A couple questions.

   - I don't think we should require the inclusion of the release commits
   in the main branch. Having leafs created right before release seems to
   simplify this and resolve any issues around force PRs, no? Or maybe I'm
   misunderstanding something? Maybe to put another way, why do we have to
   force push anything to main ever? That seems like the real problem.
   - I'm not sure I follow the comments about the java release process
   requiring any interactions with the Apache repo or a particular branch. In
   the past (note, it has been a while), I used mvn properties to override any
   of this behavior so that I might do any pushes to a personal repo rather
   than the Apache one (or an apache release branch rather than the main
   branch).

- The local release branch has some unpushed commits such as bumping to the
> next snapshot version.


Most projects I've worked on work this way:

We decide to do a release. At that point, we create a branch for the
release. In the main branch, we bump the version up immediately.
In the release branch, we do minimal commits as needed for hotfixes etc
(and all cherry pick to main).
We do release steps and push those commits (excluding the post-release
version bump) to the release branch.
We vote on the release commit and artifacts.
If there is a release failure, we revert any release commits in the release
branch and then apply addiitonal fixes.
Then do the process again.

This way main never has a force push.

(Full disclosure, the one exception I've worked on is that Calcite uses
force pushes as well as process-based branch locking. I think we should
avoid both of those patterns).

I'd also clarify, while I'm against merge commits, I think that a release
branch that forks slightly off the main branch should be fine. Do people
disagree with this?



On Wed, Nov 25, 2020 at 5:52 AM Wes McKinney  wrote:

> > The first two sound logical, but why couldn't those version bumps be a
> merge commit into master?
>
> We've made the commitment to maintaining a linear commit history in
> this project.
>
> Auto-rebasing the PRs at this point is best described as "harm
> reduction". The root cause is GitHub's UI which shows a bunch of
> unrelated commits in the UI when a PR is based on a commit that no
> longer exists due to the rebase. This can confuse people and sometimes
> cause them to engage in incorrect actions (trying to "git merge" in
> various ways) that cause their branches to get very messed up. If we
> were using another code review system (like Gerrit), this problem
> wouldn't exist.
>
> We can stop rebasing PRs after releases, but it *is* overall going to
> make life more difficult for the project's maintainers and
> contributors -- we know this based on what has occurred in the past.
> The current policy is the most pragmatic policy, even if it is not the
> most technically pure.
>
> >2. stop pushing to others' PRs without their explicit consent
>
> Since I've edited people's branches to fix problems or address minor
> issues countless times, this would create a lot of hardship for me and
> other maintainers. This is another pragmatism vs. technical purity
> question.
>
> One possible middle ground here is to create a "disallow-list" for
> contributors who don't want their PRs edited by tools.
>
> On Wed, Nov 25, 2020 at 4:33 AM Jörn Horstmann
>  wrote:
> >
> > Is there more documentation why a force push to master is necessary?
> >
> > I read
> >
> https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide#ReleaseManagementGuide-Mainsourcereleaseandvote
> > which says
> >
> > - The local release branch has some unpushed commits such as bumping to
> the
> > next snapshot version.
> > - So you need to add these commits to the master branch.
> > - This needs force push.
> >
> > The first two sound logical, but why couldn't those version bumps be a
> > merge commit into master?
> >
> > If this is somehow required only by the java/mvn setup, I'd be happy to
> > help with that. I haven't worked on the java part of arrow yet, but have
> > good experience with java and maven.
> >
> > Cheers,
> > Jörn
> >
> >
> > On Wed, Nov 25, 2020 at 10:11 AM Uwe L. Korn  wrote:
> >
> > > Hello Jorge,
> > >
> > > I know from the past on the Python/C++ side, we needed to do this for a
> > > lot of contributors to enable them to work with their branches/PRs
> again as
> > > they were overwhelmed with the complexity of these rebases.
> Personally, I
> > > wouldn't like to spend much time on whether we should rebase PRs after
> the
> > > release for everyone or not but actually get rid of the need to push to
> > > master to actually get a release candidate out. This is making the
> work of
> > > the release manager harder, confuses downstream packagers and also
> leads to
> > > the fact that all PRs are diverged when they were touched during the
> > > release process.
> > >
> > > The main headache here is that currently 

Re: [Governance] [Proposal] Stop force-pushing to PRs after release?

[Wes McKinney]

Note that at any time you can change your GitHub settings to disallow
your branches from being edited by maintainers.

On Wed, Nov 25, 2020 at 7:51 AM Wes McKinney  wrote:
>
> > The first two sound logical, but why couldn't those version bumps be a
> merge commit into master?
>
> We've made the commitment to maintaining a linear commit history in
> this project.
>
> Auto-rebasing the PRs at this point is best described as "harm
> reduction". The root cause is GitHub's UI which shows a bunch of
> unrelated commits in the UI when a PR is based on a commit that no
> longer exists due to the rebase. This can confuse people and sometimes
> cause them to engage in incorrect actions (trying to "git merge" in
> various ways) that cause their branches to get very messed up. If we
> were using another code review system (like Gerrit), this problem
> wouldn't exist.
>
> We can stop rebasing PRs after releases, but it *is* overall going to
> make life more difficult for the project's maintainers and
> contributors -- we know this based on what has occurred in the past.
> The current policy is the most pragmatic policy, even if it is not the
> most technically pure.
>
> >2. stop pushing to others' PRs without their explicit consent
>
> Since I've edited people's branches to fix problems or address minor
> issues countless times, this would create a lot of hardship for me and
> other maintainers. This is another pragmatism vs. technical purity
> question.
>
> One possible middle ground here is to create a "disallow-list" for
> contributors who don't want their PRs edited by tools.
>
> On Wed, Nov 25, 2020 at 4:33 AM Jörn Horstmann
>  wrote:
> >
> > Is there more documentation why a force push to master is necessary?
> >
> > I read
> > https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide#ReleaseManagementGuide-Mainsourcereleaseandvote
> > which says
> >
> > - The local release branch has some unpushed commits such as bumping to the
> > next snapshot version.
> > - So you need to add these commits to the master branch.
> > - This needs force push.
> >
> > The first two sound logical, but why couldn't those version bumps be a
> > merge commit into master?
> >
> > If this is somehow required only by the java/mvn setup, I'd be happy to
> > help with that. I haven't worked on the java part of arrow yet, but have
> > good experience with java and maven.
> >
> > Cheers,
> > Jörn
> >
> >
> > On Wed, Nov 25, 2020 at 10:11 AM Uwe L. Korn  wrote:
> >
> > > Hello Jorge,
> > >
> > > I know from the past on the Python/C++ side, we needed to do this for a
> > > lot of contributors to enable them to work with their branches/PRs again 
> > > as
> > > they were overwhelmed with the complexity of these rebases. Personally, I
> > > wouldn't like to spend much time on whether we should rebase PRs after the
> > > release for everyone or not but actually get rid of the need to push to
> > > master to actually get a release candidate out. This is making the work of
> > > the release manager harder, confuses downstream packagers and also leads 
> > > to
> > > the fact that all PRs are diverged when they were touched during the
> > > release process.
> > >
> > > The main headache here is that currently the release tooling on the Java
> > > side requires us to do this. I know that in the last days someone opened a
> > > JIRA to get rid of that (and hopefully someone will link to that JIRA in
> > > this thread). Solving that would be a win for all and also make this
> > > discussion unnecessary. The main caveat is that the annoyance on the Java
> > > side pops up mostly with the non-Java devs and thus it was not solved yet.
> > >
> > > Cheers
> > > Uwe
> > >
> > > On Wed, Nov 25, 2020, at 5:26 AM, Jorge Cardoso Leitão wrote:
> > > > Hi,
> > > >
> > > > Based on a discussion on PR #8481, I would like to raise a concern 
> > > > around
> > > > git and the post-actions of a release. The background is that I was
> > > really
> > > > confused that someone has force-pushed to a PR that I fielded, 
> > > > re-writing
> > > > its history and causing the PR to break.
> > > >
> > > > @wes and @kszucs quickly explained to me that this is a normal practice
> > > in
> > > > this project on every release, to which I was a bit astonished.
> > > >
> > > > AFAIK, in open source, there is a strong expectation that PRs are 
> > > > managed
> > > > by individual contributors, and committers of the project only request
> > > > contributors to make changes to them, or kindly ask before pushing (not
> > > > force-pushing) directly to the PR.
> > > >
> > > > IMO, by force-pushing to PRs, we are inverting all expectations and
> > > > sometimes even breaking PRs without consent from the contributor. This
> > > > drives any reasonable contributor to be pissed off at the team for what
> > > we
> > > > just did after a release:
> > > >
> > > >- force-pushed to master
> > > >- force-pushed to their PRs
> > > >- broke their PRs's CI
> > > > 

Re: [Governance] [Proposal] Stop force-pushing to PRs after release?

[Wes McKinney]

> The first two sound logical, but why couldn't those version bumps be a
merge commit into master?

We've made the commitment to maintaining a linear commit history in
this project.

Auto-rebasing the PRs at this point is best described as "harm
reduction". The root cause is GitHub's UI which shows a bunch of
unrelated commits in the UI when a PR is based on a commit that no
longer exists due to the rebase. This can confuse people and sometimes
cause them to engage in incorrect actions (trying to "git merge" in
various ways) that cause their branches to get very messed up. If we
were using another code review system (like Gerrit), this problem
wouldn't exist.

We can stop rebasing PRs after releases, but it *is* overall going to
make life more difficult for the project's maintainers and
contributors -- we know this based on what has occurred in the past.
The current policy is the most pragmatic policy, even if it is not the
most technically pure.

>2. stop pushing to others' PRs without their explicit consent

Since I've edited people's branches to fix problems or address minor
issues countless times, this would create a lot of hardship for me and
other maintainers. This is another pragmatism vs. technical purity
question.

One possible middle ground here is to create a "disallow-list" for
contributors who don't want their PRs edited by tools.

On Wed, Nov 25, 2020 at 4:33 AM Jörn Horstmann
 wrote:
>
> Is there more documentation why a force push to master is necessary?
>
> I read
> https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide#ReleaseManagementGuide-Mainsourcereleaseandvote
> which says
>
> - The local release branch has some unpushed commits such as bumping to the
> next snapshot version.
> - So you need to add these commits to the master branch.
> - This needs force push.
>
> The first two sound logical, but why couldn't those version bumps be a
> merge commit into master?
>
> If this is somehow required only by the java/mvn setup, I'd be happy to
> help with that. I haven't worked on the java part of arrow yet, but have
> good experience with java and maven.
>
> Cheers,
> Jörn
>
>
> On Wed, Nov 25, 2020 at 10:11 AM Uwe L. Korn  wrote:
>
> > Hello Jorge,
> >
> > I know from the past on the Python/C++ side, we needed to do this for a
> > lot of contributors to enable them to work with their branches/PRs again as
> > they were overwhelmed with the complexity of these rebases. Personally, I
> > wouldn't like to spend much time on whether we should rebase PRs after the
> > release for everyone or not but actually get rid of the need to push to
> > master to actually get a release candidate out. This is making the work of
> > the release manager harder, confuses downstream packagers and also leads to
> > the fact that all PRs are diverged when they were touched during the
> > release process.
> >
> > The main headache here is that currently the release tooling on the Java
> > side requires us to do this. I know that in the last days someone opened a
> > JIRA to get rid of that (and hopefully someone will link to that JIRA in
> > this thread). Solving that would be a win for all and also make this
> > discussion unnecessary. The main caveat is that the annoyance on the Java
> > side pops up mostly with the non-Java devs and thus it was not solved yet.
> >
> > Cheers
> > Uwe
> >
> > On Wed, Nov 25, 2020, at 5:26 AM, Jorge Cardoso Leitão wrote:
> > > Hi,
> > >
> > > Based on a discussion on PR #8481, I would like to raise a concern around
> > > git and the post-actions of a release. The background is that I was
> > really
> > > confused that someone has force-pushed to a PR that I fielded, re-writing
> > > its history and causing the PR to break.
> > >
> > > @wes and @kszucs quickly explained to me that this is a normal practice
> > in
> > > this project on every release, to which I was a bit astonished.
> > >
> > > AFAIK, in open source, there is a strong expectation that PRs are managed
> > > by individual contributors, and committers of the project only request
> > > contributors to make changes to them, or kindly ask before pushing (not
> > > force-pushing) directly to the PR.
> > >
> > > IMO, by force-pushing to PRs, we are inverting all expectations and
> > > sometimes even breaking PRs without consent from the contributor. This
> > > drives any reasonable contributor to be pissed off at the team for what
> > we
> > > just did after a release:
> > >
> > >- force-pushed to master
> > >- force-pushed to their PRs
> > >- broke their PRs's CI
> > >- no prior notice or request of any of the above
> > >
> > > IMO this is confusing and goes against what virtually every open source
> > > project does. This process also puts a lot of strain in our CIs when we
> > > have an average of 100 open PRs and force-push to all of them at once.
> > >
> > > As such, I would like to propose a small change in the post-release
> > process
> > > and to the development practices 

Re: [Governance] [Proposal] Stop force-pushing to PRs after release?

[Jörn Horstmann]

Is there more documentation why a force push to master is necessary?

I read
https://cwiki.apache.org/confluence/display/ARROW/Release+Management+Guide#ReleaseManagementGuide-Mainsourcereleaseandvote
which says

- The local release branch has some unpushed commits such as bumping to the
next snapshot version.
- So you need to add these commits to the master branch.
- This needs force push.

The first two sound logical, but why couldn't those version bumps be a
merge commit into master?

If this is somehow required only by the java/mvn setup, I'd be happy to
help with that. I haven't worked on the java part of arrow yet, but have
good experience with java and maven.

Cheers,
Jörn


On Wed, Nov 25, 2020 at 10:11 AM Uwe L. Korn  wrote:

> Hello Jorge,
>
> I know from the past on the Python/C++ side, we needed to do this for a
> lot of contributors to enable them to work with their branches/PRs again as
> they were overwhelmed with the complexity of these rebases. Personally, I
> wouldn't like to spend much time on whether we should rebase PRs after the
> release for everyone or not but actually get rid of the need to push to
> master to actually get a release candidate out. This is making the work of
> the release manager harder, confuses downstream packagers and also leads to
> the fact that all PRs are diverged when they were touched during the
> release process.
>
> The main headache here is that currently the release tooling on the Java
> side requires us to do this. I know that in the last days someone opened a
> JIRA to get rid of that (and hopefully someone will link to that JIRA in
> this thread). Solving that would be a win for all and also make this
> discussion unnecessary. The main caveat is that the annoyance on the Java
> side pops up mostly with the non-Java devs and thus it was not solved yet.
>
> Cheers
> Uwe
>
> On Wed, Nov 25, 2020, at 5:26 AM, Jorge Cardoso Leitão wrote:
> > Hi,
> >
> > Based on a discussion on PR #8481, I would like to raise a concern around
> > git and the post-actions of a release. The background is that I was
> really
> > confused that someone has force-pushed to a PR that I fielded, re-writing
> > its history and causing the PR to break.
> >
> > @wes and @kszucs quickly explained to me that this is a normal practice
> in
> > this project on every release, to which I was a bit astonished.
> >
> > AFAIK, in open source, there is a strong expectation that PRs are managed
> > by individual contributors, and committers of the project only request
> > contributors to make changes to them, or kindly ask before pushing (not
> > force-pushing) directly to the PR.
> >
> > IMO, by force-pushing to PRs, we are inverting all expectations and
> > sometimes even breaking PRs without consent from the contributor. This
> > drives any reasonable contributor to be pissed off at the team for what
> we
> > just did after a release:
> >
> >- force-pushed to master
> >- force-pushed to their PRs
> >- broke their PRs's CI
> >- no prior notice or request of any of the above
> >
> > IMO this is confusing and goes against what virtually every open source
> > project does. This process also puts a lot of strain in our CIs when we
> > have an average of 100 open PRs and force-push to all of them at once.
> >
> > As such, I would like to propose a small change in the post-release
> process
> > and to the development practices more generally:
> >
> >1. stop force-pushing to others' PRs
> >2. stop pushing to others' PRs without their explicit consent
> >3. document in the contributing guidelines
> >
> >that master is force-pushed on every release, and the steps that
> >contributors need to take to bring their PRs to the latest master
> >
> > The underlying principles here are:
> >
> >- it is the contributor's responsibility to keep the PRs in a "ready
> to
> >merge" state, rebasing them to master as master changes.
> >- A force-push to master corresponds to a change in master
> >- thus it is the contributor's responsibility to rebase their PRs
> >against the new master.
> >
> > I understand the argument that it is a burden for the contributors to
> keep
> > PRs up-to-date. However, I do not think that this justifies breaking one
> of
> > the most basic assumptions that a contributor has on an open source
> > project. Furthermore, they already have to do it anyways whenever the
> > master changes with breaking changes: the contributor's process is
> already
> > "git fetch upstream && git rebase upstream/master" whenever master
> changes.
> > Whether it changes due to a normal push or a force-push does not really
> > affect this burden when compared to when a merge conflict emerges.
> >
> > Any thoughts?
> > Best,
> > Jorge
> >
>


-- 
*Jörn Horstmann* | Senior Backend Engineer

www.signavio.com
Kurfürstenstraße 111, 10787 Berlin, Germany

Work with us! 

Re: [Governance] [Proposal] Stop force-pushing to PRs after release?

[Uwe L. Korn]

Hello Jorge,

I know from the past on the Python/C++ side, we needed to do this for a lot of 
contributors to enable them to work with their branches/PRs again as they were 
overwhelmed with the complexity of these rebases. Personally, I wouldn't like 
to spend much time on whether we should rebase PRs after the release for 
everyone or not but actually get rid of the need to push to master to actually 
get a release candidate out. This is making the work of the release manager 
harder, confuses downstream packagers and also leads to the fact that all PRs 
are diverged when they were touched during the release process. 

The main headache here is that currently the release tooling on the Java side 
requires us to do this. I know that in the last days someone opened a JIRA to 
get rid of that (and hopefully someone will link to that JIRA in this thread). 
Solving that would be a win for all and also make this discussion unnecessary. 
The main caveat is that the annoyance on the Java side pops up mostly with the 
non-Java devs and thus it was not solved yet.

Cheers
Uwe

On Wed, Nov 25, 2020, at 5:26 AM, Jorge Cardoso Leitão wrote:
> Hi,
> 
> Based on a discussion on PR #8481, I would like to raise a concern around
> git and the post-actions of a release. The background is that I was really
> confused that someone has force-pushed to a PR that I fielded, re-writing
> its history and causing the PR to break.
> 
> @wes and @kszucs quickly explained to me that this is a normal practice in
> this project on every release, to which I was a bit astonished.
> 
> AFAIK, in open source, there is a strong expectation that PRs are managed
> by individual contributors, and committers of the project only request
> contributors to make changes to them, or kindly ask before pushing (not
> force-pushing) directly to the PR.
> 
> IMO, by force-pushing to PRs, we are inverting all expectations and
> sometimes even breaking PRs without consent from the contributor. This
> drives any reasonable contributor to be pissed off at the team for what we
> just did after a release:
> 
>- force-pushed to master
>- force-pushed to their PRs
>- broke their PRs's CI
>- no prior notice or request of any of the above
> 
> IMO this is confusing and goes against what virtually every open source
> project does. This process also puts a lot of strain in our CIs when we
> have an average of 100 open PRs and force-push to all of them at once.
> 
> As such, I would like to propose a small change in the post-release process
> and to the development practices more generally:
> 
>1. stop force-pushing to others' PRs
>2. stop pushing to others' PRs without their explicit consent
>3. document in the contributing guidelines
>
>that master is force-pushed on every release, and the steps that
>contributors need to take to bring their PRs to the latest master
> 
> The underlying principles here are:
> 
>- it is the contributor's responsibility to keep the PRs in a "ready to
>merge" state, rebasing them to master as master changes.
>- A force-push to master corresponds to a change in master
>- thus it is the contributor's responsibility to rebase their PRs
>against the new master.
> 
> I understand the argument that it is a burden for the contributors to keep
> PRs up-to-date. However, I do not think that this justifies breaking one of
> the most basic assumptions that a contributor has on an open source
> project. Furthermore, they already have to do it anyways whenever the
> master changes with breaking changes: the contributor's process is already
> "git fetch upstream && git rebase upstream/master" whenever master changes.
> Whether it changes due to a normal push or a force-push does not really
> affect this burden when compared to when a merge conflict emerges.
> 
> Any thoughts?
> Best,
> Jorge
>

[Governance] [Proposal] Stop force-pushing to PRs after release?

[Jorge Cardoso Leitão]

Hi,

Based on a discussion on PR #8481, I would like to raise a concern around
git and the post-actions of a release. The background is that I was really
confused that someone has force-pushed to a PR that I fielded, re-writing
its history and causing the PR to break.

@wes and @kszucs quickly explained to me that this is a normal practice in
this project on every release, to which I was a bit astonished.

AFAIK, in open source, there is a strong expectation that PRs are managed
by individual contributors, and committers of the project only request
contributors to make changes to them, or kindly ask before pushing (not
force-pushing) directly to the PR.

IMO, by force-pushing to PRs, we are inverting all expectations and
sometimes even breaking PRs without consent from the contributor. This
drives any reasonable contributor to be pissed off at the team for what we
just did after a release:

   - force-pushed to master
   - force-pushed to their PRs
   - broke their PRs's CI
   - no prior notice or request of any of the above

IMO this is confusing and goes against what virtually every open source
project does. This process also puts a lot of strain in our CIs when we
have an average of 100 open PRs and force-push to all of them at once.

As such, I would like to propose a small change in the post-release process
and to the development practices more generally:

   1. stop force-pushing to others' PRs
   2. stop pushing to others' PRs without their explicit consent
   3. document in the contributing guidelines
   
   that master is force-pushed on every release, and the steps that
   contributors need to take to bring their PRs to the latest master

The underlying principles here are:

   - it is the contributor's responsibility to keep the PRs in a "ready to
   merge" state, rebasing them to master as master changes.
   - A force-push to master corresponds to a change in master
   - thus it is the contributor's responsibility to rebase their PRs
   against the new master.

I understand the argument that it is a burden for the contributors to keep
PRs up-to-date. However, I do not think that this justifies breaking one of
the most basic assumptions that a contributor has on an open source
project. Furthermore, they already have to do it anyways whenever the
master changes with breaking changes: the contributor's process is already
"git fetch upstream && git rebase upstream/master" whenever master changes.
Whether it changes due to a normal push or a force-push does not really
affect this burden when compared to when a merge conflict emerges.

Any thoughts?
Best,
Jorge