PRs can be opened for backports and cherry picks and you can have all of
the desired precommit reports available before any merging happens.
Publishing changes should be relatively atomic. (Is "relatively" better? I
mean on the same day.) The JIRAs that linger open in a half open state with
some commits on some branches is a bad practice that should be eliminated,
IMHO.
On Tue, Jan 19, 2021 at 8:21 AM Nick Dimiduk <ndimi...@apache.org> wrote:
> It sounds like we have rough consensus at least on removing the CHANGES and
> RELEASENOTES files from source control. I filed HBASE-25520 to track this
> work.
>
> > What is the common thing here? That the merge action is atomic! All are
> ready or none are ready! Don't stuff half done work in some branches
> leaving dangling open JIRAs and random PRs for other branches!
>
> I don't think the action of landing a patch can be atomic so long as we
> start with master and then backport. For example, I happen to prefer to use
> pre-commit checking of a PR on the backport to earlier branches. (Y'all
> should prefer this too -- it's why we have pre-commit checking!) This
> results in a workflow of (1) land PR vs master (2) cherry-pick commit to
> backport branch (3) publish PR vs backport branch (4) wait 4-24 hours for
> pre-commit check.
>
> In order to have something close to atomic commits AND get the
> peace-of-mind of a successful PR build vs. the backport branch(es), I would
> change the above workflow to prepare all backport PRs at once. Then I would
> only merge all of them once all backport PRs pass their checks.
>
> Then there's the issue of human error. People will make mistakes, post
> addendums ; need to re-open a ticket.
>
> In short, there are several reasons for a commit (even to a single branch)
> to be non-atomic. I think any process that doesn't account for an issue
> being open over the course of several days is not realistic.
>
> On Mon, Jan 18, 2021 at 4:55 PM Andrew Purtell <apurt...@apache.org>
> wrote:
>
> > Thank you Wellington. We do need for committers to pay attention to JIRA.
> > We still use JIRA for project management so we can't ignore it or allow
> it
> > to get out of sync with reality anyway. As long as we keep fix versions
> in
> > sync, we can continue to use today's create-release script and Yetus'
> > nice releasedocmaker tool for generating the change log and release
> notes.
> > (There is no substitute for these tools that use git history.)
> >
> > The release audit tool proposed on HBASE-22853 will also be really
> helpful,
> > but it is orthogonal to committers maintaining JIRA discipline. If
> > committers don't do that, the audit tool will give the RM a lot of
> warnings
> > that the RM will have to spend time cleaning up. If committers maintain
> > good JIRA discipline, the audit tool will quickly confirm the consistency
> > of JIRA fix versions and release branch commit history and the RM will
> > continue to have a pleasant day.
> >
> > My take is this:
> >
> > You have a change in mind. You will also have a target branch in mind -
> > master and/or branch-2 and/or branch-1 - depending on when and where you
> > need it to end up.
> >
> > Develop your change for your target branch. Remember that changes are
> > committed back in order like so: master -> branch-2 ( -> maybe
> branch-2.x
> > ( -> maybe branch-1 ) ). So if you are planning to commit to branch-2.4,
> > we will need PRs or patches for master, branch-2, and branch-2.4. Prepare
> > all of these patches before raising the PRs!
> >
> > Want to commit only to master? Open PR for master, merge, set fix
> version {
> > 3.0.0-alpha }, resolve, done!
> >
> > Want to commit to branch-2?: Open PR for master, cherry pick to branch-2
> > (or PR for branch-2), merge/commit all at once, set fix version {
> > 3.0.0-alpha, 2.5.0 }, resolve, done!
> >
> > Want to commit to branch-2.4? Open PR for master, cherry pick to branch-2
> > (or PR for branch-2), cherry pick to branch-2.4 (or PR for branch-2.4),
> > merge/commit all at once, set fix versions { 3.0.0-alpha, 2.5.0, 2.4.2 },
> > resolve, done!
> >
> > Want to commit to all branch-2.x? Open PR for master, cherry pick to
> > branch-2 (or PR for branch-2), cherry pick to branch-2.4 (or PR for
> > branch-2.4), cherry pick to branch-2.3 (or PR for branch-2.3),
> merge/commit
> > all at once, set fix versions { 3.0.0-alpha, 2.5.0, 2.4.2, 2.3.7 },
> > resolve, done!
> >
> > What is the common thing here? That the merge action is atomic! All are
> > ready or none are ready! Don't stuff half done work in some branches
> > leaving dangling open JIRAs and random PRs for other branches!
> >
> > If your target branch is branch-1, that can be a problem, because
> > backporting to branch-1 can be difficult. (Related: I think this year I
> > will propose EOL of branch-1, maybe in December.) So you might want to
> > select branch-2 as your initial target branch, do the above, and after
> > resolving the JIRA open a new JIRA to track the branch-1 backport.
> >
> >
> >
> > On Mon, Jan 18, 2021 at 2:32 AM Wellington Chevreuil <
> > wellington.chevre...@gmail.com> wrote:
> >
> > > While no perfect script/tool is available to solve this problem, we may
> > > agree on a preferable commiting process to make RM's life less
> difficult.
> > > My take from this thread is that something like below would help:
> > >
> > > 1) Once a commit happens on any branch, resolve the jira and update
> > > fixVersions field with value corresponding the committed branch;
> > > 2) Open sub-task jiras for each subsequent commit on different
> branches,
> > > repeating the step #1 above on each of these subtasks;
> > > - How about commit messages on these subtask jiras? Should it mention
> > both
> > > parent and subtask jira ids? Or maybe just parent id, considering most
> > > commits would be simple cherry-picks?
> > > 3) If a commit is reverted, related jira should be reopened and
> > fixVersions
> > > field should be cleaned up.
> > >
> > >
> > >
> > > Em seg., 18 de jan. de 2021 às 02:18, 张铎(Duo Zhang) <
> > palomino...@gmail.com
> > > >
> > > escreveu:
> > >
> > > > What about a Jenkins job for detecting the diff between jira and git?
> > > >
> > > > And I agree that, let's not include CHANGES.md and RELEASENOTES.md in
> > our
> > > > code base directly which means we need to sink an RC only if these
> two
> > > > files are incorrect, not the actual code.
> > > >
> > > > Andrew Purtell <apurt...@apache.org> 于2021年1月16日周六 上午11:11写道：
> > > >
> > > > > I think the discussion has already moved forward a lot with mention
> > of
> > > > > scripts I did not know existed. One well developed git<->jira diff
> > > tool,
> > > > > documented as part of the RM procedure, would be much better than
> we
> > > have
> > > > > right now, with various RMs doing ad hoc things.
> > > > >
> > > > >
> > > > > On Fri, Jan 15, 2021 at 6:04 PM 张铎(Duo Zhang) <
> palomino...@gmail.com
> > >
> > > > > wrote:
> > > > >
> > > > > > Haven't fully read all the comments but I think this is the duty
> > for
> > > > the
> > > > > > release manager? This is what I use to get the difference between
> > > jira
> > > > > and
> > > > > > git
> > > > > >
> > > > > > git log --oneline
> > > > 0167558eb31ff48308d592ef70b6d005ba6d21fb...3ee8b0c75b |
> > > > > > grep -o -E "^[0-9a-z]*\s*HBASE-[0-9]*"|awk '{print $2}' | sort
> -u >
> > > > > > in_git_2.0.0.txt
> > > > > > cat CHANGELOG.md | grep -o -E "\[HBASE-[0-9]*\]" | grep -o -E
> > > > > > "HBASE-[0-9]*" | sort -u > CHANGELOG_jiras.txt
> > > > > >
> > > > > > And then you could use comm to get the difference.
> > > > > >
> > > > > > We all make mistakes so this process can not be automatic, trust
> > me.
> > > > > > Sometimes a commit is reverted and never applied again, sometimes
> > we
> > > > > > include the wrong jira id in commit message.
> > > > > > This is why I always use a jira issue to land the CHANGES and
> > > > > RELEASENOTES,
> > > > > > tag it, and then use the release script to generate the RC.
> > > > > >
> > > > > > For me, it will cost me a lot of time when creating a new minor
> > > release
> > > > > > because there will be a lot of commits, but for a patch release
> > > usually
> > > > > it
> > > > > > is fine.
> > > > > >
> > > > > > Maybe things will be easier if we move all the development
> process
> > to
> > > > > > github?
> > > > > >
> > > > > > Thanks.
> > > > > >
> > > > > > Sean Busbey <bus...@apache.org> 于2021年1月16日周六 上午8:05写道：
> > > > > >
> > > > > > > What would the logistics look like for git as the source of
> > truth?
> > > > > > >
> > > > > > > Every release candidate I've ever done I've had to do the jira
> > and
> > > > git
> > > > > > > reconciliation. There are always errors in both. Folks commit
> > stuff
> > > > > with
> > > > > > > the wrong JIRA ID or wrong message.
> > > > > > >
> > > > > > > Would reconciliation for a RM then require revert and reapply
> of
> > > any
> > > > > such
> > > > > > > changes?
> > > > > > >
> > > > > > > On Fri, Jan 15, 2021, 17:54 Stack <st...@duboce.net> wrote:
> > > > > > >
> > > > > > > > On Fri, Jan 15, 2021 at 10:06 AM Andrew Purtell <
> > > > apurt...@apache.org
> > > > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > I have now had to sink two 2.4.1 RCs because of errors in
> the
> > > > > change
> > > > > > > log.
> > > > > > > > >
> > > > > > > > > I made a pass over git history and ensured every commit was
> > > > > > included. I
> > > > > > > > had
> > > > > > > > > also made a pass over JIRA to move out any unresolved
> issues
> > or
> > > > > > > complete
> > > > > > > > > the resolution of same. What I did not do is check that
> every
> > > > > > resolved
> > > > > > > > JIRA
> > > > > > > > > corresponded to an actual commit. This is not something RMs
> > > have
> > > > > had
> > > > > > to
> > > > > > > > do
> > > > > > > > > in the past and it asks a lot of them.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > Just to say, that making RCs, I've the done the JIRA<->GIT
> > > > > > reconciliation
> > > > > > > > described (ugly hack scripting, sorting, and comparing).
> Others
> > > > have
> > > > > > too
> > > > > > > > I've noticed. It is awful, yes, especially when issues like
> > those
> > > > > found
> > > > > > > by
> > > > > > > > Viraj in yours and Huaxiangs RCs this week. The refguide is
> > vague
> > > > on
> > > > > > what
> > > > > > > > is involved; it needs more detail. The issue HBASE-22853
> > > > description
> > > > > on
> > > > > > > the
> > > > > > > > need for a tool to do the reconcile is a bit better.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > > I know NOW that as RM I cannot currently trust committers
> to
> > > get
> > > > > fix
> > > > > > > > > versions right or care about this.
> > > > > > > > >
> > > > > > > > > That's right... Commtters cannot be trusted to correctly
> > > maintain
> > > > > > issue
> > > > > > > > > metadata in JIRA.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > Yes (says a key offender).
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > > That is not a good situation for the project to be in. Up
> > until
> > > > now
> > > > > > it
> > > > > > > > has
> > > > > > > > > not been the responsibility of the RM to check each and
> every
> > > > JIRA
> > > > > > > > status.
> > > > > > > > > It has been the collective responsibility of committers to
> > care
> > > > > about
> > > > > > > the
> > > > > > > > > project's release tracking insofar as to correctly update
> fix
> > > > > > versions
> > > > > > > in
> > > > > > > > > JIRA. For releases containing relatively few changes, like
> > > 2.4.1,
> > > > > > with
> > > > > > > > ~50
> > > > > > > > > changes, I suppose it is possible for the RM to remove all
> > > 2.4.1
> > > > > fix
> > > > > > > > > versions, walk the commit history, and set back fix
> versions
> > on
> > > > > JIRA
> > > > > > to
> > > > > > > > > actually correspond with what was truly committed. However,
> > for
> > > > > minor
> > > > > > > > > releases, with hundreds of commits, this will not be
> > possible.
> > > > > > > > >
> > > > > > > > > I think the root cause is GitHub and JIRA are two separate
> > > change
> > > > > > > > tracking
> > > > > > > > > systems with only a minimal amount of integration. It
> > requires
> > > > > manual
> > > > > > > > > effort. More and more, new committers are familiar with
> > GitHub
> > > > and
> > > > > > PRs
> > > > > > > > and
> > > > > > > > > are not familiar with JIRA and the Apache way of using JIRA
> > to
> > > > > build
> > > > > > > > change
> > > > > > > > > logs. We need to better educate new and existing committers
> > on
> > > > > their
> > > > > > > > > responsibilities with regards to maintaining JIRA metadata
> > > > > correctly.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > Agree.
> > > > > > > >
> > > > > > > > Other issues are that the release process has been evolving.
> In
> > > the
> > > > > old
> > > > > > > > days, JIRA was the source of truth; changelog was a JIRA
> > report.
> > > > The
> > > > > > > > CHANGES.md/RELEASENOTES.md have come to the fore perhaps
> making
> > > > > > > difference
> > > > > > > > between JIRA and GIT more pronounced.
> > > > > > > >
> > > > > > > > I like the suggestion made by you above (and Nick in an
> > internal
> > > > > > > > discussion) that git be the source of truth. That the
> > CHANGES.md
> > > > NOT
> > > > > be
> > > > > > > > checked-in is also a good idea so it can be regenerated w/o
> > need
> > > > of a
> > > > > > > > change to the RC if JIRA is changed (I can work on this part
> if
> > > > > > wanted).
> > > > > > > >
> > > > > > > > S
> > > > > > > >
> > > > > > > >
> > > > > > > > > --
> > > > > > > > > Best regards,
> > > > > > > > > Andrew
> > > > > > > > >
> > > > > > > > > Words like orphans lost among the crosstalk, meaning torn
> > from
> > > > > > truth's
> > > > > > > > > decrepit hands
> > > > > > > > > - A23, Crosstalk
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Best regards,
> > > > > Andrew
> > > > >
> > > > > Words like orphans lost among the crosstalk, meaning torn from
> > truth's
> > > > > decrepit hands
> > > > > - A23, Crosstalk
> > > > >
> > > >
> > >
> >
> >
> > --
> > Best regards,
> > Andrew
> >
> > Words like orphans lost among the crosstalk, meaning torn from truth's
> > decrepit hands
> > - A23, Crosstalk
> >
>
--
Best regards,
Andrew
Words like orphans lost among the crosstalk, meaning torn from truth's
decrepit hands
- A23, Crosstalk
