Thanks, Michelle. The reliance on cherry-picking as part of a work flow can definitely bring the process into conflict with other tools and processes in the git ecosystem. I've left a fairly lengthy comment on SIP-12 proposing that the mechanics be handled using gitflow, which does not use cherry-picking, just for discussion. https://github.com/apache/incubator-superset/issues/6131#issuecomment-475691949
On Wed, Mar 20, 2019 at 1:04 PM Michelle Thomas <michelle.tho...@airbnb.com.invalid> wrote: > To answer the question about using a workflow for managing > releases/branches/tags, I previously evaluated using conventional commits: > https://www.conventionalcommits.org/en/v1.0.0-beta.3/, which is a system > for tagging commits to generate the changelog. At the time it seemed like > it may have issues creating the Changelog with the way we commit everything > to master then cherry-pick fixes onto a branch, but the consistency in > tagging PRs with fix, feature, docs seemed useful. I don't think we've > heavily discussed other workflows. Sounds good to get suggestions for > improvement on SIP-12. > > On Tue, Mar 19, 2019 at 9:27 PM David Smith <dave.a.sm...@gmail.com> > wrote: > > > I'm new to this project, so I apologize if this has been discussed > > previously, but... Have we considered using a widely adopted workflow for > > managing releases/branches/tags, and using tools that the community has > > already built for them? For example Git Flow or some variant thereof? > For > > anyone who is unfamiliar, see overview and tooling--in the form of git > > extensions--here: https://github.com/nvie/gitflow > > > > I saw SIP-12 and have a few comments that I will try to document tomorrow > > on that issue, but I'm really curious whether other common > processes/flows > > were rejected explicitly or if they just didn't come up? > > > > Dave > > > > > > > > On Tue, Mar 19, 2019 at 8:45 PM Maxime Beauchemin < > > maximebeauche...@gmail.com> wrote: > > > > > Wondering what to do next, Justin, is it ok for me to push the related > > tag > > > to Github? > > > > > > Should I trigger a [VOTE] thread? > > > > > > Max > > > > > > On Tue, Mar 19, 2019 at 12:04 AM Maxime Beauchemin < > > > maximebeauche...@gmail.com> wrote: > > > > > > > My svn is a bit rusty but I made some progress tonight: > > > > https://github.com/apache/incubator-superset/pull/7054 > > > > > > > > Pushed some files to SVN > > > > > https://dist.apache.org/repos/dist/dev/incubator/superset/0.31.0rc18/ > > > > > > > > It's not intended to be an actual real RC, but it's a start. > > > > > > > > An early question is around RC numbers. Lyft & Airbnb have been > doing a > > > > lot of back and forth on the release branch already and `0.31.0` is > > > already > > > > up to rc18. I was internally debating what to do here, keeping the > > > > numbering scheme, or starting anew. Probably the right thing to do is > > to > > > > pick a number (say 0.32.0) and start clean at 0.32.0rc1, and stick to > > the > > > > Apache process, and use continuous numbers. I'm assuming the handoff > is > > > at > > > > 0.31.0rc18. > > > > > > > > Now people that would want to work on non-Apache release branches > would > > > do > > > > so in their fork, and in their own namespace. Maybe their local build > > > > instructions can become recipes for an upcoming release, but I > suggest > > > that > > > > this happens in forks from now on. > > > > > > > > Clearly this Apache process here with signing + svn commit + voting > is > > > > more involved than the usual git cherry-pick + git tag + git push, so > > > that > > > > should probably lead to less RCs (I can't imagine doing 18 votes > > around a > > > > single release). > > > > > > > > Not to mention the fact that the real complexity around releasing is > > > > raising and labeling issues with a proper target version, gather PRs, > > > > cherry-pick fixes and resolve conflicts if any. I'm hoping we can > build > > > > tooling to help with all this. Hugh started something a while back, > but > > > > there's lots to be done still in that area. > > > > > > > > Max > > > > > > > > On Mon, Mar 18, 2019 at 11:39 PM Maxime Beauchemin < > > > > maximebeauche...@gmail.com> wrote: > > > > > > > >> Not officially approved yet, I think we should start a [DISCUSS] > > thread > > > >> and eventually [VOTE] > > > >> > > > >> On Mon, Mar 18, 2019 at 10:13 PM David Smith < > dave.a.sm...@gmail.com> > > > >> wrote: > > > >> > > > >>> Thanks Max! Out of curiosity, has that release SIP-12 been approved > > > yet? > > > >>> I > > > >>> have some thoughts but if it is already a done deal I'll wait until > > > >>> another > > > >>> time. :-) > > > >>> > > > >>> On Mon, Mar 18, 2019 at 9:53 PM Maxime Beauchemin < > > > >>> maximebeauche...@gmail.com> wrote: > > > >>> > > > >>> > Hi all, > > > >>> > > > > >>> > I wanted to send an email explaining the current state of > releases > > > and > > > >>> > start a thread about what's ahead. > > > >>> > > > > >>> > Early on in the project lifecycle, I used to package releases and > > > push > > > >>> to > > > >>> > Pypi without much process. I'd package internal releases > internally > > > for > > > >>> > Airbnb. We'd roll out to staging, stabilize the release, and > launch > > > in > > > >>> > production. After a some time without major issues and > regressions, > > > I'd > > > >>> > push a new release out to Pypi and make it available to the > > > community. > > > >>> That > > > >>> > process worked ok when I reviewed or wrote most PRs and had a > > handle > > > on > > > >>> > everything that was going on. The community has grown much, and > > > clearly > > > >>> > that process has not been appropriate for quite some time now. > > > >>> > > > > >>> > Later on, we joined the Apache Software Foundation's incubator, > and > > > >>> the ASF > > > >>> > has clear process around releases > > > >>> > <http://www.apache.org/dev/release-publishing.html>. As an ASF > > > >>> project, it > > > >>> > was not reasonable for us to push to Pypi until we sorted out the > > > >>> process > > > >>> > and followed the "Apache Way". Since `0.29` we've still been > > cutting > > > >>> > release branches and labeling "release candidates", but have > > > refrained > > > >>> from > > > >>> > publishing to Pypi. Those release branches contain commits that > > have > > > >>> been > > > >>> > put into production at Lyft and/or Airbnb and elsewhere, but > didn't > > > >>> meet > > > >>> > the ASF's standards, so could not be published as releases. > > > >>> > > > > >>> > Also with the growing community, we clearly need a more > structured > > > >>> process > > > >>> > around releases. John Bodley wrote a solid SIP > > > >>> > <https://github.com/apache/incubator-superset/issues/6131> > > > proposing a > > > >>> > clear structure and process around releases, and details much of > > what > > > >>> the > > > >>> > ASF release process does not. > > > >>> > > > > >>> > All this to say, *I'm starting some work on our first > ASF-approved > > > >>> source > > > >>> > code release*. For context, the ASF makes a distinction between > > > "source > > > >>> > code releases" and "convenience releases". The former contains > the > > > >>> source > > > >>> > code and build instructions, and the latter contains binaries and > > is > > > >>> made > > > >>> > available in places like Pypi (or Maven / npm / RubyGem / ...). > > > >>> > > > > >>> > While the source code release should be pretty straight forward, > > it's > > > >>> not > > > >>> > really, well, convenient. You can't just "pip install" that... > The > > > >>> > convenience release on the other hand is great, but may require > > more > > > >>> work > > > >>> > and guidance from our mentors, ASF legal, license experts as the > > > >>> binaries > > > >>> > may contain our Javascript bundles (or maybe they don't?) but > then > > > >>> have to > > > >>> > have a gigantic LICENSE.txt file detailing the licenses of the > 500+ > > > >>> > Javascript libs that have accumulated in our `package-lock.json` > > over > > > >>> time. > > > >>> > Or maybe we ship without the bundles and add a `superset compile` > > > >>> command > > > >>> > that does the fetching/build of the JS packages. Unclear to me. > > I'll > > > >>> need > > > >>> > help figuring this out. For context here's some work I did trying > > to > > > >>> > automate the LICENSE.txt creation > > > >>> > <https://github.com/apache/incubator-superset/pull/5801> > > > >>> > > > > >>> > Anyhow, wanted people to know what's up since it's been so long > > since > > > >>> the > > > >>> > last Pypi release, and give the opportunity for contributors to > > jump > > > >>> in and > > > >>> > help out. > > > >>> > > > > >>> > Let's get the [community] release process going! > > > >>> > > > > >>> > Max > > > >>> > > > > >>> > > > >> > > > > > >
