To answer the question about using a workflow for managing releases/branches/tags, I previously evaluated using conventional commits: https://www.conventionalcommits.org/en/v1.0.0-beta.3/, which is a system for tagging commits to generate the changelog. At the time it seemed like it may have issues creating the Changelog with the way we commit everything to master then cherry-pick fixes onto a branch, but the consistency in tagging PRs with fix, feature, docs seemed useful. I don't think we've heavily discussed other workflows. Sounds good to get suggestions for improvement on SIP-12.
On Tue, Mar 19, 2019 at 9:27 PM David Smith <dave.a.sm...@gmail.com> wrote: > I'm new to this project, so I apologize if this has been discussed > previously, but... Have we considered using a widely adopted workflow for > managing releases/branches/tags, and using tools that the community has > already built for them? For example Git Flow or some variant thereof? For > anyone who is unfamiliar, see overview and tooling--in the form of git > extensions--here: https://github.com/nvie/gitflow > > I saw SIP-12 and have a few comments that I will try to document tomorrow > on that issue, but I'm really curious whether other common processes/flows > were rejected explicitly or if they just didn't come up? > > Dave > > > > On Tue, Mar 19, 2019 at 8:45 PM Maxime Beauchemin < > maximebeauche...@gmail.com> wrote: > > > Wondering what to do next, Justin, is it ok for me to push the related > tag > > to Github? > > > > Should I trigger a [VOTE] thread? > > > > Max > > > > On Tue, Mar 19, 2019 at 12:04 AM Maxime Beauchemin < > > maximebeauche...@gmail.com> wrote: > > > > > My svn is a bit rusty but I made some progress tonight: > > > https://github.com/apache/incubator-superset/pull/7054 > > > > > > Pushed some files to SVN > > > https://dist.apache.org/repos/dist/dev/incubator/superset/0.31.0rc18/ > > > > > > It's not intended to be an actual real RC, but it's a start. > > > > > > An early question is around RC numbers. Lyft & Airbnb have been doing a > > > lot of back and forth on the release branch already and `0.31.0` is > > already > > > up to rc18. I was internally debating what to do here, keeping the > > > numbering scheme, or starting anew. Probably the right thing to do is > to > > > pick a number (say 0.32.0) and start clean at 0.32.0rc1, and stick to > the > > > Apache process, and use continuous numbers. I'm assuming the handoff is > > at > > > 0.31.0rc18. > > > > > > Now people that would want to work on non-Apache release branches would > > do > > > so in their fork, and in their own namespace. Maybe their local build > > > instructions can become recipes for an upcoming release, but I suggest > > that > > > this happens in forks from now on. > > > > > > Clearly this Apache process here with signing + svn commit + voting is > > > more involved than the usual git cherry-pick + git tag + git push, so > > that > > > should probably lead to less RCs (I can't imagine doing 18 votes > around a > > > single release). > > > > > > Not to mention the fact that the real complexity around releasing is > > > raising and labeling issues with a proper target version, gather PRs, > > > cherry-pick fixes and resolve conflicts if any. I'm hoping we can build > > > tooling to help with all this. Hugh started something a while back, but > > > there's lots to be done still in that area. > > > > > > Max > > > > > > On Mon, Mar 18, 2019 at 11:39 PM Maxime Beauchemin < > > > maximebeauche...@gmail.com> wrote: > > > > > >> Not officially approved yet, I think we should start a [DISCUSS] > thread > > >> and eventually [VOTE] > > >> > > >> On Mon, Mar 18, 2019 at 10:13 PM David Smith <dave.a.sm...@gmail.com> > > >> wrote: > > >> > > >>> Thanks Max! Out of curiosity, has that release SIP-12 been approved > > yet? > > >>> I > > >>> have some thoughts but if it is already a done deal I'll wait until > > >>> another > > >>> time. :-) > > >>> > > >>> On Mon, Mar 18, 2019 at 9:53 PM Maxime Beauchemin < > > >>> maximebeauche...@gmail.com> wrote: > > >>> > > >>> > Hi all, > > >>> > > > >>> > I wanted to send an email explaining the current state of releases > > and > > >>> > start a thread about what's ahead. > > >>> > > > >>> > Early on in the project lifecycle, I used to package releases and > > push > > >>> to > > >>> > Pypi without much process. I'd package internal releases internally > > for > > >>> > Airbnb. We'd roll out to staging, stabilize the release, and launch > > in > > >>> > production. After a some time without major issues and regressions, > > I'd > > >>> > push a new release out to Pypi and make it available to the > > community. > > >>> That > > >>> > process worked ok when I reviewed or wrote most PRs and had a > handle > > on > > >>> > everything that was going on. The community has grown much, and > > clearly > > >>> > that process has not been appropriate for quite some time now. > > >>> > > > >>> > Later on, we joined the Apache Software Foundation's incubator, and > > >>> the ASF > > >>> > has clear process around releases > > >>> > <http://www.apache.org/dev/release-publishing.html>. As an ASF > > >>> project, it > > >>> > was not reasonable for us to push to Pypi until we sorted out the > > >>> process > > >>> > and followed the "Apache Way". Since `0.29` we've still been > cutting > > >>> > release branches and labeling "release candidates", but have > > refrained > > >>> from > > >>> > publishing to Pypi. Those release branches contain commits that > have > > >>> been > > >>> > put into production at Lyft and/or Airbnb and elsewhere, but didn't > > >>> meet > > >>> > the ASF's standards, so could not be published as releases. > > >>> > > > >>> > Also with the growing community, we clearly need a more structured > > >>> process > > >>> > around releases. John Bodley wrote a solid SIP > > >>> > <https://github.com/apache/incubator-superset/issues/6131> > > proposing a > > >>> > clear structure and process around releases, and details much of > what > > >>> the > > >>> > ASF release process does not. > > >>> > > > >>> > All this to say, *I'm starting some work on our first ASF-approved > > >>> source > > >>> > code release*. For context, the ASF makes a distinction between > > "source > > >>> > code releases" and "convenience releases". The former contains the > > >>> source > > >>> > code and build instructions, and the latter contains binaries and > is > > >>> made > > >>> > available in places like Pypi (or Maven / npm / RubyGem / ...). > > >>> > > > >>> > While the source code release should be pretty straight forward, > it's > > >>> not > > >>> > really, well, convenient. You can't just "pip install" that... The > > >>> > convenience release on the other hand is great, but may require > more > > >>> work > > >>> > and guidance from our mentors, ASF legal, license experts as the > > >>> binaries > > >>> > may contain our Javascript bundles (or maybe they don't?) but then > > >>> have to > > >>> > have a gigantic LICENSE.txt file detailing the licenses of the 500+ > > >>> > Javascript libs that have accumulated in our `package-lock.json` > over > > >>> time. > > >>> > Or maybe we ship without the bundles and add a `superset compile` > > >>> command > > >>> > that does the fetching/build of the JS packages. Unclear to me. > I'll > > >>> need > > >>> > help figuring this out. For context here's some work I did trying > to > > >>> > automate the LICENSE.txt creation > > >>> > <https://github.com/apache/incubator-superset/pull/5801> > > >>> > > > >>> > Anyhow, wanted people to know what's up since it's been so long > since > > >>> the > > >>> > last Pypi release, and give the opportunity for contributors to > jump > > >>> in and > > >>> > help out. > > >>> > > > >>> > Let's get the [community] release process going! > > >>> > > > >>> > Max > > >>> > > > >>> > > >> > > >
