Awesome, so I'll give this a shot with JIRA for now and if we end up moving to GH we can move it over. I've got a fork of the dashboard I've been using in Apache Beam as well as the Spark one so it shouldn't take me too long to generalize it again.
On Sun, Aug 26, 2018 at 8:50 PM Maxime Beauchemin < maximebeauche...@gmail.com> wrote: > I love the idea. I took a quick look at /databricks/spark-pr-dashboard > <https://github.com/databricks/spark-pr-dashboard> and that looks like a > nice easy start. It would be interesting to try to make a generic project > out of it that would work on top of any project/repo. Basically just > refactor all of the Spark-specific code and configurations into a > `config.py` (assuming there's not too much frontend Spark-specific > code...). > > Of course this tool assumes Jira+Github which works for Airflow, but > probably isn't as common of a setup to really generalize beyond Apache. It > seems like by embracing Github issues and dropping Jira we could be > building something much more relevant. > > Max > > On Fri, Aug 24, 2018 at 7:27 PM Holden Karau <hol...@pigscanfly.ca> wrote: > > > Update: we can do this with the dashboard code. Since this would modify > > the JIRAs I’d love a sign-off on turning that feature on from someone on > > the PMC (or at least a week with no PMC folks saying no). > > > > On Thu, Aug 23, 2018 at 8:00 AM Holden Karau <hol...@pigscanfly.ca> > wrote: > > > >> I mean a few ASF projects update JIRA tickets based on PRs > automatically. > >> Switching from JIRA to GH issues (or back) is super painful, so I’d > >> probably do more incremental improvements personally, I just don’t have > the > >> time to do something like that. > >> > >> I’ll take a look at some the K8s tools (over in Beam they’re looking at > >> one of the review tagging tools out of K8s) if the Spark one is too > >> difficult to adapt to our use case. > >> > >> On Thu, Aug 23, 2018 at 2:34 AM Eamon Keane <eamon.kea...@gmail.com> > >> wrote: > >> > >>> Kubernetes is a good place to look as they've invested a lot in github > >>> bots > >>> and label based workflows. E.g. the cherry-picking script and doc is > >>> here: > >>> > >>> > >>> > https://github.com/kubernetes/kubernetes/blob/master/hack/cherry_pick_pull.sh > >>> > >>> > https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md > >>> > >>> And more general overview here: > >>> > >>> https://github.com/kubernetes/community/tree/master/contributors/devel > >>> > >>> On Thu, Aug 23, 2018 at 5:11 AM Maxime Beauchemin < > >>> maximebeauche...@gmail.com> wrote: > >>> > >>> > I've heard many times in the past about a GH/Jira syncing tool but > >>> never > >>> > seen it in action. Personally my vote is to move issues to GH and > drop > >>> > Jira. Though in the process this will break the release helper script > >>> here: > >>> > > >>> > https://github.com/apache/incubator-airflow/blob/master/dev/airflow-jira > >>> > > >>> > We'll be working on a Github label-driven release baking magic script > >>> for > >>> > Superset, maybe we could use the same tooling on both Airflow and > >>> Superset. > >>> > The idea is that the script would use labels like > `target:apache-1.11` > >>> on > >>> > PRs to bake releases. The tool would take as input a base SHA and > >>> release > >>> > minor release number, and would craft a release branch, fetch and > >>> > cherry-pick all the right commits in the right order based on labels, > >>> > generate release tags (on minor versions) and output state into > >>> > release-info files (listing the base, all cherries, all tags, ...). > The > >>> > tricky part is resolving merge conflicts while auto-picking cherries, > >>> but > >>> > the script would guide the operator through it . > >>> > > >>> > Curious to hear about how other projects do it. I think it generally > >>> > involves a lot of manual work. Let me know if you know of open source > >>> > tooling to deal with release management. > >>> > > >>> > Max > >>> > > >>> > On Wed, Aug 22, 2018 at 6:25 PM Holden Karau <holden.ka...@gmail.com > > > >>> > wrote: > >>> > > >>> > > Thanks for the reminder, forgot to ask at coffee but I'll ask. > >>> > > > >>> > > On Wed, Aug 22, 2018, 1:52 AM Driesprong, Fokko > <fo...@driesprong.frl > >>> > > >>> > > wrote: > >>> > > > >>> > > > Hi Holden, > >>> > > > > >>> > > > Just curious if you got a hold of someone at the coffee machine > :-) > >>> > > > > >>> > > > Cheers, Fokko > >>> > > > > >>> > > > Op di 7 aug. 2018 om 09:17 schreef Holden Karau < > >>> hol...@pigscanfly.ca > >>> > >: > >>> > > > > >>> > > > > The JIRA/Github integration tooling I’m a little more fuzzy on > >>> but > >>> > I’m > >>> > > > > doing coffee with some of the folks who probably know the > details > >>> > this > >>> > > > week > >>> > > > > and I’ll report back. > >>> > > > > > >>> > > > > On Tue, Aug 7, 2018 at 12:15 AM Driesprong, Fokko > >>> > <fo...@driesprong.frl > >>> > > > > >>> > > > > wrote: > >>> > > > > > >>> > > > > > Hi Holden, > >>> > > > > > > >>> > > > > > Thanks for reaching out. Recently we've moved to Apache > Gitbox > >>> ( > >>> > > > > > https://gitbox.apache.org/), so we use the Github UI > directly > >>> > > instead > >>> > > > of > >>> > > > > > having to merge using a CLI ( > >>> > > > > > > >>> > > > >>> https://github.com/apache/incubator-airflow/blob/master/dev/airflow-pr > >>> > > > ). > >>> > > > > > > >>> > > > > > Not sure if we're already up to the game of the dashboard, > >>> which > >>> > > looks > >>> > > > > > awesome btw. But as you also mentioned in your live Airflow > PR, > >>> > we're > >>> > > > > > missing some automation of communication between Jira and > >>> Github. > >>> > For > >>> > > > > > example, as you mentioned, when a PR is opened, the status > >>> > > > automagically > >>> > > > > > changes to In Progress. Do you have any pointer of how this > is > >>> set > >>> > up > >>> > > > at, > >>> > > > > > for example the Spark or Beam project? So we can replicate > >>> this in > >>> > > > > Airflow. > >>> > > > > > > >>> > > > > > Cheers, Fokko > >>> > > > > > > >>> > > > > > 2018-08-07 7:28 GMT+02:00 Holden Karau < > holden.ka...@gmail.com > >>> >: > >>> > > > > > > >>> > > > > > > Hi Y'all, > >>> > > > > > > > >>> > > > > > > One of the comments from my livestream was asking if the > >>> code for > >>> > > the > >>> > > > > > Spark > >>> > > > > > > PR review dashboard <http://spark-prs.appspot.com/> is OSS > >>> (it > >>> > is > >>> > > > > > > <https://github.com/databricks/spark-pr-dashboard>), and I > >>> have > >>> > a > >>> > > > fork > >>> > > > > > up > >>> > > > > > > for Beam, and I was wondering if folks in Airflow would > find > >>> > > > something > >>> > > > > > like > >>> > > > > > > this useful? If so I'd be happy to set that up (if not no > >>> > stress). > >>> > > > > > > > >>> > > > > > > Cheers, > >>> > > > > > > > >>> > > > > > > Holden :) > >>> > > > > > > > >>> > > > > > > -- > >>> > > > > > > Cell : 425-233-8271 > >>> > > > > > > > >>> > > > > > > >>> > > > > -- > >>> > > > > Twitter: https://twitter.com/holdenkarau > >>> > > > > > >>> > > > > >>> > > > >>> > > >>> > >> -- > >> Twitter: https://twitter.com/holdenkarau > >> Books (Learning Spark, High Performance Spark, etc.): > >> https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9> > >> YouTube Live Streams: https://www.youtube.com/user/holdenkarau > >> > > -- > > Twitter: https://twitter.com/holdenkarau > > Books (Learning Spark, High Performance Spark, etc.): > > https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9> > > YouTube Live Streams: https://www.youtube.com/user/holdenkarau > > > -- Twitter: https://twitter.com/holdenkarau Books (Learning Spark, High Performance Spark, etc.): https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
