Alright let's keep things this way at least for now so that we can move forward with the Spark approach.
Max On Tue, Aug 28, 2018 at 10:39 AM Bolke de Bruin <bdbr...@gmail.com> wrote: > I’m not in favor of moving to GitHub issues. While JIRA is not perfect it > actually moves discussion to the mailing list. With GitHub issues the stuff > just gets lost imho. > > I do like our changelogs a lot better now. > > B. > > Verstuurd vanaf mijn iPad > > > Op 27 aug. 2018 om 05:53 heeft Holden Karau <hol...@pigscanfly.ca> het > volgende geschreven: > > > > Awesome, so I'll give this a shot with JIRA for now and if we end up > moving to GH we can move it over. I've got a fork of the dashboard I've > been using in Apache Beam as well as the Spark one so it shouldn't take me > too long to generalize it again. > > > >> On Sun, Aug 26, 2018 at 8:50 PM Maxime Beauchemin < > maximebeauche...@gmail.com> wrote: > >> I love the idea. I took a quick look at /databricks/spark-pr-dashboard > >> <https://github.com/databricks/spark-pr-dashboard> and that looks like > a > >> nice easy start. It would be interesting to try to make a generic > project > >> out of it that would work on top of any project/repo. Basically just > >> refactor all of the Spark-specific code and configurations into a > >> `config.py` (assuming there's not too much frontend Spark-specific > >> code...). > >> > >> Of course this tool assumes Jira+Github which works for Airflow, but > >> probably isn't as common of a setup to really generalize beyond Apache. > It > >> seems like by embracing Github issues and dropping Jira we could be > >> building something much more relevant. > >> > >> Max > >> > >> On Fri, Aug 24, 2018 at 7:27 PM Holden Karau <hol...@pigscanfly.ca> > wrote: > >> > >> > Update: we can do this with the dashboard code. Since this would > modify > >> > the JIRAs I’d love a sign-off on turning that feature on from someone > on > >> > the PMC (or at least a week with no PMC folks saying no). > >> > > >> > On Thu, Aug 23, 2018 at 8:00 AM Holden Karau <hol...@pigscanfly.ca> > wrote: > >> > > >> >> I mean a few ASF projects update JIRA tickets based on PRs > automatically. > >> >> Switching from JIRA to GH issues (or back) is super painful, so I’d > >> >> probably do more incremental improvements personally, I just don’t > have the > >> >> time to do something like that. > >> >> > >> >> I’ll take a look at some the K8s tools (over in Beam they’re looking > at > >> >> one of the review tagging tools out of K8s) if the Spark one is too > >> >> difficult to adapt to our use case. > >> >> > >> >> On Thu, Aug 23, 2018 at 2:34 AM Eamon Keane <eamon.kea...@gmail.com> > >> >> wrote: > >> >> > >> >>> Kubernetes is a good place to look as they've invested a lot in > github > >> >>> bots > >> >>> and label based workflows. E.g. the cherry-picking script and doc is > >> >>> here: > >> >>> > >> >>> > >> >>> > https://github.com/kubernetes/kubernetes/blob/master/hack/cherry_pick_pull.sh > >> >>> > >> >>> > https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md > >> >>> > >> >>> And more general overview here: > >> >>> > >> >>> > https://github.com/kubernetes/community/tree/master/contributors/devel > >> >>> > >> >>> On Thu, Aug 23, 2018 at 5:11 AM Maxime Beauchemin < > >> >>> maximebeauche...@gmail.com> wrote: > >> >>> > >> >>> > I've heard many times in the past about a GH/Jira syncing tool but > >> >>> never > >> >>> > seen it in action. Personally my vote is to move issues to GH and > drop > >> >>> > Jira. Though in the process this will break the release helper > script > >> >>> here: > >> >>> > > >> >>> > https://github.com/apache/incubator-airflow/blob/master/dev/airflow-jira > >> >>> > > >> >>> > We'll be working on a Github label-driven release baking magic > script > >> >>> for > >> >>> > Superset, maybe we could use the same tooling on both Airflow and > >> >>> Superset. > >> >>> > The idea is that the script would use labels like > `target:apache-1.11` > >> >>> on > >> >>> > PRs to bake releases. The tool would take as input a base SHA and > >> >>> release > >> >>> > minor release number, and would craft a release branch, fetch and > >> >>> > cherry-pick all the right commits in the right order based on > labels, > >> >>> > generate release tags (on minor versions) and output state into > >> >>> > release-info files (listing the base, all cherries, all tags, > ...). The > >> >>> > tricky part is resolving merge conflicts while auto-picking > cherries, > >> >>> but > >> >>> > the script would guide the operator through it . > >> >>> > > >> >>> > Curious to hear about how other projects do it. I think it > generally > >> >>> > involves a lot of manual work. Let me know if you know of open > source > >> >>> > tooling to deal with release management. > >> >>> > > >> >>> > Max > >> >>> > > >> >>> > On Wed, Aug 22, 2018 at 6:25 PM Holden Karau < > holden.ka...@gmail.com> > >> >>> > wrote: > >> >>> > > >> >>> > > Thanks for the reminder, forgot to ask at coffee but I'll ask. > >> >>> > > > >> >>> > > On Wed, Aug 22, 2018, 1:52 AM Driesprong, Fokko > <fo...@driesprong.frl > >> >>> > > >> >>> > > wrote: > >> >>> > > > >> >>> > > > Hi Holden, > >> >>> > > > > >> >>> > > > Just curious if you got a hold of someone at the coffee > machine :-) > >> >>> > > > > >> >>> > > > Cheers, Fokko > >> >>> > > > > >> >>> > > > Op di 7 aug. 2018 om 09:17 schreef Holden Karau < > >> >>> hol...@pigscanfly.ca > >> >>> > >: > >> >>> > > > > >> >>> > > > > The JIRA/Github integration tooling I’m a little more fuzzy > on > >> >>> but > >> >>> > I’m > >> >>> > > > > doing coffee with some of the folks who probably know the > details > >> >>> > this > >> >>> > > > week > >> >>> > > > > and I’ll report back. > >> >>> > > > > > >> >>> > > > > On Tue, Aug 7, 2018 at 12:15 AM Driesprong, Fokko > >> >>> > <fo...@driesprong.frl > >> >>> > > > > >> >>> > > > > wrote: > >> >>> > > > > > >> >>> > > > > > Hi Holden, > >> >>> > > > > > > >> >>> > > > > > Thanks for reaching out. Recently we've moved to Apache > Gitbox > >> >>> ( > >> >>> > > > > > https://gitbox.apache.org/), so we use the Github UI > directly > >> >>> > > instead > >> >>> > > > of > >> >>> > > > > > having to merge using a CLI ( > >> >>> > > > > > > >> >>> > > > >> >>> > https://github.com/apache/incubator-airflow/blob/master/dev/airflow-pr > >> >>> > > > ). > >> >>> > > > > > > >> >>> > > > > > Not sure if we're already up to the game of the dashboard, > >> >>> which > >> >>> > > looks > >> >>> > > > > > awesome btw. But as you also mentioned in your live > Airflow PR, > >> >>> > we're > >> >>> > > > > > missing some automation of communication between Jira and > >> >>> Github. > >> >>> > For > >> >>> > > > > > example, as you mentioned, when a PR is opened, the status > >> >>> > > > automagically > >> >>> > > > > > changes to In Progress. Do you have any pointer of how > this is > >> >>> set > >> >>> > up > >> >>> > > > at, > >> >>> > > > > > for example the Spark or Beam project? So we can replicate > >> >>> this in > >> >>> > > > > Airflow. > >> >>> > > > > > > >> >>> > > > > > Cheers, Fokko > >> >>> > > > > > > >> >>> > > > > > 2018-08-07 7:28 GMT+02:00 Holden Karau < > holden.ka...@gmail.com > >> >>> >: > >> >>> > > > > > > >> >>> > > > > > > Hi Y'all, > >> >>> > > > > > > > >> >>> > > > > > > One of the comments from my livestream was asking if the > >> >>> code for > >> >>> > > the > >> >>> > > > > > Spark > >> >>> > > > > > > PR review dashboard <http://spark-prs.appspot.com/> is > OSS > >> >>> (it > >> >>> > is > >> >>> > > > > > > <https://github.com/databricks/spark-pr-dashboard>), > and I > >> >>> have > >> >>> > a > >> >>> > > > fork > >> >>> > > > > > up > >> >>> > > > > > > for Beam, and I was wondering if folks in Airflow would > find > >> >>> > > > something > >> >>> > > > > > like > >> >>> > > > > > > this useful? If so I'd be happy to set that up (if not > no > >> >>> > stress). > >> >>> > > > > > > > >> >>> > > > > > > Cheers, > >> >>> > > > > > > > >> >>> > > > > > > Holden :) > >> >>> > > > > > > > >> >>> > > > > > > -- > >> >>> > > > > > > Cell : 425-233-8271 > >> >>> > > > > > > > >> >>> > > > > > > >> >>> > > > > -- > >> >>> > > > > Twitter: https://twitter.com/holdenkarau > >> >>> > > > > > >> >>> > > > > >> >>> > > > >> >>> > > >> >>> > >> >> -- > >> >> Twitter: https://twitter.com/holdenkarau > >> >> Books (Learning Spark, High Performance Spark, etc.): > >> >> https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9> > >> >> YouTube Live Streams: https://www.youtube.com/user/holdenkarau > >> >> > >> > -- > >> > Twitter: https://twitter.com/holdenkarau > >> > Books (Learning Spark, High Performance Spark, etc.): > >> > https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9> > >> > YouTube Live Streams: https://www.youtube.com/user/holdenkarau > >> > > > > > > > -- > > Twitter: https://twitter.com/holdenkarau > > Books (Learning Spark, High Performance Spark, etc.): > https://amzn.to/2MaRAG9 > > YouTube Live Streams: https://www.youtube.com/user/holdenkarau >
