Re: [DISCUSS] AIP-5: Remote DAG Fetcher

2019-04-30 Thread Chao-Han Tsai
Hi all, I would like to bump this thread up again to seek for more feedbacks from you for the AIP-5: https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-5+Remote+DAG+Fetcher In short, this AIP aims to enable Airflow to fetch DAG files from remote storage systems (git, s3,...) outside the

Re: [ANNOUNCE] Please welcome new Airflow committer Kamil Breguła

2019-04-30 Thread Jiajie Zhong
Congratulations Kamil ! Best wish. -- Jiajie From: Kaxil Naik Sent: Wednesday, May 1, 2019 5:38 To: dev@airflow.apache.org Subject: Re: [ANNOUNCE] Please welcome new Airflow committer Kamil Breguła Congratulations Kamil On Tue, Apr 30, 2019 at 10:38 PM Chao-Han

Re: [ANNOUNCE] Please welcome new Airflow committer Kevin Yang

2019-04-30 Thread Yingbo Wang
Congrats Kevin! On Tue, Apr 30, 2019 at 6:43 PM Deng Xiaodong wrote: > Congrats Kevin! > > > XD > > On Wed, May 1, 2019 at 1:09 AM Daniel Imberman < > dimberman.opensou...@gmail.com> wrote: > > > Congrats Kevin! > > > > On Tue, Apr 30, 2019 at 9:09 AM Aizhamal Nurmamat kyzy > > wrote: > > > >

Re: [ANNOUNCE] Please welcome new Airflow committer Kevin Yang

2019-04-30 Thread Deng Xiaodong
Congrats Kevin! XD On Wed, May 1, 2019 at 1:09 AM Daniel Imberman < dimberman.opensou...@gmail.com> wrote: > Congrats Kevin! > > On Tue, Apr 30, 2019 at 9:09 AM Aizhamal Nurmamat kyzy > wrote: > > > Congratulations Kevin! > > > > On Tue, Apr 30, 2019 at 02:06 Jiajie Zhong > > wrote: > > > >

[Proposal] Component changes for Apache Airflow

2019-04-30 Thread Aizhamal Nurmamat kyzy
Hello everyone, I would like to propose a few changes for the Apache Airflow JIRA. The reason behind this proposal is that the set of components is disorganized, and it could use some improvements to track the status of the project and improve the Jira triage. I outlined all the proposed changes

Re: [ANNOUNCE] Please welcome new Airflow committer Kevin Yang

2019-04-30 Thread Chao-Han Tsai
Congrats Kevin! On Tue, Apr 30, 2019 at 11:01 AM Alex Guziel wrote: > Congratulations Kevin! > > On Tue, Apr 30, 2019 at 10:58 AM Tao Feng wrote: > > > Congrats! > > > > On Tue, Apr 30, 2019 at 10:09 AM Daniel Imberman < > > dimberman.opensou...@gmail.com> wrote: > > > > > Congrats Kevin! > >

Re: [ANNOUNCE] Please welcome new Airflow committer Kamil Breguła

2019-04-30 Thread Kaxil Naik
Congratulations Kamil On Tue, Apr 30, 2019 at 10:38 PM Chao-Han Tsai wrote: > Congratulations Kamil! > > On Tue, Apr 30, 2019 at 11:54 AM Aizhamal Nurmamat kyzy > wrote: > > > This is great news! Congratulations Kamil! And thank you for all your > > contributions and caring so much for

Re: [ANNOUNCE] Please welcome new Airflow committer Kamil Breguła

2019-04-30 Thread Chao-Han Tsai
Congratulations Kamil! On Tue, Apr 30, 2019 at 11:54 AM Aizhamal Nurmamat kyzy wrote: > This is great news! Congratulations Kamil! And thank you for all your > contributions and caring so much for documentation <3 :) > > On Tue, Apr 30, 2019 at 11:43 AM Tao Feng wrote: > > > All, > > > > I'm

Airflow Question

2019-04-30 Thread Andy Srine
Team, Is there a users@ mailing list for Apache Airflow? If not, is this where I submit product questions to? Let me do it anyways. I am new to Airflow and I just deployed it on Ubuntu. I have also deployed Postgres as it's DB and Celery. I started the scheduler and the web server. The UI comes

Re: merging of airflow pull request

2019-04-30 Thread Daniel Imberman
I'm rerunning it now to see if the failing test will pass. If it does I'll merge it. On Tue, Apr 30, 2019 at 12:31 PM Ashwini Adiga wrote: > Hi Devs, > > I would like to know the update on this pull request. > https://github.com/apache/airflow/pull/4583, > This has been approved by ashb on

Re: [ANNOUNCE] Please welcome new Airflow committer Kamil Breguła

2019-04-30 Thread Aizhamal Nurmamat kyzy
This is great news! Congratulations Kamil! And thank you for all your contributions and caring so much for documentation <3 :) On Tue, Apr 30, 2019 at 11:43 AM Tao Feng wrote: > All, > > I'm pleased to announce that the Airflow PMC has voted in Kamil Breguła to > be > a committer. Kamil has

Re: [ANNOUNCE] Please welcome new Airflow committer Kamil Breguła

2019-04-30 Thread Jarek Potiuk
Welcome Kamil! Principal Software Engineer Phone: +48660796129 wt., 30 kwi 2019, 20:43 użytkownik Tao Feng napisał: > All, > > I'm pleased to announce that the Airflow PMC has voted in Kamil Breguła to > be > a committer. Kamil has been actively involved in the project for some time > now,

[ANNOUNCE] Please welcome new Airflow committer Kamil Breguła

2019-04-30 Thread Tao Feng
All, I'm pleased to announce that the Airflow PMC has voted in Kamil Breguła to be a committer. Kamil has been actively involved in the project for some time now, contributing patches and participating in the mailing list as well as AIP discussion. He spends lots of efforts improving the

Re: [DISCUSS] AIRFLOW-4192 - remove duplicate/obsolete/derived task context variables

2019-04-30 Thread Bas Harenslak
Nope. I never planned to have a discussion on execution_date but it appeared more important than the other variables mentioned in my first email :) I already wrote a summary halfway this thread. With the updates since then: * yesterday_ds, yesterday_ds_nodash, tomorrow_ds,

Season Of Docs 2019 - ibrahim-fathy

2019-04-30 Thread ibrahim Fathy
Hi I'm Ibrahim. I'm interested in your organization and would like to participate in SoD. Can you add me to Slack to talk more! my email: ibrahim_fa...@outlook.com Waiting for your invitation. :)

Re: [DISCUSS] AIRFLOW-4192 - remove duplicate/obsolete/derived task context variables

2019-04-30 Thread Ash Berlin-Taylor
Did we come to any conclusion on this topic? > On 12 Apr 2019, at 10:26, Ash Berlin-Taylor wrote: > > Does anyone actually use end_date (of either spelling) given it's value is > currently the same as `ds`: >

Re: [ANNOUNCE] Please welcome new Airflow committer Kevin Yang

2019-04-30 Thread Daniel Imberman
Congrats Kevin! On Tue, Apr 30, 2019 at 9:09 AM Aizhamal Nurmamat kyzy wrote: > Congratulations Kevin! > > On Tue, Apr 30, 2019 at 02:06 Jiajie Zhong > wrote: > > > Congratulate Kevin > > > > Best wish. > > -- Jiajie > > > > From: Jarek Potiuk > > Sent:

Re: Acceptance into Season of Docs 2019

2019-04-30 Thread aizhamal
On 2019/04/30 12:30:00, Andrew Chen wrote: > Congratulations! Your organization’s application for Season of Docs has > been accepted. > > > Thank you for your interest and enthusiasm for Season of Docs. We received > nearly 200 organization applications! There were many high quality >

Re: [PLEASE PARTICIPATE] Brainstorming ideas for Season of Docs

2019-04-30 Thread Aizhamal Nurmamat kyzy
Hello all! I am delighted to announce to the community that our application for Season of Docs got accepted! [1] 拾 Well done everybody! Now technical writers will look into our project ideas and submit their proposals. I’ll keep you updated in the dev list. Cheers, Aizhamal [1]

Re: [ANNOUNCE] Please welcome new Airflow committer Kevin Yang

2019-04-30 Thread Aizhamal Nurmamat kyzy
Congratulations Kevin! On Tue, Apr 30, 2019 at 02:06 Jiajie Zhong wrote: > Congratulate Kevin > > Best wish. > -- Jiajie > > From: Jarek Potiuk > Sent: Tuesday, April 30, 2019 16:52 > To: dev@airflow.apache.org > Subject: Re: [ANNOUNCE] Please welcome new

Re: Use -x when cherry-picking to v1-10 branches

2019-04-30 Thread Driesprong, Fokko
Good point Jarek, thanks for brining this up! Cheers, Fokko Op di 23 apr. 2019 om 01:23 schreef Deng Xiaodong : > Agree. > > XD > > On Tue, Apr 23, 2019 at 05:15 Kaxil Naik wrote: > > > Definitely, agreed. > > > > On Mon, Apr 22, 2019 at 9:28 PM Jarek Potiuk > > wrote: > > > > > Hello

Re: [ANNOUNCE] Please welcome new Airflow committer Kevin Yang

2019-04-30 Thread Jiajie Zhong
Congratulate Kevin Best wish. -- Jiajie From: Jarek Potiuk Sent: Tuesday, April 30, 2019 16:52 To: dev@airflow.apache.org Subject: Re: [ANNOUNCE] Please welcome new Airflow committer Kevin Yang Congratulate! J. On Tue, Apr 30, 2019 at 10:37 AM Driesprong,

Re: [ANNOUNCE] Please welcome new Airflow committer Kevin Yang

2019-04-30 Thread Jarek Potiuk
Congratulate! J. On Tue, Apr 30, 2019 at 10:37 AM Driesprong, Fokko wrote: > All, > > I'm pleased to announce that the Airflow PMC has voted in Kevin Yang to be > a committer. Kevin has been actively involved in the project for some time > now, contributing patches and participating in the

Re: Need more pairs of eyes for the flaky LocalExecutorTest fix

2019-04-30 Thread Ash Berlin-Taylor
Oh sorry - in the core Scheduler for the DAG parsing (inside jobs.py) not in the Celery Executor. > On 30 Apr 2019, at 08:57, Driesprong, Fokko wrote: > > Looking at the PR I only see them being used in the LocalExecutor and the > KubernetesExecutor, which makes sense. For CeleryExecutor we

[ANNOUNCE] Please welcome new Airflow committer Kevin Yang

2019-04-30 Thread Driesprong, Fokko
All, I'm pleased to announce that the Airflow PMC has voted in Kevin Yang to be a committer. Kevin has been actively involved in the project for some time now, contributing patches and participating in the mailing list. Implementing optimizations to the core of Apache Airflow to make it scale.

Re: Need more pairs of eyes for the flaky LocalExecutorTest fix

2019-04-30 Thread Driesprong, Fokko
Looking at the PR I only see them being used in the LocalExecutor and the KubernetesExecutor, which makes sense. For CeleryExecutor we have Celery in place which handles the callbacks for us, therefore we don't need to poll. Apart from that, I think this is an improvement over the current

Re: [ANNOUNCE] Please welcome new Airflow committer Joshua Carp

2019-04-30 Thread Chao-Han Tsai
Congrats Joshua! On Mon, Apr 29, 2019 at 11:20 PM Bas Harenslak < basharens...@godatadriven.com> wrote: > Congrats! > > Bas > > > On 30 Apr 2019, at 03:50, Chen Tong wrote: > > > > Congrats Joshua! > > > > Thank you! > > Chen Tong > > > > On Mon, Apr 29, 2019 at 9:23 PM Kevin Yang wrote: > > >

Re: [ANNOUNCE] Please welcome new Airflow committer Joshua Carp

2019-04-30 Thread Bas Harenslak
Congrats! Bas > On 30 Apr 2019, at 03:50, Chen Tong wrote: > > Congrats Joshua! > > Thank you! > Chen Tong > > On Mon, Apr 29, 2019 at 9:23 PM Kevin Yang wrote: > >> Congrats Joshua! >> >> Cheers, >> Kevin Y >> >> On Mon, Apr 29, 2019 at 4:40 PM Deng Xiaodong wrote: >> >>> Congrats