Re: Multiple Schedulers - "scheduler_lock"

2019-03-29 Thread Kevin Yang
Sry I'm late again :) I agree with most of Bas's idea especially if we decided to do HA maybe don't do it ourselves( Apache Helix? haven't look deep just an idea). Peter, I'm very curious about your setup and how come it took 2-3 hours to process 5000+ DAG files( I assume you meant DAG files not

Re: Hangout: Learn how to hack on Airflow (CHANGE OF DAY)

2019-03-29 Thread Kevin Yang
Thanks Ash, love to see the recording +1 Cheers, Kevin Y On Fri, Mar 29, 2019 at 6:46 PM Chao-Han Tsai wrote: > Would love to see the recording! > > > On Fri, Mar 29, 2019 at 5:29 AM Jarek Potiuk > wrote: > > > Thanks Ash, It was great to watch your developing Ariflow yesterday! > > > > I cou

Re: Hangout: Learn how to hack on Airflow (CHANGE OF DAY)

2019-03-29 Thread Chao-Han Tsai
Would love to see the recording! On Fri, Mar 29, 2019 at 5:29 AM Jarek Potiuk wrote: > Thanks Ash, It was great to watch your developing Ariflow yesterday! > > I could not stay till the very end so I would love to see the recording :) > > J. > > On Thu, Mar 28, 2019 at 5:34 PM Austin Bennett <

Re: [RFC] Prototype for an Airflow landing page

2019-03-29 Thread Pablo Estrada
Hello all, it's been a few days, and we've been busy ourselves with other work, but I was wondering if anyone is willing to share particular feedback on this iteration / prototype of the landing page for airflow? We're looking for feedback on the following areas: 1. Look and feel 2. Organization

Re: API Reference - current confusion and improvement plan

2019-03-29 Thread Jiajie Zhong
Awesome Ash ??, expecting beta2. And don't forget some minor bug still in beta1 https://issues.apache.org/jira/browse/AIRFLOW-3623?focusedCommentId=16803866&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16803866 Best wish. -- Jiajie _

Re: API Reference - current confusion and improvement plan

2019-03-29 Thread Ash Berlin-Taylor
It took pulling in about another 30 commits to get it without conflicts but I've pulled this in to the v1-10-stable branch so it will be in the 1.10.3 too! (There are 68 commits to the branch already since 1.10.3b1. Time for a beta2 I think) -a > On 29 Mar 2019, at 13:28, Jiajie Zhong wrote:

Re: API Reference - current confusion and improvement plan

2019-03-29 Thread Jiajie Zhong
Thanks Kamil, really a great change in out documentation Best wish. -- Jiajie From: Driesprong, Fokko Sent: Friday, March 29, 2019 19:16 To: dev@airflow.apache.org Subject: Re: API Reference - current confusion and improvement plan Awesome work Kamil. Thanks fo

Re: Hangout: Learn how to hack on Airflow (CHANGE OF DAY)

2019-03-29 Thread Jarek Potiuk
Thanks Ash, It was great to watch your developing Ariflow yesterday! I could not stay till the very end so I would love to see the recording :) J. On Thu, Mar 28, 2019 at 5:34 PM Austin Bennett wrote: > Would be great if recorded/available later -- have a meeting during the > 'live' time. > >

Re: API Reference - current confusion and improvement plan

2019-03-29 Thread Driesprong, Fokko
Awesome work Kamil. Thanks for giving some love to the documentation. It really needed some :-) Don't forget to remove the line from the Github template: When adding new operators/hooks/sensors, the autoclass documentation generation needs to be added. https://github.com/apache/airflow/blob/master