RE: Manual raising of AirflowRescheduleException supported?

2019-05-09 Thread Shaw, Damian P.
Thanks Stefan! Works like a charm. In fact because reschedule_date can be another timezone it's now much easier to make DAGs where tasks are rescheduled against different timezones. My users are very happy. Regards, Damian -Original Message- From: Stefan Seelmann [mailto:m...@stefan-se

Re: Sharing experience

2019-05-09 Thread Aizhamal Nurmamat kyzy
+1 into adding it to Awesome-Airflow. Thank you Germain! *From: *Jarek Potiuk *Date: *Thu, May 9, 2019 at 13:48 *To: * Very interesting writeup :). Maybe you could make PR to > https://github.com/jghoman/awesome-apache-airflow ? > > J > > On Thu, May 9, 2019 at 5:39 PM Germain TANGUY > wrote:

Re: Sharing experience

2019-05-09 Thread Jarek Potiuk
Very interesting writeup :). Maybe you could make PR to https://github.com/jghoman/awesome-apache-airflow ? J On Thu, May 9, 2019 at 5:39 PM Germain TANGUY wrote: > Hello airflow community, > > At my company, we have been using airflow in production for nearly three > years. We have shared a ta

Re: Schedule interval options

2019-05-09 Thread James Coder
Yes, but doing 1-5 would not run 5 until 1. James Coder > On May 9, 2019, at 2:40 PM, Bas Harenslak > wrote: > > Perhaps needless to say, but you can do this with a cron expression, e.g. “0 > 0 * * 1-5” to schedule on weekdays only. Or “0 0 * * 2-6” to ensure you start > running from Tuesday

Re: Schedule interval options

2019-05-09 Thread Bas Harenslak
Perhaps needless to say, but you can do this with a cron expression, e.g. “0 0 * * 1-5” to schedule on weekdays only. Or “0 0 * * 2-6” to ensure you start running from Tuesday 00:00 to Saturday 00:00 so you also process Fridays data. Does that help? Bas On 9 May 2019, at 18:53, James Coder ma

Re: Feedback Wanted for Season of Docs Proposal

2019-05-09 Thread Michael Ghen
Hello Nathan, I hear you on setting up a local environment, that was one reason I put this tutorial together back in 2017, unfortunately it maybe a little out of date now: https://github.com/mikeghen/airflow-tutorial I recently integrated a similar tutorial into the Apache Unomi website also along

Schedule interval options

2019-05-09 Thread James Coder
Hi All, I feel like this has probably been discussed more times than necessary, but I wanted to get the community opinion on running dags for business days. In my case I want to run M-F but I don't want to wait for Monday to run Fridays data. As far as I can tell the only way to do this right now i

Sharing experience

2019-05-09 Thread Germain TANGUY
Hello airflow community, At my company, we have been using airflow in production for nearly three years. We have shared a talk/slides a few months ago regarding our apache airflow experience mainly focus on how we collaborate between data engineers and data analysts. We have continued to impro