dag_run timeouts

Hi team, We wanted to enable dag_run timeouts on our dags but when we have gone through the behavior of how dagrun_timeout works, we got to know, it only works on when below conditions are met. 1) dagrun should be scheduled one i.e. not manually created 2) max_active_runs must be configured

Re: [DISCUSS] AIP-3: Drop Python2 support in Airflow 2.0

Agree with XD's second idea, we should warn Airflow users. My two cents: 1. Can we discuss on the timeline for 2.0? Another big change is FAB-UI ONLY, which is already in master branch. Personally I think it’s already worth considering 2.0, unless there is

Re: Announcing the new Portland Apache Airflow Meetup group...

Very excited to get the meetup off the ground and looking forward to sharing knowledge! Thanks, Danny Duncan Data Engineering & Infrastructure @ M Science On Thu, Feb 14, 2019 at 1:46 PM Sid Anand wrote: > Thx Ben! > > I've updated: > *

Re: Announcing the new Portland Apache Airflow Meetup group...

Thx Ben! I've updated: * https://cwiki.apache.org/confluence/display/AIRFLOW/Meetups * https://cwiki.apache.org/confluence/display/AIRFLOW/Announcements * https://twitter.com/ApacheAirflow/status/1096163577092702208 -s On Thu, Feb 14, 2019 at 12:58 PM Ben Tallman wrote: > For those in the

Announcing the new Portland Apache Airflow Meetup group...

For those in the greater Portland, OR area, we're happy to announce that we've formed the Portland Apache Airflow Meetup group, and we're working to schedule our first meetup as soon as possible. Our meetups will be based on demand and availability, but hopefully we can get quarterly at least.

Re: 'Task Instance State' FAILED: Task is in the 'running' state which is not a valid state for execution. The task must be cleared in order to be run.

More info! It appears that the Celery executor will silently fail if the credentials to a postgres results_backend are not valid. For example, we see: [2019-02-13 20:45:21,132] {{models.py:1353}} INFO - Dependencies not met for , dependency 'Task Instance Not Already Running' FAILED: Task is

Short Airflow user survey

Hi all, It occurs to me I don't really know how people use Airflow, and I'd like to know a bit more about what sort of size clusters people run. So if you have 5 minutes spare I'd appreciate filling out this short, 7 question survey: https://ashberlintaylor.typeform.com/to/hIO0Ks I'll give it

Re: [DISCUSS] AIP-3: Drop Python2 support in Airflow 2.0

My two cents: 1. Can we discuss on the timeline for 2.0? Another big change is FAB-UI ONLY, which is already in master branch. Personally I think it’s already worth considering 2.0, unless there is other existing expectation for 2.0. 2. Before we completely drop Python2 support, we can start

[DISCUSS] AIP-3: Drop Python2 support in Airflow 2.0

We talked about this in the past but I'd like to bring it up again with a more concrete proposal. At the time Airflow 2.0 wasn't very concrete, but