Jacob Ferriero created AIRFLOW-5567:
---------------------------------------
Summary: Improved primitive for building Operators that benefit
from reschedule mode
Key: AIRFLOW-5567
URL: https://issues.apache.org/jira/browse/AIRFLOW-5567
Project: Apache Airflow
Issue Type: Improvement
Components: models, operators
Affects Versions: 1.10.5
Reporter: Jacob Ferriero
Assignee: Jacob Ferriero
Often times airflow operators (derived from BaseOperator) kick-off a long
running tasks and then waits / polls, blocking a worker slot until the long
running task completes. This can be problematic in environments with many long
running tasks.
BaseSensorOperator was improved by implementing `reschedule` mode to solve the
similar issue with long running sensors blocking a worker to poll for a long
time.
This issue is to track how we could provide a primitive that would make it easy
to develop operators for long running tasks that reschedule a `poll` operation
rather than blocking in their `execute` method.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
