Dear Airflowers, I am sending this note out on behalf of Kaxil Naik, who is temporarily out of the office and asked me to start this discussion.
Over the last several months, we have discussed expanding Airflow capabilities towards more dynamic graph structures. Airflow already runs your tasks deterministicly and reliably. Loops let a runtime condition decide *how many times* to run a section of the DAG, such as a task or a task group. The dynamism focuses on the loop count and stop condition. Additionally, this is always bounded by a hard cap. The full draft is on Confluence as AIP-111 Task Loops <https://cwiki.apache.org/confluence/spaces/AIRFLOW/pages/440303747/AIP-111+Task+Loops> . We would love to hear your thoughts. Please comment on the AIP doc itself. Vikram (on behalf of Kaxil) -- Vikram Koka Chief Strategy Officer Email: [email protected] <https://www.astronomer.io/>
