Hi Kaxil and Vikram, I took a look at the proposal, and this is really great. This feature would be especially useful for AI and agentic workflows where a DAG needs to generate, evaluate, and refine repeatedly until a quality threshold or attempt budget is reached.
This is a big +1 from me. Thanks for putting this together. Pavan On Thu, Jul 9, 2026 at 5:45 PM Jarek Potiuk <[email protected]> wrote: > Hi Kaxil and Vikram, > > Huge thanks for putting this together! It’s a fantastic document and > addresses a concept many of us have been dreaming of for a long time. It > reads really well and is a very exciting step forward for Airflow. > > I’ve shared some detailed comments in the document to help clarify a few > points, but here is a quick overview of what I’m thinking: > > - Task vs. Task Group loops: I’d love to see both included, especially > for the potential optimizations Task.loop might offer. > - "Fill-to-cap" vs. "on-demand": To keep things manageable, I’d suggest > starting with one. "Fill-to-cap" feels like a safer first step, whereas > "on-demand" is a bold but higher-risk goal. > - Comparing alternatives: It would be great to highlight the specific > advantages of .loop() versus things like Deferrable Operators, while loops, > or AIP-104. > - Edge cases: We might want to dive deeper into handling skipped tasks, > loop exhaustion, and how the "condition" execution works. > - Naming: A few terms like the "loop" parameter and "gate" task could > perhaps be polished to be even clearer. > > Thanks again for the hard work on this! > > Best, > Jarek > > > On Thu, Jul 9, 2026 at 7:38 AM Kaxil Naik <[email protected]> wrote: > > > Thanks Vikram. > > > > Looking forward to feedback from all of you. > > > > On Tue, 7 Jul 2026 at 08:42, Vikram Koka via dev <[email protected] > > > > wrote: > > > > > Dear Airflowers, > > > > > > I am sending this note out on behalf of Kaxil Naik, who is temporarily > > out > > > of the office and asked me to start this discussion. > > > > > > Over the last several months, we have discussed expanding Airflow > > > capabilities towards more dynamic graph structures. > > > > > > Airflow already runs your tasks deterministicly and reliably. Loops > let a > > > runtime condition decide *how many times* to run a section of the DAG, > > such > > > as a task or a task group. The dynamism focuses on the loop count and > > stop > > > condition. Additionally, this is always bounded by a hard cap. > > > > > > The full draft is on Confluence as AIP-111 Task Loops > > > < > > > > > > https://cwiki.apache.org/confluence/spaces/AIRFLOW/pages/440303747/AIP-111+Task+Loops > > > > > > > . > > > > > > We would love to hear your thoughts. Please comment on the AIP doc > > itself. > > > > > > Vikram > > > (on behalf of Kaxil) > > > -- > > > > > > Vikram Koka > > > Chief Strategy Officer > > > Email: [email protected] > > > > > > > > > <https://www.astronomer.io/> > > > > > >
