(Yes, though we “soft-renamed” dynamic mapped tasks to just “mapped tasks” in 
3.4.)

> On 20 Nov 2026, at 16:21, Brent Bovenzi via dev <[email protected]> 
> wrote:
> 
> Planner Tasks sounds good.
> 
> We already described mapped tasks as "dynamic" so using that word again would 
> only confuse people.
> 
> On 2026/11/20 12:31:44 Amogh Desai via dev wrote:
>> Thanks for proposing this. I like the idea, and catalog + validator seems
>> like a reasonable
>> way to keep the LLM planner bounded.
>> 
>> I left a few comments on the AIP.
>> 
>> Thanks & Regards,
>> Amogh Desai
>> 
>> 
>> On Thu, Nov 19, 2026 at 12:03 AM Kaxil Naik <[email protected]> wrote:
>> 
>>> I do like Planner Tasks
>>> 
>>> On Wed, 18 Nov 2026 at 17:56, Constance Martineau via dev <
>>> [email protected]> wrote:
>>> 
>>>> For the name, I'm find myself defaulting to calling the proposed
>>>> feature Planner Tasks, especially when discussing it alongside Mapped
>>>> Tasks and Task Loops. Plan mode doesn't fit as well with the other
>>>> concepts we have.
>>>> 
>>>> On Wed, Nov 18, 2026 at 12:49 PM Kaxil Naik <[email protected]> wrote:
>>>>> 
>>>>> Hi all,
>>>>> 
>>>>> Picking this thread back up.
>>>>> 
>>>>> Airflow 3.4 shipped Task Loops and YAML Dags, and both made AIP-113
>>>>> simpler, so Ash and I reworked it on top of them: selected tasks are
>>> now
>>>>> created the way loop iterations are, and the plan uses the YAML Dag
>>>> schema.
>>>>> The page is updated:
>>>>> 
>>>>> 
>>>> 
>>> https://cwiki.apache.org/confluence/spaces/AIRFLOW/pages/440304823/AIP-113+Dynamic+Task+Groups
>>>>> 
>>>>> Thanks Przemysław for the earlier comments.
>>>>> 
>>>>> Why now: the Common AI provider reached 1.0 in October, and more of the
>>>>> Dags people write have a model deciding what happens next. Today an LLM
>>>> can
>>>>> pick which downstream tasks run (@task.llm_branch), but like any branch
>>>> it
>>>>> can only select from tasks already wired in the Dag. It can't decide
>>> the
>>>>> order or the dependencies between them, so people either put the whole
>>>>> agent loop inside one opaque task, losing per-step retries, logs and
>>>>> approvals, or generate Dag files at parse time. A few shapes we keep
>>>> seeing:
>>>>> 
>>>>> - Incident triage: an agent reads an alert and picks diagnosis steps,
>>>>> where which probe runs next depends on what the first one found, ending
>>>> in
>>>>> a fix or a page to a human.
>>>>> - Support tickets: classify, then run some of validate, refund,
>>>>> enrich, escalate and notify, in an order that depends on the ticket,
>>> with
>>>>> an approval task before anything that moves money.
>>>>> - Data-quality failures: quarantine, backfill, re-validate and notify,
>>>>> chosen per failure.
>>>>> - Selective builds, with no LLM involved: an impact analysis decides
>>>>> which models to rebuild and the edges between exactly those models.
>>>>> 
>>>>> In each case the set of allowed actions is fixed and reviewed in the
>>> Dag
>>>>> file; only the selection and wiring change per run. Each action stays a
>>>>> real task with its own retries, logs and pool, and the catalog plus a
>>>>> validator keep the model inside what the author approved.
>>>>> 
>>>>> On the name: neither of us likes "Dynamic Task Groups". It describes
>>> the
>>>>> mechanism, not what you get, which is building part of your Dag at run
>>>> time.
>>>>> We're keeping it for the AIP so the discussion doesn't get stuck on it,
>>>> but
>>>>> expect it to change before it ships.
>>>>> 
>>>>> Two that came to my mind: "Plan Mode", as in a task group in plan mode,
>>>> the
>>>>> way coding agents plan before they act, and "Runtime
>>> subgraphs/subdags",
>>>>> which pairs with Task Loops and the @planner decorator. Suggestions
>>>> welcome.
>>>>> 
>>>>> What changed since the first draft:
>>>>> 
>>>>> - Authoring is a @planner task inside a task group, plus a catalog of
>>>> what
>>>>> it
>>>>>  may pick from (@triage.catalog, or catalog= / extra_catalog_items=).
>>>> The
>>>>>  TaskFlow and context-manager forms are side by side on the page and
>>> are
>>>>>  nearly identical. Catalog items can be operator classes or
>>> import-path
>>>>>  strings, so heavy operators don't have to be imported at parse time.
>>>>> 
>>>>> - The plan uses the same schema as YAML Dags (tasks, uses, with,
>>> needs),
>>>> so
>>>>>  Airflow has one vocabulary for declaring a graph of tasks. Templating
>>>> is
>>>>>  explicit, as in YAML Dags: a value is rendered only when marked with
>>>> $t.
>>>>> 
>>>>> - No subclassing. An LLM planner would itself be a planner, shipped by
>>> a
>>>>>  provider, with a validator on top. An @task.agent returning the plan
>>> as
>>>>>  structured output should also work with no extra machinery.
>>>>> 
>>>>> - Selected entries are real task instances, not map indices of one
>>> mapped
>>>>>  task. The planner submits the plan through the Execution API while it
>>>> is
>>>>>  still running, and the API server creates the task instances, which
>>> is
>>>> the
>>>>>  approach Task Loops took. The plan is stored against the region, so
>>>> after
>>>>> a
>>>>>  clear and re-plan, the earlier plan and the task instances that ran
>>>> under
>>>>> it
>>>>>  are still there to look at.
>>>>> 
>>>>> - At parse time the group serializes as two nodes: the planner and a
>>>> merge
>>>>>  node. The merge is always an EmptyOperator, and anything downstream
>>> of
>>>> the
>>>>>  group attaches to it.
>>>>> 
>>>>> - Catalog membership is by exact class, so a subclass of an approved
>>>>> operator
>>>>>  is rejected. That can be loosened later; tightening it later would be
>>>> much
>>>>>  harder.
>>>>> 
>>>>> Comments on the page or here are both fine.
>>>>> 
>>>>> Thanks,
>>>>> Kaxil
>>>>> 
>>>>> On Tue, 18 Aug 2026 at 21:37, Przemysław Mirowski <[email protected]>
>>>> wrote:
>>>>> 
>>>>>> Thanks for the update Kaxil. Looks much better. I've added a couple
>>> of
>>>> new
>>>>>> comments.
>>>>>> 
>>>>>> On 2026/08/10 21:26:24 Kaxil Naik wrote:
>>>>>>> Thanks for your feedback Przemysław, I updated the AIP
>>>>>>> 
>>>>>>> On Fri, 7 Aug 2026 at 03:20, Przemysław Mirowski <[email protected]
>>>> 
>>>>>> wrote:
>>>>>>> 
>>>>>>>> Hi Kaxil,
>>>>>>>> 
>>>>>>>> I've added some comments to the AIP. In general, the idea looks
>>>>>>>> interesting.
>>>>>>>> 
>>>>>>>> Could we possibly rearrange the sections a bit? During the
>>> reading,
>>>>>>>> sometimes one thing was stated when the reader does not know a
>>>>>> particular
>>>>>>>> part of the change (e.g. merge policy - I added a comment next to
>>>> it).
>>>>>>>> Also, there are a lot of new definitions - maybe it would be good
>>>> to
>>>>>> create
>>>>>>>> a section which will gather them all and shortly explain?
>>>>>>>> 
>>>>>>>> On 2026/07/30 10:16:20 Kaxil Naik wrote:
>>>>>>>>> Hi all,
>>>>>>>>> 
>>>>>>>>> I would like to start a discussion on AIP-113: Dynamic Task
>>>> Groups.
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>> 
>>>> 
>>> https://cwiki.apache.org/confluence/spaces/AIRFLOW/pages/440304823/AIP-113+Dynamic+Task+Groups
>>>>>>>>> 
>>>>>>>>> tldr:
>>>>>>>>> Dynamic Task Mapping made cardinality dynamic: run the same
>>> task
>>>>>> over a
>>>>>>>>> collection discovered at run time. What is still static is the
>>>> shape,
>>>>>>>>> meaning which tasks
>>>>>>>>> run in a region and how they depend on each other.
>>>>>>>>> 
>>>>>>>>> Branching gets you half of that already, and I want to be
>>> precise
>>>>>> about
>>>>>>>>> which half, because
>>>>>>>>> I had this wrong in an earlier draft. `choose_branch` can
>>> return
>>>> a
>>>>>> list,
>>>>>>>> so
>>>>>>>>> a branch already
>>>>>>>>> selects an arbitrary subset of its declared successors. What a
>>>> branch
>>>>>>>>> cannot do is decide
>>>>>>>>> the dependency edges *among* the tasks it selected. Selection
>>> is
>>>>>> solved;
>>>>>>>>> ordering and wiring are not. That is the
>>>>>>>>> gap this AIP is aimed at.
>>>>>>>>> 
>>>>>>>>> The proposal is a Task Group whose members and internal edges
>>> are
>>>>>> decided
>>>>>>>>> at run time from a fixed catalog the author declares. A
>>> generator
>>>>>> task
>>>>>>>>> emits a plan, the group materializes
>>>>>>>>> exactly the selected entries as real task instances, and the
>>>> region
>>>>>>>>> converges on a
>>>>>>>>> synthesized merge node. The catalog is what serializes, so two
>>>> runs
>>>>>> of
>>>>>>>>> different shapes share
>>>>>>>>> one DAG version, and an entry the plan did not select never
>>>> becomes a
>>>>>>>> task
>>>>>>>>> instance. The
>>>>>>>>> planner can be a control table, an impact analysis, a triage
>>>> rule,
>>>>>> or an
>>>>>>>>> LLM.
>>>>>>>>> 
>>>>>>>>> ---
>>>>>>>>> 
>>>>>>>>> Would love to hear your thoughts & get feedback. Please comment
>>>> on
>>>>>> the
>>>>>>>> AIP
>>>>>>>>> doc itself.
>>>>>>>>> 
>>>>>>>>> Regards,
>>>>>>>>> Kaxil
>>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>> ---------------------------------------------------------------------
>>>>>>>> To unsubscribe, e-mail: [email protected]
>>>>>>>> For additional commands, e-mail: [email protected]
>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: [email protected]
>>>>>> For additional commands, e-mail: [email protected]
>>>>>> 
>>>>>> 
>>>> 
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: [email protected]
>>>> For additional commands, e-mail: [email protected]
>>>> 
>>>> 
>>> 
>> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [email protected]
> For additional commands, e-mail: [email protected]
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to