Thanks for the update Kaxil. Looks much better. I've added a couple of new comments.
On 2026/08/10 21:26:24 Kaxil Naik wrote: > Thanks for your feedback Przemysław, I updated the AIP > > On Fri, 7 Aug 2026 at 03:20, Przemysław Mirowski <[email protected]> wrote: > > > Hi Kaxil, > > > > I've added some comments to the AIP. In general, the idea looks > > interesting. > > > > Could we possibly rearrange the sections a bit? During the reading, > > sometimes one thing was stated when the reader does not know a particular > > part of the change (e.g. merge policy - I added a comment next to it). > > Also, there are a lot of new definitions - maybe it would be good to create > > a section which will gather them all and shortly explain? > > > > On 2026/07/30 10:16:20 Kaxil Naik wrote: > > > Hi all, > > > > > > I would like to start a discussion on AIP-113: Dynamic Task Groups. > > > > > > > > https://cwiki.apache.org/confluence/spaces/AIRFLOW/pages/440304823/AIP-113+Dynamic+Task+Groups > > > > > > tldr: > > > Dynamic Task Mapping made cardinality dynamic: run the same task over a > > > collection discovered at run time. What is still static is the shape, > > > meaning which tasks > > > run in a region and how they depend on each other. > > > > > > Branching gets you half of that already, and I want to be precise about > > > which half, because > > > I had this wrong in an earlier draft. `choose_branch` can return a list, > > so > > > a branch already > > > selects an arbitrary subset of its declared successors. What a branch > > > cannot do is decide > > > the dependency edges *among* the tasks it selected. Selection is solved; > > > ordering and wiring are not. That is the > > > gap this AIP is aimed at. > > > > > > The proposal is a Task Group whose members and internal edges are decided > > > at run time from a fixed catalog the author declares. A generator task > > > emits a plan, the group materializes > > > exactly the selected entries as real task instances, and the region > > > converges on a > > > synthesized merge node. The catalog is what serializes, so two runs of > > > different shapes share > > > one DAG version, and an entry the plan did not select never becomes a > > task > > > instance. The > > > planner can be a control table, an impact analysis, a triage rule, or an > > > LLM. > > > > > > --- > > > > > > Would love to hear your thoughts & get feedback. Please comment on the > > AIP > > > doc itself. > > > > > > Regards, > > > Kaxil > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > > > > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
