GitHub user potiuk added a comment to the discussion: Real Sequential Dynamic Task
Some more explanation: If I am understanding correctly - that you are essentially asking for is "Dynamic DAG" - not "Dynamic Task" -> i.e. you want to have DAG that has variable n tasks that depend on one another: ``` t1 -> t2 -> t3 -> t4 -> .... ``` Unfortunately that is something not really possible if you want to expand it at runtime because: ``` t1->t2->t3 ``` has a different DAG structure than ``` t1->t2->t3->t4 ``` Essentially airflow DAG structure should not change between runs - at least not often and the structure is established at parsing time not at the moment when DAG is run, so if your case is pretty much static - i.e. number of task does not change between runs normally, you can use Dynamic DAG generation. On the other hand - dynamic task mapping works in the way (essentially) that it does not change the DAG structure, it takes one of the "nodes" of the DAG (say t2) and replaces it with multiple task instances - basically copies of the t2 in this case. But the DAG structure remains the same only `t2` is essentially replaced with multiple copies of the same task. That's why this expansion of task is dynamic and can change between runs - every run can have different number of expanded tasks. GitHub link: https://github.com/apache/airflow/discussions/44789#discussioncomment-11517347 ---- This is an automatically sent email for commits@airflow.apache.org. To unsubscribe, please send an email to: commits-unsubscr...@airflow.apache.org