GitHub user ricardoalexanderh closed a discussion: Real Sequential Dynamic Task
I wanted to generate a dynamic tasks using expand based on a range of date. I
am able to do that, but it seems the tasks is running in parallel. I wanted it
to run in sequential, by waiting for the previous tasks to succeed.
I am able to limit the task to run only 1 at a time by setting
max_active_tasks=1, max_active_runs=1, and max_active_tis_per_dag=1. But the
issue is when one of the task failed or is retrying, it will still trigger the
next task. This is not the behaviour that I wanted. Let me give an illustration:
**Current behaviour:**
Run 8 Nov (Success) -> Run 9 Nov (Retry) -> Run 10 Nov (In Progress)
Run 8 Nov (Success) -> Run 9 Nov (Failed) -> Run 10 Nov (In Progress)
**Expected behaviour:**
Run 8 Nov (Success) -> Run 9 Nov (Retry) -> Run 10 Nov (Scheduled)
Run 8 Nov (Success) -> Run 9 Nov (Failed) -> Run 10 Nov (Failed)
The error occured on 9 Nov
**This is a part of my code:**
```python
def run_dbt_snapshot(snapshot_date: str, params: ParamsDict, **context):
scd_run_dbt_snapshot(snapshot_date, params, context)
date_range = get_date_range()
run_snapshots = run_dbt_snapshot.expand(
snapshot_date=date_range,
)
date_range >> run_snapshots
```
GitHub link: https://github.com/apache/airflow/discussions/44789
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]
