GitHub user Mrli2016 added a comment to the discussion: Dynamic mapped tasks
group arguments are interpreted as MappedArgument when provided to classic
operators
Following your method, I found that this problem can be solved using task.bash.
example:
```
from airflow import DAG
from airflow.decorators import task, task_group
from airflow.operators.bash import BashOperator
from pendulum import datetime
dag = DAG(
dag_id="airflow_issue_test",
start_date=datetime(2024, 1, 1, tz='UTC'),
catchup=False,
schedule="@daily",
default_args={"retries": 2},
)
with dag:
@task()
def list_dict_generator():
my_list = [
{
"project": "my_project",
"dataset": f"dataset_{number}",
"table": "my_table",
"partition_id": "my_partition_id",
}
for number in range(10)
]
return my_list
@task.bash
def bash_task(project, dataset, table_name, partition_id) -> str:
return f"echo {project}.{dataset}.{table_name}${partition_id}",
@task_group()
def my_tg(project, dataset, table_name, partition_id):
bash_task(project, dataset, table_name, partition_id)
partitions_to_delete = list_dict_generator()
my_tg.expand_kwargs(partitions_to_delete)
```
I'm not sure if this code will run, but I made some simple modifications based
on the code from the issue poster. However, the idea is like this, and it works
on my end.
GitHub link:
https://github.com/apache/airflow/discussions/40728#discussioncomment-12485556
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]