anishgirianish commented on PR #61975:
URL: https://github.com/apache/airflow/pull/61975#issuecomment-4163398930
> How is this different from setting the template on the tasks instead?
Hi @uranusjr, thank you for looking at this, really appreciate the question.
Please correct me if I am wrong. The main difference I see is context
availability for downstream tasks in a mapped group:
```
@task_group(map_index_template="{{ filename }}")
def file_transforms(filename):
extracted = extract(filename)
load(extracted)
file_transforms.expand(filename=["data1.json", "data2.json"])
```
extract receives filename directly, so setting map_index_template on it
individually would work. But load only sees extracted (the output of extract) —
filename isn't in its rendering context. Setting map_index_template="{{
filename }}" on load individually would raise UndefinedError during
post-execution rendering (since the DAG uses StrictUndefined by default) and
fail the task. The further downstream you go, the more disconnected tasks
become from the original expansion argument.
The group-level template addresses this by resolving the expansion args
server-side for the specific map_index and making them available to all child
tasks during rendering. This also helps avoid the workarounds folks in #40799
have been resorting to (like ti.xcom_pull(task_ids='...')[ti.map_index] on each
downstream task).
Would love your guidance on whether this is the right direction, or if you'd
suggest a different approach, happy to rework things!
Thank you
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
