jroachgolf84 opened a new pull request, #51668:
URL: https://github.com/apache/airflow/pull/51668
*Fix implemented for #51109 in `v2-11`. Closely mirrors #51556.*
#51109 outlined an issue where trying to dynamically map over an XCom return
value using a custom key resulted in unexpected behavior. Rather than
leveraging the `list` that was stored at that key, it would dynamically-map
TaskGroups using the key-value pairs in the return value itself.
This PR addresses this by applying the same logic present in the
`_TaskDecorator` class to `_TaskGroupFactory`. When attempting to map over a
list stored via a key `b` in the `dict return value of a Task `t`, the
following exception will be raised:
```
ValueError: cannot map over XCom with custom key 'b' from
<Task(_PythonDecoratedOperator): t>`
```
Rather than the faulty logic outlined in #51109, the DAG will fail to parse.
This matches the same behavior as `@task`. This logic was implemented for both
`.expand()` and `.expand_kwargs()`.
To test this, the following DAG was written. The result is now a DAG Import
Error, with the stack trace below.
``` {python}
import pendulum
from airflow.decorators import dag, task, task_group
@dag(schedule=None, start_date=pendulum.datetime(2022, 1, 1))
def pipeline():
@task
def t():
return {"b": [2, 3]}
@task_group()
def tg(a, b):
pass
tg.partial(a=1).expand(b=t()["b"])
pipeline()
```
```
Traceback (most recent call last):
File
"/opt/airflow/task-sdk/src/airflow/sdk/definitions/mappedoperator.py", line
132, in ensure_xcomarg_return_value
ensure_xcomarg_return_value(v)
File
"/opt/airflow/task-sdk/src/airflow/sdk/definitions/mappedoperator.py", line
127, in ensure_xcomarg_return_value
raise ValueError(f"cannot map over XCom with custom key {key!r} from
{operator}")
ValueError: cannot map over XCom with custom key 'b' from
<Task(_PythonDecoratedOperator): t>
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
