Hi all,

Way back when, the `output` property on operators was introduced as a more
"Pythonic" means to retrieve XComs from tasks via an XComArg object.
Meaning a DAG author could use `my_task.output` as an equivalence to:
`task_instance.xcom_pull(task_ids="my_task")`. With the use of the `output`
property, there is an option to select specific XCom keys as well (i.e.
`my_task.output["my_xcom_key"]` is equivalent to
`task_instance.xcom_pull(task_ids="my_task", key="my_xcom_key").

However, the use of `__getitem__` to change the XCom key doesn't allow
direct retrieval of nested values within XComs but rather continues to
update the key. Effectively `my_task.output["my_xcom_key"]["object_key"]`
is _not_ equivalent to `task_instance.xcom_pull(task_ids="my_task",
key="my_xcom_key")["object_key"]`. The former results in attempting to
retrieve an XCom with a key of "object_key" and the latter retrieves the
value tied to the "object_key" key of an XCom.

There is quite an old issue related to this non-equivalence
<https://github.com/apache/airflow/issues/16618> and I am bringing up the
idea to modify the means to change the XCom key by using `__getattr__`
instead as part of Airflow 3 (briefly mentioned in a comment
<https://github.com/apache/airflow/issues/16618#issuecomment-876288276> as
well). So the new, proposed method for changing XCom keys would be
`my_task.output.my_xcom_key["object_key"]`.

This change would bring parity between the `output` property and the
classic `xcom_pull()` method. The obvious drawback is this would be a
slight authoring change for existing DAGs that use the `output` property.
Perhaps if the change could be automated in migration tooling the behavior
change wouldn't be so impactful.

What do you all think about this proposed update?

Josh

Reply via email to