GitHub user merkushov created a discussion: XCom Limitations and Immutable
Operators in Airflow 3.x
I’m running into an issue with XCom and data passing between tasks.
As part of the migration from Airflow 2.x to 3.x, I’ve noticed that accessing
context.task_instance no longer works as it used to — it seems that the context
is now immutable.
In my case, I need to pass an array from one task and combine it with some
static data in the next operator. Here’s a simplified example of what I’m
trying to achieve:
```python
from airflow import DAG
from airflow.providers.standard.operators.python import PythonOperator
from airflow.providers.http.operators.http import HttpOperator
def return_list():
return [{"accounts":["apple", "banana", "cherry"]}]
def enrich_context(task):
def _enricher(context) -> None:
account_ids = []
for item in
context.get("task_instance").xcom_pull(task_ids=task.task_id):
account_ids.extend(item.get("accounts", []))
context.get("task").data["ids"] = account_ids
return None
return _enricher
with DAG(
dag_id="DEBUG_XCOM",
) as dag:
payload_task = PythonOperator(
task_id="payload",
python_callable=return_list,
)
http_task = HttpOperator(
task_id="http_request",
endpoint="/log-request",
method="POST",
data={
"foo": 1,
},
headers={"Content-Type": "application/json"},
pre_execute=enrich_context(payload_task),
)
payload_task >> http_task
```
Are there any recommended alternatives for passing data between tasks in
Airflow 3.x without relying on TaskFlow decorators?
GitHub link: https://github.com/apache/airflow/discussions/54772
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]
