GitHub user mohammadrezwankhan edited a comment on the discussion: Serialization issues with external_python decorator
Mixing `@task` and `@task.external_python` is supported, so I would not rewrite the DAG around that yet. The important clue is that the failing external task **does its work before it fails**. In the 3.3.0 code path, Airflow: 1. pickles `op_args` / `op_kwargs` in the Airflow process; 2. starts the external interpreter; 3. calls your function there; 4. pickles the function's return value back to Airflow. You can see that ordering in [`_write_args` and the subprocess call](https://github.com/apache/airflow/blob/3.3.0/providers/standard/src/airflow/providers/standard/operators/python.py#L577-L709), and in the [generated script: call at line 132, result dump at line 144](https://github.com/apache/airflow/blob/3.3.0/providers/standard/src/airflow/providers/standard/utils/python_virtualenv_script.jinja2#L131-L144). If the callable really ran, its arguments were already serialized successfully. That makes the exact value returned by the failing task the primary suspect, not the fact that an upstream value came from a normal `@task`. Also, “returns a dict/list of dicts” only checks the outer container. The error text means a function object is reachable somewhere inside the object graph. A nested SQLAlchemy row/mapping, lazy proxy, model object, bound method, or callable can hide inside a `dict`. First, if the last task is side-effect-only, make it return nothing explicitly: ```python @task.external_python(python=PYTHON) def write_result(...): # perform the write return None ``` If its output is needed, put this inside the external function and run it on the exact value immediately before `return`: ```python def require_plain(x, path="$"): if x is None or type(x) in (str, int, float, bool): return if type(x) is list: for i, value in enumerate(x): require_plain(value, f"{path}[{i}]") return if type(x) is dict: for key, value in x.items(): if type(key) is not str: raise TypeError( f"{path} has non-string key {key!r} " f"({type(key).__module__}.{type(key).__qualname__})" ) require_plain(value, f"{path}[{key!r}]") return raise TypeError(f"{path}: {type(x).__module__}.{type(x).__qualname__}") result = ... require_plain(result) return result ``` That deliberately uses `type(...) is ...`, not `isinstance`, so a dict-like or list-like wrapper cannot masquerade as a built-in container. Convert the reported leaf explicitly—for example, `dict(row._mapping)` for a SQLAlchemy row and primitive fields for model/Airflow objects—instead of switching serializers. The traceback can confirm the phase immediately: - failure in `_write_args` means an input or mapped/partial kwarg; - failure in the generated script's `dump(res, file)` means the return value. If `return None` still fails, please paste the full traceback from the first `PicklingError` plus: ```sh pip show apache-airflow apache-airflow-providers-standard ``` Airflow core `3.3.0` alone does not identify the installed Standard provider version, and the first failing frame will distinguish an argument failure from a result failure. GitHub link: https://github.com/apache/airflow/discussions/71281#discussioncomment-17962155 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
