Lee-W commented on code in PR #74695: URL: https://github.com/apache/airflow/pull/74695#discussion_r4545488412
########## providers/common/ai/docs/structured_output.rst: ########## @@ -109,5 +113,45 @@ class is not auto-registered. Add the class qualified name to that pattern work. If a downstream consumer needs the dict shape (e.g. forwarding to an external -system that expects JSON-style payloads), pass ``serialize_output=True`` and the -operator calls ``model_dump()`` before pushing to XCom. +system that expects JSON-style payloads), the operator can push the model dumped +instead of as an instance. Whether it does depends on ``serialize_output``: + +* With ``serialize_output=True``, an output that is itself a model is dumped with ``model_dump()``, so + field types such as ``date`` and ``Decimal`` are kept. +* A model inside a list, tuple, set or dict is not dumped when the worker registers its class + (``output_type=list[MyModel]`` on Airflow 3.3 and later). The XCom serializer pushes it as an + instance, with or without ``serialize_output``. +* A model whose class the worker cannot register is dumped to JSON values, whatever ``serialize_output`` + says. The one exception is a model that is itself the output when ``serialize_output=True``: it is + still dumped, but in Python mode as the first bullet describes. The worker registers the classes it + finds by reading ``output_type``, and finds none in a list of types such as + ``output_type=[MyModel, int]``, so a ``MyModel`` output of that shape is dumped. +* ``LLMOperator``, ``@task.llm``, ``LLMFileAnalysisOperator``, ``AgentOperator`` and ``@task.agent`` dump the + whole output with ``to_jsonable_python``, using field names rather than aliases (also for a model that + sets ``serialize_by_alias=True``), as soon as it holds such a model in a list, tuple, set or dict, as a + set member or as a dict key. A ``date`` or ``Decimal`` next to the model becomes a string. + +In a dump to JSON values, ``HttpUrl``, ``Path`` and ``Enum`` fields become plain JSON values. A model +that is itself the output with ``serialize_output=True`` keeps its Python field values instead, so an +``HttpUrl``, a ``Path`` or a plain ``Enum`` member in it stays an object that the default XCom backend +cannot serialize. + +Behaviour of the LLM operators after 1.1.0 +------------------------------------------- + +The changelog note above 1.1.0 describes changes made after that release. ``LLMOperator``, +``@task.llm``, ``LLMFileAnalysisOperator``, ``AgentOperator`` and ``@task.agent`` share one +helper, ``dump_output_for_xcom``, and behave as follows: + +* A model whose class the worker registers is left for the XCom serializer, in a container or not. + A top-level model is dumped with ``model_dump()`` (Python field values kept) when + ``serialize_output=True`` is set. +* A model whose class the worker cannot register is dumped to JSON values, also inside a list, + tuple, set or dict. +* Before Airflow 3.3, or with a task-sdk that has no deserialization walker, the worker registers no + class, so every model is dumped, a top-level one included. A top-level model is dumped with + ``model_dump()`` when ``serialize_output=True`` is set, and otherwise to JSON values with field + names rather than aliases, like a model inside a list, tuple, set or dict. ``LLMOperator``, + ``@task.llm`` and ``LLMFileAnalysisOperator`` pushed ``list[MyModel]``, and a top-level model + without ``serialize_output=True``, as instances there in 1.1.0 and now push dicts; Review Comment: Fixed. Before Airflow 3.3 a model that is itself the output is dumped with `model_dump()`, as in 1.1.0, and only models inside a list, tuple, set or dict change. The new "When a model arrives as a dict" section and the changelog note now say that. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
