kaxil commented on code in PR #76975:
URL: https://github.com/apache/airflow/pull/76975#discussion_r4623437675


##########
providers/common/ai/docs/changelog.rst:
##########
@@ -80,6 +80,57 @@ Changelog
   typed ``str``; in 1.1.0 it defaulted to ``None`` and was typed ``str | 
None``. This changes
   only the signature and the annotated type, not the behaviour.
 
+.. note::
+  ``@task.llm_batch`` now runs OpenAI and Anthropic batches through 
pydantic-ai's ``Model.batch_*``
+  methods and adds a Gemini adapter (``google:<model>``, Gemini Developer API 
only; a Vertex
+  ``google-cloud:`` model id is refused with the fix in the message). This 
raises the
+  ``pydantic-ai-slim`` floor to 2.66.0, the ``openai`` extra's ``openai`` 
floor to 3.31.0 and the
+  ``anthropic`` extra's ``anthropic`` floor to 1.15.0, and adds 
``google-genai`` to the ``google``
+  extra.
+
+  What changes for an existing task:
+
+  - ``request_params`` and a request's ``params`` keep working. The sampling 
settings (``temperature``,
+    ``top_p``, ...) and the provider-prefixed settings (``openai_*``,
+    ``anthropic_*``) are typed, and every other key is still sent as a request 
field. A setting the
+    model would silently drop (``seed`` on Anthropic; ``seed``, ``top_k``, 
``frequency_penalty``,
+    ``presence_penalty``, ``logit_bias`` and ``stop_sequences`` on OpenAI) now 
fails before
+    submission, and a key the operator manages (``model``, ``messages``, 
``input``, ...) is ignored
+    with a warning. A Chat Completions key has to be written the Responses way 
on OpenAI, for example
+    ``openai_reasoning_effort`` instead of ``reasoning_effort``.
+  - OpenAI batches run on the Responses API (``/v1/responses``) instead of 
Chat Completions, because
+    pydantic-ai batches only its Responses model. An ``openai-chat:`` model id 
is built as
+    ``openai:`` and logged. A gateway that implements batches only for 
``/v1/chat/completions`` stops
+    working; whether LiteLLM implements them for ``/v1/responses`` has not 
been verified. Azure OpenAI
+    still has no adapter.
+  - Structured output uses the model's native JSON-schema output instead of a 
forced tool call or
+    ``response_format``, except for models without it.
+  - A cancelled batch is recognized from its landed results, because Anthropic 
reports it as ended.
+  - OpenAI no longer recovers a batch whose submit response was never recorded 
by reading the batch's
+    metadata. Like Anthropic and Gemini, it falls through to 
``on_orphaned_intent``, whose default
+    ``"resubmit"`` submits a new batch.
+  - ``completion_window`` is ignored by every built-in adapter, and a value 
other than ``"24h"`` now logs
+    a warning.
+  - The state file records pydantic-ai's batch handle (state version 2). An 
Anthropic batch that an
+    earlier version of this provider submitted and is still running is 
re-attached; its state file is
+    left as it is. An OpenAI batch from an earlier version cannot be: it was 
created against Chat
+    Completions and its results cannot be read. The task fails with 
``LLMBatchStateReadError``
+    naming the batch; let it finish or cancel it at OpenAI, delete the task's 
state file and retry.
+    An outdated batch from an earlier version that was recorded under a 
different connection cannot
+    be cancelled automatically; the task log says so, and the batch can be 
cancelled at the provider.
+  - Anthropic and OpenAI results are streamed, so landing a batch holds only a 
small buffer of
+    responses in memory. Gemini results are still read through pydantic-ai, 
which downloads a batch's
+    results in one call, so a Gemini batch holds all of its response bodies in 
memory at once. Rows are still written one at a time. For a large
+    Gemini batch, size workers for it or split the input across mapped task 
instances with ``.expand()``.
+  - A task that was deferred by an earlier version of this provider and is 
still waiting when you
+    upgrade cannot poll its batch: the trigger reports an error naming the 
cause, and the task's retry
+    re-attaches to the batch.
+  - pydantic-ai deletes an OpenAI batch's uploaded input file only inside its 
own result collection,

Review Comment:
   Took it out of the changelog. The OpenAI section of `llm_batch.rst` keeps 
the note and says the old adapter left the file behind too.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to