kaxil opened a new pull request, #74314: URL: https://github.com/apache/airflow/pull/74314
With `AgentOperator(durable=True)`, a retry replays nothing when a tool comes from a capability that has no explicit `id`, for example `capabilities=[Toolset(FunctionToolset([...]))]`. Every step runs live again on the retry, so tools with side effects run twice and the model calls are paid for twice. The same tool passed through `toolsets=` replays correctly. Since pydantic-ai 2.40, a capability without an `id` gets a random one for each run (`<toolset:d0d75e>`, then `<toolset:78ba70>` on the retry; pydantic-ai documents that it differs every run on purpose) and stamps it on each of its tools as `ToolDefinition.capability_id`. The durable fingerprint hashed the whole `ModelRequestParameters`, so the model step-0 fingerprint never matched and `CachingModel` logged "cached model response does not match the current request" and ran the step live. The live response carries new tool call ids, so the tool steps missed as well. The fingerprint now leaves out the run-local capability ids: `capability_id` on each function and output tool definition, and `deferred_capability_ids`. Neither is sent to the model. What they influence, tool visibility and the capability catalogue in the instructions, is already in the hash through `tool_visibility`, `revealed_tool_names` and `instructions`, so renaming an explicit capability id still invalidates the cache. Stripping the ids covers every capability that contributes tools (`Toolset`, `MCP`, `CodeMode`, user capabilities). Asking users to set an `id` on every capability would not: the failure is silent, and nothing tells them to. **Message `metadata` stays in the hash.** pydantic-ai does not send it to the model, but it keeps routing state there under `__pydantic_ai__` (for example `FallbackModel`'s continuation pin), so stripping it could replay a response recorded for a different route. A new test pins that down. `test_toolset_capability_tool_replayed_on_retry` already passed a `Toolset` capability but never caught this: it calls `_build_agent().run_sync()` without `CachingModel` and uses a fixed `tool_call_id`. The new test drives `AgentOperator.execute` twice against the same durable storage, with the model issuing fresh tool call ids as a real provider does. It fails on `main` with the anonymous capability and passes with an explicit `id`, and passes for both with this change, on pydantic-ai-slim 2.48 (locked) and 2.52. Upgrading the provider changes every fingerprint once, so a task that fails before the upgrade and retries after it re-runs its steps live; that is the existing fallback for any request change. Follow-ups, not in this PR: - `revealed_tool_names` is a `set` and is hashed in iteration order, which varies with `PYTHONHASHSEED`, so a durable retry with two or more revealed tools (tool search, deferred capabilities) also misses the cache. - `durable=True` together with code mode is still rejected. That is next, after #74312. --- * Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information. Note: commit author/co-author name and email in commits become permanently public when merged. * For fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed. * When adding dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). * For significant user-facing changes create newsfragment: `{pr_number}.significant.rst`, in [airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments). You can add this file in a follow-up commit after the PR is created so you know the PR number. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
