kaxil opened a new pull request, #74697: URL: https://github.com/apache/airflow/pull/74697
With per-tool approval (#73586), a support agent allowed to refund orders asks a person about every refund, and most refunds are routine. This adds an example Dag and a section in the tool-approval guide where a decision model answers "may this call run without a person?" for each call that needs approval. A confident `allow` runs the call. A confident `deny` refuses it, and the agent is told, as after a Reject. Everything else goes to the existing approval pause: a `review` answer, low or missing confidence, a judge that cannot be reached, and any call to a tool already refused earlier in the run. It adds no operator or parameter. The handler is pydantic-ai's `HandleDeferredToolCalls` capability, passed through `capabilities=`. It sees each call waiting for approval with its validated arguments and resolves the ones it is sure about; pydantic-ai returns the rest as `DeferredToolRequests`, which `AgentOperator` already pauses on. **Why not the `.approval_required()` function.** It is typed as a synchronous `Callable[..., bool]`, so it cannot await a model call. The guide says when that function is still the right tool: a limit you can write down, such as refunds over $50, needs no model. The model is for whether the call fits the request: the right order, an amount the customer's problem supports, an action somebody asked for. **Hard limits stay in code.** The refund cap is the tool's `args_validator`, which pydantic-ai runs before the call is marked for approval, so an over-limit refund goes back to the agent without the model or a person being asked about it. I ran the judge once against TypeSafe Jev 1.13.0 on 12 proposed calls: routine refunds, a wrong order, a wrong amount, an account closure nobody asked for, and two calls where the arguments or the request argued for their own approval. Every call that should not run got `deny` at confidence 0.70 or higher, with `allow` probability at most 0.01. Two of the three routine refunds came back `review`, so its misses send work to a person rather than letting a bad call through. That is one pass with no repeats, so the guide tells readers to set the bar from their own agent's calls instead of quoting these numbers. Gotchas: - Needs Airflow 3.3+ and `pydantic-ai-slim` 2.46+ (for `UseEnumMemberDocstrings`). The test module skips below both. - The Required Actions page shows the call, not the model's answer. The handler logs the answer, its confidence and the bar to the task log. - On Airflow 3.3, a second call left for a person in the same Dag run fails the task. The new section points to the rules elsewhere on the page instead of restating that limit, so it stays correct whether or not #74686 (repeat pauses on 3.4+) lands first. The two branches merge cleanly, and the tests pass on the merge. --- * Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information. Note: commit author/co-author name and email in commits become permanently public when merged. * For fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed. * When adding dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). * For significant user-facing changes create newsfragment: `{pr_number}.significant.rst`, in [airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments). You can add this file in a follow-up commit after the PR is created so you know the PR number. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
