kaxil commented on code in PR #71403:
URL: https://github.com/apache/airflow/pull/71403#discussion_r3976952304
##########
providers/common/ai/docs/operators/llm.rst:
##########
@@ -119,16 +119,46 @@ calls within a single task.
:start-after: [START howto_operator_llm_usage_limits]
:end-before: [END howto_operator_llm_usage_limits]
+A plain ``dict`` can be passed instead of a ``UsageLimits`` instance, which
lets
+Jinja template individual fields -- e.g. a per-run cost cap driven by an
Airflow
+Variable so the budget can change per environment without editing the Dag:
+
+.. exampleinclude::
/../../ai/src/airflow/providers/common/ai/example_dags/example_llm.py
+ :language: python
+ :start-after: [START howto_operator_llm_templated_usage_limits]
+ :end-before: [END howto_operator_llm_templated_usage_limits]
+
+Each dict value is rendered by Jinja like any other ``template_fields`` entry,
+then coerced to that field's type (``Decimal``, ``int``, or ``bool``). A value
+that doesn't parse -- an unset Variable renders to ``""``, a typo renders to a
+non-numeric string -- fails the task with a ``ValueError`` naming the field and
+the rendered value, instead of silently disabling the limit. A ``UsageLimits``
+instance passed directly is used as-is and is not templated or validated.
+
Common knobs on ``UsageLimits``:
- ``request_limit`` — max model requests per run (caps retry/tool-loop
blow-ups).
pydantic-ai applies a default of ``50`` when ``UsageLimits()`` is constructed
without an explicit value, so passing
``UsageLimits(input_tokens_limit=4_000)``
- silently inherits that 50-request cap. Set ``request_limit=None`` to disable
- it explicitly when you only want a token cap.
+ (or the dict form ``{"input_tokens_limit": 4_000}``) silently inherits that
+ 50-request cap. Set ``request_limit=None`` explicitly when you only want a
+ token cap.
- ``input_tokens_limit`` / ``output_tokens_limit`` — per-run token caps.
- ``total_tokens_limit`` — combined input + output cap.
- ``tool_calls_limit`` — max tool invocations (``AgentOperator`` only).
+- ``cost_limit`` — a ``Decimal`` cap on the run's estimated USD cost. This is
**not** a
+ hard guarantee against overspend: the response that crosses the limit has
already been
+ produced and billed — pydantic-ai checks the accumulated cost *after* each
response and
+ then fails the run with ``UsageLimitExceeded``. It protects you from further
spend, not
+ from the request that broke the budget; even a single-request run fails as
soon as that
+ request's cost pushes the total over the limit. For self-hosted or unknown
+ models (e.g. Ollama, custom endpoints) pydantic-ai cannot price the
response, so cost
+ is ``None`` and ``cost_limit`` silently has no effect (a
``CostNotFoundWarning`` is
+ emitted instead of a failure). And like the other knobs above, setting
``cost_limit``
+ alone still inherits the ``request_limit=50`` default — see the
``request_limit`` note
+ above. Note that ``cost_limit`` only caps the operator's own LLM calls --
+ the meta-agent that ``LLMRetryPolicy`` runs to classify a failed task is a
separate,
+ uncapped LLM call; see :doc:`../retry_policies`.
Review Comment:
Although, we need to check how that Budget thing works i.e where does it
store the ongoing cost
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
