kaxil commented on code in PR #76000:
URL: https://github.com/apache/airflow/pull/76000#discussion_r4604279980


##########
providers/common/ai/docs/frameworks/openai_codex.rst:
##########
@@ -0,0 +1,315 @@
+ .. Licensed to the Apache Software Foundation (ASF) under one
+    or more contributor license agreements.  See the NOTICE file
+    distributed with this work for additional information
+    regarding copyright ownership.  The ASF licenses this file
+    to you under the Apache License, Version 2.0 (the
+    "License"); you may not use this file except in compliance
+    with the License.  You may obtain a copy of the License at
+
+ ..   http://www.apache.org/licenses/LICENSE-2.0
+
+ .. Unless required by applicable law or agreed to in writing,
+    software distributed under the License is distributed on an
+    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+    KIND, either express or implied.  See the License for the
+    specific language governing permissions and limitations
+    under the License.
+
+.. _howto/frameworks:openai_codex:
+
+OpenAI Codex SDK
+=================
+
+If your agent is a loop on the `OpenAI Codex SDK 
<https://pypi.org/project/openai-codex/>`__,
+which runs the ``codex app-server`` binary as a subprocess, you can run it in 
an Airflow
+task as it is and give it the toolsets this provider ships.
+:class:`~airflow.providers.common.ai.tools.openai_codex.AirflowTools` serves 
``SQLToolset``,
+``HookToolset`` and most other toolsets to Codex through an MCP server it 
starts on
+``127.0.0.1`` in the task's event loop. If you have no loop yet, 
``AgentOperator`` runs one
+for you, with durable replay and human review; see :doc:`index`.
+
+.. note::
+
+    ``AirflowTools`` is experimental: it may change in a minor release of this 
provider,
+    and it has **not been run against a model**. What a test without a model 
cannot show is
+    listed under `Not yet verified`_. Read that section before you rely on the 
defaults
+    below. The framework-neutral tool interface the adapter builds on is 
stable.
+    See :ref:`howto/stability`.
+
+Run a loop in a task
+---------------------
+
+This task answers a question about a database. The client's API key comes from 
an
+Airflow connection:
+
+.. exampleinclude:: 
/../../ai/src/airflow/providers/common/ai/example_dags/example_openai_codex.py
+    :language: python
+    :start-after: [START example_openai_codex]
+    :end-before: [END example_openai_codex]
+
+Only ``AirflowTools`` comes from Airflow: ``serve()`` starts the tool server 
and yields the
+``CodexConfig`` for ``AsyncCodex``, ``build_thread_options()`` builds the 
keyword arguments of
+``thread_start()``, and ``run()`` drives the turn. Login, the model, the 
instructions and the
+prompt are the SDK's own. ``run()`` returns the SDK's ``TurnResult``, built 
from the turn's
+notifications; ``final_response`` is the model's last answer.
+
+The model can call ``list_tables``, ``get_schema``, ``query`` and 
``check_query``, the
+tools ``SQLToolset`` serves. Each call writes a ``Tool call: <name>`` group to 
the task log.
+
+Why only ``AsyncCodex``
+-------------------------
+
+Codex has no in-process tools and no way to register a Python function, so a 
tool call
+goes from the ``codex app-server`` child process back to the worker over a 
real HTTP
+connection to the MCP server ``serve()`` started. That server runs in the same 
event loop as
+the tools, which is what lets a toolset's async hook run on the loop it was 
built for and
+share state with the run. If you drive the synchronous ``Codex`` client from 
that loop's
+thread, the loop is blocked, the server cannot answer, and the call waits 
until Codex gives
+up on the tool. Use ``AsyncCodex``, and ``AsyncThread`` and 
``AsyncTurnHandle`` with it.
+
+What each tool does
+--------------------
+
+``AirflowTools`` accepts any toolset that implements
+:class:`~airflow.providers.common.ai.tools.ToolProvider`, and individual
+:class:`~airflow.providers.common.ai.tools.AirflowTool` objects. 
``MCPToolset`` is the
+exception; give Codex that MCP server directly instead. Each tool:
+
+- Keeps the toolset tool's name, description and argument schema.
+- Runs through the toolset, so connection resolution, SQL validation,
+  ``allowed_tables``, ``allowed_methods`` and result limits behave as they do 
in
+  :class:`~airflow.providers.common.ai.operators.agent.AgentOperator`. A 
result longer than
+  100,000 characters is cut to fit and ends with a notice asking the model to 
narrow the
+  request. The limit is this adapter's own choice, and Codex's 
``tool_output_token_limit`` is
+  set to the same number so that Codex, which counts tokens, is unlikely to 
cut first; some text
+  has more than one token per character. Whether Codex applies that setting to 
MCP results is
+  not verified.
+- Passes its result through Airflow's secret masker before the model sees it.
+- Writes a ``Tool call: <name>`` group to the task log. Calls to a toolset are 
also
+  counted in the ``common_ai.tool_calls`` metric with 
``framework=openai_codex`` (see
+  :doc:`../observability`).
+
+The tool server only accepts requests that carry a bearer token. ``127.0.0.1`` 
is not a
+boundary between tasks: other processes on the same worker can connect to it. 
``serve()``
+generates a new random token for each block, compares it in constant time, 
answers ``401``
+to anything else, and gives it to Codex through the child process's 
environment, because a
+value in ``codex --config`` would be visible in ``ps``.
+
+The token keeps out other OS users and blind probes of the port. It does not 
separate tasks
+that run as the same Unix user as the worker: such a process can read 
``/proc/<pid>/environ``
+of the ``codex`` child (on Linux) and use the token. Likewise the ``codex`` 
process itself
+holds the worker's full environment, as ``CodexConfig.env`` only adds to it;
+``shell_environment_policy.inherit="none"`` protects only the commands Codex 
starts.
+
+When a tool fails
+-------------------
+
+A failure the model can correct, such as a query naming a table outside 
``allowed_tables`` or
+one the database rejects, comes back to it as an MCP tool result with 
``isError`` set.
+``SQLToolset``, ``HookToolset`` and ``ObjectStorageToolset`` allow one 
correction: a tool
+that keeps failing ends the run with
+``ToolCallError: query kept failing after 1 correction(s): ...``.
+
+Any other failure, such as a hook raising, ends the run:
+:meth:`~airflow.providers.common.ai.tools.openai_codex.AirflowTools.run` raises
+:class:`~airflow.providers.common.ai.tools.ToolCallError` after the model has 
read the (masked)
+failure, and Airflow's own retry takes over once the task fails. Before it 
raises, ``run()``
+interrupts the turn: closing the stream does not stop Codex, which would 
otherwise keep
+calling tools. If the interrupt itself fails, that is logged and the tool 
failure is still
+raised. ``run()`` also interrupts the turn when it is cancelled before the 
turn completes, for
+example when wrapped in ``asyncio.wait_for``; the cancellation still 
propagates. The interrupt is
+bounded to 10 seconds, so cancellation can take that long to unwind if the 
app-server does not
+respond. It is a request: a call already in flight, or sent before it takes 
effect, can still
+arrive. A call that arrives after the run has ended, or after a failure, is 
refused with an
+``isError`` "Not run" result and does not reach the toolset.
+
+``run()`` raises ``RuntimeError`` when the turn fails or is interrupted from 
outside, with
+Codex's error message shortened to 500 characters, and when the stream ends 
without a
+``turn/completed`` notification.
+
+The retry budget counts per run. A Codex turn spans the whole agent loop, many 
model
+responses, so the adapter cannot use it as the model turn that the Anthropic 
and Strands
+adapters count one failure per. Calls that run at the same time count as one 
failure, and
+calls one after another count separately, so a model that keeps sending a 
failing call
+exhausts the budget sooner here than it would per model turn. The adapter sets 
no limit on
+how many model responses a turn may take: bound the loop yourself, for example 
with the
+task's ``execution_timeout``.
+
+Tool timeout
+-------------
+
+Codex stops waiting for a tool after ``tool_timeout_sec``, which is 60 seconds 
unless set.
+That is too short for many SQL queries, so ``AirflowTools(..., 
tool_timeout=600)`` sets it to
+600 seconds, and you can change it. What happens to the hook when Codex gives 
up is not
+verified: Codex's result is discarded either way, but the call may be 
cancelled, or a hook that
+runs in a thread may run on.
+
+Drive one turn at a time
+--------------------------
+
+An ``AirflowTools`` instance serves one ``serve()`` block and drives one turn 
at a time:
+build a new instance per task. Entering ``serve()`` or calling ``run()`` a 
second time while
+the first is still active raises ``RuntimeError``. Leave the ``serve()`` block 
after the
+``AsyncCodex`` that uses it: leaving it stops the server and deletes the 
temporary directory
+Codex works in.
+
+Safe defaults
+---------------
+
+Left alone, ``codex app-server`` reads the worker's ``~/.codex`` 
(``config.toml``,
+``AGENTS.md``, skills, MCP servers), finds project settings in the working 
directory, writes
+its state under ``~/.codex`` and inherits every environment variable of the 
worker. Codex
+silently ignores a ``--config`` key it does not know, so a misspelled key does 
nothing.
+``serve()`` and ``build_thread_options()`` set the following. A unit test 
starts the
+``codex app-server`` that ``openai-codex-cli-bin`` ships, with no login and no 
model, and
+reads each setting back from its ``config/read``. That shows Codex knows the 
key and took the
+value; it does not show what the model can do.
+
+.. list-table::
+   :header-rows: 1
+   :widths: 34 66
+
+   * - Setting
+     - Why
+   * - ``CODEX_HOME``, ``CODEX_SQLITE_HOME`` and ``HOME`` in the child's 
environment point at
+       empty temporary directories
+     - Codex reads no user-level settings from the worker (``~/.codex``, a 
worker ``CODEX_HOME``)
+       and writes nothing to them. The system layer, 
``/etc/codex/config.toml``, is still read if
+       it exists. The environment
+       variables have to be overridden one by one, because the child inherits 
the worker's:
+       a ``CODEX_SQLITE_HOME`` set on the worker sends the state database 
outside the temporary
+       directory unless it is overridden. Whether the ``sqlite_home`` config 
key alone would do
+       is not verified. The temporary directory, about eighty files once Codex 
has started, is
+       deleted when ``serve()`` ends.
+   * - ``cwd`` is a further empty temporary directory
+     - Codex looks for a project ``.codex/config.toml`` and ``AGENTS.md`` 
upward from its
+       working directory. Pass ``cwd`` yourself and that protection is yours 
to keep.
+   * - ``cli_auth_credentials_store="ephemeral"``
+     - A login with ``login_api_key`` is meant to stay in memory instead of an 
auth file.
+       The key is accepted; that no file is written after a login is not 
verified.
+   * - ``features.shell_tool``, ``features.unified_exec``, 
``features.view_image``,
+       ``features.multi_agent``, ``features.apps``, ``features.plugins`` and
+       ``features.image_generation`` are ``false``
+     - The model must not get a shell on the worker, read an image file from 
it or use
+       capabilities that have nothing to do with Airflow's tools.
+   * - ``web_search="disabled"``
+     - Data leaves the worker only through the Airflow tools.
+   * - ``shell_environment_policy.inherit="none"``
+     - If a command still ran, it would get none of the worker's environment, 
which holds
+       ``AIRFLOW_CONN_*`` values.
+   * - ``history.persistence="none"``, ``analytics.enabled=false``, 
``feedback.enabled=false``,
+       ``check_for_update_on_startup=false`` and 
``otel.metrics_exporter="none"``
+     - No session history, usage analytics, feedback, update check or metrics.
+   * - ``approval_mode=ApprovalMode.deny_all`` and 
``sandbox=Sandbox.read_only`` in
+       ``build_thread_options()``
+     - ``AsyncCodex`` always accepts a request to run a command or change a 
file, so the only
+       way to refuse one is for Codex never to ask. See `Removing Codex's 
file-editing tool`_.
+   * - ``ephemeral=True`` in ``build_thread_options()``
+     - Codex accepts it; that it keeps the thread off disk is assumed, not 
verified.
+   * - ``mcp_servers.<name>.required=true``
+     - The thread does not start when the tool server is unreachable. Without 
it Codex would
+       run the turn with no tools. Tested: ``thread/start`` fails with 
"required MCP servers
+       failed to initialize" when the URL is wrong. ``mcpServerStatus/list`` 
does not raise in
+       that case; it reports the failure in ``tools_error``.
+   * - ``mcp_servers.<name>.supports_parallel_tool_calls=false``
+     - Hook calls run one at a time anyway; this keeps Codex from sending them 
in parallel.
+       Codex does not echo a ``false`` value, so a test can show only that the 
key is known
+       (``true`` is echoed), and whether ``false`` is the same as the default 
is not
+       confirmed.
+   * - ``mcp_servers.<name>.tool_input_schema_max_bytes``
+     - Codex compresses an MCP server's tool schemas past 5,000 bytes and 
drops parameter
+       descriptions. The adapter raises the limit to at least 65,536 bytes, or 
the size of the
+       largest tool schema.
+
+``build_thread_options()`` lets a value you pass win, and raises 
``ValueError`` if ``config``
+already defines an MCP server with the same name as ``server_name``. 
``serve()`` puts nothing
+of yours in the environment: add what you need to ``CodexConfig.env`` 
yourself, or build a
+modified copy with ``dataclasses.replace``.
+
+Removing Codex's file-editing tool
+-----------------------------------
+
+We found no user-level setting that removes ``apply_patch``, Codex's 
file-editing tool: the
+feature flag that used to control it is gone and which form the model sees is 
a field of each
+model's catalog. Overriding that catalog (``model_catalog_json``) was not 
investigated. So even
+with every default above, the model may still see a tool that writes files. 
The adapter does not remove it. It makes it harmless: the sandbox is read-only, 
and Codex is
+never allowed to ask for an exception, so a write should be refused without 
anything to
+approve. Codex's own sandbox is what does this. It depends on the host 
operating system, and
+whether it is available in a container, and what Codex does when it is not, is 
unverified.
+``sandbox`` and ``approval_mode`` are defaults that you can override; 
loosening either
+reopens this.
+
+Not yet verified
+------------------
+
+Everything above that needs a model has not been run. Someone with an OpenAI 
API key has to check
+the following before this adapter is relied on. Until then, treat the 
corresponding defaults as
+intended, not proven:
+
+1. **Login.** ``login_api_key`` succeeds, and after the run neither the 
temporary
+   ``CODEX_HOME`` nor the worker's ``~/.codex`` holds a new auth file.
+2. **Tool round trip.** Run the example Dag. The model calls a ``SQLToolset`` 
tool through MCP,
+   the task log shows the call with ``framework=openai_codex``, and a result 
containing a

Review Comment:
   Nothing in the call path logs `framework=openai_codex`; it is only a tag on 
the `common_ai.tool_calls` metric, as line 89 says. Should this step check the 
metric instead of the task log?



##########
providers/common/ai/src/airflow/providers/common/ai/tools/openai_codex.py:
##########
@@ -0,0 +1,619 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Give Airflow tools to an agent loop run by the OpenAI Codex SDK.
+
+.. note:: Experimental; see :ref:`howto/stability`.
+"""
+
+from __future__ import annotations
+
+try:
+    import uvicorn
+    from mcp import types as mcp_types
+    from mcp.server.lowlevel.server import Server
+    from openai_codex import ApprovalMode, CodexConfig, Sandbox, TurnResult
+    from openai_codex.generated.v2_all import (
+        AgentMessageThreadItem,
+        ItemCompletedNotification,
+        MessagePhase,
+        ThreadTokenUsageUpdatedNotification,
+        TurnCompletedNotification,
+        TurnStatus,
+    )
+except ImportError as e:
+    from airflow.providers.common.compat.sdk import 
AirflowOptionalProviderFeatureException
+
+    raise AirflowOptionalProviderFeatureException(e)
+
+import asyncio
+import contextlib
+import copy
+import hmac
+import json
+import logging
+import os
+import re
+import secrets
+import shutil
+import socket
+import tempfile
+import time
+from dataclasses import dataclass
+from typing import TYPE_CHECKING, Any
+
+from airflow.providers.common.ai.tools import AirflowTool, ToolCallError, 
_collect_tools
+from airflow.providers.common.ai.tools._from_toolset import tool_call_scope
+from airflow.providers.common.ai.utils.tool_definition import serialize_for_llm
+from airflow.providers.common.ai.utils.tool_metrics import calling_framework
+
+if TYPE_CHECKING:
+    from collections.abc import AsyncIterable, AsyncIterator, Awaitable, 
Callable, Iterator, MutableMapping
+
+    from mcp.server.context import ServerRequestContext
+    from openai_codex import AsyncTurnHandle
+    from openai_codex.generated.v2_all import ThreadItem, ThreadTokenUsage
+
+    from airflow.providers.common.ai.tools import ToolSource
+
+log = logging.getLogger(__name__)
+
+__all__ = ["AirflowTools"]
+
+_MCP_HINT = "Add the server to Codex's mcp_servers config instead."
+
+
+_MAX_ERROR_DETAIL_CHARS = 500
+# Every constant below was observed with openai-codex 0.163.0, not a 
documented guarantee.
+#
+# The ceiling on a tool result this adapter lets through. It is our own 
choice: unlike the Claude
+# CLI, Codex has no fixed clamp that we know of. A result over it is cut here, 
with a notice the
+# model can act on, instead of by Codex, whose truncation of an MCP result is 
unverified.
+_MAX_INLINE_RESULT_CHARS = 100_000
+# Set to the character ceiling so that Codex is unlikely to cut a result before
+# ``_truncate_result_text`` does. A token can be shorter than a character for 
some text (CJK,
+# emoji), so this is not a guarantee.
+_TOOL_OUTPUT_TOKEN_LIMIT = _MAX_INLINE_RESULT_CHARS
+# Codex's default budget is 5_000 bytes of input schema per MCP server; over 
it, Codex compresses
+# the schema and drops parameter descriptions. The largest schema of the 
toolsets this provider
+# ships is ~1 KB, so this leaves room; ``_build_config_overrides`` raises it 
for a larger one.
+_MIN_TOOL_INPUT_SCHEMA_BYTES = 65_536
+# Codex stops waiting for a tool after this long (its default is 60 s, too 
short for a SQL query).
+# What happens to the hook when Codex gives up is not verified.
+_DEFAULT_TOOL_TIMEOUT_SEC = 600.0
+# How long ``serve()`` waits for the HTTP server to start, and for open 
connections to drain on exit.
+_SERVER_START_TIMEOUT_SEC = 10.0
+# How long ``run()`` waits for Codex to confirm an interrupt before it raises 
the tool failure anyway.
+_INTERRUPT_TIMEOUT_SEC = 10.0
+_SERVER_SHUTDOWN_TIMEOUT_SEC = 5
+# The variable that carries the bearer token to the Codex child process. Only 
its name is in
+# ``codex --config`` (so in argv, visible to ``ps``); the value goes through 
the environment.
+_MCP_TOKEN_ENV_VAR = "AIRFLOW_CODEX_MCP_TOKEN"
+# ``server_name`` becomes an unquoted TOML key in 
``mcp_servers.<server_name>.*``.
+_SERVER_NAME_RE = re.compile(r"[A-Za-z0-9_-]+")
+
+
+def _truncate_result_text(text: str) -> str:
+    if len(text) <= _MAX_INLINE_RESULT_CHARS:
+        return text
+    notice = (
+        f"\n\n[Truncated: this result had {len(text)} characters and only the 
beginning is shown. "
+        "Narrow the request, for example by adding filters, selecting fewer 
columns or lowering "
+        "the LIMIT, to get a smaller result.]"
+    )
+    return text[: _MAX_INLINE_RESULT_CHARS - len(notice)] + notice
+
+
+def _bound_detail(text: str) -> str:

Review Comment:
   `_bound_detail`, `_MAX_ERROR_DETAIL_CHARS` and `_close_stream` are copies of 
the ones in `claude_agent_sdk.py`. Could they move to `_from_toolset.py`, which 
both adapters already import? Fine as a follow-up.



##########
providers/common/ai/src/airflow/providers/common/ai/example_dags/example_openai_codex.py:
##########
@@ -0,0 +1,98 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Answer a question about a database with the OpenAI Codex SDK's agent loop, in 
an Airflow task.
+
+The agent loop is Codex's own, run by the ``codex app-server`` subprocess that 
``AsyncCodex``
+starts. Airflow supplies the API key, from a connection, and the tools:
+:class:`~airflow.providers.common.ai.tools.openai_codex.AirflowTools` serves a 
``SQLToolset``
+to Codex through an MCP server on ``127.0.0.1`` in the task's event loop, with 
read-only SQL
+validation, bounded results and Airflow's secret masker applied to every 
result.
+``AirflowTools.serve`` and ``build_thread_options`` also point Codex at empty 
temporary
+directories instead of the worker's ``~/.codex``, ask Codex to turn off its 
own shell and the
+other built-in tools that do not belong here, and run the thread in a 
read-only sandbox that
+never asks for approval.
+
+This example is experimental and was not run against a model: the tool round 
trip, that the
+built-in tools are gone from the model's tool list, and that the sandbox 
refuses writes still
+need to be checked by hand; see the guide.
+
+Before running:
+
+1. Install this provider's ``openai-codex`` extra:
+   ``pip install "apache-airflow-providers-common-ai[openai-codex]"``.
+2. Create a connection (``LLM_CONN_ID``, default ``openai_default``) whose 
password is
+   an OpenAI API key.
+3. Create a database connection (``DB_CONN_ID``, default ``sql_default``) 
whose hook is a
+   ``DbApiHook`` (e.g. SQLite, Postgres, MySQL).
+"""
+
+from __future__ import annotations
+
+import asyncio
+import os
+
+from airflow.providers.common.ai.toolsets.sql import SQLToolset

Review Comment:
   `toolsets/sql.py` raises `AirflowOptionalProviderFeatureException` on import 
without the `sql` extra, so this now fails Dag parsing for anyone who installed 
only `openai-codex`. That is the reason the comment at line 67 gives, and the 
other examples import it inside the task or under `try`. Can it go back next to 
the `AirflowTools` import?



##########
providers/common/ai/src/airflow/providers/common/ai/tools/openai_codex.py:
##########
@@ -0,0 +1,619 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Give Airflow tools to an agent loop run by the OpenAI Codex SDK.
+
+.. note:: Experimental; see :ref:`howto/stability`.
+"""
+
+from __future__ import annotations
+
+try:
+    import uvicorn
+    from mcp import types as mcp_types
+    from mcp.server.lowlevel.server import Server
+    from openai_codex import ApprovalMode, CodexConfig, Sandbox, TurnResult
+    from openai_codex.generated.v2_all import (
+        AgentMessageThreadItem,
+        ItemCompletedNotification,
+        MessagePhase,
+        ThreadTokenUsageUpdatedNotification,
+        TurnCompletedNotification,
+        TurnStatus,
+    )
+except ImportError as e:
+    from airflow.providers.common.compat.sdk import 
AirflowOptionalProviderFeatureException
+
+    raise AirflowOptionalProviderFeatureException(e)
+
+import asyncio
+import contextlib
+import copy
+import hmac
+import json
+import logging
+import os
+import re
+import secrets
+import shutil
+import socket
+import tempfile
+import time
+from dataclasses import dataclass
+from typing import TYPE_CHECKING, Any
+
+from airflow.providers.common.ai.tools import AirflowTool, ToolCallError, 
_collect_tools
+from airflow.providers.common.ai.tools._from_toolset import tool_call_scope
+from airflow.providers.common.ai.utils.tool_definition import serialize_for_llm
+from airflow.providers.common.ai.utils.tool_metrics import calling_framework
+
+if TYPE_CHECKING:
+    from collections.abc import AsyncIterable, AsyncIterator, Awaitable, 
Callable, Iterator, MutableMapping
+
+    from mcp.server.context import ServerRequestContext
+    from openai_codex import AsyncTurnHandle
+    from openai_codex.generated.v2_all import ThreadItem, ThreadTokenUsage
+
+    from airflow.providers.common.ai.tools import ToolSource
+
+log = logging.getLogger(__name__)
+
+__all__ = ["AirflowTools"]
+
+_MCP_HINT = "Add the server to Codex's mcp_servers config instead."
+
+
+_MAX_ERROR_DETAIL_CHARS = 500
+# Every constant below was observed with openai-codex 0.163.0, not a 
documented guarantee.
+#
+# The ceiling on a tool result this adapter lets through. It is our own 
choice: unlike the Claude
+# CLI, Codex has no fixed clamp that we know of. A result over it is cut here, 
with a notice the
+# model can act on, instead of by Codex, whose truncation of an MCP result is 
unverified.
+_MAX_INLINE_RESULT_CHARS = 100_000
+# Set to the character ceiling so that Codex is unlikely to cut a result before
+# ``_truncate_result_text`` does. A token can be shorter than a character for 
some text (CJK,
+# emoji), so this is not a guarantee.
+_TOOL_OUTPUT_TOKEN_LIMIT = _MAX_INLINE_RESULT_CHARS
+# Codex's default budget is 5_000 bytes of input schema per MCP server; over 
it, Codex compresses
+# the schema and drops parameter descriptions. The largest schema of the 
toolsets this provider
+# ships is ~1 KB, so this leaves room; ``_build_config_overrides`` raises it 
for a larger one.
+_MIN_TOOL_INPUT_SCHEMA_BYTES = 65_536
+# Codex stops waiting for a tool after this long (its default is 60 s, too 
short for a SQL query).
+# What happens to the hook when Codex gives up is not verified.
+_DEFAULT_TOOL_TIMEOUT_SEC = 600.0
+# How long ``serve()`` waits for the HTTP server to start, and for open 
connections to drain on exit.
+_SERVER_START_TIMEOUT_SEC = 10.0
+# How long ``run()`` waits for Codex to confirm an interrupt before it raises 
the tool failure anyway.
+_INTERRUPT_TIMEOUT_SEC = 10.0
+_SERVER_SHUTDOWN_TIMEOUT_SEC = 5
+# The variable that carries the bearer token to the Codex child process. Only 
its name is in
+# ``codex --config`` (so in argv, visible to ``ps``); the value goes through 
the environment.
+_MCP_TOKEN_ENV_VAR = "AIRFLOW_CODEX_MCP_TOKEN"
+# ``server_name`` becomes an unquoted TOML key in 
``mcp_servers.<server_name>.*``.
+_SERVER_NAME_RE = re.compile(r"[A-Za-z0-9_-]+")
+
+
+def _truncate_result_text(text: str) -> str:
+    if len(text) <= _MAX_INLINE_RESULT_CHARS:
+        return text
+    notice = (
+        f"\n\n[Truncated: this result had {len(text)} characters and only the 
beginning is shown. "
+        "Narrow the request, for example by adding filters, selecting fewer 
columns or lowering "
+        "the LIMIT, to get a smaller result.]"
+    )
+    return text[: _MAX_INLINE_RESULT_CHARS - len(notice)] + notice
+
+
+def _bound_detail(text: str) -> str:
+    omitted = len(text) - _MAX_ERROR_DETAIL_CHARS
+    return text if omitted <= 0 else 
f"{text[:_MAX_ERROR_DETAIL_CHARS]}...(+{omitted} chars)"
+
+
+def _to_toml_string(value: str) -> str:
+    # A JSON string is a valid TOML basic string.
+    return json.dumps(value)
+
+
+def _find_final_response(items: list[ThreadItem]) -> str | None:
+    """Pick the final answer the way the SDK's ``thread.run()`` does, which 
keeps that helper private."""
+    last_unphased: str | None = None
+    for item in reversed(items):
+        message = getattr(item, "root", item)
+        if not isinstance(message, AgentMessageThreadItem):
+            continue
+        if message.phase == MessagePhase.final_answer:
+            return message.text
+        if message.phase is None and last_unphased is None:
+            last_unphased = message.text
+    return last_unphased
+
+
+async def _close_stream(events: AsyncIterable[Any]) -> None:
+    # ``AsyncIterable`` does not guarantee ``aclose``; the stream 
``AsyncTurnHandle.stream()`` returns does.
+    aclose = getattr(events, "aclose", None)
+    if aclose is not None:
+        await aclose()
+
+
+class _Run:
+    """
+    One run of :meth:`AirflowTools.run`.
+
+    Collects its calls' failures and records whether it has ended.
+    """
+
+    def __init__(self) -> None:
+        self.failures: list[ToolCallError] = []
+        self.ended = False
+
+
+@dataclass(frozen=True)
+class _Serving:
+    """What a ``serve()`` block hands to 
:meth:`AirflowTools.build_thread_options`."""
+
+    cwd: str
+
+
+class _BearerTokenMiddleware:
+    """Let a request through only when it carries the run's bearer token; deny 
every other scope by default."""
+
+    def __init__(self, app: Any, token: str) -> None:
+        self._app = app
+        self._expected = f"Bearer {token}".encode()
+
+    async def __call__(self, scope: MutableMapping[str, Any], receive: Any, 
send: Any) -> None:
+        scope_type = scope["type"]
+        if scope_type == "lifespan":
+            await self._app(scope, receive, send)
+            return
+        supplied = next((value for name, value in scope.get("headers", []) if 
name == b"authorization"), b"")
+        if hmac.compare_digest(supplied, self._expected):
+            await self._app(scope, receive, send)
+        elif scope_type == "http":
+            await send(
+                {
+                    "type": "http.response.start",
+                    "status": 401,
+                    "headers": [(b"content-type", b"text/plain")],
+                }
+            )
+            await send({"type": "http.response.body", "body": b"Unauthorized"})
+        elif scope_type == "websocket":
+            await send({"type": "websocket.close", "code": 1008})
+
+
+class _InProcessServer(uvicorn.Server):
+    @contextlib.contextmanager
+    def capture_signals(self) -> Iterator[None]:
+        # uvicorn would replace the worker's SIGTERM and SIGINT handlers for 
as long as it serves,
+        # so the task could no longer be stopped by them. The server is 
stopped by ``serve()`` alone.
+        yield
+
+
+class AirflowTools:
+    """
+    Airflow's tools for an agent loop run by the OpenAI Codex SDK.
+
+    Pass toolsets that implement
+    :class:`~airflow.providers.common.ai.tools.ToolProvider`, such as 
``SQLToolset``
+    and ``HookToolset``, optionally wrapped with ``.filtered()``,
+    ``.prefixed()`` or ``.renamed()`` (a ``.filtered()`` function runs once, 
when this
+    is built, with ``ctx.deps`` set to ``None``), or individual
+    :class:`~airflow.providers.common.ai.tools.AirflowTool` objects. Codex has 
no
+    in-process tools, so :meth:`serve` starts an MCP server on ``127.0.0.1`` 
in the
+    running event loop and points a ``codex app-server`` child process at it. 
Start a
+    thread with :meth:`build_thread_options` and drive its turn with 
:meth:`run`:
+
+    .. code-block:: python
+
+        from openai_codex import AsyncCodex
+
+        from airflow.providers.common.ai.tools.openai_codex import AirflowTools
+        from airflow.providers.common.ai.toolsets.sql import SQLToolset
+
+        tools = AirflowTools(SQLToolset(db_conn_id="warehouse", 
allowed_tables=["orders"]))
+        async with tools.serve() as config, AsyncCodex(config) as codex:
+            await codex.login_api_key(api_key)
+            thread = await codex.thread_start(**tools.build_thread_options())
+            result = await tools.run(await thread.turn("..."))
+
+    Each tool keeps the source tool's name, description and argument schema, 
and every
+    result passes through Airflow's secret masker first. A failure the model 
can
+    correct, such as a query naming a missing column, reaches it as an 
``isError``
+    tool result. A failure the model cannot fix, such as a hook raising, makes
+    :meth:`run` raise 
:class:`~airflow.providers.common.ai.tools.ToolCallError`, so the
+    task fails and Airflow retries it.
+
+    Only the async client, ``AsyncCodex``, is supported. The MCP server runs 
in the same
+    event loop as the tools, so a synchronous ``Codex`` call that blocks that 
loop's
+    thread keeps the server from answering, and the call waits until Codex 
gives up on
+    the tool.
+
+    An instance runs one turn at a time and serves one :meth:`serve` block at 
a time:
+    build a new ``AirflowTools`` per task. A tool call that arrives after a 
run has ended
+    is refused until the next :meth:`run` starts. A call that lands after the 
next run has
+    started cannot be told apart and is served by that run.
+
+    :param sources: Toolsets and tools to serve, in order.
+    :param server_name: The MCP server name the tools are served under. 
Letters, digits,
+        ``_`` and ``-`` only, as it becomes part of a Codex config key.
+    :param tool_timeout: Seconds Codex waits for one tool call. What happens 
to a hook still
+        running when Codex gives up is not verified: the call may be 
cancelled, or a hook that
+        runs in a thread may run on, and its result is discarded either way.
+    :raises ValueError: when ``server_name`` has other characters.
+    """
+
+    def __init__(
+        self,
+        *sources: ToolSource,
+        server_name: str = "airflow",
+        tool_timeout: float = _DEFAULT_TOOL_TIMEOUT_SEC,
+    ) -> None:
+        if not _SERVER_NAME_RE.fullmatch(server_name):
+            raise ValueError(
+                f"server_name {server_name!r} must contain only letters, 
digits, '_' and '-', "
+                "because it is used as a Codex config key."
+            )
+        self.server_name = server_name
+        self.tool_timeout = tool_timeout
+        self._tools = _collect_tools(sources, mcp_hint=_MCP_HINT)
+        # ``_run`` is the run in progress (guards re-entry); ``_last_run`` is 
the one that most
+        # recently ended (refuses late calls). It is only read while ``_run`` 
is None.
+        self._run: _Run | None = None
+        self._last_run: _Run | None = None
+        self._serving: _Serving | None = None
+        self._handlers = {t.name: self._build_handler(t) for t in self._tools}
+
+    @contextlib.asynccontextmanager
+    async def serve(self) -> AsyncIterator[CodexConfig]:
+        """
+        Serve the tools over MCP for the duration of the block and yield the 
``CodexConfig`` to use.
+
+        Creates an empty ``CODEX_HOME``, an empty working directory and an 
empty ``HOME`` in a
+        temporary directory, starts the MCP server on a free ``127.0.0.1`` 
port with a
+        random bearer token, and yields a ``CodexConfig`` that points ``codex 
app-server`` at it
+        and applies the safe defaults described in
+        :ref:`the OpenAI Codex guide <howto/frameworks:openai_codex>`. On exit 
it stops the
+        server and deletes the temporary directory, so leave the block after 
the ``AsyncCodex``
+        that uses it.
+
+        The yielded ``CodexConfig`` holds the bearer token in ``env``, so its 
``repr`` does too:
+        do not log it.
+
+        Do not use the synchronous ``Codex`` client inside the block; see the 
class docstring.
+
+        :raises RuntimeError: when this instance is already serving.
+        """
+        if self._serving is not None:
+            raise RuntimeError(
+                f"{self!r} is already serving. Build a separate AirflowTools 
instance "
+                "for a concurrent run, or leave the first serve() block before 
starting another."
+            )
+        token = secrets.token_urlsafe(32)
+        base = tempfile.mkdtemp(prefix="airflow-codex-")
+        sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+        server: _InProcessServer | None = None
+        task: asyncio.Task[None] | None = None
+        try:
+            codex_home, cwd, home = (f"{base}/{name}" for name in 
("codex_home", "cwd", "home"))
+            for path in (codex_home, cwd, home):
+                os.mkdir(path)
+            # Before the first await, so that a second concurrent serve() is 
refused.
+            self._serving = _Serving(cwd=cwd)
+            # Not "localhost": it may resolve to ::1 while the socket is IPv4.
+            sock.bind(("127.0.0.1", 0))
+            sock.listen(16)
+            port = sock.getsockname()[1]
+            app = _BearerTokenMiddleware(
+                
self._build_mcp_server().streamable_http_app(host="127.0.0.1"), token
+            )
+            server = _InProcessServer(
+                uvicorn.Config(

Review Comment:
   `uvicorn.Config` defaults `log_config` to uvicorn's `LOGGING_CONFIG` and 
passes it to `logging.config.dictConfig` in `__init__`, which runs 
`_clearExistingHandlers()` and closes every handler in the task process. On 
Airflow 2 that includes the task log handler: `S3TaskHandler.close()` uploads 
and sets `closed`, so the remote log stops at `serve()` and the real upload at 
task end is skipped. Can this pass `log_config=None`, like 
`api_server_command.py` does?



##########
providers/common/ai/src/airflow/providers/common/ai/tools/openai_codex.py:
##########
@@ -0,0 +1,619 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""
+Give Airflow tools to an agent loop run by the OpenAI Codex SDK.
+
+.. note:: Experimental; see :ref:`howto/stability`.
+"""
+
+from __future__ import annotations
+
+try:
+    import uvicorn
+    from mcp import types as mcp_types
+    from mcp.server.lowlevel.server import Server
+    from openai_codex import ApprovalMode, CodexConfig, Sandbox, TurnResult
+    from openai_codex.generated.v2_all import (
+        AgentMessageThreadItem,
+        ItemCompletedNotification,
+        MessagePhase,
+        ThreadTokenUsageUpdatedNotification,
+        TurnCompletedNotification,
+        TurnStatus,
+    )
+except ImportError as e:
+    from airflow.providers.common.compat.sdk import 
AirflowOptionalProviderFeatureException
+
+    raise AirflowOptionalProviderFeatureException(e)
+
+import asyncio
+import contextlib
+import copy
+import hmac
+import json
+import logging
+import os
+import re
+import secrets
+import shutil
+import socket
+import tempfile
+import time
+from dataclasses import dataclass
+from typing import TYPE_CHECKING, Any
+
+from airflow.providers.common.ai.tools import AirflowTool, ToolCallError, 
_collect_tools
+from airflow.providers.common.ai.tools._from_toolset import tool_call_scope
+from airflow.providers.common.ai.utils.tool_definition import serialize_for_llm
+from airflow.providers.common.ai.utils.tool_metrics import calling_framework
+
+if TYPE_CHECKING:
+    from collections.abc import AsyncIterable, AsyncIterator, Awaitable, 
Callable, Iterator, MutableMapping
+
+    from mcp.server.context import ServerRequestContext
+    from openai_codex import AsyncTurnHandle
+    from openai_codex.generated.v2_all import ThreadItem, ThreadTokenUsage
+
+    from airflow.providers.common.ai.tools import ToolSource
+
+log = logging.getLogger(__name__)
+
+__all__ = ["AirflowTools"]
+
+_MCP_HINT = "Add the server to Codex's mcp_servers config instead."
+
+
+_MAX_ERROR_DETAIL_CHARS = 500
+# Every constant below was observed with openai-codex 0.163.0, not a 
documented guarantee.
+#
+# The ceiling on a tool result this adapter lets through. It is our own 
choice: unlike the Claude
+# CLI, Codex has no fixed clamp that we know of. A result over it is cut here, 
with a notice the
+# model can act on, instead of by Codex, whose truncation of an MCP result is 
unverified.
+_MAX_INLINE_RESULT_CHARS = 100_000
+# Set to the character ceiling so that Codex is unlikely to cut a result before
+# ``_truncate_result_text`` does. A token can be shorter than a character for 
some text (CJK,
+# emoji), so this is not a guarantee.
+_TOOL_OUTPUT_TOKEN_LIMIT = _MAX_INLINE_RESULT_CHARS
+# Codex's default budget is 5_000 bytes of input schema per MCP server; over 
it, Codex compresses
+# the schema and drops parameter descriptions. The largest schema of the 
toolsets this provider
+# ships is ~1 KB, so this leaves room; ``_build_config_overrides`` raises it 
for a larger one.
+_MIN_TOOL_INPUT_SCHEMA_BYTES = 65_536
+# Codex stops waiting for a tool after this long (its default is 60 s, too 
short for a SQL query).
+# What happens to the hook when Codex gives up is not verified.
+_DEFAULT_TOOL_TIMEOUT_SEC = 600.0
+# How long ``serve()`` waits for the HTTP server to start, and for open 
connections to drain on exit.
+_SERVER_START_TIMEOUT_SEC = 10.0
+# How long ``run()`` waits for Codex to confirm an interrupt before it raises 
the tool failure anyway.
+_INTERRUPT_TIMEOUT_SEC = 10.0
+_SERVER_SHUTDOWN_TIMEOUT_SEC = 5
+# The variable that carries the bearer token to the Codex child process. Only 
its name is in
+# ``codex --config`` (so in argv, visible to ``ps``); the value goes through 
the environment.
+_MCP_TOKEN_ENV_VAR = "AIRFLOW_CODEX_MCP_TOKEN"
+# ``server_name`` becomes an unquoted TOML key in 
``mcp_servers.<server_name>.*``.
+_SERVER_NAME_RE = re.compile(r"[A-Za-z0-9_-]+")
+
+
+def _truncate_result_text(text: str) -> str:
+    if len(text) <= _MAX_INLINE_RESULT_CHARS:
+        return text
+    notice = (
+        f"\n\n[Truncated: this result had {len(text)} characters and only the 
beginning is shown. "
+        "Narrow the request, for example by adding filters, selecting fewer 
columns or lowering "
+        "the LIMIT, to get a smaller result.]"
+    )
+    return text[: _MAX_INLINE_RESULT_CHARS - len(notice)] + notice
+
+
+def _bound_detail(text: str) -> str:
+    omitted = len(text) - _MAX_ERROR_DETAIL_CHARS
+    return text if omitted <= 0 else 
f"{text[:_MAX_ERROR_DETAIL_CHARS]}...(+{omitted} chars)"
+
+
+def _to_toml_string(value: str) -> str:
+    # A JSON string is a valid TOML basic string.
+    return json.dumps(value)
+
+
+def _find_final_response(items: list[ThreadItem]) -> str | None:
+    """Pick the final answer the way the SDK's ``thread.run()`` does, which 
keeps that helper private."""
+    last_unphased: str | None = None
+    for item in reversed(items):
+        message = getattr(item, "root", item)
+        if not isinstance(message, AgentMessageThreadItem):
+            continue
+        if message.phase == MessagePhase.final_answer:
+            return message.text
+        if message.phase is None and last_unphased is None:
+            last_unphased = message.text
+    return last_unphased
+
+
+async def _close_stream(events: AsyncIterable[Any]) -> None:
+    # ``AsyncIterable`` does not guarantee ``aclose``; the stream 
``AsyncTurnHandle.stream()`` returns does.
+    aclose = getattr(events, "aclose", None)
+    if aclose is not None:
+        await aclose()
+
+
+class _Run:
+    """
+    One run of :meth:`AirflowTools.run`.
+
+    Collects its calls' failures and records whether it has ended.
+    """
+
+    def __init__(self) -> None:
+        self.failures: list[ToolCallError] = []
+        self.ended = False
+
+
+@dataclass(frozen=True)
+class _Serving:
+    """What a ``serve()`` block hands to 
:meth:`AirflowTools.build_thread_options`."""
+
+    cwd: str
+
+
+class _BearerTokenMiddleware:
+    """Let a request through only when it carries the run's bearer token; deny 
every other scope by default."""
+
+    def __init__(self, app: Any, token: str) -> None:
+        self._app = app
+        self._expected = f"Bearer {token}".encode()
+
+    async def __call__(self, scope: MutableMapping[str, Any], receive: Any, 
send: Any) -> None:
+        scope_type = scope["type"]
+        if scope_type == "lifespan":
+            await self._app(scope, receive, send)
+            return
+        supplied = next((value for name, value in scope.get("headers", []) if 
name == b"authorization"), b"")
+        if hmac.compare_digest(supplied, self._expected):
+            await self._app(scope, receive, send)
+        elif scope_type == "http":
+            await send(
+                {
+                    "type": "http.response.start",
+                    "status": 401,
+                    "headers": [(b"content-type", b"text/plain")],
+                }
+            )
+            await send({"type": "http.response.body", "body": b"Unauthorized"})
+        elif scope_type == "websocket":
+            await send({"type": "websocket.close", "code": 1008})
+
+
+class _InProcessServer(uvicorn.Server):
+    @contextlib.contextmanager
+    def capture_signals(self) -> Iterator[None]:
+        # uvicorn would replace the worker's SIGTERM and SIGINT handlers for 
as long as it serves,
+        # so the task could no longer be stopped by them. The server is 
stopped by ``serve()`` alone.
+        yield
+
+
+class AirflowTools:
+    """
+    Airflow's tools for an agent loop run by the OpenAI Codex SDK.
+
+    Pass toolsets that implement
+    :class:`~airflow.providers.common.ai.tools.ToolProvider`, such as 
``SQLToolset``
+    and ``HookToolset``, optionally wrapped with ``.filtered()``,
+    ``.prefixed()`` or ``.renamed()`` (a ``.filtered()`` function runs once, 
when this
+    is built, with ``ctx.deps`` set to ``None``), or individual
+    :class:`~airflow.providers.common.ai.tools.AirflowTool` objects. Codex has 
no
+    in-process tools, so :meth:`serve` starts an MCP server on ``127.0.0.1`` 
in the
+    running event loop and points a ``codex app-server`` child process at it. 
Start a
+    thread with :meth:`build_thread_options` and drive its turn with 
:meth:`run`:
+
+    .. code-block:: python
+
+        from openai_codex import AsyncCodex
+
+        from airflow.providers.common.ai.tools.openai_codex import AirflowTools
+        from airflow.providers.common.ai.toolsets.sql import SQLToolset
+
+        tools = AirflowTools(SQLToolset(db_conn_id="warehouse", 
allowed_tables=["orders"]))
+        async with tools.serve() as config, AsyncCodex(config) as codex:
+            await codex.login_api_key(api_key)
+            thread = await codex.thread_start(**tools.build_thread_options())
+            result = await tools.run(await thread.turn("..."))
+
+    Each tool keeps the source tool's name, description and argument schema, 
and every
+    result passes through Airflow's secret masker first. A failure the model 
can
+    correct, such as a query naming a missing column, reaches it as an 
``isError``
+    tool result. A failure the model cannot fix, such as a hook raising, makes
+    :meth:`run` raise 
:class:`~airflow.providers.common.ai.tools.ToolCallError`, so the
+    task fails and Airflow retries it.
+
+    Only the async client, ``AsyncCodex``, is supported. The MCP server runs 
in the same
+    event loop as the tools, so a synchronous ``Codex`` call that blocks that 
loop's
+    thread keeps the server from answering, and the call waits until Codex 
gives up on
+    the tool.
+
+    An instance runs one turn at a time and serves one :meth:`serve` block at 
a time:
+    build a new ``AirflowTools`` per task. A tool call that arrives after a 
run has ended
+    is refused until the next :meth:`run` starts. A call that lands after the 
next run has
+    started cannot be told apart and is served by that run.
+
+    :param sources: Toolsets and tools to serve, in order.
+    :param server_name: The MCP server name the tools are served under. 
Letters, digits,
+        ``_`` and ``-`` only, as it becomes part of a Codex config key.
+    :param tool_timeout: Seconds Codex waits for one tool call. What happens 
to a hook still
+        running when Codex gives up is not verified: the call may be 
cancelled, or a hook that
+        runs in a thread may run on, and its result is discarded either way.
+    :raises ValueError: when ``server_name`` has other characters.
+    """
+
+    def __init__(
+        self,
+        *sources: ToolSource,
+        server_name: str = "airflow",
+        tool_timeout: float = _DEFAULT_TOOL_TIMEOUT_SEC,
+    ) -> None:
+        if not _SERVER_NAME_RE.fullmatch(server_name):
+            raise ValueError(
+                f"server_name {server_name!r} must contain only letters, 
digits, '_' and '-', "
+                "because it is used as a Codex config key."
+            )
+        self.server_name = server_name
+        self.tool_timeout = tool_timeout
+        self._tools = _collect_tools(sources, mcp_hint=_MCP_HINT)
+        # ``_run`` is the run in progress (guards re-entry); ``_last_run`` is 
the one that most
+        # recently ended (refuses late calls). It is only read while ``_run`` 
is None.
+        self._run: _Run | None = None
+        self._last_run: _Run | None = None
+        self._serving: _Serving | None = None
+        self._handlers = {t.name: self._build_handler(t) for t in self._tools}
+
+    @contextlib.asynccontextmanager
+    async def serve(self) -> AsyncIterator[CodexConfig]:
+        """
+        Serve the tools over MCP for the duration of the block and yield the 
``CodexConfig`` to use.
+
+        Creates an empty ``CODEX_HOME``, an empty working directory and an 
empty ``HOME`` in a
+        temporary directory, starts the MCP server on a free ``127.0.0.1`` 
port with a
+        random bearer token, and yields a ``CodexConfig`` that points ``codex 
app-server`` at it
+        and applies the safe defaults described in
+        :ref:`the OpenAI Codex guide <howto/frameworks:openai_codex>`. On exit 
it stops the
+        server and deletes the temporary directory, so leave the block after 
the ``AsyncCodex``
+        that uses it.
+
+        The yielded ``CodexConfig`` holds the bearer token in ``env``, so its 
``repr`` does too:
+        do not log it.
+
+        Do not use the synchronous ``Codex`` client inside the block; see the 
class docstring.
+
+        :raises RuntimeError: when this instance is already serving.
+        """
+        if self._serving is not None:
+            raise RuntimeError(
+                f"{self!r} is already serving. Build a separate AirflowTools 
instance "
+                "for a concurrent run, or leave the first serve() block before 
starting another."
+            )
+        token = secrets.token_urlsafe(32)
+        base = tempfile.mkdtemp(prefix="airflow-codex-")
+        sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+        server: _InProcessServer | None = None
+        task: asyncio.Task[None] | None = None
+        try:
+            codex_home, cwd, home = (f"{base}/{name}" for name in 
("codex_home", "cwd", "home"))
+            for path in (codex_home, cwd, home):
+                os.mkdir(path)
+            # Before the first await, so that a second concurrent serve() is 
refused.
+            self._serving = _Serving(cwd=cwd)
+            # Not "localhost": it may resolve to ::1 while the socket is IPv4.
+            sock.bind(("127.0.0.1", 0))
+            sock.listen(16)
+            port = sock.getsockname()[1]
+            app = _BearerTokenMiddleware(
+                
self._build_mcp_server().streamable_http_app(host="127.0.0.1"), token
+            )
+            server = _InProcessServer(
+                uvicorn.Config(
+                    app,
+                    log_level="warning",
+                    access_log=False,
+                    lifespan="on",
+                    timeout_graceful_shutdown=_SERVER_SHUTDOWN_TIMEOUT_SEC,
+                )
+            )
+            task = asyncio.create_task(server.serve(sockets=[sock]))
+            await self._wait_for_server(server, task)
+            yield CodexConfig(
+                config_overrides=self._build_config_overrides(port),
+                cwd=cwd,
+                # CodexConfig.env is merged over the worker's environment, so 
each variable that
+                # points Codex at worker state has to be overridden here.
+                env={
+                    # Without it Codex reads ~/.codex (config.toml, AGENTS.md, 
skills, mcp_servers)
+                    # and writes its state there.
+                    "CODEX_HOME": codex_home,
+                    # A CODEX_SQLITE_HOME in the worker's environment would 
send the state DB outside
+                    # the temporary directory (observed). Whether the 
``sqlite_home`` config key alone
+                    # is enough is unverified, so this variable is what we 
rely on.
+                    "CODEX_SQLITE_HOME": codex_home,
+                    "HOME": home,
+                    _MCP_TOKEN_ENV_VAR: token,
+                },
+            )
+        finally:
+            self._serving = None
+            if server is not None and task is not None:
+                server.should_exit = True
+                for outcome in await asyncio.gather(task, 
return_exceptions=True):
+                    if isinstance(outcome, Exception):
+                        log.warning("The MCP server for the Codex tools ended 
with an error: %r", outcome)
+            sock.close()
+            shutil.rmtree(base, ignore_errors=True)
+
+    def build_thread_options(self, **kwargs: Any) -> dict[str, Any]:
+        """
+        Build the keyword arguments for ``AsyncCodex.thread_start``, with safe 
defaults.
+
+        Defaults ``approval_mode`` to ``ApprovalMode.deny_all`` and 
``sandbox`` to
+        ``Sandbox.read_only``, ``cwd`` to the empty directory :meth:`serve` 
created (so that no
+        project ``.codex/config.toml`` or ``AGENTS.md`` is found), and 
``ephemeral`` to ``True``.
+        ``AsyncCodex`` always accepts approval requests for command execution 
and file changes,
+        so the only way to refuse them is to make Codex never ask. A keyword 
you pass wins.
+        The model and anything else are passed through unchanged.
+
+        Call it inside the :meth:`serve` block.
+
+        :raises RuntimeError: when called outside the :meth:`serve` block.
+        :raises ValueError: when ``config`` already has an ``mcp_servers`` 
entry named
+            :attr:`server_name`.
+        """
+        if self._serving is None:
+            raise RuntimeError("build_thread_options() has to be called inside 
the serve() block.")
+        mcp_servers = (kwargs.get("config") or {}).get("mcp_servers") or {}
+        if self.server_name in mcp_servers:
+            raise ValueError(
+                f"config already has an MCP server named {self.server_name!r}. 
Pass a different "
+                "server_name to AirflowTools, or drop it from config."
+            )
+        kwargs.setdefault("approval_mode", ApprovalMode.deny_all)
+        kwargs.setdefault("sandbox", Sandbox.read_only)
+        kwargs.setdefault("cwd", self._serving.cwd)
+        # Observed to be accepted by 0.163.0; that it keeps the thread off 
disk is assumed.
+        kwargs.setdefault("ephemeral", True)
+        return kwargs
+
+    async def run(self, turn: AsyncTurnHandle) -> TurnResult:
+        """
+        Drive ``turn``, the handle ``AsyncThread.turn()`` returns, until it 
completes.
+
+        On a tool failure the model cannot fix, the turn is interrupted before 
the failure
+        is raised. It is also interrupted when ``run()`` is cancelled before 
the turn completes,
+        for example by a timeout: closing the stream would not stop Codex, 
which would keep
+        calling tools. The interrupt is a request, bounded to 10 seconds; if 
it fails, that is
+        logged and does not replace the original exception, and a cancellation 
still propagates
+        as ``asyncio.CancelledError``.
+        A tool call that arrives after this returns or raises is answered with 
an ``isError``
+        "Not run" result.
+
+        :raises RuntimeError: when this instance is already running a turn, 
when the stream ends
+            without a ``turn/completed`` notification, or when the turn did 
not complete
+            (it failed, or was interrupted by something other than a tool 
failure).
+        :raises ToolCallError: when a tool failed in a way the model cannot 
fix.
+        """
+        if self._run is not None:
+            raise RuntimeError(
+                f"{self!r} is already running a turn. Build a separate 
AirflowTools instance "
+                "for a concurrent run, or await the first run before starting 
another."
+            )
+        events = turn.stream()
+        state = _Run()
+        self._run = state
+        raised = True
+        completed: TurnCompletedNotification | None = None
+        interrupted = False
+        try:
+            items: list[ThreadItem] = []
+            usage: ThreadTokenUsage | None = None
+            async for event in events:
+                payload = event.payload
+                if isinstance(payload, ItemCompletedNotification) and 
payload.turn_id == turn.id:
+                    items.append(payload.item)
+                elif isinstance(payload, ThreadTokenUsageUpdatedNotification) 
and payload.turn_id == turn.id:
+                    usage = payload.token_usage
+                elif isinstance(payload, TurnCompletedNotification) and 
payload.turn.id == turn.id:
+                    completed = payload
+                if state.failures:
+                    interrupted = True
+                    await self._interrupt_turn(turn)
+                    raise state.failures[0]
+            if completed is None:
+                if state.failures:
+                    raise state.failures[0]
+                raise RuntimeError("The Codex turn stream ended without a 
turn/completed notification.")
+            finished = completed.turn
+            if finished.status != TurnStatus.completed:
+                detail = f": {_bound_detail(finished.error.message)}" if 
finished.error else ""
+                raise RuntimeError(f"The Codex turn ended with status 
{finished.status.value}{detail}")
+            raised = False
+            return TurnResult(
+                id=finished.id,
+                status=finished.status,
+                error=finished.error,
+                started_at=finished.started_at,
+                completed_at=finished.completed_at,
+                duration_ms=finished.duration_ms,
+                final_response=_find_final_response(items),
+                items=items,
+                usage=usage,
+            )
+        except asyncio.CancelledError:
+            # Interrupt before the stream is closed and ``self._run`` is 
cleared: a late tool call
+            # from this turn would otherwise resolve to the next run's state.
+            if completed is None and not interrupted:
+                await self._interrupt_turn(turn)
+            raise
+        finally:
+            try:
+                await _close_stream(events)
+            except Exception as e:
+                if not raised:
+                    raise
+                log.warning("Closing the Codex turn stream failed: %s", e)
+            finally:
+                state.ended = True
+                self._last_run = state
+                self._run = None
+
+    @staticmethod
+    async def _interrupt_turn(turn: AsyncTurnHandle) -> None:
+        try:
+            # The SDK runs the interrupt in a worker thread (``to_thread``). 
On timeout we stop
+            # waiting, but that thread is only released when ``AsyncCodex`` 
closes; keep the timeout,
+            # or an unresponsive app-server hangs the task instead of failing 
it.
+            await asyncio.wait_for(turn.interrupt(), _INTERRUPT_TIMEOUT_SEC)
+        except Exception as e:
+            log.warning("Interrupting the Codex turn failed: %s", e)
+
+    @staticmethod
+    async def _wait_for_server(server: uvicorn.Server, task: 
asyncio.Task[None]) -> None:
+        deadline = time.monotonic() + _SERVER_START_TIMEOUT_SEC
+        while not server.started:
+            if task.done():
+                raise RuntimeError("The MCP server for the Codex tools stopped 
while starting.") from (
+                    None if task.cancelled() else task.exception()
+                )
+            if time.monotonic() > deadline:
+                raise RuntimeError(
+                    f"The MCP server for the Codex tools did not start within 
{_SERVER_START_TIMEOUT_SEC} seconds."
+                )
+            await asyncio.sleep(0.01)
+
+    def _build_config_overrides(self, port: int) -> tuple[str, ...]:
+        """
+        Build the ``codex --config key=value`` overrides.
+
+        Codex silently ignores a key it does not know, so every key is covered 
by a test that
+        reads it back from a real ``codex app-server``. Values are TOML.
+        """
+        prefix = f"mcp_servers.{self.server_name}"
+        largest_schema = max((len(json.dumps(t.parameters)) for t in 
self._tools), default=0)
+        return (
+            # Login credentials stay in memory instead of an auth file in 
CODEX_HOME.
+            'cli_auth_credentials_store="ephemeral"',
+            # The model must not get a shell on the worker. Whether these 
flags remove the tool
+            # from the model's tool list is unverified without a model.
+            "features.shell_tool=false",
+            "features.unified_exec=false",
+            # The model must not read an image file from the worker, outside 
the secret masker.
+            "features.view_image=false",
+            # Capabilities with no use for Airflow tools.
+            "features.multi_agent=false",
+            "features.apps=false",
+            "features.plugins=false",
+            "features.image_generation=false",
+            # Data leaves the worker only through the Airflow tools, never 
through a web search.
+            'web_search="disabled"',
+            # If a command still ran, it would get none of the worker's 
environment (AIRFLOW_CONN_*).
+            'shell_environment_policy.inherit="none"',
+            # Keep session history, analytics, feedback, update checks and 
metrics out of the picture.
+            'history.persistence="none"',
+            "analytics.enabled=false",
+            "feedback.enabled=false",
+            "check_for_update_on_startup=false",
+            'otel.metrics_exporter="none"',
+            # See _TOOL_OUTPUT_TOKEN_LIMIT. Whether this limit applies to MCP 
results is unverified.
+            f"tool_output_token_limit={_TOOL_OUTPUT_TOKEN_LIMIT}",
+            f"{prefix}.url={_to_toml_string(f'http://127.0.0.1:{port}/mcp')}",
+            # The token reaches the child through its environment, not argv.
+            
f"{prefix}.bearer_token_env_var={_to_toml_string(_MCP_TOKEN_ENV_VAR)}",
+            # Fail the thread start when the server is unreachable instead of 
running with no tools.
+            f"{prefix}.required=true",
+            f"{prefix}.tool_timeout_sec={self.tool_timeout}",
+            # The docs promise that hook calls run one at a time. Codex does 
not echo ``false``, so
+            # whether it equals the default is unconfirmed; ``true`` is echoed.
+            f"{prefix}.supports_parallel_tool_calls=false",

Review Comment:
   If this does what the guide says and Codex sends one response's calls one 
after another, the `turn=None` budget never sees two calls at the same time. 
Two failing calls to the same tool in one model response then use up the 
default `max_retries=1` before the model gets to correct, and the guide's line 
about calls at the same time counting as one never applies here. Is this 
needed, given `sequential` tools already queue in `_from_toolset`? 
Non-blocking, since the guide already says the budget runs out sooner here.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to