This is an automated email from the ASF dual-hosted git repository.
jason810496 pushed a change to branch
jason/core-taskhandler-refactor/06-go-sdk-task-handler-parse
in repository https://gitbox.apache.org/repos/asf/airflow.git
discard b1151dc3b23 Describe the two task handler bindings for Language SDK
authors
discard dc549f98eb3 Go SDK: declare every lone struct handler as named
discard 54363fc4a9d State that a task handler parse answer depends only on the
artifact
discard 8f74493f2be Go SDK: answer a task handler parse with every handler
discard dd9027215fb Tidy names, mocks and cases in the Go probe and digest
tests
discard 5e38a0ba4ee Move Go SDK comments back to the code they describe
discard de6d01540b6 Resolve the probed bundle path and tighten the digest docs
discard 609c46c0038 Go SDK: take the bundle digests from the staged executable
discard 6ef66f078a5 Document the task handler parse for Language SDK authors
discard b1f99665ded Read a Go bundle's stored cache digest without hashing it
discard 5f5c3397476 Let the Executable coordinator probe a bundle's task
handlers
discard 5dc64b4eef2 Go SDK: record integrity and cache digests in bundle
metadata
discard bfadfbe16b6 Go SDK: regenerate genmodels for the named_open binding
discard 9fa167c731c Go SDK: clarify parse timeout and nested-null docs, test
anyOf
discard 795b97cb9cd Go SDK: answer TaskHandlerParseRequest with registered
handlers
discard 9a8627af5d3 Go SDK: describe task handler params with JSON Schema
discard faf5e976f0c Go SDK: regenerate genmodels from the supervisor schema
discard 7a8bab885bc Build the probe test's params without the dropped required
discard eb406929ad3 Probe a task handler artifact without naming Dags
discard f1ac0ab61d4 Close the fake runtime's sockets when its reply is done
discard 1206141d55b Name the runtime transport test helpers with verbs
discard fde3390849a Patch with decorators in the runtime transport tests
discard c1861502159 Hang the fake runtimes without sleeping
discard b12cf16c735 Test that a start failing before the fork closes its
listeners
discard 4acdded53e1 Spec the mocks in the runtime transport and probe tests
discard 8e0f64d7ee3 Import ctypes at the top of the subprocess coordinator
module
discard 25a918a97cb Pass the binding mode in the task-handler probe tests
discard 7a7f588a6bc Tidy the runtime transport and the coordinator lookup
discard c32ed26aa98 Clean up a Lang-SDK runtime parse whose start fails after
the fork
discard 9856d8b0cc1 Do not exec a Lang-SDK runtime whose parent has already
exited
discard a1782e45de9 Kill what a Lang-SDK runtime leaves behind when it exits
discard 6a37b6d6166 Read a Lang-SDK runtime's frames without blocking the
parse loop
discard 0ab3e095f6d Probe a Lang-SDK artifact for its task handlers
discard b3d5f26dcdd Keep a nested supervised child's standard streams open
discard 400d08b04d4 Make the Dag file processor base generic over its result
discard 7e229e82f86 Let a subprocess coordinator exec a task-handler parse
runtime
discard 48f58db7caf Describe the two task handler binding modes in the ADRs
discard 6c3f0eda7c9 Regenerate TS SDK supervisor types for the two binding
modes
discard 69ac8da87fb Bind task handlers only by position or by name
discard b88bd4f5fe9 Describe the all-handlers probe in ADR-0011 and ADR-0012
discard 561cc63ae9c Regenerate TS SDK supervisor types for the probe without
dag_ids
discard c4346b16be0 Ask a task handler runtime for every handler it registers
discard 51b4d1ae876 Add the named_open binding to the Lang-SDK ADRs
discard cf24a55709f Regenerate TS SDK supervisor types for the named_open
binding
discard b3daf463995 Add a named_open binding for non-exhaustive task-handler
params
discard 00fb770fe2d Describe the binding-mode parse check in ADR-0011 and
ADR-0013
discard 0c5783121c4 Describe the task-handler binding mode in ADR-0012
discard 44b9e8a330c Regenerate TS SDK supervisor types for the binding mode
discard 22b1ecde5ed Add a binding mode to task-handler declarations
discard 2d63e560647 Document the version rules for a new supervisor schema body
discard 6787551b0ff Regenerate TS SDK supervisor types for task-handler
messages
discard bd591739323 Add task-handler parse messages and their parse-channel
union
discard 59a5bc27503 Test closing a Dag file processor that has no parse log
file
discard ca2c7c99499 Move the Dag file processor's shared plumbing into a base
class
discard 9bb714bdf05 Allow a NULL cache digest in ADR-0013's artifact table SQL
discard 45a0221d614 Let a task handler artifact store no cache digest
discard 45e7b7b351b Move the handler declarations to the artifact in
ADR-0013's SQL
discard 6d194fdee7d Cache an artifact's task handlers on its row, not on each
binding
discard 96789418983 Make session keyword-only in the task handler model test
helpers
discard 860dad5b421 Add handler_binding to ADR-0013's task handler table SQL
discard faa8558073b Store the task handler binding mode beside its params
discard 88a952d6923 Regenerate the task handler migration revision id
discard 34595f1e611 Describe LangSDKTaskHandler as a stub task binding
discard 1c782678224 Test that a referenced task handler artifact cannot be
deleted
discard 73715f836fd Keep the artifact path hash in sync on ORM updates
discard 4da20330927 Match ADR-0013 task handler table SQL to the migration
discard 37beb8ec4f5 Add lang-SDK task handler binding tables
discard 7d1d54f8df1 Say the Dag processor needs the language SDK setup too
discard 19efe03bcd7 Drop em-dashes from the rewritten doc sentences
discard e6c81d92775 Say artifact Dag bundles are registered on every component
discard 4a48d373b16 Tidy names and mocks in the artifact bundle tests
discard 3dc8488c286 Document artifact Dag bundles for language SDK coordinators
discard 4cb590183ac Serve lang-SDK harness artifacts from Dag bundles
discard d919f7ef61f Replace coordinator artifact roots with
task_handler_bundle_name
add 16f974f6a35 Run the `common.ai` agent framework adapter tests in CI
(#73912)
add e364ee76483 Keep agent framework spans when worker tracing is off
(#73903)
add 2e803e58997 Add key-pair JWT authentication to Snowflake Cortex Agent
hook (#73815)
add e92147c12aa UI: Add a time-window scope to the Dags list run state
filter (#70470)
add 63a5966473b Check PyPI history before a new provider's first release
(#73887)
add ede78e095b6 Add embedding kwargs to common AI hooks and operators
(#72002)
add 64943ba6dbb Fix `run_duration` calculation in airflow dags test by
decoupling start_date (#68837)
add 9e67831f2bb Allow filtering the Dag runs list by Dag tags (#70861)
add e22c9d02b2e Add Has Events filter to Assets search (#72460)
add f31b572df18 Fix airflowctl commands crashing instead of printing their
result (#72675)
add 1559dae90ab UI: Use TeamName component for all team name displays
(#73910)
add 489316fbc72 Migrate execution API reads to async sessions (#73403)
add a8e95debbc6 Fix flaky AgentOperator durable usage budget tests (#73996)
add e33e45fe67a Add `AssetAndTimeSchedule` timetable (#58543)
add a91216d5ea9 TS SDK: serialize a native Dag to Dag JSON (#73441)
add 884e687ea77 Fix short-circuit inside a mapped task group not skipping
downstream (#73959)
add 3f19c146545 Go SDK: declare data edges with airflow.Inputs (#73895)
add 13adb691464 Close German Translation Gaps 2026-09-30 (#73980)
add d886882ac14 Let Common AI toolsets use the agent's tool retry budget
(#73957)
add 35633455ed7 Add first-class `capabilities` to Common AI
`AgentOperator` (#73984)
add adaa16f88af Parse Dag files through the Task SDK importers (#73853)
add 12127b07978 Fix check-conn-id-templated crashing on a vanishing
__pycache__ (#74016)
add 4d304d0876c Export files an agent built in a sandbox to object storage
(#73990)
add e9c552cc827 Add a vendor-neutral managed-agent hook contract to Common
AI (#73532)
add a33fe27cace Add async client to `AnthropicHook` (#73967)
add e6a9de5d832 Go SDK: bring the coordinator-protocol models back to the
supervisor schema (#73963)
add 27cd88ca74c Cache repeated agent prompts by default in AgentOperator
(#73994)
add 529f429400a Go SDK: reject a task handler and a Dag that share a
dag_id (#73876)
add bc9ca5b1227 Correctly shutdown async sessions on exit. (#73838)
add f1b36db6093 Back off between GitDagBundle bare clone attempts so a
fresh GitHub App token can propagate (#73878)
add 6b847411de1 Stop rebuilding the FAB app on every users and roles API
call (#73104)
add 56817294cdd Go SDK: add airflow.TriggerDagRun, a task that triggers a
Dag run (#74003)
add b8475094703 Fix go-sdk generated drift (#74028)
add 8f3e8466c67 Support Airflow 2.11 in the Common AI provider (#73991)
add f0cde671827 Use "Airflow versions" in user-facing docs, not "cores"
(#74045)
add 0bb09cb6ba5 Close the providers testing issue right after announcing
the release (#73890)
add 43377466a06 Release the DuckDB provider as 0.2.0 (#73888)
add 53e43f29130 Speed up CI image restores with zstd-compressed stashes
(#73997)
add e4d397f021e Bound get_schema results for very wide tables in the SQL
toolsets (#74017)
add 7d6244bf191 Clarify `max_db_retries` doc wording to avoid off-by-one
confusion (#73722)
add bb463bf9230 Correlate executor task events by attempt UUID (#73916)
add c19faf0c5ae Render the HITL review workflow as a Mermaid diagram
(#74024)
add 2aa31ae7bff Use airflow.sdk imports in Airflow 3 upgrade guide example
(#74015)
add f3532c60eda Use Task SDK TaskInstanceState in OpenSearch log handler
(#72444)
add 0ec26723858 Add GKEPodExecOperator for existing Pods (#72577)
add 740986adfce Add missing cycle assertion to test_semi_complex (#71973)
add fca1c01c73e Fix Windows UnicodeDecodeError during pytest collection
(#73536)
add 1787ef7cbba Run the deferred AWS hook configuration check as a prek
hook (#73681)
add 413ae46a045 Align timetable docs with Airflow 3 data interval and
catchup behaviour (#70658)
add bd9be97f70e Replace coordinator artifact roots with
task_handler_bundle_name
add ace4b2eeaa9 Serve lang-SDK harness artifacts from Dag bundles
add 4bfc18153e8 Document artifact Dag bundles for language SDK coordinators
add b9cde7fe612 Tidy names and mocks in the artifact bundle tests
add 23898f0586c Say artifact Dag bundles are registered on every component
add 8997a357c46 Drop em-dashes from the rewritten doc sentences
add 8983f6dc234 Say the Dag processor needs the language SDK setup too
add f538f1f2f56 Add lang-SDK task handler binding tables
add 2db819349d4 Match ADR-0013 task handler table SQL to the migration
add fbae76d020f Keep the artifact path hash in sync on ORM updates
add 8b1b2554fd9 Test that a referenced task handler artifact cannot be
deleted
add 78f7154fc41 Describe LangSDKTaskHandler as a stub task binding
add 0c9521838f0 Regenerate the task handler migration revision id
add b8ea4dc87d9 Store the task handler binding mode beside its params
add 08b3428d5fd Add handler_binding to ADR-0013's task handler table SQL
add 6f8fa4d28a6 Make session keyword-only in the task handler model test
helpers
add 50274e47e05 Cache an artifact's task handlers on its row, not on each
binding
add 65e8396ac6b Move the handler declarations to the artifact in
ADR-0013's SQL
add b2a45718777 Let a task handler artifact store no cache digest
add 1db024e76b7 Allow a NULL cache digest in ADR-0013's artifact table SQL
add 438e073af11 Move the Dag file processor's shared plumbing into a base
class
add c6ea332cc53 Test closing a Dag file processor that has no parse log
file
add d148fa37175 Add task-handler parse messages and their parse-channel
union
add 65e405f46e1 Regenerate TS SDK supervisor types for task-handler
messages
add ae2f0e35146 Document the version rules for a new supervisor schema body
add 259f707da38 Add a binding mode to task-handler declarations
add 6b256280a83 Regenerate TS SDK supervisor types for the binding mode
add 65f47584d62 Describe the task-handler binding mode in ADR-0012
add a60b181c39a Describe the binding-mode parse check in ADR-0011 and
ADR-0013
add b8de183cce0 Add a named_open binding for non-exhaustive task-handler
params
add e0d7a0a24bf Regenerate TS SDK supervisor types for the named_open
binding
add f202225ab80 Add the named_open binding to the Lang-SDK ADRs
add c090d5f84b4 Ask a task handler runtime for every handler it registers
add 0fe6c7bd317 Regenerate TS SDK supervisor types for the probe without
dag_ids
add dd1174d4558 Describe the all-handlers probe in ADR-0011 and ADR-0012
add 13853f06ade Bind task handlers only by position or by name
add 31154a6ffe9 Regenerate TS SDK supervisor types for the two binding
modes
add 37eb81323aa Regenerate Go SDK models for task-handler messages
add 017a4010644 Describe the two task handler binding modes in the ADRs
add 80c989b2c05 Keep a nested supervised child's standard streams open
add d9f8c3aba41 Let a subprocess coordinator exec a task-handler parse
runtime
add 2f07e848745 Probe a Lang-SDK artifact for its task handlers
add 71ee14604ac Read a Lang-SDK runtime's frames without blocking the
parse loop
add d432d42e704 Kill what a Lang-SDK runtime leaves behind when it exits
add b4025a25e30 Do not exec a Lang-SDK runtime whose parent has already
exited
add 69e2b807dd8 Clean up a Lang-SDK runtime parse whose start fails after
the fork
add ad605d636bb Go SDK: describe task handler params with JSON Schema
add 51e0a222867 Go SDK: answer TaskHandlerParseRequest with registered
handlers
add cc8a102f52d Go SDK: clarify parse timeout and nested-null docs, test
anyOf
add 2acb75863aa Go SDK: record integrity and cache digests in bundle
metadata
add cd5d45c1ef1 Let the Executable coordinator probe a bundle's task
handlers
add 641aa09e0d4 Read a Go bundle's stored cache digest without hashing it
add 113c3818b44 Document the task handler parse for Language SDK authors
add 3d796536894 Go SDK: take the bundle digests from the staged executable
add 51248188736 Resolve the probed bundle path and tighten the digest docs
add 146ec8981c8 Move Go SDK comments back to the code they describe
add 405713a5cbb Tidy names, mocks and cases in the Go probe and digest
tests
add 8dc677e3331 Go SDK: answer a task handler parse with every handler
add ddb48ee3075 State that a task handler parse answer depends only on the
artifact
add 9e1936b10ff Go SDK: declare every lone struct handler as named
add 0fb5c4c2fda Describe the two task handler bindings for Language SDK
authors
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (b1151dc3b23)
\
N -- N -- N
refs/heads/jason/core-taskhandler-refactor/06-go-sdk-task-handler-parse
(0fb5c4c2fda)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
...-image-checks.yml => agent-framework-tests.yml} | 68 +-
.github/workflows/ci-amd.yml | 19 +
.github/workflows/ci-arm.yml | 19 +
.github/workflows/ci-image-build.yml | 8 +
.pre-commit-config.yaml | 27 +-
.../lang-sdk/0011-mixed-language-dag-processing.md | 4 +-
.../adr/lang-sdk/0012-lang-sdk-parse-protocol.md | 10 +-
.../0013-persisted-task-handler-bindings.md | 4 +-
.../authoring-and-scheduling/asset-scheduling.rst | 31 +-
.../language-sdks/typescript.rst | 23 +
.../docs/authoring-and-scheduling/timetable.rst | 179 +++--
airflow-core/docs/core-concepts/executor/index.rst | 23 +-
.../docs/installation/upgrading_to_airflow3.rst | 2 +-
airflow-core/docs/migrations-ref.rst | 4 +-
airflow-core/newsfragments/58543.feature.rst | 1 +
airflow-core/newsfragments/68496.significant.rst | 4 +-
airflow-core/src/airflow/api_fastapi/app.py | 2 +
.../api_fastapi/common/parameters/__init__.py | 3 +
.../airflow/api_fastapi/common/parameters/asset.py | 23 +
.../airflow/api_fastapi/common/parameters/dag.py | 46 ++
.../api_fastapi/common/parameters/dag_run.py | 35 +-
.../api_fastapi/core_api/openapi/_private_ui.yaml | 24 +
.../core_api/openapi/v2-rest-api-generated.yaml | 19 +
.../api_fastapi/core_api/routes/public/dag_run.py | 4 +
.../api_fastapi/core_api/routes/ui/assets.py | 3 +
.../src/airflow/api_fastapi/execution_api/app.py | 2 +
.../api_fastapi/execution_api/routes/assets.py | 14 +-
.../api_fastapi/execution_api/routes/dag_runs.py | 18 +-
.../api_fastapi/execution_api/routes/hitl.py | 12 +-
.../execution_api/routes/task_instances.py | 28 +-
.../execution_api/routes/task_reschedules.py | 6 +-
.../src/airflow/config_templates/config.yml | 43 +-
.../src/airflow/dag_processing/collection.py | 21 +-
airflow-core/src/airflow/dag_processing/dagbag.py | 232 +++---
.../airflow/dag_processing/importers/__init__.py | 39 -
.../src/airflow/dag_processing/importers/base.py | 266 -------
.../dag_processing/importers/python_importer.py | 360 ---------
.../src/airflow/dag_processing/lang_sdk_runtime.py | 553 --------------
airflow-core/src/airflow/dag_processing/manager.py | 2 +
.../src/airflow/dag_processing/processor.py | 55 +-
.../dag_processing/task_handler_processor.py | 548 +++++++++++--
.../src/airflow/executors/base_executor.py | 123 ++-
.../src/airflow/executors/local_executor.py | 13 +-
.../src/airflow/executors/workloads/task.py | 3 +-
.../src/airflow/executors/workloads/types.py | 17 +-
.../src/airflow/jobs/scheduler_job_runner.py | 256 ++++---
...3_4_0_add_timetable_asset_gated_to_dagmodel.py} | 29 +-
...0142_3_4_0_add_lang_sdk_task_handler_tables.py} | 4 +-
airflow-core/src/airflow/models/dag.py | 32 +-
airflow-core/src/airflow/models/dagcode.py | 45 +-
airflow-core/src/airflow/models/dagrun.py | 2 +-
airflow-core/src/airflow/models/serialized_dag.py | 19 +-
airflow-core/src/airflow/models/taskinstance.py | 2 +-
airflow-core/src/airflow/serialization/encoders.py | 9 +
airflow-core/src/airflow/settings.py | 14 +-
.../ti_deps/deps/not_previously_skipped_dep.py | 6 +-
airflow-core/src/airflow/timetables/assets.py | 112 ++-
airflow-core/src/airflow/timetables/base.py | 16 +
airflow-core/src/airflow/timetables/simple.py | 1 +
.../src/airflow/ui/openapi-gen/queries/common.ts | 16 +-
.../ui/openapi-gen/queries/ensureQueryData.ts | 20 +-
.../src/airflow/ui/openapi-gen/queries/prefetch.ts | 20 +-
.../src/airflow/ui/openapi-gen/queries/queries.ts | 20 +-
.../src/airflow/ui/openapi-gen/queries/suspense.ts | 20 +-
.../ui/openapi-gen/requests/services.gen.ts | 8 +
.../airflow/ui/openapi-gen/requests/types.gen.ts | 10 +
.../airflow/ui/public/i18n/locales/ar/dags.json | 2 -
.../airflow/ui/public/i18n/locales/ca/dags.json | 4 -
.../airflow/ui/public/i18n/locales/de/browse.json | 30 +
.../airflow/ui/public/i18n/locales/de/common.json | 26 +
.../ui/public/i18n/locales/de/components.json | 5 +
.../src/airflow/ui/public/i18n/locales/de/dag.json | 5 +
.../airflow/ui/public/i18n/locales/de/dags.json | 24 +-
.../ui/public/i18n/locales/de/dashboard.json | 17 +-
.../airflow/ui/public/i18n/locales/el/dags.json | 2 -
.../airflow/ui/public/i18n/locales/en/assets.json | 1 +
.../airflow/ui/public/i18n/locales/en/common.json | 9 +-
.../airflow/ui/public/i18n/locales/en/dags.json | 6 +-
.../airflow/ui/public/i18n/locales/es/dags.json | 4 -
.../airflow/ui/public/i18n/locales/fr/dags.json | 2 -
.../airflow/ui/public/i18n/locales/he/dags.json | 2 -
.../airflow/ui/public/i18n/locales/hi/dags.json | 2 -
.../airflow/ui/public/i18n/locales/ko/dags.json | 4 -
.../airflow/ui/public/i18n/locales/pl/dags.json | 4 -
.../airflow/ui/public/i18n/locales/ru/dags.json | 2 -
.../airflow/ui/public/i18n/locales/zh-CN/dags.json | 4 -
.../airflow/ui/public/i18n/locales/zh-TW/dags.json | 4 -
.../FilterBar/filters/RunStateFilter.tsx | 234 ++++++
.../FilterBar/filters/runStateParams.test.ts | 95 +++
.../components/FilterBar/filters/runStateParams.ts | 76 ++
.../airflow/ui/src/components/FilterBar/types.ts | 14 +-
.../src/airflow/ui/src/constants/filterConfigs.tsx | 38 +-
.../src/airflow/ui/src/constants/searchParams.ts | 3 +
.../src/airflow/ui/src/mocks/handlers/dag_runs.ts | 49 +-
.../src/airflow/ui/src/mocks/handlers/dags.ts | 10 +
.../ui/src/pages/AssetsList/AssetsList.test.tsx | 28 +
.../airflow/ui/src/pages/AssetsList/AssetsList.tsx | 11 +
.../ui/src/pages/Connections/Connections.tsx | 4 +
.../airflow/ui/src/pages/DagRuns/DagRuns.test.tsx | 27 +
.../src/airflow/ui/src/pages/DagRuns/DagRuns.tsx | 6 +
.../ui/src/pages/DagRuns/DagRunsFilters.tsx | 1 +
.../src/pages/DagsList/DagsFilters/DagsFilters.tsx | 3 +-
.../ui/src/pages/DagsList/DagsList.test.tsx | 41 +-
.../src/airflow/ui/src/pages/DagsList/DagsList.tsx | 5 +
.../src/airflow/ui/src/pages/Events/Events.tsx | 4 +
airflow-core/src/airflow/ui/src/pages/Jobs.tsx | 11 +-
.../src/airflow/ui/src/pages/Pools/PoolBarCard.tsx | 12 +-
.../airflow/ui/src/pages/Variables/Variables.tsx | 4 +
.../src/airflow/ui/src/queries/useDags.tsx | 3 +
.../ui/src/utils/useFiltersHandler.test.tsx | 63 ++
.../src/airflow/ui/src/utils/useFiltersHandler.ts | 22 +-
.../src/airflow/ui/tests/e2e/pages/DagsPage.ts | 18 +-
airflow-core/src/airflow/utils/file.py | 10 +
.../api_fastapi/auth/managers/simple/conftest.py | 3 +-
airflow-core/tests/unit/api_fastapi/conftest.py | 27 +-
.../core_api/routes/public/test_auth.py | 28 +-
.../core_api/routes/public/test_backfills.py | 20 +-
.../core_api/routes/public/test_connections.py | 9 +-
.../core_api/routes/public/test_dag_bundles.py | 43 +-
.../core_api/routes/public/test_dag_parsing.py | 18 +-
.../core_api/routes/public/test_dag_run.py | 134 ++--
.../core_api/routes/public/test_task_instances.py | 111 ++-
.../api_fastapi/core_api/routes/ui/test_assets.py | 23 +
.../api_fastapi/core_api/routes/ui/test_dags.py | 43 ++
.../unit/api_fastapi/execution_api/test_app.py | 84 +-
.../versions/head/test_task_instances.py | 5 +-
airflow-core/tests/unit/api_fastapi/test_app.py | 108 ++-
airflow-core/tests/unit/core/test_settings.py | 120 +++
...ke_lang_sdk.py => fake_task_handler_runtime.py} | 0
.../unit/dag_processing/importers/test_registry.py | 99 ---
.../tests/unit/dag_processing/test_collection.py | 86 ++-
.../tests/unit/dag_processing/test_dagbag.py | 263 ++++---
.../unit/dag_processing/test_lang_sdk_runtime.py | 724 ------------------
.../tests/unit/dag_processing/test_manager.py | 25 +
.../tests/unit/dag_processing/test_processor.py | 31 +-
.../dag_processing/test_task_handler_processor.py | 830 ++++++++++++++++++--
.../test_task_handler_processor_go.py | 4 +-
.../tests/unit/executors/test_base_executor.py | 278 ++++++-
.../tests/unit/executors/test_local_executor.py | 10 +-
.../tests/unit/executors/test_workloads.py | 16 +-
airflow-core/tests/unit/jobs/test_scheduler_job.py | 747 ++++++++++++++++--
airflow-core/tests/unit/models/test_dag.py | 31 +
airflow-core/tests/unit/models/test_dagcode.py | 39 +
.../tests/unit/models/test_taskinstance.py | 12 +
airflow-core/tests/unit/state/test_metastore.py | 9 +
.../deps/test_not_previously_skipped_dep.py | 44 ++
.../tests/unit/timetables/test_assets_timetable.py | 227 +++++-
airflow-core/tests/unit/utils/test_session.py | 18 +-
airflow-ctl/src/airflowctl/ctl/cli_config.py | 12 +-
.../tests/airflow_ctl/ctl/test_cli_config.py | 34 +-
.../airflow_e2e_tests/openlineage_tests/README.md | 2 +-
.../dags_extra/e2e_versioned_bundle.py | 4 +-
contributing-docs/30_new_language_sdk.rst | 6 +-
dev/README_RELEASE_PROVIDERS.md | 87 ++-
dev/breeze/doc/ci/04_selective_checks.md | 1 +
dev/breeze/doc/ci/05_workflows.md | 26 +
.../airflow_breeze/commands/testing_commands.py | 2 +-
dev/breeze/src/airflow_breeze/global_constants.py | 2 +-
.../src/airflow_breeze/utils/selective_checks.py | 18 +
dev/breeze/tests/test_selective_checks.py | 56 +-
devel-common/src/tests_common/pytest_plugin.py | 9 +-
devel-common/src/tests_common/test_utils/config.py | 11 +-
.../src/tests_common/test_utils/mock_executor.py | 5 +-
.../src/tests_common/test_utils/providers.py | 2 +-
docs/spelling_wordlist.txt | 2 +
generated/known_sdk_imports_in_core.txt | 9 +-
go-sdk/README.md | 7 +-
go-sdk/airflow/bundle.go | 46 +-
go-sdk/airflow/bundle_test.go | 111 +++
go-sdk/airflow/dag.go | 101 ++-
go-sdk/airflow/dag_test.go | 6 +
go-sdk/airflow/inputs.go | 148 ++++
go-sdk/airflow/inputs_test.go | 392 ++++++++++
go-sdk/airflow/spec.gen.go | 3 +-
go-sdk/airflow/task_handler.go | 12 +-
go-sdk/airflow/task_option.go | 11 +-
go-sdk/airflow/trigger_dag_run.go | 177 +++++
go-sdk/airflow/trigger_dag_run_test.go | 272 +++++++
go-sdk/internal/bundle/task.go | 28 +-
go-sdk/internal/genspec/authoring.go | 2 +-
go-sdk/pkg/binding/binding.go | 56 +-
go-sdk/pkg/binding/binding_test.go | 27 +
go-sdk/pkg/execution/genmodels/models.gen.go | 21 +-
providers/.pre-commit-config.yaml | 7 +
providers/amazon/docs/index.rst | 2 +
providers/amazon/docs/operators/bedrock.rst | 40 +
providers/amazon/pyproject.toml | 4 +
.../airflow/providers/amazon/aws/hooks/bedrock.py | 229 +++++-
.../tests/unit/amazon/aws/hooks/test_bedrock.py | 280 +++++++
.../amazon/aws/test_deferred_hook_configuration.py | 242 +-----
providers/anthropic/pyproject.toml | 2 +-
.../airflow/providers/anthropic/hooks/anthropic.py | 99 ++-
.../tests/unit/anthropic/hooks/test_anthropic.py | 141 +++-
.../unit/celery/executors/test_celery_executor.py | 4 +
.../executors/test_kubernetes_executor.py | 2 +
providers/common/ai/README.rst | 3 +-
providers/common/ai/docs/approval_gates.rst | 2 +-
providers/common/ai/docs/capabilities.rst | 137 ++++
providers/common/ai/docs/changelog.rst | 12 +
providers/common/ai/docs/code_mode.rst | 7 +-
providers/common/ai/docs/features.rst | 6 +-
providers/common/ai/docs/guardrails.rst | 66 --
providers/common/ai/docs/hitl_review.rst | 63 +-
providers/common/ai/docs/hooks/langchain.rst | 20 +-
providers/common/ai/docs/hooks/llamaindex.rst | 12 +
providers/common/ai/docs/index.rst | 5 +-
providers/common/ai/docs/installation.rst | 33 +-
providers/common/ai/docs/observability.rst | 3 +
providers/common/ai/docs/operators/agent.rst | 108 ++-
.../ai/docs/operators/llamaindex_embedding.rst | 26 +-
.../ai/docs/operators/llamaindex_retrieval.rst | 18 +-
providers/common/ai/docs/operators/llm_batch.rst | 4 +-
providers/common/ai/docs/operators/llm_branch.rst | 2 +-
providers/common/ai/docs/quickstart.rst | 3 +-
providers/common/ai/docs/redirects.txt | 1 +
providers/common/ai/docs/sandbox/backends.rst | 11 +-
providers/common/ai/docs/sandbox/configuration.rst | 59 +-
providers/common/ai/docs/sandbox/index.rst | 5 +-
providers/common/ai/docs/self_hosted_models.rst | 3 +-
providers/common/ai/docs/stability.rst | 2 +-
providers/common/ai/docs/tool_approval.rst | 5 +-
providers/common/ai/docs/toolsets/datafusion.rst | 13 +-
providers/common/ai/docs/toolsets/hook.rst | 3 +
providers/common/ai/docs/toolsets/index.rst | 77 +-
.../common/ai/docs/toolsets/managed_agent.rst | 337 ++++----
.../common/ai/docs/toolsets/object_storage.rst | 3 +
providers/common/ai/docs/toolsets/sql.rst | 45 +-
providers/common/ai/docs/troubleshooting.rst | 8 +-
providers/common/ai/pyproject.toml | 7 +-
.../ai/src/airflow/providers/common/ai/__init__.py | 4 +-
.../airflow/providers/common/ai/batch/anthropic.py | 5 +-
.../airflow/providers/common/ai/batch/openai.py | 5 +-
.../airflow/providers/common/ai/batch/results.py | 5 +-
.../providers/common/ai/decorators/agent.py | 2 +-
.../airflow/providers/common/ai/decorators/llm.py | 2 +-
.../providers/common/ai/decorators/llm_batch.py | 2 +-
.../providers/common/ai/decorators/llm_branch.py | 2 +-
.../common/ai/decorators/llm_file_analysis.py | 2 +-
.../common/ai/decorators/llm_schema_compare.py | 2 +-
.../providers/common/ai/decorators/llm_sql.py | 2 +-
.../providers/common/ai/durable/caching_model.py | 4 +-
.../providers/common/ai/durable/caching_toolset.py | 17 +-
.../providers/common/ai/durable/fingerprint.py | 12 +-
.../airflow/providers/common/ai/durable/storage.py | 7 +-
.../common/ai/durable/task_state_store.py | 4 +-
.../ai/example_dags/example_agent_capabilities.py | 27 +-
.../ai/example_dags/example_sandbox_toolset.py | 70 +-
.../src/airflow/providers/common/ai/exceptions.py | 17 +-
.../airflow/providers/common/ai/hooks/langchain.py | 30 +-
.../providers/common/ai/hooks/llamaindex.py | 25 +-
.../providers/common/ai/hooks/pydantic_ai.py | 10 +
.../providers/common/ai/managed_agents/__init__.py | 49 ++
.../providers/common/ai/managed_agents/base.py | 251 ++++++
.../providers/common/ai/managed_agents/failover.py | 145 ++++
.../airflow/providers/common/ai/mixins/approval.py | 6 +-
.../airflow/providers/common/ai/observability.py | 19 +-
.../airflow/providers/common/ai/operators/agent.py | 193 +++--
.../common/ai/operators/llamaindex_embedding.py | 15 +-
.../common/ai/operators/llamaindex_retrieval.py | 16 +-
.../airflow/providers/common/ai/operators/llm.py | 17 +-
.../airflow/providers/common/ai/sandbox/base.py | 167 +++-
.../providers/common/ai/sandbox/opensandbox.py | 71 +-
.../src/airflow/providers/common/ai/sandbox/sbx.py | 90 ++-
.../providers/common/ai/tools/_from_toolset.py | 18 +-
.../providers/common/ai/toolsets/__init__.py | 4 +-
.../providers/common/ai/toolsets/datafusion.py | 45 +-
.../airflow/providers/common/ai/toolsets/hook.py | 11 +-
.../common/ai/toolsets/langchain_bridge.py | 5 +-
.../providers/common/ai/toolsets/managed_agent.py | 365 ++++-----
.../providers/common/ai/toolsets/object_storage.py | 11 +-
.../providers/common/ai/toolsets/sandbox.py | 209 ++++-
.../airflow/providers/common/ai/toolsets/sql.py | 43 +-
.../airflow/providers/common/ai/utils/logging.py | 18 +-
.../providers/common/ai/utils/prompt_cache.py | 140 ++++
.../providers/common/ai/utils/query_results.py | 202 ++++-
.../providers/common/ai/utils/task_logger.py | 67 ++
.../providers/common/ai/utils/toolset_base.py | 19 +
.../providers/common/ai/utils/usage_budget.py | 11 +-
.../ai/tests/unit/common/ai/batch/test_dispatch.py | 2 +-
.../ai/tests/unit/common/ai/batch/test_results.py | 2 +-
.../ai/tests/unit/common/ai/batch/test_state.py | 2 +-
.../common/ai/tests/unit/common/ai/conftest.py | 11 +-
.../tests/unit/common/ai/decorators/test_agent.py | 22 +-
.../unit/common/ai/decorators/test_llm_batch.py | 3 +-
.../common/ai/decorators/test_llm_file_analysis.py | 5 +-
.../ai/tests/unit/common/ai/durable/test_base.py | 2 +-
.../unit/common/ai/durable/test_caching_toolset.py | 55 ++
.../unit/common/ai/durable/test_fingerprint.py | 20 +
.../unit/common/ai/durable/test_replay_cost.py | 2 +-
.../common/ai/durable/test_replay_verification.py | 2 +-
.../tests/unit/common/ai/durable/test_storage.py | 2 +-
.../common/ai/durable/test_task_state_store.py | 2 +-
.../tests/unit/common/ai/hooks/test_langchain.py | 66 ++
.../tests/unit/common/ai/hooks/test_llamaindex.py | 72 ++
.../tests/unit/common/ai/hooks/test_pydantic_ai.py | 15 +
.../unit/common/ai/managed_agents}/__init__.py | 0
.../unit/common/ai/managed_agents/test_base.py | 137 ++++
.../unit/common/ai/managed_agents/test_failover.py | 205 +++++
.../tests/unit/common/ai/mixins/test_approval.py | 6 +-
.../tests/unit/common/ai/operators/test_agent.py | 313 +++++++-
.../common/ai/operators/test_document_loader.py | 12 +-
.../ai/operators/test_llamaindex_embedding.py | 17 +-
.../ai/operators/test_llamaindex_retrieval.py | 19 +-
.../ai/tests/unit/common/ai/operators/test_llm.py | 24 +-
.../unit/common/ai/operators/test_llm_batch.py | 3 +-
.../unit/common/ai/operators/test_llm_branch.py | 4 +-
.../common/ai/operators/test_llm_file_analysis.py | 11 +-
.../tests/unit/common/ai/operators/test_llm_sql.py | 4 +-
.../ai/tests/unit/common/ai/sandbox/test_base.py | 163 +++-
.../unit/common/ai/sandbox/test_opensandbox.py | 137 ++++
.../ai/tests/unit/common/ai/sandbox/test_sbx.py | 106 +++
.../ai/tests/unit/common/ai/test_observability.py | 19 +
.../unit/common/ai/tools/test__from_toolset.py | 33 +-
.../unit/common/ai/toolsets/test_datafusion.py | 55 +-
.../unit/common/ai/toolsets/test_managed_agent.py | 403 ++++------
.../unit/common/ai/toolsets/test_object_storage.py | 8 +-
.../unit/common/ai/toolsets/test_retry_budget.py | 98 +++
.../tests/unit/common/ai/toolsets/test_sandbox.py | 356 +++++++++
.../ai/tests/unit/common/ai/toolsets/test_sql.py | 62 +-
.../ai/tests/unit/common/ai/utils/test_logging.py | 55 +-
.../unit/common/ai/utils/test_prompt_cache.py | 209 +++++
.../unit/common/ai/utils/test_query_results.py | 167 +++-
.../tests/unit/common/ai/utils/test_task_logger.py | 82 ++
.../common/compat/_set_during_execution.py | 40 +
.../src/airflow/providers/common/compat/sdk.py | 14 +-
.../common/compat/security/test_access_view.py | 8 +-
.../common/compat/test__set_during_execution.py} | 22 +-
.../compat/tests/unit/common/compat/test_sdk.py | 18 +-
providers/duckdb/README.rst | 6 +-
providers/duckdb/docs/changelog.rst | 9 +-
providers/duckdb/docs/index.rst | 6 +-
providers/duckdb/provider.yaml | 2 +-
providers/duckdb/pyproject.toml | 6 +-
.../src/airflow/providers/duckdb/__init__.py | 2 +-
providers/fab/pyproject.toml | 2 +-
.../fab/auth_manager/api_fastapi/routes/roles.py | 14 +-
.../fab/auth_manager/api_fastapi/routes/users.py | 12 +-
.../providers/fab/auth_manager/fab_auth_manager.py | 4 +-
.../unit/fab/auth_manager/api_fastapi/conftest.py | 21 +
.../auth_manager/api_fastapi/routes/test_roles.py | 161 ++--
.../auth_manager/api_fastapi/routes/test_users.py | 128 ++--
.../git/src/airflow/providers/git/bundles/git.py | 31 +-
.../git/src/airflow/providers/git/hooks/git.py | 7 +-
providers/git/tests/unit/git/bundles/test_git.py | 44 ++
providers/git/tests/unit/git/hooks/test_git.py | 10 +
providers/google/docs/index.rst | 10 +-
.../docs/operators/cloud/kubernetes_engine.rst | 20 +
.../google/docs/operators/cloud/vertex_ai.rst | 24 +
providers/google/pyproject.toml | 6 +-
.../google/cloud/hooks/vertex_ai/agent_engine.py | 159 +++-
.../google/cloud/operators/kubernetes_engine.py | 114 +++
.../kubernetes_engine/example_kubernetes_engine.py | 105 ++-
.../cloud/hooks/vertex_ai/test_agent_engine.py | 145 ++++
.../cloud/operators/test_kubernetes_engine.py | 146 +++-
.../providers/opensearch/log/os_task_handler.py | 3 +-
.../smtp/src/airflow/providers/smtp/hooks/smtp.py | 2 +-
.../snowflake/hooks/snowflake_cortex_agent.py | 42 +-
.../snowflake/hooks/test_snowflake_cortex_agent.py | 106 ++-
.../airflow/providers/standard/operators/hitl.py | 6 +-
.../tests/unit/standard/operators/test_hitl.py | 4 +-
pyproject.toml | 4 +-
.../ci/lang_sdk_serialization}/__init__.py | 0
scripts/ci/lang_sdk_serialization/compare.py | 233 ++++++
.../ci/lang_sdk_serialization/serialize_python.py | 140 ++++
scripts/ci/lang_sdk_serialization/test_dags.yaml | 166 ++++
scripts/ci/prek/check_conn_id_templated.py | 18 +-
.../ci/prek/check_deferred_hook_configuration.py | 294 +++++++
scripts/ci/prek/check_go_sdk_generated_drift.py | 187 +++++
scripts/ci/prek/check_go_sdk_spec_drift.py | 140 ----
scripts/ci/prek/common_prek_utils.py | 26 +-
scripts/ci/prek/update_providers_dependencies.py | 14 +-
scripts/in_container/run_agent_framework_tests.sh | 90 +++
.../tests/ci/lang_sdk_serialization}/__init__.py | 0
.../ci/lang_sdk_serialization/test_compare.py | 245 ++++++
.../tests/ci/prek/test_check_conn_id_templated.py | 26 +-
.../prek/test_check_deferred_hook_configuration.py | 373 +++++++++
...ift.py => test_check_go_sdk_generated_drift.py} | 83 +-
.../observability/metrics/metrics_template.yaml | 20 +-
task-sdk/docs/api.rst | 2 +
task-sdk/src/airflow/sdk/__init__.py | 3 +
task-sdk/src/airflow/sdk/__init__.pyi | 2 +
task-sdk/src/airflow/sdk/bases/timetable.py | 7 +
.../src/airflow/sdk/coordinators/_subprocess.py | 104 ++-
task-sdk/src/airflow/sdk/definitions/dag.py | 4 +-
.../airflow/sdk/definitions/timetables/assets.py | 25 +
.../airflow/sdk/execution_time/schema/schema.json | 33 +
.../sdk/execution_time/schema/versions/__init__.py | 8 +-
.../execution_time/schema/versions/v2026_10_30.py | 12 +
.../src/airflow/sdk/execution_time/task_runner.py | 12 +
task-sdk/src/airflow/sdk/importers/base.py | 2 +
task-sdk/src/airflow/sdk/serde/__init__.py | 7 +-
.../tests/task_sdk/coordinators/test_subprocess.py | 12 +-
task-sdk/tests/task_sdk/definitions/test_dag.py | 18 +-
.../task_sdk/execution_time/test_task_runner.py | 64 ++
ts-sdk/.pre-commit-config.yaml | 14 +
ts-sdk/package.json | 3 +-
ts-sdk/pnpm-lock.yaml | 27 +-
..._lint.py => check_serialization_conformance.py} | 16 +-
ts-sdk/src/coordinator/serde.ts | 751 ++++++++++++++++++
ts-sdk/src/generated/supervisor.ts | 18 +
ts-sdk/src/sdk/dag.ts | 25 +-
ts-sdk/tests/conformance/serialize_typescript.ts | 163 ++++
ts-sdk/tests/coordinator/arg-binding.test.ts | 19 +
ts-sdk/tests/coordinator/serde.test.ts | 846 +++++++++++++++++++++
ts-sdk/tests/public-api.test.ts | 4 +
uv.lock | 20 +-
406 files changed, 18805 insertions(+), 5080 deletions(-)
copy .github/workflows/{additional-ci-image-checks.yml =>
agent-framework-tests.yml} (55%)
create mode 100644 airflow-core/newsfragments/58543.feature.rst
delete mode 100644
airflow-core/src/airflow/dag_processing/importers/__init__.py
delete mode 100644 airflow-core/src/airflow/dag_processing/importers/base.py
delete mode 100644
airflow-core/src/airflow/dag_processing/importers/python_importer.py
delete mode 100644 airflow-core/src/airflow/dag_processing/lang_sdk_runtime.py
copy
airflow-core/src/airflow/migrations/versions/{0111_3_3_0_add_timetable_periodic_to_dag.py
=> 0141_3_4_0_add_timetable_asset_gated_to_dagmodel.py} (63%)
rename
airflow-core/src/airflow/migrations/versions/{0141_3_4_0_add_lang_sdk_task_handler_tables.py
=> 0142_3_4_0_add_lang_sdk_task_handler_tables.py} (98%)
create mode 100644
airflow-core/src/airflow/ui/src/components/FilterBar/filters/RunStateFilter.tsx
create mode 100644
airflow-core/src/airflow/ui/src/components/FilterBar/filters/runStateParams.test.ts
create mode 100644
airflow-core/src/airflow/ui/src/components/FilterBar/filters/runStateParams.ts
rename airflow-core/tests/unit/dag_processing/{fake_lang_sdk.py =>
fake_task_handler_runtime.py} (100%)
delete mode 100644
airflow-core/tests/unit/dag_processing/importers/test_registry.py
delete mode 100644
airflow-core/tests/unit/dag_processing/test_lang_sdk_runtime.py
create mode 100644 go-sdk/airflow/inputs.go
create mode 100644 go-sdk/airflow/inputs_test.go
create mode 100644 go-sdk/airflow/trigger_dag_run.go
create mode 100644 go-sdk/airflow/trigger_dag_run_test.go
create mode 100644 providers/common/ai/docs/capabilities.rst
delete mode 100644 providers/common/ai/docs/guardrails.rst
create mode 100644
providers/common/ai/src/airflow/providers/common/ai/managed_agents/__init__.py
create mode 100644
providers/common/ai/src/airflow/providers/common/ai/managed_agents/base.py
create mode 100644
providers/common/ai/src/airflow/providers/common/ai/managed_agents/failover.py
create mode 100644
providers/common/ai/src/airflow/providers/common/ai/utils/prompt_cache.py
create mode 100644
providers/common/ai/src/airflow/providers/common/ai/utils/task_logger.py
rename {airflow-core/tests/unit/dag_processing/importers =>
providers/common/ai/tests/unit/common/ai/managed_agents}/__init__.py (100%)
create mode 100644
providers/common/ai/tests/unit/common/ai/managed_agents/test_base.py
create mode 100644
providers/common/ai/tests/unit/common/ai/managed_agents/test_failover.py
create mode 100644
providers/common/ai/tests/unit/common/ai/toolsets/test_retry_budget.py
create mode 100644
providers/common/ai/tests/unit/common/ai/utils/test_prompt_cache.py
create mode 100644
providers/common/ai/tests/unit/common/ai/utils/test_task_logger.py
create mode 100644
providers/common/compat/src/airflow/providers/common/compat/_set_during_execution.py
copy providers/{amazon/tests/unit/amazon/aws/cli/test_definition.py =>
common/compat/tests/unit/common/compat/test__set_during_execution.py} (52%)
copy {airflow-core/src/airflow/_shared =>
scripts/ci/lang_sdk_serialization}/__init__.py (100%)
create mode 100644 scripts/ci/lang_sdk_serialization/compare.py
create mode 100644 scripts/ci/lang_sdk_serialization/serialize_python.py
create mode 100644 scripts/ci/lang_sdk_serialization/test_dags.yaml
create mode 100755 scripts/ci/prek/check_deferred_hook_configuration.py
create mode 100755 scripts/ci/prek/check_go_sdk_generated_drift.py
delete mode 100755 scripts/ci/prek/check_go_sdk_spec_drift.py
create mode 100755 scripts/in_container/run_agent_framework_tests.sh
copy {airflow-core/src/airflow/_shared =>
scripts/tests/ci/lang_sdk_serialization}/__init__.py (100%)
create mode 100644 scripts/tests/ci/lang_sdk_serialization/test_compare.py
create mode 100644
scripts/tests/ci/prek/test_check_deferred_hook_configuration.py
rename scripts/tests/ci/prek/{test_check_go_sdk_spec_drift.py =>
test_check_go_sdk_generated_drift.py} (52%)
copy ts-sdk/scripts/ci/prek/{ts_sdk_lint.py =>
check_serialization_conformance.py} (67%)
create mode 100644 ts-sdk/src/coordinator/serde.ts
create mode 100644 ts-sdk/tests/conformance/serialize_typescript.ts
create mode 100644 ts-sdk/tests/coordinator/serde.test.ts
