This is an automated email from the ASF dual-hosted git repository.
jason810496 pushed a change to branch
jason/core-taskhandler-refactor/06-go-sdk-task-handler-parse
in repository https://gitbox.apache.org/repos/asf/airflow.git
omit ccecc8115d5 Describe the two task handler bindings for Language SDK
authors
omit 3d9eaf255fe Go SDK: declare every lone struct handler as named
omit 5155067e2c7 State that a task handler parse answer depends only on the
artifact
omit f479c8aef71 Go SDK: answer a task handler parse with every handler
omit a9d24bac054 Tidy names, mocks and cases in the Go probe and digest
tests
omit f731a54bab2 Move Go SDK comments back to the code they describe
omit 262ca91a98a Resolve the probed bundle path and tighten the digest docs
omit 9ecf47d459c Go SDK: take the bundle digests from the staged executable
omit 6cf833f9f89 Document the task handler parse for Language SDK authors
omit eb196107f46 Read a Go bundle's stored cache digest without hashing it
omit c440bf74d68 Let the Executable coordinator probe a bundle's task
handlers
omit f3d803492c5 Go SDK: record integrity and cache digests in bundle
metadata
omit fc06e0786b7 Go SDK: clarify parse timeout and nested-null docs, test
anyOf
omit 4619601b5de Go SDK: answer TaskHandlerParseRequest with registered
handlers
omit aeff5da3dc4 Go SDK: describe task handler params with JSON Schema
omit b45c6ec8df6 Clean up a Lang-SDK runtime parse whose start fails after
the fork
omit 1b9321728ac Do not exec a Lang-SDK runtime whose parent has already
exited
omit 896cc6faf9b Kill what a Lang-SDK runtime leaves behind when it exits
omit bf185776a42 Read a Lang-SDK runtime's frames without blocking the
parse loop
omit 4cb786b1d35 Probe a Lang-SDK artifact for its task handlers
omit 1ac623cb01a Let a subprocess coordinator exec a task-handler parse
runtime
omit 32db985ad1d Keep a nested supervised child's standard streams open
omit 07b5199cce6 Describe the two task handler binding modes in the ADRs
omit 1d793f9974d Regenerate Go SDK models for task-handler messages
omit a4598c4102a Regenerate TS SDK supervisor types for the two binding
modes
omit 1bd56509d09 Bind task handlers only by position or by name
omit c9eaf687a1e Describe the all-handlers probe in ADR-0011 and ADR-0012
omit d86cead24b2 Regenerate TS SDK supervisor types for the probe without
dag_ids
omit d33f9041f22 Ask a task handler runtime for every handler it registers
omit 270011518de Add the named_open binding to the Lang-SDK ADRs
omit 12b142d0ab9 Regenerate TS SDK supervisor types for the named_open
binding
omit 06386fb2291 Add a named_open binding for non-exhaustive task-handler
params
omit de96d42d536 Describe the binding-mode parse check in ADR-0011 and
ADR-0013
omit 7160a104c3d Describe the task-handler binding mode in ADR-0012
omit f3601d6e8a0 Regenerate TS SDK supervisor types for the binding mode
omit a2b0c2936d4 Add a binding mode to task-handler declarations
omit 2826ec45c7f Document the version rules for a new supervisor schema body
omit 7e0327ce1e5 Regenerate TS SDK supervisor types for task-handler
messages
omit 4f4b3bdc559 Add task-handler parse messages and their parse-channel
union
omit 7e26f0d11be Test closing a Dag file processor that has no parse log
file
omit db4f8470d08 Move the Dag file processor's shared plumbing into a base
class
omit da3372e04cd Allow a NULL cache digest in ADR-0013's artifact table SQL
omit 2c96b482aa6 Let a task handler artifact store no cache digest
omit 03613039ea9 Move the handler declarations to the artifact in
ADR-0013's SQL
omit 7b572ab02d5 Cache an artifact's task handlers on its row, not on each
binding
omit d18d0afe556 Make session keyword-only in the task handler model test
helpers
omit fce1160d9b8 Add handler_binding to ADR-0013's task handler table SQL
omit 61b2d76b38e Store the task handler binding mode beside its params
omit 396cac2600d Regenerate the task handler migration revision id
omit 5da25f7206e Describe LangSDKTaskHandler as a stub task binding
omit ab8f0257223 Test that a referenced task handler artifact cannot be
deleted
omit dc15082f135 Keep the artifact path hash in sync on ORM updates
omit 6762729ecfc Match ADR-0013 task handler table SQL to the migration
omit c174aa41637 Add lang-SDK task handler binding tables
omit 68f03d615b8 Mark ADR-0013 as accepted
omit 9d00ead095f Add newsfragment for the removed coordinator artifact roots
omit c7055408b06 Validate task_handler_bundle_name only for routed
coordinators
omit 8983f6dc234 Say the Dag processor needs the language SDK setup too
omit 8997a357c46 Drop em-dashes from the rewritten doc sentences
omit 23898f0586c Say artifact Dag bundles are registered on every component
omit b9cde7fe612 Tidy names and mocks in the artifact bundle tests
omit 4bfc18153e8 Document artifact Dag bundles for language SDK coordinators
omit ace4b2eeaa9 Serve lang-SDK harness artifacts from Dag bundles
omit bd9be97f70e Replace coordinator artifact roots with
task_handler_bundle_name
add 64f9f3ac784 Update issue templates now that Airflow 2 has reached end
of life (#74056)
add 6330b4de9c5 Go SDK: reject a second TaskSpec or Inputs when dag.Task
applies it (#74057)
add 383a9a87a98 Stop wiping the uv cache before every CI image build
(#74052)
add 58e987defec Fix Non-DB test collection failure in Google Stackdriver
hook test (#74066)
add 498724fc13a Go SDK: enumerate every user-facing enum in the airflow
package (#74064)
add 5e476d7cf89 TS SDK: answer the Dag-parsing request from bundle.serve
(#73442)
add dd271599939 Go SDK: vendor the schemas it generates from (#74047)
add 77927390c17 Move Dag cycle detection from the Task SDK to shared
(#74034)
add a3e07159a0b Add Dag ID filtering to Assets search (#70971)
add d3b9a0dd2b3 Add KafkaSharedStreamProducer and KafkaSharedStreamTrigger
(#68625)
add ae6139d58a8 Fix DecreasingPriorityStrategy example by initializing
try_number before weight evaluation (#62148)
add 402d7885194 Java SDK: Resolve a task's arguments from the Dag's own
wiring (#73596)
add e14854221ad Group and filter Dags by folder on the Dags page (#68544)
add 18af65032e3 Render the Language SDK Spec diagrams as Mermaid (#74065)
add be53013349e Support azure-datalake-store 1.x in AzureDataLakeHook
(#73819)
add bc15e1093b5 Close German Translation Gaps 2026-10-02 (#74088)
add 3045451b8b0 Treat an empty Dag run config as no config in the UI
(#74083)
add c31888641b1 Persist the match-anywhere search toggle in the URL
(#74019)
add 4803c0b3f79 Discover Dag definitions through the Task SDK importers
(#74020)
add f5283f3da9b Move the Dag file processor's shared plumbing into a base
class (#74040)
add e3dd28607b9 Document team-scoped plugins in multi-team mode (#73995)
add b62299ced8a Cap microsoft-kiota-http below 1.13 in the Microsoft Azure
provider (#74090)
add d7b672f44e0 Update docs and diagrams for the Airflow 3 Dag processor
(#74073)
add ec5b621a9cc Prepare ad-hoc providers release 2026-10-02 (#74097)
add 7800fc44c07 Fix scheduler crash on restart with queued or running Edge
tasks (#73832)
add 49a52987956 Stop version-check hooks from clobbering the committing
worktree's index (#74115)
add d7b88f36197 UI: Fix external plugin nav links not navigating on click
(#73913)
add 66d751bfd22 Skip already-released providers in
update-providers-next-version (#74098)
add 04194b1de1f Fix flaky Kafka trigger tests by awaiting the event
instead of a fixed sleep (#74124)
add 4c0ba9e41da Make Edge executor adoption test immune to executor_loader
reloads (#74122)
add 7a8ce83f385 Go SDK: reject a task function whose last result type is
not error (#74077)
add 9c14681a4a0 Add create and update support to Snowflake Cortex Agent
hook (#70703)
add e029fcef2bb Bump the registry-security-updates group across 1
directory with 3 updates (#74099)
add da28f1fcac4 Bump the auth-ui-package-updates group across 1 directory
with 5 updates (#74101)
add 971e8db54ea Bump the edge-ui-package-updates group across 1 directory
with 12 updates (#74107)
add 536f535d9dd Bump the core-ui-package-updates group across 1 directory
with 12 updates (#74105)
add 95826e38547 Bump the uv-dependency-updates group in /dev/breeze with 3
updates (#74100)
add df76e2e0d5e Bump the ui-plugin-template-package-updates group across 1
directory with 7 updates (#74102)
add 954d21df92d Bump vitest from 4.1.7 to 5.0.1 in /ts-sdk (#73483)
add 0adf1378017 Bump swagger-ui-dist from 5.32.15 to 5.33.0 in /registry
(#73480)
add e37a39aa34e Bump pyjwt from 2.14.0 to 2.15.0 in /dev/breeze (#73969)
add da53196d812 Time-box and retry pre-extras SDK downloads (#71544)
add 232c264968f Deprecate Dataproc ClusterGenerator helper class (#71427)
add b9c52951378 Create GCS folder markers as directories in
GCSToSFTPOperator (#73880)
add 91353550246 Honor download_num_attempts and upload_num_attempts in
GCSTimeSpanFileTransformOperator (#72953)
add 02f2c852c80 Shrink the CI image by skipping unused Rust docs and the
Go archive (#74076)
add 5523443d98c Set end date and duration on tasks skipped by a Dag run
timeout (#74128)
add 0f3a356e768 Add a user setting for the default search match mode
(#74021)
add 12ee0862774 Carry task attempt UUIDs through remote executor providers
(#73917)
add 867327c48dd Run only the tests that import a changed test helper
(#74055)
add da5af33fd02 Create the Athena Spark work group and results bucket in
the system test (#73929)
add c011b5960ac Install ibmmq from a wheel in IBM MQ lowest-dependency
tests (#74132)
add 855a2ce92f9 Isolate Edge job state by task attempt UUID (#73918)
add 79de1d48039 Fix `refresh_from_task` not applied on retries (#65932)
add bf558f6f29a Go SDK: add dag.If with Then and Else so native Dags can
branch (#74075)
add d21496f7581 Fetch existing task instances in instead of rebuilding
them in tests (#74146)
add 11d25a587ac Run leaking provider tests with providers changed after
them in CI (#74123)
add a89208e5405 Bump the github-actions-updates group across 1 directory
with 5 updates (#74110)
add 4ae133cd807 Bump @chakra-ui/react from 3.36.1 to 3.37.0 in
/airflow-core/src/airflow/ui (#72524)
add 484a87b54bc Add get_async_extra_dejson to the common.compat connection
helpers (#74147)
add e4736b46968 Pass ClientTimeout to aiohttp in Cloud Composer async hook
(#74164)
add f070997ba63 Go SDK: order tasks with Before, After and Label (#74119)
add 803cc82370d Update supported Kubernetes versions (#73798)
add 4a73cb7d5fa Keep builtin annotations from resolving to project
attributes in Sphinx 9 (#74168)
add 79bc6439a9c Always clear the session cookie on logout (#73788)
add a1b02e26f3d Support Python patch exclusions in provider metadata
(#73598)
add bdfc5555305 Add configurable request timeout for the Airbyte API
(#71092)
add b2c215cc037 Fix Beam async hook launching pipelines through a shell
(#72510)
add d2883eead5c Fix Cloud SQL 409 operationInProgress on import/export
operations (#68361)
add c688a6f0abc Add breeze verify to list local checks for changed files
(#73241)
add 07c9ceba527 Fail provider DB test jobs when their tests fail (#74172)
add 043f97c2b00 Test get_async_extra_dejson against the real supervisor
comms (#74166)
add 592cf3ffdd2 Use the current user in the HITL detail tests instead of
hardcoding (#74184)
add 8331bb485c5 Read the msgraph connection extra with
get_async_extra_dejson (#74161)
add b156582855c Fix OpenSearch provider log pagination (#73947)
add c8b136358ce Allow OpensearchTaskHandler and OpensearchRemoteLogIO to
take empty username and password (#71692)
add cc94831e213 Fix docstring cross-references that Sphinx 9 cannot
resolve (#74169)
add b58cfa912a9 Add an API endpoint and UI tab comparing two stored Dag
versions (#73322)
add d1d5b1af26e Adapt GCP BigtableHook to run in private cloud (#73181)
add b3674d43be0 `SparkSubmitHook`: Mask `_mask_cmd` secrets in linear time
(#71711)
add 8d5814c09d5 Bump the react group across 1 directory with 4 updates
(#73334)
add 7e448f70af5 Use async database access for Variable key listing (#73407)
add 523567e47d0 Add tests for Google Go module subprocess utility (#73268)
add 4a766db2b03 Bump the typescript group across 1 directory with 4
updates (#72994)
add 7fad35d9c99 Add deadline alert coverage to Dag endpoint tests (#72296)
add 6f5d5d06b17 Align standard provider tests with source paths (#73267)
add c2f600f6ec6 Add airflowctl tasks state command (#71206)
add 86b8a521837 Bind the AWS auth manager SAML response to the browser
that started the login (#73698)
add cefc0c3239d Bump the java-sdk-dependency-updates group across 1
directory with 3 updates (#74108)
add 2a274ab3c93 Refuse nested secret ids looked up with no team in
multi-team mode (#73789)
add 8055f6a8717 Skip creating an empty hook lineage collector for
OpenLineage events (#74190)
add 38fefc1ca95 Fix invalid cron expression in OpenLineage system test Dag
(#74187)
add 27f827fcdce Honor the polling interval for deferred Kubernetes jobs
(#74143)
add 3bd4df6b2da Fix Snowflake SQL API wait_for_query timing out on
finished queries (#73761)
add f74a685978c Use configured default queue for Celery executor callbacks
(#73552)
add ad3bb1f676c Replace coordinator artifact roots with
task_handler_bundle_name (#73970)
add 1c3f78e1d57 Validate a serialized Dag and fill its unset settings from
the Airflow config (#74041)
add a44fd6d1f31 Allow explicit runs to drain a paused Dag (#73763)
add 40832901c59 Improve LocalExecutor bookkeeping: correctly add tasks to
the `running` list (#74142)
add 5487c080bea Add Azure Blob Storage support to the DataFusion object
storage layer (#73374)
add b191cc4767b Fail instead of silently falling behind when a Lang-SDK
vendored Dag schema drifts from airflow-core (#74091)
add e0202f40c78 Let a Dag importer find the coordinator that parses its
files (#74042)
add 94a677e8f87 Keep callback triggers assigned to healthy triggerers
(#72848)
add 4e1f2aca67f Keep the Dag trigger button one-click when a run has
config (#74039)
add bfe84598a52 Fix destination path validation in GCS-to-SFTP transfers
(#74186)
add 8239997bf20 Scope UI next_run_assets endpoint to assets the caller may
read (#72862)
add 7df6f0e69cc Parse native Lang-SDK Dags in the Dag processor with their
runtime (#74035)
add cffc14d173b Parse native Lang-SDK Dags in a Dag bag (#74043)
add 9c6cd0e3396 Read JAR manifests per the JAR specification
add 1f361c2bcbc Build the JVM command that parses a Dag JAR
add c1a0bd225e1 Parse native Java Dags with their coordinator
add f6b87a7cae4 Treat an unreadable JAR like one without a manifest
add bb23794a6af Test the Java Dag source placeholder for a JAR without a
manifest
add 1e85316e871 Name the JAR manifest attributes the Java coordinator reads
add a28bdee1c03 Document native Java Dag parsing
add 45761df8bb7 Run a task on its run's version when dag_bundle_name names
its bundle
add 801e5a1f843 Pass the task's Dag file to the coordinator's command
builder
add 4c9eaf9b573 Report why a Java Dag JAR cannot be read
add e6933d81c5b Run the JAR a native Java Dag was parsed from
add 1e5f033d9fc Reject a Java Dag JAR whose SDK cannot answer a parse
add cb7ff0320fb Stop promising that Java dependency JARs are not parsed
add 2f3c1b7271c Keep the Java parse_dag test from changing the pytest
worker
add f950fb8b502 Hand out the Java Dag importer through get_dag_importer
add 3cd507c37b9 Read Java Dag sources from the bundle's sources.json index
add e63ffc3be9e Let the Java Dag importer find its coordinator
add d1ffa7d818d Fail a task cleanly when its runtime cannot be launched
add 03ac85138ab Run a native Dag's task from its own Dag file
add b5f144dc0ad Fail a native task cleanly when its file or coordinator
class cannot load
add 7a243c19d33 Say tasks list runs a native Java Dag's main class
add 3b6a9d0d48b Say a JRE parses native Java Dags, and keep main's bundle
test mocks
add 12307d82ea3 Parse native TypeScript Dags with the Node coordinator
add d1821478b82 Document parsing native TypeScript Dags
add 761ce1ab1af Share the bundle section mutation helper between Node tests
add db26372198b Run a native TypeScript Dag's task from its own bundle
add 7be05790dd6 Say when the Dag processor needs Node.js for TypeScript
bundles
add 2485d7c38c3 Align the TypeScript build docs with the Code view
add 58cf8d1a3e6 Warn against defining a Dag in both TypeScript and Python
add 861ffd6e9ef Import NodeDagImporter at the top of the Node coordinator
add fce03d56bcc Let NodeDagImporter source read errors reach the caller
add aac11e6a559 Pin the Code view to the entry module in Node bundle tests
add 881795f9d20 Hand out the Node Dag importer through get_dag_importer
add 158894e8564 Let the Node Dag importer find its coordinator
add 2b606617e9a Run a native TypeScript Dag's own bundle through the
native task branch
add 02f1e5169c0 Document how a NodeCoordinator is picked for parsing
TypeScript Dags
add 1e09e5fdfbe Say tasks list runs a native TypeScript Dag's bundle
add 6817c69979e Add task-handler parse messages and their parse-channel
union
add 8e37efefb71 Regenerate TS SDK supervisor types for task-handler
messages
add f30a00a88f5 Regenerate Go SDK models for task-handler messages
add 68d0a434bc5 Document the version rules for a new supervisor schema body
add 802d8f05333 Describe the task handler messages and binding modes in
ADR-0011 and ADR-0012
add a6ba6e22279 Use the task handler message names and shapes in ADR-0013
add c845f9b401c Keep a nested supervised child's standard streams open
add ae727d1490a Let a subprocess coordinator exec a task-handler parse
runtime
add 4ab1318d387 Do not exec a Lang-SDK runtime whose parent has already
exited
add 8eaf1e5c98a Let a subprocess coordinator find the artifact a stub task
runs
add a86bda071e7 Start a task handler parse only on a runtime that can
answer it
add 3cd63f1f4e1 Probe a Lang-SDK artifact for its task handlers
add 29b8267afc4 Read a Lang-SDK runtime's frames without blocking the
parse loop
add d564c46209f Kill what a probed runtime leaves in its process group
add 26e49c53b0c Clean up a Lang-SDK runtime parse whose start fails after
the fork
add c154387d563 Stop a task handler probe at its caller's deadline
add 4a128770b4d Name the task handler probe and its coordinator hooks in
ADR-0012 and ADR-0013
add 2a35a43d713 Go SDK: describe task handler params with JSON Schema
add f2cd42ff406 Go SDK: answer TaskHandlerParseRequest with every task
handler
add 9f81daacb49 Scan for executable bundles in sorted path order
add 02100d8c2f4 Let the Executable coordinator find a stub task's bundle
and probe it
add 23782e5265c Probe a packed Go bundle for its task handlers in a gated
test
add cf557b954a8 Document the task handler parse for Language SDK authors
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (ccecc8115d5)
\
N -- N -- N
refs/heads/jason/core-taskhandler-refactor/06-go-sdk-task-handler-parse
(cf557b954a8)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.agents/skills/airflow-publish-changes/SKILL.md | 2 +-
.../prepare-providers-documentation/SKILL.md | 6 +-
.github/ISSUE_TEMPLATE/1-airflow_bug_report.yml | 7 +-
.github/ISSUE_TEMPLATE/2-feature_request.yml | 3 +-
.github/workflows/ci-notification.yml | 2 +-
.github/workflows/codeql-analysis.yml | 6 +-
.github/workflows/java-sdk-dependency-security.yml | 6 +-
.github/workflows/notify-uv-lock-conflicts.yml | 2 +-
.github/workflows/registry-tests.yml | 2 +-
.../scheduled-verify-release-calendar.yml | 2 +-
.pre-commit-config.yaml | 43 +-
AGENTS.md | 1 +
Dockerfile | 5 +-
Dockerfile.ci | 7 +-
README.md | 16 +-
airflow-core/adr/lang-sdk/0004-dag-parsing.md | 12 +-
.../adr/lang-sdk/0010-native-dag-processing.md | 118 +-
.../lang-sdk/0011-mixed-language-dag-processing.md | 31 +-
.../adr/lang-sdk/0012-lang-sdk-parse-protocol.md | 123 +-
.../0013-persisted-task-handler-bindings.md | 77 +-
.../cluster-policies.rst | 11 +-
.../modules_management.rst | 2 +-
.../docs/administration-and-deployment/plugins.rst | 172 +-
.../authoring-and-scheduling/language-sdks/go.rst | 5 +-
.../language-sdks/java.rst | 111 +-
.../language-sdks/typescript.rst | 104 +-
airflow-core/docs/best-practices.rst | 10 +-
airflow-core/docs/core-concepts/backfill.rst | 7 +-
airflow-core/docs/core-concepts/dags.rst | 19 +-
airflow-core/docs/core-concepts/multi-team.rst | 16 +-
airflow-core/docs/core-concepts/overview.rst | 4 +-
.../img/diagram_basic_airflow_architecture.md5sum | 2 +-
.../img/diagram_basic_airflow_architecture.png | Bin 116194 -> 127853
bytes
.../docs/img/diagram_basic_airflow_architecture.py | 9 +-
...agram_dag_processor_airflow_architecture.md5sum | 2 +-
.../diagram_dag_processor_airflow_architecture.png | Bin 237657 -> 237572
bytes
.../diagram_dag_processor_airflow_architecture.py | 23 +-
...diagram_distributed_airflow_architecture.md5sum | 2 +-
.../diagram_distributed_airflow_architecture.png | Bin 192497 -> 215541
bytes
.../diagram_distributed_airflow_architecture.py | 20 +-
airflow-core/docs/installation/prerequisites.rst | 2 +-
airflow-core/docs/migrations-ref.rst | 5 +-
airflow-core/docs/security/api_permissions_ref.rst | 4 +
airflow-core/docs/security/security_model.rst | 9 +
airflow-core/newsfragments/72407.significant.rst | 10 +-
.../auth/managers/simple/ui/package.json | 10 +-
.../auth/managers/simple/ui/pnpm-lock.yaml | 253 +--
.../src/airflow/api_fastapi/auth/tokens.py | 24 +-
.../api_fastapi/common/parameters/__init__.py | 1 +
.../airflow/api_fastapi/common/parameters/dag.py | 41 +
.../api_fastapi/core_api/datamodels/backfills.py | 8 +
.../api_fastapi/core_api/datamodels/dag_run.py | 7 +
.../core_api/datamodels/dag_versions.py | 158 ++
.../api_fastapi/core_api/datamodels/ui/dags.py | 14 +
.../api_fastapi/core_api/openapi/_private_ui.yaml | 90 +
.../core_api/openapi/v2-rest-api-generated.yaml | 301 +++
.../api_fastapi/core_api/routes/public/assets.py | 10 +
.../api_fastapi/core_api/routes/public/auth.py | 9 +-
.../core_api/routes/public/backfills.py | 17 +-
.../api_fastapi/core_api/routes/public/dag_run.py | 11 +
.../core_api/routes/public/dag_versions.py | 98 +-
.../api_fastapi/core_api/routes/ui/assets.py | 1 +
.../airflow/api_fastapi/core_api/routes/ui/dags.py | 51 +
.../api_fastapi/execution_api/routes/variables.py | 10 +-
.../src/airflow/cli/commands/dag_command.py | 9 +-
.../src/airflow/cli/commands/task_command.py | 5 +-
.../src/airflow/config_templates/config.yml | 14 +
airflow-core/src/airflow/dag_processing/dagbag.py | 167 +-
.../src/airflow/dag_processing/importer_routing.py | 52 +
..._handler_processor.py => lang_sdk_processor.py} | 452 ++--
airflow-core/src/airflow/dag_processing/manager.py | 179 +-
.../src/airflow/dag_processing/processor.py | 82 +-
.../dag_processing/task_handler_processor.py | 245 +--
.../src/airflow/executors/local_executor.py | 75 +-
.../src/airflow/jobs/scheduler_job_runner.py | 3 +-
.../0142_3_4_0_add_lang_sdk_task_handler_tables.py | 91 -
airflow-core/src/airflow/models/__init__.py | 1 -
airflow-core/src/airflow/models/backfill.py | 3 +
airflow-core/src/airflow/models/dag.py | 16 +
airflow-core/src/airflow/models/dagrun.py | 4 +
.../src/airflow/models/lang_sdk_task_handler.py | 128 --
airflow-core/src/airflow/models/taskinstance.py | 2 +-
airflow-core/src/airflow/models/trigger.py | 1 +
airflow-core/src/airflow/provider.yaml.schema.json | 5 +-
.../src/airflow/serialization/dag_version_diff.py | 3 +-
.../src/airflow/serialization/definitions/dag.py | 2 +
airflow-core/src/airflow/serialization/schema.json | 13 +-
.../airflow/serialization/serialized_objects.py | 96 +
.../src/airflow/ui/openapi-gen/queries/common.ts | 27 +-
.../ui/openapi-gen/queries/ensureQueryData.ts | 63 +-
.../src/airflow/ui/openapi-gen/queries/prefetch.ts | 63 +-
.../src/airflow/ui/openapi-gen/queries/queries.ts | 63 +-
.../src/airflow/ui/openapi-gen/queries/suspense.ts | 63 +-
.../airflow/ui/openapi-gen/requests/schemas.gen.ts | 243 +++
.../ui/openapi-gen/requests/services.gen.ts | 99 +-
.../airflow/ui/openapi-gen/requests/types.gen.ts | 187 ++
airflow-core/src/airflow/ui/package.json | 42 +-
airflow-core/src/airflow/ui/pnpm-lock.yaml | 2258 ++++++++++----------
.../ui/public/i18n/locales/ar/components.json | 4 +-
.../ui/public/i18n/locales/ca/components.json | 4 +-
.../airflow/ui/public/i18n/locales/de/assets.json | 1 +
.../airflow/ui/public/i18n/locales/de/browse.json | 2 +-
.../airflow/ui/public/i18n/locales/de/common.json | 15 +-
.../ui/public/i18n/locales/de/components.json | 6 +-
.../airflow/ui/public/i18n/locales/de/dags.json | 6 +-
.../ui/public/i18n/locales/de/dashboard.json | 8 +-
.../ui/public/i18n/locales/el/components.json | 4 +-
.../airflow/ui/public/i18n/locales/en/common.json | 7 +
.../ui/public/i18n/locales/en/components.json | 16 +-
.../src/airflow/ui/public/i18n/locales/en/dag.json | 63 +
.../airflow/ui/public/i18n/locales/en/dags.json | 7 +
.../ui/public/i18n/locales/es/components.json | 4 +-
.../ui/public/i18n/locales/fr/components.json | 4 +-
.../ui/public/i18n/locales/he/components.json | 4 +-
.../ui/public/i18n/locales/hi/components.json | 4 +-
.../ui/public/i18n/locales/hu/components.json | 4 +-
.../ui/public/i18n/locales/it/components.json | 4 +-
.../ui/public/i18n/locales/ja/components.json | 4 +-
.../ui/public/i18n/locales/ko/components.json | 4 +-
.../ui/public/i18n/locales/nl/components.json | 4 +-
.../ui/public/i18n/locales/pl/components.json | 4 +-
.../ui/public/i18n/locales/pt/components.json | 4 +-
.../ui/public/i18n/locales/ru/components.json | 4 +-
.../ui/public/i18n/locales/th/components.json | 4 +-
.../ui/public/i18n/locales/tr/components.json | 4 +-
.../ui/public/i18n/locales/zh-CN/components.json | 4 +-
.../ui/public/i18n/locales/zh-TW/components.json | 4 +-
.../components/DagActions/RunBackfillForm.test.tsx | 96 +-
.../src/components/DagActions/RunBackfillForm.tsx | 35 +-
.../ui/src/components/FilterBar/FilterBar.test.tsx | 21 +
.../ui/src/components/FilterBar/FilterBar.tsx | 31 +-
.../TriggerDag/PausedDagOptions.test.tsx | 250 +++
.../src/components/TriggerDag/PausedDagOptions.tsx | 123 ++
.../TriggerDag/TriggerDAGButton.test.tsx | 190 ++
.../src/components/TriggerDag/TriggerDAGButton.tsx | 172 +-
.../components/TriggerDag/TriggerDAGForm.test.tsx | 78 +-
.../src/components/TriggerDag/TriggerDAGForm.tsx | 32 +-
.../components/TriggerDag/TriggerDAGModal.test.tsx | 238 +++
.../src/components/TriggerDag/TriggerDAGModal.tsx | 21 +-
.../airflow/ui/src/components/TriggerDag/types.ts | 3 +
.../src/components/VersionCompareSelect.test.tsx | 82 +
.../Code => components}/VersionCompareSelect.tsx | 4 +-
.../src/airflow/ui/src/constants/localStorage.ts | 2 +
.../src/airflow/ui/src/constants/searchParams.ts | 6 +
airflow-core/src/airflow/ui/src/constants/tab.ts | 1 +
.../ui/src/hooks/useAdvancedSearch.test.tsx | 175 ++
.../src/airflow/ui/src/hooks/useAdvancedSearch.ts | 39 +-
.../airflow/ui/src/hooks/useUserSettings.test.tsx | 21 +
.../src/airflow/ui/src/hooks/useUserSettings.ts | 7 +
.../ui/src/layouts/Details/DetailsLayout.tsx | 1 -
.../airflow/ui/src/layouts/Nav/NavButton.test.tsx | 14 +
.../src/airflow/ui/src/layouts/Nav/NavButton.tsx | 8 +-
.../src/pages/Asset/CreateAssetEventModal.test.tsx | 161 +-
.../ui/src/pages/Asset/CreateAssetEventModal.tsx | 20 +-
.../ui/src/pages/AssetsList/AssetsList.test.tsx | 85 +-
.../airflow/ui/src/pages/AssetsList/AssetsList.tsx | 10 +-
.../src/airflow/ui/src/pages/Dag/Code/Code.tsx | 2 +-
airflow-core/src/airflow/ui/src/pages/Dag/Dag.tsx | 3 +-
.../pages/Dag/Versions/VersionDiff.i18n.test.tsx | 83 +
.../ui/src/pages/Dag/Versions/VersionDiff.test.tsx | 275 +++
.../ui/src/pages/Dag/Versions/VersionDiff.tsx | 197 ++
.../ui/src/pages/Dag/Versions/Versions.test.tsx | 227 ++
.../airflow/ui/src/pages/Dag/Versions/Versions.tsx | 182 ++
.../types.ts => pages/Dag/Versions/index.ts} | 19 +-
.../src/airflow/ui/src/pages/DagRuns/DagRuns.tsx | 4 +-
.../ui/src/pages/DagsList/DagCardActions.tsx | 1 -
.../DagsList/DagFolderTree/DagFolderTree.test.tsx | 323 +++
.../pages/DagsList/DagFolderTree/DagFolderTree.tsx | 216 ++
.../DagsList/DagFolderTree/buildFolderTree.test.ts | 119 ++
.../DagsList/DagFolderTree/buildFolderTree.ts | 152 ++
.../DagsList/DagFolderTree/index.ts} | 19 +-
.../src/airflow/ui/src/pages/DagsList/DagsList.tsx | 189 +-
.../src/airflow/ui/src/pages/Run/Details.tsx | 4 +-
.../ui/src/pages/Settings/Settings.test.tsx | 18 +
.../src/airflow/ui/src/pages/Settings/Settings.tsx | 16 +
.../ui/src/queries/useCreateBackfill.test.tsx | 99 +
.../airflow/ui/src/queries/useCreateBackfill.ts | 39 +-
.../types.ts => queries/useDagFolders.ts} | 29 +-
.../src/airflow/ui/src/queries/useDags.tsx | 6 +
.../src/airflow/ui/src/queries/useTrigger.test.tsx | 98 +
.../src/airflow/ui/src/queries/useTrigger.ts | 31 +-
airflow-core/src/airflow/ui/src/router.tsx | 6 +
.../ui/src/system-components/Menu/MenuTrigger.tsx | 2 +-
.../utils/{index.ts => hasDagRunConfig.test.ts} | 25 +-
.../types.ts => utils/hasDagRunConfig.ts} | 22 +-
airflow-core/src/airflow/ui/src/utils/index.ts | 1 +
airflow-core/src/airflow/utils/cli.py | 48 +-
airflow-core/src/airflow/utils/db.py | 2 +-
airflow-core/src/airflow/utils/file.py | 48 -
.../tests/unit/always/test_project_structure.py | 4 -
.../tests/unit/always/test_provider_yaml_schema.py | 10 +
.../tests/unit/api_fastapi/auth/test_tokens.py | 43 +
airflow-core/tests/unit/api_fastapi/conftest.py | 13 +
.../core_api/routes/public/test_assets.py | 48 +-
.../core_api/routes/public/test_auth.py | 45 +-
.../core_api/routes/public/test_backfills.py | 72 +-
.../core_api/routes/public/test_dag_run.py | 75 +-
.../core_api/routes/public/test_dag_versions.py | 296 +++
.../core_api/routes/public/test_dags.py | 175 +-
.../core_api/routes/public/test_hitl.py | 3 +-
.../api_fastapi/core_api/routes/ui/test_assets.py | 79 +
.../api_fastapi/core_api/routes/ui/test_dags.py | 183 ++
.../tests/unit/cli/commands/test_dag_command.py | 3 +
.../tests/unit/cli/commands/test_task_command.py | 26 +
.../tests/unit/dag_processing/fake_lang_sdk.py | 144 ++
.../dag_processing/fake_task_handler_runtime.py | 6 +-
.../lang_sdk_fixtures/java_native.json | 151 ++
.../lang_sdk_fixtures/ts_conformance.json | 661 ++++++
.../tests/unit/dag_processing/test_dagbag.py | 334 ++-
.../unit/dag_processing/test_importer_routing.py | 63 +
.../unit/dag_processing/test_lang_sdk_processor.py | 868 ++++++++
.../tests/unit/dag_processing/test_manager.py | 542 ++++-
.../tests/unit/dag_processing/test_processor.py | 57 +-
.../dag_processing/test_task_handler_processor.py | 451 +++-
.../test_task_handler_processor_go.py | 52 +-
.../tests/unit/executors/test_local_executor.py | 455 +++-
.../executors/test_local_executor_check_workers.py | 5 +-
airflow-core/tests/unit/jobs/test_scheduler_job.py | 83 +-
airflow-core/tests/unit/models/test_backfill.py | 133 +-
airflow-core/tests/unit/models/test_dag.py | 45 +-
airflow-core/tests/unit/models/test_dagrun.py | 79 +-
.../unit/models/test_lang_sdk_task_handler.py | 233 --
.../tests/unit/models/test_serialized_dag.py | 4 +-
.../tests/unit/models/test_taskinstance.py | 25 +-
airflow-core/tests/unit/models/test_trigger.py | 45 +
.../unit/serialization/test_dag_serialization.py | 215 ++
.../unit/serialization/test_dag_version_diff.py | 1 -
airflow-core/tests/unit/utils/test_cli_util.py | 79 +
airflow-core/tests/unit/utils/test_db_cleanup.py | 2 -
airflow-core/tests/unit/utils/test_file.py | 37 +-
.../airflowctl_tests/test_airflowctl_commands.py | 2 +
airflow-ctl/docs/images/command_hashes.txt | 2 +-
airflow-ctl/docs/images/output_tasks.svg | 78 +-
.../src/airflowctl/api/datamodels/generated.py | 178 ++
airflow-ctl/src/airflowctl/api/operations.py | 7 +-
airflow-ctl/src/airflowctl/ctl/cli_config.py | 17 +
.../src/airflowctl/ctl/commands/task_command.py | 53 +-
.../tests/airflow_ctl/api/test_operations.py | 54 +-
.../airflow_ctl/ctl/commands/test_task_command.py | 275 ++-
chart/README.md | 2 +-
contributing-docs/08_static_code_checks.rst | 1 +
contributing-docs/12_provider_distributions.rst | 14 +-
contributing-docs/30_new_language_sdk.rst | 58 +-
dev/README_RELEASE_PROVIDERS.md | 5 +-
dev/breeze/doc/03_developer_tasks.rst | 55 +
dev/breeze/doc/ci/04_selective_checks.md | 38 +-
dev/breeze/doc/images/output-commands.svg | 68 +-
.../doc/images/output_k8s_configure-cluster.svg | 8 +-
.../doc/images/output_k8s_configure-cluster.txt | 2 +-
.../doc/images/output_k8s_create-cluster.svg | 8 +-
.../doc/images/output_k8s_create-cluster.txt | 2 +-
.../doc/images/output_k8s_delete-cluster.svg | 4 +-
.../doc/images/output_k8s_delete-cluster.txt | 2 +-
.../doc/images/output_k8s_deploy-airflow.svg | 8 +-
.../doc/images/output_k8s_deploy-airflow.txt | 2 +-
.../doc/images/output_k8s_deploy-cluster.svg | 4 +-
.../doc/images/output_k8s_deploy-cluster.txt | 2 +-
dev/breeze/doc/images/output_k8s_dev.svg | 4 +-
dev/breeze/doc/images/output_k8s_dev.txt | 2 +-
dev/breeze/doc/images/output_k8s_k9s.svg | 4 +-
dev/breeze/doc/images/output_k8s_k9s.txt | 2 +-
dev/breeze/doc/images/output_k8s_logs.svg | 4 +-
dev/breeze/doc/images/output_k8s_logs.txt | 2 +-
.../doc/images/output_k8s_run-complete-tests.svg | 8 +-
.../doc/images/output_k8s_run-complete-tests.txt | 2 +-
.../doc/images/output_k8s_setup-lang-sdk-test.svg | 4 +-
.../doc/images/output_k8s_setup-lang-sdk-test.txt | 2 +-
dev/breeze/doc/images/output_k8s_shell.svg | 4 +-
dev/breeze/doc/images/output_k8s_shell.txt | 2 +-
.../doc/images/output_k8s_smoke-test-overlay.svg | 4 +-
.../doc/images/output_k8s_smoke-test-overlay.txt | 2 +-
dev/breeze/doc/images/output_k8s_status.svg | 4 +-
dev/breeze/doc/images/output_k8s_status.txt | 2 +-
dev/breeze/doc/images/output_k8s_tests.svg | 8 +-
dev/breeze/doc/images/output_k8s_tests.txt | 2 +-
.../doc/images/output_k8s_upload-k8s-image.svg | 8 +-
.../doc/images/output_k8s_upload-k8s-image.txt | 2 +-
.../output_setup_check-all-params-in-groups.svg | 4 +-
.../output_setup_check-all-params-in-groups.txt | 2 +-
.../output_setup_regenerate-command-images.svg | 4 +-
.../output_setup_regenerate-command-images.txt | 2 +-
.../doc/images/output_testing_helm-tests.svg | 4 +-
.../doc/images/output_testing_helm-tests.txt | 2 +-
dev/breeze/doc/images/output_verify.svg | 128 ++
dev/breeze/doc/images/output_verify.txt | 1 +
dev/breeze/src/airflow_breeze/breeze.py | 1 +
.../airflow_breeze/commands/ci_image_commands.py | 4 +-
.../commands/developer_commands_config.py | 11 +
.../airflow_breeze/commands/testing_commands.py | 10 +
.../src/airflow_breeze/commands/verify_commands.py | 180 ++
dev/breeze/src/airflow_breeze/global_constants.py | 16 +-
dev/breeze/src/airflow_breeze/utils/packages.py | 26 +
.../src/airflow_breeze/utils/reproduce_ci.py | 10 +-
.../src/airflow_breeze/utils/selective_checks.py | 235 +-
.../src/airflow_breeze/utils/verification_plan.py | 170 ++
dev/breeze/tests/test_ci_image_commands.py | 20 +
dev/breeze/tests/test_packages.py | 63 +
dev/breeze/tests/test_reproduce_ci.py | 12 +
dev/breeze/tests/test_run_test_args.py | 35 +-
dev/breeze/tests/test_selective_checks.py | 358 +++-
dev/breeze/tests/test_verification_plan.py | 244 +++
dev/breeze/tests/test_verify_commands.py | 327 +++
dev/breeze/uv.lock | 44 +-
.../react_plugin_template/package.json | 14 +-
.../react_plugin_template/pnpm-lock.yaml | 369 ++--
dev/skill-evals/last-eval-hash.txt | 2 +-
devel-common/src/docs/utils/conf_constants.py | 1 +
.../src/sphinx_exts/python_builtin_xrefs.py | 66 +
.../unit/sphinx_exts/test_python_builtin_xrefs.py | 92 +
docs/spelling_wordlist.txt | 3 +
generated/PYPI_README.md | 16 +-
generated/known_airflow_exceptions.txt | 4 +-
generated/known_sdk_imports_in_core.txt | 6 +-
go-sdk/README.md | 59 +-
go-sdk/adr/0008-native-dag-interface.md | 7 +-
go-sdk/airflow/bundle.go | 15 +-
go-sdk/airflow/bundle_test.go | 54 +
go-sdk/airflow/dag.go | 152 +-
go-sdk/airflow/dag_test.go | 22 +-
go-sdk/airflow/enums.go | 156 ++
go-sdk/airflow/enums_test.go | 221 ++
go-sdk/airflow/if.go | 228 ++
go-sdk/airflow/if_test.go | 693 ++++++
go-sdk/airflow/inputs.go | 78 +-
go-sdk/airflow/inputs_test.go | 55 +-
go-sdk/airflow/node.go | 383 ++++
go-sdk/airflow/node_test.go | 538 +++++
go-sdk/airflow/serve.go | 3 +-
go-sdk/airflow/serve_test.go | 62 +
go-sdk/airflow/spec.go | 33 +-
go-sdk/airflow/spec_test.go | 71 -
go-sdk/airflow/task_handler_test.go | 9 +
go-sdk/airflow/task_option.go | 35 +-
.../foreignnode/main.go} | 37 +-
go-sdk/airflow/trigger_dag_run.go | 36 +-
go-sdk/airflow/trigger_dag_run_test.go | 28 +-
go-sdk/cmd/airflow-go-pack/inspect_test.go | 2 +-
go-sdk/cmd/airflow-go-pack/pack.go | 102 +-
.../cmd/airflow-go-pack/pack_integration_test.go | 11 -
go-sdk/cmd/airflow-go-pack/pack_test.go | 157 +-
go-sdk/internal/airflowmetadata/airflowmetadata.go | 4 +-
go-sdk/internal/bundle/task.go | 127 +-
go-sdk/internal/bundle/task_test.go | 242 +++
go-sdk/internal/bundlefooter/footer.go | 8 +-
go-sdk/internal/genspec/authoring.go | 6 +-
go-sdk/internal/genspec/main.go | 9 +-
go-sdk/internal/genspec/normalize_test.go | 5 +-
go-sdk/pkg/binding/declaration.go | 10 +-
go-sdk/pkg/binding/declaration_test.go | 23 +
go-sdk/pkg/execution/client.go | 8 +
go-sdk/pkg/execution/client_test.go | 47 +
go-sdk/pkg/execution/comms.go | 8 +-
go-sdk/pkg/execution/genmodels/gen.go | 13 +-
go-sdk/pkg/execution/genmodels/models.gen.go | 3 +
go-sdk/pkg/execution/integration_test.go | 127 +-
go-sdk/pkg/execution/messages_test.go | 23 +-
go-sdk/pkg/execution/task_runner.go | 1 +
{ts-sdk => go-sdk}/schema/dag-schema.json | 13 +-
.../schema/supervisor-schema.json | 2 +-
java-sdk/buildSrc/build.gradle.kts | 2 +-
java-sdk/gradle/verification-metadata.xml | 410 ++--
.../org/apache/airflow/sdk/BuilderProcessor.kt | 2 +-
.../kotlin/org/apache/airflow/sdk/BuilderTest.kt | 4 +-
java-sdk/sdk/build.gradle.kts | 19 +-
java-sdk/sdk/schema/dag-schema.json | 13 +-
.../src/main/kotlin/org/apache/airflow/sdk/Arg.kt | 11 +-
.../main/kotlin/org/apache/airflow/sdk/Context.kt | 10 +-
.../main/kotlin/org/apache/airflow/sdk/DagDef.kt | 1 +
.../kotlin/org/apache/airflow/sdk/InputTask.kt | 2 +-
.../org/apache/airflow/sdk/execution/Task.kt | 7 +-
.../org/apache/airflow/sdk/internal/ArgValues.kt | 124 ++
.../org/apache/airflow/sdk/internal/TaskArgs.kt | 42 +-
.../org/apache/airflow/sdk/ArgTestSupport.kt | 15 +
.../kotlin/org/apache/airflow/sdk/ArgValuesTest.kt | 2 +-
.../kotlin/org/apache/airflow/sdk/InputTaskTest.kt | 90 +
.../org/apache/airflow/sdk/execution/TaskTest.kt | 24 +
.../apache/airflow/sdk/internal/ArgValuesTest.kt | 266 +++
.../kubernetes_tests/test_kubernetes_executor.py | 89 +-
providers/.last_release_date.txt | 2 +-
providers/MANAGING_PROVIDERS_LIFECYCLE.rst | 20 +
providers/airbyte/docs/connections.rst | 10 +-
.../src/airflow/providers/airbyte/hooks/airbyte.py | 37 +-
.../tests/unit/airbyte/hooks/test_airbyte.py | 61 +
providers/akeyless/docs/changelog.rst | 7 +
.../airflow/providers/akeyless/secrets/akeyless.py | 28 +-
.../tests/unit/akeyless/secrets/test_akeyless.py | 27 +-
.../docs/auth-manager/setup/identity-center.rst | 36 +
providers/amazon/docs/executors/batch-executor.rst | 4 +
providers/amazon/docs/executors/ecs-executor.rst | 4 +
providers/amazon/docs/executors/general.rst | 13 +
.../amazon/docs/executors/lambda-executor.rst | 5 +
providers/amazon/provider.yaml | 15 +
.../providers/amazon/aws/auth_manager/constants.py | 1 +
.../amazon/aws/auth_manager/routes/login.py | 200 +-
.../aws/executors/aws_lambda/lambda_executor.py | 67 +-
.../amazon/aws/executors/batch/batch_executor.py | 15 +-
.../providers/amazon/aws/executors/batch/utils.py | 6 +-
.../amazon/aws/executors/ecs/ecs_executor.py | 5 +-
.../airflow/providers/amazon/get_provider_info.py | 7 +
.../system/amazon/aws/example_athena_spark.py | 52 +-
.../amazon/aws/auth_manager/routes/test_login.py | 359 +++-
.../executors/aws_lambda/test_lambda_executor.py | 150 +-
.../aws/executors/batch/test_batch_executor.py | 105 +-
.../tests/unit/amazon/aws/executors/conftest.py} | 36 +-
.../amazon/aws/executors/ecs/test_ecs_executor.py | 135 +-
.../airflow/providers/apache/beam/hooks/beam.py | 45 +-
.../beam/tests/unit/apache/beam/hooks/test_beam.py | 42 +
providers/apache/kafka/docs/triggers.rst | 33 +
providers/apache/kafka/provider.yaml | 1 +
.../providers/apache/kafka/get_provider_info.py | 1 +
.../apache/kafka/triggers/shared_stream.py | 407 ++++
.../providers/apache/kafka/version_compat.py | 2 +
.../apache/kafka/triggers/test_shared_stream.py | 259 +++
.../apache/kafka/triggers/test_await_message.py | 49 +-
.../unit/apache/kafka/triggers/test_msg_queue.py | 5 +-
.../apache/kafka/triggers/test_shared_stream.py | 481 +++++
.../providers/apache/spark/hooks/spark_submit.py | 98 +-
.../unit/apache/spark/hooks/test_spark_submit.py | 118 +
providers/celery/docs/celery_executor.rst | 8 +
.../providers/celery/executors/celery_executor.py | 34 +-
.../integration/celery/test_celery_executor.py | 30 +-
.../unit/celery/executors/test_celery_executor.py | 287 ++-
providers/cncf/kubernetes/docs/changelog.rst | 5 +
.../cncf/kubernetes/docs/kubernetes_executor.rst | 9 +
providers/cncf/kubernetes/provider.yaml | 2 +-
.../kubernetes/executors/kubernetes_executor.py | 123 +-
.../executors/kubernetes_executor_types.py | 43 +-
.../executors/kubernetes_executor_utils.py | 37 +-
.../providers/cncf/kubernetes/triggers/job.py | 4 +-
.../executors/test_kubernetes_executor.py | 818 ++++++-
.../executors/test_kubernetes_executor_types.py | 89 +
.../unit/cncf/kubernetes/triggers/test_job.py | 13 +-
.../ai/docs/operators/llm_schema_compare.rst | 2 +-
providers/common/ai/docs/operators/llm_sql.rst | 2 +-
providers/common/ai/docs/toolsets/datafusion.rst | 2 +-
.../providers/common/compat/connection/__init__.py | 30 +-
.../common/compat/connection/test_connection.py | 58 +-
.../connection/test_connection_supervisor.py | 214 ++
providers/common/sql/docs/operators.rst | 60 +-
.../sql/src/airflow/providers/common/sql/config.py | 3 +
.../providers/common/sql/datafusion/engine.py | 185 ++
.../sql/datafusion/object_storage_provider.py | 36 +-
.../common/sql/example_dags/example_analytics.py | 13 +
.../unit/common/sql/datafusion/test_engine.py | 447 ++++
.../sql/datafusion/test_object_storage_provider.py | 48 +
.../sql/tests/unit/common/sql/test_config.py | 1 +
providers/duckdb/provider.yaml | 2 +-
providers/edge3/docs/changelog.rst | 8 +
providers/edge3/docs/migrations-ref.rst | 4 +-
providers/edge3/provider.yaml | 14 +-
.../src/airflow/providers/edge3/cli/api_client.py | 14 +-
.../src/airflow/providers/edge3/cli/worker.py | 22 +-
.../providers/edge3/executors/edge_executor.py | 214 +-
.../airflow/providers/edge3/get_provider_info.py | 12 +-
.../0006_5_0_0_add_task_instance_id_to_edge_job.py | 71 +
.../edge3/src/airflow/providers/edge3/models/db.py | 1 +
.../src/airflow/providers/edge3/models/edge_job.py | 10 +
.../www/openapi-gen/requests/schemas.gen.ts | 12 +
.../plugins/www/openapi-gen/requests/types.gen.ts | 10 +
.../providers/edge3/plugins/www/package.json | 24 +-
.../providers/edge3/plugins/www/pnpm-lock.yaml | 817 +++----
.../edge3/plugins/www/src/pages/JobsPage.test.tsx | 105 +
.../edge3/plugins/www/src/pages/JobsPage.tsx | 17 +-
.../providers/edge3/plugins/www/vite.config.ts | 4 +-
.../providers/edge3/worker_api/datamodels.py | 7 +
.../providers/edge3/worker_api/datamodels_ui.py | 2 +
.../providers/edge3/worker_api/routes/jobs.py | 84 +-
.../providers/edge3/worker_api/routes/ui.py | 1 +
.../edge3/worker_api/v2-edge-generated.yaml | 39 +
.../edge3/tests/unit/edge3/cli/test_api_client.py | 76 +-
.../edge3/tests/unit/edge3/cli/test_worker.py | 46 +-
.../unit/edge3/executors/test_edge_executor.py | 456 +++-
.../migrations/test_task_instance_identity.py | 170 ++
providers/edge3/tests/unit/edge3/models/test_db.py | 30 +-
.../unit/edge3/worker_api/routes/test_jobs.py | 144 +-
.../tests/unit/edge3/worker_api/routes/test_ui.py | 31 +
providers/fab/docs/auth-manager/access-control.rst | 4 +
providers/google/docs/changelog.rst | 9 +
providers/google/docs/operators/cloud/dataproc.rst | 14 +-
.../providers/google/cloud/hooks/bigtable.py | 24 +
.../providers/google/cloud/hooks/cloud_composer.py | 4 +-
.../providers/google/cloud/hooks/cloud_sql.py | 57 +-
.../providers/google/cloud/hooks/compute.py | 3 -
.../google/cloud/hooks/vertex_ai/auto_ml.py | 2 +-
.../providers/google/cloud/operators/dataproc.py | 19 +-
.../providers/google/cloud/operators/gcs.py | 24 +-
.../google/cloud/transfers/gcs_to_sftp.py | 84 +-
.../dataproc/example_dataproc_batch_persistent.py | 32 +-
.../dataproc/example_dataproc_cluster_generator.py | 156 --
.../tests/unit/google/cloud/hooks/test_bigtable.py | 26 +
.../unit/google/cloud/hooks/test_cloud_composer.py | 24 +
.../unit/google/cloud/hooks/test_cloud_sql.py | 78 +-
.../unit/google/cloud/hooks/test_stackdriver.py | 2 +-
.../unit/google/cloud/operators/test_dataproc.py | 7 +-
.../tests/unit/google/cloud/operators/test_gcs.py | 135 ++
.../google/cloud/transfers/test_gcs_to_sftp.py | 122 +-
.../tests/unit/google/test_go_module_utils.py | 73 +
providers/hashicorp/docs/changelog.rst | 7 +
.../airflow/providers/hashicorp/secrets/vault.py | 31 +-
.../tests/unit/hashicorp/secrets/test_vault.py | 55 +
providers/ibm/mq/docs/index.rst | 2 +-
providers/ibm/mq/pre_extras_install.yaml | 33 -
providers/ibm/mq/pyproject.toml | 4 +-
providers/ibm/mq/tests/conftest.py | 8 +-
providers/microsoft/azure/README.rst | 6 +-
providers/microsoft/azure/docs/index.rst | 6 +-
providers/microsoft/azure/pyproject.toml | 15 +-
.../providers/microsoft/azure/hooks/data_lake.py | 13 +-
.../providers/microsoft/azure/hooks/msgraph.py | 13 +-
.../unit/microsoft/azure/hooks/test_data_lake.py | 61 +-
.../unit/microsoft/azure/hooks/test_msgraph.py | 17 +
.../providers/openlineage/extractors/manager.py | 31 +
...schedule_multiple_cron_trigger_timetable_dag.py | 2 +-
...hedule_multiple_cron_trigger_timetable_dag.json | 12 +-
.../unit/openlineage/extractors/test_manager.py | 71 +-
.../providers/opensearch/log/os_task_handler.py | 32 +-
.../opensearch/log/test_os_remote_log_io.py | 18 +
.../unit/opensearch/log/test_os_task_handler.py | 102 +-
.../snowflake/hooks/snowflake_cortex_agent.py | 176 ++
.../providers/snowflake/hooks/snowflake_sql_api.py | 12 +-
.../snowflake/hooks/test_snowflake_cortex_agent.py | 211 ++
.../unit/snowflake/hooks/test_snowflake_sql_api.py | 35 +-
.../tests/unit/standard/operators/test_empty.py | 56 +
...ternal_task_sensor.py => test_external_task.py} | 34 +-
pyproject.toml | 1 -
registry/package.json | 2 +-
registry/pnpm-lock.yaml | 44 +-
.../ci/lang_sdk_serialization/serialize_python.py | 23 +-
.../prek/check_contextmanager_class_decorators.py | 1 +
scripts/ci/prek/check_excluded_provider_markers.py | 36 +-
scripts/ci/prek/check_execution_api_versions.py | 15 +-
scripts/ci/prek/check_go_sdk_generated_drift.py | 25 +-
.../ci/prek/check_supervisor_schemas_versions.py | 11 +-
scripts/ci/prek/common_prek_utils.py | 21 +
scripts/ci/prek/sync_go_sdk_schemas.py | 145 ++
scripts/ci/prek/update_airflow_pyproject_toml.py | 35 +-
scripts/ci/testing/run_unit_tests.sh | 7 +-
scripts/docker/entrypoint_ci.sh | 2 +-
scripts/docker/install_os_dependencies.sh | 5 +-
scripts/in_container/run_pre_extras_install.py | 64 +-
.../test_check_contextmanager_class_decorators.py | 80 +
.../prek/test_check_excluded_provider_markers.py | 56 +
.../ci/prek/test_check_go_sdk_generated_drift.py | 13 +-
scripts/tests/ci/prek/test_sync_go_sdk_schemas.py | 127 ++
.../ci/prek/test_update_airflow_pyproject_toml.py | 58 +
scripts/tests/ci/testing/test_run_unit_tests.py | 15 +
.../in_container/test_run_pre_extras_install.py | 142 ++
shared/dagnode/src/airflow_shared/dagnode/cycle.py | 73 +
shared/dagnode/tests/dagnode/test_cycle.py | 79 +
task-sdk/docs/airflow-metadata.schema.json | 17 -
task-sdk/docs/conf.py | 1 +
task-sdk/docs/deferred-vs-async-operators.rst | 2 +-
task-sdk/docs/executable-bundle-spec.rst | 39 +-
task-sdk/docs/lang-sdk-spec.rst | 113 +-
.../airflow/sdk/coordinators/_bundle_metadata.py | 3 +-
.../src/airflow/sdk/coordinators/_dag_importer.py | 165 ++
.../src/airflow/sdk/coordinators/_subprocess.py | 289 ++-
.../sdk/coordinators/executable/coordinator.py | 91 +-
.../airflow/sdk/coordinators/java/_dag_importer.py | 122 ++
.../airflow/sdk/coordinators/java/_jar_manifest.py | 71 +
.../airflow/sdk/coordinators/java/coordinator.py | 138 +-
.../sdk/coordinators/node/_bundle_reader.py | 29 +-
.../airflow/sdk/coordinators/node/_dag_importer.py | 62 +
.../airflow/sdk/coordinators/node/coordinator.py | 27 +-
task-sdk/src/airflow/sdk/definitions/dag.py | 41 +-
.../src/airflow/sdk/execution_time/coordinator.py | 193 +-
.../airflow/sdk/execution_time/schema/__init__.py | 4 +-
.../airflow/sdk/execution_time/schema/schema.json | 2 +-
.../src/airflow/sdk/execution_time/supervisor.py | 94 +-
.../src/airflow/sdk/execution_time/task_runner.py | 38 +-
task-sdk/src/airflow/sdk/importers/base.py | 85 +-
.../src/airflow/sdk/importers/python_importer.py | 22 +-
.../coordinators/executable/test_coordinator.py | 119 +-
.../task_sdk/coordinators/java/_jar_test_utils.py | 53 +
.../task_sdk/coordinators/java/test_coordinator.py | 279 ++-
.../coordinators/java/test_dag_importer.py | 297 +++
.../coordinators/java/test_jar_manifest.py | 94 +
.../coordinators/node/_bundle_test_utils.py | 5 +
.../coordinators/node/test_bundle_reader.py | 10 +-
.../task_sdk/coordinators/node/test_coordinator.py | 114 +-
.../coordinators/node/test_dag_importer.py | 243 +++
.../task_sdk/coordinators/test_dag_importer.py | 214 ++
.../tests/task_sdk/coordinators/test_subprocess.py | 556 ++++-
.../task_sdk/execution_time/test_coordinator.py | 339 ++-
.../task_sdk/execution_time/test_supervisor.py | 257 +++
.../task_sdk/execution_time/test_task_runner.py | 130 ++
.../task_sdk/importers/test_python_importer.py | 20 +
task-sdk/tests/task_sdk/importers/test_registry.py | 152 ++
ts-sdk/package.json | 2 +-
ts-sdk/pnpm-lock.yaml | 413 +---
ts-sdk/schema/dag-schema.json | 13 +-
ts-sdk/src/coordinator/runtime.ts | 69 +-
ts-sdk/src/generated/supervisor.ts | 2 +
ts-sdk/tests/coordinator/integration.test.ts | 108 +-
uv.lock | 21 +-
595 files changed, 35311 insertions(+), 6826 deletions(-)
create mode 100644 airflow-core/src/airflow/dag_processing/importer_routing.py
copy airflow-core/src/airflow/dag_processing/{task_handler_processor.py =>
lang_sdk_processor.py} (54%)
delete mode 100644
airflow-core/src/airflow/migrations/versions/0142_3_4_0_add_lang_sdk_task_handler_tables.py
delete mode 100644 airflow-core/src/airflow/models/lang_sdk_task_handler.py
create mode 100644
airflow-core/src/airflow/ui/src/components/TriggerDag/PausedDagOptions.test.tsx
create mode 100644
airflow-core/src/airflow/ui/src/components/TriggerDag/PausedDagOptions.tsx
create mode 100644
airflow-core/src/airflow/ui/src/components/TriggerDag/TriggerDAGButton.test.tsx
create mode 100644
airflow-core/src/airflow/ui/src/components/TriggerDag/TriggerDAGModal.test.tsx
create mode 100644
airflow-core/src/airflow/ui/src/components/VersionCompareSelect.test.tsx
rename airflow-core/src/airflow/ui/src/{pages/Dag/Code =>
components}/VersionCompareSelect.tsx (97%)
create mode 100644
airflow-core/src/airflow/ui/src/hooks/useAdvancedSearch.test.tsx
create mode 100644
airflow-core/src/airflow/ui/src/pages/Dag/Versions/VersionDiff.i18n.test.tsx
create mode 100644
airflow-core/src/airflow/ui/src/pages/Dag/Versions/VersionDiff.test.tsx
create mode 100644
airflow-core/src/airflow/ui/src/pages/Dag/Versions/VersionDiff.tsx
create mode 100644
airflow-core/src/airflow/ui/src/pages/Dag/Versions/Versions.test.tsx
create mode 100644
airflow-core/src/airflow/ui/src/pages/Dag/Versions/Versions.tsx
copy airflow-core/src/airflow/ui/src/{components/TriggerDag/types.ts =>
pages/Dag/Versions/index.ts} (59%)
create mode 100644
airflow-core/src/airflow/ui/src/pages/DagsList/DagFolderTree/DagFolderTree.test.tsx
create mode 100644
airflow-core/src/airflow/ui/src/pages/DagsList/DagFolderTree/DagFolderTree.tsx
create mode 100644
airflow-core/src/airflow/ui/src/pages/DagsList/DagFolderTree/buildFolderTree.test.ts
create mode 100644
airflow-core/src/airflow/ui/src/pages/DagsList/DagFolderTree/buildFolderTree.ts
copy airflow-core/src/airflow/ui/src/{components/TriggerDag/types.ts =>
pages/DagsList/DagFolderTree/index.ts} (59%)
create mode 100644
airflow-core/src/airflow/ui/src/queries/useCreateBackfill.test.tsx
copy airflow-core/src/airflow/ui/src/{components/TriggerDag/types.ts =>
queries/useDagFolders.ts} (59%)
create mode 100644 airflow-core/src/airflow/ui/src/queries/useTrigger.test.tsx
copy airflow-core/src/airflow/ui/src/utils/{index.ts =>
hasDagRunConfig.test.ts} (55%)
copy airflow-core/src/airflow/ui/src/{components/TriggerDag/types.ts =>
utils/hasDagRunConfig.ts} (59%)
create mode 100644 airflow-core/tests/unit/dag_processing/fake_lang_sdk.py
create mode 100644
airflow-core/tests/unit/dag_processing/lang_sdk_fixtures/java_native.json
create mode 100644
airflow-core/tests/unit/dag_processing/lang_sdk_fixtures/ts_conformance.json
create mode 100644
airflow-core/tests/unit/dag_processing/test_importer_routing.py
create mode 100644
airflow-core/tests/unit/dag_processing/test_lang_sdk_processor.py
delete mode 100644 airflow-core/tests/unit/models/test_lang_sdk_task_handler.py
create mode 100644 dev/breeze/doc/images/output_verify.svg
create mode 100644 dev/breeze/doc/images/output_verify.txt
create mode 100644 dev/breeze/src/airflow_breeze/commands/verify_commands.py
create mode 100644 dev/breeze/src/airflow_breeze/utils/verification_plan.py
create mode 100644 dev/breeze/tests/test_verification_plan.py
create mode 100644 dev/breeze/tests/test_verify_commands.py
create mode 100644 devel-common/src/sphinx_exts/python_builtin_xrefs.py
create mode 100644
devel-common/tests/unit/sphinx_exts/test_python_builtin_xrefs.py
create mode 100644 go-sdk/airflow/enums.go
create mode 100644 go-sdk/airflow/enums_test.go
create mode 100644 go-sdk/airflow/if.go
create mode 100644 go-sdk/airflow/if_test.go
create mode 100644 go-sdk/airflow/node.go
create mode 100644 go-sdk/airflow/node_test.go
delete mode 100644 go-sdk/airflow/spec_test.go
copy go-sdk/airflow/{task_option.go => testdata/foreignnode/main.go} (50%)
copy {ts-sdk => go-sdk}/schema/dag-schema.json (97%)
copy task-sdk/src/airflow/sdk/execution_time/schema/schema.json =>
go-sdk/schema/supervisor-schema.json (99%)
create mode 100644
java-sdk/sdk/src/test/kotlin/org/apache/airflow/sdk/internal/ArgValuesTest.kt
copy providers/{google/tests/unit/google/cloud/hooks/test_stackdriver.py =>
amazon/tests/unit/amazon/aws/executors/conftest.py} (54%)
create mode 100644
providers/apache/kafka/src/airflow/providers/apache/kafka/triggers/shared_stream.py
create mode 100644
providers/apache/kafka/tests/integration/apache/kafka/triggers/test_shared_stream.py
create mode 100644
providers/apache/kafka/tests/unit/apache/kafka/triggers/test_shared_stream.py
create mode 100644
providers/cncf/kubernetes/tests/unit/cncf/kubernetes/executors/test_kubernetes_executor_types.py
create mode 100644
providers/common/compat/tests/unit/common/compat/connection/test_connection_supervisor.py
create mode 100644
providers/edge3/src/airflow/providers/edge3/migrations/versions/0006_5_0_0_add_task_instance_id_to_edge_job.py
create mode 100644
providers/edge3/src/airflow/providers/edge3/plugins/www/src/pages/JobsPage.test.tsx
create mode 100644
providers/edge3/tests/unit/edge3/migrations/test_task_instance_identity.py
delete mode 100644
providers/google/tests/system/google/cloud/dataproc/example_dataproc_cluster_generator.py
create mode 100644 providers/google/tests/unit/google/test_go_module_utils.py
delete mode 100644 providers/ibm/mq/pre_extras_install.yaml
create mode 100644
providers/standard/tests/unit/standard/operators/test_empty.py
rename
providers/standard/tests/unit/standard/sensors/{test_external_task_sensor.py =>
test_external_task.py} (98%)
create mode 100755 scripts/ci/prek/sync_go_sdk_schemas.py
create mode 100644
scripts/tests/ci/prek/test_check_contextmanager_class_decorators.py
create mode 100644 scripts/tests/ci/prek/test_sync_go_sdk_schemas.py
create mode 100644 scripts/tests/ci/prek/test_update_airflow_pyproject_toml.py
create mode 100644 scripts/tests/in_container/test_run_pre_extras_install.py
create mode 100644 shared/dagnode/src/airflow_shared/dagnode/cycle.py
create mode 100644 shared/dagnode/tests/dagnode/test_cycle.py
create mode 100644 task-sdk/src/airflow/sdk/coordinators/_dag_importer.py
create mode 100644 task-sdk/src/airflow/sdk/coordinators/java/_dag_importer.py
create mode 100644 task-sdk/src/airflow/sdk/coordinators/java/_jar_manifest.py
create mode 100644 task-sdk/src/airflow/sdk/coordinators/node/_dag_importer.py
create mode 100644 task-sdk/tests/task_sdk/coordinators/java/_jar_test_utils.py
create mode 100644
task-sdk/tests/task_sdk/coordinators/java/test_dag_importer.py
create mode 100644
task-sdk/tests/task_sdk/coordinators/java/test_jar_manifest.py
create mode 100644
task-sdk/tests/task_sdk/coordinators/node/test_dag_importer.py
create mode 100644 task-sdk/tests/task_sdk/coordinators/test_dag_importer.py
