This is an automated email from the ASF dual-hosted git repository.
shahar1 pushed a change to branch drop-py310/6-ruff-dev-scripts
in repository https://gitbox.apache.org/repos/asf/airflow.git
discard 6db80151461 Apply ruff Python 3.11 fixes to dev tooling, scripts and
test suites
discard 3f74b3e6f21 Apply ruff Python 3.11 fixes to providers
discard 36cfc7f6adf Apply ruff Python 3.11 fixes to core, Task SDK, shared and
airflowctl
discard c0a3969a808 Remove Python 3.10 compatibility shims
discard f4ade48bbdb Refresh diagram checksums for the Python 3.11 script
headers
discard 76279c79c10 Bump inline script metadata to Python 3.11
discard 1cd3fc190f7 Drop Python 3.10 from CI, Breeze and the container images
discard c66c7a3d722 [DO NOT MERGE] Cherry-pick #74200 (deflake backfill e2e on
WebKit) to verify CI
add 38fefc1ca95 Fix invalid cron expression in OpenLineage system test Dag
(#74187)
add 27f827fcdce Honor the polling interval for deferred Kubernetes jobs
(#74143)
add 3bd4df6b2da Fix Snowflake SQL API wait_for_query timing out on
finished queries (#73761)
add f74a685978c Use configured default queue for Celery executor callbacks
(#73552)
add ad3bb1f676c Replace coordinator artifact roots with
task_handler_bundle_name (#73970)
add 1c3f78e1d57 Validate a serialized Dag and fill its unset settings from
the Airflow config (#74041)
add a44fd6d1f31 Allow explicit runs to drain a paused Dag (#73763)
add 40832901c59 Improve LocalExecutor bookkeeping: correctly add tasks to
the `running` list (#74142)
add 5487c080bea Add Azure Blob Storage support to the DataFusion object
storage layer (#73374)
add b191cc4767b Fail instead of silently falling behind when a Lang-SDK
vendored Dag schema drifts from airflow-core (#74091)
add e0202f40c78 Let a Dag importer find the coordinator that parses its
files (#74042)
add 94a677e8f87 Keep callback triggers assigned to healthy triggerers
(#72848)
add 4e1f2aca67f Keep the Dag trigger button one-click when a run has
config (#74039)
add bfe84598a52 Fix destination path validation in GCS-to-SFTP transfers
(#74186)
add 8239997bf20 Scope UI next_run_assets endpoint to assets the caller may
read (#72862)
add 7df6f0e69cc Parse native Lang-SDK Dags in the Dag processor with their
runtime (#74035)
add cffc14d173b Parse native Lang-SDK Dags in a Dag bag (#74043)
add e08216d0081 Go SDK: delete the XComs of earlier tries before a task
runs (#74178)
add ae082ab651f Go SDK: add dag.TaskGroup with nesting and group-level
edges (#74214)
add fa65343163d Remove FAB-related exclusion in SQLA tests (#74180)
add dbee3c6c11b Fix concurrent SFTP directory transfers dropping parent
hook's connection overrides (#73647)
add b79acc8a8d6 Fix google provider BQ openlineage checks (#71777)
add d6ab1aad3c6 Register TypeScript, Java and Go grammars for the Dag Code
tab (#74231)
add 747f69b1a8c Warn when db clean skips only some rows of a batch (#73568)
add 9b2cc1352c3 Handle plain Step Functions execution errors (#73881)
add d339bb2767a Fix Dataproc triggers swallowing task cancellation (#74192)
add e0e3454998f Add v3-4-test to CI, dependabot, backport, and fork-sync
configuration (#74219)
add 98e753847a3 Distinguish credential failures from AWS waiter exhaustion
(#73870)
add 71f98d3ac00 UI: Group digits of counters according to the selected
locale (#73713)
add 2d734dc9842 UI: Deflake backfill e2e tests on WebKit (#74200)
add 20f90a6e105 UI: Keep Dags list table rows on a single line (#74002)
add dafa0555a0e Bump stylelint (#74252)
add 08fae625769 Add datafusion handler for google cleanup script (#74234)
add 11bf87404c2 Restrict the second render pass in
SqlToSlackWebhookOperator (#71402)
add a048842a72a Fix google provider system tests dag version increase
issues (#72194)
add 48f6b517397 Require moto 5.2.2 or newer for the Amazon provider tests
(#74224)
add eb89b4092b9 Fix SSH commands failing when the task holds over 1024
file descriptors (#74211)
add 86aef85e35c Keep S3 Dag bundle downloads within the configured
directory (#73756)
add 1ab53cd8e38 Keep the Dag asset schedule the same shape for every
caller (#73805)
add fbdec9503c7 Support Table objects in BigQueryHook.create_table
table_resource (#73099)
add 67853607ab2 Add async Variable methods (aget, aset, adelete, akeys)
(#72329)
add 353d856281c Stop Kafka hook tests from leaving live clients behind
(#74285)
add b2786de47b7 Close German Translation Gaps 2026-10-05 (#74264)
add 2b33f02539a UI: Open provider connection fields by default and title
them by connection type (#74273)
add d4a47caaf09 Notify downstream Dags when COPY INTO writes a Unity
(Databricks) table (#74191)
add f554a64e021 Drop Python 3.10 from CI, Breeze and the container images
add 500333099fa Bump inline script metadata to Python 3.11
add daa68a43d74 Refresh diagram checksums for the Python 3.11 script
headers
add 2911db93332 Remove Python 3.10 compatibility shims
add 0f51ddbac21 Apply ruff Python 3.11 fixes to core, Task SDK, shared and
airflowctl
add 39372d9490d Apply ruff Python 3.11 fixes to providers
add db48c5336b7 Apply ruff Python 3.11 fixes to dev tooling, scripts and
test suites
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (6db80151461)
\
N -- N -- N refs/heads/drop-py310/6-ruff-dev-scripts
(db48c5336b7)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.agents/skills/airflow-java-sdk/SKILL.md | 11 +-
.github/boring-cyborg.yml | 6 +-
.github/dependabot.yml | 34 +-
.github/workflows/ci-notification.yml | 2 +-
.github/workflows/milestone-tag-assistant.yml | 1 +
...t.yml => scheduled-upgrade-check-v3-4-test.yml} | 8 +-
.pre-commit-config.yaml | 42 +-
Dockerfile.ci | 7 +-
airflow-core/adr/lang-sdk/0004-dag-parsing.md | 12 +-
.../adr/lang-sdk/0010-native-dag-processing.md | 118 +-
.../0013-persisted-task-handler-bindings.md | 2 +-
.../authoring-and-scheduling/language-sdks/go.rst | 93 +-
.../language-sdks/index.rst | 6 +-
.../language-sdks/java.rst | 112 +-
.../language-sdks/typescript.rst | 85 +-
airflow-core/docs/core-concepts/backfill.rst | 7 +-
airflow-core/docs/core-concepts/dags.rst | 13 +
airflow-core/newsfragments/72407.significant.rst | 10 +-
airflow-core/newsfragments/73970.significant.rst | 5 +
.../api_fastapi/core_api/datamodels/backfills.py | 8 +
.../api_fastapi/core_api/datamodels/dag_run.py | 7 +
.../api_fastapi/core_api/datamodels/ui/assets.py | 6 +
.../api_fastapi/core_api/openapi/_private_ui.yaml | 4 +
.../core_api/openapi/v2-rest-api-generated.yaml | 28 +
.../api_fastapi/core_api/routes/public/assets.py | 10 +
.../core_api/routes/public/backfills.py | 17 +-
.../api_fastapi/core_api/routes/public/dag_run.py | 11 +
.../api_fastapi/core_api/routes/ui/assets.py | 19 +-
.../src/airflow/cli/commands/dag_command.py | 9 +-
.../src/airflow/cli/commands/task_command.py | 5 +-
.../src/airflow/config_templates/config.yml | 31 +-
airflow-core/src/airflow/dag_processing/dagbag.py | 157 ++-
.../src/airflow/dag_processing/importer_routing.py | 52 +
.../airflow/dag_processing/lang_sdk_processor.py | 579 +++++++++
airflow-core/src/airflow/dag_processing/manager.py | 48 +-
.../src/airflow/dag_processing/processor.py | 71 +-
.../src/airflow/executors/local_executor.py | 75 +-
airflow-core/src/airflow/models/backfill.py | 3 +
airflow-core/src/airflow/models/dag.py | 16 +
airflow-core/src/airflow/models/trigger.py | 1 +
.../src/airflow/serialization/definitions/dag.py | 2 +
airflow-core/src/airflow/serialization/schema.json | 13 +-
.../airflow/serialization/serialized_objects.py | 96 ++
.../airflow/ui/openapi-gen/requests/schemas.gen.ts | 23 +
.../airflow/ui/openapi-gen/requests/types.gen.ts | 13 +
.../airflow/ui/public/i18n/locales/ar/admin.json | 1 -
.../ui/public/i18n/locales/ar/components.json | 4 +-
.../airflow/ui/public/i18n/locales/ca/admin.json | 1 -
.../ui/public/i18n/locales/ca/components.json | 4 +-
.../airflow/ui/public/i18n/locales/de/admin.json | 1 -
.../airflow/ui/public/i18n/locales/de/common.json | 7 +
.../ui/public/i18n/locales/de/components.json | 16 +-
.../src/airflow/ui/public/i18n/locales/de/dag.json | 63 +
.../airflow/ui/public/i18n/locales/de/dags.json | 7 +
.../airflow/ui/public/i18n/locales/el/admin.json | 1 -
.../ui/public/i18n/locales/el/components.json | 4 +-
.../airflow/ui/public/i18n/locales/en/admin.json | 2 +-
.../ui/public/i18n/locales/en/components.json | 15 +-
.../airflow/ui/public/i18n/locales/es/admin.json | 1 -
.../ui/public/i18n/locales/es/components.json | 4 +-
.../airflow/ui/public/i18n/locales/fr/admin.json | 1 -
.../ui/public/i18n/locales/fr/components.json | 4 +-
.../airflow/ui/public/i18n/locales/he/admin.json | 1 -
.../ui/public/i18n/locales/he/components.json | 4 +-
.../airflow/ui/public/i18n/locales/hi/admin.json | 1 -
.../ui/public/i18n/locales/hi/components.json | 4 +-
.../airflow/ui/public/i18n/locales/hu/admin.json | 1 -
.../ui/public/i18n/locales/hu/components.json | 4 +-
.../airflow/ui/public/i18n/locales/it/admin.json | 1 -
.../ui/public/i18n/locales/it/components.json | 4 +-
.../airflow/ui/public/i18n/locales/ja/admin.json | 1 -
.../ui/public/i18n/locales/ja/components.json | 4 +-
.../airflow/ui/public/i18n/locales/ko/admin.json | 1 -
.../ui/public/i18n/locales/ko/components.json | 4 +-
.../airflow/ui/public/i18n/locales/nl/admin.json | 1 -
.../ui/public/i18n/locales/nl/components.json | 4 +-
.../airflow/ui/public/i18n/locales/pl/admin.json | 1 -
.../ui/public/i18n/locales/pl/components.json | 4 +-
.../airflow/ui/public/i18n/locales/pt/admin.json | 1 -
.../ui/public/i18n/locales/pt/components.json | 4 +-
.../airflow/ui/public/i18n/locales/ru/admin.json | 1 -
.../ui/public/i18n/locales/ru/components.json | 4 +-
.../airflow/ui/public/i18n/locales/th/admin.json | 1 -
.../ui/public/i18n/locales/th/components.json | 4 +-
.../airflow/ui/public/i18n/locales/tr/admin.json | 1 -
.../ui/public/i18n/locales/tr/components.json | 4 +-
.../ui/public/i18n/locales/zh-CN/admin.json | 1 -
.../ui/public/i18n/locales/zh-CN/components.json | 4 +-
.../ui/public/i18n/locales/zh-TW/admin.json | 1 -
.../ui/public/i18n/locales/zh-TW/components.json | 4 +-
.../components/ActionAccordion/ActionAccordion.tsx | 6 +-
.../src/components/AssetExpression/AssetNode.tsx | 7 +-
.../ui/src/components/AssetProgressCell.tsx | 6 +-
.../ui/src/components/Assets/AssetEvents.tsx | 6 +-
.../ui/src/components/Assets/ListPopover.tsx | 46 +-
.../ui/src/components/Assets/TriggeredRuns.tsx | 6 +-
.../DAGWarningsModal.test.tsx} | 25 +-
.../airflow/ui/src/components/DAGWarningsModal.tsx | 4 +
.../components/DagActions/RunBackfillForm.test.tsx | 97 +-
.../src/components/DagActions/RunBackfillForm.tsx | 35 +-
.../ui/src/components/DataTable/DataTable.tsx | 4 +-
.../ImportErrorCount.test.tsx} | 36 +-
.../airflow/ui/src/components/ImportErrorCount.tsx | 7 +-
.../src/components/MonacoEditor/configureMonaco.ts | 122 +-
.../airflow/ui/src/components/NeedsReviewBadge.tsx | 6 +-
.../src/airflow/ui/src/components/PoolBar.test.tsx | 24 +-
.../src/airflow/ui/src/components/PoolBar.tsx | 7 +-
.../ui/src/components/RollupKeyChecklist.tsx | 51 +-
.../StatsCard.test.tsx} | 23 +-
.../src/airflow/ui/src/components/StatsCard.tsx | 7 +-
.../ui/src/components/TaskInstanceTooltip.test.tsx | 30 +-
.../ui/src/components/TaskInstanceTooltip.tsx | 6 +-
.../src/airflow/ui/src/components/TaskName.tsx | 9 +-
.../airflow/ui/src/components/TrendCountButton.tsx | 12 +-
.../TriggerDag/PausedDagOptions.test.tsx | 250 ++++
.../src/components/TriggerDag/PausedDagOptions.tsx | 123 ++
.../TriggerDag/TriggerDAGButton.test.tsx | 102 +-
.../src/components/TriggerDag/TriggerDAGButton.tsx | 171 +--
.../components/TriggerDag/TriggerDAGForm.test.tsx | 78 +-
.../src/components/TriggerDag/TriggerDAGForm.tsx | 32 +-
.../components/TriggerDag/TriggerDAGModal.test.tsx | 238 ++++
.../src/components/TriggerDag/TriggerDAGModal.tsx | 21 +-
.../airflow/ui/src/components/TriggerDag/types.ts | 3 +
.../ui/src/layouts/Details/DetailsLayout.tsx | 7 +-
.../src/pages/Asset/CreateAssetEventModal.test.tsx | 161 ++-
.../ui/src/pages/Asset/CreateAssetEventModal.tsx | 20 +-
.../ui/src/pages/AssetsList/DependencyPopover.tsx | 6 +-
.../src/pages/Connections/ConnectionForm.test.tsx | 121 ++
.../ui/src/pages/Connections/ConnectionForm.tsx | 11 +-
.../ui/src/pages/Connections/Connections.tsx | 6 +-
.../ui/src/pages/Dag/Calendar/CalendarTooltip.tsx | 8 +-
.../src/airflow/ui/src/pages/Dag/Details.tsx | 12 +-
.../src/airflow/ui/src/pages/Dag/Header.test.tsx | 12 +
.../src/airflow/ui/src/pages/Dag/Header.tsx | 7 +-
.../airflow/ui/src/pages/DagBundle/BundleFiles.tsx | 7 +-
.../src/airflow/ui/src/pages/DagBundle/Header.tsx | 5 +-
.../src/pages/DagRuns/BulkDeleteDagRunsButton.tsx | 5 +-
.../src/pages/DagRuns/BulkMarkDagRunsAsButton.tsx | 5 +-
.../src/airflow/ui/src/pages/DagRuns/DagRuns.tsx | 6 +-
.../ui/src/pages/DagsList/AssetSchedule.test.tsx | 125 ++
.../ui/src/pages/DagsList/AssetSchedule.tsx | 12 +-
.../pages/DagsList/BulkPauseDrainDagsButton.tsx | 5 +-
.../src/pages/DagsList/BulkUnpauseDagsButton.tsx | 5 +-
.../ui/src/pages/DagsList/DagCardActions.tsx | 1 -
.../src/pages/DagsList/DagRunStateCounts.test.tsx | 13 +-
.../ui/src/pages/DagsList/DagRunStateCounts.tsx | 5 +-
.../ui/src/pages/DagsList/DagsList.test.tsx | 34 +
.../src/airflow/ui/src/pages/DagsList/DagsList.tsx | 46 +-
.../DagsList/Schedule.test.tsx} | 47 +-
.../src/airflow/ui/src/pages/DagsList/Schedule.tsx | 2 +-
.../Dashboard/HistoricalMetrics/MetricSection.tsx | 3 +-
.../Dashboard/PoolSummary/PoolSummary.test.tsx | 1 +
.../src/pages/Dashboard/Stats/DagImportErrors.tsx | 4 +-
.../pages/Dashboard/Stats/PluginImportErrors.tsx | 4 +-
.../ui/src/pages/Dashboard/Stats/Stats.test.tsx | 5 +-
.../ui/src/pages/GroupTaskInstance/Header.tsx | 6 +-
.../ui/src/pages/MappedTaskInstance/Details.tsx | 6 +-
.../ui/src/pages/MappedTaskInstance/Header.tsx | 8 +-
.../MappedTaskInstance/MappedTaskInstance.tsx | 10 +-
.../src/airflow/ui/src/pages/Pools/PoolBarCard.tsx | 7 +-
.../BulkDeleteTaskInstancesButton.tsx | 5 +-
.../BulkMarkTaskInstancesAsButton.tsx | 5 +-
.../ui/src/pages/TaskInstances/TaskInstances.tsx | 6 +-
.../airflow/ui/src/pages/Variables/Variables.tsx | 6 +-
.../ui/src/queries/useCreateBackfill.test.tsx | 99 ++
.../airflow/ui/src/queries/useCreateBackfill.ts | 39 +-
.../src/airflow/ui/src/queries/useTrigger.test.tsx | 98 ++
.../src/airflow/ui/src/queries/useTrigger.ts | 31 +-
.../ui/src/system-components/Menu/MenuTrigger.tsx | 2 +-
.../ui/src/system-components/Pagination/Item.tsx | 6 +-
.../src/system-components/Pagination/PageText.tsx | 10 +-
.../ui/src/system-components/Select/ValueText.tsx | 6 +-
.../src/airflow/ui/src/utils/datetimeUtils.ts | 31 +-
.../context.ts => utils/formatNumber.test.ts} | 24 +-
.../ui/selectors.ts => src/utils/formatNumber.ts} | 14 +-
airflow-core/src/airflow/ui/src/utils/index.ts | 1 +
airflow-core/src/airflow/ui/src/utils/intlCache.ts | 48 +
airflow-core/src/airflow/ui/src/vite-env.d.ts | 23 +-
airflow-core/src/airflow/utils/cli.py | 48 +-
airflow-core/src/airflow/utils/db_cleanup.py | 30 +-
airflow-core/tests/unit/api_fastapi/conftest.py | 13 +
.../core_api/routes/public/test_assets.py | 48 +-
.../core_api/routes/public/test_backfills.py | 72 +-
.../core_api/routes/public/test_dag_run.py | 75 +-
.../core_api/routes/public/test_dags.py | 2 +-
.../api_fastapi/core_api/routes/ui/test_assets.py | 113 +-
.../tests/unit/cli/commands/test_dag_command.py | 3 +
.../tests/unit/cli/commands/test_task_command.py | 26 +
.../tests/unit/dag_processing/fake_lang_sdk.py | 144 +++
.../lang_sdk_fixtures/java_native.json | 151 +++
.../lang_sdk_fixtures/ts_conformance.json | 661 ++++++++++
.../tests/unit/dag_processing/test_dagbag.py | 304 +++++
.../unit/dag_processing/test_importer_routing.py | 63 +
.../unit/dag_processing/test_lang_sdk_processor.py | 839 +++++++++++++
.../tests/unit/dag_processing/test_manager.py | 241 +++-
.../tests/unit/dag_processing/test_processor.py | 69 +-
.../tests/unit/executors/test_local_executor.py | 455 ++++++-
.../executors/test_local_executor_check_workers.py | 5 +-
airflow-core/tests/unit/jobs/test_scheduler_job.py | 9 +-
airflow-core/tests/unit/models/test_backfill.py | 133 +-
airflow-core/tests/unit/models/test_dag.py | 40 +
airflow-core/tests/unit/models/test_dagrun.py | 6 +-
.../tests/unit/models/test_serialized_dag.py | 4 +-
airflow-core/tests/unit/models/test_trigger.py | 45 +
.../unit/serialization/test_dag_serialization.py | 215 ++++
.../unit/serialization/test_dag_version_diff.py | 1 -
airflow-core/tests/unit/utils/test_cli_util.py | 79 ++
airflow-core/tests/unit/utils/test_db_cleanup.py | 62 +-
.../src/airflowctl/api/datamodels/generated.py | 21 +
airflow-ctl/src/airflowctl/api/operations.py | 7 +-
.../tests/airflow_ctl/api/test_operations.py | 54 +-
.../tests/airflow_e2e_tests/conftest.py | 56 +-
contributing-docs/30_new_language_sdk.rst | 8 +-
dev/sync_fork.sh | 2 +-
docker-tests/tests/docker_tests/test_prod_image.py | 3 -
generated/known_sdk_imports_in_core.txt | 2 +
go-sdk/Justfile | 2 +-
go-sdk/README.md | 50 +-
go-sdk/adr/0008-native-dag-interface.md | 21 +-
go-sdk/airflow/bundle.go | 11 +-
go-sdk/airflow/dag.go | 179 ++-
go-sdk/airflow/if.go | 30 +-
go-sdk/airflow/if_test.go | 47 +-
go-sdk/airflow/inputs.go | 14 +-
go-sdk/airflow/inputs_test.go | 4 +-
go-sdk/airflow/node.go | 172 ++-
go-sdk/airflow/node_test.go | 11 +-
go-sdk/airflow/spec.gen.go | 31 +-
go-sdk/airflow/spec.go | 6 +-
go-sdk/airflow/task_group.go | 553 +++++++++
go-sdk/airflow/task_group_test.go | 1274 ++++++++++++++++++++
go-sdk/airflow/task_option.go | 10 +-
go-sdk/airflow/trigger_dag_run.go | 4 +-
go-sdk/example/bundle/Justfile | 5 +-
go-sdk/internal/bundle/task.go | 22 +-
go-sdk/internal/bundle/task_test.go | 35 +-
go-sdk/internal/genspec/authoring.go | 35 +-
go-sdk/internal/genspec/authoring_test.go | 22 +-
go-sdk/internal/genspec/main.go | 4 +-
go-sdk/internal/genspec/normalize.go | 14 +-
go-sdk/internal/genspec/normalize_test.go | 33 +-
go-sdk/pkg/execution/client.go | 32 +-
go-sdk/pkg/execution/client_test.go | 59 +
go-sdk/pkg/execution/integration_test.go | 326 ++++-
go-sdk/pkg/execution/task_runner.go | 64 +-
go-sdk/schema/dag-schema.json | 13 +-
java-sdk/README.md | 30 +-
.../apache/airflow/sdk/plugin/AirflowSdkPlugin.kt | 4 +-
java-sdk/scala_spark_example/README.md | 6 +-
java-sdk/sdk/build.gradle.kts | 15 +
java-sdk/sdk/schema/dag-schema.json | 13 +-
kubernetes-tests/lang_sdk/README.md | 14 +-
kubernetes-tests/lang_sdk/config/values.yaml | 14 +-
.../lang_sdk/pod_templates/lang_sdk_golang.yaml | 3 +-
.../lang_sdk/pod_templates/lang_sdk_java.yaml | 3 +-
kubernetes-tests/lang_sdk/stage_artifacts.py | 8 +-
providers/amazon/docs/bundles/index.rst | 5 +
providers/amazon/docs/changelog.rst | 6 +
providers/amazon/docs/operators/step_functions.rst | 11 +
providers/amazon/pyproject.toml | 2 +-
.../src/airflow/providers/amazon/aws/bundles/s3.py | 6 +-
.../src/airflow/providers/amazon/aws/exceptions.py | 4 +
.../amazon/aws/operators/step_function.py | 5 +-
.../amazon/aws/utils/waiter_with_logging.py | 24 +-
.../tests/unit/amazon/aws/bundles/test_s3.py | 33 +
.../amazon/aws/operators/test_step_function.py | 51 +
.../amazon/aws/utils/test_waiter_with_logging.py | 135 ++-
.../tests/unit/apache/kafka/hooks/test_consume.py | 9 +-
.../tests/unit/apache/kafka/hooks/test_produce.py | 9 +-
.../providers/celery/executors/celery_executor.py | 4 +-
.../unit/celery/executors/test_celery_executor.py | 29 +-
.../providers/cncf/kubernetes/triggers/job.py | 4 +-
.../unit/cncf/kubernetes/triggers/test_job.py | 13 +-
.../ai/docs/operators/llm_schema_compare.rst | 2 +-
providers/common/ai/docs/operators/llm_sql.rst | 2 +-
providers/common/ai/docs/toolsets/datafusion.rst | 2 +-
providers/common/sql/docs/operators.rst | 60 +-
.../sql/src/airflow/providers/common/sql/config.py | 3 +
.../providers/common/sql/datafusion/engine.py | 185 +++
.../sql/datafusion/object_storage_provider.py | 36 +-
.../common/sql/example_dags/example_analytics.py | 13 +
.../unit/common/sql/datafusion/test_engine.py | 447 +++++++
.../sql/datafusion/test_object_storage_provider.py | 48 +
.../sql/tests/unit/common/sql/test_config.py | 1 +
providers/databricks/docs/operators/copy_into.rst | 81 ++
.../providers/databricks/assets/databricks.py | 37 +
.../databricks/operators/databricks_sql.py | 82 +-
.../unit/databricks/assets/test_databricks.py | 55 +
.../databricks/operators/test_databricks_copy.py | 215 +++-
.../fab/src/airflow/providers/fab/www/package.json | 2 +-
.../src/airflow/providers/fab/www/pnpm-lock.yaml | 165 +--
.../providers/google/cloud/hooks/bigquery.py | 13 +-
.../google/cloud/transfers/gcs_to_sftp.py | 29 +-
.../providers/google/cloud/triggers/dataproc.py | 3 +
.../google/cloud/bigquery/example_bigquery_dts.py | 3 +-
.../cloud/bigquery/example_bigquery_sensors.py | 3 -
.../cloud/bigquery/example_bigquery_tables.py | 21 +-
.../cloud/bigquery/example_bigquery_value_check.py | 2 +-
.../resources/openlineage/bigquery_dts.json | 18 +-
.../resources/openlineage/bigquery_operations.json | 8 +-
.../resources/openlineage/bigquery_tables.json | 12 +-
.../resources/openlineage/bigquery_transfer.json | 8 +-
.../cloud/composer/example_cloud_composer.py | 4 +-
.../google/cloud/dataplex/example_dataplex_dp.py | 2 +-
.../example_cloud_storage_transfer_service_aws.py | 64 +-
.../example_cloud_storage_transfer_service_gcp.py | 55 +-
.../system/google/cloud/tasks/example_tasks.py | 15 +-
.../marketing_platform/example_campaign_manager.py | 6 +-
.../system/google/resources_cleanup/README.md | 8 +
.../handlers/__init__.py | 2 +
.../handlers/{sqladmin.py => datafusion.py} | 42 +-
.../tests/unit/google/cloud/hooks/test_bigquery.py | 55 +-
.../google/cloud/transfers/test_gcs_to_sftp.py | 50 +-
.../unit/google/cloud/triggers/test_dataproc.py | 63 +-
.../google/resources_cleanup/test_cmd_delete.py | 1 +
.../resources_cleanup/test_datafusion_handler.py | 67 +
...schedule_multiple_cron_trigger_timetable_dag.py | 2 +-
...hedule_multiple_cron_trigger_timetable_dag.json | 12 +-
.../sftp/src/airflow/providers/sftp/hooks/sftp.py | 44 +-
providers/sftp/tests/unit/sftp/hooks/test_sftp.py | 92 ++
.../slack/transfers/sql_to_slack_webhook.py | 28 +-
.../slack/transfers/test_sql_to_slack_webhook.py | 61 +
.../providers/snowflake/hooks/snowflake_sql_api.py | 12 +-
.../unit/snowflake/hooks/test_snowflake_sql_api.py | 35 +-
.../ssh/src/airflow/providers/ssh/hooks/ssh.py | 68 +-
providers/ssh/tests/unit/ssh/hooks/test_ssh.py | 106 +-
.../unit/standard/sensors/test_external_task.py | 2 +-
.../ci/lang_sdk_serialization/serialize_python.py | 23 +-
scripts/ci/prek/check_go_sdk_generated_drift.py | 5 +-
scripts/ci/prek/sync_go_sdk_schemas.py | 21 +-
scripts/docker/entrypoint_ci.sh | 7 +-
task-sdk/docs/executable-bundle-spec.rst | 31 +-
task-sdk/docs/ts-bundle-spec.rst | 26 +-
.../airflow/sdk/coordinators/_bundle_metadata.py | 45 +-
.../src/airflow/sdk/coordinators/_dag_importer.py | 162 +++
.../src/airflow/sdk/coordinators/_subprocess.py | 258 ++--
.../sdk/coordinators/executable/coordinator.py | 30 +-
.../airflow/sdk/coordinators/java/coordinator.py | 34 +-
.../airflow/sdk/coordinators/node/coordinator.py | 31 +-
task-sdk/src/airflow/sdk/definitions/variable.py | 55 +-
task-sdk/src/airflow/sdk/execution_time/context.py | 155 +++
.../src/airflow/sdk/execution_time/coordinator.py | 190 ++-
.../src/airflow/sdk/execution_time/task_runner.py | 36 +-
task-sdk/src/airflow/sdk/importers/base.py | 51 +-
.../coordinators/executable/test_coordinator.py | 73 +-
.../task_sdk/coordinators/java/test_coordinator.py | 96 +-
.../task_sdk/coordinators/node/test_coordinator.py | 45 +-
.../task_sdk/coordinators/test_dag_importer.py | 214 ++++
.../tests/task_sdk/coordinators/test_subprocess.py | 309 ++---
.../tests/task_sdk/definitions/test_variables.py | 139 ++-
.../tests/task_sdk/execution_time/test_context.py | 350 +++++-
.../task_sdk/execution_time/test_coordinator.py | 412 ++++++-
.../task_sdk/execution_time/test_task_runner.py | 130 ++
task-sdk/tests/task_sdk/importers/test_registry.py | 138 +++
ts-sdk/README.md | 23 +-
ts-sdk/example/README.md | 14 +-
ts-sdk/schema/dag-schema.json | 13 +-
uv.lock | 2 +-
358 files changed, 15832 insertions(+), 2276 deletions(-)
rename .github/workflows/{scheduled-upgrade-check-v3-3-test.yml =>
scheduled-upgrade-check-v3-4-test.yml} (89%)
create mode 100644 airflow-core/newsfragments/73970.significant.rst
create mode 100644 airflow-core/src/airflow/dag_processing/importer_routing.py
create mode 100644
airflow-core/src/airflow/dag_processing/lang_sdk_processor.py
copy airflow-core/src/airflow/ui/src/{system-components/Pagination/context.ts
=> components/DAGWarningsModal.test.tsx} (64%)
copy airflow-core/src/airflow/ui/src/{pages/Dag/Code/FileLocation.test.tsx =>
components/ImportErrorCount.test.tsx} (54%)
copy
airflow-core/src/airflow/ui/src/{pages/Connections/NothingFoundInfo.test.tsx =>
components/StatsCard.test.tsx} (63%)
create mode 100644
airflow-core/src/airflow/ui/src/components/TriggerDag/PausedDagOptions.test.tsx
create mode 100644
airflow-core/src/airflow/ui/src/components/TriggerDag/PausedDagOptions.tsx
create mode 100644
airflow-core/src/airflow/ui/src/components/TriggerDag/TriggerDAGModal.test.tsx
create mode 100644
airflow-core/src/airflow/ui/src/pages/Connections/ConnectionForm.test.tsx
create mode 100644
airflow-core/src/airflow/ui/src/pages/DagsList/AssetSchedule.test.tsx
copy airflow-core/src/airflow/ui/src/{layouts/Details/Grid/GridButton.test.tsx
=> pages/DagsList/Schedule.test.tsx} (52%)
create mode 100644
airflow-core/src/airflow/ui/src/queries/useCreateBackfill.test.tsx
create mode 100644 airflow-core/src/airflow/ui/src/queries/useTrigger.test.tsx
copy airflow-core/src/airflow/ui/src/{system-components/Pagination/context.ts
=> utils/formatNumber.test.ts} (64%)
copy airflow-core/src/airflow/ui/{tests/e2e/utils/ui/selectors.ts =>
src/utils/formatNumber.ts} (54%)
create mode 100644 airflow-core/src/airflow/ui/src/utils/intlCache.ts
create mode 100644 airflow-core/tests/unit/dag_processing/fake_lang_sdk.py
create mode 100644
airflow-core/tests/unit/dag_processing/lang_sdk_fixtures/java_native.json
create mode 100644
airflow-core/tests/unit/dag_processing/lang_sdk_fixtures/ts_conformance.json
create mode 100644
airflow-core/tests/unit/dag_processing/test_importer_routing.py
create mode 100644
airflow-core/tests/unit/dag_processing/test_lang_sdk_processor.py
create mode 100644 go-sdk/airflow/task_group.go
create mode 100644 go-sdk/airflow/task_group_test.go
copy
providers/google/tests/system/google/resources_cleanup/airflow_google_provider_resource_cleanup/handlers/{sqladmin.py
=> datafusion.py} (56%)
create mode 100644
providers/google/tests/unit/google/resources_cleanup/test_datafusion_handler.py
create mode 100644 task-sdk/src/airflow/sdk/coordinators/_dag_importer.py
create mode 100644 task-sdk/tests/task_sdk/coordinators/test_dag_importer.py
