This is an automated email from the ASF dual-hosted git repository.
github-actions[bot] pushed a change to branch ci-upgrade-main
in repository https://gitbox.apache.org/repos/asf/airflow.git
omit 99e5324a97f [main] CI: Upgrade important CI environment
add b70e056895f Fix stackdriver list alert policies JSON typo (#73148)
add 1370a121cee Update python client version/changelog for 3.3.2 release
(#73309)
add 42741924a79 Support assigned reviewers in LLM approval reviews (#72157)
add a1c2c50f4e4 Add SetVariable and DeleteVariable to the Go SDK task
client (#73084)
add b6d67b7fbb1 Fix Dag callbacks dropped when bundle versioning is
disabled (#72930)
add 9611649802d Add taskflow-binding capability to the Lang SDK
compatibility matrix (#73314)
add d878befd4c6 Document how retry policies and durable execution work
together (#73160)
add 1b4c2f30a78 i18n(zh-CN): add missing translations (#73113)
add f3faf90ff69 TS SDK: minify packed bundles and find them by metadata,
not filename (#73126)
add 9556dcf25b3 TS SDK: bind TaskFlow call arguments by folding names on
both sides (#73189)
add b295dde10fd TS SDK: resolve upstream XComs for bound TaskFlow
arguments (#73190)
add 7484939508d Add Insignis to the list of companies using Apache Airflow
(#73321)
add 613c9d3330d Stop backporting the scheduled CI environment upgrade
(#73319)
add 9db04b8693f Document that Core API session authorisation is resolved
at token issue (#73306)
add 92c919de773 Stop reporting components as degraded after an instance
restart (#73222)
add 24ff1f58d18 Report missing Anthropic WIF fields clearly (#71359)
add 60633c810c7 Skip provider dependency generation for help command in
breeze cli (#71689)
add 86ff1dc8186 Anchor top-level nav and needs-review links to root paths
(#73202)
add 44094b2661b Fix airflowctl tracebacks on redirects and non-JSON error
bodies (#73307)
add 257fe6e8511 UI: Show xcom before task state store in storage tabs
(#73313)
add 1e72a331931 Bump the auth-ui-package-updates group across 1 directory
with 11 updates (#73340)
add 9dcd6a0a2ee Hide unreadable assets in the asset expression served by
next_run_assets (#72864)
add a6ae1b1f169 Fix impersonation_chain and poll_sleep ignored by
deferrable Beam Dataflow pipelines (#72146)
add cd1f0e66277 Keep date-only range ends inclusive (#72860)
add c0cc98d00f9 Bump the github-actions-updates group with 5 updates
(#73345)
add edededd02dc Contain remote log upload paths within base_log_folder
(#72162)
add e386b821844 Fix typos, grammar, and broken references across the docs
(#73146)
add 656902875d5 Make EdgeExecutor respect [core] parallelism (#72048)
add 4ceb1fff01e Preserve SageMaker job lifecycle states in trigger events
(#71653)
add dd9ad598d4b Fix broken class xrefs in standard provider docs (#73360)
add 8cdb183d09e TS SDK: embed the entrypoint source in the Dag bundle
(#73127)
add bd65f7998b1 TS SDK: add withArgNames for explicit argument renames
(#73191)
add 99c9012db77 Go SDK: introduce the `airflow` package with
`airflow.Context` (#73277)
add 3c19778c2ec Allow DataSourceConfig to represent a plain database table
(#73273)
add 2a47d8c3347 Fix LLMBranchOperator sending branch options in a
different order on each worker (#73366)
add f2112c85667 Add Cortex Agent management methods to
SnowflakeCortexAgentHook (#70101)
add 42164201d42 Make Google deprecation date parsing locale independent
(#69405)
add 985e5007dc1 Add environments and trigger named parameters to
Databricks operators (#72505)
add 1050685b721 Add connection-driven provider failover for common.ai LLM
calls (#72156)
add 224d34121c8 Fix wrong session type in state-store custom-backend doc
example (#73361)
add f8d6f40fe95 Fix duplicate Dataproc job submission after a triggerer
restart (#72685)
add 274035646e1 Fix unformatted Kubernetes pod trigger error messages
(#73034)
add 47d28272bbc Demote ADC credential log from info to debug (#73199)
add 7da73c89cb0 Make the constraints check follow the cooldown rules the
constraints use (#73316)
add 45bad9f54dd Use async DB session for Execution API task-instance
heartbeat (#67800)
add 61db43fba9a Assert the panic value in TestNewContextRejectsNilArgs
(#73372)
add f546f7f0702 Clarify when code comments preserve useful context (#73359)
add 23c8d820870 Add GCS support to the DataFusion object storage layer
(#73370)
add 6ede6646097 Add SnowflakeNotebookOperator for executing Snowflake
Notebooks (#63470)
add 407485ef8cb Escape values interpolated into Google Drive q= queries
(#72166)
add 056806e8412 Keep object store credentials out of the task log in
Teradata transfers (#72176)
add 559330b276a TS SDK: add setVariable and deleteVariable to the task
client (#73357)
add 9ed42b0da2c Add missing Korean translations (#73115)
add 1dc7efe27ab Add a decision guide for choosing between common.ai
toolsets (#72936)
add 82f45f42261 Fix Test connection rejecting a connection that has no
port (#73298)
add 905bc89f6e4 Keep myself focussed temporarily on AI integrations
(#73404)
add 56d5432c1d5 Go SDK: require airflow.Context first and stop injecting
by type (#73394)
add ed72629410a Add branch_descriptions to LLMBranchOperator so the model
reads what each branch means (#73367)
add 2f846065252 Fix exception messages rendering as a tuple instead of the
value (#73296)
add c5d7f60fffe Fix EMR Serverless delete operator skipping deletion when
deferrable (#73323)
add 79991cd4db0 Fix id_to_job_info memory leak in the AWS Batch executor
(#72186)
add 0a41e28ec2a Preserve OpenSearch Serverless hook configuration when
deferring (#72472)
add fcdbf2d5b38 Log when a Snowpark container job returns no logs (#73011)
add aa4aeef0a14 Reject a DataSourceConfig with no uri or format in
DataFusionEngine (#73287)
add 471b82e12f7 Fix DataFusion storage-type error message on Python 3.11+
(#73413)
add 71a1f29213e Fix dataflow_config service_account for Beam Python and Go
pipelines (#73236)
add a675418bfaa Fix Google IdP credentials failing when project id cannot
be resolved (#72799)
add bda84d19fa3 Document that clear_on_success also removes the KPO pod
identity (#71749)
add af1c66d7426 Fix Teradata compute cluster trigger swallowing task
cancellation (#72695)
add e75094dbafc Resolve GCSFileTransformOperator destination fallbacks
after rendering (#70488)
add 259ed8698fa Avoid parsing templated DateTimeSensorAsync targets at Dag
parse time (#72659)
add a7132cce963 Log the HTTP response in HttpToGCSOperator when
log_response is set (#72952)
add 5d3cf0b49f1 Add tests for Hive plugin registration (#71970)
add 9ccb991e6ea Preserve DELETE request bodies in HttpAsyncHook (#72720)
add 37b96609ed7 Skip Dag params with no value when forwarding them to
Databricks (#71782)
add 25b192e043d Document proxy setup for Google provider using PySocks
(#71941)
add 10b07b4519b Fix deferrable Kubernetes 401s with a default exec-based
kubeconfig (#72300)
add 1a9091e4c5f contain gemini batch-job results file within
results_folder (#71832)
add 134b7332319 Add support for TypeSafe Jev classifier models (#73363)
add 834b6af975f Add a decision policy so an LLM operator asks a person
when the model is unsure (#73368)
add 1f7396d9a39 Require explicit task_id on TypeScript mixed-lang task
handlers (#73325)
add 51876b066a0 Add a Modal backend for the sandbox toolset (#72910)
add 730d6c4c4ea [main] CI: Upgrade important CI environment
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (99e5324a97f)
\
N -- N -- N refs/heads/ci-upgrade-main (730d6c4c4ea)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.github/CODEOWNERS | 16 +-
.github/instructions/code-review.instructions.md | 8 +-
.github/workflows/airflow-e2e-tests.yml | 2 +-
.github/workflows/basic-tests.yml | 2 +-
.github/workflows/ci-amd.yml | 2 +-
.github/workflows/ci-arm.yml | 2 +-
.github/workflows/ci-notification.yml | 2 +-
.github/workflows/codeql-analysis.yml | 8 +-
.../java-sdk-dependabot-verification-metadata.yml | 2 +-
.github/workflows/java-sdk-dependency-security.yml | 2 +-
.github/workflows/java-sdk-release-verify.yml | 4 +-
.github/workflows/k8s-tests.yml | 2 +-
.github/workflows/notify-uv-lock-conflicts.yml | 2 +-
.github/workflows/registry-tests.yml | 2 +-
.../scheduled-verify-release-calendar.yml | 2 +-
.github/workflows/upgrade-check.yml | 23 +
.pre-commit-config.yaml | 2 +-
.rat-excludes | 3 +
AGENTS.md | 9 +-
Dockerfile | 2 +-
Dockerfile.ci | 2 +-
INTHEWILD.md | 1 +
.../cluster-policies.rst | 4 +-
.../administration-and-deployment/dag-bundles.rst | 6 +-
.../dag-serialization.rst | 4 +-
.../dagfile-processing.rst | 10 +-
.../administration-and-deployment/kubernetes.rst | 4 +-
.../administration-and-deployment/listeners.rst | 4 +-
.../logging-monitoring/check-health.rst | 46 +-
.../logging-monitoring/errors.rst | 2 +-
.../modules_management.rst | 2 +-
.../priority-weight.rst | 2 +-
.../production-deployment.rst | 10 +-
.../administration-and-deployment/scheduler.rst | 6 +-
.../task-and-asset-state-store.rst | 8 +-
.../docs/authoring-and-scheduling/assets.rst | 16 +-
.../docs/authoring-and-scheduling/connections.rst | 2 +-
.../docs/authoring-and-scheduling/cron.rst | 2 +-
.../docs/authoring-and-scheduling/deferring.rst | 2 +-
.../dynamic-task-mapping.rst | 8 +-
.../authoring-and-scheduling/language-sdks/go.rst | 22 +-
.../language-sdks/typescript.rst | 135 +-
.../docs/authoring-and-scheduling/serializers.rst | 4 +-
.../docs/authoring-and-scheduling/timetable.rst | 8 +-
airflow-core/docs/best-practices.rst | 8 +-
airflow-core/docs/core-concepts/backfill.rst | 2 +-
airflow-core/docs/core-concepts/dag-run.rst | 2 +-
airflow-core/docs/core-concepts/dags.rst | 4 +-
airflow-core/docs/core-concepts/debug.rst | 2 +-
airflow-core/docs/core-concepts/params.rst | 7 +-
.../docs/core-concepts/resumable-tasks.rst | 39 +
airflow-core/docs/core-concepts/sensors.rst | 2 +-
.../docs/core-concepts/task-state-store.rst | 2 +-
airflow-core/docs/core-concepts/taskflow.rst | 4 +-
airflow-core/docs/core-concepts/tasks.rst | 8 +
airflow-core/docs/core-concepts/xcoms.rst | 2 +-
airflow-core/docs/extra-packages-ref.rst | 2 +-
airflow-core/docs/faq.rst | 10 +-
airflow-core/docs/howto/connection.rst | 2 +-
airflow-core/docs/howto/custom-operator.rst | 2 +-
airflow-core/docs/howto/custom-view-plugin.rst | 2 +-
airflow-core/docs/howto/define-extra-link.rst | 2 +-
airflow-core/docs/howto/docker-compose/index.rst | 4 +-
airflow-core/docs/howto/export-more-env-vars.rst | 2 +-
airflow-core/docs/howto/memory-profiling.rst | 2 +-
airflow-core/docs/howto/run-behind-proxy.rst | 2 +-
airflow-core/docs/howto/set-config.rst | 2 +-
airflow-core/docs/howto/set-up-database.rst | 4 +-
airflow-core/docs/howto/setup-and-teardown.rst | 4 +-
airflow-core/docs/howto/sla-to-deadlines.rst | 2 +-
airflow-core/docs/howto/timetable.rst | 6 +-
airflow-core/docs/installation/dependencies.rst | 14 +-
airflow-core/docs/installation/index.rst | 4 +-
.../docs/installation/installing-from-sources.rst | 10 +-
airflow-core/docs/installation/upgrading.rst | 2 +-
.../docs/installation/upgrading_to_airflow3.rst | 2 +-
airflow-core/docs/public-airflow-interface.rst | 7 +-
airflow-core/docs/security/audit_logs.rst | 14 +-
airflow-core/docs/security/kerberos.rst | 4 +-
airflow-core/docs/security/security_model.rst | 28 +-
airflow-core/docs/security/sql.rst | 2 +-
.../vulnerabilities-in-3rd-party-dependencies.rst | 2 +-
airflow-core/docs/start.rst | 2 +-
airflow-core/docs/templates-ref.rst | 10 +-
airflow-core/docs/tutorial/fundamentals.rst | 2 +-
airflow-core/docs/tutorial/hitl.rst | 4 +-
airflow-core/docs/ui.rst | 4 +-
.../src/airflow/api/common/airflow_health.py | 174 ++-
.../auth/managers/simple/ui/package.json | 22 +-
.../auth/managers/simple/ui/pnpm-lock.yaml | 520 +++---
.../airflow/api_fastapi/common/asset_expression.py | 62 +
.../src/airflow/api_fastapi/common/db/common.py | 2 +-
.../api_fastapi/core_api/datamodels/common.py | 9 +-
.../api_fastapi/core_api/datamodels/monitor.py | 17 +-
.../api_fastapi/core_api/openapi/_private_ui.yaml | 23 +-
.../core_api/openapi/v2-rest-api-generated.yaml | 78 +-
.../core_api/routes/public/connections.py | 6 +-
.../api_fastapi/core_api/routes/ui/assets.py | 12 +-
.../execution_api/routes/task_instances.py | 32 +-
.../src/airflow/config_templates/config.yml | 7 +-
.../src/airflow/dag_processing/bundles/base.py | 4 +-
.../src/airflow/dag_processing/bundles/manager.py | 36 +-
airflow-core/src/airflow/dag_processing/manager.py | 38 +-
airflow-core/src/airflow/settings.py | 34 +-
.../airflow/ui/openapi-gen/requests/schemas.gen.ts | 117 +-
.../airflow/ui/openapi-gen/requests/types.gen.ts | 36 +-
.../airflow/ui/public/i18n/locales/ko/browse.json | 16 +
.../airflow/ui/public/i18n/locales/ko/common.json | 5 +
.../ui/public/i18n/locales/ko/components.json | 4 +
.../airflow/ui/public/i18n/locales/ko/dags.json | 18 +
.../ui/public/i18n/locales/zh-CN/admin.json | 19 +-
.../ui/public/i18n/locales/zh-CN/assets.json | 21 +-
.../ui/public/i18n/locales/zh-CN/browse.json | 18 +
.../ui/public/i18n/locales/zh-CN/common.json | 169 +-
.../ui/public/i18n/locales/zh-CN/components.json | 31 +-
.../airflow/ui/public/i18n/locales/zh-CN/dag.json | 23 +-
.../airflow/ui/public/i18n/locales/zh-CN/dags.json | 53 +-
.../ui/public/i18n/locales/zh-CN/dashboard.json | 22 +-
.../airflow/ui/public/i18n/locales/zh-CN/hitl.json | 14 +
.../FilterBar/filters/DateRangeFilter.test.tsx | 27 +
.../ui/src/components/NeedsReviewButton.tsx | 2 +-
.../src/airflow/ui/src/hooks/useDateRangeFilter.ts | 10 +-
.../src/airflow/ui/src/layouts/Nav/Nav.tsx | 4 +-
.../src/airflow/ui/src/layouts/StorageLayout.tsx | 2 +-
.../ui/src/pages/TaskInstance/TaskInstance.tsx | 4 +-
airflow-core/src/airflow/ui/src/router.tsx | 2 +-
airflow-core/src/airflow/utils/db_cleanup.py | 9 +-
.../tests/unit/always/test_project_structure.py | 1 -
.../tests/unit/api/common/test_airflow_health.py | 361 +++--
.../api_fastapi/common/test_asset_expression.py | 73 +
.../api_fastapi/core_api/datamodels/test_common.py | 14 +-
.../core_api/routes/public/test_connections.py | 31 +
.../api_fastapi/core_api/routes/ui/test_assets.py | 48 +-
.../versions/head/test_task_instances.py | 71 +-
airflow-core/tests/unit/core/test_settings.py | 57 +-
.../bundles/test_dag_bundle_manager.py | 40 +-
.../tests/unit/dag_processing/test_manager.py | 300 +++-
airflow-ctl/docs/cli-and-env-variables-ref.rst | 2 +-
airflow-ctl/docs/howto/index.rst | 4 +-
airflow-ctl/docs/installation/index.rst | 2 +-
.../docs/installation/installing-from-sources.rst | 14 +-
airflow-ctl/docs/installation/prerequisites.rst | 5 +-
.../airflowctl/api/datamodels/auth_generated.py | 2 +-
.../src/airflowctl/api/datamodels/generated.py | 61 +-
airflow-ctl/src/airflowctl/ctl/cli_config.py | 11 +
.../tests/airflow_ctl/ctl/test_cli_config.py | 37 +
.../tests/airflow_e2e_tests/conftest.py | 3 +-
...rrent_xcom.py => test_go_sdk_variable_write.py} | 54 +-
.../ts_sdk_tests/test_ts_sdk_dag.py | 102 +-
chart/docs/extending-the-chart.rst | 2 +-
chart/docs/manage-dag-files.rst | 2 +-
chart/docs/parameters-ref.rst | 2 +-
chart/docs/production-guide.rst | 12 +-
chart/docs/setting-resources-for-containers.rst | 2 +-
clients/python/CHANGELOG.md | 13 +
clients/python/version.txt | 2 +-
contributing-docs/30_new_language_sdk.rst | 5 +
dev/breeze/doc/ci/02_images.md | 2 +-
dev/breeze/pyproject.toml | 8 +-
.../src/airflow_breeze/commands/ci_commands.py | 46 +
.../src/airflow_breeze/commands/main_command.py | 3 +-
.../commands/release_management_commands.py | 4 +-
dev/breeze/src/airflow_breeze/global_constants.py | 2 +-
.../utils/constraints_version_check.py | 75 +-
dev/breeze/tests/test_constraints_version_check.py | 73 +-
dev/breeze/tests/test_remove_backport_labels.py | 85 +
dev/breeze/uv.lock | 128 +-
dev/skill-evals/last-eval-hash.txt | 2 +-
devel-common/pyproject.toml | 4 +-
docs/spelling_wordlist.txt | 9 +
generated/known_airflow_exceptions.txt | 2 +-
go-sdk/README.md | 1 +
go-sdk/airflow/context.go | 117 ++
go-sdk/airflow/context_test.go | 179 +++
go-sdk/airflow/doc.go | 41 +
go-sdk/bundle/bundlev1/registry.go | 17 +-
go-sdk/bundle/bundlev1/registry_test.go | 31 +-
go-sdk/bundle/bundlev1/task_test.go | 214 +--
go-sdk/capabilities.yaml | 3 +
.../cmd/airflow-go-pack/pack_integration_test.go | 3 +
go-sdk/dags/go_examples.py | 19 +-
.../bundle/concurrentxcom/concurrentxcom.go | 15 +-
.../bundle/concurrentxcom/concurrentxcom_test.go | 19 +-
go-sdk/example/bundle/main.go | 55 +-
go-sdk/example/bundle/main_test.go | 21 +-
.../bundle/taskflowbinding/taskflowbinding.go | 61 +-
.../bundle/taskflowbinding/taskflowbinding_test.go | 56 +-
.../example/bundle/variablewrite/variablewrite.go | 53 +
go-sdk/pkg/binding/binding.go | 133 +-
go-sdk/pkg/binding/binding_test.go | 382 +++--
go-sdk/pkg/execution/client.go | 19 +
go-sdk/pkg/execution/client_test.go | 113 ++
go-sdk/pkg/execution/integration_test.go | 117 +-
go-sdk/pkg/execution/task_runner.go | 11 +-
go-sdk/pkg/sdkcontext/keys.go | 20 +-
go-sdk/sdk/context.go | 49 +-
go-sdk/sdk/doc.go | 20 +-
go-sdk/sdk/sdk.go | 21 +-
java-sdk/README.md | 1 +
java-sdk/capabilities.yaml | 3 +
java-sdk/sdk/module.md | 1 +
kubernetes-tests/lang_sdk/go_example/main.go | 13 +-
performance/pyproject.toml | 2 +-
providers/airbyte/pyproject.toml | 2 +-
providers/akeyless/pyproject.toml | 2 +-
providers/alibaba/pyproject.toml | 2 +-
providers/amazon/pyproject.toml | 2 +-
.../providers/amazon/aws/executors/batch/utils.py | 1 +
.../providers/amazon/aws/log/s3_task_handler.py | 9 +
.../airflow/providers/amazon/aws/operators/emr.py | 19 +-
.../amazon/aws/sensors/opensearch_serverless.py | 3 +
.../amazon/aws/triggers/opensearch_serverless.py | 18 +-
.../providers/amazon/aws/triggers/sagemaker.py | 36 +
.../unit/amazon/aws/executors/batch/test_utils.py | 4 +-
.../unit/amazon/aws/log/test_s3_task_handler.py | 23 +
.../amazon/aws/operators/test_emr_serverless.py | 62 +-
.../aws/sensors/test_opensearch_serverless.py | 20 +-
.../aws/triggers/test_opensearch_serverless.py | 77 +-
.../unit/amazon/aws/triggers/test_sagemaker.py | 114 ++
providers/anthropic/pyproject.toml | 2 +-
.../airflow/providers/anthropic/hooks/anthropic.py | 16 +-
.../tests/unit/anthropic/hooks/test_anthropic.py | 63 +
providers/apache/arrow/pyproject.toml | 2 +-
providers/apache/beam/pyproject.toml | 2 +-
.../providers/apache/beam/operators/beam.py | 11 +-
.../tests/unit/apache/beam/operators/test_beam.py | 49 +-
providers/apache/cassandra/pyproject.toml | 2 +-
providers/apache/drill/pyproject.toml | 2 +-
providers/apache/druid/pyproject.toml | 2 +-
providers/apache/flink/pyproject.toml | 2 +-
providers/apache/hdfs/pyproject.toml | 2 +-
providers/apache/hive/pyproject.toml | 2 +-
.../tests/unit/apache/hive/plugins}/__init__.py | 0
.../unit/apache/hive/plugins/test_hive.py} | 11 +-
providers/apache/iceberg/pyproject.toml | 2 +-
providers/apache/impala/pyproject.toml | 2 +-
providers/apache/kafka/pyproject.toml | 2 +-
providers/apache/kylin/pyproject.toml | 2 +-
providers/apache/livy/pyproject.toml | 2 +-
providers/apache/pig/pyproject.toml | 2 +-
providers/apache/pinot/pyproject.toml | 2 +-
providers/apache/spark/pyproject.toml | 2 +-
providers/apache/tinkerpop/pyproject.toml | 2 +-
providers/apprise/pyproject.toml | 2 +-
providers/arangodb/pyproject.toml | 2 +-
providers/asana/pyproject.toml | 2 +-
providers/atlassian/jira/pyproject.toml | 2 +-
providers/celery/pyproject.toml | 2 +-
providers/clickhousedb/pyproject.toml | 2 +-
providers/cloudant/pyproject.toml | 2 +-
providers/cncf/kubernetes/docs/operators.rst | 17 +-
providers/cncf/kubernetes/pyproject.toml | 2 +-
.../providers/cncf/kubernetes/hooks/kubernetes.py | 63 +-
.../providers/cncf/kubernetes/triggers/pod.py | 21 +-
.../unit/cncf/kubernetes/hooks/test_kubernetes.py | 101 ++
.../unit/cncf/kubernetes/triggers/test_pod.py | 40 +-
providers/cohere/pyproject.toml | 2 +-
providers/common/ai/README.rst | 2 +
providers/common/ai/docs/changelog.rst | 11 +
providers/common/ai/docs/choosing_a_toolset.rst | 610 ++++++++
providers/common/ai/docs/classifier_models.rst | 189 +++
.../common/ai/docs/connections/pydantic_ai.rst | 20 +
.../ai/docs/connections/pydantic_ai_azure.rst | 24 +-
.../ai/docs/connections/pydantic_ai_bedrock.rst | 14 +
.../ai/docs/connections/pydantic_ai_vertex.rst | 24 +-
providers/common/ai/docs/examples.rst | 8 +
providers/common/ai/docs/index.rst | 22 +-
providers/common/ai/docs/operators/agent.rst | 16 +-
providers/common/ai/docs/operators/llm.rst | 68 +-
providers/common/ai/docs/operators/llm_branch.rst | 200 ++-
.../common/ai/docs/operators/llm_file_analysis.rst | 6 +-
.../ai/docs/operators/llm_schema_compare.rst | 24 +-
providers/common/ai/docs/operators/llm_sql.rst | 9 +-
providers/common/ai/docs/provider_fallback.rst | 231 +++
providers/common/ai/docs/retry_policies.rst | 36 +
providers/common/ai/docs/sandbox.rst | 821 ++++++++++
providers/common/ai/docs/toolsets.rst | 335 +---
providers/common/ai/provider.yaml | 39 +
providers/common/ai/pyproject.toml | 14 +
.../airflow/providers/common/ai/decorators/llm.py | 2 +-
.../providers/common/ai/decorators/llm_branch.py | 2 +-
.../ai/example_dags/example_classifier_model.py | 134 ++
.../common/ai/example_dags/example_llm_branch.py | 94 +-
.../common/ai/example_dags/example_llm_fallback.py | 101 ++
.../ai/example_dags/example_llm_schema_compare.py | 7 +-
.../ai/example_dags/example_sandbox_toolset.py | 282 ++++
.../src/airflow/providers/common/ai/exceptions.py | 10 +
.../providers/common/ai/get_provider_info.py | 27 +-
.../providers/common/ai/hooks/pydantic_ai.py | 403 ++++-
.../airflow/providers/common/ai/mixins/approval.py | 34 +-
.../airflow/providers/common/ai/operators/agent.py | 11 +
.../airflow/providers/common/ai/operators/llm.py | 210 ++-
.../providers/common/ai/operators/llm_branch.py | 228 ++-
.../common/ai/operators/llm_file_analysis.py | 14 +-
.../common/ai/operators/llm_schema_compare.py | 24 +-
.../providers/common/ai/operators/llm_sql.py | 24 +-
.../providers/common/ai/policies/decision.py | 110 ++
.../providers/common/ai/sandbox/__init__.py | 14 +
.../airflow/providers/common/ai/sandbox/base.py | 21 +-
.../airflow/providers/common/ai/sandbox/modal.py | 868 ++++++++++
.../src/airflow/providers/common/ai/sandbox/sbx.py | 19 +-
.../providers/common/ai/toolsets/sandbox.py | 52 +-
.../airflow/providers/common/ai/utils/decision.py | 271 ++++
.../common/ai/example_sandbox_toolset_modal.py | 258 +++
.../ai/tests/unit/common/ai/decorators/test_llm.py | 19 +
.../tests/unit/common/ai/hooks/test_pydantic_ai.py | 897 ++++++++++-
.../tests/unit/common/ai/mixins/test_approval.py | 19 +
.../tests/unit/common/ai/operators/test_agent.py | 47 +-
.../ai/tests/unit/common/ai/operators/test_llm.py | 417 ++++-
.../unit/common/ai/operators/test_llm_branch.py | 771 ++++++++-
.../common/ai/operators/test_llm_file_analysis.py | 1 +
.../tests/unit/common/ai/operators/test_llm_sql.py | 31 +
.../tests/unit/common/ai/policies/test_decision.py | 90 ++
.../ai/tests/unit/common/ai/sandbox/fake_modal.py | 322 ++++
.../ai/tests/unit/common/ai/sandbox/test_base.py | 17 +
.../ai/tests/unit/common/ai/sandbox/test_modal.py | 1120 +++++++++++++
.../tests/unit/common/ai/toolsets/test_sandbox.py | 107 ++
.../ai/tests/unit/common/ai/utils/test_decision.py | 250 +++
.../ai/tests/unit/common/ai/utils/test_logging.py | 29 +
providers/common/compat/pyproject.toml | 2 +-
providers/common/dataquality/pyproject.toml | 2 +-
providers/common/io/docs/xcom_backend.rst | 2 +-
providers/common/io/pyproject.toml | 2 +-
providers/common/messaging/pyproject.toml | 2 +-
providers/common/sql/docs/operators.rst | 23 +-
providers/common/sql/pyproject.toml | 2 +-
.../sql/src/airflow/providers/common/sql/config.py | 19 +-
.../providers/common/sql/datafusion/engine.py | 27 +
.../sql/datafusion/object_storage_provider.py | 61 +-
.../common/sql/example_dags/example_analytics.py | 13 +
.../unit/common/sql/datafusion/test_engine.py | 93 +-
.../common/sql/datafusion/test_format_handlers.py | 2 +-
.../sql/datafusion/test_object_storage_provider.py | 91 ++
.../sql/tests/unit/common/sql/test_config.py | 29 +
.../databricks/docs/operators/jobs_create.rst | 5 +-
providers/databricks/docs/operators/run_now.rst | 5 +-
providers/databricks/docs/operators/submit_run.rst | 4 +
providers/databricks/pyproject.toml | 2 +-
.../providers/databricks/operators/databricks.py | 54 +-
.../unit/databricks/operators/test_databricks.py | 196 +++
providers/datadog/pyproject.toml | 2 +-
providers/dbt/cloud/pyproject.toml | 2 +-
providers/dingding/pyproject.toml | 2 +-
providers/discord/pyproject.toml | 2 +-
providers/docker/pyproject.toml | 2 +-
providers/duckdb/pyproject.toml | 2 +-
providers/edge3/docs/changelog.rst | 6 +
.../providers/edge3/executors/edge_executor.py | 104 +-
.../src/airflow/providers/edge3/models/edge_job.py | 31 +-
.../src/airflow/providers/edge3/models/types.py | 19 +
.../unit/edge3/executors/test_edge_executor.py | 218 ++-
.../edge3/tests/unit/edge3/models/test_edge_job.py | 29 +-
providers/elasticsearch/pyproject.toml | 2 +-
providers/exasol/pyproject.toml | 2 +-
providers/facebook/pyproject.toml | 2 +-
providers/ftp/pyproject.toml | 2 +-
providers/git/pyproject.toml | 2 +-
providers/github/pyproject.toml | 2 +-
providers/google/docs/connections/gcp.rst | 86 +
providers/google/pyproject.toml | 2 +-
.../providers/google/cloud/hooks/stackdriver.py | 2 +-
.../providers/google/cloud/log/gcs_task_handler.py | 9 +
.../providers/google/cloud/operators/gcs.py | 10 +-
.../providers/google/cloud/operators/gen_ai.py | 29 +-
.../google/cloud/transfers/http_to_gcs.py | 2 +
.../providers/google/cloud/triggers/dataproc.py | 70 +-
.../google/cloud/utils/credentials_provider.py | 7 +-
.../airflow/providers/google/common/deprecated.py | 29 +-
.../airflow/providers/google/suite/hooks/drive.py | 21 +-
.../unit/google/cloud/hooks/test_stackdriver.py | 24 +
.../unit/google/cloud/log/test_gcs_task_handler.py | 23 +
.../tests/unit/google/cloud/operators/test_gcs.py | 55 +
.../unit/google/cloud/operators/test_gen_ai.py | 162 ++
.../google/cloud/transfers/test_http_to_gcs.py | 22 +
.../unit/google/cloud/triggers/test_dataproc.py | 230 +++
.../cloud/utils/test_credentials_provider.py | 17 +-
.../tests/unit/google/common/test_deprecated.py | 60 +-
.../tests/unit/google/suite/hooks/test_drive.py | 70 +-
providers/grpc/pyproject.toml | 2 +-
providers/hashicorp/pyproject.toml | 2 +-
providers/http/pyproject.toml | 2 +-
.../http/src/airflow/providers/http/hooks/http.py | 2 +-
providers/http/tests/unit/http/hooks/test_http.py | 24 +
providers/ibm/db2/pyproject.toml | 2 +-
providers/ibm/mq/pyproject.toml | 2 +-
providers/imap/pyproject.toml | 2 +-
providers/influxdb/pyproject.toml | 2 +-
providers/informatica/pyproject.toml | 2 +-
providers/jdbc/pyproject.toml | 2 +-
providers/jenkins/pyproject.toml | 2 +-
providers/keycloak/pyproject.toml | 2 +-
providers/microsoft/azure/pyproject.toml | 2 +-
.../microsoft/azure/log/wasb_task_handler.py | 9 +
.../microsoft/azure/log/test_wasb_task_handler.py | 28 +
providers/microsoft/mssql/pyproject.toml | 2 +-
providers/microsoft/psrp/pyproject.toml | 2 +-
providers/microsoft/winrm/pyproject.toml | 2 +-
providers/mongo/pyproject.toml | 2 +-
providers/mysql/pyproject.toml | 2 +-
providers/neo4j/pyproject.toml | 2 +-
providers/odbc/pyproject.toml | 2 +-
providers/openai/pyproject.toml | 2 +-
providers/openfaas/pyproject.toml | 2 +-
providers/openlineage/pyproject.toml | 2 +-
providers/opensearch/pyproject.toml | 2 +-
providers/opsgenie/pyproject.toml | 2 +-
providers/oracle/pyproject.toml | 2 +-
providers/pagerduty/pyproject.toml | 2 +-
providers/papermill/pyproject.toml | 2 +-
providers/pgvector/pyproject.toml | 2 +-
providers/pinecone/pyproject.toml | 2 +-
providers/postgres/pyproject.toml | 2 +-
providers/presto/pyproject.toml | 2 +-
providers/qdrant/pyproject.toml | 2 +-
providers/redis/pyproject.toml | 2 +-
providers/salesforce/pyproject.toml | 2 +-
providers/samba/pyproject.toml | 2 +-
providers/segment/pyproject.toml | 2 +-
providers/sendgrid/pyproject.toml | 2 +-
providers/sftp/pyproject.toml | 2 +-
providers/singularity/pyproject.toml | 2 +-
providers/slack/pyproject.toml | 2 +-
providers/smtp/pyproject.toml | 2 +-
providers/snowflake/docs/operators/snowflake.rst | 36 +
providers/snowflake/pyproject.toml | 2 +-
.../snowflake/hooks/snowflake_cortex_agent.py | 184 ++-
.../providers/snowflake/operators/snowflake.py | 61 +-
.../snowflake/operators/snowpark_containers.py | 1 +
.../system/snowflake/example_snowflake_notebook.py | 75 +
.../snowflake/hooks/test_snowflake_cortex_agent.py | 237 ++-
.../unit/snowflake/operators/test_snowflake.py | 334 ++++
.../operators/test_snowpark_containers.py | 7 +-
providers/sqlite/pyproject.toml | 2 +-
providers/ssh/pyproject.toml | 2 +-
providers/standard/docs/sensors/datetime.rst | 2 +-
providers/standard/pyproject.toml | 2 +-
.../providers/standard/operators/trigger_dagrun.py | 2 +-
.../providers/standard/sensors/date_time.py | 11 +
.../providers/standard/sensors/filesystem.py | 2 +-
.../unit/standard/operators/test_trigger_dagrun.py | 4 +-
.../tests/unit/standard/sensors/test_date_time.py | 12 +
.../tests/unit/standard/sensors/test_filesystem.py | 14 +-
providers/tableau/pyproject.toml | 2 +-
providers/telegram/pyproject.toml | 2 +-
providers/teradata/pyproject.toml | 2 +-
.../teradata/transfers/azure_blob_to_teradata.py | 14 +
.../providers/teradata/transfers/s3_to_teradata.py | 15 +
.../teradata/triggers/teradata_compute_cluster.py | 2 -
.../airflow/providers/teradata/utils/constants.py | 1 -
.../transfers/test_azure_blob_to_teradata.py | 48 +
.../unit/teradata/transfers/test_s3_to_teradata.py | 83 +
.../triggers/test_teradata_compute_cluster.py | 16 +
providers/trino/pyproject.toml | 2 +-
providers/vertica/pyproject.toml | 2 +-
providers/vespa/pyproject.toml | 2 +-
providers/weaviate/pyproject.toml | 2 +-
providers/yandex/pyproject.toml | 2 +-
providers/ydb/pyproject.toml | 2 +-
providers/zendesk/pyproject.toml | 2 +-
pyproject.toml | 10 +-
scripts/ci/prek/check_extra_packages_ref.py | 2 +-
scripts/ci/prek/check_imports_in_providers.py | 2 +-
scripts/ci/prek/lang_sdk_compat_matrix.py | 1 +
scripts/ci/prek/ruff_format.py | 2 +-
.../ci/prek/validate_operators_init_exemptions.txt | 1 -
scripts/tools/setup_breeze | 2 +-
task-sdk/docs/deferred-vs-async-operators.rst | 2 +-
task-sdk/docs/examples.rst | 2 +-
task-sdk/docs/executable-bundle-spec.rst | 1 +
task-sdk/docs/ts-bundle-spec.rst | 169 +-
.../src/airflow/sdk/api/datamodels/_generated.py | 2 +-
.../airflow/sdk/coordinators/_bundle_metadata.py | 58 +-
.../sdk/coordinators/node/_bundle_reader.py | 99 +-
.../airflow/sdk/coordinators/node/coordinator.py | 22 +-
.../coordinators/node/_bundle_test_utils.py | 28 +-
.../coordinators/node/test_bundle_reader.py | 210 ++-
.../task_sdk/coordinators/node/test_coordinator.py | 82 +-
ts-sdk/.prettierignore | 3 +
ts-sdk/README.md | 144 +-
ts-sdk/adr/0001-mixed-lang-dag-interface.md | 8 +-
ts-sdk/adr/0002-native-dag-interface.md | 3 +-
ts-sdk/api-docs/dag-authoring-api.ts | 3 +-
ts-sdk/capabilities.yaml | 8 +-
ts-sdk/docs/index.md | 18 +
ts-sdk/example/README.md | 9 +-
ts-sdk/example/dags/typescript_example.py | 5 +
ts-sdk/example/dags/typescript_taskflow_example.py | 30 +-
ts-sdk/example/src/main.ts | 20 +-
ts-sdk/example/src/taskflow.ts | 93 +-
ts-sdk/src/cli/bundle-encoder.ts | 72 +-
ts-sdk/src/cli/pack.ts | 82 +-
ts-sdk/src/cli/validate.ts | 6 +-
ts-sdk/src/coordinator/arg-binding.ts | 299 ++++
ts-sdk/src/coordinator/client.ts | 100 +-
ts-sdk/src/coordinator/manifest.ts | 17 +-
ts-sdk/src/coordinator/protocol.ts | 2 +
ts-sdk/src/coordinator/runtime.ts | 28 +-
ts-sdk/src/index.ts | 2 +
ts-sdk/src/sdk/arg-names.ts | 112 ++
ts-sdk/src/sdk/bundle.ts | 5 +-
ts-sdk/src/sdk/client.ts | 18 +
ts-sdk/src/sdk/dag.ts | 4 +-
ts-sdk/src/sdk/task-handler.ts | 12 +-
ts-sdk/src/sdk/task.ts | 22 +-
ts-sdk/tests/cli/fixtures/bundle-v1.min.mjs | 18 +
ts-sdk/tests/cli/fixtures/bundle-v1.mjs | 24 -
ts-sdk/tests/cli/pack.test.ts | 234 ++-
ts-sdk/tests/cli/validate.test.ts | 4 +-
ts-sdk/tests/coordinator/arg-binding.test.ts | 505 ++++++
ts-sdk/tests/coordinator/client.test.ts | 93 +-
ts-sdk/tests/coordinator/integration.test.ts | 199 +++
ts-sdk/tests/coordinator/runtime-manifest.test.ts | 14 +-
ts-sdk/tests/public-api.test.ts | 64 +-
ts-sdk/tests/sdk/arg-names.test.ts | 112 ++
uv.lock | 1653 +++++++++++---------
515 files changed, 21252 insertions(+), 3675 deletions(-)
create mode 100644
airflow-core/src/airflow/api_fastapi/common/asset_expression.py
create mode 100644
airflow-core/tests/unit/api_fastapi/common/test_asset_expression.py
copy
airflow-e2e-tests/tests/airflow_e2e_tests/go_sdk_tests/{test_go_sdk_concurrent_xcom.py
=> test_go_sdk_variable_write.py} (51%)
create mode 100644 dev/breeze/tests/test_remove_backport_labels.py
create mode 100644 go-sdk/airflow/context.go
create mode 100644 go-sdk/airflow/context_test.go
create mode 100644 go-sdk/airflow/doc.go
create mode 100644 go-sdk/example/bundle/variablewrite/variablewrite.go
copy {airflow-core/src/airflow/_shared =>
providers/apache/hive/tests/unit/apache/hive/plugins}/__init__.py (100%)
copy providers/apache/hive/{src/airflow/providers/apache/hive/plugins/hive.py
=> tests/unit/apache/hive/plugins/test_hive.py} (78%)
create mode 100644 providers/common/ai/docs/choosing_a_toolset.rst
create mode 100644 providers/common/ai/docs/classifier_models.rst
create mode 100644 providers/common/ai/docs/provider_fallback.rst
create mode 100644 providers/common/ai/docs/sandbox.rst
create mode 100644
providers/common/ai/src/airflow/providers/common/ai/example_dags/example_classifier_model.py
create mode 100644
providers/common/ai/src/airflow/providers/common/ai/example_dags/example_llm_fallback.py
create mode 100644
providers/common/ai/src/airflow/providers/common/ai/example_dags/example_sandbox_toolset.py
create mode 100644
providers/common/ai/src/airflow/providers/common/ai/policies/decision.py
create mode 100644
providers/common/ai/src/airflow/providers/common/ai/sandbox/modal.py
create mode 100644
providers/common/ai/src/airflow/providers/common/ai/utils/decision.py
create mode 100644
providers/common/ai/tests/system/common/ai/example_sandbox_toolset_modal.py
create mode 100644
providers/common/ai/tests/unit/common/ai/policies/test_decision.py
create mode 100644
providers/common/ai/tests/unit/common/ai/sandbox/fake_modal.py
create mode 100644
providers/common/ai/tests/unit/common/ai/sandbox/test_modal.py
create mode 100644
providers/common/ai/tests/unit/common/ai/utils/test_decision.py
create mode 100644
providers/snowflake/tests/system/snowflake/example_snowflake_notebook.py
create mode 100644 ts-sdk/src/coordinator/arg-binding.ts
create mode 100644 ts-sdk/src/sdk/arg-names.ts
create mode 100644 ts-sdk/tests/cli/fixtures/bundle-v1.min.mjs
delete mode 100644 ts-sdk/tests/cli/fixtures/bundle-v1.mjs
create mode 100644 ts-sdk/tests/coordinator/arg-binding.test.ts
create mode 100644 ts-sdk/tests/sdk/arg-names.test.ts
