This is an automated email from the ASF dual-hosted git repository.
ephraimbuddy pushed a change to branch dag-processor-activation
in repository https://gitbox.apache.org/repos/asf/airflow.git
discard fa348acf4dc Release Dag processor claims for unpublishable sources
discard 6c68ab0a8a8 Separate trusted Dag bundle catalog provisioning from
processing
discard 961d7a8f2f5 Restore isolated Dag processor regression coverage
discard a5cfaf43112 Keep API Dag processors recoverable through delivery
failures
discard 7ab82fbeeb3 Move API work delivery state out of the Dag processor
manager
discard d3ef7048e43 Allow independent priority parses to finish in either
order in recovery tests
discard 3ff4451edc1 Preserve requested Dag processing work during bundle
failures
discard c510658194b Wait for fresh inventories before running priority parses
discard 71d44886f23 Preserve Dag processor work across retries and reparses
discard 26da5622257 Run Dag processors without metadata database access
discard 820006c6f5d Filter invalid Dag discovery paths and deduplicate parse
sources
discard ae49e8bcafe Recover failed Dag parses and bound inventory publication
discard e8afa30fefd Introduce a source interface for requested Dag processing
work
discard b4e013cedc7 Avoid database sessions when collecting API-mode Dag
results
discard 65278d51030 Back off inventory retries and report results the API
would refuse
discard 4ea6bed7994 Publish Dag processor results through the Execution API
discard 1e44909d9d6 Carry publication receipts through Dag processor
maintenance
discard 9e55df8b2ff Preserve Dag cleanup and exact priority reparse targets
discard 81899d430c6 Fix Dag processor requested-work cleanup and delivery
discard 7e856b14e2d Adapt legacy Dag upgrade regression to bundle discovery
results
discard 46e06da0ff5 Align Dag processor claim migrations with main
discard 05f111ca978 Test legacy Dag deactivation after bundle-name backfill
discard 68761fb0f39 Fix Dag processor work retries and metadata cleanup
discard 1ba66079796 Retain Dag processor work until delivery is acknowledged
discard fa11b277fce Preserve compact source publications and actionable Dag
diagnostics
discard 6651abba6f3 Clean asset references for stale Dags removed from bundles
discard fd008778841 Align API-mode Dag publication with DB-mode cleanup
discard 3a3614205bc Allow Dag processors to publish metadata through the
Execution API
discard 8e824a66c34 Keep Dag parsing and metrics healthy during bundle outages
discard 821762e750a Revise Dag bundle refresh compatibility guidance
discard d4a70c2505b Clarify bundle refresh compatibility and simplify its
regression test
discard 5f5cbcc4c82 Keep Git bundles importable on Airflow 3.4
discard b0b605f2f39 Keep Dag parsing responsive during bundle refresh failures
discard e59ac582fe0 Make the bundle refresh regression test independent of
uptime
discard 2a218d1983d Avoid parsing stalls while a bundle refresh waits for
imports
discard 4cc0bfc41ff Keep Dag parsing results aligned with refreshed bundle
sources
add d8532e28ac6 Remove duplicate Keycloak token-binding test (#75882)
add af2c3efed7b Remove unused FAB test log assertion helpers (#75876)
add 107c2c0a447 Remove obsolete SQLAlchemy 1 driver compatibility check
(#75892)
add c84db2fa3a2 Fix async SmtpHook connections failing when STARTTLS is
enabled (#75529)
add 3722c7f84d7 Add unit tests for auth manager login datamodels (#72265)
(#75544)
add d5ca289f414 Add CLI login discovery to auth managers (#75672)
add 71bdce5fa51 Run lang-SDK serialization conformance checks in CI
instead of prek (#75872)
add 2c560aad523 Let a generic pydanticai connection set a default model
provider (#75909)
add eeae004bd25 Keep release-candidate provider pins until each is
released or yanked (#75919)
add aec750da2e4 Remove duplicate GCS wildcard transfer test (#75897)
add 5376be8924c Remove duplicate Calendar-to-GCS transfer test (#75895)
add 0a07c9ac1f1 Remove unused milestone selection helpers (#75848)
add f7abf6fec2d Remove unused decorator default comparison helper (#75849)
add a7e1de351a4 Remove unused HITL browser workflow helpers (#75851)
add ef1162a158b Remove duplicate default Kerberos chart tests (#75878)
add 8bf71ce1069 Parametrize Spark sensor waiting-state tests (#75879)
add 50189f81dfe Remove unused operator-helper test scaffolding (#75880)
add 983b387136e Remove unused S3 bundle test upload helper (#75888)
add 6edfe624bc9 Remove duplicate Teradata suspend-success test (#75881)
add 6056deb572e Remove unused CLI process test helpers (#75883)
add 5f337adeb19 Remove unused API and SDK test helpers (#75884)
add 7e1a4504fcc Remove unused Sphinx snippet renderer (#75886)
add d72dd42520a Remove orphaned model, asset and timetable test helpers
(#75887)
add 69b1ce81da3 Remove duplicate Databricks retry test (#75889)
add 67301393a50 Remove unrequested core test fixtures (#75890)
add 1387cd96177 Remove duplicate Databricks timeout retry test (#75891)
add 99bd160d063 Remove empty Databricks mixin test placeholders (#75893)
add 652da05dc1e Remove duplicate task-instance 404 tests (#75894)
add eb4803e6d63 Remove duplicate HTTP header override test (#75899)
add 7e6bccecd8e Parametrize airflowctl environment output tests (#75900)
add 6061e081f6c Remove duplicate SFTP missing-file sensor test (#75901)
add c6690015a28 Parametrize airflowctl partition-date tests (#75902)
add ddb89d5e006 Run the slow root prek checks in parallel (#75437)
add a211be1c115 Fix MySQL HITL responded-by filters (#67323)
add 24f33707582 Add query and report inspection operators for Google Bid
Manager (#71947)
add 6a8e8642939 Don't force dags reserialize in e2e startup (races
dag-processor) (#75739)
add dac3da4ccfd Add missing tests for google.cloud.utils BigQuery helpers
(#72276) (#72448)
add d561f37ea54 Fix OpenLineage task events lost for Dags with mapped
tasks (#75860)
add d9ca6241562 Match file datasets in exclude_datasets without a doubled
slash (#75921)
add 20e4e570971 Avoid unbounded IN lists for queued asset events (#75943)
add fa160f83538 Fix dashboard Active Dags filtering (#75350)
add eafb6ed8a7d Reject non-object JSON files when importing variables
(#75384)
add e0a95793c95 Preserve XCom value types when editing in the UI (#75496)
add 226de8a1911 Publish usage, run id and model name from LLMOperator's
subclasses (#75208)
add b394b884046 Add OpenLineage system tests for the emission policy
(#75678)
add 1040a319e0e Fix the durable MSGraph and PowerBI operators from
post-merge review (#75272)
add 6f0a4b5d519 Fix hourly cron schedules running an hour off across a DST
change (#75847)
add fb6534f1db4 Prepare deferred providers RC2 for 2026-10-29 (#75947)
add 97d4d4e2aa9 Skip log accumulator disk spill when reading finished task
logs (#70671)
add 7468d3145cc Show UI doc screenshots in synced dark/light tabs (#75957)
add cd1ed360b9a Refresh Dag cards after deleting Dag runs (#75931)
add 040173d818a Refresh the Magpie reconciliation stamp (#75952)
add 34ad14c27b3 i18n(ko): add missing translations(Oct 27) (#75740)
add 9b37c269b4a Use only file:// URIs in the OpenLineage exclude_datasets
authoring system test (#75961)
add f605f1040f3 UI: Fix the Dag filters test that fails lint and vitest
(#75963)
add 5761c2ac1ea Remove the unused React SWC plugin from the UI (#75875)
add 90628f4fbdc Preserve task instance bulk selections across pages
(#75905)
add 63c8c9bb9a8 Keep Dag parsing results aligned with refreshed bundle
sources
add 8bb1297d4ba Avoid parsing stalls while a bundle refresh waits for
imports
add 73a139115df Make the bundle refresh regression test independent of
uptime
add 040e0c587b6 Keep Dag parsing responsive during bundle refresh failures
add f11a0cc5b37 Keep Git bundles importable on Airflow 3.4
add 36be438af12 Clarify bundle refresh compatibility and simplify its
regression test
add 78401bac6a7 Revise Dag bundle refresh compatibility guidance
add 509cb5dcaa0 Keep Dag parsing and metrics healthy during bundle outages
add df5919166b7 Stop repeated bundle drains and keep cleanup overrides
working
add ef49963c778 Describe the git provider's refresh-error probe accurately
add 9edca876774 Allow Dag processors to publish metadata through the
Execution API
add eb57eb39899 Align API-mode Dag publication with DB-mode cleanup
add 8c5e46d88f9 Clean asset references for stale Dags removed from bundles
add 8d08c505ccd Preserve compact source publications and actionable Dag
diagnostics
add 29254b8b730 Reject Dags without a timetable or with incomplete
deadline alerts
add 1232175f661 Cover publication API rejections and clarify its docs and
names
add 3293711654c Retain Dag processor work until delivery is acknowledged
add 82d25c08cb2 Fix Dag processor work retries and metadata cleanup
add a8876fcf8d6 Test legacy Dag deactivation after bundle-name backfill
add d013c6fc32f Align Dag processor claim migrations with main
add 22dc859ffe9 Adapt legacy Dag upgrade regression to bundle discovery
results
add 8e4cd50bd33 Fix Dag processor requested-work cleanup and delivery
add 34e2f9dee76 Preserve Dag cleanup and exact priority reparse targets
add 703c4482689 Carry publication receipts through Dag processor
maintenance
add 322fdd84fe6 Fix Dag processor maintenance logs and stale Dag warning
cleanup
add ed8e8b74bf0 Tighten test coverage of Dag processor requested-work
delivery
add e65c037b002 Publish Dag processor results through the Execution API
add e42f1ccd785 Back off inventory retries and report results the API
would refuse
add 41d0c607e0a Avoid database sessions when collecting API-mode Dag
results
add 3ee3f7757bf Introduce a source interface for requested Dag processing
work
add b10fa205aa4 Recover failed Dag parses and bound inventory publication
add b947711337d Filter invalid Dag discovery paths and deduplicate parse
sources
add e333564e330 Handle Execution API failures in the API-mode Dag processor
add f16729c2982 Close gaps in the Dag processor's API-mode tests
add 2858274818e Run Dag processors without metadata database access
add 55f5549dc51 Preserve Dag processor work across retries and reparses
add 7bd497f4f66 Wait for fresh inventories before running priority parses
add 4ba83a1ffdf Preserve requested Dag processing work during bundle
failures
add f5460ae50eb Allow independent priority parses to finish in either
order in recovery tests
add abb40fad8a3 Move API work delivery state out of the Dag processor
manager
add be9609e3a67 Keep API Dag processors recoverable through delivery
failures
add 3a8b420ab70 Restore isolated Dag processor regression coverage
add 2a339ec8250 Separate trusted Dag bundle catalog provisioning from
processing
add 5ce0f9aa347 Release Dag processor claims for unpublishable sources
add f713f5e1702 Fix Dag processor API-mode shutdown, completion and claim
capacity
add 82ef8a0d664 Correct the Dag processor API-mode docs and Compose example
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (fa348acf4dc)
\
N -- N -- N refs/heads/dag-processor-activation (82ef8a0d664)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.apache-magpie.lock | 6 +-
.github/workflows/ci-amd.yml | 58 ++
.github/workflows/ci-arm.yml | 58 ++
.github/workflows/generate-constraints.yml | 14 +-
.github/workflows/release-constraints.yml | 3 +
.pre-commit-config.yaml | 106 ++-
.../administration-and-deployment/dag-bundles.rst | 14 +-
.../administration-and-deployment/listeners.rst | 6 +
.../docs/core-concepts/auth-manager/index.rst | 25 +
airflow-core/docs/security/api_permissions_ref.rst | 4 +
.../security/dag-processor-api/docker-compose.yaml | 4 +
.../docs/security/jwt_token_authentication.rst | 31 +-
airflow-core/docs/ui.rst | 432 ++++++++----
.../api_fastapi/auth/managers/base_auth_manager.py | 11 +-
.../src/airflow/api_fastapi/common/types.py | 21 +-
.../datamodels/{extra_links.py => auth.py} | 15 +-
.../core_api/openapi/v2-rest-api-generated.yaml | 52 ++
.../api_fastapi/core_api/routes/public/auth.py | 14 +
.../execution_api/datamodels/dag_parsing.py | 4 +-
.../execution_api/routes/dag_parsing.py | 77 ++-
.../src/airflow/config_templates/config.yml | 2 +
.../src/airflow/dag_processing/api_client.py | 54 +-
.../src/airflow/dag_processing/maintenance.py | 112 +--
airflow-core/src/airflow/dag_processing/manager.py | 122 +++-
.../src/airflow/dag_processing/processor.py | 19 +-
.../src/airflow/jobs/dag_processor_job_runner.py | 25 +-
.../src/airflow/jobs/scheduler_job_runner.py | 158 +++--
.../src/airflow/listeners/spec/importerrors.py | 12 +-
...52_3_5_0_add_dag_processor_publication_state.py | 3 +-
airflow-core/src/airflow/models/hitl.py | 17 +-
.../airflow/serialization/serialized_objects.py | 18 +-
airflow-core/src/airflow/timetables/_cron.py | 69 +-
airflow-core/src/airflow/timetables/base.py | 4 +-
.../src/airflow/ui/openapi-gen/queries/common.ts | 4 +
.../ui/openapi-gen/queries/ensureQueryData.ts | 7 +
.../src/airflow/ui/openapi-gen/queries/prefetch.ts | 7 +
.../src/airflow/ui/openapi-gen/queries/queries.ts | 7 +
.../src/airflow/ui/openapi-gen/queries/suspense.ts | 7 +
.../airflow/ui/openapi-gen/requests/schemas.gen.ts | 56 ++
.../ui/openapi-gen/requests/services.gen.ts | 15 +-
.../airflow/ui/openapi-gen/requests/types.gen.ts | 23 +
airflow-core/src/airflow/ui/package.json | 2 -
airflow-core/src/airflow/ui/pnpm-lock.yaml | 181 -----
.../airflow/ui/public/i18n/locales/ko/browse.json | 1 +
.../airflow/ui/public/i18n/locales/ko/common.json | 1 +
.../ui/public/i18n/locales/ko/components.json | 1 +
.../src/airflow/ui/public/i18n/locales/ko/dag.json | 4 +
.../airflow/ui/public/i18n/locales/ko/hitl.json | 1 +
.../components/DataTable/useRowSelection.test.tsx | 86 +++
.../src/components/DataTable/useRowSelection.tsx | 19 +-
.../FilterBar/filters/pausedStateParams.ts | 19 +-
.../src/airflow/ui/src/constants/filterConfigs.tsx | 2 +
.../src/airflow/ui/src/mocks/handlers/dags.ts | 4 +
.../DagsList/DagsFilters/DagsFilters.test.tsx | 24 +-
.../src/airflow/ui/src/pages/DagsList/DagsList.tsx | 4 +-
.../src/pages/TaskInstances/TaskInstances.test.tsx | 50 +-
.../ui/src/pages/TaskInstances/TaskInstances.tsx | 19 +-
.../pages/Variables/ImportVariablesForm.test.tsx | 83 +++
.../ui/src/pages/Variables/ImportVariablesForm.tsx | 12 +-
.../airflow/ui/src/pages/XCom/XComModal.test.tsx | 60 ++
.../src/airflow/ui/src/pages/XCom/XComModal.tsx | 6 +-
.../airflow/ui/src/queries/useBulkDeleteDagRuns.ts | 6 +
.../ui/src/queries/useDeleteDagRun.test.tsx | 82 +++
.../src/airflow/ui/src/queries/useDeleteDagRun.ts | 6 +
.../src/queries/useRecentTaskStateCounts.test.tsx | 20 +
.../ui/src/queries/useRecentTaskStateCounts.tsx | 9 +-
.../ui/tests/e2e/pages/RequiredActionsPage.ts | 357 +---------
airflow-core/src/airflow/utils/db.py | 8 +-
.../src/airflow/utils/log/file_task_handler.py | 40 +-
airflow-core/tests/conftest.py | 36 -
.../dag_processing/test_api_isolation.py | 10 +-
.../tests/unit/always/test_project_structure.py | 4 -
.../auth/managers/test_base_auth_manager.py | 5 +-
.../unit/api_fastapi/common/test_exceptions.py | 11 -
.../core_api/routes/public/test_assets.py | 10 -
.../core_api/routes/public/test_auth.py | 47 +-
.../core_api/routes/public/test_task_instances.py | 18 -
.../core_api/routes/public/test_variables.py | 6 -
.../api_fastapi/core_api/routes/test_routes.py | 1 +
.../versions/head/test_dag_parsing.py | 412 ++++++++---
.../execution_api/versions/head/test_jobs.py | 43 +-
airflow-core/tests/unit/assets/test_manager.py | 10 +-
.../tests/unit/cli/commands/_common_cli_classes.py | 43 --
.../cli/commands/test_dag_processor_command.py | 18 +-
.../tests/unit/dag_processing/test_api_client.py | 112 +++
.../tests/unit/dag_processing/test_claims.py | 4 +-
.../tests/unit/dag_processing/test_maintenance.py | 117 ++--
.../tests/unit/dag_processing/test_manager.py | 552 ++++++++++++++-
.../dag_processing/test_manager_api_failures.py | 128 +++-
.../dag_processing/test_manager_api_inventory.py | 83 ++-
.../dag_processing/test_manager_api_isolation.py | 206 ++++++
.../dag_processing/test_manager_api_results.py | 66 +-
.../test_priority_inventory_sequence.py | 2 +
.../tests/unit/dag_processing/test_processor.py | 78 ++-
.../dag_processing/test_requested_work_recovery.py | 10 +-
.../unit/jobs/test_dag_processor_job_runner.py | 91 ++-
airflow-core/tests/unit/jobs/test_scheduler_job.py | 270 ++++++--
airflow-core/tests/unit/models/test_dag.py | 14 -
airflow-core/tests/unit/models/test_hitl.py | 55 ++
.../tests/unit/models/test_taskinstance.py | 38 -
.../unit/serialization/test_dag_serialization.py | 39 ++
.../tests/unit/timetables/test_assets_timetable.py | 18 -
.../tests/unit/timetables/test_cron_mixin.py | 279 +++++++-
.../tests/unit/utils/log/test_log_reader.py | 4 +-
airflow-core/tests/unit/utils/test_log_handlers.py | 109 ++-
.../tests/unit/utils/test_operator_helpers.py | 37 -
.../src/airflowctl/api/datamodels/generated.py | 12 +
.../airflow_ctl/ctl/commands/test_auth_command.py | 40 +-
.../airflow_ctl/ctl/commands/test_dag_command.py | 46 +-
airflow-e2e-tests/docker/openlineage.yml | 88 ++-
.../tests/airflow_e2e_tests/conftest.py | 5 +-
.../airflow_aux/test_pod_template_file.py | 8 -
chart/tests/helm_tests/airflow_core/test_worker.py | 10 -
contributing-docs/08_static_code_checks.rst | 6 +
dev/README_RELEASE_AIRFLOW.md | 37 +-
dev/breeze/doc/09_release_management_tasks.rst | 15 +-
dev/breeze/doc/ci/04_selective_checks.md | 28 +-
...put_release-management_generate-constraints.svg | 78 ++-
...put_release-management_generate-constraints.txt | 2 +-
.../src/airflow_breeze/commands/ci_commands.py | 39 --
.../src/airflow_breeze/commands/common_options.py | 7 +
.../commands/release_management_commands.py | 5 +
.../commands/release_management_commands_config.py | 1 +
.../src/airflow_breeze/params/shell_params.py | 2 +
.../src/airflow_breeze/utils/selective_checks.py | 52 +-
.../src/airflow_breeze/utils/workflow_status.py | 4 +
dev/breeze/tests/test_selective_checks.py | 91 ++-
dev/breeze/tests/test_set_milestone.py | 48 --
dev/breeze/tests/test_shell_params.py | 6 +
.../breeze/tests/test_workflow_status.py | 34 +-
.../src/sphinx_exts/docs_build/code_utils.py | 45 --
generated/known_airflow_exceptions.txt | 2 +-
go-sdk/DEVELOPMENT.md | 10 +-
go-sdk/airflow/conformance_test.go | 4 +-
java-sdk/sdk/build.gradle.kts | 2 +-
providers/.last_release_date.txt | 2 +-
.../aws/auth_manager/datamodels}/__init__.py | 0
.../datamodels/test_login.py} | 23 +-
.../tests/unit/amazon/aws/bundles/test_s3.py | 10 -
.../unit/amazon/aws/log/test_s3_task_handler.py | 4 +-
.../kubernetes/sensors/test_spark_kubernetes.py | 78 +--
providers/common/ai/README.rst | 55 +-
providers/common/ai/docs/changelog.rst | 42 +-
.../common/ai/docs/connections/pydantic_ai.rst | 61 +-
providers/common/ai/docs/features.rst | 4 +-
providers/common/ai/docs/model_providers.rst | 4 +-
providers/common/ai/docs/observability.rst | 43 +-
providers/common/ai/docs/operators/llm_batch.rst | 8 +-
providers/common/ai/docs/operators/llm_branch.rst | 6 +-
.../common/ai/docs/operators/llm_file_analysis.rst | 10 +
.../ai/docs/operators/llm_schema_compare.rst | 3 +
providers/common/ai/docs/operators/llm_sql.rst | 3 +
providers/common/ai/docs/provider_fallback.rst | 8 +-
providers/common/ai/docs/self_hosted_models.rst | 19 +-
providers/common/ai/docs/structured_output.rst | 10 +-
providers/common/ai/docs/troubleshooting.rst | 9 +-
providers/common/ai/provider.yaml | 11 +-
.../src/airflow/providers/common/ai/batch/base.py | 14 +-
.../airflow/providers/common/ai/batch/dispatch.py | 13 +-
.../providers/common/ai/get_provider_info.py | 7 +-
.../providers/common/ai/hooks/pydantic_ai.py | 86 ++-
.../airflow/providers/common/ai/operators/llm.py | 101 +--
.../providers/common/ai/operators/llm_batch.py | 31 +-
.../providers/common/ai/plugins/model_panel.py | 14 +-
.../providers/common/ai/plugins/www/src/model.tsx | 4 +-
.../ai/tests/unit/common/ai/batch/test_base.py | 8 +
.../ai/tests/unit/common/ai/batch/test_dispatch.py | 4 +
.../common/ai/tests/unit/common/ai/conftest.py | 57 ++
.../unit/common/ai/decorators/test_llm_branch.py | 12 +-
.../common/ai/decorators/test_llm_file_analysis.py | 8 +-
.../ai/decorators/test_llm_schema_compare.py | 2 +-
.../unit/common/ai/decorators/test_llm_sql.py | 12 +-
.../tests/unit/common/ai/hooks/test_pydantic_ai.py | 160 ++++-
.../unit/common/ai/operators/test_llm_batch.py | 80 +++
.../unit/common/ai/operators/test_llm_branch.py | 84 ++-
.../common/ai/operators/test_llm_file_analysis.py | 42 +-
.../common/ai/operators/test_llm_schema_compare.py | 26 +-
.../tests/unit/common/ai/operators/test_llm_sql.py | 18 +-
.../unit/common/ai/plugins/test_model_panel.py | 34 +-
.../tests/unit/databricks/hooks/test_databricks.py | 19 -
.../unit/databricks/hooks/test_databricks_base.py | 4 -
.../tests/unit/databricks/utils/test_mixins.py | 10 -
providers/fab/docs/auth-manager/access-control.rst | 4 +
.../api_fastapi/datamodels/test_login.py} | 24 +-
providers/fab/tests/unit/fab/utils.py | 109 ---
.../git/src/airflow/providers/git/bundles/git.py | 2 +-
providers/git/tests/unit/git/bundles/test_git.py | 8 +-
.../operators/marketing_platform/bid_manager.rst | 76 +-
.../google/marketing_platform/hooks/bid_manager.py | 21 +-
.../marketing_platform/operators/bid_manager.py | 237 ++++++-
.../marketing_platform/sensors/bid_manager.py | 6 +-
.../marketing_platform/example_bid_manager.py | 37 +-
.../unit/google/cloud/log/test_gcs_task_handler.py | 4 +-
.../google/cloud/transfers/test_calendar_to_gcs.py | 54 --
.../unit/google/cloud/transfers/test_gcs_to_gcs.py | 18 -
.../tests/unit/google/cloud/utils/test_bigquery.py | 58 ++
.../google/cloud/utils/test_bigquery_get_data.py | 77 +++
.../marketing_platform/hooks/test_bid_manager.py | 37 +-
.../operators/test_bid_manager.py | 135 +++-
.../marketing_platform/sensors/test_bid_manager.py | 23 +-
providers/http/tests/unit/http/hooks/test_http.py | 4 -
.../auth_manager/test_keycloak_auth_manager.py | 23 -
providers/microsoft/azure/docs/changelog.rst | 20 +-
.../microsoft/azure/docs/operators/msgraph.rst | 5 +-
.../microsoft/azure/docs/operators/powerbi.rst | 14 +-
providers/microsoft/azure/provider.yaml | 2 +-
.../providers/microsoft/azure/exceptions.py | 9 +
.../providers/microsoft/azure/operators/msgraph.py | 124 +++-
.../providers/microsoft/azure/operators/powerbi.py | 53 +-
.../providers/microsoft/azure/version_compat.py | 2 +
.../microsoft/azure/log/test_wasb_task_handler.py | 2 +-
.../unit/microsoft/azure/operators/test_msgraph.py | 145 +++-
.../unit/microsoft/azure/operators/test_powerbi.py | 87 ++-
providers/openlineage/docs/changelog.rst | 24 +
providers/openlineage/docs/emission_policy.rst | 13 +-
providers/openlineage/provider.yaml | 2 +-
.../providers/openlineage/api/emission_policy.py | 3 +-
.../providers/openlineage/utils/emission_policy.py | 8 +-
.../airflow/providers/openlineage/utils/utils.py | 61 +-
...le_openlineage_policy_authoring_controls_dag.py | 140 ++++
...example_openlineage_policy_conf_controls_dag.py | 200 ++++++
..._openlineage_policy_conf_locked_controls_dag.py | 156 +++++
.../example_openlineage_policy_conf_locked_dag.py | 2 +-
...mple_openlineage_policy_conf_source_code_dag.py | 2 +-
...xample_openlineage_policy_dag_emit_false_dag.py | 2 +-
...mple_openlineage_policy_dag_events_false_dag.py | 4 +-
...le_openlineage_policy_dag_events_trigger_dag.py | 123 ++++
...ple_openlineage_policy_dag_override_task_dag.py | 2 +-
...ineage_policy_exclude_datasets_authoring_dag.py | 285 ++++++++
...openlineage_policy_exclude_datasets_conf_dag.py | 246 +++++++
...enlineage_policy_exclude_datasets_global_dag.py | 224 ++++++
...nlineage_policy_exclude_datasets_sources_dag.py | 342 +++++++++
...ineage_policy_exclude_datasets_versioned_dag.py | 125 ++++
...mple_openlineage_policy_extract_metadata_dag.py | 2 +-
...xample_openlineage_policy_full_task_info_dag.py | 2 +-
.../example_openlineage_policy_hook_lineage_dag.py | 2 +-
.../example_openlineage_policy_source_code_dag.py | 2 +-
...ample_openlineage_policy_task_emit_false_dag.py | 7 +-
...json => openlineage_defer_simple_dag__af2.json} | 0
.../openlineage_policy_authoring_controls_dag.json | 455 ++++++++++++
.../openlineage_policy_conf_controls_dag.json | 759 ++++++++++++++++++++
...penlineage_policy_conf_locked_controls_dag.json | 308 +++++++++
.../openlineage_policy_dag_events_trigger_dag.json | 487 +++++++++++++
...eage_policy_exclude_datasets_authoring_dag.json | 494 +++++++++++++
...enlineage_policy_exclude_datasets_conf_dag.json | 614 +++++++++++++++++
...lineage_policy_exclude_datasets_global_dag.json | 410 +++++++++++
...ineage_policy_exclude_datasets_sources_dag.json | 765 +++++++++++++++++++++
...eage_policy_exclude_datasets_versioned_dag.json | 172 +++++
...exclude_datasets_versioned_dag__af3_1_plus.json | 201 ++++++
...exclude_datasets_versioned_dag__af3_2_plus.json | 201 ++++++
.../openlineage_policy_task_emit_false_dag.json | 17 +-
.../tests/system/openlineage/operator.py | 56 +-
.../tests/unit/openlineage/plugins/test_utils.py | 48 +-
.../unit/openlineage/utils/test_emission_policy.py | 7 +
.../tests/unit/openlineage/utils/test_utils.py | 118 +++-
.../sftp/tests/unit/sftp/sensors/test_sftp.py | 13 -
providers/smtp/README.rst | 2 +-
providers/smtp/docs/index.rst | 2 +-
providers/smtp/pyproject.toml | 2 +-
.../smtp/src/airflow/providers/smtp/hooks/smtp.py | 6 +-
providers/smtp/tests/unit/smtp/hooks/test_smtp.py | 31 +-
.../triggers/test_teradata_compute_cluster.py | 21 -
.../check_go_sdk.py} | 2 +-
.../ci/lang_sdk_serialization/check_java_sdk.py | 4 +-
.../ci/lang_sdk_serialization/check_ts_sdk.py | 7 +-
scripts/ci/lang_sdk_serialization/compare.py | 4 +-
scripts/ci/prek/AGENTS.md | 30 +-
scripts/ci/prek/check_init_decorator_arguments.py | 15 -
scripts/ci/prek/check_prek_parallel_hooks.py | 97 +++
scripts/in_container/run_generate_constraints.py | 129 +++-
.../test_check_go_sdk.py} | 4 +-
.../ci/prek/test_check_prek_parallel_hooks.py | 75 ++
.../in_container/test_run_generate_constraints.py | 167 ++++-
.../observability/metrics/metrics_template.yaml | 3 +-
.../src/airflow/sdk/api/datamodels/_generated.py | 4 +-
.../task_sdk/definitions/test_asset_decorators.py | 7 -
ts-sdk/.pre-commit-config.yaml | 16 -
ts-sdk/tests/conformance/serialize_typescript.ts | 2 +-
uv.lock | 2 +-
279 files changed, 14117 insertions(+), 2818 deletions(-)
copy airflow-core/src/airflow/api_fastapi/core_api/datamodels/{extra_links.py
=> auth.py} (72%)
create mode 100644
airflow-core/src/airflow/ui/src/components/DataTable/useRowSelection.test.tsx
create mode 100644
airflow-core/src/airflow/ui/src/pages/Variables/ImportVariablesForm.test.tsx
create mode 100644
airflow-core/src/airflow/ui/src/queries/useDeleteDagRun.test.tsx
create mode 100644
airflow-core/tests/unit/dag_processing/test_manager_api_isolation.py
create mode 100644 airflow-core/tests/unit/models/test_hitl.py
copy
airflow-core/src/airflow/migrations/versions/0070_3_1_0_add_deadline_to_dag.py
=> dev/breeze/tests/test_workflow_status.py (58%)
copy {airflow-core/src/airflow/_shared =>
providers/amazon/tests/unit/amazon/aws/auth_manager/datamodels}/__init__.py
(100%)
copy providers/amazon/tests/unit/amazon/aws/{cli/test_definition.py =>
auth_manager/datamodels/test_login.py} (58%)
copy providers/{amazon/tests/unit/amazon/aws/cli/test_definition.py =>
fab/tests/unit/fab/auth_manager/api_fastapi/datamodels/test_login.py} (54%)
create mode 100644
providers/google/tests/unit/google/cloud/utils/test_bigquery.py
create mode 100644
providers/google/tests/unit/google/cloud/utils/test_bigquery_get_data.py
create mode 100644
providers/openlineage/tests/system/openlineage/example_openlineage_policy_authoring_controls_dag.py
create mode 100644
providers/openlineage/tests/system/openlineage/example_openlineage_policy_conf_controls_dag.py
create mode 100644
providers/openlineage/tests/system/openlineage/example_openlineage_policy_conf_locked_controls_dag.py
create mode 100644
providers/openlineage/tests/system/openlineage/example_openlineage_policy_dag_events_trigger_dag.py
create mode 100644
providers/openlineage/tests/system/openlineage/example_openlineage_policy_exclude_datasets_authoring_dag.py
create mode 100644
providers/openlineage/tests/system/openlineage/example_openlineage_policy_exclude_datasets_conf_dag.py
create mode 100644
providers/openlineage/tests/system/openlineage/example_openlineage_policy_exclude_datasets_global_dag.py
create mode 100644
providers/openlineage/tests/system/openlineage/example_openlineage_policy_exclude_datasets_sources_dag.py
create mode 100644
providers/openlineage/tests/system/openlineage/example_openlineage_policy_exclude_datasets_versioned_dag.py
rename
providers/openlineage/tests/system/openlineage/expected_events/{openlineage_defer_simple_dag.json
=> openlineage_defer_simple_dag__af2.json} (100%)
create mode 100644
providers/openlineage/tests/system/openlineage/expected_events/openlineage_policy_authoring_controls_dag.json
create mode 100644
providers/openlineage/tests/system/openlineage/expected_events/openlineage_policy_conf_controls_dag.json
create mode 100644
providers/openlineage/tests/system/openlineage/expected_events/openlineage_policy_conf_locked_controls_dag.json
create mode 100644
providers/openlineage/tests/system/openlineage/expected_events/openlineage_policy_dag_events_trigger_dag.json
create mode 100644
providers/openlineage/tests/system/openlineage/expected_events/openlineage_policy_exclude_datasets_authoring_dag.json
create mode 100644
providers/openlineage/tests/system/openlineage/expected_events/openlineage_policy_exclude_datasets_conf_dag.json
create mode 100644
providers/openlineage/tests/system/openlineage/expected_events/openlineage_policy_exclude_datasets_global_dag.json
create mode 100644
providers/openlineage/tests/system/openlineage/expected_events/openlineage_policy_exclude_datasets_sources_dag.json
create mode 100644
providers/openlineage/tests/system/openlineage/expected_events/openlineage_policy_exclude_datasets_versioned_dag.json
create mode 100644
providers/openlineage/tests/system/openlineage/expected_events/openlineage_policy_exclude_datasets_versioned_dag__af3_1_plus.json
create mode 100644
providers/openlineage/tests/system/openlineage/expected_events/openlineage_policy_exclude_datasets_versioned_dag__af3_2_plus.json
rename scripts/ci/{prek/check_go_sdk_serialization_conformance.py =>
lang_sdk_serialization/check_go_sdk.py} (96%)
rename java-sdk/scripts/ci/prek/check_serialization_conformance.py =>
scripts/ci/lang_sdk_serialization/check_java_sdk.py (94%)
rename ts-sdk/scripts/ci/prek/check_serialization_conformance.py =>
scripts/ci/lang_sdk_serialization/check_ts_sdk.py (92%)
create mode 100755 scripts/ci/prek/check_prek_parallel_hooks.py
rename scripts/tests/ci/{prek/test_check_go_sdk_serialization_conformance.py
=> lang_sdk_serialization/test_check_go_sdk.py} (94%)
create mode 100644 scripts/tests/ci/prek/test_check_prek_parallel_hooks.py
