This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a change to branch ci-upgrade-main
in repository https://gitbox.apache.org/repos/asf/airflow.git
discard b188b158f18 Fix CI breakages from the dependency upgrade
omit 7aa3e0e3087 Make fab and standard provider typing hold for SQLAlchemy
2.1
omit e90f0899b21 Fix "prek run update-providers-build-files --all-files"
omit 1bd55b09626 [main] CI: Upgrade important CI environment
add 6f358052a42 Send task inlets and outlets as JSON lists in OpenLineage
facet (#74666)
add e0c54d330d0 Clean up deleted worktrees' Breeze resources automatically
(#74282)
add 27c8cd7059c Reserve graph header width for loop-group iteration badge
(#75076)
add 7cae03be33f Speed up API server requests by keeping JWT checks off the
event loop (#74856)
add cffe6dfdcf1 Fix flaky LocalExecutor concurrent result refill test
(#75030)
add a26f9f9926c Fix missing and incorrect endpoints in the API permission
reference (#74871)
add 5ce21c5ad74 Fix mypy errors from SQLAlchemy 2.1 typing in fab sorting
and standard sensor_helper (#75075)
add 1d66775efcb Limit Glue retry task UUID scan (#71546)
add 57fb394afe8 Validate CallbackTrigger callback path before running the
callback (#74842)
add 325360c2b92 Reject reserved serialization keys in Dag run conf (#74834)
add 99f2b8618df Scope asset Dag and task references to readable Dags
(#74836)
add a1109f86030 Mask provider-declared sensitive connection extras on
every connection type (#73794)
add 6f408e1715b Scope upstream asset events of a Dag run to assets and
Dags the caller may read (#72924)
add fa83bd029cb Remove unused type: ignore comments in airflow-core
(#75078)
add 46ff76ec9c2 Fix the @asset example in the asset state store docs
(#75070)
add cae1728f127 Add EmrServerlessStopSessionOperator to Amazon provider
(#74870)
add ab1acec7aa8 Open the Dags menu from the Dags nav button instead of
navigating (#75092)
add d2a8457e768 Run the lang-SDK conformance serializer in its own
virtualenv (#75085)
add 233f9c3fc78 Decode deadline callback kwargs where the callback runs
(#74830)
add bfa23909190 Mask registered secrets at any nesting depth (#73793)
add a26eb122498 Mask XCom values in the audit log (#73791)
add b0239d215e1 Only rebuild built-in value types when converting
next_kwargs for older workers (#73577)
add 11174724838 Add the resume event to stored next_kwargs without
decoding them (#73576)
add a7ac9379cf9 Read XCom without object reconstruction during dependency
evaluation (#72718)
add a10e4c37666 [main] CI: Upgrade important CI environment
add e0934d2c6f8 Fix "prek run update-providers-build-files --all-files"
add a20c177f464 Fix CI breakages from the dependency upgrade
add 43058e2344b Type Backfill.duration expression against the class
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (b188b158f18)
\
N -- N -- N refs/heads/ci-upgrade-main (43058e2344b)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.../docs/core-concepts/asset-state-store.rst | 4 +-
airflow-core/docs/security/api_permissions_ref.rst | 54 ++-
airflow-core/newsfragments/73577.significant.rst | 5 +
airflow-core/newsfragments/73791.improvement.rst | 1 +
airflow-core/newsfragments/73793.improvement.rst | 1 +
airflow-core/newsfragments/73794.improvement.rst | 1 +
airflow-core/newsfragments/74830.significant.rst | 10 +
airflow-core/newsfragments/74834.bugfix.rst | 1 +
airflow-core/newsfragments/74836.bugfix.rst | 1 +
airflow-core/newsfragments/74842.improvement.rst | 1 +
.../api_fastapi/auth/managers/base_auth_manager.py | 4 +-
.../api_fastapi/auth/middlewares/refresh_token.py | 154 +++++---
airflow-core/src/airflow/api_fastapi/compat.py | 2 +-
.../src/airflow/api_fastapi/core_api/app.py | 6 +-
.../api_fastapi/core_api/datamodels/assets.py | 19 +-
.../api_fastapi/core_api/datamodels/backfills.py | 8 +-
.../api_fastapi/core_api/datamodels/common.py | 14 +
.../api_fastapi/core_api/datamodels/connections.py | 57 +++
.../api_fastapi/core_api/datamodels/dag_run.py | 8 +-
.../api_fastapi/core_api/routes/public/assets.py | 59 +---
.../api_fastapi/core_api/routes/public/dag_run.py | 48 ++-
.../core_api/routes/public/extra_links.py | 5 +-
.../core_api/routes/public/task_instances.py | 20 +-
.../api_fastapi/core_api/routes/ui/assets.py | 9 +-
.../core_api/services/public/connections.py | 12 +-
.../api_fastapi/execution_api/datamodels/dagrun.py | 8 +-
.../api_fastapi/execution_api/routes/__init__.py | 2 +-
.../execution_api/versions/v2025_08_10.py | 2 +-
.../execution_api/versions/v2025_10_27.py | 2 +-
.../execution_api/versions/v2025_11_05.py | 2 +-
.../execution_api/versions/v2026_04_06.py | 240 ++++++++++++-
.../execution_api/versions/v2026_06_30.py | 14 +-
.../execution_api/versions/v2026_10_30.py | 6 +-
.../src/airflow/api_fastapi/logging/decorators.py | 16 +
.../src/airflow/executors/base_executor.py | 8 +
.../src/airflow/executors/workloads/callback.py | 6 +
.../src/airflow/jobs/triggerer_job_runner.py | 8 +-
airflow-core/src/airflow/models/backfill.py | 7 +-
airflow-core/src/airflow/models/callback.py | 52 ++-
airflow-core/src/airflow/models/deadline.py | 33 +-
airflow-core/src/airflow/models/serialized_dag.py | 4 +-
airflow-core/src/airflow/models/taskinstance.py | 21 +-
airflow-core/src/airflow/models/trigger.py | 90 +++--
airflow-core/src/airflow/models/xcom.py | 43 +++
airflow-core/src/airflow/serialization/decoders.py | 89 ++++-
.../serialization/definitions/mappedoperator.py | 2 +-
airflow-core/src/airflow/serialization/encoders.py | 68 +++-
.../ti_deps/deps/not_previously_skipped_dep.py | 66 +++-
.../src/airflow/ti_deps/deps/prev_dagrun_dep.py | 14 +
.../src/airflow/ti_deps/deps/trigger_rule_dep.py | 12 +-
airflow-core/src/airflow/timetables/simple.py | 2 +-
airflow-core/src/airflow/triggers/callback.py | 27 ++
.../ui/src/components/Graph/elkGraphUtils.ts | 5 +
.../airflow/ui/src/layouts/Nav/DagsButton.test.tsx | 36 +-
.../src/airflow/ui/src/layouts/Nav/DagsButton.tsx | 4 +-
.../airflow/ui/src/layouts/Nav/NavButton.test.tsx | 18 -
.../src/airflow/ui/src/layouts/Nav/NavButton.tsx | 17 +-
.../src/airflow/utils/cli_action_loggers.py | 2 +-
airflow-core/src/airflow/utils/db_cleanup.py | 2 +-
airflow-core/src/airflow/utils/memray_utils.py | 2 +
airflow-core/src/airflow/utils/retries.py | 2 +-
.../auth/managers/test_base_auth_manager.py | 28 ++
.../auth/middlewares/test_refresh_token.py | 387 +++++++++++----------
.../core_api/datamodels/test_connections.py | 60 ++++
.../core_api/datamodels/test_dag_run.py | 66 +++-
.../core_api/routes/public/test_assets.py | 108 +++++-
.../core_api/routes/public/test_backfills.py | 27 ++
.../core_api/routes/public/test_connections.py | 133 +++++++
.../core_api/routes/public/test_dag_run.py | 113 +++++-
.../core_api/routes/public/test_xcom.py | 41 +++
.../api_fastapi/core_api/routes/ui/test_assets.py | 42 ++-
.../execution_api/versions/head/test_dag_runs.py | 21 ++
.../versions/v2026_04_06/test_task_instances.py | 339 ++++++++++++++++++
.../unit/api_fastapi/logging/test_decorators.py | 15 +
airflow-core/tests/unit/api_fastapi/test_app.py | 10 +
.../tests/unit/cli/commands/test_dag_command.py | 2 +-
.../tests/unit/executors/test_base_executor.py | 6 +-
.../tests/unit/executors/test_local_executor.py | 31 +-
.../tests/unit/executors/test_workloads.py | 4 +-
airflow-core/tests/unit/jobs/test_scheduler_job.py | 59 ++++
airflow-core/tests/unit/jobs/test_triggerer_job.py | 41 +++
.../unit/migrations/test_migration_patterns.py | 2 +-
airflow-core/tests/unit/models/test_callback.py | 200 ++++++++++-
.../tests/unit/models/test_connection_test.py | 2 +-
airflow-core/tests/unit/models/test_deadline.py | 167 ++++++++-
airflow-core/tests/unit/models/test_trigger.py | 70 ++++
airflow-core/tests/unit/models/test_xcom.py | 90 +++++
.../tests/unit/plugins/test_plugins_manager.py | 4 +-
.../unit/serialization/test_serialized_objects.py | 86 ++++-
.../deps/test_not_previously_skipped_dep.py | 311 +++++++++++++++++
.../unit/ti_deps/deps/test_trigger_rule_dep.py | 62 +++-
airflow-core/tests/unit/triggers/test_callback.py | 114 +++++-
airflow-core/tests/unit/utils/test_db_manager.py | 2 +-
dev/breeze/doc/01_installation.rst | 4 +
dev/breeze/doc/03_developer_tasks.rst | 38 +-
dev/breeze/doc/04_troubleshooting.rst | 5 +-
...19-isolate-docker-resources-per-git-worktree.md | 154 ++++++++
dev/breeze/doc/images/output_setup.svg | 14 +-
dev/breeze/doc/images/output_setup.txt | 2 +-
dev/breeze/doc/images/output_setup_config.svg | 100 ++++--
dev/breeze/doc/images/output_setup_config.txt | 2 +-
.../airflow_breeze/commands/developer_commands.py | 15 +-
.../src/airflow_breeze/commands/setup_commands.py | 28 +-
.../commands/setup_commands_config.py | 1 +
.../src/airflow_breeze/params/shell_params.py | 8 +-
.../airflow_breeze/utils/docker_command_utils.py | 89 ++++-
dev/breeze/src/airflow_breeze/utils/path_utils.py | 73 +++-
.../src/airflow_breeze/utils/worktree_watcher.py | 168 +++++++++
dev/breeze/tests/conftest.py | 16 +
dev/breeze/tests/test_developer_commands.py | 54 ++-
dev/breeze/tests/test_docker_command_utils.py | 196 ++++++++++-
dev/breeze/tests/test_git_worktree.py | 123 ++++++-
dev/breeze/tests/test_shell_params.py | 31 +-
dev/breeze/tests/test_worktree_watcher.py | 193 ++++++++++
.../src/tests_common/test_utils/mock_context.py | 1 +
generated/known_sdk_imports_in_core.txt | 4 +-
.../amazon/docs/operators/emr/emr_serverless.rst | 14 +
providers/amazon/docs/operators/glue.rst | 4 +-
.../airflow/providers/amazon/aws/operators/emr.py | 92 +++++
.../airflow/providers/amazon/aws/operators/glue.py | 27 +-
.../airflow/providers/amazon/aws/triggers/emr.py | 48 +++
.../amazon/aws/waiters/emr-serverless.json | 19 +
.../amazon/aws/example_emr_serverless_session.py | 10 +
.../aws/operators/test_emr_serverless_session.py | 115 +++++-
.../tests/unit/amazon/aws/operators/test_glue.py | 107 ++++++
.../tests/unit/amazon/aws/triggers/test_emr.py | 60 ++++
.../unit/common/ai/utils/test_prompt_cache.py | 50 +--
providers/fab/docs/auth-manager/access-control.rst | 54 ++-
.../fab/auth_manager/api_fastapi/sorting.py | 9 +-
.../airflow/providers/openlineage/utils/utils.py | 19 +
.../openlineage_base_complex_dag__af2.json | 56 +++
.../openlineage_base_complex_dag__af3.json | 62 ++++
.../openlineage_base_simple_dag__af2.json | 4 +
.../openlineage_base_simple_dag__af3.json | 4 +
.../openlineage_custom_operator_failure_dag.json | 2 +
.../openlineage_manual_lineage_dag.json | 4 +
.../openlineage_taskflow_simple_dag.json | 4 +
.../tests/unit/openlineage/utils/test_utils.py | 102 +++++-
.../providers/standard/utils/sensor_helper.py | 36 +-
scripts/ci/docker-compose/backend-mysql.yml | 2 +
scripts/ci/docker-compose/backend-postgres.yml | 2 +
scripts/ci/docker-compose/base.yml | 2 +
.../ci/docker-compose/integration-cassandra.yml | 2 +
scripts/ci/docker-compose/integration-celery.yml | 3 +
scripts/ci/docker-compose/integration-drill.yml | 2 +
.../docker-compose/integration-elasticsearch.yml | 2 +
scripts/ci/docker-compose/integration-kafka.yml | 1 +
scripts/ci/docker-compose/integration-kerberos.yml | 2 +
scripts/ci/docker-compose/integration-keycloak.yml | 1 +
.../ci/docker-compose/integration-localstack.yml | 1 +
scripts/ci/docker-compose/integration-mongo.yml | 2 +
scripts/ci/docker-compose/integration-mssql.yml | 2 +
.../ci/docker-compose/integration-openlineage.yml | 2 +
.../ci/docker-compose/integration-opensearch.yml | 2 +
scripts/ci/docker-compose/integration-otel.yml | 4 +
scripts/ci/docker-compose/integration-pinot.yml | 1 +
scripts/ci/docker-compose/integration-qdrant.yml | 1 +
scripts/ci/docker-compose/integration-redis.yml | 3 +
scripts/ci/docker-compose/integration-statsd.yml | 3 +
.../ci/docker-compose/integration-tinkerpop.yml | 2 +
scripts/ci/docker-compose/integration-trino.yml | 2 +
scripts/ci/docker-compose/integration-ydb.yml | 1 +
scripts/ci/docker-compose/local-all-sources.yml | 2 +
scripts/ci/docker-compose/local.yml | 2 +
scripts/ci/lang_sdk_serialization/compare.py | 13 +-
scripts/ci/prek/extract_permissions.py | 35 +-
scripts/ci/prek/fab_permissions_doc.py | 13 +-
scripts/tests/ci/prek/test_extract_permissions.py | 148 ++++++++
.../secrets_masker/secrets_masker.py | 15 +-
.../tests/secrets_masker/test_secrets_masker.py | 11 +-
.../sdk/execution_time/callback_supervisor.py | 29 ++
.../execution_time/test_callback_supervisor.py | 107 +++++-
172 files changed, 5959 insertions(+), 761 deletions(-)
create mode 100644 airflow-core/newsfragments/73577.significant.rst
create mode 100644 airflow-core/newsfragments/73791.improvement.rst
create mode 100644 airflow-core/newsfragments/73793.improvement.rst
create mode 100644 airflow-core/newsfragments/73794.improvement.rst
create mode 100644 airflow-core/newsfragments/74830.significant.rst
create mode 100644 airflow-core/newsfragments/74834.bugfix.rst
create mode 100644 airflow-core/newsfragments/74836.bugfix.rst
create mode 100644 airflow-core/newsfragments/74842.improvement.rst
create mode 100644
dev/breeze/doc/adr/0019-isolate-docker-resources-per-git-worktree.md
create mode 100644 dev/breeze/src/airflow_breeze/utils/worktree_watcher.py
create mode 100644 dev/breeze/tests/test_worktree_watcher.py
