This is an automated email from the ASF dual-hosted git repository.
github-actions[bot] pushed a change to branch ci-upgrade-main
in repository https://gitbox.apache.org/repos/asf/airflow.git
discard 3b1e17c94f2 [main] CI: Upgrade important CI environment
add 4aafb9523be Correct the example config for the coordinators (#68929)
add 94405c425ac SnowflakeHook: extract OAuth token lifecycle management
into dedicated helper (#68549)
add 73c88418feb Fix DatabricksWorkflowTaskGroup ignoring
upstream/downstream set before with block (#68924)
add 0b1912c1061 Propagate verify and botocore_config through
EC2StateSensorTrigger (#68921)
add 31f7ba74209 Fix race condition on sort param when returning dagRuns
(#68842)
add 88e61106fdc Exclude non-successful runs from AVERAGE_RUNTIME deadline
calculation (#68647)
add 76f9210e5e0 Propagate AWS hook parameters through
RedshiftClusterTrigger (#68925)
add b9148fa1c55 Fix CLI connection display reporting wrong
is_extra_encrypted flag (#67561)
add a6115b172f1 Allow forcing trace head-sampling per DAG run via conf
(#68860)
add 37d8acbd5a7 Persist batchoperator deferrable xcom links (#64745)
add 7286fa9d921 Fix CloudWatch remote logging for ephemeral lifecycle
executor (#68779)
add 5928304a33c Refactor Snowflake provider tests to remove legacy
unittest imports (#68898)
add 4bbcc163321 Standardize ECS TaskDoneTrigger on region_name and AWS
hook parameters (#68923)
add d42096908f6 Make Amazon SageMaker triggers inherit AWS base classes
(#68927)
add feea09d6107 AIP-103: Allow for `get`/`set`/`delete`/`clear` to run for
`AssetStateStoreAccessor` on Triggerer (#68900)
add 50227092c79 Fix flaky test_trigger_logger_fd_closed_when_removed on
slow runners (#68965)
add 653aef3bac2 Reject Conventional Commits style commit messages (#68633)
add 5303dbc0959 Gate raw Dag deserialization error detail behind
api.expose_stacktrace and log it server-side (#68388)
add 4d39a855b1c Walk nested lists/tuples/sets in secrets masker key-name
redaction (#68422)
add a2f8a7ab39d Fix Java coordinator rejecting macOS dual-stack loopback
connections (#68960)
add 94b365fcd57 Add partition_date to DagRun detail page (#68969)
add e886dfd05ee Allow missing `api_auth.jwt_secret` for
`InProcessExecutionAPI` (#68980)
add 7dd84f3eed5 Fix TaskInstance duration calculation with SQLite (#68142)
add e34b725b382 Raise a warning instead of an error if extra teams are
defined in the auth manager (#68935)
add 90f061288b3 Rename PartitionAtRuntime as PartitionedAtRuntime for
consistency (#68978)
add 41623bc01fe Publish schema files with doc publishing workflow (#68976)
add 70bb308b19a Add keyboard shortcut help dialog to discover available
shortcuts (? key) (#68200)
add 74c59481b64 Disable branch protection on v3-3-stable (#68993)
add 4bfafaeca2f Add query tags to `DatabricksSqlSensor` and
`DatabricksPartitionSensor` (#68704)
add 2161027d844 Fix grammar in PR guidelines (#68992)
add 30a70719c03 Rename Neo4jOperator `sql` parameter to `cypher` (#68970)
add 779efc6983b Require existing-connection read access when testing an
existing connection (#67620)
add b4114239a89 Handle missing Keycloak resources as access denied (#68951)
add 65cd588ae89 Fix flaky CloudSQL custom-universe trigger test on loaded
runners (#68981)
add a4c269a6f9e Don't force full CI tests for non-test workflow and
prek-only changes (#68802)
add 8e9a13bd57c Thread version_data through BundleInfo to worker-side
bundle initialization (#67217)
add 3905dc509bb Cache Celery apps when publishing workloads (#67127)
add 275730a9cf2 Java-SDK: replace AirflowJulHandler.install() with setup()
(#68986)
add 4b925364105 Add owners for Java and Go SDKs (#69009)
add 473e33259cd Remove stale `dag_` parameter from create_test_pipeline
docstring (#68787)
add a037bec9664 [main] CI: Upgrade important CI environment
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (3b1e17c94f2)
\
N -- N -- N refs/heads/ci-upgrade-main (a037bec9664)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.asf.yaml | 5 +
.github/CODEOWNERS | 5 +-
.github/workflows/ci-image-checks.yml | 12 +
.github/workflows/publish-docs-to-s3.yml | 62 +++
.pre-commit-config.yaml | 11 +-
AGENTS.md | 9 +
.../docs/authoring-and-scheduling/assets.rst | 12 +-
.../docs/core-concepts/asset-state-store.rst | 74 +++
airflow-core/docs/howto/deadline-alerts.rst | 6 +-
.../api_fastapi/auth/managers/base_auth_manager.py | 12 +-
.../src/airflow/api_fastapi/common/exceptions.py | 14 +-
.../src/airflow/api_fastapi/common/parameters.py | 2 +-
.../core_api/routes/public/connections.py | 51 +-
.../src/airflow/api_fastapi/execution_api/app.py | 19 +-
.../execution_api/routes/asset_state_store.py | 38 +-
.../src/airflow/cli/commands/connection_command.py | 4 +-
.../src/airflow/config_templates/config.yml | 7 +
.../src/airflow/dag_processing/bundles/base.py | 4 +
.../src/airflow/dag_processing/bundles/manager.py | 11 +-
.../example_dags/example_asset_partition.py | 12 +-
.../src/airflow/example_dags/example_skip_dag.py | 1 -
.../src/airflow/executors/workloads/base.py | 9 +-
.../src/airflow/executors/workloads/task.py | 4 +
.../src/airflow/jobs/scheduler_job_runner.py | 6 +
.../src/airflow/jobs/triggerer_job_runner.py | 48 ++
airflow-core/src/airflow/models/dagrun.py | 20 +
airflow-core/src/airflow/models/taskinstance.py | 12 +-
.../airflow/serialization/definitions/deadline.py | 11 +-
airflow-core/src/airflow/serialization/encoders.py | 8 +-
airflow-core/src/airflow/timetables/base.py | 2 +-
airflow-core/src/airflow/timetables/simple.py | 6 +-
.../airflow/ui/public/i18n/locales/en/common.json | 42 ++
.../src/airflow/ui/public/i18n/locales/en/dag.json | 4 +-
.../ui/public/i18n/locales/zh-TW/common.json | 1 +
.../ui/src/components/Clear/Run/ClearRunButton.tsx | 13 +-
.../Clear/TaskInstance/ClearTaskInstanceButton.tsx | 13 +-
.../FilterBar/filters/TextSearchFilter.tsx | 13 +-
.../airflow/ui/src/components/GraphTaskFilters.tsx | 9 +-
.../KeyboardShortcuts/KeyboardShortcutsModal.tsx | 106 ++++
.../KeyboardShortcuts/formatShortcutCombo.test.ts} | 33 +-
.../KeyboardShortcuts/formatShortcutCombo.ts | 66 +++
.../{DataTable => KeyboardShortcuts}/index.ts | 4 +-
.../src/components/MarkAs/Run/MarkRunAsButton.tsx | 23 +-
.../MarkAs/TaskGroup/MarkTaskGroupAsButton.tsx | 23 +-
.../TaskInstance/MarkTaskInstanceAsButton.tsx | 23 +-
.../src/airflow/ui/src/components/SearchBar.tsx | 13 +-
.../src/components/SearchDags/SearchDagsButton.tsx | 15 +-
.../ui/src/context/keyboardShortcuts/Context.ts | 58 ++
.../keyboardShortcuts/ShortcutRegistryProvider.tsx | 45 ++
.../keyboardShortcuts/index.ts} | 12 +-
.../ui/src/context/keyboardShortcuts/shortcuts.ts | 179 +++++++
.../keyboardShortcuts/useShortcutRegistry.ts} | 10 +-
.../src/hooks/navigation/useKeyboardNavigation.ts | 19 +-
.../src/airflow/ui/src/hooks/useShortcut.test.tsx | 111 ++++
.../src/airflow/ui/src/hooks/useShortcut.ts | 76 +++
.../src/airflow/ui/src/layouts/BaseLayout.tsx | 2 +
.../ui/src/layouts/Details/PanelButtons.tsx | 26 +-
airflow-core/src/airflow/ui/src/main.tsx | 5 +-
.../src/airflow/ui/src/pages/Dag/Code/Code.tsx | 14 +-
.../ui/src/pages/Dag/Overview/FailedLogs.tsx | 8 +-
.../src/airflow/ui/src/pages/Run/Details.tsx | 8 +
.../src/pages/TaskInstance/Logs/LogSearchInput.tsx | 13 +-
.../ui/src/pages/TaskInstance/Logs/Logs.tsx | 33 +-
.../src/pages/TaskInstance/Logs/TaskLogContent.tsx | 15 +-
.../tests/unit/api/common/test_trigger_dag.py | 12 +-
.../auth/managers/test_base_auth_manager.py | 22 +-
.../unit/api_fastapi/common/test_exceptions.py | 21 +
.../unit/api_fastapi/common/test_parameters.py | 21 +
.../core_api/routes/public/test_assets.py | 6 +-
.../core_api/routes/public/test_connections.py | 127 +++++
.../core_api/routes/public/test_dag_run.py | 26 +-
.../unit/api_fastapi/execution_api/test_app.py | 10 +
.../versions/head/test_asset_state_store.py | 72 +++
airflow-core/tests/unit/assets/test_manager.py | 4 +-
.../unit/cli/commands/test_connection_command.py | 34 ++
.../tests/unit/dag_processing/bundles/test_base.py | 13 +
.../tests/unit/dag_processing/test_collection.py | 4 +-
.../tests/unit/executors/test_workloads.py | 79 +++
airflow-core/tests/unit/jobs/test_scheduler_job.py | 6 +-
airflow-core/tests/unit/jobs/test_triggerer_job.py | 126 ++++-
airflow-core/tests/unit/models/test_dag.py | 6 +-
airflow-core/tests/unit/models/test_dagrun.py | 32 ++
airflow-core/tests/unit/models/test_deadline.py | 65 +++
.../tests/unit/models/test_taskinstance.py | 75 ++-
.../unit/serialization/test_dag_serialization.py | 10 +-
contributing-docs/05_pull_requests.rst | 2 +-
dev/breeze/doc/09_release_management_tasks.rst | 33 ++
dev/breeze/doc/ci/04_selective_checks.md | 23 +-
dev/breeze/doc/images/output-commands.svg | 2 +-
.../doc/images/output_release-management.svg | 24 +-
.../doc/images/output_release-management.txt | 2 +-
...ut_release-management_publish-schemas-to-s3.svg | 139 +++++
...ut_release-management_publish-schemas-to-s3.txt | 1 +
.../output_setup_check-all-params-in-groups.svg | 6 +-
.../output_setup_check-all-params-in-groups.txt | 2 +-
.../output_setup_regenerate-command-images.svg | 30 +-
.../output_setup_regenerate-command-images.txt | 2 +-
.../commands/release_management_commands.py | 62 +++
.../commands/release_management_commands_config.py | 13 +
dev/breeze/src/airflow_breeze/global_constants.py | 5 +
.../src/airflow_breeze/utils/publish_docs_to_s3.py | 122 +++++
.../src/airflow_breeze/utils/selective_checks.py | 23 +-
dev/breeze/tests/test_selective_checks.py | 48 +-
dev/breeze/uv.lock | 60 +--
docs/spelling_wordlist.txt | 1 +
.../apache/airflow/sdk/jul/AirflowJulHandler.kt | 25 +-
.../airflow/sdk/jul/AirflowJulHandlerTest.kt | 54 +-
.../amazon/aws/log/cloudwatch_task_handler.py | 55 +-
.../providers/amazon/aws/operators/batch.py | 133 +++--
.../airflow/providers/amazon/aws/operators/ecs.py | 4 +-
.../providers/amazon/aws/operators/sagemaker.py | 22 +-
.../airflow/providers/amazon/aws/sensors/ec2.py | 2 +
.../amazon/aws/sensors/redshift_cluster.py | 3 +
.../airflow/providers/amazon/aws/triggers/ec2.py | 16 +-
.../airflow/providers/amazon/aws/triggers/ecs.py | 38 +-
.../amazon/aws/triggers/redshift_cluster.py | 25 +-
.../providers/amazon/aws/triggers/sagemaker.py | 181 ++++---
.../aws/executors/batch/test_batch_executor.py | 1 +
.../amazon/aws/executors/ecs/test_ecs_executor.py | 1 +
.../amazon/aws/log/test_cloudwatch_task_handler.py | 58 ++
.../tests/unit/amazon/aws/operators/test_batch.py | 161 +++++-
.../tests/unit/amazon/aws/triggers/test_ec2.py | 22 +
.../tests/unit/amazon/aws/triggers/test_ecs.py | 82 +++
.../amazon/aws/triggers/test_redshift_cluster.py | 25 +
.../unit/amazon/aws/triggers/test_sagemaker.py | 173 +++++-
.../celery/executors/celery_executor_utils.py | 43 +-
.../integration/celery/test_celery_executor.py | 6 +
.../unit/celery/executors/test_celery_executor.py | 99 ++++
providers/databricks/docs/operators/sql.rst | 18 +
.../databricks/docs/operators/sql_statements.rst | 8 +
.../providers/databricks/operators/databricks.py | 17 +-
.../databricks/operators/databricks_sql.py | 37 +-
.../databricks/operators/databricks_workflow.py | 9 +
.../providers/databricks/sensors/databricks.py | 9 +
.../databricks/sensors/databricks_partition.py | 14 +
.../providers/databricks/sensors/databricks_sql.py | 12 +
.../providers/databricks/utils/query_tags.py | 75 +++
.../unit/databricks/operators/test_databricks.py | 101 ++++
.../databricks/operators/test_databricks_sql.py | 6 -
.../operators/test_databricks_workflow.py | 76 +++
.../unit/databricks/sensors/test_databricks.py | 90 ++++
.../sensors/test_databricks_partition.py | 69 +++
.../unit/databricks/sensors/test_databricks_sql.py | 69 +++
.../tests/unit/databricks/utils/test_query_tags.py | 106 ++++
.../edge3/worker_api/v2-edge-generated.yaml | 6 +
.../unit/google/cloud/triggers/test_cloud_sql.py | 9 +-
.../keycloak/auth_manager/keycloak_auth_manager.py | 10 +
.../auth_manager/test_keycloak_auth_manager.py | 37 ++
providers/neo4j/docs/operators/neo4j.rst | 8 +-
.../src/airflow/providers/neo4j/operators/neo4j.py | 32 +-
.../neo4j/tests/system/neo4j/example_neo4j.py | 16 +-
.../tests/system/neo4j/example_neo4j_query.py | 55 --
.../tests/system/neo4j/example_neo4j_sensor.py | 2 +-
.../neo4j/tests/unit/neo4j/operators/test_neo4j.py | 38 +-
.../airflow/providers/snowflake/hooks/snowflake.py | 232 ++++----
.../tests/unit/snowflake/hooks/test_snowflake.py | 6 +-
.../unit/snowflake/hooks/test_snowflake_sql_api.py | 25 +-
.../prek/check_no_conventional_commit_message.py | 128 +++++
scripts/ci/prek/check_trigger_serialize_init.py | 3 +
scripts/ci/prek/known_airflow_exceptions.txt | 2 +-
.../test_check_no_conventional_commit_message.py | 108 ++++
.../observability/traces/__init__.py | 50 +-
.../tests/observability/test_traces.py | 39 ++
.../secrets_masker/secrets_masker.py | 40 +-
.../tests/secrets_masker/test_secrets_masker.py | 14 +
task-sdk/dev/generate_task_sdk_models.py | 2 -
task-sdk/docs/api.rst | 2 +-
task-sdk/src/airflow/sdk/__init__.py | 6 +-
task-sdk/src/airflow/sdk/__init__.pyi | 4 +-
.../src/airflow/sdk/api/datamodels/_generated.py | 1 +
task-sdk/src/airflow/sdk/bases/timetable.py | 2 +-
.../src/airflow/sdk/coordinators/_subprocess.py | 20 +-
.../sdk/coordinators/executable/coordinator.py | 7 +-
.../airflow/sdk/coordinators/java/coordinator.py | 9 +-
.../airflow/sdk/definitions/timetables/assets.py | 2 +-
.../sdk/execution_time/callback_supervisor.py | 4 +-
.../airflow/sdk/execution_time/request_handlers.py | 94 ++++
.../airflow/sdk/execution_time/schema/schema.json | 13 +
.../src/airflow/sdk/execution_time/task_runner.py | 1 +
.../tests/task_sdk/coordinators/test_subprocess.py | 35 ++
task-sdk/tests/task_sdk/definitions/test_dag.py | 17 +-
.../execution_time/test_callback_supervisor.py | 1 +
.../task_sdk/execution_time/test_coordinator.py | 32 +-
.../execution_time/test_request_handlers.py | 167 ++++++
uv.lock | 594 +++++++++++----------
185 files changed, 5590 insertions(+), 1131 deletions(-)
create mode 100644
airflow-core/src/airflow/ui/src/components/KeyboardShortcuts/KeyboardShortcutsModal.tsx
copy airflow-core/src/airflow/ui/{tests/e2e/specs/configuration.spec.ts =>
src/components/KeyboardShortcuts/formatShortcutCombo.test.ts} (51%)
create mode 100644
airflow-core/src/airflow/ui/src/components/KeyboardShortcuts/formatShortcutCombo.ts
copy airflow-core/src/airflow/ui/src/components/{DataTable =>
KeyboardShortcuts}/index.ts (86%)
create mode 100644
airflow-core/src/airflow/ui/src/context/keyboardShortcuts/Context.ts
create mode 100644
airflow-core/src/airflow/ui/src/context/keyboardShortcuts/ShortcutRegistryProvider.tsx
copy airflow-core/src/airflow/ui/src/{queries/useConfig.tsx =>
context/keyboardShortcuts/index.ts} (73%)
create mode 100644
airflow-core/src/airflow/ui/src/context/keyboardShortcuts/shortcuts.ts
copy
airflow-core/src/airflow/{api_fastapi/auth/managers/simple/ui/rules/levels.js
=> ui/src/context/keyboardShortcuts/useShortcutRegistry.ts} (77%)
create mode 100644 airflow-core/src/airflow/ui/src/hooks/useShortcut.test.tsx
create mode 100644 airflow-core/src/airflow/ui/src/hooks/useShortcut.ts
create mode 100644
dev/breeze/doc/images/output_release-management_publish-schemas-to-s3.svg
create mode 100644
dev/breeze/doc/images/output_release-management_publish-schemas-to-s3.txt
create mode 100644
providers/databricks/src/airflow/providers/databricks/utils/query_tags.py
create mode 100644
providers/databricks/tests/unit/databricks/utils/test_query_tags.py
delete mode 100644 providers/neo4j/tests/system/neo4j/example_neo4j_query.py
create mode 100755 scripts/ci/prek/check_no_conventional_commit_message.py
create mode 100644
scripts/tests/ci/prek/test_check_no_conventional_commit_message.py
create mode 100644
task-sdk/tests/task_sdk/execution_time/test_request_handlers.py
