This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a change to branch python3.12
in repository https://gitbox.apache.org/repos/asf/airflow.git
omit c45ba0a9d8 Add Python 3.12 support
add fae63102aa Filter Datasets by associated dag_ids (GET /datasets)
(#37512)
add 492d90c396 Remove usage of deprecated `flask._request_ctx_stack`
(#37522)
add f33b68656a Simplify checks for package versions (#37585)
add 103a19e3f8 add airflow_home to loggrommersidecar (#37588)
add 52d2032887 Fix typo on DataflowStartFlexTemplateOperator documentation
(#37595)
add b38d59bdce Revert "Fix future DagRun rarely triggered by race
conditions when max_active_runs reached its upper limit. (#31414)" (#37596)
add 69d48ed7b5 Prepare docs ad hoc Teradata provider February 2024 (#37589)
add 581e2e42e9 Change AirflowTaskTimeout to inherit BaseException (#35653)
add 564adcfb7a Deprecate PY* constants into the airflow module (#37575)
add f971232ab4 Add conditional logic for dataset triggering (#37016)
add 2699f8ce39 Fix spelling of `airflow` (#37601)
add 2cb96a8e85 resolve template fields init checks for `bigquery` (#37586)
add 622d271048 Add tests to increase DagFileProcessorAgent coverage
(#37605)
add d5d6f4b188 Make Datasets hashable (#37465)
add c0e30cb3f8 Set parsing context dag_id in dag test command (#37606)
add 5fc866aebe `CloudRunExecuteJobOperator`: Add project_id to
hook.get_job calls (#37201)
add 810fb5f2a8 feat(GKEPodAsyncHook): use async credentials token
implementation (#37486)
add 0900055585 Refactor dataset class inheritance (#37590)
add 4e2f74333f Check if Python 3.12 is used for release management
commands as well (#37615)
add f9317611d7 Enhancing breeze commands with PACKAGE_LIST env variable
(#37502)
add 3c9863c8ea Introduce new config variable to control whether DAG
processor outputs to stdout (#37439)
add 1b7036ef05 Clarify skip-message for test requiring database. (#37612)
add 45eeff4476 Use new exception type inheriting BaseException for
SIGTERMs (#37613)
add 0c2d2c6544 Implement `batch_is_authorized_*` APIs in AWS auth manager
(#37430)
add 5289140a03 chore: Update comments and logging in OpenLineage
ExtractorManager (#37622)
add 3f8c0b9180 Show dataset events above task/run details in grid view
(#37603)
add 332e3cc349 Update min kubernetes version in docs and update release
instructions (#37623)
add b2f0279cc1 Prepare docs ad hoc Teradata provider RC2 February 2024
(#37624)
add 1c3789194c Add datasets to dag graph (#37604)
add 2cb78c4263 Filter datasets graph by dag_id (#37464)
add a6679413fa Fix task duration selection (#37630)
add ca4c559865 Fix Async GCSObjectsWithPrefixExistenceSensor xcom push
(#37634)
add 254d7eb898 Fix few small release issues found during 2.8.2 preparation
(#37633)
add 383b7b72c5 Update build dependencies to latest (#37639)
add 977d8cb5d9 Fix passing packages using arguments (#37636)
add cef52cbdbb Remove extra space in tests warning message (#37643)
add d663d8bc76 Clean up webserver endpoints adding to audit log (#37580)
add 8eb7d97e65 Fix passing packages as arguments to build-docs (#37646)
add 457a2fdd0c Small improvements in package preparation output (#37642)
add 2f4dda95a8 Add local airflow + configuration cleanup (#37644)
add 185e158b18 Correct wording typo in tests/conftest.py (#37645)
add 2294a2f5ff Install latest docker CLI instead of specific one (#37651)
add 3568b09e8c Fix unexpected keyword argument 'secret_path' in
secrets.kv.v2 (#37626)
add 0b9e2a78cc Add excluded/included events to get_event_logs api (#37641)
add 310a427ece Update PMC reproducibiliy check instructions (#37656)
add 8c05e59922 fix bq_to_mysql init checks (#37653)
add d2ba126404 Adding optional SSL verification for druid operator (#37629)
add 6a707e38c3 Tell users what to do if their scanners find issues in the
image (#37652)
add e0fc8034dc fix ImportError on examples dags (#37571)
add f36d17ce75 Implement `filter_permitted_menu_items` in AWS auth manager
(#37627)
add 0f4babed5a Fix hashicorp labels (#37663)
add a2db179d9c Init checks for aws gcs_to_s3 (#37662)
add bc34603123 Update to even newer version of trove classifier as build
dependency (#37669)
add 3bb6568cbe Add Python 3.12 support
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (c45ba0a9d8)
\
N -- N -- N refs/heads/python3.12 (3bb6568cbe)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.github/boring-cyborg.yml | 6 +-
.pre-commit-config.yaml | 3 -
Dockerfile | 25 +-
Dockerfile.ci | 25 +-
airflow/__init__.py | 26 +-
airflow/api/auth/backend/kerberos_auth.py | 66 +++--
.../api_connexion/endpoints/dataset_endpoint.py | 10 +
.../api_connexion/endpoints/event_log_endpoint.py | 8 +
.../endpoints/task_instance_endpoint.py | 1 -
airflow/api_connexion/openapi/v1.yaml | 25 ++
airflow/auth/managers/base_auth_manager.py | 49 ++--
airflow/auth/managers/models/batch_apis.py | 5 -
airflow/cli/commands/dag_command.py | 4 +-
airflow/config_templates/airflow_local_settings.py | 17 ++
airflow/config_templates/config.yml | 11 +-
airflow/datasets/__init__.py | 73 ++++-
airflow/example_dags/example_branch_operator.py | 217 +++++++-------
.../example_branch_operator_decorator.py | 155 +++++-----
airflow/exceptions.py | 9 +-
airflow/jobs/scheduler_job_runner.py | 17 +-
airflow/models/dag.py | 104 ++++---
airflow/models/dataset.py | 2 +-
airflow/models/taskinstance.py | 19 +-
.../amazon/aws/auth_manager/avp/facade.py | 130 ++++++++-
.../amazon/aws/auth_manager/aws_auth_manager.py | 325 ++++++++++++++++++++-
.../providers/amazon/aws/transfers/gcs_to_s3.py | 8 +-
airflow/providers/apache/druid/hooks/druid.py | 9 +-
airflow/providers/apache/druid/operators/druid.py | 6 +
.../celery/executors/celery_executor_utils.py | 6 +-
.../google/cloud/hooks/kubernetes_engine.py | 83 +++---
.../providers/google/cloud/operators/bigquery.py | 31 +-
.../providers/google/cloud/operators/cloud_run.py | 4 +-
.../providers/google/cloud/operators/dataflow.py | 4 +-
airflow/providers/google/cloud/sensors/gcs.py | 2 +
.../google/cloud/transfers/bigquery_to_mysql.py | 6 +-
.../hashicorp/_internal_client/vault_client.py | 4 +-
.../providers/openlineage/extractors/manager.py | 12 +-
airflow/providers/teradata/CHANGELOG.rst | 10 +-
airflow/providers/teradata/__init__.py | 12 +-
airflow/providers/teradata/provider.yaml | 2 +-
airflow/serialization/enums.py | 2 +
airflow/serialization/schema.json | 36 ++-
airflow/serialization/serialized_objects.py | 30 +-
airflow/timetables/datasets.py | 37 ++-
airflow/utils/context.pyi | 2 +-
airflow/utils/hashlib_wrapper.py | 5 +-
airflow/utils/pydantic.py | 12 +-
airflow/utils/sqlalchemy.py | 11 +-
airflow/utils/timezone.py | 4 +-
airflow/www/static/js/api/index.ts | 4 +-
.../www/static/js/api/useDatasetDependencies.ts | 121 +++++---
airflow/www/static/js/api/useDatasets.ts | 60 +---
.../api/{useDatasets.ts => useDatasetsSummary.ts} | 6 +-
.../www/static/js/api/useUpstreamDatasetEvents.ts | 2 +-
.../js/dag/details/dagRun/DatasetTriggerEvents.tsx | 13 +-
.../dag/details/dagRun/{index.tsx => Details.tsx} | 49 +---
airflow/www/static/js/dag/details/dagRun/index.tsx | 175 +----------
.../graph/{Node.test.tsx => DagNode.test.tsx} | 12 +-
.../js/dag/details/graph/{Node.tsx => DagNode.tsx} | 75 +----
.../static/js/dag/details/graph/DatasetNode.tsx | 112 +++++++
airflow/www/static/js/dag/details/graph/Node.tsx | 218 +++-----------
airflow/www/static/js/dag/details/graph/index.tsx | 61 +++-
airflow/www/static/js/dag/details/index.tsx | 1 +
.../details/taskInstance/DatasetUpdateEvents.tsx | 15 +-
.../static/js/dag/details/taskInstance/Details.tsx | 44 +--
.../js/dag/details/taskInstance/TriggererInfo.tsx | 65 +++++
.../static/js/dag/details/taskInstance/index.tsx | 6 +
airflow/www/static/js/datasets/DagFilter.tsx | 117 ++++++++
airflow/www/static/js/datasets/DatasetEvents.tsx | 7 +
airflow/www/static/js/datasets/Graph/DagNode.tsx | 37 ++-
airflow/www/static/js/datasets/Graph/Node.tsx | 11 +-
airflow/www/static/js/datasets/Graph/index.tsx | 100 ++++---
airflow/www/static/js/datasets/List.test.tsx | 20 +-
airflow/www/static/js/datasets/List.tsx | 35 ++-
airflow/www/static/js/datasets/Main.tsx | 49 +++-
airflow/www/static/js/types/api-generated.ts | 16 +
airflow/www/static/js/types/index.ts | 8 +-
airflow/www/static/js/utils/graph.ts | 3 +-
airflow/www/templates/airflow/dag.html | 1 +
airflow/www/templates/airflow/datasets.html | 2 +-
airflow/www/templates/appbuilder/navbar_menu.html | 2 +-
airflow/www/views.py | 43 +--
.../dag-processor/dag-processor-deployment.yaml | 2 +
.../templates/scheduler/scheduler-deployment.yaml | 2 +
.../templates/triggerer/triggerer-deployment.yaml | 2 +
chart/templates/workers/worker-deployment.yaml | 2 +
dev/README_RELEASE_AIRFLOW.md | 15 +-
dev/README_RELEASE_HELM_CHART.md | 10 +
dev/README_RELEASE_PROVIDER_PACKAGES.md | 52 ++++
dev/README_RELEASE_PYTHON_CLIENT.md | 2 +-
dev/breeze/doc/images/output_build-docs.svg | 40 ++-
dev/breeze/doc/images/output_build-docs.txt | 2 +-
...elease-management_prepare-provider-packages.svg | 36 ++-
...elease-management_prepare-provider-packages.txt | 2 +-
.../output_release-management_publish-docs.svg | 54 ++--
.../output_release-management_publish-docs.txt | 2 +-
.../src/airflow_breeze/commands/ci_commands.py | 6 +-
.../airflow_breeze/commands/developer_commands.py | 20 ++
.../commands/developer_commands_config.py | 1 +
.../src/airflow_breeze/commands/main_command.py | 9 +-
.../commands/release_candidate_command.py | 16 +-
.../commands/release_management_commands.py | 103 +++++--
.../commands/release_management_commands_config.py | 2 +
dev/breeze/src/airflow_breeze/utils/path_utils.py | 1 +
.../src/airflow_breeze/utils/python_versions.py | 15 +-
.../src/airflow_breeze/utils/reproducible.py | 6 +-
.../changelog.rst | 6 +
docs/apache-airflow-providers-teradata/commits.rst | 25 +-
docs/apache-airflow-providers-teradata/index.rst | 18 +-
docs/apache-airflow/core-concepts/auth-manager.rst | 2 +-
docs/conf.py | 2 +-
docs/docker-stack/index.rst | 133 +++++++--
docs/helm-chart/index.rst | 2 +-
hatch_build.py | 6 +-
newsfragments/35653.significant.rst | 21 ++
newsfragments/37627.significant.rst | 1 +
pyproject.toml | 17 +-
.../ci/pre_commit/pre_commit_compile_www_assets.py | 2 +-
scripts/cov/core_coverage.py | 2 +-
.../cov/{restapi_coverage.py => other_coverage.py} | 50 +++-
scripts/docker/install_os_dependencies.sh | 25 +-
.../in_container/run_prepare_airflow_packages.py | 29 ++
.../endpoints/test_dataset_endpoint.py | 60 +++-
.../endpoints/test_event_log_endpoint.py | 24 ++
tests/auth/managers/test_base_auth_manager.py | 4 +-
tests/charts/log_groomer.py | 12 +
tests/cli/commands/test_dag_command.py | 25 +-
tests/conftest.py | 9 +-
tests/core/test_airflow_module.py | 54 ++++
tests/core/test_core.py | 9 +-
tests/dag_processing/test_job_runner.py | 208 ++++++++++++-
tests/dag_processing/test_processor.py | 4 +-
...lid_dup_task.py => test_dag_parsing_context.py} | 20 +-
tests/datasets/test_dataset.py | 217 +++++++++++++-
tests/decorators/test_python.py | 3 +-
tests/models/test_taskinstance.py | 5 +-
tests/operators/test_python.py | 2 +-
.../amazon/aws/auth_manager/avp/test_facade.py | 129 ++++++++
.../aws/auth_manager/test_aws_auth_manager.py | 281 ++++++++++++++++++
tests/providers/apache/druid/hooks/test_druid.py | 19 ++
.../providers/apache/druid/operators/test_druid.py | 13 +-
tests/providers/apache/hive/hooks/test_hive.py | 11 +-
.../google/cloud/hooks/test_kubernetes_engine.py | 52 ++--
.../google/cloud/operators/test_cloud_run.py | 12 +
tests/providers/google/cloud/sensors/test_gcs.py | 14 +
.../utils/test_mlengine_prediction_summary.py | 7 +-
.../_internal_client/test_vault_client.py | 8 +-
tests/providers/hashicorp/hooks/test_vault.py | 6 +-
.../microsoft/azure/hooks/test_synapse.py | 3 +-
tests/providers/papermill/hooks/test_kernel.py | 5 +-
.../papermill/operators/test_papermill.py | 5 +-
.../serialization/serializers/test_serializers.py | 8 +-
tests/serialization/test_dag_serialization.py | 14 +-
tests/timetables/test_datasets_timetable.py | 1 -
tests/utils/test_sqlalchemy.py | 32 +-
tests/www/api/experimental/test_endpoints.py | 6 +-
tests/www/views/test_views_decorators.py | 36 +--
157 files changed, 3698 insertions(+), 1523 deletions(-)
copy airflow/www/static/js/api/{useDatasets.ts => useDatasetsSummary.ts} (92%)
copy airflow/www/static/js/dag/details/dagRun/{index.tsx => Details.tsx} (81%)
rename airflow/www/static/js/dag/details/graph/{Node.test.tsx =>
DagNode.test.tsx} (91%)
copy airflow/www/static/js/dag/details/graph/{Node.tsx => DagNode.tsx} (74%)
create mode 100644 airflow/www/static/js/dag/details/graph/DatasetNode.tsx
create mode 100644
airflow/www/static/js/dag/details/taskInstance/TriggererInfo.tsx
create mode 100644 airflow/www/static/js/datasets/DagFilter.tsx
create mode 100644 newsfragments/35653.significant.rst
create mode 100644 newsfragments/37627.significant.rst
copy scripts/cov/{restapi_coverage.py => other_coverage.py} (60%)
create mode 100644 tests/core/test_airflow_module.py
copy tests/dags/{test_invalid_dup_task.py => test_dag_parsing_context.py} (67%)