This is an automated email from the ASF dual-hosted git repository.
ephraimanierobi pushed a change to branch v2-8-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
omit d6c7d33d43 Update RELEASE_NOTES.rst
omit 729c1df9eb Update version to 2.8.0
new 1fc0633f89 Add v2-8 branches to codecov.yml and .asf.yaml (#35750)
new 70e2f419e6 Update minor release command (#35751)
new 6198558843 Remove backcompat with Airflow 2.3/2.4 in providers (#35727)
new b88d0d7ce2 OpenLineage integration tried to use non-existed method in
SnowflakeHook (#35752)
new 25990d159c Remove usage of deprecated method from
BigQueryToBigQueryOperator (#35605)
new bd644fab3e feature(providers): added `OpsgenieNotifier` (#35530)
new 38afe9ffb1 Rename --aditional-extras flag to
--aditional-airflow-extras (#35760)
new 9b832f6acd More detail on mandatory task arguments (#35740)
new 7642b29f5c Update README.md to reflect changes we agreed to the
versioning (#35764)
new 5d69fc142d Add basic metrics to stats collector. (#35368)
new d9cfdd8131 Extend task context logging support for remote logging
using Elasticsearch (#32977)
new 7310693277 Remove backcompat inheritance for DbApiHook (#35754)
new 26d5e3f4e7 Improve ownership fixing for Breeze (#35759)
new 1df306a337 Fix broken link to Weaviate docs (#35776)
new f40e1c17ee Set mark_end_on_close after set_context (#35761)
new c0a1dfe9ff Make passing build args explicit in ci/prod builds (#35768)
new 4454fc870c Move `BaseOperatorLink` into the separate module (#35032)
new b51aaf59d2 Add OpenLineage support to GCSToBigQueryOperator (#35778)
new e2e89668d4 Implement login and logout in AWS auth manager (#35488)
new feaeb8c5fb Check attr on parent not self re TaskContextLogger
set_context (#35780)
new 528d2bc51b Remove --force-build command in cache steps in CI (#35784)
new 599189e41e Fix DataFusion example type annotations (#35753)
new b904523c5a Fix permission check on menus (#35781)
new 2a240e65d5 Remove pendulum as dependency of breeze (#35786)
new 58743f28d5 Reflect drop/add support of DB Backends versions in
documentation (#35785)
new e0736092a8 Update emr.py (#35787)
new 7937cba0dc Add missing docker test_exceptions.py (#35674)
new 08785f50f1 added Topic params for schema_settings and
message_retention_duration. (#35767)
new 0e6b5d8f8e Added retry strategy parameter to Amazon AWS provider Batch
Operator to allow dynamic Batch retry strategies (#35789)
new 673f7f837f Fix for infinite recursion due to secrets_masker (#35048)
new d568543d87 feat: K8S resource operator - CRD (#35600)
new b331cb2e0e Upgrade to Pydantic v2 (#35551)
new e022710039 Fix HttpOperator pagination with `str` data (#35782)
new 840707cedf Updated docstring: `check_key_async` is now in line with
description of `_check_key_async` (#35799)
new 55914e14cb Add borderWidthRight to grid for Firefox scrollbar (#35346)
new 20cb92e52a Update version to 2.8.0
new 5bbc46005f Update RELEASE_NOTES.rst
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (d6c7d33d43)
\
N -- N -- N refs/heads/v2-8-test (5bbc46005f)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 37 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.asf.yaml | 3 +
.github/actions/build-ci-images/action.yml | 4 -
.github/actions/build-prod-images/action.yml | 4 -
.github/actions/post_tests_failure/action.yml | 3 -
.github/actions/post_tests_success/action.yml | 3 -
.github/workflows/build-images.yml | 3 -
.github/workflows/ci.yml | 55 ---
.github/workflows/release_dockerhub_image.yml | 3 -
.pre-commit-config.yaml | 23 +-
BREEZE.rst | 2 +-
CONTRIBUTING.rst | 6 +-
Dockerfile | 4 +-
Dockerfile.ci | 6 +-
IMAGES.rst | 10 +-
INSTALL | 6 +-
README.md | 17 +-
RELEASE_NOTES.rst | 7 +
STATIC_CODE_CHECKS.rst | 3 +-
airflow/auth/managers/base_auth_manager.py | 32 +-
airflow/auth/managers/fab/fab_auth_manager.py | 16 +-
airflow/auth/managers/fab/models/__init__.py | 3 +
airflow/auth/managers/models/base_user.py | 7 +-
airflow/configuration.py | 4 -
airflow/models/__init__.py | 5 +-
airflow/models/abstractoperator.py | 3 +-
airflow/models/baseoperator.py | 49 ++-
airflow/models/baseoperatorlink.py | 57 ++++
airflow/models/mappedoperator.py | 3 +-
airflow/operators/trigger_dagrun.py | 3 +-
.../amazon/aws/auth_manager}/__init__.py | 0
.../amazon/aws/auth_manager/aws_auth_manager.py | 143 ++++++++
.../providers/amazon/aws/auth_manager/constants.py | 10 +-
.../aws/auth_manager/security_manager}/__init__.py | 0
.../aws_security_manager_override.py} | 25 +-
airflow/providers/amazon/aws/auth_manager/user.py | 51 +++
.../amazon/aws/auth_manager/views}/__init__.py | 0
.../amazon/aws/auth_manager/views/auth.py | 149 ++++++++
airflow/providers/amazon/aws/hooks/s3.py | 6 +-
.../providers/amazon/aws/log/s3_task_handler.py | 4 +-
airflow/providers/amazon/aws/operators/batch.py | 5 +
airflow/providers/amazon/aws/operators/emr.py | 18 +-
airflow/providers/amazon/provider.yaml | 25 ++
airflow/providers/apache/impala/hooks/impala.py | 3 +-
.../cncf/kubernetes/operators/resource.py | 62 +++-
.../cncf/kubernetes/utils/k8s_resource_iterator.py | 46 +++
airflow/providers/common/sql/hooks/sql.py | 18 +-
airflow/providers/common/sql/hooks/sql.pyi | 4 +-
.../providers/elasticsearch/hooks/elasticsearch.py | 4 +-
.../providers/elasticsearch/log/es_task_handler.py | 48 ++-
airflow/providers/google/cloud/hooks/gcs.py | 9 +-
airflow/providers/google/cloud/hooks/pubsub.py | 10 +
.../providers/google/cloud/log/gcs_task_handler.py | 4 +-
airflow/providers/google/cloud/operators/pubsub.py | 7 +
.../google/cloud/secrets/secret_manager.py | 14 +-
.../google/cloud/transfers/bigquery_to_bigquery.py | 88 +++--
.../google/cloud/transfers/gcs_to_bigquery.py | 77 +++++
airflow/providers/http/operators/http.py | 50 ++-
airflow/providers/http/triggers/http.py | 2 +-
.../microsoft/azure/log/wasb_task_handler.py | 4 +-
.../providers/microsoft/azure/secrets/key_vault.py | 14 +-
airflow/providers/openlineage/plugins/adapter.py | 5 +-
.../opsgenie/notifications}/__init__.py | 0
.../providers/opsgenie/notifications/opsgenie.py | 81 +++++
airflow/providers/opsgenie/provider.yaml | 3 +
.../opsgenie/typing}/__init__.py | 0
airflow/providers/opsgenie/typing/opsgenie.py | 59 ++++
airflow/providers/snowflake/hooks/snowflake.py | 2 +-
airflow/providers/trino/hooks/trino.py | 19 +-
airflow/sensors/external_task.py | 2 +-
airflow/serialization/pydantic/dag.py | 29 +-
airflow/serialization/pydantic/dag_run.py | 9 +-
airflow/serialization/pydantic/dataset.py | 27 +-
airflow/serialization/pydantic/job.py | 8 +-
airflow/serialization/pydantic/taskinstance.py | 9 +-
airflow/serialization/serde.py | 9 +-
airflow/serialization/serialized_objects.py | 7 +-
airflow/utils/log/secrets_masker.py | 9 +-
airflow/utils/log/task_context_logger.py | 2 +-
airflow/www/extensions/init_appbuilder.py | 2 +-
airflow/www/extensions/init_auth_manager.py | 6 +-
airflow/www/extensions/init_security.py | 4 +-
airflow/www/security_manager.py | 56 +--
airflow/www/static/js/dag/grid/index.tsx | 2 +-
airflow/www/utils.py | 5 +-
codecov.yml | 2 +
dev/breeze/README.md | 2 +-
...-root-ownership-after-exiting-docker-command.md | 60 ++++
dev/breeze/pyproject.toml | 1 -
.../airflow_breeze/commands/ci_image_commands.py | 119 +++++--
.../commands/ci_image_commands_config.py | 29 +-
.../airflow_breeze/commands/developer_commands.py | 7 +
.../commands/minor_release_command.py | 1 +
.../commands/production_image_commands.py | 160 +++++++--
.../commands/production_image_commands_config.py | 46 ++-
.../commands/release_management_commands.py | 21 ++
.../airflow_breeze/commands/testing_commands.py | 5 +
.../src/airflow_breeze/params/build_ci_params.py | 113 +++---
.../src/airflow_breeze/params/build_prod_params.py | 115 +++----
.../airflow_breeze/params/common_build_params.py | 84 +++--
.../src/airflow_breeze/utils/common_options.py | 14 +-
.../airflow_breeze/utils/docker_command_utils.py | 59 +---
dev/breeze/src/airflow_breeze/utils/parallel.py | 6 +
dev/breeze/src/airflow_breeze/utils/run_utils.py | 2 +-
docs/apache-airflow-providers-opsgenie/index.rst | 1 +
.../notifications/index.rst | 30 ++
.../notifications/opsgenie_notifier.rst | 33 ++
.../connections.rst | 2 +-
.../administration-and-deployment/scheduler.rst | 14 +-
docs/apache-airflow/extra-packages-ref.rst | 2 +
docs/apache-airflow/howto/define-extra-link.rst | 9 +-
docs/apache-airflow/howto/set-up-database.rst | 10 +-
docs/apache-airflow/installation/prerequisites.rst | 11 +-
docs/apache-airflow/public-airflow-interface.rst | 7 +
docs/apache-airflow/tutorial/fundamentals.rst | 7 +-
docs/conf.py | 1 +
docs/docker-stack/changelog.rst | 3 +
docs/spelling_wordlist.txt | 1 +
generated/PYPI_README.md | 2 +-
images/breeze/output_ci-image_build.svg | 160 +++++----
images/breeze/output_ci-image_build.txt | 2 +-
images/breeze/output_prod-image_build.svg | 190 ++++++-----
images/breeze/output_prod-image_build.txt | 2 +-
..._release-management_prepare-airflow-package.txt | 2 +-
...elease-management_prepare-provider-packages.txt | 2 +-
pyproject.toml | 1 +
scripts/docker/entrypoint_ci.sh | 2 -
scripts/docker/install_os_dependencies.sh | 4 +-
scripts/in_container/_in_container_utils.sh | 40 ---
scripts/in_container/run_fix_ownership.py | 84 +++++
.../in_container/run_prepare_airflow_packages.py | 12 +-
setup.cfg | 6 +-
setup.py | 7 +
tests/always/test_project_structure.py | 1 -
.../endpoints/test_extra_link_endpoint.py | 2 +-
.../endpoints/test_forward_to_fab_endpoint.py | 2 +-
.../endpoints/test_plugin_endpoint.py | 2 +-
tests/api_connexion/schemas/test_plugin_schema.py | 2 +-
tests/auth/managers/fab/test_fab_auth_manager.py | 38 +--
tests/auth/managers/test_base_auth_manager.py | 48 ++-
.../providers/amazon/aws/auth_manager}/__init__.py | 0
.../aws/auth_manager/security_manager}/__init__.py | 0
.../test_aws_security_manager_override.py | 56 +++
.../aws/auth_manager/test_aws_auth_manager.py | 114 +++++++
.../amazon/aws/auth_manager/test_constants.py | 23 +-
.../providers/amazon/aws/auth_manager/test_user.py | 32 +-
.../amazon/aws/auth_manager/views}/__init__.py | 0
.../amazon/aws/auth_manager/views/test_auth.py | 156 +++++++++
tests/providers/amazon/aws/operators/test_batch.py | 6 +
.../cncf/kubernetes/operators/test_resource.py | 63 ++++
.../kubernetes/utils/test_k8s_resource_iterator.py | 65 ++++
tests/providers/docker/test_exceptions.py | 80 +++++
tests/providers/google/cloud/hooks/test_pubsub.py | 9 +-
.../google/cloud/operators/test_pubsub.py | 4 +
.../cloud/transfers/test_bigquery_to_bigquery.py | 94 +++--
.../google/cloud/transfers/test_gcs_to_bigquery.py | 380 ++++++++++++++++++++-
tests/providers/http/operators/test_http.py | 63 +++-
tests/providers/odbc/hooks/test_odbc.py | 3 +-
.../plugins/test_openlineage_adapter.py | 75 +++-
.../providers/opsgenie/notifications}/__init__.py | 0
.../opsgenie/notifications/test_opsgenie.py | 119 +++++++
.../providers/opsgenie/typing}/__init__.py | 0
.../providers/opsgenie/typing/test_opsgenie.py | 16 +-
tests/providers/trino/hooks/test_trino.py | 38 ++-
tests/serialization/test_dag_serialization.py | 3 +-
.../google/cloud/datafusion/example_datafusion.py | 4 +-
.../opsgenie/example_opsgenie_notifier.py | 31 +-
tests/test_utils/mock_operators.py | 3 +-
tests/utils/log/test_secrets_masker.py | 18 +
tests/www/test_security_manager.py | 115 +++++++
tests/www/views/test_views_extra_links.py | 3 +-
170 files changed, 3609 insertions(+), 1151 deletions(-)
create mode 100644 airflow/models/baseoperatorlink.py
copy airflow/{api_connexion => providers/amazon/aws/auth_manager}/__init__.py
(100%)
create mode 100644
airflow/providers/amazon/aws/auth_manager/aws_auth_manager.py
rename scripts/in_container/run_fix_ownership.sh =>
airflow/providers/amazon/aws/auth_manager/constants.py (82%)
mode change 100755 => 100644
copy airflow/{api_connexion =>
providers/amazon/aws/auth_manager/security_manager}/__init__.py (100%)
copy airflow/{auth/managers/models/base_user.py =>
providers/amazon/aws/auth_manager/security_manager/aws_security_manager_override.py}
(52%)
create mode 100644 airflow/providers/amazon/aws/auth_manager/user.py
copy airflow/{api_connexion =>
providers/amazon/aws/auth_manager/views}/__init__.py (100%)
create mode 100644 airflow/providers/amazon/aws/auth_manager/views/auth.py
create mode 100644
airflow/providers/cncf/kubernetes/utils/k8s_resource_iterator.py
copy airflow/{api_connexion => providers/opsgenie/notifications}/__init__.py
(100%)
create mode 100644 airflow/providers/opsgenie/notifications/opsgenie.py
copy airflow/{api_connexion => providers/opsgenie/typing}/__init__.py (100%)
create mode 100644 airflow/providers/opsgenie/typing/opsgenie.py
create mode 100644
dev/breeze/doc/adr/0014-fix-root-ownership-after-exiting-docker-command.md
create mode 100644
docs/apache-airflow-providers-opsgenie/notifications/index.rst
create mode 100644
docs/apache-airflow-providers-opsgenie/notifications/opsgenie_notifier.rst
create mode 100755 scripts/in_container/run_fix_ownership.py
copy {airflow/api_connexion =>
tests/providers/amazon/aws/auth_manager}/__init__.py (100%)
copy {airflow/api_connexion =>
tests/providers/amazon/aws/auth_manager/security_manager}/__init__.py (100%)
create mode 100644
tests/providers/amazon/aws/auth_manager/security_manager/test_aws_security_manager_override.py
create mode 100644
tests/providers/amazon/aws/auth_manager/test_aws_auth_manager.py
copy airflow/auth/managers/models/base_user.py =>
tests/providers/amazon/aws/auth_manager/test_constants.py (64%)
copy airflow/auth/managers/models/base_user.py =>
tests/providers/amazon/aws/auth_manager/test_user.py (55%)
copy {airflow/api_connexion =>
tests/providers/amazon/aws/auth_manager/views}/__init__.py (100%)
create mode 100644 tests/providers/amazon/aws/auth_manager/views/test_auth.py
create mode 100644
tests/providers/cncf/kubernetes/utils/test_k8s_resource_iterator.py
create mode 100644 tests/providers/docker/test_exceptions.py
copy {airflow/api_connexion =>
tests/providers/opsgenie/notifications}/__init__.py (100%)
create mode 100644 tests/providers/opsgenie/notifications/test_opsgenie.py
copy {airflow/api_connexion => tests/providers/opsgenie/typing}/__init__.py
(100%)
copy airflow/auth/managers/models/base_user.py =>
tests/providers/opsgenie/typing/test_opsgenie.py (74%)
copy airflow/auth/managers/models/base_user.py =>
tests/system/providers/opsgenie/example_opsgenie_notifier.py (51%)
create mode 100644 tests/www/test_security_manager.py
