(airflow) branch main updated: Fix SNOWFLAKE_CONN_ID and DAG_ID in Snowpark system tests (#42952)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new db9c130a5c Fix SNOWFLAKE_CONN_ID and DAG_ID in Snowpark system tests (#42952) db9c130a5c is described below commit db9c130a5c987a12ee0f1dba624275c5844c226c Author: Jianzhun Du <68252326+sfc-gh-...@users.noreply.github.com> AuthorDate: Fri Oct 11 17:41:32 2024 -0700 Fix SNOWFLAKE_CONN_ID and DAG_ID in Snowpark system tests (#42952) --- providers/tests/system/snowflake/example_snowpark_decorator.py | 4 ++-- providers/tests/system/snowflake/example_snowpark_operator.py | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/providers/tests/system/snowflake/example_snowpark_decorator.py b/providers/tests/system/snowflake/example_snowpark_decorator.py index 3f74b97d0f..4c2319a91b 100644 --- a/providers/tests/system/snowflake/example_snowpark_decorator.py +++ b/providers/tests/system/snowflake/example_snowpark_decorator.py @@ -30,8 +30,8 @@ if TYPE_CHECKING: from airflow import DAG from airflow.decorators import task -SNOWFLAKE_CONN_ID = "snowflake_default" -DAG_ID = "example_snowpark" +SNOWFLAKE_CONN_ID = "my_snowflake_conn" +DAG_ID = "example_snowpark_decorator" with DAG( DAG_ID, diff --git a/providers/tests/system/snowflake/example_snowpark_operator.py b/providers/tests/system/snowflake/example_snowpark_operator.py index 0543596ec9..1d7cce34f7 100644 --- a/providers/tests/system/snowflake/example_snowpark_operator.py +++ b/providers/tests/system/snowflake/example_snowpark_operator.py @@ -30,8 +30,8 @@ if TYPE_CHECKING: from airflow import DAG from airflow.providers.snowflake.operators.snowpark import SnowparkOperator -SNOWFLAKE_CONN_ID = "snowflake_default" -DAG_ID = "example_snowpark" +SNOWFLAKE_CONN_ID = "my_snowflake_conn" +DAG_ID = "example_snowpark_operator" with DAG( DAG_ID,
(airflow-site) branch add-documentation-2024-10-10 created (now 964acc0f27)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch add-documentation-2024-10-10 in repository https://gitbox.apache.org/repos/asf/airflow-site.git at 964acc0f27 Add documentation for packages - 2024-10-10 This branch includes the following new commits: new 964acc0f27 Add documentation for packages - 2024-10-10 The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
svn commit: r72229 [1/2] - /dev/airflow/providers/
Author: eladkal Date: Wed Oct 9 16:17:47 2024 New Revision: 72229 Log: Add artifacts for Airflow Providers 2024-10-09 Added: dev/airflow/providers/apache_airflow_providers_amazon-9.0.0-py3-none-any.whl (with props) dev/airflow/providers/apache_airflow_providers_amazon-9.0.0-py3-none-any.whl.asc dev/airflow/providers/apache_airflow_providers_amazon-9.0.0-py3-none-any.whl.sha512 dev/airflow/providers/apache_airflow_providers_amazon-9.0.0.tar.gz (with props) dev/airflow/providers/apache_airflow_providers_amazon-9.0.0.tar.gz.asc dev/airflow/providers/apache_airflow_providers_amazon-9.0.0.tar.gz.sha512 dev/airflow/providers/apache_airflow_providers_apache_beam-5.8.1-py3-none-any.whl (with props) dev/airflow/providers/apache_airflow_providers_apache_beam-5.8.1-py3-none-any.whl.asc dev/airflow/providers/apache_airflow_providers_apache_beam-5.8.1-py3-none-any.whl.sha512 dev/airflow/providers/apache_airflow_providers_apache_beam-5.8.1.tar.gz (with props) dev/airflow/providers/apache_airflow_providers_apache_beam-5.8.1.tar.gz.asc dev/airflow/providers/apache_airflow_providers_apache_beam-5.8.1.tar.gz.sha512 dev/airflow/providers/apache_airflow_providers_apache_kafka-1.6.1-py3-none-any.whl (with props) dev/airflow/providers/apache_airflow_providers_apache_kafka-1.6.1-py3-none-any.whl.asc dev/airflow/providers/apache_airflow_providers_apache_kafka-1.6.1-py3-none-any.whl.sha512 dev/airflow/providers/apache_airflow_providers_apache_kafka-1.6.1.tar.gz (with props) dev/airflow/providers/apache_airflow_providers_apache_kafka-1.6.1.tar.gz.asc dev/airflow/providers/apache_airflow_providers_apache_kafka-1.6.1.tar.gz.sha512 dev/airflow/providers/apache_airflow_providers_apache_spark-4.11.1-py3-none-any.whl (with props) dev/airflow/providers/apache_airflow_providers_apache_spark-4.11.1-py3-none-any.whl.asc dev/airflow/providers/apache_airflow_providers_apache_spark-4.11.1-py3-none-any.whl.sha512 dev/airflow/providers/apache_airflow_providers_apache_spark-4.11.1.tar.gz (with props) dev/airflow/providers/apache_airflow_providers_apache_spark-4.11.1.tar.gz.asc dev/airflow/providers/apache_airflow_providers_apache_spark-4.11.1.tar.gz.sha512 dev/airflow/providers/apache_airflow_providers_celery-3.8.3-py3-none-any.whl (with props) dev/airflow/providers/apache_airflow_providers_celery-3.8.3-py3-none-any.whl.asc dev/airflow/providers/apache_airflow_providers_celery-3.8.3-py3-none-any.whl.sha512 dev/airflow/providers/apache_airflow_providers_celery-3.8.3.tar.gz (with props) dev/airflow/providers/apache_airflow_providers_celery-3.8.3.tar.gz.asc dev/airflow/providers/apache_airflow_providers_celery-3.8.3.tar.gz.sha512 dev/airflow/providers/apache_airflow_providers_cloudant-4.0.1-py3-none-any.whl (with props) dev/airflow/providers/apache_airflow_providers_cloudant-4.0.1-py3-none-any.whl.asc dev/airflow/providers/apache_airflow_providers_cloudant-4.0.1-py3-none-any.whl.sha512 dev/airflow/providers/apache_airflow_providers_cloudant-4.0.1.tar.gz (with props) dev/airflow/providers/apache_airflow_providers_cloudant-4.0.1.tar.gz.asc dev/airflow/providers/apache_airflow_providers_cloudant-4.0.1.tar.gz.sha512 dev/airflow/providers/apache_airflow_providers_cncf_kubernetes-9.0.0-py3-none-any.whl (with props) dev/airflow/providers/apache_airflow_providers_cncf_kubernetes-9.0.0-py3-none-any.whl.asc dev/airflow/providers/apache_airflow_providers_cncf_kubernetes-9.0.0-py3-none-any.whl.sha512 dev/airflow/providers/apache_airflow_providers_cncf_kubernetes-9.0.0.tar.gz (with props) dev/airflow/providers/apache_airflow_providers_cncf_kubernetes-9.0.0.tar.gz.asc dev/airflow/providers/apache_airflow_providers_cncf_kubernetes-9.0.0.tar.gz.sha512 dev/airflow/providers/apache_airflow_providers_common_compat-1.2.1-py3-none-any.whl (with props) dev/airflow/providers/apache_airflow_providers_common_compat-1.2.1-py3-none-any.whl.asc dev/airflow/providers/apache_airflow_providers_common_compat-1.2.1-py3-none-any.whl.sha512 dev/airflow/providers/apache_airflow_providers_common_compat-1.2.1.tar.gz (with props) dev/airflow/providers/apache_airflow_providers_common_compat-1.2.1.tar.gz.asc dev/airflow/providers/apache_airflow_providers_common_compat-1.2.1.tar.gz.sha512 dev/airflow/providers/apache_airflow_providers_common_io-1.4.2-py3-none-any.whl (with props) dev/airflow/providers/apache_airflow_providers_common_io-1.4.2-py3-none-any.whl.asc dev/airflow/providers/apache_airflow_providers_common_io-1.4.2-py3-none-any.whl.sha512 dev/airflow/providers/apache_airflow_providers_common_io-1.4.2.tar.gz (with props) dev/airflow/providers/apache_airflow_providers_common_io-1.4.2.tar.gz.asc dev/airflow/providers/apache_airflow_providers_common_io-1.4.2.tar.gz.sha512 dev
svn commit: r72229 [2/2] - /dev/airflow/providers/
Added: dev/airflow/providers/apache_airflow_providers_ydb-1.4.0.tar.gz.asc == --- dev/airflow/providers/apache_airflow_providers_ydb-1.4.0.tar.gz.asc (added) +++ dev/airflow/providers/apache_airflow_providers_ydb-1.4.0.tar.gz.asc Wed Oct 9 16:17:47 2024 @@ -0,0 +1,7 @@ +-BEGIN PGP SIGNATURE- + +iIkEABYKADEWIQSDQO8ECQokO9vDRUWG4IhmPszevgUCZwar8RMcZWxhZGthbEBh +cGFjaGUub3JnAAoJEIbgiGY+zN6+EgAA/0C7pbIvP9CZfikK1jeCaXs50PFxabu+ +VMWQ0uQ9bjTbAQCT0LN3hYuYPayXr55gGJAdUqzbTnI0jwNF2QRTndtADQ== +=2geR +-END PGP SIGNATURE- Added: dev/airflow/providers/apache_airflow_providers_ydb-1.4.0.tar.gz.sha512 == --- dev/airflow/providers/apache_airflow_providers_ydb-1.4.0.tar.gz.sha512 (added) +++ dev/airflow/providers/apache_airflow_providers_ydb-1.4.0.tar.gz.sha512 Wed Oct 9 16:17:47 2024 @@ -0,0 +1 @@ +f0a277c47958694ed2722ed637d271ab932664704c0f00cc91bfe5a54e318fea5c9512d9c4bd2f5e3f4490b10be7829dcb1931e07fb2a2a7caf83543b1f12417 apache_airflow_providers_ydb-1.4.0.tar.gz
(airflow) branch main updated: Revert "Remove `sqlalchemy-redshift` dependency from Amazon provider (#42830)" (#42864)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new d395c1f1f8d Revert "Remove `sqlalchemy-redshift` dependency from Amazon provider (#42830)" (#42864) d395c1f1f8d is described below commit d395c1f1f8dad18a1daf5275a522f4f62bcf6aec Author: Maciej Obuchowski AuthorDate: Wed Oct 9 17:28:06 2024 +0200 Revert "Remove `sqlalchemy-redshift` dependency from Amazon provider (#42830)" (#42864) This reverts commit 3b4c73a644f40013927fb72227dfe018dc47a97f. --- airflow/providers/amazon/provider.yaml | 1 + docs/apache-airflow-providers-amazon/index.rst | 1 + generated/provider_dependencies.json | 1 + 3 files changed, 3 insertions(+) diff --git a/airflow/providers/amazon/provider.yaml b/airflow/providers/amazon/provider.yaml index 741efb70e7e..1316cd05231 100644 --- a/airflow/providers/amazon/provider.yaml +++ b/airflow/providers/amazon/provider.yaml @@ -105,6 +105,7 @@ dependencies: - watchtower>=3.0.0,!=3.3.0,<4 - jsonpath_ng>=1.5.3 - redshift_connector>=2.0.918 + - sqlalchemy_redshift>=0.8.6 - asgiref>=2.3.0 - PyAthena>=3.0.10 - jmespath>=0.7.0 diff --git a/docs/apache-airflow-providers-amazon/index.rst b/docs/apache-airflow-providers-amazon/index.rst index 88e869bba80..c58847dcd88 100644 --- a/docs/apache-airflow-providers-amazon/index.rst +++ b/docs/apache-airflow-providers-amazon/index.rst @@ -119,6 +119,7 @@ PIP package Version required ``watchtower`` ``>=3.0.0,!=3.3.0,<4`` ``jsonpath_ng`` ``>=1.5.3`` ``redshift_connector`` ``>=2.0.918`` +``sqlalchemy_redshift`` ``>=0.8.6`` ``asgiref`` ``>=2.3.0`` ``PyAthena````>=3.0.10`` ``jmespath````>=0.7.0`` diff --git a/generated/provider_dependencies.json b/generated/provider_dependencies.json index 57bca636ed6..2d0ab90a350 100644 --- a/generated/provider_dependencies.json +++ b/generated/provider_dependencies.json @@ -38,6 +38,7 @@ "jsonpath_ng>=1.5.3", "python3-saml>=1.16.0", "redshift_connector>=2.0.918", + "sqlalchemy_redshift>=0.8.6", "watchtower>=3.0.0,!=3.3.0,<4" ], "devel-deps": [
(airflow) branch revert-42830-vincbeck/sqlalchemy-redshift deleted (was 502d0cb710d)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch revert-42830-vincbeck/sqlalchemy-redshift in repository https://gitbox.apache.org/repos/asf/airflow.git was 502d0cb710d Revert "Remove `sqlalchemy-redshift` dependency from Amazon provider (#42830)" The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
(airflow) branch main updated (0711581891 -> b0a18d9019)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 0711581891 Typo correction in 07_local_virtualenv.rst (#42727) add b0a18d9019 Handle empty login and password with opensearch client (#39982) No new revisions were added by this update. Summary of changes: airflow/providers/opensearch/hooks/opensearch.py | 19 +++--- .../providers/opensearch/hooks/test_opensearch.py | 29 ++ 2 files changed, 44 insertions(+), 4 deletions(-)
svn commit: r72007 - /dev/airflow/providers/ /release/airflow/providers/
Author: eladkal Date: Tue Oct 1 09:03:29 2024 New Revision: 72007 Log: Release Airflow Providers on 2024-10-01 Added: release/airflow/providers/apache_airflow_providers_common_sql-1.17.1-py3-none-any.whl - copied unchanged from r72006, dev/airflow/providers/apache_airflow_providers_common_sql-1.17.1-py3-none-any.whl release/airflow/providers/apache_airflow_providers_common_sql-1.17.1-py3-none-any.whl.asc - copied unchanged from r72006, dev/airflow/providers/apache_airflow_providers_common_sql-1.17.1-py3-none-any.whl.asc release/airflow/providers/apache_airflow_providers_common_sql-1.17.1-py3-none-any.whl.sha512 - copied unchanged from r72006, dev/airflow/providers/apache_airflow_providers_common_sql-1.17.1-py3-none-any.whl.sha512 release/airflow/providers/apache_airflow_providers_common_sql-1.17.1.tar.gz - copied unchanged from r72006, dev/airflow/providers/apache_airflow_providers_common_sql-1.17.1.tar.gz release/airflow/providers/apache_airflow_providers_common_sql-1.17.1.tar.gz.asc - copied unchanged from r72006, dev/airflow/providers/apache_airflow_providers_common_sql-1.17.1.tar.gz.asc release/airflow/providers/apache_airflow_providers_common_sql-1.17.1.tar.gz.sha512 - copied unchanged from r72006, dev/airflow/providers/apache_airflow_providers_common_sql-1.17.1.tar.gz.sha512 release/airflow/providers/apache_airflow_providers_openlineage-1.12.1-py3-none-any.whl - copied unchanged from r72006, dev/airflow/providers/apache_airflow_providers_openlineage-1.12.1-py3-none-any.whl release/airflow/providers/apache_airflow_providers_openlineage-1.12.1-py3-none-any.whl.asc - copied unchanged from r72006, dev/airflow/providers/apache_airflow_providers_openlineage-1.12.1-py3-none-any.whl.asc release/airflow/providers/apache_airflow_providers_openlineage-1.12.1-py3-none-any.whl.sha512 - copied unchanged from r72006, dev/airflow/providers/apache_airflow_providers_openlineage-1.12.1-py3-none-any.whl.sha512 release/airflow/providers/apache_airflow_providers_openlineage-1.12.1.tar.gz - copied unchanged from r72006, dev/airflow/providers/apache_airflow_providers_openlineage-1.12.1.tar.gz release/airflow/providers/apache_airflow_providers_openlineage-1.12.1.tar.gz.asc - copied unchanged from r72006, dev/airflow/providers/apache_airflow_providers_openlineage-1.12.1.tar.gz.asc release/airflow/providers/apache_airflow_providers_openlineage-1.12.1.tar.gz.sha512 - copied unchanged from r72006, dev/airflow/providers/apache_airflow_providers_openlineage-1.12.1.tar.gz.sha512 Removed: dev/airflow/providers/apache_airflow_providers_common_sql-1.17.1-py3-none-any.whl dev/airflow/providers/apache_airflow_providers_common_sql-1.17.1-py3-none-any.whl.asc dev/airflow/providers/apache_airflow_providers_common_sql-1.17.1-py3-none-any.whl.sha512 dev/airflow/providers/apache_airflow_providers_common_sql-1.17.1.tar.gz dev/airflow/providers/apache_airflow_providers_common_sql-1.17.1.tar.gz.asc dev/airflow/providers/apache_airflow_providers_common_sql-1.17.1.tar.gz.sha512 dev/airflow/providers/apache_airflow_providers_openlineage-1.12.1-py3-none-any.whl dev/airflow/providers/apache_airflow_providers_openlineage-1.12.1-py3-none-any.whl.asc dev/airflow/providers/apache_airflow_providers_openlineage-1.12.1-py3-none-any.whl.sha512 dev/airflow/providers/apache_airflow_providers_openlineage-1.12.1.tar.gz dev/airflow/providers/apache_airflow_providers_openlineage-1.12.1.tar.gz.asc dev/airflow/providers/apache_airflow_providers_openlineage-1.12.1.tar.gz.sha512 release/airflow/providers/apache_airflow_providers_common_sql-1.17.0-py3-none-any.whl release/airflow/providers/apache_airflow_providers_common_sql-1.17.0-py3-none-any.whl.asc release/airflow/providers/apache_airflow_providers_common_sql-1.17.0-py3-none-any.whl.sha512 release/airflow/providers/apache_airflow_providers_common_sql-1.17.0.tar.gz release/airflow/providers/apache_airflow_providers_common_sql-1.17.0.tar.gz.asc release/airflow/providers/apache_airflow_providers_common_sql-1.17.0.tar.gz.sha512 release/airflow/providers/apache_airflow_providers_openlineage-1.12.0-py3-none-any.whl release/airflow/providers/apache_airflow_providers_openlineage-1.12.0-py3-none-any.whl.asc release/airflow/providers/apache_airflow_providers_openlineage-1.12.0-py3-none-any.whl.sha512 release/airflow/providers/apache_airflow_providers_openlineage-1.12.0.tar.gz release/airflow/providers/apache_airflow_providers_openlineage-1.12.0.tar.gz.asc release/airflow/providers/apache_airflow_providers_openlineage-1.12.0.tar.gz.sha512
(airflow-site) branch add-documentation-2024-09-27 deleted (was af5a765605)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch add-documentation-2024-09-27 in repository https://gitbox.apache.org/repos/asf/airflow-site.git was af5a765605 Add documentation for packages - 2024-09-27 The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
(airflow) annotated tag providers-common-sql/1.17.1 updated (bcee926d49 -> 3202c4efc2)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to annotated tag providers-common-sql/1.17.1 in repository https://gitbox.apache.org/repos/asf/airflow.git *** WARNING: tag providers-common-sql/1.17.1 was modified! *** from bcee926d49 (commit) to 3202c4efc2 (tag) tagging bcee926d494cabf4ddfa9f2569e36acc5b4d281d (commit) replaces providers-airbyte/4.0.0 by Elad Kalif on Tue Oct 1 12:05:14 2024 +0300 - Log - Release 2024-10-01 of providers --- No new revisions were added by this update. Summary of changes:
(airflow) branch main updated: Update providers metadata 2024-10-01 (#42611)
This is an automated email from the ASF dual-hosted git repository.
eladkal pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new 05c43eeacc Update providers metadata 2024-10-01 (#42611)
05c43eeacc is described below
commit 05c43eeacc537cfa6b1affa6fdd8c3202f70c14b
Author: Elad Kalif <45845474+elad...@users.noreply.github.com>
AuthorDate: Tue Oct 1 22:31:17 2024 +0700
Update providers metadata 2024-10-01 (#42611)
---
generated/provider_metadata.json | 8
1 file changed, 8 insertions(+)
diff --git a/generated/provider_metadata.json b/generated/provider_metadata.json
index a73e3da9f6..56199e2f82 100644
--- a/generated/provider_metadata.json
+++ b/generated/provider_metadata.json
@@ -2763,6 +2763,10 @@
"1.17.0": {
"associated_airflow_version": "2.10.1",
"date_released": "2024-09-24T13:49:56Z"
+},
+"1.17.1": {
+"associated_airflow_version": "2.10.1",
+"date_released": "2024-10-01T09:05:14Z"
}
},
"databricks": {
@@ -6225,6 +6229,10 @@
"1.12.0": {
"associated_airflow_version": "2.10.1",
"date_released": "2024-09-24T13:49:56Z"
+},
+"1.12.1": {
+"associated_airflow_version": "2.10.1",
+"date_released": "2024-10-01T09:05:14Z"
}
},
"opensearch": {
(airflow) annotated tag providers-openlineage/1.12.1 updated (bcee926d49 -> f95648c47e)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to annotated tag providers-openlineage/1.12.1 in repository https://gitbox.apache.org/repos/asf/airflow.git *** WARNING: tag providers-openlineage/1.12.1 was modified! *** from bcee926d49 (commit) to f95648c47e (tag) tagging bcee926d494cabf4ddfa9f2569e36acc5b4d281d (commit) replaces providers-airbyte/4.0.0 by Elad Kalif on Tue Oct 1 12:05:14 2024 +0300 - Log - Release 2024-10-01 of providers --- No new revisions were added by this update. Summary of changes:
(airflow-site) branch add-documentation-2024-09-27 created (now af5a765605)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch add-documentation-2024-09-27 in repository https://gitbox.apache.org/repos/asf/airflow-site.git at af5a765605 Add documentation for packages - 2024-09-27 This branch includes the following new commits: new af5a765605 Add documentation for packages - 2024-09-27 The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
svn commit: r71923 - /dev/airflow/providers/
Author: eladkal Date: Fri Sep 27 01:45:25 2024 New Revision: 71923 Log: Add artifacts for Airflow Providers 2024-09-27 Added: dev/airflow/providers/apache_airflow_providers_common_sql-1.17.1-py3-none-any.whl (with props) dev/airflow/providers/apache_airflow_providers_common_sql-1.17.1-py3-none-any.whl.asc dev/airflow/providers/apache_airflow_providers_common_sql-1.17.1-py3-none-any.whl.sha512 dev/airflow/providers/apache_airflow_providers_common_sql-1.17.1.tar.gz (with props) dev/airflow/providers/apache_airflow_providers_common_sql-1.17.1.tar.gz.asc dev/airflow/providers/apache_airflow_providers_common_sql-1.17.1.tar.gz.sha512 dev/airflow/providers/apache_airflow_providers_openlineage-1.12.1-py3-none-any.whl (with props) dev/airflow/providers/apache_airflow_providers_openlineage-1.12.1-py3-none-any.whl.asc dev/airflow/providers/apache_airflow_providers_openlineage-1.12.1-py3-none-any.whl.sha512 dev/airflow/providers/apache_airflow_providers_openlineage-1.12.1.tar.gz (with props) dev/airflow/providers/apache_airflow_providers_openlineage-1.12.1.tar.gz.asc dev/airflow/providers/apache_airflow_providers_openlineage-1.12.1.tar.gz.sha512 Added: dev/airflow/providers/apache_airflow_providers_common_sql-1.17.1-py3-none-any.whl == Binary file - no diff available. Propchange: dev/airflow/providers/apache_airflow_providers_common_sql-1.17.1-py3-none-any.whl -- svn:mime-type = application/octet-stream Added: dev/airflow/providers/apache_airflow_providers_common_sql-1.17.1-py3-none-any.whl.asc == --- dev/airflow/providers/apache_airflow_providers_common_sql-1.17.1-py3-none-any.whl.asc (added) +++ dev/airflow/providers/apache_airflow_providers_common_sql-1.17.1-py3-none-any.whl.asc Fri Sep 27 01:45:25 2024 @@ -0,0 +1,7 @@ +-BEGIN PGP SIGNATURE- + +iIkEABYKADEWIQSDQO8ECQokO9vDRUWG4IhmPszevgUCZvYN+BMcZWxhZGthbEBh +cGFjaGUub3JnAAoJEIbgiGY+zN6+bU0BAKNxG6Mx1qySogyHFUiUphogJfEZcEnN +PGg1Fg+xG5IpAQCp1aHWkmXAZgOsAmVuWdEY2fa/JOOkpTWt3wIKhmqHBw== +=yDG6 +-END PGP SIGNATURE- Added: dev/airflow/providers/apache_airflow_providers_common_sql-1.17.1-py3-none-any.whl.sha512 == --- dev/airflow/providers/apache_airflow_providers_common_sql-1.17.1-py3-none-any.whl.sha512 (added) +++ dev/airflow/providers/apache_airflow_providers_common_sql-1.17.1-py3-none-any.whl.sha512 Fri Sep 27 01:45:25 2024 @@ -0,0 +1 @@ +b2e00183000763db3d2b1fa550c1d84e13e5d26b0bf815a23c290edd99cba0c22279077fd74d1810599f16ddd1bc7fbf97bb4f7c90baaf0869db63f5ec13d6b2 apache_airflow_providers_common_sql-1.17.1-py3-none-any.whl Added: dev/airflow/providers/apache_airflow_providers_common_sql-1.17.1.tar.gz == Binary file - no diff available. Propchange: dev/airflow/providers/apache_airflow_providers_common_sql-1.17.1.tar.gz -- svn:mime-type = application/octet-stream Added: dev/airflow/providers/apache_airflow_providers_common_sql-1.17.1.tar.gz.asc == --- dev/airflow/providers/apache_airflow_providers_common_sql-1.17.1.tar.gz.asc (added) +++ dev/airflow/providers/apache_airflow_providers_common_sql-1.17.1.tar.gz.asc Fri Sep 27 01:45:25 2024 @@ -0,0 +1,7 @@ +-BEGIN PGP SIGNATURE- + +iIkEABYKADEWIQSDQO8ECQokO9vDRUWG4IhmPszevgUCZvYN/xMcZWxhZGthbEBh +cGFjaGUub3JnAAoJEIbgiGY+zN6+nJwA/0PEP6r0ATKLxm0bbTqkdWVuiQxCaMqp +FxK0CYf26kvfAQCe1Vakpm7G82keuEP9EYzm6Q5zGvm6sRs1jXcVYGWYCQ== +=NAv2 +-END PGP SIGNATURE- Added: dev/airflow/providers/apache_airflow_providers_common_sql-1.17.1.tar.gz.sha512 == --- dev/airflow/providers/apache_airflow_providers_common_sql-1.17.1.tar.gz.sha512 (added) +++ dev/airflow/providers/apache_airflow_providers_common_sql-1.17.1.tar.gz.sha512 Fri Sep 27 01:45:25 2024 @@ -0,0 +1 @@ +7b13fd09e43d34487761aa97b8323534eb3ebe3c366652f0516beed0b07492ef1e59b819bdd224b756be21718bf4e48fdf77d0f64ccef27865bb8ade6ceec84f apache_airflow_providers_common_sql-1.17.1.tar.gz Added: dev/airflow/providers/apache_airflow_providers_openlineage-1.12.1-py3-none-any.whl == Binary file - no diff available. Propchange: dev/airflow/providers/apache_airflow_providers_openlineage-1.12.1-py3-none-any.whl -- svn:mime-type = application/octet-stream Added: dev/airflow/providers
(airflow) annotated tag providers-openlineage/1.12.1rc1 updated (bcee926d49 -> dc229bb271)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to annotated tag providers-openlineage/1.12.1rc1 in repository https://gitbox.apache.org/repos/asf/airflow.git *** WARNING: tag providers-openlineage/1.12.1rc1 was modified! *** from bcee926d49 (commit) to dc229bb271 (tag) tagging bcee926d494cabf4ddfa9f2569e36acc5b4d281d (commit) replaces providers-airbyte/4.0.0 by Elad Kalif on Fri Sep 27 04:50:47 2024 +0300 - Log - Release 2024-09-27 of providers --- No new revisions were added by this update. Summary of changes:
(airflow) annotated tag providers-common-sql/1.17.1rc1 updated (bcee926d49 -> a625f90107)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to annotated tag providers-common-sql/1.17.1rc1 in repository https://gitbox.apache.org/repos/asf/airflow.git *** WARNING: tag providers-common-sql/1.17.1rc1 was modified! *** from bcee926d49 (commit) to a625f90107 (tag) tagging bcee926d494cabf4ddfa9f2569e36acc5b4d281d (commit) replaces providers-airbyte/4.0.0 by Elad Kalif on Fri Sep 27 04:50:47 2024 +0300 - Log - Release 2024-09-27 of providers --- No new revisions were added by this update. Summary of changes:
(airflow) branch main updated: Prepare docs for Sep 2nd adhoc wave of providers (#42519)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new bcee926d49 Prepare docs for Sep 2nd adhoc wave of providers (#42519) bcee926d49 is described below commit bcee926d494cabf4ddfa9f2569e36acc5b4d281d Author: Elad Kalif <45845474+elad...@users.noreply.github.com> AuthorDate: Fri Sep 27 08:40:54 2024 +0700 Prepare docs for Sep 2nd adhoc wave of providers (#42519) --- airflow/providers/common/sql/CHANGELOG.rst| 9 + airflow/providers/common/sql/__init__.py | 2 +- airflow/providers/common/sql/provider.yaml| 3 ++- airflow/providers/openlineage/CHANGELOG.rst | 9 + airflow/providers/openlineage/__init__.py | 2 +- airflow/providers/openlineage/provider.yaml | 3 ++- docs/apache-airflow-providers-common-sql/commits.rst | 15 ++- docs/apache-airflow-providers-common-sql/index.rst| 6 +++--- docs/apache-airflow-providers-openlineage/commits.rst | 15 ++- docs/apache-airflow-providers-openlineage/index.rst | 6 +++--- 10 files changed, 58 insertions(+), 12 deletions(-) diff --git a/airflow/providers/common/sql/CHANGELOG.rst b/airflow/providers/common/sql/CHANGELOG.rst index ff4a7a74d5..531353f8c8 100644 --- a/airflow/providers/common/sql/CHANGELOG.rst +++ b/airflow/providers/common/sql/CHANGELOG.rst @@ -25,6 +25,15 @@ Changelog - +1.17.1 +.. + +Bug Fixes +~ + +* ``fix(providers/common/sql): add dummy connection setter for backward compatibility (#42490)`` +* ``Changed type hinting for handler function (#42275)`` + 1.17.0 .. diff --git a/airflow/providers/common/sql/__init__.py b/airflow/providers/common/sql/__init__.py index e1c93c3efb..6ef37aa0ed 100644 --- a/airflow/providers/common/sql/__init__.py +++ b/airflow/providers/common/sql/__init__.py @@ -29,7 +29,7 @@ from airflow import __version__ as airflow_version __all__ = ["__version__"] -__version__ = "1.17.0" +__version__ = "1.17.1" if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse( "2.8.0" diff --git a/airflow/providers/common/sql/provider.yaml b/airflow/providers/common/sql/provider.yaml index f600acc1fa..ec487aca3f 100644 --- a/airflow/providers/common/sql/provider.yaml +++ b/airflow/providers/common/sql/provider.yaml @@ -22,9 +22,10 @@ description: | `Common SQL Provider <https://en.wikipedia.org/wiki/SQL>`__ state: ready -source-date-epoch: 1723970051 +source-date-epoch: 1727372263 # note that those versions are maintained by release manager - do not update them manually versions: + - 1.17.1 - 1.17.0 - 1.16.0 - 1.15.0 diff --git a/airflow/providers/openlineage/CHANGELOG.rst b/airflow/providers/openlineage/CHANGELOG.rst index 318d0d92b2..0e35dab6de 100644 --- a/airflow/providers/openlineage/CHANGELOG.rst +++ b/airflow/providers/openlineage/CHANGELOG.rst @@ -26,6 +26,15 @@ Changelog - +1.12.1 +.. + +Bug Fixes +~ + +* ``fix: OpenLineage dag start event not being emitted (#42448)`` +* ``fix: typo in error stack trace formatting for clearer output (#42017)`` + 1.12.0 .. diff --git a/airflow/providers/openlineage/__init__.py b/airflow/providers/openlineage/__init__.py index 6c3c88bb92..664e5530eb 100644 --- a/airflow/providers/openlineage/__init__.py +++ b/airflow/providers/openlineage/__init__.py @@ -29,7 +29,7 @@ from airflow import __version__ as airflow_version __all__ = ["__version__"] -__version__ = "1.12.0" +__version__ = "1.12.1" if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse( "2.8.0" diff --git a/airflow/providers/openlineage/provider.yaml b/airflow/providers/openlineage/provider.yaml index af13b1954b..b249ff46c8 100644 --- a/airflow/providers/openlineage/provider.yaml +++ b/airflow/providers/openlineage/provider.yaml @@ -22,9 +22,10 @@ description: | `OpenLineage <https://openlineage.io/>`__ state: ready -source-date-epoch: 1726861079 +source-date-epoch: 1727372276 # note that those versions are maintained by release manager - do not update them manually versions: + - 1.12.1 - 1.12.0 - 1.11.0 - 1.10.0 diff --git a/docs/apache-airflow-providers-common-sql/commits.rst b/docs/apache-airflow-providers-common-sql/commits.rst index 95e835b0bf..f719dd7b39 100644 --- a/docs/apache-airflow-providers-common-sql/commits.rst +++ b/docs/apache-airflow-providers-common-sql/commits.rst @@ -35,14 +35,27 @@ For high-level changelog, see :doc:`package information including changelog https://github.com/apache/airflow/commit/7ad586ed6a6cb93fc25ae0405eb9f6c17d360ef0>`_ 2024-09-26 `
(airflow) branch main updated (2bbb37931e -> 84e8cdf674)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 2bbb37931e Add basic endpoints for managing backfill entities (#42455) add 84e8cdf674 fix: OL dag start event not being emitted (#42448) No new revisions were added by this update. Summary of changes: airflow/providers/openlineage/plugins/listener.py | 1 - 1 file changed, 1 deletion(-)
(airflow) branch main updated (d87f9b0505 -> 193defd289)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from d87f9b0505 Refactor AWS Auth manager user output (#42454) add 193defd289 (bugfix): Paginated results in MSGraphAsyncOperator (#42414) No new revisions were added by this update. Summary of changes: .../providers/microsoft/azure/operators/msgraph.py | 56 -- .../microsoft/azure/operators/test_msgraph.py | 36 +- 2 files changed, 66 insertions(+), 26 deletions(-)
(airflow) branch main updated: Refactor AWS Auth manager user output (#42454)
This is an automated email from the ASF dual-hosted git repository.
eladkal pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new d87f9b0505 Refactor AWS Auth manager user output (#42454)
d87f9b0505 is described below
commit d87f9b050505747ee070327ef482a5e0f5551ec1
Author: Niko Oliveira
AuthorDate: Wed Sep 25 06:58:22 2024 -0700
Refactor AWS Auth manager user output (#42454)
AWS auth manager has incredible tooling to setup the required resources,
however one piece needs to be done manually. This PR updates the docs
and user output to make it more clear what needs to happen next.
Removing the stacktrace (which usually indicates a critical failure in
a piece of code) and replacing with a more clearly marked output
message. Also update the docs to more clearly indicate that the script
will most likely need user intervention.
---
airflow/providers/amazon/aws/auth_manager/cli/idc_commands.py | 10 +++---
.../auth-manager/setup/identity-center.rst | 6 +-
2 files changed, 8 insertions(+), 8 deletions(-)
diff --git a/airflow/providers/amazon/aws/auth_manager/cli/idc_commands.py
b/airflow/providers/amazon/aws/auth_manager/cli/idc_commands.py
index 388948765a..c4901351b2 100644
--- a/airflow/providers/amazon/aws/auth_manager/cli/idc_commands.py
+++ b/airflow/providers/amazon/aws/auth_manager/cli/idc_commands.py
@@ -19,6 +19,7 @@
from __future__ import annotations
import logging
+import sys
from typing import TYPE_CHECKING
import boto3
@@ -139,10 +140,13 @@ def _create_application(client: BaseClient, instance_arn:
str | None, args) -> s
# Remove this part when it is supported
if "is not supported for this action" in
e.response["Error"]["Message"]:
print(
-"Creation of SAML applications is only supported in AWS
console today. "
-"Please create the application through the console."
+
"*\n"
+"*ACTION REQUIRED
*\n"
+"* Creation of SAML applications is only supported in AWS
console today. *\n"
+"* Please create the application through the console.
*\n"
+
"*\n"
)
-raise
+sys.exit(1)
print(f"Application created: '{response['ApplicationArn']}'")
diff --git
a/docs/apache-airflow-providers-amazon/auth-manager/setup/identity-center.rst
b/docs/apache-airflow-providers-amazon/auth-manager/setup/identity-center.rst
index a134dfe0dd..acf3727bf9 100644
---
a/docs/apache-airflow-providers-amazon/auth-manager/setup/identity-center.rst
+++
b/docs/apache-airflow-providers-amazon/auth-manager/setup/identity-center.rst
@@ -48,11 +48,7 @@ To create the resources, please run the following command:
airflow aws-auth-manager init-identity-center
-The CLI command should exit successfully with the message: ::
-
- AWS IAM Identity Center resources created successfully.
-
-If the CLI command exited with an error, please look carefully at the CLI
command output to understand which resource(s)
+The CLI command will ask you to create any resources manually if they cannot
be automatically created. Please look carefully at the CLI command output to
understand which resource(s)
have or have not been created successfully. The resource(s) which have not
been successfully created need to be
:ref:`created manually `.
(airflow) branch main updated: Update providers metadata 2024-09-24 (#42445)
This is an automated email from the ASF dual-hosted git repository.
eladkal pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new c239c5f66c Update providers metadata 2024-09-24 (#42445)
c239c5f66c is described below
commit c239c5f66cfe620e707bebc8ea88c7013af0e843
Author: Elad Kalif <45845474+elad...@users.noreply.github.com>
AuthorDate: Tue Sep 24 17:07:35 2024 +0300
Update providers metadata 2024-09-24 (#42445)
---
generated/provider_metadata.json | 376 ---
1 file changed, 237 insertions(+), 139 deletions(-)
diff --git a/generated/provider_metadata.json b/generated/provider_metadata.json
index 4ca06608c7..a73e3da9f6 100644
--- a/generated/provider_metadata.json
+++ b/generated/provider_metadata.json
@@ -85,8 +85,12 @@
"date_released": "2024-05-30T06:38:15Z"
},
"3.9.0": {
-"associated_airflow_version": "2.9.2",
+"associated_airflow_version": "2.10.1",
"date_released": "2024-08-22T10:37:58Z"
+},
+"4.0.0": {
+"associated_airflow_version": "2.10.1",
+"date_released": "2024-09-24T13:49:56Z"
}
},
"alibaba": {
@@ -179,8 +183,12 @@
"date_released": "2024-05-30T06:38:15Z"
},
"2.9.0": {
-"associated_airflow_version": "2.9.2",
+"associated_airflow_version": "2.10.1",
"date_released": "2024-08-22T10:37:58Z"
+},
+"2.9.1": {
+"associated_airflow_version": "2.10.1",
+"date_released": "2024-09-24T13:49:56Z"
}
},
"amazon": {
@@ -433,8 +441,12 @@
"date_released": "2024-08-06T20:34:43Z"
},
"8.28.0": {
-"associated_airflow_version": "2.10.0",
+"associated_airflow_version": "2.10.1",
"date_released": "2024-08-22T10:37:58Z"
+},
+"8.29.0": {
+"associated_airflow_version": "2.10.1",
+"date_released": "2024-09-24T13:49:56Z"
}
},
"apache.beam": {
@@ -567,7 +579,7 @@
"date_released": "2024-08-06T20:34:43Z"
},
"5.8.0": {
-"associated_airflow_version": "2.10.0",
+"associated_airflow_version": "2.10.1",
"date_released": "2024-08-22T10:37:58Z"
}
},
@@ -649,7 +661,7 @@
"date_released": "2024-05-30T06:38:15Z"
},
"3.6.0": {
-"associated_airflow_version": "2.9.2",
+"associated_airflow_version": "2.10.1",
"date_released": "2024-08-22T10:37:59Z"
}
},
@@ -751,7 +763,7 @@
"date_released": "2024-08-06T20:34:44Z"
},
"2.8.0": {
-"associated_airflow_version": "2.10.0",
+"associated_airflow_version": "2.10.1",
"date_released": "2024-08-22T10:37:58Z"
}
},
@@ -877,7 +889,7 @@
"date_released": "2024-08-06T20:34:43Z"
},
"3.11.0": {
-"associated_airflow_version": "2.10.0",
+"associated_airflow_version": "2.10.1",
"date_released": "2024-08-22T10:37:57Z"
}
},
@@ -927,8 +939,12 @@
"date_released": "2024-06-27T07:50:54Z"
},
"1.5.0": {
-"associated_airflow_version": "2.9.3",
+"associated_airflow_version": "2.10.1",
"date_released": "2024-08-22T10:37:58Z"
+},
+"1.5.1": {
+"associated_airflow_version": "2.10.1",
+"date_released": "2024-09-24T13:49:56Z"
}
},
"apache.hdfs": {
@@ -1033,8 +1049,12 @@
"date_released": "2024-06-27T07:50:54Z"
},
"4.5.0": {
-"associated_airflow_version": "2.9.3",
+"associated_airflow_version&qu
(airflow-site) branch add-documentation-2nd-2024-09-21 deleted (was f63dc049ca)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch add-documentation-2nd-2024-09-21 in repository https://gitbox.apache.org/repos/asf/airflow-site.git was f63dc049ca Add documentation for packages - 2024-09-21 The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
svn commit: r71831 - /dev/airflow/providers/ /release/airflow/providers/
Author: eladkal Date: Tue Sep 24 13:44:14 2024 New Revision: 71831 Log: Release Airflow Providers on 2024-09-24 Added: release/airflow/providers/apache_airflow_providers_airbyte-4.0.0-py3-none-any.whl - copied unchanged from r71830, dev/airflow/providers/apache_airflow_providers_airbyte-4.0.0-py3-none-any.whl release/airflow/providers/apache_airflow_providers_airbyte-4.0.0-py3-none-any.whl.asc - copied unchanged from r71830, dev/airflow/providers/apache_airflow_providers_airbyte-4.0.0-py3-none-any.whl.asc release/airflow/providers/apache_airflow_providers_airbyte-4.0.0-py3-none-any.whl.sha512 - copied unchanged from r71830, dev/airflow/providers/apache_airflow_providers_airbyte-4.0.0-py3-none-any.whl.sha512 release/airflow/providers/apache_airflow_providers_airbyte-4.0.0.tar.gz - copied unchanged from r71830, dev/airflow/providers/apache_airflow_providers_airbyte-4.0.0.tar.gz release/airflow/providers/apache_airflow_providers_airbyte-4.0.0.tar.gz.asc - copied unchanged from r71830, dev/airflow/providers/apache_airflow_providers_airbyte-4.0.0.tar.gz.asc release/airflow/providers/apache_airflow_providers_airbyte-4.0.0.tar.gz.sha512 - copied unchanged from r71830, dev/airflow/providers/apache_airflow_providers_airbyte-4.0.0.tar.gz.sha512 release/airflow/providers/apache_airflow_providers_alibaba-2.9.1-py3-none-any.whl - copied unchanged from r71830, dev/airflow/providers/apache_airflow_providers_alibaba-2.9.1-py3-none-any.whl release/airflow/providers/apache_airflow_providers_alibaba-2.9.1-py3-none-any.whl.asc - copied unchanged from r71830, dev/airflow/providers/apache_airflow_providers_alibaba-2.9.1-py3-none-any.whl.asc release/airflow/providers/apache_airflow_providers_alibaba-2.9.1-py3-none-any.whl.sha512 - copied unchanged from r71830, dev/airflow/providers/apache_airflow_providers_alibaba-2.9.1-py3-none-any.whl.sha512 release/airflow/providers/apache_airflow_providers_alibaba-2.9.1.tar.gz - copied unchanged from r71830, dev/airflow/providers/apache_airflow_providers_alibaba-2.9.1.tar.gz release/airflow/providers/apache_airflow_providers_alibaba-2.9.1.tar.gz.asc - copied unchanged from r71830, dev/airflow/providers/apache_airflow_providers_alibaba-2.9.1.tar.gz.asc release/airflow/providers/apache_airflow_providers_alibaba-2.9.1.tar.gz.sha512 - copied unchanged from r71830, dev/airflow/providers/apache_airflow_providers_alibaba-2.9.1.tar.gz.sha512 release/airflow/providers/apache_airflow_providers_amazon-8.29.0-py3-none-any.whl - copied unchanged from r71830, dev/airflow/providers/apache_airflow_providers_amazon-8.29.0-py3-none-any.whl release/airflow/providers/apache_airflow_providers_amazon-8.29.0-py3-none-any.whl.asc - copied unchanged from r71830, dev/airflow/providers/apache_airflow_providers_amazon-8.29.0-py3-none-any.whl.asc release/airflow/providers/apache_airflow_providers_amazon-8.29.0-py3-none-any.whl.sha512 - copied unchanged from r71830, dev/airflow/providers/apache_airflow_providers_amazon-8.29.0-py3-none-any.whl.sha512 release/airflow/providers/apache_airflow_providers_amazon-8.29.0.tar.gz - copied unchanged from r71830, dev/airflow/providers/apache_airflow_providers_amazon-8.29.0.tar.gz release/airflow/providers/apache_airflow_providers_amazon-8.29.0.tar.gz.asc - copied unchanged from r71830, dev/airflow/providers/apache_airflow_providers_amazon-8.29.0.tar.gz.asc release/airflow/providers/apache_airflow_providers_amazon-8.29.0.tar.gz.sha512 - copied unchanged from r71830, dev/airflow/providers/apache_airflow_providers_amazon-8.29.0.tar.gz.sha512 release/airflow/providers/apache_airflow_providers_apache_flink-1.5.1-py3-none-any.whl - copied unchanged from r71830, dev/airflow/providers/apache_airflow_providers_apache_flink-1.5.1-py3-none-any.whl release/airflow/providers/apache_airflow_providers_apache_flink-1.5.1-py3-none-any.whl.asc - copied unchanged from r71830, dev/airflow/providers/apache_airflow_providers_apache_flink-1.5.1-py3-none-any.whl.asc release/airflow/providers/apache_airflow_providers_apache_flink-1.5.1-py3-none-any.whl.sha512 - copied unchanged from r71830, dev/airflow/providers/apache_airflow_providers_apache_flink-1.5.1-py3-none-any.whl.sha512 release/airflow/providers/apache_airflow_providers_apache_flink-1.5.1.tar.gz - copied unchanged from r71830, dev/airflow/providers/apache_airflow_providers_apache_flink-1.5.1.tar.gz release/airflow/providers/apache_airflow_providers_apache_flink-1.5.1.tar.gz.asc - copied unchanged from r71830, dev/airflow/providers/apache_airflow_providers_apache_flink-1.5.1.tar.gz.asc release/airflow/providers/apache_airflow_providers_apache_flink-1.5.1.tar.gz.sha512 - copied unchanged from r71830, dev/airflow/providers/apache_airflow_providers_apache_flink-1.5.1
(airflow) branch main updated (3d2f9c11d1 -> 583af0294d)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 3d2f9c11d1 Add option in simple auth manager to disable authentication and allow everyone as admin (#42355) add 583af0294d Resolve failing test when request and ddb table schema mismatch (#42419) No new revisions were added by this update. Summary of changes: .../providers/amazon/aws/sensors/test_dynamodb.py | 31 +- 1 file changed, 24 insertions(+), 7 deletions(-)
(airflow-site) branch add-documentation-2024-09-21 deleted (was 5ca48e41b5)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch add-documentation-2024-09-21 in repository https://gitbox.apache.org/repos/asf/airflow-site.git was 5ca48e41b5 Add documentation for packages - 2024-09-21 The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
(airflow-site) branch add-documentation-2nd-2024-09-21 created (now f63dc049ca)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch add-documentation-2nd-2024-09-21 in repository https://gitbox.apache.org/repos/asf/airflow-site.git at f63dc049ca Add documentation for packages - 2024-09-21 This branch includes the following new commits: new f63dc049ca Add documentation for packages - 2024-09-21 The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
(airflow) branch main updated (7628d47d04 -> ba1c602533)
This is an automated email from the ASF dual-hosted git repository.
eladkal pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
from 7628d47d04 Prepare docs for Sep 1st wave of providers (#42387)
add ba1c602533 (feat): Add opensearch logging integration (#41799)
No new revisions were added by this update.
Summary of changes:
airflow/config_templates/airflow_local_settings.py | 31 +-
airflow/config_templates/unit_tests.cfg| 24 +
.../opensearch/log}/__init__.py| 0
.../log/os_json_formatter.py} | 2 +-
.../log/os_response.py}| 6 +-
.../log/os_task_handler.py}| 507 +
airflow/providers/opensearch/provider.yaml | 143 ++
airflow/utils/db.py| 10 +
.../configurations-ref.rst | 0
docs/apache-airflow-providers-opensearch/index.rst | 2 +
.../logging/index.rst | 72 +++
tests/core/test_configuration.py | 2 +
tests/providers/opensearch/conftest.py | 120 +
.../providers/opensearch/log}/__init__.py | 0
.../log/test_os_json_formatter.py} | 32 +-
.../log/test_os_response.py} | 107 +
.../opensearch/log/test_os_task_handler.py | 499
17 files changed, 1154 insertions(+), 403 deletions(-)
copy airflow/{api_connexion => providers/opensearch/log}/__init__.py (100%)
copy airflow/providers/{elasticsearch/log/es_json_formatter.py =>
opensearch/log/os_json_formatter.py} (97%)
copy airflow/providers/{elasticsearch/log/es_response.py =>
opensearch/log/os_response.py} (96%)
copy airflow/providers/{elasticsearch/log/es_task_handler.py =>
opensearch/log/os_task_handler.py} (76%)
copy docs/{apache-airflow-providers-amazon =>
apache-airflow-providers-opensearch}/configurations-ref.rst (100%)
create mode 100644 docs/apache-airflow-providers-opensearch/logging/index.rst
copy {airflow/api_connexion => tests/providers/opensearch/log}/__init__.py
(100%)
copy tests/providers/{elasticsearch/log/test_es_json_formatter.py =>
opensearch/log/test_os_json_formatter.py} (74%)
copy tests/providers/{elasticsearch/log/test_es_response.py =>
opensearch/log/test_os_response.py} (59%)
create mode 100644 tests/providers/opensearch/log/test_os_task_handler.py
(airflow-site) branch add-documentation-2024-09-21 created (now 5ca48e41b5)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch add-documentation-2024-09-21 in repository https://gitbox.apache.org/repos/asf/airflow-site.git at 5ca48e41b5 Add documentation for packages - 2024-09-21 This branch includes the following new commits: new 5ca48e41b5 Add documentation for packages - 2024-09-21 The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
svn commit: r71788 [2/2] - /dev/airflow/providers/
Added: dev/airflow/providers/apache_airflow_providers_http-4.13.1.tar.gz.asc == --- dev/airflow/providers/apache_airflow_providers_http-4.13.1.tar.gz.asc (added) +++ dev/airflow/providers/apache_airflow_providers_http-4.13.1.tar.gz.asc Sat Sep 21 06:12:32 2024 @@ -0,0 +1,7 @@ +-BEGIN PGP SIGNATURE- + +iIkEABYKADEWIQSDQO8ECQokO9vDRUWG4IhmPszevgUCZu5jAhMcZWxhZGthbEBh +cGFjaGUub3JnAAoJEIbgiGY+zN6+GeUBALHFpr3JtERzKebytOahJ6hTkcq6LYsW +pg0v5ItdmL1YAP4pLFyTu6icwzf0QLSt5ilSN+hQ1MwVJ1cHt5oa4rpjCQ== +=jlJg +-END PGP SIGNATURE- Added: dev/airflow/providers/apache_airflow_providers_http-4.13.1.tar.gz.sha512 == --- dev/airflow/providers/apache_airflow_providers_http-4.13.1.tar.gz.sha512 (added) +++ dev/airflow/providers/apache_airflow_providers_http-4.13.1.tar.gz.sha512 Sat Sep 21 06:12:32 2024 @@ -0,0 +1 @@ +bd45f519010c994cde945862cc4a3550a0bfab15fba738727a2558d5183f58ed1d85ff91186eb308bf779b2082d7f5d9c966c5af89a2cff13ec21135d7ab0251 apache_airflow_providers_http-4.13.1.tar.gz Added: dev/airflow/providers/apache_airflow_providers_influxdb-2.7.1-py3-none-any.whl == Binary file - no diff available. Propchange: dev/airflow/providers/apache_airflow_providers_influxdb-2.7.1-py3-none-any.whl -- svn:mime-type = application/octet-stream Added: dev/airflow/providers/apache_airflow_providers_influxdb-2.7.1-py3-none-any.whl.asc == --- dev/airflow/providers/apache_airflow_providers_influxdb-2.7.1-py3-none-any.whl.asc (added) +++ dev/airflow/providers/apache_airflow_providers_influxdb-2.7.1-py3-none-any.whl.asc Sat Sep 21 06:12:32 2024 @@ -0,0 +1,7 @@ +-BEGIN PGP SIGNATURE- + +iIkEABYKADEWIQSDQO8ECQokO9vDRUWG4IhmPszevgUCZu5jAhMcZWxhZGthbEBh +cGFjaGUub3JnAAoJEIbgiGY+zN6+KLMBAJ6uNNJv/qFBQtPh3Kw9p8dTdqi+zHI7 +D8hM4A28b+WPAP4nFBIJ98/EsdgqLO7DJer4fBBZ4vxzgoxv6OTK+Ci8Ag== +=fdR8 +-END PGP SIGNATURE- Added: dev/airflow/providers/apache_airflow_providers_influxdb-2.7.1-py3-none-any.whl.sha512 == --- dev/airflow/providers/apache_airflow_providers_influxdb-2.7.1-py3-none-any.whl.sha512 (added) +++ dev/airflow/providers/apache_airflow_providers_influxdb-2.7.1-py3-none-any.whl.sha512 Sat Sep 21 06:12:32 2024 @@ -0,0 +1 @@ +03eff0c201ec3aabf2ce5441c197eb08dc292a161c42d691c1e78f40eea0455f05a4aa0ef860b653b32e8acf1728bb15959317bab111408f8d25a52a960664bf apache_airflow_providers_influxdb-2.7.1-py3-none-any.whl Added: dev/airflow/providers/apache_airflow_providers_influxdb-2.7.1.tar.gz == Binary file - no diff available. Propchange: dev/airflow/providers/apache_airflow_providers_influxdb-2.7.1.tar.gz -- svn:mime-type = application/octet-stream Added: dev/airflow/providers/apache_airflow_providers_influxdb-2.7.1.tar.gz.asc == --- dev/airflow/providers/apache_airflow_providers_influxdb-2.7.1.tar.gz.asc (added) +++ dev/airflow/providers/apache_airflow_providers_influxdb-2.7.1.tar.gz.asc Sat Sep 21 06:12:32 2024 @@ -0,0 +1,7 @@ +-BEGIN PGP SIGNATURE- + +iIkEABYKADEWIQSDQO8ECQokO9vDRUWG4IhmPszevgUCZu5jAhMcZWxhZGthbEBh +cGFjaGUub3JnAAoJEIbgiGY+zN6+TrgA/0/EcfCPZXg1VYiGB5tsuw/jFrirt4hl +S5rN2AENlndQAP91/LXQbNcjSGNVLbLEt3XJfwPHG8MZ4z/liRWu2GNtAA== +=aqsK +-END PGP SIGNATURE- Added: dev/airflow/providers/apache_airflow_providers_influxdb-2.7.1.tar.gz.sha512 == --- dev/airflow/providers/apache_airflow_providers_influxdb-2.7.1.tar.gz.sha512 (added) +++ dev/airflow/providers/apache_airflow_providers_influxdb-2.7.1.tar.gz.sha512 Sat Sep 21 06:12:32 2024 @@ -0,0 +1 @@ +8cd29211260d17b2a3741514c80fbdf835f63cb24abe58bff86072e6e726ede8635043daa3f2f7d5b5dd00fb468460cdd465f559faaa3a4051a8b736b554c681 apache_airflow_providers_influxdb-2.7.1.tar.gz Added: dev/airflow/providers/apache_airflow_providers_jdbc-4.5.1-py3-none-any.whl == Binary file - no diff available. Propchange: dev/airflow/providers/apache_airflow_providers_jdbc-4.5.1-py3-none-any.whl -- svn:mime-type = application/octet-stream Added: dev/airflow/providers/apache_airflow_providers_jdbc-4.5.1-py3-none-any.whl.asc == --- dev/airflow/pr
svn commit: r71788 [1/2] - /dev/airflow/providers/
Author: eladkal Date: Sat Sep 21 06:12:32 2024 New Revision: 71788 Log: Add artifacts for Airflow Providers 2024-09-21 Added: dev/airflow/providers/apache_airflow_providers_airbyte-4.0.0-py3-none-any.whl (with props) dev/airflow/providers/apache_airflow_providers_airbyte-4.0.0-py3-none-any.whl.asc dev/airflow/providers/apache_airflow_providers_airbyte-4.0.0-py3-none-any.whl.sha512 dev/airflow/providers/apache_airflow_providers_airbyte-4.0.0.tar.gz (with props) dev/airflow/providers/apache_airflow_providers_airbyte-4.0.0.tar.gz.asc dev/airflow/providers/apache_airflow_providers_airbyte-4.0.0.tar.gz.sha512 dev/airflow/providers/apache_airflow_providers_alibaba-2.9.1-py3-none-any.whl (with props) dev/airflow/providers/apache_airflow_providers_alibaba-2.9.1-py3-none-any.whl.asc dev/airflow/providers/apache_airflow_providers_alibaba-2.9.1-py3-none-any.whl.sha512 dev/airflow/providers/apache_airflow_providers_alibaba-2.9.1.tar.gz (with props) dev/airflow/providers/apache_airflow_providers_alibaba-2.9.1.tar.gz.asc dev/airflow/providers/apache_airflow_providers_alibaba-2.9.1.tar.gz.sha512 dev/airflow/providers/apache_airflow_providers_amazon-8.29.0-py3-none-any.whl (with props) dev/airflow/providers/apache_airflow_providers_amazon-8.29.0-py3-none-any.whl.asc dev/airflow/providers/apache_airflow_providers_amazon-8.29.0-py3-none-any.whl.sha512 dev/airflow/providers/apache_airflow_providers_amazon-8.29.0.tar.gz (with props) dev/airflow/providers/apache_airflow_providers_amazon-8.29.0.tar.gz.asc dev/airflow/providers/apache_airflow_providers_amazon-8.29.0.tar.gz.sha512 dev/airflow/providers/apache_airflow_providers_apache_flink-1.5.1-py3-none-any.whl (with props) dev/airflow/providers/apache_airflow_providers_apache_flink-1.5.1-py3-none-any.whl.asc dev/airflow/providers/apache_airflow_providers_apache_flink-1.5.1-py3-none-any.whl.sha512 dev/airflow/providers/apache_airflow_providers_apache_flink-1.5.1.tar.gz (with props) dev/airflow/providers/apache_airflow_providers_apache_flink-1.5.1.tar.gz.asc dev/airflow/providers/apache_airflow_providers_apache_flink-1.5.1.tar.gz.sha512 dev/airflow/providers/apache_airflow_providers_apache_hdfs-4.5.1-py3-none-any.whl (with props) dev/airflow/providers/apache_airflow_providers_apache_hdfs-4.5.1-py3-none-any.whl.asc dev/airflow/providers/apache_airflow_providers_apache_hdfs-4.5.1-py3-none-any.whl.sha512 dev/airflow/providers/apache_airflow_providers_apache_hdfs-4.5.1.tar.gz (with props) dev/airflow/providers/apache_airflow_providers_apache_hdfs-4.5.1.tar.gz.asc dev/airflow/providers/apache_airflow_providers_apache_hdfs-4.5.1.tar.gz.sha512 dev/airflow/providers/apache_airflow_providers_apache_impala-1.5.1-py3-none-any.whl (with props) dev/airflow/providers/apache_airflow_providers_apache_impala-1.5.1-py3-none-any.whl.asc dev/airflow/providers/apache_airflow_providers_apache_impala-1.5.1-py3-none-any.whl.sha512 dev/airflow/providers/apache_airflow_providers_apache_impala-1.5.1.tar.gz (with props) dev/airflow/providers/apache_airflow_providers_apache_impala-1.5.1.tar.gz.asc dev/airflow/providers/apache_airflow_providers_apache_impala-1.5.1.tar.gz.sha512 dev/airflow/providers/apache_airflow_providers_apache_livy-3.9.1-py3-none-any.whl (with props) dev/airflow/providers/apache_airflow_providers_apache_livy-3.9.1-py3-none-any.whl.asc dev/airflow/providers/apache_airflow_providers_apache_livy-3.9.1-py3-none-any.whl.sha512 dev/airflow/providers/apache_airflow_providers_apache_livy-3.9.1.tar.gz (with props) dev/airflow/providers/apache_airflow_providers_apache_livy-3.9.1.tar.gz.asc dev/airflow/providers/apache_airflow_providers_apache_livy-3.9.1.tar.gz.sha512 dev/airflow/providers/apache_airflow_providers_apache_spark-4.11.0-py3-none-any.whl (with props) dev/airflow/providers/apache_airflow_providers_apache_spark-4.11.0-py3-none-any.whl.asc dev/airflow/providers/apache_airflow_providers_apache_spark-4.11.0-py3-none-any.whl.sha512 dev/airflow/providers/apache_airflow_providers_apache_spark-4.11.0.tar.gz (with props) dev/airflow/providers/apache_airflow_providers_apache_spark-4.11.0.tar.gz.asc dev/airflow/providers/apache_airflow_providers_apache_spark-4.11.0.tar.gz.sha512 dev/airflow/providers/apache_airflow_providers_celery-3.8.2-py3-none-any.whl (with props) dev/airflow/providers/apache_airflow_providers_celery-3.8.2-py3-none-any.whl.asc dev/airflow/providers/apache_airflow_providers_celery-3.8.2-py3-none-any.whl.sha512 dev/airflow/providers/apache_airflow_providers_celery-3.8.2.tar.gz (with props) dev/airflow/providers/apache_airflow_providers_celery-3.8.2.tar.gz.asc dev/airflow/providers/apache_airflow_providers_celery-3.8.2.tar.gz.sha512 dev/airflow/providers
(airflow) branch main updated: Fix pre-commit for auto update of fab migration versions (#42382)
This is an automated email from the ASF dual-hosted git repository.
eladkal pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new 6a527c9fac Fix pre-commit for auto update of fab migration versions
(#42382)
6a527c9fac is described below
commit 6a527c9facc649b3d64f36459cd655bcb03a9cb1
Author: Ephraim Anierobi
AuthorDate: Sat Sep 21 00:15:44 2024 +0100
Fix pre-commit for auto update of fab migration versions (#42382)
* Fix pre-commit for auto update of fab migration versions
This fixes the pre-commit for updating the version in FAB migration and
also, updated the actual version as 1.4 since that's going to be the next
version
At the same time, added Daniel's fix for airflow alembic.ini file
* fixup! Fix pre-commit for auto update of fab migration versions
* fixup! fixup! Fix pre-commit for auto update of fab migration versions
---
.pre-commit-config.yaml | 4 +++-
airflow/providers/fab/alembic.ini | 2 +-
airflow/providers/fab/auth_manager/models/db.py | 2 +-
...0_placeholder_migration.py => 0001_1_4_0_placeholder_migration.py} | 2 +-
docs/apache-airflow-providers-fab/migrations-ref.rst | 2 +-
5 files changed, 7 insertions(+), 5 deletions(-)
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
index a356de6dec..4ca7f304d1 100644
--- a/.pre-commit-config.yaml
+++ b/.pre-commit-config.yaml
@@ -510,7 +510,9 @@ repos:
(?x)
^scripts/ci/pre_commit/version_heads_map\.py$|
^airflow/migrations/versions/.*$|^airflow/migrations/versions|
- ^airflow/utils/db.py$
+
^airflow/providers/fab/migrations/versions/.*$|^airflow/providers/fab/migrations/versions|
+ ^airflow/utils/db.py$|
+ ^airflow/providers/fab/auth_manager/models/db.py$
additional_dependencies: ['packaging','google-re2']
- id: update-version
name: Update versions in docs
diff --git a/airflow/providers/fab/alembic.ini
b/airflow/providers/fab/alembic.ini
index 55c7338f9b..75d42ee16d 100644
--- a/airflow/providers/fab/alembic.ini
+++ b/airflow/providers/fab/alembic.ini
@@ -20,7 +20,7 @@
[alembic]
# path to migration scripts
# Use forward slashes (/) also on windows to provide an os agnostic path
-script_location = migrations
+script_location = %(here)s/migrations
# template used to generate migration file names; The default value is
%%(rev)s_%%(slug)s
# Uncomment the line below if you want the files to be prepended with date and
time
diff --git a/airflow/providers/fab/auth_manager/models/db.py
b/airflow/providers/fab/auth_manager/models/db.py
index f72e1fcc65..efca4a1041 100644
--- a/airflow/providers/fab/auth_manager/models/db.py
+++ b/airflow/providers/fab/auth_manager/models/db.py
@@ -28,7 +28,7 @@ from airflow.utils.db_manager import BaseDBManager
PACKAGE_DIR = os.path.dirname(airflow.__file__)
_REVISION_HEADS_MAP: dict[str, str] = {
-"1.3.0": "6709f7a774b9",
+"1.4.0": "6709f7a774b9",
}
diff --git
a/airflow/providers/fab/migrations/versions/0001_1_3_0_placeholder_migration.py
b/airflow/providers/fab/migrations/versions/0001_1_4_0_placeholder_migration.py
similarity index 98%
rename from
airflow/providers/fab/migrations/versions/0001_1_3_0_placeholder_migration.py
rename to
airflow/providers/fab/migrations/versions/0001_1_4_0_placeholder_migration.py
index 685216779d..722c39198a 100644
---
a/airflow/providers/fab/migrations/versions/0001_1_3_0_placeholder_migration.py
+++
b/airflow/providers/fab/migrations/versions/0001_1_4_0_placeholder_migration.py
@@ -36,7 +36,7 @@ revision = "6709f7a774b9"
down_revision = None
branch_labels = None
depends_on = None
-fab_version = "1.3.0"
+fab_version = "1.4.0"
def upgrade() -> None: ...
diff --git a/docs/apache-airflow-providers-fab/migrations-ref.rst
b/docs/apache-airflow-providers-fab/migrations-ref.rst
index dbddd6e7d4..bf27837e84 100644
--- a/docs/apache-airflow-providers-fab/migrations-ref.rst
+++ b/docs/apache-airflow-providers-fab/migrations-ref.rst
@@ -34,7 +34,7 @@ Here's the list of all the Database Migrations that are
executed via when you ru
+-+--+---++
| Revision ID | Revises ID | Fab Version | Description
|
+=+==+===++
-| ``6709f7a774b9`` (head) | ``None`` | ``1.3.0`` | placeholder
migration. |
+| ``6709f7a774b9`` (head) | ``None`` | ``1.4.0`` | placeholder
migration. |
+-+--+---++
.. End of auto-generated table
(airflow) branch main updated: Refactor system test for Campaign Manager 360 (#42346)
This is an automated email from the ASF dual-hosted git repository.
eladkal pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new 447b221390 Refactor system test for Campaign Manager 360 (#42346)
447b221390 is described below
commit 447b221390547364c380d9dfa53ca7135a05f2b1
Author: max <42827971+moiseen...@users.noreply.github.com>
AuthorDate: Thu Sep 19 12:17:01 2024 +
Refactor system test for Campaign Manager 360 (#42346)
---
.../marketing_platform/example_campaign_manager.py | 155 ++---
1 file changed, 135 insertions(+), 20 deletions(-)
diff --git
a/tests/system/providers/google/marketing_platform/example_campaign_manager.py
b/tests/system/providers/google/marketing_platform/example_campaign_manager.py
index bace148179..f779458e78 100644
---
a/tests/system/providers/google/marketing_platform/example_campaign_manager.py
+++
b/tests/system/providers/google/marketing_platform/example_campaign_manager.py
@@ -17,18 +17,30 @@
# under the License.
"""
Example Airflow DAG that shows how to use CampaignManager.
+
+This system test relies on a service account with proper settings in Campaign
Manager 360.
+That's why before running this system test locally, make sure your service
account corresponds all the
+secrets that the DAG reads. If your service account doesn't have access but
you know another one which has
+then simply specify it in the environment variable CM360_IMPERSONATION_CHAIN.
"""
from __future__ import annotations
+import json
+import logging
import os
import time
import uuid
from datetime import datetime
from typing import cast
+from google.api_core.exceptions import NotFound
+
+from airflow.decorators import task
+from airflow.models import Connection
from airflow.models.dag import DAG
from airflow.models.xcom_arg import XComArg
+from airflow.providers.google.cloud.hooks.secret_manager import
GoogleCloudSecretManagerHook
from airflow.providers.google.cloud.operators.gcs import
GCSCreateBucketOperator, GCSDeleteBucketOperator
from airflow.providers.google.marketing_platform.operators.campaign_manager
import (
GoogleCampaignManagerBatchInsertConversionsOperator,
@@ -41,24 +53,36 @@ from
airflow.providers.google.marketing_platform.operators.campaign_manager impo
from airflow.providers.google.marketing_platform.sensors.campaign_manager
import (
GoogleCampaignManagerReportSensor,
)
+from airflow.settings import Session
from airflow.utils.trigger_rule import TriggerRule
from tests.system.providers.google import DEFAULT_GCP_SYSTEM_TEST_PROJECT_ID
ENV_ID = os.environ.get("SYSTEM_TESTS_ENV_ID")
PROJECT_ID = os.environ.get("SYSTEM_TESTS_GCP_PROJECT") or
DEFAULT_GCP_SYSTEM_TEST_PROJECT_ID
+CM360_IMPERSONATION_CHAIN = os.environ.get("IMPERSONATION_CHAIN", None)
+
+DAG_ID = "campaign_manager"
+
+SECRET_ACCOUNT_ID = "cm360_account_id"
+SECRET_DCLID = "cm360_dclid"
+SECRET_ENCRYPTION_ENTITY_ID = "cm360_encryption_entity_id"
+SECRET_FLOODLIGHT_ACTIVITY_ID = "cm360_floodlight_activity_id"
+SECRET_FLOODLIGHT_CONFIGURATION_ID = "cm360_floodlight_configuration_id"
+SECRET_USER_PROFILE_ID = "cm360_user_profile_id"
-DAG_ID = "example_campaign_manager"
+ACCOUNT_ID = "{{ task_instance.xcom_pull('get_account_id') }}"
+DCLID = "{{ task_instance.xcom_pull('get_dclid') }}"
+ENCRYPTION_ENTITY_ID = "{{ task_instance.xcom_pull('get_encryption_entity_id')
}}"
+FLOODLIGHT_ACTIVITY_ID = "{{
task_instance.xcom_pull('get_floodlight_activity_id') }}"
+FLOODLIGHT_CONFIGURATION_ID = "{{
task_instance.xcom_pull('get_floodlight_configuration_id') }}"
+USER_PROFILE_ID = "{{ task_instance.xcom_pull('get_user_profile_id') }}"
-PROFILE_ID = os.environ.get("MARKETING_PROFILE_ID", "123456789")
-FLOODLIGHT_ACTIVITY_ID = int(os.environ.get("FLOODLIGHT_ACTIVITY_ID", 12345))
-FLOODLIGHT_CONFIGURATION_ID =
int(os.environ.get("FLOODLIGHT_CONFIGURATION_ID", 12345))
-ENCRYPTION_ENTITY_ID = int(os.environ.get("ENCRYPTION_ENTITY_ID", 12345))
-DEVICE_ID = os.environ.get("DEVICE_ID", "12345")
BUCKET_NAME = f"bucket_{DAG_ID}_{ENV_ID}"
REPORT_NAME = f"report_{DAG_ID}_{ENV_ID}"
FILE_NAME = f"file_{DAG_ID}_{ENV_ID}"
-ACCOUNT_ID = f"account_{DAG_ID}_{ENV_ID}"
FORMAT = "CSV"
+CONNECTION_ID = f"connection_{DAG_ID}_{ENV_ID}"
+
# For more information, please check
# https://developers.google.com/doubleclick-advertisers/rest/v4/reports#type
@@ -85,7 +109,7 @@ CONVERSION = {
"kind": "dfareporting#conversion",
&
(airflow) branch main updated: Add new Google Search 360 Reporting Operators (#42255)
This is an automated email from the ASF dual-hosted git repository.
eladkal pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new 567926535a Add new Google Search 360 Reporting Operators (#42255)
567926535a is described below
commit 567926535a719f7ea93565e3fee3f2fbd9a03cb9
Author: M. Olcay Tercanlı
AuthorDate: Thu Sep 19 07:36:06 2024 +0200
Add new Google Search 360 Reporting Operators (#42255)
* Implement new Google Search 360 operators
Remove the decommissioned operators
Co-authored-by: Elad Kalif <45845474+elad...@users.noreply.github.com>
-
Co-authored-by: Jarek Potiuk
Co-authored-by: Elad Kalif <45845474+elad...@users.noreply.github.com>
---
airflow/providers/google/CHANGELOG.rst | 12 +
.../google/marketing_platform/hooks/search_ads.py | 199 +--
.../marketing_platform/operators/search_ads.py | 373 -
.../marketing_platform/sensors/search_ads.py | 92 -
airflow/providers/google/provider.yaml | 3 -
.../operators/marketing_platform/search_ads.rst| 87 +++--
tests/always/test_project_structure.py | 2 +-
.../marketing_platform/hooks/test_search_ads.py| 175 --
.../operators/test_search_ads.py | 240 +++--
.../marketing_platform/sensors/test_search_ads.py | 45 ---
.../marketing_platform/example_search_ads.py | 94 +++---
11 files changed, 770 insertions(+), 552 deletions(-)
diff --git a/airflow/providers/google/CHANGELOG.rst
b/airflow/providers/google/CHANGELOG.rst
index 64ae3b4e91..ff54591fd6 100644
--- a/airflow/providers/google/CHANGELOG.rst
+++ b/airflow/providers/google/CHANGELOG.rst
@@ -27,6 +27,18 @@
Changelog
-
+Main
+...
+
+.. warning::
+ The previous Search Ads 360 Reporting API
<https://developers.google.com/search-ads/v2/how-tos/reporting>
+ (which is currently in use in google-provider) was already decommissioned on
June 30, 2024
+ (see details <https://developers.google.com/search-ads/v2/migration>).
+ All new reporting development should use the new Search Ads 360 Reporting
API.
+ Currently, the Reporting operators, sensors and hooks are failing due to the
decommission.
+ The new API is not a replacement for the old one, it has a different
approach and endpoints.
+ Therefore, new operators implemented for the new API.
+
10.22.0
...
diff --git a/airflow/providers/google/marketing_platform/hooks/search_ads.py
b/airflow/providers/google/marketing_platform/hooks/search_ads.py
index 85cc25c458..d57a89a821 100644
--- a/airflow/providers/google/marketing_platform/hooks/search_ads.py
+++ b/airflow/providers/google/marketing_platform/hooks/search_ads.py
@@ -19,73 +19,212 @@
from __future__ import annotations
-from typing import Any, Sequence
+from functools import cached_property
+from typing import TYPE_CHECKING, Any, Sequence
+from google.oauth2.credentials import Credentials
from googleapiclient.discovery import build
+from airflow.exceptions import AirflowException
from airflow.providers.google.common.hooks.base_google import GoogleBaseHook
+if TYPE_CHECKING:
+from googleapiclient.discovery import Resource
-class GoogleSearchAdsHook(GoogleBaseHook):
-"""Hook for Google Search Ads 360."""
+
+class GoogleSearchAdsReportingHook(GoogleBaseHook):
+"""Hook for the Google Search Ads 360 Reporting API."""
_conn: build | None = None
+default_api_version: str = "v0"
def __init__(
self,
-api_version: str = "v2",
-gcp_conn_id: str = "google_cloud_default",
-delegate_to: str | None = None,
-impersonation_chain: str | Sequence[str] | None = None,
+api_version: str | None = None,
+gcp_conn_id: str = "google_search_ads_default",
) -> None:
super().__init__(
gcp_conn_id=gcp_conn_id,
-delegate_to=delegate_to,
-impersonation_chain=impersonation_chain,
)
-self.api_version = api_version
+self.api_version = api_version or self.default_api_version
-def get_conn(self):
-"""Retrieve connection to Google SearchAds."""
+def _get_config(self) -> None:
+"""
+Set up Google Search Ads config from Connection.
+
+This pulls the connections from db, and uses it to set up
+``google_search_ads_client``.
+"""
+conn = self.get_connection(self.gcp_conn_id)
+if "google_search_ads_client" not in conn.extra_dejson:
+raise AirflowException("google_search_ads_client not found in
extra field")
+
+self.
(airflow) branch v2-10-test updated: fix: Correct docstring format in _get_template_context (#42244) (#42272)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a commit to branch v2-10-test in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/v2-10-test by this push: new 8808224d36 fix: Correct docstring format in _get_template_context (#42244) (#42272) 8808224d36 is described below commit 8808224d368195f451647f80b046897365cede17 Author: rom sharon <33751805+romsharo...@users.noreply.github.com> AuthorDate: Tue Sep 17 13:33:53 2024 +0300 fix: Correct docstring format in _get_template_context (#42244) (#42272) (cherry picked from commit 8fe286108720ba1f9a303c066022349d6656f735) Co-authored-by: Wonseok Yang --- airflow/models/taskinstance.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/airflow/models/taskinstance.py b/airflow/models/taskinstance.py index b34c71bc9f..c180d7b508 100644 --- a/airflow/models/taskinstance.py +++ b/airflow/models/taskinstance.py @@ -943,7 +943,7 @@ def _get_template_context( Return TI Context. :param task_instance: the task instance for the task -:param dag for the task +:param dag: dag for the task :param session: SQLAlchemy ORM Session :param ignore_param_exceptions: flag to suppress value exceptions while initializing the ParamsDict
(airflow) branch main updated: Deprecated kerberos auth removed (#41693)
This is an automated email from the ASF dual-hosted git repository.
eladkal pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new db7f92787a Deprecated kerberos auth removed (#41693)
db7f92787a is described below
commit db7f92787ab6f0e9646cc0e2a7ad5044f1d9ade8
Author: Gopal Dirisala <39794726+dir...@users.noreply.github.com>
AuthorDate: Tue Sep 17 08:48:11 2024 +0530
Deprecated kerberos auth removed (#41693)
* deprecatd kerberos auth airflow.api.auth.backend.kerberos_auth and
airflow.auth.managers.fab.api.auth.backend.kerberos_aut removed
* news fragment added
* deprecatd kerberos auth airflow.api.auth.backend.kerberos_auth and
airflow.auth.managers.fab.api.auth.backend.kerberos_aut removed
---
airflow/api/auth/backend/kerberos_auth.py | 182 -
.../managers/fab/api/auth/backend/kerberos_auth.py | 43 -
.../auth_manager/api/auth/backend/kerberos_auth.py | 114 -
.../auth-manager/api-authentication.rst| 2 +-
newsfragments/41693.significant.rst| 1 +
.../api/auth/backend/test_kerberos_auth.py | 5 +-
6 files changed, 109 insertions(+), 238 deletions(-)
diff --git a/airflow/api/auth/backend/kerberos_auth.py
b/airflow/api/auth/backend/kerberos_auth.py
deleted file mode 100644
index 100d52dd77..00
--- a/airflow/api/auth/backend/kerberos_auth.py
+++ /dev/null
@@ -1,182 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied. See the License for the
-# specific language governing permissions and limitations
-# under the License.
-from __future__ import annotations
-
-import warnings
-
-from airflow.exceptions import RemovedInAirflow3Warning
-from airflow.utils.airflow_flask_app import get_airflow_app
-
-#
-# Copyright (c) 2013, Michael Komitee
-# All rights reserved.
-#
-# Redistribution and use in source and binary forms, with or without
modification,
-# are permitted provided that the following conditions are met:
-#
-# 1. Redistributions of source code must retain the above copyright notice,
-# this list of conditions and the following disclaimer.
-#
-# 2. Redistributions in binary form must reproduce the above copyright notice,
-# this list of conditions and the following disclaimer in the documentation
-# and/or other materials provided with the distribution.
-#
-# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND
-# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
-# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
-# DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR
-# ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES
-# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
-# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
-# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
-# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
-# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-"""Kerberos authentication module"""
-import logging
-import os
-from functools import wraps
-from typing import TYPE_CHECKING, Callable, NamedTuple, TypeVar, cast
-
-import kerberos
-from flask import Response, g, make_response, request
-
-from airflow.configuration import conf
-from airflow.utils.net import getfqdn
-
-if TYPE_CHECKING:
-from airflow.auth.managers.models.base_user import BaseUser
-
-log = logging.getLogger(__name__)
-
-
-class KerberosService:
-"""Class to keep information about the Kerberos Service initialized."""
-
-def __init__(self):
-self.service_name = None
-
-
-class _KerberosAuth(NamedTuple):
-return_code: int | None
-user: str = ""
-token: str | None = None
-
-
-# Stores currently initialized Kerberos Service
-_KERBEROS_SERVICE = KerberosService()
-
-
-def init_app(app):
-"""Initialize application with kerberos."""
-hostname = app.config.get("SERVER_NAME")
-if not hostname:
-
(airflow) branch main updated: ECSExecutor: Drop params that aren't compatible with EC2 (#42228)
This is an automated email from the ASF dual-hosted git repository.
eladkal pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new 4afc2569b1 ECSExecutor: Drop params that aren't compatible with EC2
(#42228)
4afc2569b1 is described below
commit 4afc2569b1f7e1d83cd20dead8933194c08cc8df
Author: Niko Oliveira
AuthorDate: Sun Sep 15 22:56:17 2024 -0700
ECSExecutor: Drop params that aren't compatible with EC2 (#42228)
If using the ECS executor with the EC2 launch type, then
platform_version and assign_public_ip are not applicable
and cause the Boto call to fail.
---
.../amazon/aws/executors/ecs/ecs_executor_config.py | 17 +
airflow/providers/amazon/aws/executors/ecs/utils.py | 10 --
2 files changed, 21 insertions(+), 6 deletions(-)
diff --git a/airflow/providers/amazon/aws/executors/ecs/ecs_executor_config.py
b/airflow/providers/amazon/aws/executors/ecs/ecs_executor_config.py
index 4f57b72d96..c0b6ea5dd5 100644
--- a/airflow/providers/amazon/aws/executors/ecs/ecs_executor_config.py
+++ b/airflow/providers/amazon/aws/executors/ecs/ecs_executor_config.py
@@ -35,6 +35,8 @@ from json import JSONDecodeError
from airflow.configuration import conf
from airflow.providers.amazon.aws.executors.ecs.utils import (
CONFIG_GROUP_NAME,
+ECS_LAUNCH_TYPE_EC2,
+ECS_LAUNCH_TYPE_FARGATE,
AllEcsConfigKeys,
RunTaskKwargsConfigKeys,
camelize_dict_keys,
@@ -56,13 +58,15 @@ def _fetch_config_values() -> dict[str, str]:
def build_task_kwargs() -> dict:
+all_config_keys = AllEcsConfigKeys()
# This will put some kwargs at the root of the dictionary that do NOT
belong there. However,
# the code below expects them to be there and will rearrange them as
necessary.
task_kwargs = _fetch_config_values()
task_kwargs.update(_fetch_templated_kwargs())
-has_launch_type: bool = "launch_type" in task_kwargs
-has_capacity_provider: bool = "capacity_provider_strategy" in task_kwargs
+has_launch_type: bool = all_config_keys.LAUNCH_TYPE in task_kwargs
+has_capacity_provider: bool = all_config_keys.CAPACITY_PROVIDER_STRATEGY
in task_kwargs
+is_launch_type_ec2: bool = task_kwargs.get(all_config_keys.LAUNCH_TYPE,
None) == ECS_LAUNCH_TYPE_EC2
if has_capacity_provider and has_launch_type:
raise ValueError(
@@ -75,7 +79,12 @@ def build_task_kwargs() -> dict:
# the final fallback.
cluster =
EcsHook().conn.describe_clusters(clusters=[task_kwargs["cluster"]])["clusters"][0]
if not cluster.get("defaultCapacityProviderStrategy"):
-task_kwargs["launch_type"] = "FARGATE"
+task_kwargs[all_config_keys.LAUNCH_TYPE] = ECS_LAUNCH_TYPE_FARGATE
+
+# If you're using the EC2 launch type, you should not/can not provide the
platform_version. In this
+# case we'll drop it on the floor on behalf of the user, instead of
throwing an exception.
+if is_launch_type_ec2:
+task_kwargs.pop(all_config_keys.PLATFORM_VERSION, None)
# There can only be 1 count of these containers
task_kwargs["count"] = 1 # type: ignore
@@ -105,7 +114,7 @@ def build_task_kwargs() -> dict:
"awsvpcConfiguration": {
"subnets": str(subnets).split(",") if subnets else None,
"securityGroups": str(security_groups).split(",") if
security_groups else None,
-"assignPublicIp": parse_assign_public_ip(assign_public_ip),
+"assignPublicIp": parse_assign_public_ip(assign_public_ip,
is_launch_type_ec2),
}
}
)
diff --git a/airflow/providers/amazon/aws/executors/ecs/utils.py
b/airflow/providers/amazon/aws/executors/ecs/utils.py
index cb730acac9..10d9162d5a 100644
--- a/airflow/providers/amazon/aws/executors/ecs/utils.py
+++ b/airflow/providers/amazon/aws/executors/ecs/utils.py
@@ -40,6 +40,9 @@ CommandType = List[str]
ExecutorConfigFunctionType = Callable[[CommandType], dict]
ExecutorConfigType = Dict[str, Any]
+ECS_LAUNCH_TYPE_EC2 = "EC2"
+ECS_LAUNCH_TYPE_FARGATE = "FARGATE"
+
CONFIG_GROUP_NAME = "aws_ecs_executor"
CONFIG_DEFAULTS = {
@@ -247,9 +250,12 @@ def _recursive_flatten_dict(nested_dict):
return dict(items)
-def parse_assign_public_ip(assign_public_ip):
+def parse_assign_public_ip(assign_public_ip, is_launch_type_ec2=False):
"""Convert "assign_public_ip" from True/False to ENABLE/DISABLE."""
-return "ENABLED" if assign_public_ip == "True" else "DISABLED"
+# If the launch type is EC2, you cannot/should not provide the
assignPublicIp parameter (which is
+# specific to Fargate)
+if not is_launch_type_ec2:
+return "ENABLED" if assign_public_ip == "True" else "DISABLED"
def camelize_dict_keys(nested_dict) -> dict:
(airflow) branch main updated (3f6497b528 -> 26b3b3d81f)
This is an automated email from the ASF dual-hosted git repository.
eladkal pushed a change to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
from 3f6497b528 Fix documentation for cpu and memory usage (#42147)
add 26b3b3d81f Add pgbouncer and statsd ingress (#41759)
No new revisions were added by this update.
Summary of changes:
.../pgbouncer-ingress.yaml}| 43 +++---
chart/templates/pgbouncer/pgbouncer-service.yaml | 2 +-
.../statsd-ingress.yaml} | 43 +++---
chart/values.schema.json | 170 +
chart/values.yaml | 60
helm_tests/other/test_pgbouncer.py | 35 -
helm_tests/other/test_statsd.py| 31
7 files changed, 331 insertions(+), 53 deletions(-)
copy chart/templates/{flower/flower-ingress.yaml =>
pgbouncer/pgbouncer-ingress.yaml} (56%)
copy chart/templates/{flower/flower-ingress.yaml =>
statsd/statsd-ingress.yaml} (56%)
(airflow) branch main updated: Fix MyPy Checks introduced by newer version of InfluxDB (#42227)
This is an automated email from the ASF dual-hosted git repository.
eladkal pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new 1272c1cc61 Fix MyPy Checks introduced by newer version of InfluxDB
(#42227)
1272c1cc61 is described below
commit 1272c1cc6146ab80d80a26daaf7db933fe342407
Author: Jarek Potiuk
AuthorDate: Fri Sep 13 11:21:49 2024 -0700
Fix MyPy Checks introduced by newer version of InfluxDB (#42227)
---
airflow/providers/influxdb/hooks/influxdb.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/airflow/providers/influxdb/hooks/influxdb.py
b/airflow/providers/influxdb/hooks/influxdb.py
index b1b001730a..902b968a05 100644
--- a/airflow/providers/influxdb/hooks/influxdb.py
+++ b/airflow/providers/influxdb/hooks/influxdb.py
@@ -58,7 +58,7 @@ class InfluxDBHook(BaseHook):
super().__init__(*args, **kwargs)
self.influxdb_conn_id = conn_id
self.connection = kwargs.pop("connection", None)
-self.client = None
+self.client: InfluxDBClient | None = None
self.extras: dict = {}
self.uri = None
@@ -78,7 +78,7 @@ class InfluxDBHook(BaseHook):
),
}
-def get_client(self, uri, kwargs):
+def get_client(self, uri, kwargs) -> InfluxDBClient:
return InfluxDBClient(url=uri, **kwargs)
def get_uri(self, conn: Connection):
(airflow) branch v2-10-test updated: update requires (#42223)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a commit to branch v2-10-test in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/v2-10-test by this push: new 50445e34e4 update requires (#42223) 50445e34e4 is described below commit 50445e34e438c4ae5aca53cc90623c66f8270534 Author: rom sharon <33751805+romsharo...@users.noreply.github.com> AuthorDate: Fri Sep 13 11:20:17 2024 -0700 update requires (#42223) * update requires * update requires (cherry picked from commit 44eba061c6dcfae75d0b00ec80196f1607536e4a) --- pyproject.toml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/pyproject.toml b/pyproject.toml index 194c4b268c..618a6cb782 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -30,7 +30,7 @@ requires = [ "pluggy==1.5.0", "smmap==5.0.1", "tomli==2.0.1; python_version < '3.11'", -"trove-classifiers==2024.7.2", +"trove-classifiers==2024.9.12", ] build-backend = "hatchling.build"
(airflow) branch main updated (d113ce6249 -> 70865f344a)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from d113ce6249 fix broken main (#42221) add 70865f344a Added infoworks.io in INTOTHEWILD.md (#42212) No new revisions were added by this update. Summary of changes: INTHEWILD.md | 1 + 1 file changed, 1 insertion(+)
(airflow) branch main updated (9df31442bc -> d113ce6249)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 9df31442bc Page routing in new UI (#42185) add d113ce6249 fix broken main (#42221) No new revisions were added by this update. Summary of changes: pyproject.toml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
(airflow) branch main updated (d2cb6119fc -> 25cc8b267f)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from d2cb6119fc document that running task instances will be marked as skipped when a dagrun times out (#41410) add 25cc8b267f Add Airflow 3 development readme (#41457) No new revisions were added by this update. Summary of changes: dev/README_AIRFLOW3_DEV.md | 160 + 1 file changed, 160 insertions(+) create mode 100644 dev/README_AIRFLOW3_DEV.md
(airflow) branch revert-41736-feature/remove-deprecated-params-from-core-operators deleted (was c1e54dd3b1)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch revert-41736-feature/remove-deprecated-params-from-core-operators in repository https://gitbox.apache.org/repos/asf/airflow.git was c1e54dd3b1 Revert "Remove deprecated parameters from airflow (core) Operators (#41736)" The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
(airflow) branch main updated: docstring update (#41929)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 86af316c87 docstring update (#41929) 86af316c87 is described below commit 86af316c8716c0535e8bebb95c10dfa8227a5049 Author: GPK AuthorDate: Mon Sep 2 12:36:27 2024 +0100 docstring update (#41929) --- airflow/providers/cncf/kubernetes/hooks/kubernetes.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/airflow/providers/cncf/kubernetes/hooks/kubernetes.py b/airflow/providers/cncf/kubernetes/hooks/kubernetes.py index c3b473bea3..9f7e33696e 100644 --- a/airflow/providers/cncf/kubernetes/hooks/kubernetes.py +++ b/airflow/providers/cncf/kubernetes/hooks/kubernetes.py @@ -845,6 +845,7 @@ class AsyncKubernetesHook(KubernetesHook): :param name: Name of Pod to fetch. :param namespace: Namespace of the Pod. :param container_name: name of the container within the pod to monitor +:param poll_interval: Interval in seconds between polling the container status """ while True: pod = await self.get_pod(name=name, namespace=namespace) @@ -862,6 +863,7 @@ class AsyncKubernetesHook(KubernetesHook): :param name: Name of Pod to fetch. :param namespace: Namespace of the Pod. :param container_name: name of the container within the pod to monitor +:param poll_interval: Interval in seconds between polling the container status """ while True: pod = await self.get_pod(name=name, namespace=namespace)
(airflow) branch main updated: Fix treatment of "#" in S3Hook.parse_s3_url() (#41796)
This is an automated email from the ASF dual-hosted git repository.
eladkal pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new 062fb3a438 Fix treatment of "#" in S3Hook.parse_s3_url() (#41796)
062fb3a438 is described below
commit 062fb3a4380d3e849f5eec8b9b21eb7065c597ce
Author: GlenboLake
AuthorDate: Thu Aug 29 17:31:43 2024 -0400
Fix treatment of "#" in S3Hook.parse_s3_url() (#41796)
The current implementation of parse_s3_url will truncate a key if it
contains
an octothorpe character. By passing the allow_fragments=False argument to
urlsplit, keys will be correctly parsed.
---
airflow/providers/amazon/aws/hooks/s3.py| 2 +-
tests/providers/amazon/aws/hooks/test_s3.py | 74 +
2 files changed, 56 insertions(+), 20 deletions(-)
diff --git a/airflow/providers/amazon/aws/hooks/s3.py
b/airflow/providers/amazon/aws/hooks/s3.py
index 5f2c136640..76aed19782 100644
--- a/airflow/providers/amazon/aws/hooks/s3.py
+++ b/airflow/providers/amazon/aws/hooks/s3.py
@@ -238,7 +238,7 @@ class S3Hook(AwsBaseHook):
valid_s3_virtual_hosted_format =
"https://bucket-name.s3.region-code.amazonaws.com/key-name";
format = s3url.split("//")
if re.match(r"s3[na]?:", format[0], re.IGNORECASE):
-parsed_url = urlsplit(s3url)
+parsed_url = urlsplit(s3url, allow_fragments=False)
if not parsed_url.netloc:
raise S3HookUriParseFailure(
"Please provide a bucket name using a valid format of the
form: "
diff --git a/tests/providers/amazon/aws/hooks/test_s3.py
b/tests/providers/amazon/aws/hooks/test_s3.py
index 9dade82004..97696c64b6 100644
--- a/tests/providers/amazon/aws/hooks/test_s3.py
+++ b/tests/providers/amazon/aws/hooks/test_s3.py
@@ -81,25 +81,61 @@ class TestAwsS3Hook:
with pytest.raises(TypeError, match="transfer_config_args expected
dict, got .*"):
S3Hook(transfer_config_args=transfer_config_args)
-def test_parse_s3_url(self):
-parsed = S3Hook.parse_s3_url("s3://test/this/is/not/a-real-key.txt")
-assert parsed == ("test", "this/is/not/a-real-key.txt"), "Incorrect
parsing of the s3 url"
-
-def test_parse_s3_url_s3a_style(self):
-parsed = S3Hook.parse_s3_url("s3a://test/this/is/not/a-real-key.txt")
-assert parsed == ("test", "this/is/not/a-real-key.txt"), "Incorrect
parsing of the s3 url"
-
-def test_parse_s3_url_s3n_style(self):
-parsed = S3Hook.parse_s3_url("s3n://test/this/is/not/a-real-key.txt")
-assert parsed == ("test", "this/is/not/a-real-key.txt"), "Incorrect
parsing of the s3 url"
-
-def test_parse_s3_url_path_style(self):
-parsed =
S3Hook.parse_s3_url("https://s3.us-west-2.amazonaws.com/DOC-EXAMPLE-BUCKET1/test.jpg";)
-assert parsed == ("DOC-EXAMPLE-BUCKET1", "test.jpg"), "Incorrect
parsing of the s3 url"
-
-def test_parse_s3_url_virtual_hosted_style(self):
-parsed =
S3Hook.parse_s3_url("https://DOC-EXAMPLE-BUCKET1.s3.us-west-2.amazonaws.com/test.png";)
-assert parsed == ("DOC-EXAMPLE-BUCKET1", "test.png"), "Incorrect
parsing of the s3 url"
+@pytest.mark.parametrize(
+"url, expected",
+[
+pytest.param(
+"s3://test/this/is/not/a-real-key.txt", ("test",
"this/is/not/a-real-key.txt"), id="s3 style"
+),
+pytest.param(
+"s3a://test/this/is/not/a-real-key.txt",
+("test", "this/is/not/a-real-key.txt"),
+id="s3a style",
+),
+pytest.param(
+"s3n://test/this/is/not/a-real-key.txt",
+("test", "this/is/not/a-real-key.txt"),
+id="s3n style",
+),
+pytest.param(
+
"https://s3.us-west-2.amazonaws.com/DOC-EXAMPLE-BUCKET1/test.jpg";,
+("DOC-EXAMPLE-BUCKET1", "test.jpg"),
+id="path style",
+),
+pytest.param(
+
"https://DOC-EXAMPLE-BUCKET1.s3.us-west-2.amazonaws.com/test.png";,
+("DOC-EXAMPLE-BUCKET1", "test.png"),
+id="virtual hosted style",
+),
+pytest.param(
+"s3://test/this/is/not/a-real-key #2.txt",
+("test", "this/is/not/a-re
(airflow) branch main updated: Make code style fix in Dataform system test (#41750)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 55f9068d53 Make code style fix in Dataform system test (#41750) 55f9068d53 is described below commit 55f9068d53ad2189ce4843f34c10485a12ff1e92 Author: Maksim AuthorDate: Thu Aug 29 09:43:28 2024 +0200 Make code style fix in Dataform system test (#41750) --- tests/system/providers/google/cloud/dataform/example_dataform.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/system/providers/google/cloud/dataform/example_dataform.py b/tests/system/providers/google/cloud/dataform/example_dataform.py index 78ea252f5c..3f2ad987f2 100644 --- a/tests/system/providers/google/cloud/dataform/example_dataform.py +++ b/tests/system/providers/google/cloud/dataform/example_dataform.py @@ -269,7 +269,7 @@ with DAG( # [END howto_operator_remove_directory] delete_dataset = BigQueryDeleteDatasetOperator( -task_id="delete_dataset", +task_id="delete-dataset", dataset_id=DATAFORM_SCHEMA_NAME, delete_contents=True, trigger_rule=TriggerRule.ALL_DONE,
(airflow) branch main updated (032ac87b1d -> c2d5ef4525)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 032ac87b1d logout link in no roles error page fix (#41813) add c2d5ef4525 Update provider to use Airbyte API Python SDK (#41122) No new revisions were added by this update. Summary of changes: airflow/providers/airbyte/CHANGELOG.rst| 15 ++ airflow/providers/airbyte/hooks/airbyte.py | 241 - airflow/providers/airbyte/operators/airbyte.py | 74 +++ airflow/providers/airbyte/provider.yaml| 3 +- airflow/providers/airbyte/sensors/airbyte.py | 46 ++-- airflow/providers/airbyte/triggers/airbyte.py | 20 +- dev/breeze/tests/test_selective_checks.py | 32 +-- .../connections.rst| 40 ++-- .../operators/airbyte.rst | 6 +- generated/provider_dependencies.json | 6 +- tests/providers/airbyte/hooks/test_airbyte.py | 159 -- tests/providers/airbyte/operators/test_airbyte.py | 24 +- tests/providers/airbyte/sensors/test_airbyte.py| 42 ++-- tests/providers/airbyte/triggers/test_airbyte.py | 28 +-- 14 files changed, 350 insertions(+), 386 deletions(-)
(airflow-site) branch add-documentation-2024-08-25 deleted (was dbfad7341e)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch add-documentation-2024-08-25 in repository https://gitbox.apache.org/repos/asf/airflow-site.git was dbfad7341e move tabular provider index entry to removed section The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
svn commit: r71132 - /dev/airflow/providers/ /release/airflow/providers/
Author: eladkal Date: Wed Aug 28 10:28:13 2024 New Revision: 71132 Log: Release Airflow Providers on 2024-08-28 Added: release/airflow/providers/apache_airflow_providers_celery-3.8.1-py3-none-any.whl - copied unchanged from r71131, dev/airflow/providers/apache_airflow_providers_celery-3.8.1-py3-none-any.whl release/airflow/providers/apache_airflow_providers_celery-3.8.1-py3-none-any.whl.asc - copied unchanged from r71131, dev/airflow/providers/apache_airflow_providers_celery-3.8.1-py3-none-any.whl.asc release/airflow/providers/apache_airflow_providers_celery-3.8.1-py3-none-any.whl.sha512 - copied unchanged from r71131, dev/airflow/providers/apache_airflow_providers_celery-3.8.1-py3-none-any.whl.sha512 release/airflow/providers/apache_airflow_providers_celery-3.8.1.tar.gz - copied unchanged from r71131, dev/airflow/providers/apache_airflow_providers_celery-3.8.1.tar.gz release/airflow/providers/apache_airflow_providers_celery-3.8.1.tar.gz.asc - copied unchanged from r71131, dev/airflow/providers/apache_airflow_providers_celery-3.8.1.tar.gz.asc release/airflow/providers/apache_airflow_providers_celery-3.8.1.tar.gz.sha512 - copied unchanged from r71131, dev/airflow/providers/apache_airflow_providers_celery-3.8.1.tar.gz.sha512 release/airflow/providers/apache_airflow_providers_cncf_kubernetes-8.4.1-py3-none-any.whl - copied unchanged from r71131, dev/airflow/providers/apache_airflow_providers_cncf_kubernetes-8.4.1-py3-none-any.whl release/airflow/providers/apache_airflow_providers_cncf_kubernetes-8.4.1-py3-none-any.whl.asc - copied unchanged from r71131, dev/airflow/providers/apache_airflow_providers_cncf_kubernetes-8.4.1-py3-none-any.whl.asc release/airflow/providers/apache_airflow_providers_cncf_kubernetes-8.4.1-py3-none-any.whl.sha512 - copied unchanged from r71131, dev/airflow/providers/apache_airflow_providers_cncf_kubernetes-8.4.1-py3-none-any.whl.sha512 release/airflow/providers/apache_airflow_providers_cncf_kubernetes-8.4.1.tar.gz - copied unchanged from r71131, dev/airflow/providers/apache_airflow_providers_cncf_kubernetes-8.4.1.tar.gz release/airflow/providers/apache_airflow_providers_cncf_kubernetes-8.4.1.tar.gz.asc - copied unchanged from r71131, dev/airflow/providers/apache_airflow_providers_cncf_kubernetes-8.4.1.tar.gz.asc release/airflow/providers/apache_airflow_providers_cncf_kubernetes-8.4.1.tar.gz.sha512 - copied unchanged from r71131, dev/airflow/providers/apache_airflow_providers_cncf_kubernetes-8.4.1.tar.gz.sha512 release/airflow/providers/apache_airflow_providers_openlineage-1.11.0-py3-none-any.whl - copied unchanged from r71131, dev/airflow/providers/apache_airflow_providers_openlineage-1.11.0-py3-none-any.whl release/airflow/providers/apache_airflow_providers_openlineage-1.11.0-py3-none-any.whl.asc - copied unchanged from r71131, dev/airflow/providers/apache_airflow_providers_openlineage-1.11.0-py3-none-any.whl.asc release/airflow/providers/apache_airflow_providers_openlineage-1.11.0-py3-none-any.whl.sha512 - copied unchanged from r71131, dev/airflow/providers/apache_airflow_providers_openlineage-1.11.0-py3-none-any.whl.sha512 release/airflow/providers/apache_airflow_providers_openlineage-1.11.0.tar.gz - copied unchanged from r71131, dev/airflow/providers/apache_airflow_providers_openlineage-1.11.0.tar.gz release/airflow/providers/apache_airflow_providers_openlineage-1.11.0.tar.gz.asc - copied unchanged from r71131, dev/airflow/providers/apache_airflow_providers_openlineage-1.11.0.tar.gz.asc release/airflow/providers/apache_airflow_providers_openlineage-1.11.0.tar.gz.sha512 - copied unchanged from r71131, dev/airflow/providers/apache_airflow_providers_openlineage-1.11.0.tar.gz.sha512 release/airflow/providers/apache_airflow_providers_ssh-3.13.1-py3-none-any.whl - copied unchanged from r71131, dev/airflow/providers/apache_airflow_providers_ssh-3.13.1-py3-none-any.whl release/airflow/providers/apache_airflow_providers_ssh-3.13.1-py3-none-any.whl.asc - copied unchanged from r71131, dev/airflow/providers/apache_airflow_providers_ssh-3.13.1-py3-none-any.whl.asc release/airflow/providers/apache_airflow_providers_ssh-3.13.1-py3-none-any.whl.sha512 - copied unchanged from r71131, dev/airflow/providers/apache_airflow_providers_ssh-3.13.1-py3-none-any.whl.sha512 release/airflow/providers/apache_airflow_providers_ssh-3.13.1.tar.gz - copied unchanged from r71131, dev/airflow/providers/apache_airflow_providers_ssh-3.13.1.tar.gz release/airflow/providers/apache_airflow_providers_ssh-3.13.1.tar.gz.asc - copied unchanged from r71131, dev/airflow/providers/apache_airflow_providers_ssh-3.13.1.tar.gz.asc release/airflow/providers/apache_airflow_providers_ssh-3.13.1.tar.gz.sha512 - copied unchanged from r71131, dev/airflow
(airflow) branch revert-41736-feature/remove-deprecated-params-from-core-operators created (now c1e54dd3b1)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch revert-41736-feature/remove-deprecated-params-from-core-operators in repository https://gitbox.apache.org/repos/asf/airflow.git at c1e54dd3b1 Revert "Remove deprecated parameters from airflow (core) Operators (#41736)" This branch includes the following new commits: new c1e54dd3b1 Revert "Remove deprecated parameters from airflow (core) Operators (#41736)" The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
(airflow) 01/01: Revert "Remove deprecated parameters from airflow (core) Operators (#41736)"
This is an automated email from the ASF dual-hosted git repository.
eladkal pushed a commit to branch
revert-41736-feature/remove-deprecated-params-from-core-operators
in repository https://gitbox.apache.org/repos/asf/airflow.git
commit c1e54dd3b1d60797cc6ddefc8992a19ff0398f7e
Author: Elad Kalif <45845474+elad...@users.noreply.github.com>
AuthorDate: Tue Aug 27 19:28:44 2024 +0300
Revert "Remove deprecated parameters from airflow (core) Operators (#41736)"
This reverts commit 27fe45bdfe25eabba24c9d4de0b2e1807ea36840.
---
airflow/operators/datetime.py | 11 ++-
airflow/operators/trigger_dagrun.py| 12
airflow/operators/weekday.py | 11 +++
newsfragments/41736.significant.rst| 7 ---
tests/operators/test_datetime.py | 16
tests/operators/test_trigger_dagrun.py | 29 -
tests/operators/test_weekday.py| 20
7 files changed, 97 insertions(+), 9 deletions(-)
diff --git a/airflow/operators/datetime.py b/airflow/operators/datetime.py
index 4455b84dd3..732b380077 100644
--- a/airflow/operators/datetime.py
+++ b/airflow/operators/datetime.py
@@ -17,9 +17,10 @@
from __future__ import annotations
import datetime
+import warnings
from typing import TYPE_CHECKING, Iterable
-from airflow.exceptions import AirflowException
+from airflow.exceptions import AirflowException, RemovedInAirflow3Warning
from airflow.operators.branch import BaseBranchOperator
from airflow.utils import timezone
@@ -55,6 +56,7 @@ class BranchDateTimeOperator(BaseBranchOperator):
target_lower: datetime.datetime | datetime.time | None,
target_upper: datetime.datetime | datetime.time | None,
use_task_logical_date: bool = False,
+use_task_execution_date: bool = False,
**kwargs,
) -> None:
super().__init__(**kwargs)
@@ -69,6 +71,13 @@ class BranchDateTimeOperator(BaseBranchOperator):
self.follow_task_ids_if_true = follow_task_ids_if_true
self.follow_task_ids_if_false = follow_task_ids_if_false
self.use_task_logical_date = use_task_logical_date
+if use_task_execution_date:
+self.use_task_logical_date = use_task_execution_date
+warnings.warn(
+"Parameter ``use_task_execution_date`` is deprecated. Use
``use_task_logical_date``.",
+RemovedInAirflow3Warning,
+stacklevel=2,
+)
def choose_branch(self, context: Context) -> str | Iterable[str]:
if self.use_task_logical_date:
diff --git a/airflow/operators/trigger_dagrun.py
b/airflow/operators/trigger_dagrun.py
index 539506ff9b..2521297dcf 100644
--- a/airflow/operators/trigger_dagrun.py
+++ b/airflow/operators/trigger_dagrun.py
@@ -20,6 +20,7 @@ from __future__ import annotations
import datetime
import json
import time
+import warnings
from typing import TYPE_CHECKING, Any, Sequence, cast
from sqlalchemy import select
@@ -33,6 +34,7 @@ from airflow.exceptions import (
AirflowSkipException,
DagNotFound,
DagRunAlreadyExists,
+RemovedInAirflow3Warning,
)
from airflow.models.baseoperator import BaseOperator
from airflow.models.baseoperatorlink import BaseOperatorLink
@@ -108,6 +110,7 @@ class TriggerDagRunOperator(BaseOperator):
DAG for the same logical date already exists.
:param deferrable: If waiting for completion, whether or not to defer the
task until done,
default is ``False``.
+:param execution_date: Deprecated parameter; same as ``logical_date``.
"""
template_fields: Sequence[str] = (
@@ -136,6 +139,7 @@ class TriggerDagRunOperator(BaseOperator):
failed_states: list[str | DagRunState] | None = None,
skip_when_already_exists: bool = False,
deferrable: bool = conf.getboolean("operators", "default_deferrable",
fallback=False),
+execution_date: str | datetime.datetime | None = None,
**kwargs,
) -> None:
super().__init__(**kwargs)
@@ -156,6 +160,14 @@ class TriggerDagRunOperator(BaseOperator):
self.skip_when_already_exists = skip_when_already_exists
self._defer = deferrable
+if execution_date is not None:
+warnings.warn(
+"Parameter 'execution_date' is deprecated. Use 'logical_date'
instead.",
+RemovedInAirflow3Warning,
+stacklevel=2,
+)
+logical_date = execution_date
+
if logical_date is not None and not isinstance(logical_date, (str,
datetime.datetime)):
type_name = type(logical_date).__name__
raise TypeError(
diff --git a/airflow/operators/weekday.py b/airflow/operators/weekday.py
index f59a1da134..af3e332899 100644
--- a/airflow/operators/weekday.py
+++ b/airflow/operators/weekda
(airflow) branch main updated (59dc98178b -> 8019a75f76)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 59dc98178b Separate FAB migration from Core Airflow migration (#41437) add 8019a75f76 revert removing soft_fail from TimeDeltaSensorAsync (#41741) No new revisions were added by this update. Summary of changes: airflow/sensors/time_delta.py | 9 +++-- 1 file changed, 7 insertions(+), 2 deletions(-)
(airflow-site) branch add-documentation-2024-08-25 updated (f94d8a0916 -> dbfad7341e)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch add-documentation-2024-08-25 in repository https://gitbox.apache.org/repos/asf/airflow-site.git from f94d8a0916 Add documentation for packages - 2024-08-25 add dbfad7341e move tabular provider index entry to removed section No new revisions were added by this update. Summary of changes: landing-pages/site/content/en/docs/_index.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
(airflow-site) branch add-documentation-2024-08-25 created (now f94d8a0916)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch add-documentation-2024-08-25 in repository https://gitbox.apache.org/repos/asf/airflow-site.git at f94d8a0916 Add documentation for packages - 2024-08-25 This branch includes the following new commits: new f94d8a0916 Add documentation for packages - 2024-08-25 The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
svn commit: r71091 - /dev/airflow/providers/
Author: eladkal Date: Sun Aug 25 07:41:17 2024 New Revision: 71091 Log: Add artifacts for Airflow Providers 2024-08-25 Added: dev/airflow/providers/apache_airflow_providers_celery-3.8.1-py3-none-any.whl (with props) dev/airflow/providers/apache_airflow_providers_celery-3.8.1-py3-none-any.whl.asc dev/airflow/providers/apache_airflow_providers_celery-3.8.1-py3-none-any.whl.sha512 dev/airflow/providers/apache_airflow_providers_celery-3.8.1.tar.gz (with props) dev/airflow/providers/apache_airflow_providers_celery-3.8.1.tar.gz.asc dev/airflow/providers/apache_airflow_providers_celery-3.8.1.tar.gz.sha512 dev/airflow/providers/apache_airflow_providers_cncf_kubernetes-8.4.1-py3-none-any.whl (with props) dev/airflow/providers/apache_airflow_providers_cncf_kubernetes-8.4.1-py3-none-any.whl.asc dev/airflow/providers/apache_airflow_providers_cncf_kubernetes-8.4.1-py3-none-any.whl.sha512 dev/airflow/providers/apache_airflow_providers_cncf_kubernetes-8.4.1.tar.gz (with props) dev/airflow/providers/apache_airflow_providers_cncf_kubernetes-8.4.1.tar.gz.asc dev/airflow/providers/apache_airflow_providers_cncf_kubernetes-8.4.1.tar.gz.sha512 dev/airflow/providers/apache_airflow_providers_openlineage-1.11.0-py3-none-any.whl (with props) dev/airflow/providers/apache_airflow_providers_openlineage-1.11.0-py3-none-any.whl.asc dev/airflow/providers/apache_airflow_providers_openlineage-1.11.0-py3-none-any.whl.sha512 dev/airflow/providers/apache_airflow_providers_openlineage-1.11.0.tar.gz (with props) dev/airflow/providers/apache_airflow_providers_openlineage-1.11.0.tar.gz.asc dev/airflow/providers/apache_airflow_providers_openlineage-1.11.0.tar.gz.sha512 dev/airflow/providers/apache_airflow_providers_ssh-3.13.1-py3-none-any.whl (with props) dev/airflow/providers/apache_airflow_providers_ssh-3.13.1-py3-none-any.whl.asc dev/airflow/providers/apache_airflow_providers_ssh-3.13.1-py3-none-any.whl.sha512 dev/airflow/providers/apache_airflow_providers_ssh-3.13.1.tar.gz (with props) dev/airflow/providers/apache_airflow_providers_ssh-3.13.1.tar.gz.asc dev/airflow/providers/apache_airflow_providers_ssh-3.13.1.tar.gz.sha512 dev/airflow/providers/apache_airflow_providers_tabular-1.6.1-py3-none-any.whl (with props) dev/airflow/providers/apache_airflow_providers_tabular-1.6.1-py3-none-any.whl.asc dev/airflow/providers/apache_airflow_providers_tabular-1.6.1-py3-none-any.whl.sha512 dev/airflow/providers/apache_airflow_providers_tabular-1.6.1.tar.gz (with props) dev/airflow/providers/apache_airflow_providers_tabular-1.6.1.tar.gz.asc dev/airflow/providers/apache_airflow_providers_tabular-1.6.1.tar.gz.sha512 Added: dev/airflow/providers/apache_airflow_providers_celery-3.8.1-py3-none-any.whl == Binary file - no diff available. Propchange: dev/airflow/providers/apache_airflow_providers_celery-3.8.1-py3-none-any.whl -- svn:mime-type = application/octet-stream Added: dev/airflow/providers/apache_airflow_providers_celery-3.8.1-py3-none-any.whl.asc == --- dev/airflow/providers/apache_airflow_providers_celery-3.8.1-py3-none-any.whl.asc (added) +++ dev/airflow/providers/apache_airflow_providers_celery-3.8.1-py3-none-any.whl.asc Sun Aug 25 07:41:17 2024 @@ -0,0 +1,7 @@ +-BEGIN PGP SIGNATURE- + +iIkEABYKADEWIQSDQO8ECQokO9vDRUWG4IhmPszevgUCZsrePBMcZWxhZGthbEBh +cGFjaGUub3JnAAoJEIbgiGY+zN6+w9UBANSB3C0deKUZbQH9fqD4UQ6tQcwG+6uo +CoS2pPkrGmvPAQDu5fhJuYIUhRSTVwDH1JytmiP8AWi6M55ui4Ovkm55DQ== +=pSBl +-END PGP SIGNATURE- Added: dev/airflow/providers/apache_airflow_providers_celery-3.8.1-py3-none-any.whl.sha512 == --- dev/airflow/providers/apache_airflow_providers_celery-3.8.1-py3-none-any.whl.sha512 (added) +++ dev/airflow/providers/apache_airflow_providers_celery-3.8.1-py3-none-any.whl.sha512 Sun Aug 25 07:41:17 2024 @@ -0,0 +1 @@ +f3e554292ea143cfc9ac79502140e04c6a9580dec6ee0a95a37a66b7b67ea59ab75ca03669de1cbdf8455133d6d2ad8bbf2c3eba409461adc29358330a8099fb apache_airflow_providers_celery-3.8.1-py3-none-any.whl Added: dev/airflow/providers/apache_airflow_providers_celery-3.8.1.tar.gz == Binary file - no diff available. Propchange: dev/airflow/providers/apache_airflow_providers_celery-3.8.1.tar.gz -- svn:mime-type = application/octet-stream Added: dev/airflow/providers/apache_airflow_providers_celery-3.8.1.tar.gz.asc == --- dev/airflow
(airflow) branch main updated: Prepare docs for Aug 3rd wave of providers (#41714)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 2f952909f5 Prepare docs for Aug 3rd wave of providers (#41714) 2f952909f5 is described below commit 2f952909f5028e416c951084727bd71ff8f22b72 Author: Elad Kalif <45845474+elad...@users.noreply.github.com> AuthorDate: Sun Aug 25 10:29:49 2024 +0300 Prepare docs for Aug 3rd wave of providers (#41714) --- airflow/providers/celery/CHANGELOG.rst | 12 +++ airflow/providers/celery/__init__.py | 2 +- airflow/providers/celery/provider.yaml | 3 +- airflow/providers/cncf/kubernetes/CHANGELOG.rst| 11 +++ airflow/providers/cncf/kubernetes/__init__.py | 2 +- airflow/providers/cncf/kubernetes/provider.yaml| 3 +- airflow/providers/openlineage/CHANGELOG.rst| 3 ++ airflow/providers/ssh/CHANGELOG.rst| 8 + airflow/providers/ssh/__init__.py | 2 +- airflow/providers/ssh/provider.yaml| 3 +- airflow/providers/tabular/CHANGELOG.rst| 15 + airflow/providers/tabular/provider.yaml| 3 +- docs/apache-airflow-providers-celery/commits.rst | 14 +++- docs/apache-airflow-providers-celery/index.rst | 6 ++-- .../commits.rst| 17 +- .../index.rst | 2 +- .../commits.rst| 5 ++- .../apache-airflow-providers-openlineage/index.rst | 19 ++- docs/apache-airflow-providers-ssh/commits.rst | 14 +++- docs/apache-airflow-providers-ssh/index.rst| 2 +- docs/apache-airflow-providers-tabular/commits.rst | 14 +++- docs/apache-airflow-providers-tabular/index.rst| 38 +- 22 files changed, 143 insertions(+), 55 deletions(-) diff --git a/airflow/providers/celery/CHANGELOG.rst b/airflow/providers/celery/CHANGELOG.rst index 48343927c1..2709f0bada 100644 --- a/airflow/providers/celery/CHANGELOG.rst +++ b/airflow/providers/celery/CHANGELOG.rst @@ -27,6 +27,18 @@ Changelog - +3.8.1 +. + +Bug Fixes +~ + +* ``fix: Missing 'slots_occupied' in 'CeleryKubernetesExecutor' and 'LocalKubernetesExecutor' (#41602)`` + + +.. Below changes are excluded from the changelog. Move them to + appropriate section above if needed. Do not delete the lines(!): + 3.8.0 . diff --git a/airflow/providers/celery/__init__.py b/airflow/providers/celery/__init__.py index 26be5e5fc4..322e6e3cce 100644 --- a/airflow/providers/celery/__init__.py +++ b/airflow/providers/celery/__init__.py @@ -29,7 +29,7 @@ from airflow import __version__ as airflow_version __all__ = ["__version__"] -__version__ = "3.8.0" +__version__ = "3.8.1" if packaging.version.parse(packaging.version.parse(airflow_version).base_version) < packaging.version.parse( "2.8.0" diff --git a/airflow/providers/celery/provider.yaml b/airflow/providers/celery/provider.yaml index ba78272ba6..ca60a25e13 100644 --- a/airflow/providers/celery/provider.yaml +++ b/airflow/providers/celery/provider.yaml @@ -22,9 +22,10 @@ description: | `Celery <https://docs.celeryq.dev/en/stable/>`__ state: ready -source-date-epoch: 1723969853 +source-date-epoch: 1724520600 # note that those versions are maintained by release manager - do not update them manually versions: + - 3.8.1 - 3.8.0 - 3.7.3 - 3.7.2 diff --git a/airflow/providers/cncf/kubernetes/CHANGELOG.rst b/airflow/providers/cncf/kubernetes/CHANGELOG.rst index 62972f7631..c26d3d76dd 100644 --- a/airflow/providers/cncf/kubernetes/CHANGELOG.rst +++ b/airflow/providers/cncf/kubernetes/CHANGELOG.rst @@ -27,6 +27,17 @@ Changelog - +8.4.1 +. + +Bug Fixes +~ + +* ``fix: 'KubernetesExecutor' failing the task in case the watcher receives an event with the reason ProviderFailed (#41186)`` +* ``fix: 'do_xcom_push' and 'get_logs' functionality for KubernetesJobOperator (#40814)`` +* ``fix: 'KubernetesHook' loading config file with '_is_in_cluster' set as False (#41464)`` +* ``fix: Missing 'slots_occupied' in 'CeleryKubernetesExecutor' and 'LocalKubernetesExecutor' (#41602)`` + 8.4.0 . diff --git a/airflow/providers/cncf/kubernetes/__init__.py b/airflow/providers/cncf/kubernetes/__init__.py index e4d9114b5a..289c5675cc 100644 --- a/airflow/providers/cncf/kubernetes/__init__.py +++ b/airflow/providers/cncf/kubernetes/__init__.py @@ -29,7 +29,7 @@ from airflow import __version__ as airflow_version __all__ = ["__version__"] -__version__ = "8.4.0" +__version__ = "8.4.1" if packaging.version.pars
(airflow) branch main updated: Fix pinecone PINECONE_DEBUG_CURL unit test (#41703)
This is an automated email from the ASF dual-hosted git repository.
eladkal pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new 56ca940567 Fix pinecone PINECONE_DEBUG_CURL unit test (#41703)
56ca940567 is described below
commit 56ca940567fd7aa9977e7047c495bb5bc9face12
Author: Niko Oliveira
AuthorDate: Fri Aug 23 20:43:27 2024 -0700
Fix pinecone PINECONE_DEBUG_CURL unit test (#41703)
Previously the unit test was mocking too early and the code which sets
the env variable was not even running. Remove mocking and create client
only to ensure the env variable is being set.
---
tests/providers/pinecone/hooks/test_pinecone.py | 6 ++
1 file changed, 2 insertions(+), 4 deletions(-)
diff --git a/tests/providers/pinecone/hooks/test_pinecone.py
b/tests/providers/pinecone/hooks/test_pinecone.py
index 82a01e5319..1bd80b7a27 100644
--- a/tests/providers/pinecone/hooks/test_pinecone.py
+++ b/tests/providers/pinecone/hooks/test_pinecone.py
@@ -52,11 +52,9 @@ class TestPineconeHook:
self.pinecone_hook.list_indexes()
mock_list_indexes.assert_called_once()
-
@patch("airflow.providers.pinecone.hooks.pinecone.PineconeHook.list_indexes")
-def test_debug_curl_setting(self, mock_list_indexes):
+def test_debug_curl_setting(self):
"""Test that the PINECONE_DEBUG_CURL environment variable is set when
initializing Pinecone Object."""
-self.pinecone_hook.list_indexes()
-mock_list_indexes.assert_called_once()
+self.pinecone_hook.pinecone_client
assert os.environ.get("PINECONE_DEBUG_CURL") == "true"
@patch("airflow.providers.pinecone.hooks.pinecone.PineconeHook.create_index")
(airflow) branch main updated: Mark tabular provider as removed (#41629)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 9674af5f88 Mark tabular provider as removed (#41629) 9674af5f88 is described below commit 9674af5f88929f5057158241330f4ef4fd08beb2 Author: Elad Kalif <45845474+elad...@users.noreply.github.com> AuthorDate: Thu Aug 22 18:01:58 2024 +0300 Mark tabular provider as removed (#41629) * Mark tabular provider as removed --- airflow/providers/tabular/provider.yaml| 2 +- dev/breeze/doc/images/output-commands.svg | 42 +++--- ...management_generate-issue-content-providers.svg | 12 +++ ...management_generate-issue-content-providers.txt | 2 +- ...e-management_prepare-provider-documentation.svg | 26 +++--- ...e-management_prepare-provider-documentation.txt | 2 +- ...elease-management_prepare-provider-packages.svg | 28 +++ ...elease-management_prepare-provider-packages.txt | 2 +- dev/breeze/tests/test_packages.py | 2 +- generated/provider_dependencies.json | 2 +- 10 files changed, 60 insertions(+), 60 deletions(-) diff --git a/airflow/providers/tabular/provider.yaml b/airflow/providers/tabular/provider.yaml index 2e0530313d..8ea1c559f0 100644 --- a/airflow/providers/tabular/provider.yaml +++ b/airflow/providers/tabular/provider.yaml @@ -21,7 +21,7 @@ name: Tabular description: | `Tabular <https://tabular.io/>`__ -state: ready +state: removed source-date-epoch: 1723970601 # note that those versions are maintained by release manager - do not update them manually versions: diff --git a/dev/breeze/doc/images/output-commands.svg b/dev/breeze/doc/images/output-commands.svg index 08d3dc2a13..5888d1fc86 100644 --- a/dev/breeze/doc/images/output-commands.svg +++ b/dev/breeze/doc/images/output-commands.svg @@ -298,53 +298,53 @@ Usage:breeze[OPTIONS â•â”€Â Execution mode │--python-p│--python-pPython major/minor version use [...] │(>3.8< | 3.9 | 3.10 | 3.11 | 3.12)  │[default: 3.8]  [...] -│--integrationIntegration(s) to enable [...] +│--integrationIntegration(s) to enable when running (can be more than one).   [...] │(all | all-testable | cassandra | celery | drill | kafka | kerberos | mongo | mssql │| openlineage | otel | pinot | qdrant | redis | statsd | trino | ydb)   │--standalone-dag-processor│--database-isolation│--standalone-dag-processorRun standalone dag processor for start-airflow.│--database-isolationRun airflow in database isolation mode.╰──╯ â•â”€Â Docker Compose selection and cleanup ──│--project-name│project name and you can use `breeze down -│--project-nameName of the docker-compose project to bring down. The `docker-compose` is for legacy& [...] +│project name and you can use `breeze down --project-name│belonging to it. [...] │(breeze | pre-commit | docker-compose)   [...] │[default: breeze] [...] -│--docker-host│--docker-hostOptional - docker host to use when running docker commands. When set, the `│ignored when building images.  [...] │(TEXT)& [...] ╰──╯ â•â”€Â Database ──│--backend-b│--backend-bDatabase backend to [...] │configuration, meaning there will be no database available, and any attempts to connect to  │the Airflow database will fail.&# [...] │(>sqlite< | mysql | postgres | none) [...] │[default: sqlite] & [...] -│--postgres-version│--mysql-version│--db-reset│--postgres-version-PVersion of Postgre [...] +│--mysql-version-MVersion of MySQL used.│--db-reset-dReset DB when enterin [...] ╰──╯ â•â”€Â Build CI image (before entering shell) < [...] -│--github-repository│--builderBuildx builder used to perf [...] +│--github-repository-gGitHub repository [...] +│--builderBuildx builder used to perfo
(airflow-site) branch add-documentation-2024-08-19 deleted (was 116692bcf3)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch add-documentation-2024-08-19 in repository https://gitbox.apache.org/repos/asf/airflow-site.git was 116692bcf3 exclude openlineage provider The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
svn commit: r71053 [1/2] - /dev/airflow/providers/ /release/airflow/providers/
Author: eladkal Date: Thu Aug 22 10:30:45 2024 New Revision: 71053 Log: Release Airflow Providers on 2024-08-22 Added: release/airflow/providers/apache_airflow_providers_airbyte-3.9.0-py3-none-any.whl - copied unchanged from r71052, dev/airflow/providers/apache_airflow_providers_airbyte-3.9.0-py3-none-any.whl release/airflow/providers/apache_airflow_providers_airbyte-3.9.0-py3-none-any.whl.asc - copied unchanged from r71052, dev/airflow/providers/apache_airflow_providers_airbyte-3.9.0-py3-none-any.whl.asc release/airflow/providers/apache_airflow_providers_airbyte-3.9.0-py3-none-any.whl.sha512 - copied unchanged from r71052, dev/airflow/providers/apache_airflow_providers_airbyte-3.9.0-py3-none-any.whl.sha512 release/airflow/providers/apache_airflow_providers_airbyte-3.9.0.tar.gz - copied unchanged from r71052, dev/airflow/providers/apache_airflow_providers_airbyte-3.9.0.tar.gz release/airflow/providers/apache_airflow_providers_airbyte-3.9.0.tar.gz.asc - copied unchanged from r71052, dev/airflow/providers/apache_airflow_providers_airbyte-3.9.0.tar.gz.asc release/airflow/providers/apache_airflow_providers_airbyte-3.9.0.tar.gz.sha512 - copied unchanged from r71052, dev/airflow/providers/apache_airflow_providers_airbyte-3.9.0.tar.gz.sha512 release/airflow/providers/apache_airflow_providers_alibaba-2.9.0-py3-none-any.whl - copied unchanged from r71052, dev/airflow/providers/apache_airflow_providers_alibaba-2.9.0-py3-none-any.whl release/airflow/providers/apache_airflow_providers_alibaba-2.9.0-py3-none-any.whl.asc - copied unchanged from r71052, dev/airflow/providers/apache_airflow_providers_alibaba-2.9.0-py3-none-any.whl.asc release/airflow/providers/apache_airflow_providers_alibaba-2.9.0-py3-none-any.whl.sha512 - copied unchanged from r71052, dev/airflow/providers/apache_airflow_providers_alibaba-2.9.0-py3-none-any.whl.sha512 release/airflow/providers/apache_airflow_providers_alibaba-2.9.0.tar.gz - copied unchanged from r71052, dev/airflow/providers/apache_airflow_providers_alibaba-2.9.0.tar.gz release/airflow/providers/apache_airflow_providers_alibaba-2.9.0.tar.gz.asc - copied unchanged from r71052, dev/airflow/providers/apache_airflow_providers_alibaba-2.9.0.tar.gz.asc release/airflow/providers/apache_airflow_providers_alibaba-2.9.0.tar.gz.sha512 - copied unchanged from r71052, dev/airflow/providers/apache_airflow_providers_alibaba-2.9.0.tar.gz.sha512 release/airflow/providers/apache_airflow_providers_amazon-8.28.0-py3-none-any.whl - copied unchanged from r71052, dev/airflow/providers/apache_airflow_providers_amazon-8.28.0-py3-none-any.whl release/airflow/providers/apache_airflow_providers_amazon-8.28.0-py3-none-any.whl.asc - copied unchanged from r71052, dev/airflow/providers/apache_airflow_providers_amazon-8.28.0-py3-none-any.whl.asc release/airflow/providers/apache_airflow_providers_amazon-8.28.0-py3-none-any.whl.sha512 - copied unchanged from r71052, dev/airflow/providers/apache_airflow_providers_amazon-8.28.0-py3-none-any.whl.sha512 release/airflow/providers/apache_airflow_providers_amazon-8.28.0.tar.gz - copied unchanged from r71052, dev/airflow/providers/apache_airflow_providers_amazon-8.28.0.tar.gz release/airflow/providers/apache_airflow_providers_amazon-8.28.0.tar.gz.asc - copied unchanged from r71052, dev/airflow/providers/apache_airflow_providers_amazon-8.28.0.tar.gz.asc release/airflow/providers/apache_airflow_providers_amazon-8.28.0.tar.gz.sha512 - copied unchanged from r71052, dev/airflow/providers/apache_airflow_providers_amazon-8.28.0.tar.gz.sha512 release/airflow/providers/apache_airflow_providers_apache_beam-5.8.0-py3-none-any.whl - copied unchanged from r71052, dev/airflow/providers/apache_airflow_providers_apache_beam-5.8.0-py3-none-any.whl release/airflow/providers/apache_airflow_providers_apache_beam-5.8.0-py3-none-any.whl.asc - copied unchanged from r71052, dev/airflow/providers/apache_airflow_providers_apache_beam-5.8.0-py3-none-any.whl.asc release/airflow/providers/apache_airflow_providers_apache_beam-5.8.0-py3-none-any.whl.sha512 - copied unchanged from r71052, dev/airflow/providers/apache_airflow_providers_apache_beam-5.8.0-py3-none-any.whl.sha512 release/airflow/providers/apache_airflow_providers_apache_beam-5.8.0.tar.gz - copied unchanged from r71052, dev/airflow/providers/apache_airflow_providers_apache_beam-5.8.0.tar.gz release/airflow/providers/apache_airflow_providers_apache_beam-5.8.0.tar.gz.asc - copied unchanged from r71052, dev/airflow/providers/apache_airflow_providers_apache_beam-5.8.0.tar.gz.asc release/airflow/providers/apache_airflow_providers_apache_beam-5.8.0.tar.gz.sha512 - copied unchanged from r71052, dev/airflow/providers/apache_airflow_providers_apache_beam-5.8.0.tar.gz.sha512
svn commit: r71053 [2/2] - /dev/airflow/providers/ /release/airflow/providers/
(airflow-site) branch chart deleted (was fcf72dbe44)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch chart in repository https://gitbox.apache.org/repos/asf/airflow-site.git was fcf72dbe44 Fix helm chart 1.15.0 release date The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
(airflow-site) branch chart created (now fcf72dbe44)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch chart in repository https://gitbox.apache.org/repos/asf/airflow-site.git at fcf72dbe44 Fix helm chart 1.15.0 release date This branch includes the following new commits: new fcf72dbe44 Fix helm chart 1.15.0 release date The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
(airflow-site) 01/01: Fix helm chart 1.15.0 release date
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a commit to branch chart in repository https://gitbox.apache.org/repos/asf/airflow-site.git commit fcf72dbe444a9eea133e1bf45bffc212fbeed6d5 Author: Elad Kalif <45845474+elad...@users.noreply.github.com> AuthorDate: Thu Aug 22 09:28:50 2024 +0300 Fix helm chart 1.15.0 release date --- docs-archive/helm-chart/1.15.0/release_notes.html | 114 +++--- 1 file changed, 57 insertions(+), 57 deletions(-) diff --git a/docs-archive/helm-chart/1.15.0/release_notes.html b/docs-archive/helm-chart/1.15.0/release_notes.html index a8756f2ca7..7524835263 100644 --- a/docs-archive/helm-chart/1.15.0/release_notes.html +++ b/docs-archive/helm-chart/1.15.0/release_notes.html @@ -42,9 +42,9 @@ - + - + @@ -83,38 +83,38 @@ - + Community - + Meetups - + Documentation - + Use Cases - + Announcements - + Blog - + Ecosystem - + - + @@ -158,38 +158,38 @@ - + Community - + Meetups - + Documentation - + Use Cases - + Announcements - + Blog - + Ecosystem - + - + @@ -200,8 +200,8 @@ - - + + @@ -350,7 +350,7 @@ Parameters Release Notes -Airflow Helm Chart 1.15.0 (2022-07-24) +Airflow Helm Chart 1.15.0 (2024-07-24) Significant Changes Default Airflow image is updated to 2.9.3 (#40816) Default PgBouncer Exporter image has been updated (#40318) @@ -644,10 +644,10 @@ - + - - + + @@ -716,7 +716,7 @@ Parameters Release Notes -Airflow Helm Chart 1.15.0 (2022-07-24) +Airflow Helm Chart 1.15.0 (2024-07-24) Significant Changes Default Airflow image is updated to 2.9.3 (#40816) Default PgBouncer Exporter image has been updated (#40318) @@ -1007,15 +1007,15 @@ - - + + - - + + @@ -1023,20 +1023,20 @@ - + Home - + Release Notes - + - + - + @@ -1046,7 +1046,7 @@ Apache Airflow Helm Chart Releases -Airflow Helm Chart 1.15.0 (2022-07-24) +Airflow Helm Chart 1.15.0 (2024-07-24) Airflow Helm Chart 1.14.0 (2024-06-18) Airflow Helm Chart 1.13.1 (2024-03-25) Airflow Helm Chart 1.13.0 (2024-03-05) @@ -1065,8 +1065,8 @@ Run helm repo update before upgrading the chart to the latest version. - -Airflow Helm Chart 1.15.0 (2022-07-24)¶ + +Airflow Helm Chart 1.15.0 (2024-07-24)¶ Significant Changes¶ @@ -2245,17 +2245,17 @@ and tolerations - + - - - + + + Release Notes -Airflow Helm Chart 1.15.0 (2022-07-24) +Airflow Helm Chart 1.15.0 (2024-07-24) Significant Changes Default Airflow image is updated to 2.9.3 (#40816) Default PgBouncer Exporter image has been updated (#40318) @@ -2479,12 +2479,10 @@ and toler
(airflow) branch main updated: fix: Missing 'slots_occupied' in `CeleryKubernetesExecutor` and `LocalKubernetesExecutor` (#41602)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 5cb582b51f fix: Missing 'slots_occupied' in `CeleryKubernetesExecutor` and `LocalKubernetesExecutor` (#41602) 5cb582b51f is described below commit 5cb582b51fb17a7724d22ed2eeca80b1a50610f3 Author: Niko Oliveira AuthorDate: Mon Aug 19 23:11:58 2024 -0700 fix: Missing 'slots_occupied' in `CeleryKubernetesExecutor` and `LocalKubernetesExecutor` (#41602) Each time the base executor changes, often those need to be copied into the old (soon to be deprecated) hybrid executors. This was missed for the new slots_occupied property. --- airflow/providers/celery/executors/celery_kubernetes_executor.py | 5 + .../providers/cncf/kubernetes/executors/local_kubernetes_executor.py | 5 + 2 files changed, 10 insertions(+) diff --git a/airflow/providers/celery/executors/celery_kubernetes_executor.py b/airflow/providers/celery/executors/celery_kubernetes_executor.py index e981e75fa1..bc2ed7904f 100644 --- a/airflow/providers/celery/executors/celery_kubernetes_executor.py +++ b/airflow/providers/celery/executors/celery_kubernetes_executor.py @@ -116,6 +116,11 @@ class CeleryKubernetesExecutor(LoggingMixin): """Number of new tasks this executor instance can accept.""" return self.celery_executor.slots_available +@property +def slots_occupied(self): +"""Number of tasks this executor instance is currently managing.""" +return len(self.running) + len(self.queued_tasks) + def queue_command( self, task_instance: TaskInstance, diff --git a/airflow/providers/cncf/kubernetes/executors/local_kubernetes_executor.py b/airflow/providers/cncf/kubernetes/executors/local_kubernetes_executor.py index 8c948b0d64..75de1101c5 100644 --- a/airflow/providers/cncf/kubernetes/executors/local_kubernetes_executor.py +++ b/airflow/providers/cncf/kubernetes/executors/local_kubernetes_executor.py @@ -103,6 +103,11 @@ class LocalKubernetesExecutor(LoggingMixin): """Number of new tasks this executor instance can accept.""" return self.local_executor.slots_available +@property +def slots_occupied(self): +"""Number of tasks this executor instance is currently managing.""" +return len(self.running) + len(self.queued_tasks) + def queue_command( self, task_instance: TaskInstance,
(airflow-site) branch add-documentation-2024-08-19 updated (94f038df93 -> 116692bcf3)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch add-documentation-2024-08-19 in repository https://gitbox.apache.org/repos/asf/airflow-site.git from 94f038df93 fixes add 116692bcf3 exclude openlineage provider No new revisions were added by this update. Summary of changes: .../1.11.0/.buildinfo | 4 - .../airflow/providers/openlineage/conf/index.html | 1057 -- .../openlineage/extractors/base/index.html | 1030 -- .../openlineage/extractors/bash/index.html | 868 -- .../providers/openlineage/extractors/index.html| 907 -- .../openlineage/extractors/manager/index.html | 967 -- .../openlineage/extractors/python/index.html | 868 -- .../providers/openlineage/facets/index.html| 866 -- .../_api/airflow/providers/openlineage/index.html | 918 -- .../openlineage/plugins/adapter/index.html | 1046 -- .../openlineage/plugins/facets/index.html | 1136 -- .../providers/openlineage/plugins/index.html | 881 -- .../openlineage/plugins/listener/index.html| 990 -- .../openlineage/plugins/macros/index.html | 960 -- .../openlineage/plugins/openlineage/index.html | 911 -- .../providers/openlineage/sqlparser/index.html | 1222 --- .../airflow/providers/openlineage/utils/index.html | 879 -- .../openlineage/utils/selective_enable/index.html | 999 -- .../providers/openlineage/utils/sql/index.html | 1085 -- .../providers/openlineage/utils/utils/index.html | 1403 --- .../_modules/airflow/providers/openlineage.html| 874 -- .../airflow/providers/openlineage/conf.html| 990 -- .../providers/openlineage/extractors/base.html | 994 -- .../providers/openlineage/extractors/manager.html | 1105 -- .../providers/openlineage/plugins/adapter.html | 1332 --- .../providers/openlineage/plugins/facets.html | 967 -- .../providers/openlineage/plugins/listener.html| 1319 --- .../providers/openlineage/plugins/macros.html | 924 -- .../providers/openlineage/plugins/openlineage.html | 881 -- .../airflow/providers/openlineage/sqlparser.html | 1233 --- .../openlineage/utils/selective_enable.html| 927 -- .../airflow/providers/openlineage/utils/sql.html | 1085 -- .../airflow/providers/openlineage/utils/utils.html | 1531 --- .../1.11.0/_modules/index.html | 846 -- .../providers/openlineage/conf/index.rst.txt | 109 - .../openlineage/extractors/base/index.rst.txt | 112 - .../openlineage/extractors/bash/index.rst.txt | 6 - .../providers/openlineage/extractors/index.rst.txt |36 - .../openlineage/extractors/manager/index.rst.txt |54 - .../openlineage/extractors/python/index.rst.txt| 6 - .../providers/openlineage/facets/index.rst.txt | 6 - .../airflow/providers/openlineage/index.rst.txt|36 - .../openlineage/plugins/adapter/index.rst.txt | 107 - .../openlineage/plugins/facets/index.rst.txt | 213 - .../providers/openlineage/plugins/index.rst.txt|19 - .../openlineage/plugins/listener/index.rst.txt |63 - .../openlineage/plugins/macros/index.rst.txt |64 - .../openlineage/plugins/openlineage/index.rst.txt |35 - .../providers/openlineage/sqlparser/index.rst.txt | 247 - .../providers/openlineage/utils/index.rst.txt |17 - .../utils/selective_enable/index.rst.txt |82 - .../providers/openlineage/utils/sql/index.rst.txt | 146 - .../openlineage/utils/utils/index.rst.txt | 317 - .../1.11.0/_sources/changelog.rst.txt |25 - .../1.11.0/_sources/commits.rst.txt| 385 - .../1.11.0/_sources/configurations-ref.rst.txt |21 - .../1.11.0/_sources/guides/developer.rst.txt | 619 -- .../1.11.0/_sources/guides/structure.rst.txt |67 - .../1.11.0/_sources/guides/user.rst.txt| 447 - .../1.11.0/_sources/index.rst.txt | 132 - .../installing-providers-from-sources.rst.txt |18 - .../1.11.0/_sources/macros.rst.txt |72 - .../1.11.0/_sources/security.rst.txt |18 - .../1.11.0/_sources/supported_classes.rst.txt |35 - ...-style.1e8bd061cd6da7fc9cf755528e8ffc24.min.css | 1 - .../1.11.0/_sphinx_design_static/design-tabs.js|27 - .../1.11.0/_static/_gen/css/main-custom.min.css| 1 - .../1.11.0/_static/_gen/css/main.min.css | 7 - .../1.11.0/_static/_gen/js/docs.js | 1 - .../_sphinx_javascript_frameworks_compat.js| 134 - .../1.11.0/_static/basic.css | 899 -- .../1.11.0/_static/check-solid.svg | 4 - .../1.11.0/_static/clipboard.min.js| 7 - .../1.11.0/_static/copy-button.svg | 5 - .../1.11.0/_static
(airflow-site) branch add-documentation-2024-08-19 updated (25db37b8e8 -> 94f038df93)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch add-documentation-2024-08-19 in repository https://gitbox.apache.org/repos/asf/airflow-site.git from 25db37b8e8 Add documentation for packages - 2024-08-19 add 94f038df93 fixes No new revisions were added by this update. Summary of changes: docs-archive/apache-airflow-providers-tableau/stable.txt | 2 +- docs-archive/apache-airflow-providers-tabular/stable.txt | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
(airflow-site) branch add-documentation-2024-08-19 created (now 25db37b8e8)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch add-documentation-2024-08-19 in repository https://gitbox.apache.org/repos/asf/airflow-site.git at 25db37b8e8 Add documentation for packages - 2024-08-19 This branch includes the following new commits: new 25db37b8e8 Add documentation for packages - 2024-08-19 The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
svn commit: r70984 [3/4] - /dev/airflow/providers/
Propchange: dev/airflow/providers/apache_airflow_providers_google-10.22.0.tar.gz -- svn:mime-type = application/octet-stream Added: dev/airflow/providers/apache_airflow_providers_google-10.22.0.tar.gz.asc == --- dev/airflow/providers/apache_airflow_providers_google-10.22.0.tar.gz.asc (added) +++ dev/airflow/providers/apache_airflow_providers_google-10.22.0.tar.gz.asc Mon Aug 19 06:22:21 2024 @@ -0,0 +1,7 @@ +-BEGIN PGP SIGNATURE- + +iIkEABYKADEWIQSDQO8ECQokO9vDRUWG4IhmPszevgUCZsLjkRMcZWxhZGthbEBh +cGFjaGUub3JnAAoJEIbgiGY+zN6+dLABAMlKI89lGdMuBBKzPCAYAKa3+7gkqcqL +yNOWsu8BEl7nAQDD0QcM/t4jTc1bScUs108NNr6HDMPo0JH5bVlyc+wODw== +=g6Yw +-END PGP SIGNATURE- Added: dev/airflow/providers/apache_airflow_providers_google-10.22.0.tar.gz.sha512 == --- dev/airflow/providers/apache_airflow_providers_google-10.22.0.tar.gz.sha512 (added) +++ dev/airflow/providers/apache_airflow_providers_google-10.22.0.tar.gz.sha512 Mon Aug 19 06:22:21 2024 @@ -0,0 +1 @@ +d0bf3185c1318bb9646d996f6132f9e2d07942cbab17ac14f1e5ea7139ba821512f72fb4f3917a4ef7bc6b472507286e08ea0c542ab8dede7ca54a16eaef5b71 apache_airflow_providers_google-10.22.0.tar.gz Added: dev/airflow/providers/apache_airflow_providers_grpc-3.6.0-py3-none-any.whl == Binary file - no diff available. Propchange: dev/airflow/providers/apache_airflow_providers_grpc-3.6.0-py3-none-any.whl -- svn:mime-type = application/octet-stream Added: dev/airflow/providers/apache_airflow_providers_grpc-3.6.0-py3-none-any.whl.asc == --- dev/airflow/providers/apache_airflow_providers_grpc-3.6.0-py3-none-any.whl.asc (added) +++ dev/airflow/providers/apache_airflow_providers_grpc-3.6.0-py3-none-any.whl.asc Mon Aug 19 06:22:21 2024 @@ -0,0 +1,7 @@ +-BEGIN PGP SIGNATURE- + +iIkEABYKADEWIQSDQO8ECQokO9vDRUWG4IhmPszevgUCZsLjkhMcZWxhZGthbEBh +cGFjaGUub3JnAAoJEIbgiGY+zN6+2a0BAKxK8Fw6PBgVeovH3epCwsbHpFWAhBfE +GzPvCbSmKrrWAPsEliEaI4xrX0PEOmzB/nw0RHqWz5gBQXzyXM12wDPTCw== +=t2GF +-END PGP SIGNATURE- Added: dev/airflow/providers/apache_airflow_providers_grpc-3.6.0-py3-none-any.whl.sha512 == --- dev/airflow/providers/apache_airflow_providers_grpc-3.6.0-py3-none-any.whl.sha512 (added) +++ dev/airflow/providers/apache_airflow_providers_grpc-3.6.0-py3-none-any.whl.sha512 Mon Aug 19 06:22:21 2024 @@ -0,0 +1 @@ +135483406e18a60ac1433128803ba752e56077133790a45fad0f4b2e244e516d43bc8924d136354ab438cbbe6bbe7a5d340b9540161e4c8bcf23a10733ec125f apache_airflow_providers_grpc-3.6.0-py3-none-any.whl Added: dev/airflow/providers/apache_airflow_providers_grpc-3.6.0.tar.gz == Binary file - no diff available. Propchange: dev/airflow/providers/apache_airflow_providers_grpc-3.6.0.tar.gz -- svn:mime-type = application/octet-stream Added: dev/airflow/providers/apache_airflow_providers_grpc-3.6.0.tar.gz.asc == --- dev/airflow/providers/apache_airflow_providers_grpc-3.6.0.tar.gz.asc (added) +++ dev/airflow/providers/apache_airflow_providers_grpc-3.6.0.tar.gz.asc Mon Aug 19 06:22:21 2024 @@ -0,0 +1,7 @@ +-BEGIN PGP SIGNATURE- + +iIkEABYKADEWIQSDQO8ECQokO9vDRUWG4IhmPszevgUCZsLjkhMcZWxhZGthbEBh +cGFjaGUub3JnAAoJEIbgiGY+zN6+FWABAI5wM2r5juyivcObfmKMEsjIOBipAssy +jNKM7w/wnt8fAQDC0gZwd+9P7BRriNZ9bMYoLQM0uAdU8zAEt/n+qfp6Bw== +=D8+J +-END PGP SIGNATURE- Added: dev/airflow/providers/apache_airflow_providers_grpc-3.6.0.tar.gz.sha512 == --- dev/airflow/providers/apache_airflow_providers_grpc-3.6.0.tar.gz.sha512 (added) +++ dev/airflow/providers/apache_airflow_providers_grpc-3.6.0.tar.gz.sha512 Mon Aug 19 06:22:21 2024 @@ -0,0 +1 @@ +6487ec46050dd6ed2d56dbbef4ab737f92ad3f4594b9e74d506aa7d75936a059c3e16ff3fdd3aabcd58ae80c7bbe5f2bc5395e7b4d64dd4819768b29daab35de apache_airflow_providers_grpc-3.6.0.tar.gz Added: dev/airflow/providers/apache_airflow_providers_hashicorp-3.8.0-py3-none-any.whl == Binary file - no diff available. Propchange: dev/airflow/providers/apache_airflow_providers_hashicorp-3.8.0-py3-none-any.whl -- svn:mime-type = application/octet-stream Added: dev/airflow/p
svn commit: r70984 [4/4] - /dev/airflow/providers/
Added: dev/airflow/providers/apache_airflow_providers_qdrant-1.2.0-py3-none-any.whl.sha512 == --- dev/airflow/providers/apache_airflow_providers_qdrant-1.2.0-py3-none-any.whl.sha512 (added) +++ dev/airflow/providers/apache_airflow_providers_qdrant-1.2.0-py3-none-any.whl.sha512 Mon Aug 19 06:22:21 2024 @@ -0,0 +1 @@ +ab41582c56320339bd224a7db05b12a2e1af11bb20aa33e74e998f09f737a1a47b6953382d4e93621d9aff7d3e7abdbf2f286040c958ab91a24177343207866f apache_airflow_providers_qdrant-1.2.0-py3-none-any.whl Added: dev/airflow/providers/apache_airflow_providers_qdrant-1.2.0.tar.gz == Binary file - no diff available. Propchange: dev/airflow/providers/apache_airflow_providers_qdrant-1.2.0.tar.gz -- svn:mime-type = application/octet-stream Added: dev/airflow/providers/apache_airflow_providers_qdrant-1.2.0.tar.gz.asc == --- dev/airflow/providers/apache_airflow_providers_qdrant-1.2.0.tar.gz.asc (added) +++ dev/airflow/providers/apache_airflow_providers_qdrant-1.2.0.tar.gz.asc Mon Aug 19 06:22:21 2024 @@ -0,0 +1,7 @@ +-BEGIN PGP SIGNATURE- + +iIkEABYKADEWIQSDQO8ECQokO9vDRUWG4IhmPszevgUCZsLjmxMcZWxhZGthbEBh +cGFjaGUub3JnAAoJEIbgiGY+zN6+bkwBAI8f99KrSYtpZ7yVXHW/lgm+wD4afWLU +jaX1V3dcXgRLAP4nrHiBnpyTFTtnh+mtLC0E1KiYg5LJO/TpeEttFZRACQ== +=22MP +-END PGP SIGNATURE- Added: dev/airflow/providers/apache_airflow_providers_qdrant-1.2.0.tar.gz.sha512 == --- dev/airflow/providers/apache_airflow_providers_qdrant-1.2.0.tar.gz.sha512 (added) +++ dev/airflow/providers/apache_airflow_providers_qdrant-1.2.0.tar.gz.sha512 Mon Aug 19 06:22:21 2024 @@ -0,0 +1 @@ +d18b6342f4ef405c5f14c072879f2599442c1dae2ee0fc0fee5be3af5d591ec1fe06642ee0dac11edb8fe6d138711749ce70e2cdc15cfcce82dec1e367dca99d apache_airflow_providers_qdrant-1.2.0.tar.gz Added: dev/airflow/providers/apache_airflow_providers_redis-3.8.0-py3-none-any.whl == Binary file - no diff available. Propchange: dev/airflow/providers/apache_airflow_providers_redis-3.8.0-py3-none-any.whl -- svn:mime-type = application/octet-stream Added: dev/airflow/providers/apache_airflow_providers_redis-3.8.0-py3-none-any.whl.asc == --- dev/airflow/providers/apache_airflow_providers_redis-3.8.0-py3-none-any.whl.asc (added) +++ dev/airflow/providers/apache_airflow_providers_redis-3.8.0-py3-none-any.whl.asc Mon Aug 19 06:22:21 2024 @@ -0,0 +1,7 @@ +-BEGIN PGP SIGNATURE- + +iIkEABYKADEWIQSDQO8ECQokO9vDRUWG4IhmPszevgUCZsLjmxMcZWxhZGthbEBh +cGFjaGUub3JnAAoJEIbgiGY+zN6+Cj4BAPprI6kmP80vKIoigPObms/e0o7HZDK5 +gkWZ7kL9zpxOAP9gli4Dy+35LWaescdL5flkXWur8Emwov9+ZvKSDnvrBA== +=u+10 +-END PGP SIGNATURE- Added: dev/airflow/providers/apache_airflow_providers_redis-3.8.0-py3-none-any.whl.sha512 == --- dev/airflow/providers/apache_airflow_providers_redis-3.8.0-py3-none-any.whl.sha512 (added) +++ dev/airflow/providers/apache_airflow_providers_redis-3.8.0-py3-none-any.whl.sha512 Mon Aug 19 06:22:21 2024 @@ -0,0 +1 @@ +cb9afda3176c661286bfb75dde1c085cf41d2732d05445d03cbfd836ac90046077757f8f7509383d3eb8e898173183a63939b4406bb0db6771f192daef257e01 apache_airflow_providers_redis-3.8.0-py3-none-any.whl Added: dev/airflow/providers/apache_airflow_providers_redis-3.8.0.tar.gz == Binary file - no diff available. Propchange: dev/airflow/providers/apache_airflow_providers_redis-3.8.0.tar.gz -- svn:mime-type = application/octet-stream Added: dev/airflow/providers/apache_airflow_providers_redis-3.8.0.tar.gz.asc == --- dev/airflow/providers/apache_airflow_providers_redis-3.8.0.tar.gz.asc (added) +++ dev/airflow/providers/apache_airflow_providers_redis-3.8.0.tar.gz.asc Mon Aug 19 06:22:21 2024 @@ -0,0 +1,7 @@ +-BEGIN PGP SIGNATURE- + +iIkEABYKADEWIQSDQO8ECQokO9vDRUWG4IhmPszevgUCZsLjmxMcZWxhZGthbEBh +cGFjaGUub3JnAAoJEIbgiGY+zN6+s30A/0313sT2I/6vBp9ZDLZ0raB0PO6e/JmJ +BPQvT53WA2rwAP9r9V6rjSTtUNxNQwG08HWFsMruBct/TCwiy6GSe4UwAg== +=lWhD +-END PGP SIGNATURE- Added: dev/airflow/providers/apache_airflow_providers_redis-3.8.0.tar.gz.sha512 == --- dev/airflow/providers/apache_airflow_prov
svn commit: r70984 [1/4] - /dev/airflow/providers/
Author: eladkal Date: Mon Aug 19 06:22:21 2024 New Revision: 70984 Log: Add artifacts for Airflow Providers 2024-08-19 Added: dev/airflow/providers/apache_airflow_providers_airbyte-3.9.0-py3-none-any.whl (with props) dev/airflow/providers/apache_airflow_providers_airbyte-3.9.0-py3-none-any.whl.asc dev/airflow/providers/apache_airflow_providers_airbyte-3.9.0-py3-none-any.whl.sha512 dev/airflow/providers/apache_airflow_providers_airbyte-3.9.0.tar.gz (with props) dev/airflow/providers/apache_airflow_providers_airbyte-3.9.0.tar.gz.asc dev/airflow/providers/apache_airflow_providers_airbyte-3.9.0.tar.gz.sha512 dev/airflow/providers/apache_airflow_providers_alibaba-2.9.0-py3-none-any.whl (with props) dev/airflow/providers/apache_airflow_providers_alibaba-2.9.0-py3-none-any.whl.asc dev/airflow/providers/apache_airflow_providers_alibaba-2.9.0-py3-none-any.whl.sha512 dev/airflow/providers/apache_airflow_providers_alibaba-2.9.0.tar.gz (with props) dev/airflow/providers/apache_airflow_providers_alibaba-2.9.0.tar.gz.asc dev/airflow/providers/apache_airflow_providers_alibaba-2.9.0.tar.gz.sha512 dev/airflow/providers/apache_airflow_providers_amazon-8.28.0-py3-none-any.whl (with props) dev/airflow/providers/apache_airflow_providers_amazon-8.28.0-py3-none-any.whl.asc dev/airflow/providers/apache_airflow_providers_amazon-8.28.0-py3-none-any.whl.sha512 dev/airflow/providers/apache_airflow_providers_amazon-8.28.0.tar.gz (with props) dev/airflow/providers/apache_airflow_providers_amazon-8.28.0.tar.gz.asc dev/airflow/providers/apache_airflow_providers_amazon-8.28.0.tar.gz.sha512 dev/airflow/providers/apache_airflow_providers_apache_beam-5.8.0-py3-none-any.whl (with props) dev/airflow/providers/apache_airflow_providers_apache_beam-5.8.0-py3-none-any.whl.asc dev/airflow/providers/apache_airflow_providers_apache_beam-5.8.0-py3-none-any.whl.sha512 dev/airflow/providers/apache_airflow_providers_apache_beam-5.8.0.tar.gz (with props) dev/airflow/providers/apache_airflow_providers_apache_beam-5.8.0.tar.gz.asc dev/airflow/providers/apache_airflow_providers_apache_beam-5.8.0.tar.gz.sha512 dev/airflow/providers/apache_airflow_providers_apache_cassandra-3.6.0-py3-none-any.whl (with props) dev/airflow/providers/apache_airflow_providers_apache_cassandra-3.6.0-py3-none-any.whl.asc dev/airflow/providers/apache_airflow_providers_apache_cassandra-3.6.0-py3-none-any.whl.sha512 dev/airflow/providers/apache_airflow_providers_apache_cassandra-3.6.0.tar.gz (with props) dev/airflow/providers/apache_airflow_providers_apache_cassandra-3.6.0.tar.gz.asc dev/airflow/providers/apache_airflow_providers_apache_cassandra-3.6.0.tar.gz.sha512 dev/airflow/providers/apache_airflow_providers_apache_drill-2.8.0-py3-none-any.whl (with props) dev/airflow/providers/apache_airflow_providers_apache_drill-2.8.0-py3-none-any.whl.asc dev/airflow/providers/apache_airflow_providers_apache_drill-2.8.0-py3-none-any.whl.sha512 dev/airflow/providers/apache_airflow_providers_apache_drill-2.8.0.tar.gz (with props) dev/airflow/providers/apache_airflow_providers_apache_drill-2.8.0.tar.gz.asc dev/airflow/providers/apache_airflow_providers_apache_drill-2.8.0.tar.gz.sha512 dev/airflow/providers/apache_airflow_providers_apache_druid-3.11.0-py3-none-any.whl (with props) dev/airflow/providers/apache_airflow_providers_apache_druid-3.11.0-py3-none-any.whl.asc dev/airflow/providers/apache_airflow_providers_apache_druid-3.11.0-py3-none-any.whl.sha512 dev/airflow/providers/apache_airflow_providers_apache_druid-3.11.0.tar.gz (with props) dev/airflow/providers/apache_airflow_providers_apache_druid-3.11.0.tar.gz.asc dev/airflow/providers/apache_airflow_providers_apache_druid-3.11.0.tar.gz.sha512 dev/airflow/providers/apache_airflow_providers_apache_flink-1.5.0-py3-none-any.whl (with props) dev/airflow/providers/apache_airflow_providers_apache_flink-1.5.0-py3-none-any.whl.asc dev/airflow/providers/apache_airflow_providers_apache_flink-1.5.0-py3-none-any.whl.sha512 dev/airflow/providers/apache_airflow_providers_apache_flink-1.5.0.tar.gz (with props) dev/airflow/providers/apache_airflow_providers_apache_flink-1.5.0.tar.gz.asc dev/airflow/providers/apache_airflow_providers_apache_flink-1.5.0.tar.gz.sha512 dev/airflow/providers/apache_airflow_providers_apache_hdfs-4.5.0-py3-none-any.whl (with props) dev/airflow/providers/apache_airflow_providers_apache_hdfs-4.5.0-py3-none-any.whl.asc dev/airflow/providers/apache_airflow_providers_apache_hdfs-4.5.0-py3-none-any.whl.sha512 dev/airflow/providers/apache_airflow_providers_apache_hdfs-4.5.0.tar.gz (with props) dev/airflow/providers/apache_airflow_providers_apache_hdfs-4.5.0.tar.gz.asc dev/airflow/providers/apache_airflow_providers_apache_hdfs-4.5.0
svn commit: r70984 [2/4] - /dev/airflow/providers/
Added: dev/airflow/providers/apache_airflow_providers_apache_kafka-1.6.0.tar.gz.sha512 == --- dev/airflow/providers/apache_airflow_providers_apache_kafka-1.6.0.tar.gz.sha512 (added) +++ dev/airflow/providers/apache_airflow_providers_apache_kafka-1.6.0.tar.gz.sha512 Mon Aug 19 06:22:21 2024 @@ -0,0 +1 @@ +2c88af6ad660f679b422cd828e046ff01c3cdb32417856c3b77ee9125f4fbd930b8007347f627afe4035c0513543c1e30b7d3884c16608d1600d6483b0c070b0 apache_airflow_providers_apache_kafka-1.6.0.tar.gz Added: dev/airflow/providers/apache_airflow_providers_apache_kylin-3.7.0-py3-none-any.whl == Binary file - no diff available. Propchange: dev/airflow/providers/apache_airflow_providers_apache_kylin-3.7.0-py3-none-any.whl -- svn:mime-type = application/octet-stream Added: dev/airflow/providers/apache_airflow_providers_apache_kylin-3.7.0-py3-none-any.whl.asc == --- dev/airflow/providers/apache_airflow_providers_apache_kylin-3.7.0-py3-none-any.whl.asc (added) +++ dev/airflow/providers/apache_airflow_providers_apache_kylin-3.7.0-py3-none-any.whl.asc Mon Aug 19 06:22:21 2024 @@ -0,0 +1,7 @@ +-BEGIN PGP SIGNATURE- + +iIkEABYKADEWIQSDQO8ECQokO9vDRUWG4IhmPszevgUCZsLjiBMcZWxhZGthbEBh +cGFjaGUub3JnAAoJEIbgiGY+zN6+aSIA/i0ATb1XeXyJuGJ7mzBF/UsoEHzw25S5 +g1Xon7CiKYrKAP9GXYLLu2acEHFbdW+tgJegyZsXH2gnOnP7OUVEbMwpAQ== +=V8mP +-END PGP SIGNATURE- Added: dev/airflow/providers/apache_airflow_providers_apache_kylin-3.7.0-py3-none-any.whl.sha512 == --- dev/airflow/providers/apache_airflow_providers_apache_kylin-3.7.0-py3-none-any.whl.sha512 (added) +++ dev/airflow/providers/apache_airflow_providers_apache_kylin-3.7.0-py3-none-any.whl.sha512 Mon Aug 19 06:22:21 2024 @@ -0,0 +1 @@ +0b5a59ca42850a4525ddca3f993c8a89824261b7e82b06c11052232919660c17308c2602458745145f5d97fce685666d45fb2b3c560ef69e4c920fda3c68513c apache_airflow_providers_apache_kylin-3.7.0-py3-none-any.whl Added: dev/airflow/providers/apache_airflow_providers_apache_kylin-3.7.0.tar.gz == Binary file - no diff available. Propchange: dev/airflow/providers/apache_airflow_providers_apache_kylin-3.7.0.tar.gz -- svn:mime-type = application/octet-stream Added: dev/airflow/providers/apache_airflow_providers_apache_kylin-3.7.0.tar.gz.asc == --- dev/airflow/providers/apache_airflow_providers_apache_kylin-3.7.0.tar.gz.asc (added) +++ dev/airflow/providers/apache_airflow_providers_apache_kylin-3.7.0.tar.gz.asc Mon Aug 19 06:22:21 2024 @@ -0,0 +1,7 @@ +-BEGIN PGP SIGNATURE- + +iIkEABYKADEWIQSDQO8ECQokO9vDRUWG4IhmPszevgUCZsLjiBMcZWxhZGthbEBh +cGFjaGUub3JnAAoJEIbgiGY+zN6+7zkBAOXumUh/313V7ce10EoM+5jp78FkYpH4 +hTF1J/3C+u7XAQDJ4k/iM5uZ4b6NBoKXXyE3E9XlFaHBMNZ17yaMwha7Cg== +=x+IL +-END PGP SIGNATURE- Added: dev/airflow/providers/apache_airflow_providers_apache_kylin-3.7.0.tar.gz.sha512 == --- dev/airflow/providers/apache_airflow_providers_apache_kylin-3.7.0.tar.gz.sha512 (added) +++ dev/airflow/providers/apache_airflow_providers_apache_kylin-3.7.0.tar.gz.sha512 Mon Aug 19 06:22:21 2024 @@ -0,0 +1 @@ +905a4d23c1a14df4565a88baeb3099a31f31372af0bb4ac3ac67ce7c0736746f47f826c1ec51dca36fe7a04383f72d8483f25df9b283292271bac2c058560c64 apache_airflow_providers_apache_kylin-3.7.0.tar.gz Added: dev/airflow/providers/apache_airflow_providers_apache_livy-3.9.0-py3-none-any.whl == Binary file - no diff available. Propchange: dev/airflow/providers/apache_airflow_providers_apache_livy-3.9.0-py3-none-any.whl -- svn:mime-type = application/octet-stream Added: dev/airflow/providers/apache_airflow_providers_apache_livy-3.9.0-py3-none-any.whl.asc == --- dev/airflow/providers/apache_airflow_providers_apache_livy-3.9.0-py3-none-any.whl.asc (added) +++ dev/airflow/providers/apache_airflow_providers_apache_livy-3.9.0-py3-none-any.whl.asc Mon Aug 19 06:22:21 2024 @@ -0,0 +1,7 @@ +-BEGIN PGP SIGNATURE- + +iIkEABYKADEWIQSDQO8ECQokO9vDRUWG4IhmPszevgUCZsLjiBMcZWxhZGthbEBh +cGFjaGUub3JnAAoJEIbgiGY+zN6+XTwA/jhS4s1yQxLeHrJy83x+wkNaUk5k8152 +LE5nEvyIMKywAQCwe18juVaZv2HqiN3RXj3C0RRjHE9PARQOkgsGL4mkAQ== +=Wv1R +-END PGP SIGNATURE- Added: dev
(airflow) branch main updated (7622192441 -> e92cf81805)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 7622192441 Fix `ElasticsearchSQLHook` fails with `AttributeError: __enter__` (#41537) add e92cf81805 Fix `AwsTaskLogFetcher` missing logs (#41515) No new revisions were added by this update. Summary of changes: airflow/providers/amazon/aws/utils/task_log_fetcher.py | 12 1 file changed, 12 insertions(+)
(airflow) branch main updated (8c2f56375a -> 7622192441)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 8c2f56375a Skip task in example_dynamodb_to_s3.py (#41546) add 7622192441 Fix `ElasticsearchSQLHook` fails with `AttributeError: __enter__` (#41537) No new revisions were added by this update. Summary of changes: airflow/providers/elasticsearch/hooks/elasticsearch.py| 8 tests/providers/elasticsearch/hooks/test_elasticsearch.py | 15 +++ .../elasticsearch/example_elasticsearch_query.py | 9 - 3 files changed, 27 insertions(+), 5 deletions(-)
(airflow) branch main updated (3b42286aa4 -> fcbff15bda)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 3b42286aa4 Fix missing source link for the mapped task with index 0 (#41403) add fcbff15bda Bump minimum Airflow version in providers to Airflow 2.8.0 (#41396) No new revisions were added by this update. Summary of changes: PROVIDERS.rst | 4 +- airflow/providers/airbyte/__init__.py | 4 +- airflow/providers/airbyte/provider.yaml | 2 +- airflow/providers/alibaba/__init__.py | 4 +- airflow/providers/alibaba/provider.yaml | 2 +- airflow/providers/amazon/__init__.py | 4 +- airflow/providers/amazon/provider.yaml| 2 +- airflow/providers/apache/beam/__init__.py | 4 +- airflow/providers/apache/beam/provider.yaml | 2 +- airflow/providers/apache/cassandra/__init__.py| 4 +- airflow/providers/apache/cassandra/provider.yaml | 2 +- airflow/providers/apache/drill/__init__.py| 4 +- airflow/providers/apache/drill/provider.yaml | 2 +- airflow/providers/apache/druid/__init__.py| 4 +- airflow/providers/apache/druid/provider.yaml | 2 +- airflow/providers/apache/flink/__init__.py| 4 +- airflow/providers/apache/flink/provider.yaml | 2 +- airflow/providers/apache/hdfs/__init__.py | 4 +- airflow/providers/apache/hdfs/provider.yaml | 2 +- airflow/providers/apache/hive/__init__.py | 4 +- airflow/providers/apache/hive/provider.yaml | 2 +- airflow/providers/apache/iceberg/__init__.py | 4 +- airflow/providers/apache/iceberg/provider.yaml| 2 +- airflow/providers/apache/impala/__init__.py | 4 +- airflow/providers/apache/impala/provider.yaml | 2 +- airflow/providers/apache/kafka/__init__.py| 4 +- airflow/providers/apache/kafka/provider.yaml | 2 +- airflow/providers/apache/kylin/__init__.py| 4 +- airflow/providers/apache/kylin/provider.yaml | 2 +- airflow/providers/apache/livy/__init__.py | 4 +- airflow/providers/apache/livy/provider.yaml | 2 +- airflow/providers/apache/pig/__init__.py | 4 +- airflow/providers/apache/pig/provider.yaml| 2 +- airflow/providers/apache/pinot/__init__.py| 4 +- airflow/providers/apache/pinot/provider.yaml | 2 +- airflow/providers/apache/spark/__init__.py| 4 +- airflow/providers/apache/spark/provider.yaml | 2 +- airflow/providers/apprise/__init__.py | 4 +- airflow/providers/apprise/provider.yaml | 2 +- airflow/providers/arangodb/__init__.py| 4 +- airflow/providers/arangodb/provider.yaml | 2 +- airflow/providers/asana/__init__.py | 4 +- airflow/providers/asana/provider.yaml | 2 +- airflow/providers/atlassian/jira/__init__.py | 4 +- airflow/providers/atlassian/jira/provider.yaml| 2 +- airflow/providers/celery/__init__.py | 4 +- airflow/providers/celery/provider.yaml| 2 +- airflow/providers/cloudant/__init__.py| 4 +- airflow/providers/cloudant/provider.yaml | 2 +- airflow/providers/cncf/kubernetes/__init__.py | 4 +- airflow/providers/cncf/kubernetes/provider.yaml | 2 +- airflow/providers/cohere/__init__.py | 4 +- airflow/providers/cohere/provider.yaml| 2 +- airflow/providers/common/compat/__init__.py | 4 +- airflow/providers/common/compat/provider.yaml | 2 +- airflow/providers/common/sql/__init__.py | 4 +- airflow/providers/common/sql/provider.yaml| 2 +- airflow/providers/databricks/__init__.py | 4 +- airflow/providers/databricks/provider.yaml| 2 +- airflow/providers/datadog/__init__.py | 4 +- airflow/providers/datadog/provider.yaml | 2 +- airflow/providers/dbt/cloud/__init__.py | 4 +- airflow/providers/dbt/cloud/provider.yaml | 2 +- airflow/providers/dingding/__init__.py| 4 +- airflow/providers/dingding/provider.yaml | 2 +- airflow/providers/discord/__init__.py | 4 +- airflow/providers/discord/provider.yaml | 2 +- airflow/providers/docker/__init__.py | 4 +- airflow/providers/docker/provider.yaml| 2 +- airflow/providers/elasticsearch/__init__.py | 4 +- airflow/providers/elasticsearch/provider.yaml | 2 +- airflow/providers/exasol/__init__.py | 4 +- airflow/providers/exasol/provider.yaml| 2 +- airflow/providers/facebook/__init__.py| 4 +- airflow/providers/facebook/provider.yaml | 2 +- airflow/providers/ftp/__init__.py | 4 +- airflow/providers/ftp
(airflow) branch main updated: Add comment on methodtools in mssql provider.yaml (#41402)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 4e62909ff5 Add comment on methodtools in mssql provider.yaml (#41402) 4e62909ff5 is described below commit 4e62909ff5fc92eb19676fb3266cd935cd42df55 Author: Elad Kalif <45845474+elad...@users.noreply.github.com> AuthorDate: Mon Aug 12 15:46:02 2024 +0300 Add comment on methodtools in mssql provider.yaml (#41402) --- airflow/providers/microsoft/mssql/provider.yaml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/airflow/providers/microsoft/mssql/provider.yaml b/airflow/providers/microsoft/mssql/provider.yaml index 8675eded52..f34c59a373 100644 --- a/airflow/providers/microsoft/mssql/provider.yaml +++ b/airflow/providers/microsoft/mssql/provider.yaml @@ -56,6 +56,8 @@ dependencies: - apache-airflow>=2.7.0 - apache-airflow-providers-common-sql>=1.14.1 - pymssql>=2.3.0 + # The methodtools dependency can be removed with min airflow version >=2.9.1 + # as it was added in https://github.com/apache/airflow/pull/37757 - methodtools>=0.4.7 integrations:
(airflow) branch main updated (6b9b1b225d -> b7fc211a69)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 6b9b1b225d Remove deprecations from `airflow.executors` & `airflow.utils` (#41395) add b7fc211a69 fix broken main on news fragment (#41404) No new revisions were added by this update. Summary of changes: newsfragments/41395.significant.rst | 1 - 1 file changed, 1 deletion(-)
(airflow) branch main updated: reorder docstring of `SparkKubernetesOperator` (#41372)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 2b1ae5f811 reorder docstring of `SparkKubernetesOperator` (#41372) 2b1ae5f811 is described below commit 2b1ae5f811457f95aefb589f9d85dd0eac58267f Author: Gopal Dirisala <39794726+dir...@users.noreply.github.com> AuthorDate: Mon Aug 12 10:55:32 2024 +0530 reorder docstring of `SparkKubernetesOperator` (#41372) * spark kubernetes operator arguments description reordering * spark kubernetes operator arguments description reordering --- .../cncf/kubernetes/operators/spark_kubernetes.py| 16 +--- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/airflow/providers/cncf/kubernetes/operators/spark_kubernetes.py b/airflow/providers/cncf/kubernetes/operators/spark_kubernetes.py index d9c3425f6e..82df0a2ec9 100644 --- a/airflow/providers/cncf/kubernetes/operators/spark_kubernetes.py +++ b/airflow/providers/cncf/kubernetes/operators/spark_kubernetes.py @@ -48,24 +48,26 @@ class SparkKubernetesOperator(KubernetesPodOperator): For more detail about Spark Application Object have a look at the reference: https://github.com/GoogleCloudPlatform/spark-on-k8s-operator/blob/v1beta2-1.3.3-3.1.1/docs/api-docs.md#sparkapplication -:param application_file: filepath to kubernetes custom_resource_definition of sparkApplication -:param kubernetes_conn_id: the connection to Kubernetes cluster :param image: Docker image you wish to launch. Defaults to hub.docker.com, :param code_path: path to the spark code in image, :param namespace: kubernetes namespace to put sparkApplication -:param cluster_context: context of the cluster -:param application_file: yaml file if passed +:param name: name of the pod in which the task will run, will be used (plus a random +suffix if random_name_suffix is True) to generate a pod id (DNS-1123 subdomain, +containing only [a-z0-9.-]). +:param application_file: filepath to kubernetes custom_resource_definition of sparkApplication +:param template_spec: kubernetes sparkApplication specification :param get_logs: get the stdout of the container as logs of the tasks. :param do_xcom_push: If True, the content of the file /airflow/xcom/return.json in the container will also be pushed to an XCom when the container completes. :param success_run_history_limit: Number of past successful runs of the application to keep. -:param delete_on_termination: What to do when the pod reaches its final -state, or the execution is interrupted. If True (default), delete the -pod; if False, leave the pod. :param startup_timeout_seconds: timeout in seconds to startup the pod. :param log_events_on_failure: Log the pod's events if a failure occurs :param reattach_on_restart: if the scheduler dies while the pod is running, reattach and monitor +:param delete_on_termination: What to do when the pod reaches its final +state, or the execution is interrupted. If True (default), delete the +pod; if False, leave the pod. +:param kubernetes_conn_id: the connection to Kubernetes cluster """ template_fields = ["application_file", "namespace", "template_spec"]
(airflow) branch main updated: Add `CloudRunServiceHook` and `CloudRunCreateServiceOperator` (#40008)
This is an automated email from the ASF dual-hosted git repository.
eladkal pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new 0aad0c8107 Add `CloudRunServiceHook` and
`CloudRunCreateServiceOperator` (#40008)
0aad0c8107 is described below
commit 0aad0c81078530a3512612d6424e1dcbd01f6752
Author: Jaejun <63435794+jx2...@users.noreply.github.com>
AuthorDate: Sat Aug 10 18:45:15 2024 +0900
Add `CloudRunServiceHook` and `CloudRunCreateServiceOperator` (#40008)
* init cloud_run service hook & operator
* init docs
* added example_cloud_run_service
* remove parent init
Co-authored-by: Shahar Epstein <60007259+shah...@users.noreply.github.com>
* apply reveiw - more clearer
* typo
* add ENV_ID in system tests
* validate inputs
* add deleteOperator & test
* typo
* modified second review
* 404/409 case in operator
* pass static checks
* typo
* modified service creation in hook & operator
* fixed by reviews in tests
* modified docs
* fixed test code to success
* modified hook
* add exceptions
* modified test codes
* static checks
-
Co-authored-by: Shahar Epstein <60007259+shah...@users.noreply.github.com>
---
airflow/providers/google/cloud/hooks/cloud_run.py | 124 ++
.../providers/google/cloud/operators/cloud_run.py | 145 -
.../operators/cloud/cloud_run.rst | 42 ++
.../providers/google/cloud/hooks/test_cloud_run.py | 139 +++-
.../google/cloud/operators/test_cloud_run.py | 133 ++-
.../cloud/cloud_run/example_cloud_run_service.py | 97 ++
6 files changed, 676 insertions(+), 4 deletions(-)
diff --git a/airflow/providers/google/cloud/hooks/cloud_run.py
b/airflow/providers/google/cloud/hooks/cloud_run.py
index 710357a60b..5eee37973a 100644
--- a/airflow/providers/google/cloud/hooks/cloud_run.py
+++ b/airflow/providers/google/cloud/hooks/cloud_run.py
@@ -22,13 +22,19 @@ from typing import TYPE_CHECKING, Any, Iterable, Sequence
from google.cloud.run_v2 import (
CreateJobRequest,
+CreateServiceRequest,
DeleteJobRequest,
+DeleteServiceRequest,
GetJobRequest,
+GetServiceRequest,
Job,
JobsAsyncClient,
JobsClient,
ListJobsRequest,
RunJobRequest,
+Service,
+ServicesAsyncClient,
+ServicesClient,
UpdateJobRequest,
)
from google.longrunning import operations_pb2 # type: ignore[attr-defined]
@@ -39,6 +45,7 @@ from airflow.providers.google.common.hooks.base_google import
PROVIDE_PROJECT_ID
if TYPE_CHECKING:
from google.api_core import operation
+from google.api_core.operation_async import AsyncOperation
from google.cloud.run_v2.services.jobs import pagers
@@ -183,3 +190,120 @@ class CloudRunAsyncHook(GoogleBaseHook):
return await self.get_conn().get_operation(
operations_pb2.GetOperationRequest(name=operation_name),
timeout=120
)
+
+
+class CloudRunServiceHook(GoogleBaseHook):
+"""
+Hook for the Google Cloud Run services.
+
+:param gcp_conn_id: The connection ID to use when fetching connection info.
+:param impersonation_chain: Optional service account to impersonate using
short-term
+credentials, or chained list of accounts required to get the
access_token
+of the last account in the list, which will be impersonated in the
request.
+If set as a string, the account must grant the originating account
+the Service Account Token Creator IAM role.
+If set as a sequence, the identities from the list must grant
+Service Account Token Creator IAM role to the directly preceding
identity, with first
+account from the list granting this role to the originating account.
+"""
+
+def __init__(
+self,
+gcp_conn_id: str = "google_cloud_default",
+impersonation_chain: str | Sequence[str] | None = None,
+):
+self._client: ServicesClient | None = None
+super().__init__(gcp_conn_id=gcp_conn_id,
impersonation_chain=impersonation_chain)
+
+def get_conn(self):
+if self._client is None:
+self._client = ServicesClient(credentials=self.get_credentials(),
client_info=CLIENT_INFO)
+
+return self._client
+
+@GoogleBaseHook.fallback_to_default_project_id
+def get_service(self, service_name: str, region: str, project_id: str =
PROVIDE_PROJECT_ID):
+get_service_request = GetServiceRequest(
+
name=f"projects/{project_id}/locations/{region}/services/{service_name}&
(airflow) branch main updated (7f5b8a57df -> 4d45c005b7)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 7f5b8a57df Resolve `AirflowProviderDeprecationWarning` in `SparkSqlOperator` (#41358) add 4d45c005b7 EKS Overrides for AWS Batch submit_job (#40718) No new revisions were added by this update. Summary of changes: airflow/providers/amazon/aws/hooks/batch_client.py | 3 + airflow/providers/amazon/aws/operators/batch.py| 8 ++ tests/providers/amazon/aws/operators/test_batch.py | 105 - 3 files changed, 115 insertions(+), 1 deletion(-)
(airflow) branch main updated: Resolve `AirflowProviderDeprecationWarning` in `SparkSqlOperator` (#41358)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 7f5b8a57df Resolve `AirflowProviderDeprecationWarning` in `SparkSqlOperator` (#41358) 7f5b8a57df is described below commit 7f5b8a57df2c6ec90da8dae0f3ef998a85eca2ca Author: phi-friday AuthorDate: Sat Aug 10 01:22:28 2024 +0900 Resolve `AirflowProviderDeprecationWarning` in `SparkSqlOperator` (#41358) --- airflow/providers/apache/spark/operators/spark_sql.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/airflow/providers/apache/spark/operators/spark_sql.py b/airflow/providers/apache/spark/operators/spark_sql.py index 7e48b748c6..ba0227f453 100644 --- a/airflow/providers/apache/spark/operators/spark_sql.py +++ b/airflow/providers/apache/spark/operators/spark_sql.py @@ -116,7 +116,7 @@ class SparkSqlOperator(BaseOperator): def _get_hook(self) -> SparkSqlHook: """Get SparkSqlHook.""" return SparkSqlHook( -sql=self._sql, +sql=self.sql, conf=self._conf, conn_id=self._conn_id, total_executor_cores=self._total_executor_cores,
(airflow) branch main updated (6deb4812be -> 71c61086d4)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 6deb4812be Add 2-10 branches to protected branches and codecoverage (#40940) add 71c61086d4 Recommend reserializing DAGs after downgrade (#41347) No new revisions were added by this update. Summary of changes: docs/apache-airflow/howto/usage-cli.rst | 4 1 file changed, 4 insertions(+)
(airflow) branch main updated (d310545798 -> 08589a76af)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from d310545798 docs: adding tip on configuration for healthcheck (#41289) add 08589a76af Fix improper script tag usage for dark theme toggle (#41337) No new revisions were added by this update. Summary of changes: airflow/www/templates/airflow/main.html | 2 +- airflow/www/webpack.config.js | 1 + 2 files changed, 2 insertions(+), 1 deletion(-)
(airflow) branch main updated: docs: adding tip on configuration for healthcheck (#41289)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new d310545798 docs: adding tip on configuration for healthcheck (#41289) d310545798 is described below commit d3105457983fdf54d7f6fec1ca0c81fa7050d9c3 Author: Everton Seiei Arakaki AuthorDate: Thu Aug 8 18:16:01 2024 +0200 docs: adding tip on configuration for healthcheck (#41289) * docs: adding tip on configuration for healthcheck * Format touchup * Fix trailing whitespaces - Co-authored-by: Tzu-ping Chung --- .../logging-monitoring/check-health.rst | 5 + 1 file changed, 5 insertions(+) diff --git a/docs/apache-airflow/administration-and-deployment/logging-monitoring/check-health.rst b/docs/apache-airflow/administration-and-deployment/logging-monitoring/check-health.rst index ead97bf94d..610d53758d 100644 --- a/docs/apache-airflow/administration-and-deployment/logging-monitoring/check-health.rst +++ b/docs/apache-airflow/administration-and-deployment/logging-monitoring/check-health.rst @@ -143,6 +143,11 @@ CLI Check for Celery Workers To verify that the Celery workers are working correctly, you can use the ``celery inspect ping`` command. On failure, the command will exit with a non-zero error code. +.. note:: + + For this check to work, ``[celery]worker_enable_remote_control`` must be ``True``. + If the parameter is set to ``False``, the command will exit with a non-zero error code. + To check if the worker running on the local host is working correctly, run: .. code-block:: bash
(airflow) branch main updated: add `env_vars` into `_BasePythonVirtualenvOperator` (#40942)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 463e28b9ce add `env_vars` into `_BasePythonVirtualenvOperator` (#40942) 463e28b9ce is described below commit 463e28b9ce6b58a64795c96ae7b94aba37770c93 Author: Youngha, Park <32288527+softyoun...@users.noreply.github.com> AuthorDate: Fri Aug 9 00:09:13 2024 +0900 add `env_vars` into `_BasePythonVirtualenvOperator` (#40942) * add env_vars into _BasePythonVirtualenvOperator * fix BranchPythonVirtualenvOperator * update feedback --- airflow/decorators/__init__.pyi | 16 +++ airflow/operators/python.py | 31 - airflow/utils/process_utils.py| 7 ++- tests/operators/test_python.py| 93 +++ tests/utils/test_process_utils.py | 4 ++ 5 files changed, 148 insertions(+), 3 deletions(-) diff --git a/airflow/decorators/__init__.pyi b/airflow/decorators/__init__.pyi index 9b4fbccc2e..faf77e8240 100644 --- a/airflow/decorators/__init__.pyi +++ b/airflow/decorators/__init__.pyi @@ -122,6 +122,8 @@ class TaskDecoratorCollection: index_urls: None | Collection[str] | str = None, venv_cache_path: None | str = None, show_return_value_in_logs: bool = True, +env_vars: dict[str, str] | None = None, +inherit_env: bool = True, use_dill: bool = False, **kwargs, ) -> TaskDecorator: @@ -165,6 +167,12 @@ class TaskDecoratorCollection: logs. Defaults to True, which allows return value log output. It can be set to False to prevent log output of return value when you return huge data such as transmission a large amount of XCom to TaskAPI. +:param env_vars: A dictionary containing additional environment variables to set for the virtual +environment when it is executed. +:param inherit_env: Whether to inherit the current environment variables when executing the virtual +environment. If set to ``True``, the virtual environment will inherit the environment variables +of the parent process (``os.environ``). If set to ``False``, the virtual environment will be +executed with a clean environment. :param use_dill: Deprecated, use ``serializer`` instead. Whether to use dill to serialize the args and result (pickle is default). This allows more complex types but requires you to include dill in your requirements. @@ -181,6 +189,8 @@ class TaskDecoratorCollection: serializer: Literal["pickle", "cloudpickle", "dill"] | None = None, templates_dict: Mapping[str, Any] | None = None, show_return_value_in_logs: bool = True, +env_vars: dict[str, str] | None = None, +inherit_env: bool = True, use_dill: bool = False, **kwargs, ) -> TaskDecorator: @@ -206,6 +216,12 @@ class TaskDecoratorCollection: logs. Defaults to True, which allows return value log output. It can be set to False to prevent log output of return value when you return huge data such as transmission a large amount of XCom to TaskAPI. +:param env_vars: A dictionary containing additional environment variables to set for the virtual +environment when it is executed. +:param inherit_env: Whether to inherit the current environment variables when executing the virtual +environment. If set to ``True``, the virtual environment will inherit the environment variables +of the parent process (``os.environ``). If set to ``False``, the virtual environment will be +executed with a clean environment. :param use_dill: Deprecated, use ``serializer`` instead. Whether to use dill to serialize the args and result (pickle is default). This allows more complex types but requires you to include dill in your requirements. diff --git a/airflow/operators/python.py b/airflow/operators/python.py index 46dbff9f01..fdfe575fb9 100644 --- a/airflow/operators/python.py +++ b/airflow/operators/python.py @@ -439,6 +439,8 @@ class _BasePythonVirtualenvOperator(PythonOperator, metaclass=ABCMeta): templates_exts: list[str] | None = None, expect_airflow: bool = True, skip_on_exit_code: int | Container[int] | None = None, +env_vars: dict[str, str] | None = None, +inherit_env: bool = True, use_dill: bool = False, **kwargs, ): @@ -490,6 +492,8 @@ class _BasePythonVirtualenvOperator(PythonOperator, metaclass=ABCMeta): if skip_on_exit_code is not None else [] ) +self.env_vars = env_vars +self.inherit_env = i
(airflow) branch main updated (472e683732 -> 45658a8963)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 472e683732 feat(dataset_event): add showSource and showTriggeredDagRuns to DatasetEventCard (#41300) add 45658a8963 Clarify that the TI try selector only appears when you have multiple tries (#41319) No new revisions were added by this update. Summary of changes: docs/apache-airflow/core-concepts/dag-run.rst | 6 +- 1 file changed, 5 insertions(+), 1 deletion(-)
(airflow) branch main updated: Fix RedshiftDataOperator not running in deferred mode when it should (#41206)
This is an automated email from the ASF dual-hosted git repository.
eladkal pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new 454b5bbf52 Fix RedshiftDataOperator not running in deferred mode when
it should (#41206)
454b5bbf52 is described below
commit 454b5bbf529ea2a9b0b69a871803ff8920af0bb5
Author: Boris Morel <2323800+bori...@users.noreply.github.com>
AuthorDate: Wed Aug 7 19:35:03 2024 +0800
Fix RedshiftDataOperator not running in deferred mode when it should
(#41206)
* Ensure operator goes into deferrable mode
* Remove commented out code
* Test when not waiting for completion
* Add entry to changelog
* Rephrase warning
---
airflow/providers/amazon/CHANGELOG.rst | 14
.../amazon/aws/operators/redshift_data.py | 6 ++--
.../amazon/aws/operators/test_redshift_data.py | 38 --
3 files changed, 53 insertions(+), 5 deletions(-)
diff --git a/airflow/providers/amazon/CHANGELOG.rst
b/airflow/providers/amazon/CHANGELOG.rst
index ed3b50f1ab..1dfc37909c 100644
--- a/airflow/providers/amazon/CHANGELOG.rst
+++ b/airflow/providers/amazon/CHANGELOG.rst
@@ -26,6 +26,20 @@
Changelog
-
+Main
+..
+
+.. warning:: When deferrable mode was introduced for ``RedshiftDataOperator``,
in version 8.17.0, tasks configured with
+ ``deferrable=True`` and ``wait_for_completion=True`` wouldn't enter the
deferred state. Instead, the task would occupy
+ an executor slot until the statement was completed. A workaround may have
been to set ``wait_for_completion=False``.
+ In this version, tasks set up with ``wait_for_completion=False`` will not
wait anymore, regardless of the value of
+ ``deferrable``.
+
+Bug Fixes
+~
+
+* ``Fix deferred mode for 'RedshiftDataOperator' (#41206)``
+
8.27.0
..
diff --git a/airflow/providers/amazon/aws/operators/redshift_data.py
b/airflow/providers/amazon/aws/operators/redshift_data.py
index 54e3c2c7ae..45fee2a919 100644
--- a/airflow/providers/amazon/aws/operators/redshift_data.py
+++ b/airflow/providers/amazon/aws/operators/redshift_data.py
@@ -127,8 +127,8 @@ class
RedshiftDataOperator(AwsBaseOperator[RedshiftDataHook]):
# Set wait_for_completion to False so that it waits for the status in
the deferred task.
wait_for_completion = self.wait_for_completion
-if self.deferrable and self.wait_for_completion:
-self.wait_for_completion = False
+if self.deferrable:
+wait_for_completion = False
self.statement_id = self.hook.execute_query(
database=self.database,
@@ -144,7 +144,7 @@ class
RedshiftDataOperator(AwsBaseOperator[RedshiftDataHook]):
poll_interval=self.poll_interval,
)
-if self.deferrable:
+if self.deferrable and self.wait_for_completion:
is_finished = self.hook.check_query_is_finished(self.statement_id)
if not is_finished:
self.defer(
diff --git a/tests/providers/amazon/aws/operators/test_redshift_data.py
b/tests/providers/amazon/aws/operators/test_redshift_data.py
index a02515441b..fa021395a4 100644
--- a/tests/providers/amazon/aws/operators/test_redshift_data.py
+++ b/tests/providers/amazon/aws/operators/test_redshift_data.py
@@ -51,7 +51,7 @@ def deferrable_operator():
secret_arn=secret_arn,
statement_name=statement_name,
parameters=parameters,
-wait_for_completion=False,
+wait_for_completion=True,
poll_interval=poll_interval,
deferrable=True,
)
@@ -276,7 +276,6 @@ class TestRedshiftDataOperator:
poll_interval=poll_interval,
)
-#
@mock.patch("airflow.providers.amazon.aws.operators.redshift_data.RedshiftDataOperator.defer")
@mock.patch(
"airflow.providers.amazon.aws.hooks.redshift_data.RedshiftDataHook.check_query_is_finished",
return_value=False,
@@ -315,3 +314,38 @@ class TestRedshiftDataOperator:
== "uuid"
)
mock_log_info.assert_called_with("%s completed successfully.", TASK_ID)
+
+
@mock.patch("airflow.providers.amazon.aws.operators.redshift_data.RedshiftDataOperator.defer")
+
@mock.patch("airflow.providers.amazon.aws.hooks.redshift_data.RedshiftDataHook.check_query_is_finished")
+
@mock.patch("airflow.providers.amazon.aws.hooks.redshift_data.RedshiftDataHook.execute_query")
+def test_no_wait_for_completion(self, mock_exec_query,
mock_check_query_is_finished, mock_defer):
+"""Tests that the operator does not check for completion nor defers
when wait_for_completion is False,
+no matter the value of deferrable"""
+cluster_identifier = "
(airflow) branch main updated: Fix almost 100 tests for taskinstance for DB isolation mode (#41296)
This is an automated email from the ASF dual-hosted git repository.
eladkal pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new eda4329d65 Fix almost 100 tests for taskinstance for DB isolation mode
(#41296)
eda4329d65 is described below
commit eda4329d65a466786522488f159782803f9eeb29
Author: Jarek Potiuk
AuthorDate: Wed Aug 7 08:43:43 2024 +0200
Fix almost 100 tests for taskinstance for DB isolation mode (#41296)
Related: #41067
---
tests/models/test_taskinstance.py | 22 +++---
1 file changed, 15 insertions(+), 7 deletions(-)
diff --git a/tests/models/test_taskinstance.py
b/tests/models/test_taskinstance.py
index 32158dc00d..7be4eda365 100644
--- a/tests/models/test_taskinstance.py
+++ b/tests/models/test_taskinstance.py
@@ -82,7 +82,7 @@ from airflow.operators.python import PythonOperator
from airflow.sensors.base import BaseSensorOperator
from airflow.sensors.python import PythonSensor
from airflow.serialization.serialized_objects import SerializedBaseOperator,
SerializedDAG
-from airflow.settings import TIMEZONE
+from airflow.settings import TIMEZONE, TracebackSessionForTests
from airflow.stats import Stats
from airflow.ti_deps.dep_context import DepContext
from airflow.ti_deps.dependencies_deps import REQUEUEABLE_DEPS, RUNNING_DEPS
@@ -104,7 +104,7 @@ from tests.test_utils.config import conf_vars
from tests.test_utils.db import clear_db_connections, clear_db_runs
from tests.test_utils.mock_operators import MockOperator
-pytestmark = pytest.mark.db_test
+pytestmark = [pytest.mark.db_test]
@pytest.fixture
@@ -288,7 +288,7 @@ class TestTaskInstance:
assert not ti.test_mode
@patch.object(DAG, "get_concurrency_reached")
-def test_requeue_over_dag_concurrency(self, mock_concurrency_reached,
create_task_instance):
+def test_requeue_over_dag_concurrency(self, mock_concurrency_reached,
create_task_instance, dag_maker):
mock_concurrency_reached.return_value = True
ti = create_task_instance(
@@ -1403,7 +1403,9 @@ class TestTaskInstance:
assert task.start_date is not None
run_date = task.start_date + datetime.timedelta(days=5)
-ti =
dag_maker.create_dagrun(execution_date=run_date).get_task_instance(downstream.task_id)
+dr = dag_maker.create_dagrun(execution_date=run_date)
+dag_maker.session.commit()
+ti = dr.get_task_instance(downstream.task_id)
ti.task = downstream
dep_results = TriggerRuleDep()._evaluate_trigger_rule(
@@ -1413,6 +1415,8 @@ class TestTaskInstance:
)
completed = all(dep.passed for dep in dep_results)
+ti = dr.get_task_instance(downstream.task_id)
+
assert completed == expect_passed
assert ti.state == expect_state
@@ -1511,15 +1515,16 @@ class TestTaskInstance:
do_something_else.expand(i=nums)
dr = dag_maker.create_dagrun()
-
+dag_maker.session.commit()
monkeypatch.setattr(_UpstreamTIStates, "calculate", lambda *_:
upstream_states)
ti = dr.get_task_instance("do_something_else", session=session)
ti.map_index = 0
for map_index in range(1, 5):
-ti = TaskInstance(ti.task, run_id=dr.run_id, map_index=map_index)
+ti = TaskInstance(dr.task_instances[-1].task, run_id=dr.run_id,
map_index=map_index)
session.add(ti)
ti.dag_run = dr
session.flush()
+session.commit()
downstream = ti.task
ti = dr.get_task_instance(task_id="do_something_else", map_index=3,
session=session)
ti.task = downstream
@@ -1528,7 +1533,10 @@ class TestTaskInstance:
dep_context=DepContext(flag_upstream_failed=flag_upstream_failed),
session=session,
)
+TracebackSessionForTests.set_allow_db_access(session, True)
completed = all(dep.passed for dep in dep_results)
+TracebackSessionForTests.set_allow_db_access(session, False)
+ti = dr.get_task_instance(task_id="do_something_else", map_index=3,
session=session)
assert completed == expect_completed
assert ti.state == expect_state
@@ -2946,7 +2954,7 @@ class TestTaskInstance:
dag_maker,
) -> list:
dag_id = "test_previous_dates"
-with dag_maker(dag_id=dag_id, schedule=schedule_interval,
catchup=catchup):
+with dag_maker(dag_id=dag_id, schedule=schedule_interval,
catchup=catchup, serialized=True):
task = EmptyOperator(task_id="task")
def get_test_ti(execution_date: pendulum.DateTime, state: str) -> TI:
(airflow) branch main updated: Move Create Dataset Event button above tabs (#41190)
This is an automated email from the ASF dual-hosted git repository.
eladkal pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new 55872ab51c Move Create Dataset Event button above tabs (#41190)
55872ab51c is described below
commit 55872ab51c533c922d0a78f1d62e376b780cb258
Author: Brent Bovenzi
AuthorDate: Wed Aug 7 00:21:54 2024 -0400
Move Create Dataset Event button above tabs (#41190)
---
.../www/static/js/datasets/CreateDatasetEvent.tsx | 16 +--
airflow/www/static/js/datasets/DatasetDetails.tsx | 31 --
airflow/www/static/js/datasets/Main.tsx| 108 ++---
3 files changed, 82 insertions(+), 73 deletions(-)
diff --git a/airflow/www/static/js/datasets/CreateDatasetEvent.tsx
b/airflow/www/static/js/datasets/CreateDatasetEvent.tsx
index 6a1249e32c..39de114371 100644
--- a/airflow/www/static/js/datasets/CreateDatasetEvent.tsx
+++ b/airflow/www/static/js/datasets/CreateDatasetEvent.tsx
@@ -34,13 +34,12 @@ import {
} from "@chakra-ui/react";
import { useContainerRef } from "src/context/containerRef";
-import { useCreateDatasetEvent } from "src/api";
-import type { Dataset } from "src/types/api-generated";
+import { useCreateDatasetEvent, useDataset } from "src/api";
interface Props {
isOpen: boolean;
onClose: () => void;
- dataset: Dataset;
+ uri: string;
}
function checkJsonString(str: string) {
@@ -52,16 +51,17 @@ function checkJsonString(str: string) {
return true;
}
-const CreateDatasetEventModal = ({ dataset, isOpen, onClose }: Props) => {
+const CreateDatasetEventModal = ({ uri, isOpen, onClose }: Props) => {
const containerRef = useContainerRef();
const [extra, setExtra] = useState("");
+ const { data: dataset } = useDataset({ uri });
const isJson = checkJsonString(extra);
const isDisabled = !!extra && !isJson;
const { mutate: createDatasetEvent, isLoading } = useCreateDatasetEvent({
-datasetId: dataset.id,
-uri: dataset.uri,
+datasetId: dataset?.id,
+uri: dataset?.uri,
});
const onSubmit = () => {
@@ -69,6 +69,8 @@ const CreateDatasetEventModal = ({ dataset, isOpen, onClose
}: Props) => {
onClose();
};
+ if (!dataset) return null;
+
return (
{
>
-Manually create event for {dataset.uri}
+Manually create event for {dataset?.uri}
diff --git a/airflow/www/static/js/datasets/DatasetDetails.tsx
b/airflow/www/static/js/datasets/DatasetDetails.tsx
index a4a4797a06..715c2bf308 100644
--- a/airflow/www/static/js/datasets/DatasetDetails.tsx
+++ b/airflow/www/static/js/datasets/DatasetDetails.tsx
@@ -21,24 +21,18 @@ import React from "react";
import {
Spinner,
Flex,
- IconButton,
- useDisclosure,
Grid,
GridItem,
Heading,
Link,
Box,
} from "@chakra-ui/react";
-import { MdPlayArrow } from "react-icons/md";
import { isEmpty } from "lodash";
import { useDataset } from "src/api";
-import { useContainerRef } from "src/context/containerRef";
-import Tooltip from "src/components/Tooltip";
import { getMetaValue } from "src/utils";
import RenderedJsonField from "src/components/RenderedJsonField";
-import CreateDatasetEventModal from "./CreateDatasetEvent";
import Events from "./DatasetEvents";
const gridUrl = getMetaValue("grid_url");
@@ -49,8 +43,6 @@ interface Props {
const DatasetDetails = ({ uri }: Props) => {
const { data: dataset, isLoading } = useDataset({ uri });
- const { isOpen, onToggle, onClose } = useDisclosure();
- const containerRef = useContainerRef();
const hasProducingTasks = !!dataset?.producingTasks?.length;
const hasConsumingDags = !!dataset?.consumingDags?.length;
@@ -100,22 +92,6 @@ const DatasetDetails = ({ uri }: Props) => {
})}
)}
-
-
-
-
-
-
-
{dataset?.extra && !isEmpty(dataset?.extra) && (
{
{dataset && dataset.id && }
- {dataset && (
-
- )}
);
};
diff --git a/airflow/www/static/js/datasets/Main.tsx
b/airflow/www/static/js/datasets/Main.tsx
index 04380e2665..3d9b5e51c3 100644
--- a/airflow/www/static/js/datasets/Main.tsx
+++ b/airflow/www/static/js/datasets/Main.tsx
@@ -32,15 +32,20 @@ import {
TabPanel,
TabPanels,
Text,
+ Flex,
+ useDisclosure,
+ IconButton,
} from "@chakra-ui/react";
import { HiDatabase } from "react-icons/hi";
-import { MdEvent, MdAccountTree, MdDetails } from "react-icons/md";
+import { MdEvent, MdAccountTree, MdDetails,
(airflow) branch main updated: Fix PlainAsserts tests in Database Isolation Mode (#41299)
This is an automated email from the ASF dual-hosted git repository.
eladkal pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new 00991d5880 Fix PlainAsserts tests in Database Isolation Mode (#41299)
00991d5880 is described below
commit 00991d5880f943b75b8875a22fb724d698c6a2b5
Author: Jens Scheffler <95105677+jsche...@users.noreply.github.com>
AuthorDate: Wed Aug 7 04:19:27 2024 +0200
Fix PlainAsserts tests in Database Isolation Mode (#41299)
---
airflow/utils/types.py | 2 ++
1 file changed, 2 insertions(+)
diff --git a/airflow/utils/types.py b/airflow/utils/types.py
index abe4003232..c3a6b7e5a9 100644
--- a/airflow/utils/types.py
+++ b/airflow/utils/types.py
@@ -57,6 +57,8 @@ class AttributeRemoved:
self.attribute_name = attribute_name
def __getattr__(self, item):
+if item == "attribute_name":
+return super().__getattribute__(item)
raise RuntimeError(
f"Attribute {self.attribute_name} was removed on "
f"serialization and must be set again - found when accessing
{item}."
(airflow-site) branch add-documentation-2024-08-03 deleted (was 38980fdec4)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch add-documentation-2024-08-03 in repository https://gitbox.apache.org/repos/asf/airflow-site.git was 38980fdec4 Add documentation for packages - 2024-08-03 The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
svn commit: r70731 - /dev/airflow/providers/ /release/airflow/providers/
Author: eladkal Date: Tue Aug 6 20:28:43 2024 New Revision: 70731 Log: Release Airflow Providers on 2024-08-06 Added: release/airflow/providers/apache_airflow_providers_amazon-8.27.0-py3-none-any.whl - copied unchanged from r70730, dev/airflow/providers/apache_airflow_providers_amazon-8.27.0-py3-none-any.whl release/airflow/providers/apache_airflow_providers_amazon-8.27.0-py3-none-any.whl.asc - copied unchanged from r70730, dev/airflow/providers/apache_airflow_providers_amazon-8.27.0-py3-none-any.whl.asc release/airflow/providers/apache_airflow_providers_amazon-8.27.0-py3-none-any.whl.sha512 - copied unchanged from r70730, dev/airflow/providers/apache_airflow_providers_amazon-8.27.0-py3-none-any.whl.sha512 release/airflow/providers/apache_airflow_providers_amazon-8.27.0.tar.gz - copied unchanged from r70730, dev/airflow/providers/apache_airflow_providers_amazon-8.27.0.tar.gz release/airflow/providers/apache_airflow_providers_amazon-8.27.0.tar.gz.asc - copied unchanged from r70730, dev/airflow/providers/apache_airflow_providers_amazon-8.27.0.tar.gz.asc release/airflow/providers/apache_airflow_providers_amazon-8.27.0.tar.gz.sha512 - copied unchanged from r70730, dev/airflow/providers/apache_airflow_providers_amazon-8.27.0.tar.gz.sha512 release/airflow/providers/apache_airflow_providers_apache_beam-5.7.2-py3-none-any.whl - copied unchanged from r70730, dev/airflow/providers/apache_airflow_providers_apache_beam-5.7.2-py3-none-any.whl release/airflow/providers/apache_airflow_providers_apache_beam-5.7.2-py3-none-any.whl.asc - copied unchanged from r70730, dev/airflow/providers/apache_airflow_providers_apache_beam-5.7.2-py3-none-any.whl.asc release/airflow/providers/apache_airflow_providers_apache_beam-5.7.2-py3-none-any.whl.sha512 - copied unchanged from r70730, dev/airflow/providers/apache_airflow_providers_apache_beam-5.7.2-py3-none-any.whl.sha512 release/airflow/providers/apache_airflow_providers_apache_beam-5.7.2.tar.gz - copied unchanged from r70730, dev/airflow/providers/apache_airflow_providers_apache_beam-5.7.2.tar.gz release/airflow/providers/apache_airflow_providers_apache_beam-5.7.2.tar.gz.asc - copied unchanged from r70730, dev/airflow/providers/apache_airflow_providers_apache_beam-5.7.2.tar.gz.asc release/airflow/providers/apache_airflow_providers_apache_beam-5.7.2.tar.gz.sha512 - copied unchanged from r70730, dev/airflow/providers/apache_airflow_providers_apache_beam-5.7.2.tar.gz.sha512 release/airflow/providers/apache_airflow_providers_apache_drill-2.7.3-py3-none-any.whl - copied unchanged from r70730, dev/airflow/providers/apache_airflow_providers_apache_drill-2.7.3-py3-none-any.whl release/airflow/providers/apache_airflow_providers_apache_drill-2.7.3-py3-none-any.whl.asc - copied unchanged from r70730, dev/airflow/providers/apache_airflow_providers_apache_drill-2.7.3-py3-none-any.whl.asc release/airflow/providers/apache_airflow_providers_apache_drill-2.7.3-py3-none-any.whl.sha512 - copied unchanged from r70730, dev/airflow/providers/apache_airflow_providers_apache_drill-2.7.3-py3-none-any.whl.sha512 release/airflow/providers/apache_airflow_providers_apache_drill-2.7.3.tar.gz - copied unchanged from r70730, dev/airflow/providers/apache_airflow_providers_apache_drill-2.7.3.tar.gz release/airflow/providers/apache_airflow_providers_apache_drill-2.7.3.tar.gz.asc - copied unchanged from r70730, dev/airflow/providers/apache_airflow_providers_apache_drill-2.7.3.tar.gz.asc release/airflow/providers/apache_airflow_providers_apache_drill-2.7.3.tar.gz.sha512 - copied unchanged from r70730, dev/airflow/providers/apache_airflow_providers_apache_drill-2.7.3.tar.gz.sha512 release/airflow/providers/apache_airflow_providers_apache_druid-3.10.2-py3-none-any.whl - copied unchanged from r70730, dev/airflow/providers/apache_airflow_providers_apache_druid-3.10.2-py3-none-any.whl release/airflow/providers/apache_airflow_providers_apache_druid-3.10.2-py3-none-any.whl.asc - copied unchanged from r70730, dev/airflow/providers/apache_airflow_providers_apache_druid-3.10.2-py3-none-any.whl.asc release/airflow/providers/apache_airflow_providers_apache_druid-3.10.2-py3-none-any.whl.sha512 - copied unchanged from r70730, dev/airflow/providers/apache_airflow_providers_apache_druid-3.10.2-py3-none-any.whl.sha512 release/airflow/providers/apache_airflow_providers_apache_druid-3.10.2.tar.gz - copied unchanged from r70730, dev/airflow/providers/apache_airflow_providers_apache_druid-3.10.2.tar.gz release/airflow/providers/apache_airflow_providers_apache_druid-3.10.2.tar.gz.asc - copied unchanged from r70730, dev/airflow/providers/apache_airflow_providers_apache_druid-3.10.2.tar.gz.asc release/airflow/providers/apache_airflow_providers_apache_druid-3.10.2
(airflow-site) branch add-documentation-2024-08-03 created (now 38980fdec4)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch add-documentation-2024-08-03 in repository https://gitbox.apache.org/repos/asf/airflow-site.git at 38980fdec4 Add documentation for packages - 2024-08-03 This branch includes the following new commits: new 38980fdec4 Add documentation for packages - 2024-08-03 The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
svn commit: r70673 [1/2] - /dev/airflow/providers/
Author: eladkal Date: Sat Aug 3 14:51:36 2024 New Revision: 70673 Log: Add artifacts for Airflow Providers 2024-08-03 Added: dev/airflow/providers/apache_airflow_providers_amazon-8.27.0-py3-none-any.whl (with props) dev/airflow/providers/apache_airflow_providers_amazon-8.27.0-py3-none-any.whl.asc dev/airflow/providers/apache_airflow_providers_amazon-8.27.0-py3-none-any.whl.sha512 dev/airflow/providers/apache_airflow_providers_amazon-8.27.0.tar.gz (with props) dev/airflow/providers/apache_airflow_providers_amazon-8.27.0.tar.gz.asc dev/airflow/providers/apache_airflow_providers_amazon-8.27.0.tar.gz.sha512 dev/airflow/providers/apache_airflow_providers_apache_beam-5.7.2-py3-none-any.whl (with props) dev/airflow/providers/apache_airflow_providers_apache_beam-5.7.2-py3-none-any.whl.asc dev/airflow/providers/apache_airflow_providers_apache_beam-5.7.2-py3-none-any.whl.sha512 dev/airflow/providers/apache_airflow_providers_apache_beam-5.7.2.tar.gz (with props) dev/airflow/providers/apache_airflow_providers_apache_beam-5.7.2.tar.gz.asc dev/airflow/providers/apache_airflow_providers_apache_beam-5.7.2.tar.gz.sha512 dev/airflow/providers/apache_airflow_providers_apache_drill-2.7.3-py3-none-any.whl (with props) dev/airflow/providers/apache_airflow_providers_apache_drill-2.7.3-py3-none-any.whl.asc dev/airflow/providers/apache_airflow_providers_apache_drill-2.7.3-py3-none-any.whl.sha512 dev/airflow/providers/apache_airflow_providers_apache_drill-2.7.3.tar.gz (with props) dev/airflow/providers/apache_airflow_providers_apache_drill-2.7.3.tar.gz.asc dev/airflow/providers/apache_airflow_providers_apache_drill-2.7.3.tar.gz.sha512 dev/airflow/providers/apache_airflow_providers_apache_druid-3.10.2-py3-none-any.whl (with props) dev/airflow/providers/apache_airflow_providers_apache_druid-3.10.2-py3-none-any.whl.asc dev/airflow/providers/apache_airflow_providers_apache_druid-3.10.2-py3-none-any.whl.sha512 dev/airflow/providers/apache_airflow_providers_apache_druid-3.10.2.tar.gz (with props) dev/airflow/providers/apache_airflow_providers_apache_druid-3.10.2.tar.gz.asc dev/airflow/providers/apache_airflow_providers_apache_druid-3.10.2.tar.gz.sha512 dev/airflow/providers/apache_airflow_providers_apache_impala-1.4.2-py3-none-any.whl (with props) dev/airflow/providers/apache_airflow_providers_apache_impala-1.4.2-py3-none-any.whl.asc dev/airflow/providers/apache_airflow_providers_apache_impala-1.4.2-py3-none-any.whl.sha512 dev/airflow/providers/apache_airflow_providers_apache_impala-1.4.2.tar.gz (with props) dev/airflow/providers/apache_airflow_providers_apache_impala-1.4.2.tar.gz.asc dev/airflow/providers/apache_airflow_providers_apache_impala-1.4.2.tar.gz.sha512 dev/airflow/providers/apache_airflow_providers_apache_pinot-4.4.2-py3-none-any.whl (with props) dev/airflow/providers/apache_airflow_providers_apache_pinot-4.4.2-py3-none-any.whl.asc dev/airflow/providers/apache_airflow_providers_apache_pinot-4.4.2-py3-none-any.whl.sha512 dev/airflow/providers/apache_airflow_providers_apache_pinot-4.4.2.tar.gz (with props) dev/airflow/providers/apache_airflow_providers_apache_pinot-4.4.2.tar.gz.asc dev/airflow/providers/apache_airflow_providers_apache_pinot-4.4.2.tar.gz.sha512 dev/airflow/providers/apache_airflow_providers_apprise-1.3.2-py3-none-any.whl (with props) dev/airflow/providers/apache_airflow_providers_apprise-1.3.2-py3-none-any.whl.asc dev/airflow/providers/apache_airflow_providers_apprise-1.3.2-py3-none-any.whl.sha512 dev/airflow/providers/apache_airflow_providers_apprise-1.3.2.tar.gz (with props) dev/airflow/providers/apache_airflow_providers_apprise-1.3.2.tar.gz.asc dev/airflow/providers/apache_airflow_providers_apprise-1.3.2.tar.gz.sha512 dev/airflow/providers/apache_airflow_providers_celery-3.7.3-py3-none-any.whl (with props) dev/airflow/providers/apache_airflow_providers_celery-3.7.3-py3-none-any.whl.asc dev/airflow/providers/apache_airflow_providers_celery-3.7.3-py3-none-any.whl.sha512 dev/airflow/providers/apache_airflow_providers_celery-3.7.3.tar.gz (with props) dev/airflow/providers/apache_airflow_providers_celery-3.7.3.tar.gz.asc dev/airflow/providers/apache_airflow_providers_celery-3.7.3.tar.gz.sha512 dev/airflow/providers/apache_airflow_providers_cncf_kubernetes-8.3.4-py3-none-any.whl (with props) dev/airflow/providers/apache_airflow_providers_cncf_kubernetes-8.3.4-py3-none-any.whl.asc dev/airflow/providers/apache_airflow_providers_cncf_kubernetes-8.3.4-py3-none-any.whl.sha512 dev/airflow/providers/apache_airflow_providers_cncf_kubernetes-8.3.4.tar.gz (with props) dev/airflow/providers/apache_airflow_providers_cncf_kubernetes-8.3.4.tar.gz.asc dev/airflow/providers/apache_airflow_providers_cncf_kubernetes-8.3.4
svn commit: r70673 [2/2] - /dev/airflow/providers/
Propchange: dev/airflow/providers/apache_airflow_providers_odbc-4.6.3-py3-none-any.whl -- svn:mime-type = application/octet-stream Added: dev/airflow/providers/apache_airflow_providers_odbc-4.6.3-py3-none-any.whl.asc == --- dev/airflow/providers/apache_airflow_providers_odbc-4.6.3-py3-none-any.whl.asc (added) +++ dev/airflow/providers/apache_airflow_providers_odbc-4.6.3-py3-none-any.whl.asc Sat Aug 3 14:51:36 2024 @@ -0,0 +1,7 @@ +-BEGIN PGP SIGNATURE- + +iIkEABYKADEWIQSDQO8ECQokO9vDRUWG4IhmPszevgUCZq5CgRMcZWxhZGthbEBh +cGFjaGUub3JnAAoJEIbgiGY+zN6+oWYBAIBdiI3knviww/NuXuUuxXeUluNH31Ee +hDCCjDPpTEQKAP9G/1hk/fZbNlCQoMRiRAOzP2n7D9TX3/zthuT125YQDA== +=ibAH +-END PGP SIGNATURE- Added: dev/airflow/providers/apache_airflow_providers_odbc-4.6.3-py3-none-any.whl.sha512 == --- dev/airflow/providers/apache_airflow_providers_odbc-4.6.3-py3-none-any.whl.sha512 (added) +++ dev/airflow/providers/apache_airflow_providers_odbc-4.6.3-py3-none-any.whl.sha512 Sat Aug 3 14:51:36 2024 @@ -0,0 +1 @@ +3e28590ef9245d5592f7f20febc8cb7ce3ae8dfd11d59870d16856c685c211d147a9ae850d06a9e45335cf1ea4e7eb295deb9fb71f5fe8223014eb8dfc08559b apache_airflow_providers_odbc-4.6.3-py3-none-any.whl Added: dev/airflow/providers/apache_airflow_providers_odbc-4.6.3.tar.gz == Binary file - no diff available. Propchange: dev/airflow/providers/apache_airflow_providers_odbc-4.6.3.tar.gz -- svn:mime-type = application/octet-stream Added: dev/airflow/providers/apache_airflow_providers_odbc-4.6.3.tar.gz.asc == --- dev/airflow/providers/apache_airflow_providers_odbc-4.6.3.tar.gz.asc (added) +++ dev/airflow/providers/apache_airflow_providers_odbc-4.6.3.tar.gz.asc Sat Aug 3 14:51:36 2024 @@ -0,0 +1,7 @@ +-BEGIN PGP SIGNATURE- + +iIkEABYKADEWIQSDQO8ECQokO9vDRUWG4IhmPszevgUCZq5CgRMcZWxhZGthbEBh +cGFjaGUub3JnAAoJEIbgiGY+zN6+yvQBALXDZf7ObUxaXuNqiZ9gHIrAZUPZvSKb +mT22x0mNbA39AQCdWnT1HEqM/zqfsePKfQkOzpLtK8btGp3kb9PXim3ACQ== +=pYTc +-END PGP SIGNATURE- Added: dev/airflow/providers/apache_airflow_providers_odbc-4.6.3.tar.gz.sha512 == --- dev/airflow/providers/apache_airflow_providers_odbc-4.6.3.tar.gz.sha512 (added) +++ dev/airflow/providers/apache_airflow_providers_odbc-4.6.3.tar.gz.sha512 Sat Aug 3 14:51:36 2024 @@ -0,0 +1 @@ +1da242c774217fc24722ef3113ed311db5929a8aff3a8c496e35f096870680c217064bc2ce4699d890323e9763e215887371b94db9fcf8ea7048b13c8288d41f apache_airflow_providers_odbc-4.6.3.tar.gz Added: dev/airflow/providers/apache_airflow_providers_openlineage-1.10.0-py3-none-any.whl == Binary file - no diff available. Propchange: dev/airflow/providers/apache_airflow_providers_openlineage-1.10.0-py3-none-any.whl -- svn:mime-type = application/octet-stream Added: dev/airflow/providers/apache_airflow_providers_openlineage-1.10.0-py3-none-any.whl.asc == --- dev/airflow/providers/apache_airflow_providers_openlineage-1.10.0-py3-none-any.whl.asc (added) +++ dev/airflow/providers/apache_airflow_providers_openlineage-1.10.0-py3-none-any.whl.asc Sat Aug 3 14:51:36 2024 @@ -0,0 +1,7 @@ +-BEGIN PGP SIGNATURE- + +iIkEABYKADEWIQSDQO8ECQokO9vDRUWG4IhmPszevgUCZq5CghMcZWxhZGthbEBh +cGFjaGUub3JnAAoJEIbgiGY+zN6+fcYBAMjAYgt7uZP4PtAm3ptTcER7mQdxoBrt +s4qx/3f+7yiSAP9teeZpSQXpRNAsyrZYU/oXTglLPFBw4VPq23v89wGrCQ== +=iovW +-END PGP SIGNATURE- Added: dev/airflow/providers/apache_airflow_providers_openlineage-1.10.0-py3-none-any.whl.sha512 == --- dev/airflow/providers/apache_airflow_providers_openlineage-1.10.0-py3-none-any.whl.sha512 (added) +++ dev/airflow/providers/apache_airflow_providers_openlineage-1.10.0-py3-none-any.whl.sha512 Sat Aug 3 14:51:36 2024 @@ -0,0 +1 @@ +9c91a17e70071bb39ccbb062fa714886fa59395c927960bc4fe1ad88370256e2fa0457b8cf7ba932622eb9330b54f5553007328dfd84775ab03552b4a3ea302e apache_airflow_providers_openlineage-1.10.0-py3-none-any.whl Added: dev/airflow/providers/apache_airflow_providers_openlineage-1.10.0.tar.gz == Binary file - no diff available. Propchange: dev/airflow/providers/apache_airflow_providers_openlineage-1.10.0.tar.gz ---
(airflow) branch main updated (5147028758 -> d23881c648)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 5147028758 Fix tests/always/test_secrets_backends.py by adding test_utils/db.py to accepted DB access (#41234) add d23881c648 Prepare docs for Aug 1st wave of providers (#41230) No new revisions were added by this update. Summary of changes: airflow/providers/amazon/CHANGELOG.rst | 5 ++- airflow/providers/apache/drill/CHANGELOG.rst | 8 airflow/providers/apache/drill/__init__.py | 2 +- airflow/providers/apache/drill/provider.yaml | 3 +- airflow/providers/apache/druid/CHANGELOG.rst | 14 +++ airflow/providers/apache/druid/__init__.py | 2 +- airflow/providers/apache/druid/provider.yaml | 3 +- .../apache/hdfs/.latest-doc-only-change.txt| 2 +- .../apache/hive/.latest-doc-only-change.txt| 2 +- .../apache/iceberg/.latest-doc-only-change.txt | 2 +- airflow/providers/apache/impala/CHANGELOG.rst | 12 ++ airflow/providers/apache/impala/__init__.py| 2 +- airflow/providers/apache/impala/provider.yaml | 3 +- .../apache/kafka/.latest-doc-only-change.txt | 2 +- .../apache/kylin/.latest-doc-only-change.txt | 2 +- .../apache/livy/.latest-doc-only-change.txt| 2 +- .../apache/pig/.latest-doc-only-change.txt | 2 +- airflow/providers/apache/pinot/CHANGELOG.rst | 12 ++ airflow/providers/apache/pinot/__init__.py | 2 +- airflow/providers/apache/pinot/provider.yaml | 3 +- airflow/providers/apprise/CHANGELOG.rst| 12 ++ airflow/providers/apprise/__init__.py | 2 +- airflow/providers/apprise/provider.yaml| 3 +- .../atlassian/jira/.latest-doc-only-change.txt | 2 +- airflow/providers/celery/CHANGELOG.rst | 14 +++ airflow/providers/celery/__init__.py | 2 +- airflow/providers/celery/provider.yaml | 3 +- airflow/providers/cncf/kubernetes/CHANGELOG.rst| 14 +++ airflow/providers/cncf/kubernetes/__init__.py | 2 +- airflow/providers/cncf/kubernetes/provider.yaml| 3 +- .../providers/cohere/.latest-doc-only-change.txt | 2 +- airflow/providers/common/compat/CHANGELOG.rst | 14 +++ airflow/providers/common/io/CHANGELOG.rst | 21 +++ airflow/providers/common/io/__init__.py| 2 +- airflow/providers/common/io/provider.yaml | 3 +- airflow/providers/common/sql/CHANGELOG.rst | 12 ++ airflow/providers/common/sql/__init__.py | 2 +- airflow/providers/common/sql/provider.yaml | 3 +- airflow/providers/databricks/CHANGELOG.rst | 28 ++ airflow/providers/databricks/__init__.py | 2 +- airflow/providers/databricks/provider.yaml | 3 +- .../dbt/cloud/.latest-doc-only-change.txt | 2 +- .../providers/dingding/.latest-doc-only-change.txt | 2 +- airflow/providers/docker/CHANGELOG.rst | 12 ++ airflow/providers/docker/__init__.py | 2 +- airflow/providers/docker/provider.yaml | 3 +- airflow/providers/elasticsearch/CHANGELOG.rst | 14 +++ airflow/providers/elasticsearch/__init__.py| 2 +- airflow/providers/elasticsearch/provider.yaml | 3 +- airflow/providers/exasol/CHANGELOG.rst | 14 +++ airflow/providers/exasol/__init__.py | 2 +- airflow/providers/exasol/provider.yaml | 3 +- .../providers/facebook/.latest-doc-only-change.txt | 2 +- airflow/providers/ftp/CHANGELOG.rst| 12 ++ airflow/providers/ftp/__init__.py | 2 +- airflow/providers/ftp/provider.yaml| 3 +- .../providers/github/.latest-doc-only-change.txt | 2 +- airflow/providers/google/CHANGELOG.rst | 2 + airflow/providers/http/.latest-doc-only-change.txt | 2 +- airflow/providers/jdbc/CHANGELOG.rst | 19 ++ airflow/providers/jdbc/__init__.py | 2 +- airflow/providers/jdbc/provider.yaml | 3 +- .../providers/jenkins/.latest-doc-only-change.txt | 2 +- airflow/providers/microsoft/azure/CHANGELOG.rst| 17 + airflow/providers/microsoft/azure/__init__.py | 2 +- airflow/providers/microsoft/azure/provider.yaml| 3 +- airflow/providers/microsoft/mssql/CHANGELOG.rst| 17 + airflow/providers/microsoft/mssql/__init__.py | 2 +- airflow/providers/microsoft/mssql/provider.yaml| 3 +- .../microsoft/psrp/.latest-doc-only-change.txt | 2 +- airflow/providers/mysql/CHANGELOG.rst | 12 ++ airflow/providers/mysql/__init__.py| 2 +- airflow/providers/mysql/provider.yaml | 3 +- airflow/providers/odbc/CHANGELOG.rst | 14 +++ airflow/providers
(airflow) branch main updated (ed3a0e49bf -> 4d27069d9c)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from ed3a0e49bf Fix the right affected provider version (#41215) add 4d27069d9c Adjust default extractor's on_failure detection for airflow 2.10 fix (#41094) No new revisions were added by this update. Summary of changes: airflow/providers/openlineage/extractors/base.py | 10 +- airflow/providers/openlineage/plugins/listener.py | 9 +++-- airflow/providers/openlineage/utils/utils.py | 4 +-- .../amazon/aws/operators/test_redshift_sql.py | 2 +- .../providers/openlineage/extractors/test_base.py | 42 ++ 5 files changed, 58 insertions(+), 9 deletions(-)
(airflow) branch main updated: Add common-compat to boring-cyborg.yml (#41180)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 4d849e87ec Add common-compat to boring-cyborg.yml (#41180) 4d849e87ec is described below commit 4d849e87ecce654e100c36dca3d0fbe19bcd8b6b Author: Elad Kalif <45845474+elad...@users.noreply.github.com> AuthorDate: Fri Aug 2 06:21:19 2024 +0300 Add common-compat to boring-cyborg.yml (#41180) --- .github/boring-cyborg.yml | 5 + 1 file changed, 5 insertions(+) diff --git a/.github/boring-cyborg.yml b/.github/boring-cyborg.yml index a6c1ed0919..516fc2b457 100644 --- a/.github/boring-cyborg.yml +++ b/.github/boring-cyborg.yml @@ -170,6 +170,11 @@ labelPRBasedOnFilePath: - tests/providers/cohere/**/* - tests/system/providers/cohere/**/* + provider:common-compat: +- airflow/providers/common/compat/**/* +- docs/apache-airflow-providers-common-compat/**/* +- tests/providers/common/compat/**/* + provider:common-io: - airflow/providers/common/io/**/* - docs/apache-airflow-providers-common-io/**/*
(airflow) branch main updated: Fix 120 or so test_python tests in db_isolation mode (#41173)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new fbb6639657 Fix 120 or so test_python tests in db_isolation mode (#41173) fbb6639657 is described below commit fbb6639657b32ca0188ab14353b540efc9710f96 Author: Jarek Potiuk AuthorDate: Thu Aug 1 14:49:29 2024 +0200 Fix 120 or so test_python tests in db_isolation mode (#41173) Serialized dags are needed in a number of places where DB isolation mode needs to work. This one adds seialization to test_python tests to make them work. --- tests/operators/test_python.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tests/operators/test_python.py b/tests/operators/test_python.py index 728b5aae8d..66e3c9a823 100644 --- a/tests/operators/test_python.py +++ b/tests/operators/test_python.py @@ -69,7 +69,7 @@ from tests.test_utils.db import clear_db_runs if TYPE_CHECKING: from airflow.models.dagrun import DagRun -pytestmark = pytest.mark.db_test +pytestmark = [pytest.mark.db_test, pytest.mark.need_serialized_dag] TI = TaskInstance @@ -1734,6 +1734,7 @@ class TestCurrentContextRuntime: op.run(ignore_first_depends_on_past=True, ignore_ti_state=True) +@pytest.mark.need_serialized_dag(False) class TestShortCircuitWithTeardown: @pytest.mark.parametrize( "ignore_downstream_trigger_rules, with_teardown, should_skip, expected",
(airflow) branch main updated: Update providers metadata 2024-07-31 (#41159)
This is an automated email from the ASF dual-hosted git repository.
eladkal pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new 6a5ae50891 Update providers metadata 2024-07-31 (#41159)
6a5ae50891 is described below
commit 6a5ae508916f0b084cbf004e71f85708ef150e4c
Author: Elad Kalif <45845474+elad...@users.noreply.github.com>
AuthorDate: Wed Jul 31 17:42:24 2024 +0300
Update providers metadata 2024-07-31 (#41159)
---
generated/provider_metadata.json | 4
1 file changed, 4 insertions(+)
diff --git a/generated/provider_metadata.json b/generated/provider_metadata.json
index ab234121a1..ed9bb1a0d8 100644
--- a/generated/provider_metadata.json
+++ b/generated/provider_metadata.json
@@ -3511,6 +3511,10 @@
"1.2.1": {
"associated_airflow_version": "2.9.3",
"date_released": "2024-07-12T12:38:31Z"
+},
+"1.2.2": {
+"associated_airflow_version": "2.9.3",
+"date_released": "2024-07-31T14:18:50Z"
}
},
"facebook": {
