[airflow-site] branch add-andrey updated (7664dc1a47 -> 8189ecbbba)

2022-12-01 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch add-andrey in repository https://gitbox.apache.org/repos/asf/airflow-site.git omit 7664dc1a47 Add Andrey as committer add 8189ecbbba Add Andrey as committer This update added new revisi

[airflow-site] branch add-andrey updated (570c2b30c0 -> 7664dc1a47)

2022-12-01 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch add-andrey in repository https://gitbox.apache.org/repos/asf/airflow-site.git omit 570c2b30c0 Add Andrey as committer add 7664dc1a47 Add Andrey as committer This update added new revisi

[GitHub] [airflow] potiuk opened a new pull request, #28050: Add Andrey as committer

2022-12-01 Thread GitBox
potiuk opened a new pull request, #28050: URL: https://github.com/apache/airflow/pull/28050 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more info

[GitHub] [airflow-site] potiuk commented on pull request #691: Add Andrey as committer

2022-12-01 Thread GitBox
potiuk commented on PR #691: URL: https://github.com/apache/airflow-site/pull/691#issuecomment-1334860868 cc: @taragolis - let's see if you can already approve it -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[airflow-site] 01/01: Add Andrey as committer

2022-12-01 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch add-andrey in repository https://gitbox.apache.org/repos/asf/airflow-site.git commit 570c2b30c07f534ab2f8dedbe68d9e7cdcc018ef Author: Jarek Potiuk AuthorDate: Fri Dec 2 08:34:43 2022 +0100 A

[airflow-site] branch add-andrey created (now 570c2b30c0)

2022-12-01 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch add-andrey in repository https://gitbox.apache.org/repos/asf/airflow-site.git at 570c2b30c0 Add Andrey as committer This branch includes the following new commits: new 570c2b30c0 Add

[GitHub] [airflow] dstandish commented on a diff in pull request #28047: Ensure KubernetesExecutor's multi_namespace_mode work without cluster-scope role

2022-12-01 Thread GitBox
dstandish commented on code in PR #28047: URL: https://github.com/apache/airflow/pull/28047#discussion_r1037857487 ## airflow/executors/kubernetes_executor.py: ## @@ -288,15 +294,33 @@ def _make_kube_watcher(self) -> KubernetesJobWatcher: watcher.start() return

[GitHub] [airflow] dstandish commented on a diff in pull request #28047: Ensure KubernetesExecutor's multi_namespace_mode work without cluster-scope role

2022-12-01 Thread GitBox
dstandish commented on code in PR #28047: URL: https://github.com/apache/airflow/pull/28047#discussion_r1037854334 ## airflow/executors/kubernetes_executor.py: ## @@ -66,7 +67,7 @@ class ResourceVersion: """Singleton for tracking resourceVersion from Kubernetes""" _i

[GitHub] [airflow] dstandish opened a new pull request, #28049: Remove unused task_queue attr from k8s scheduler class

2022-12-01 Thread GitBox
dstandish opened a new pull request, #28049: URL: https://github.com/apache/airflow/pull/28049 Not used, so think we can remove it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

[GitHub] [airflow] Taragolis commented on issue #28026: webserver login does not work by AD with config `AUTH_LDAP_BIND_USER` but with `auth_ldap_bind_user`

2022-12-01 Thread GitBox
Taragolis commented on issue #28026: URL: https://github.com/apache/airflow/issues/28026#issuecomment-1334830090 This more refers to LDAP protocol and its implementation rather than Airflow itself - Is this MS Active Directory or OpenLDAP - How it actually implemented in python: https

[GitHub] [airflow] MatrixManAtYrService opened a new issue, #28048: Try to serialize a set to XCom, get unhelpful error message.

2022-12-01 Thread GitBox
MatrixManAtYrService opened a new issue, #28048: URL: https://github.com/apache/airflow/issues/28048 ### Apache Airflow version main (development) ### What happened Here's a dag, and an exception from its task logs, as generate from 2.5.0 (as of e2455d870056391eed13e32e2

[GitHub] [airflow] Taragolis commented on a diff in pull request #28039: Migrate amazon provider hooks tests from `unittests` to `pytest`

2022-12-01 Thread GitBox
Taragolis commented on code in PR #28039: URL: https://github.com/apache/airflow/pull/28039#discussion_r1037832131 ## tests/providers/amazon/aws/hooks/test_emr_containers.py: ## @@ -110,7 +109,9 @@ def test_query_status_polling_with_timeout(self, mock_session): mock_se

[GitHub] [airflow] Taragolis commented on a diff in pull request #28039: Migrate amazon provider hooks tests from `unittests` to `pytest`

2022-12-01 Thread GitBox
Taragolis commented on code in PR #28039: URL: https://github.com/apache/airflow/pull/28039#discussion_r1037814650 ## tests/providers/amazon/aws/hooks/test_batch_client.py: ## @@ -299,16 +300,17 @@ def test_job_splunk_logs(self): } ] } -

[GitHub] [airflow] johnwarburton commented on issue #27399: CronTriggerTimetable lost one task occasionally

2022-12-01 Thread GitBox
johnwarburton commented on issue #27399: URL: https://github.com/apache/airflow/issues/27399#issuecomment-1334760796 Hi @RNHTTR - no, I am waiting to see if there is a fix in the milestones `CronTriggerTimetable` must have a timezone attached: ``` | timetable=CronTriggerTi

[GitHub] [airflow] XD-DENG commented on pull request #28047: Ensure KubernetesExecutor's multi_namespace_mode work without cluster-scope role

2022-12-01 Thread GitBox
XD-DENG commented on PR #28047: URL: https://github.com/apache/airflow/pull/28047#issuecomment-1334704956 Hi @potiuk and @ferruzzi tagging you both, to follow up [our earlier discussion in the mail list](https://lists.apache.org/thread/xxsppw7qwvky78l6nx41vlz593gj4zqb). -- This is an aut

[GitHub] [airflow] XD-DENG opened a new pull request, #28047: Ensure KubernetesExecutor's multi_namespace_mode work without cluster-scope role

2022-12-01 Thread GitBox
XD-DENG opened a new pull request, #28047: URL: https://github.com/apache/airflow/pull/28047 Currently `KubernetesExecutor`'s `multi_namespace_mode` requires the Scheduler to have cluster-scope role on the Kubernetes Cluster, because it's using function `list_pod_for_all_namespaces()`.

[GitHub] [airflow] HwiLu commented on issue #28026: webserver login does not work by AD with config `AUTH_LDAP_BIND_USER` but with `auth_ldap_bind_user`

2022-12-01 Thread GitBox
HwiLu commented on issue #28026: URL: https://github.com/apache/airflow/issues/28026#issuecomment-1334687100 It still shows the same error ``` 192.168.208.171 - - [02/Dec/2022:10:26:50 +0800] "GET /static/pin_32.png HTTP/1.1" 304 0 "http://hadoop-180-91:8086/login/"; "Mozilla/5.0 (Win

[GitHub] [airflow] vandonr-amz commented on pull request #28039: Migrate amazon provider hooks tests from `unittests` to `pytest`

2022-12-01 Thread GitBox
vandonr-amz commented on PR #28039: URL: https://github.com/apache/airflow/pull/28039#issuecomment-1334671366 > > nice. I noticed that on some `setup_method`s you added the optional `method` parameter without using it, while you skipped it on others. I think it'd be best to omit it wherever

[GitHub] [airflow] o-nikolas opened a new pull request, #28046: Increase pod start timeout for eks fargate system test

2022-12-01 Thread GitBox
o-nikolas opened a new pull request, #28046: URL: https://github.com/apache/airflow/pull/28046 When running these tests in AWS we often come close to the default of pod start timeout of 120 seconds and every now and then we breach that limit (see screenshot below). Increasing the timeout to

[GitHub] [airflow] boring-cyborg[bot] commented on issue #28045: GoogleDisplayVideo360 (hooks|operators|sensors) aren't compatible with v2 of the Display & Video 360 API

2022-12-01 Thread GitBox
boring-cyborg[bot] commented on issue #28045: URL: https://github.com/apache/airflow/issues/28045#issuecomment-1334613416 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the mess

[GitHub] [airflow] github-sang opened a new issue, #28045: GoogleDisplayVideo360 (hooks|operators|sensors) aren't compatible with v2 of the Display & Video 360 API

2022-12-01 Thread GitBox
github-sang opened a new issue, #28045: URL: https://github.com/apache/airflow/issues/28045 ### Apache Airflow Provider(s) google ### Versions of Apache Airflow Providers apache-airflow-providers-google 8.4.0 ### Apache Airflow version airfl

[GitHub] [airflow] dstandish commented on pull request #28019: Use asserts instead of exceptions for executor not started

2022-12-01 Thread GitBox
dstandish commented on PR #28019: URL: https://github.com/apache/airflow/pull/28019#issuecomment-1334608169 > we want to use assert for if TYPE_CHECKING when we want to get rid of mypy warning that None has no property etc Yes, and by my reading, it appears that's exactly what these a

[GitHub] [airflow] jedcunningham commented on a diff in pull request #28034: Added custom environment variables to statsd

2022-12-01 Thread GitBox
jedcunningham commented on code in PR #28034: URL: https://github.com/apache/airflow/pull/28034#discussion_r1037668526 ## chart/templates/statsd/statsd-deployment.yaml: ## @@ -115,6 +115,10 @@ spec: - name: config mountPath: /etc/statsd-exporter/mappi

[GitHub] [airflow] Taragolis closed pull request #28039: Migrate amazon provider hooks tests from `unittests` to `pytest`

2022-12-01 Thread GitBox
Taragolis closed pull request #28039: Migrate amazon provider hooks tests from `unittests` to `pytest` URL: https://github.com/apache/airflow/pull/28039 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [airflow] Taragolis commented on pull request #28039: Migrate amazon provider hooks tests from `unittests` to `pytest`

2022-12-01 Thread GitBox
Taragolis commented on PR #28039: URL: https://github.com/apache/airflow/pull/28039#issuecomment-1334568914 > nice. I noticed that on some `setup_method`s you added the optional `method` parameter without using it, while you skipped it on others. I think it'd be best to omit it wherever pos

[airflow] branch constraints-main updated: Updating constraints. Build id:

2022-12-01 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 2ad12f81ad Updating con

[GitHub] [airflow] potiuk commented on pull request #28019: Use asserts instead of exceptions for executor not started

2022-12-01 Thread GitBox
potiuk commented on PR #28019: URL: https://github.com/apache/airflow/pull/28019#issuecomment-1334545117 > OK -- I updated to use the `if TYPE_CHECKING` style I think this is misunderstanding. This "if TYPE_CHECKING" is not something that you should add. There are specific places wher

[GitHub] [airflow] italovinicius18 commented on issue #28023: Dataproc operators ClusterGenerator can't find parameter of ShieldedInstanceConfig

2022-12-01 Thread GitBox
italovinicius18 commented on issue #28023: URL: https://github.com/apache/airflow/issues/28023#issuecomment-1334537305 I know now, that the syntax needs to be the same of protobuf protocol, but I found the same error: ValueError: Protocol message GceClusterConfig has no "shielded_ins

[GitHub] [airflow] potiuk closed issue #28043: Scheduler keeps dying - Killing DAGFileProcessorProcess

2022-12-01 Thread GitBox
potiuk closed issue #28043: Scheduler keeps dying - Killing DAGFileProcessorProcess URL: https://github.com/apache/airflow/issues/28043 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

[GitHub] [airflow] potiuk commented on issue #27976: `SQLColumnCheckOperator` failures after upgrading to `common-sql==1.3.0`

2022-12-01 Thread GitBox
potiuk commented on issue #27976: URL: https://github.com/apache/airflow/issues/27976#issuecomment-1334531255 @mag3141592 - can you please check if 4.0.2rc1 solves the problem and report it in https://github.com/apache/airflow/issues/28036 -- This is an automated message from the Apache G

[GitHub] [airflow] potiuk commented on issue #28036: Status of testing Providers that were prepared on December 01, 2022

2022-12-01 Thread GitBox
potiuk commented on issue #28036: URL: https://github.com/apache/airflow/issues/28036#issuecomment-1334529582 # 27997 #28006 checked by @park-peter https://github.com/apache/airflow/issues/27978#issuecomment-1334408599 -- This is an automated message from the Apache Git Service. To respon

[GitHub] [airflow] vandonr-amz commented on a diff in pull request #28039: Migrate amazon provider hooks tests from `unittests` to `pytest`

2022-12-01 Thread GitBox
vandonr-amz commented on code in PR #28039: URL: https://github.com/apache/airflow/pull/28039#discussion_r1037610212 ## tests/providers/amazon/aws/hooks/test_glue_crawler.py: ## @@ -83,18 +82,17 @@ } -class TestGlueCrawlerHook(unittest.TestCase): -@classmethod -def

[GitHub] [airflow] dimon222 commented on issue #8212: Can't read S3 remote logs when using gevent/eventlent webserver workers.

2022-12-01 Thread GitBox
dimon222 commented on issue #8212: URL: https://github.com/apache/airflow/issues/8212#issuecomment-1334519505 Only got to test patches now. I'm on 2.4.3 1. --preload doesn't help 2. Monkey patching of default settings.py still works with new minor adjustment - new annotation import hav

[airflow] branch main updated (7f0332de2d -> e2455d8700)

2022-12-01 Thread bbovenzi
This is an automated email from the ASF dual-hosted git repository. bbovenzi pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 7f0332de2d Enhance chart to allow over-riding command-line args to statsd exporter (#28041) add e2455d8700 Auto

[GitHub] [airflow] bbovenzi merged pull request #27805: Automatically save and allow restore of recent DAG run configs

2022-12-01 Thread GitBox
bbovenzi merged PR #27805: URL: https://github.com/apache/airflow/pull/27805 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #27805: Automatically save and allow restore of recent DAG run configs

2022-12-01 Thread GitBox
boring-cyborg[bot] commented on PR #27805: URL: https://github.com/apache/airflow/pull/27805#issuecomment-1334504199 Awesome work, congrats on your first merged pull request! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [airflow] bbovenzi closed issue #27714: Re-use recent DagRun JSON-configurations

2022-12-01 Thread GitBox
bbovenzi closed issue #27714: Re-use recent DagRun JSON-configurations URL: https://github.com/apache/airflow/issues/27714 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [airflow] ferruzzi commented on a diff in pull request #28039: Migrate amazon provider hooks tests from `unittests` to `pytest`

2022-12-01 Thread GitBox
ferruzzi commented on code in PR #28039: URL: https://github.com/apache/airflow/pull/28039#discussion_r1037535947 ## tests/providers/amazon/aws/hooks/test_glacier.py: ## @@ -47,25 +47,21 @@ def test_retrieve_inventory_should_return_job_id(self, mock_conn): assert job_i

[GitHub] [airflow] dstandish commented on a diff in pull request #28019: Use asserts instead of exceptions for executor not started

2022-12-01 Thread GitBox
dstandish commented on code in PR #28019: URL: https://github.com/apache/airflow/pull/28019#discussion_r1037587532 ## airflow/executors/base_executor.py: ## @@ -33,8 +33,6 @@ PARALLELISM: int = conf.getint("core", "PARALLELISM") -NOT_STARTED_MESSAGE = "The executor should b

[GitHub] [airflow] jedcunningham commented on pull request #28041: Enhance chart to allow over-riding command-line args to statsd exporter

2022-12-01 Thread GitBox
jedcunningham commented on PR #28041: URL: https://github.com/apache/airflow/pull/28041#issuecomment-1334453147 Thanks @rob-1126! Congrats on your first commit 🎉 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[airflow] branch main updated: Enhance chart to allow over-riding command-line args to statsd exporter (#28041)

2022-12-01 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 7f0332de2d Enhance chart to allow over-ridin

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #28041: Enhance chart to allow over-riding command-line args to statsd exporter

2022-12-01 Thread GitBox
boring-cyborg[bot] commented on PR #28041: URL: https://github.com/apache/airflow/pull/28041#issuecomment-1334452547 Awesome work, congrats on your first merged pull request! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [airflow] jedcunningham merged pull request #28041: Enhance chart to allow over-riding command-line args to statsd exporter

2022-12-01 Thread GitBox
jedcunningham merged PR #28041: URL: https://github.com/apache/airflow/pull/28041 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@ai

[GitHub] [airflow] jedcunningham commented on pull request #28041: Enhance chart to allow over-riding command-line args to statsd exporter

2022-12-01 Thread GitBox
jedcunningham commented on PR #28041: URL: https://github.com/apache/airflow/pull/28041#issuecomment-1334452314 Merging, sqlite tests failure is unrelated (and flaky). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

[GitHub] [airflow] dstandish commented on pull request #28019: Use asserts instead of exceptions for executor not started

2022-12-01 Thread GitBox
dstandish commented on PR #28019: URL: https://github.com/apache/airflow/pull/28019#issuecomment-1334451150 OK -- I updated to use the `if TYPE_CHECKING` style -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

[GitHub] [airflow] shubham22 commented on a diff in pull request #27823: Amazon Provider Package user agent

2022-12-01 Thread GitBox
shubham22 commented on code in PR #27823: URL: https://github.com/apache/airflow/pull/27823#discussion_r103755 ## airflow/providers/amazon/aws/hooks/base_aws.py: ## @@ -25,9 +25,13 @@ from __future__ import annotations Review Comment: I confirmed with the managed serv

[GitHub] [airflow] ferruzzi commented on a diff in pull request #28039: Migrate amazon provider hooks tests from `unittests` to `pytest`

2022-12-01 Thread GitBox
ferruzzi commented on code in PR #28039: URL: https://github.com/apache/airflow/pull/28039#discussion_r1037539606 ## tests/providers/amazon/aws/hooks/test_glue_crawler.py: ## @@ -83,18 +82,17 @@ } -class TestGlueCrawlerHook(unittest.TestCase): -@classmethod -def set

[GitHub] [airflow] boring-cyborg[bot] commented on issue #28043: Scheduler keeps dying - Killing DAGFileProcessorProcess

2022-12-01 Thread GitBox
boring-cyborg[bot] commented on issue #28043: URL: https://github.com/apache/airflow/issues/28043#issuecomment-1334415036 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the mess

[GitHub] [airflow] gabrielboehme opened a new issue, #28043: Scheduler keeps dying - Killing DAGFileProcessorProcess

2022-12-01 Thread GitBox
gabrielboehme opened a new issue, #28043: URL: https://github.com/apache/airflow/issues/28043 ### Apache Airflow version 2.4.3 ### What happened I'm running airflow on Docker (inside Windows WSL2) with Celery Executor. Suddenly, from one day to another, my airflow schedu

[GitHub] [airflow] park-peter commented on issue #27978: KeyError: 0 error with common-sql version 1.3.0

2022-12-01 Thread GitBox
park-peter commented on issue #27978: URL: https://github.com/apache/airflow/issues/27978#issuecomment-1334408599 @potiuk with snowflake 4.0.2rc1 and common-sql 1.3.1, I can confirm that things look good! -- This is an automated message from the Apache Git Service. To respond to the messa

[GitHub] [airflow] jh242 commented on pull request #27805: Automatically save and allow restore of recent DAG run configs

2022-12-01 Thread GitBox
jh242 commented on PR #27805: URL: https://github.com/apache/airflow/pull/27805#issuecomment-1334353906 updated our branch to be up-to-date with airflow. Thanks again @bbovenzi @pierrejeambrun :D -- This is an automated message from the Apache Git Service. To respond to the message, pleas

[GitHub] [airflow] ferruzzi commented on pull request #28039: Migrate amazon provider hooks tests from `unittests` to `pytest`

2022-12-01 Thread GitBox
ferruzzi commented on PR #28039: URL: https://github.com/apache/airflow/pull/28039#issuecomment-1334330677 That's awesome. I was literally looking at this earlier this week and have some local code where I started working on it. :+1: Will review ASAP -- This is an automated message fro

[GitHub] [airflow] p-madduri opened a new issue, #28042: Type Error while using dynamodb_to_s3 operator

2022-12-01 Thread GitBox
p-madduri opened a new issue, #28042: URL: https://github.com/apache/airflow/issues/28042 ### Apache Airflow version Other Airflow 2 version (please specify below) ### What happened https://github.com/apache/airflow/blob/430e930902792fc37cdd2c517783f7dd544fbebf/airflow/p

[GitHub] [airflow] boring-cyborg[bot] commented on issue #28042: Type Error while using dynamodb_to_s3 operator

2022-12-01 Thread GitBox
boring-cyborg[bot] commented on issue #28042: URL: https://github.com/apache/airflow/issues/28042#issuecomment-1334258640 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the mess

[GitHub] [airflow] rob-1126 opened a new pull request, #28041: Enhance chart to allow over-riding command-line args to statsd exporter

2022-12-01 Thread GitBox
rob-1126 opened a new pull request, #28041: URL: https://github.com/apache/airflow/pull/28041 Add a new configurable option (statsd.args) to the helm chart to allow args on the statsd-exporter container to be over-ridden in the same manner as the args on other containers. -- This is

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #28041: Enhance chart to allow over-riding command-line args to statsd exporter

2022-12-01 Thread GitBox
boring-cyborg[bot] commented on PR #28041: URL: https://github.com/apache/airflow/pull/28041#issuecomment-1334231497 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution

[GitHub] [airflow] Taragolis commented on a diff in pull request #28040: Migrate google provider cloud utils tests from `unittests` to `pytest`

2022-12-01 Thread GitBox
Taragolis commented on code in PR #28040: URL: https://github.com/apache/airflow/pull/28040#discussion_r1037451071 ## tests/providers/google/cloud/utils/test_credentials_provider.py: ## @@ -131,35 +128,21 @@ def test_provide_gcp_conn_and_credentials(self, mock_builder):

[GitHub] [airflow] Taragolis opened a new pull request, #28040: Migrate google provider cloud utils tests from `unittests` to `pytest`

2022-12-01 Thread GitBox
Taragolis opened a new pull request, #28040: URL: https://github.com/apache/airflow/pull/28040 Migrate google cloud provider's utils and secret backend tests to `pytest`. All changes are more or less straightforward: - Get rid of `unittests.TestCase` class and **TestCase.assert*** m

[GitHub] [airflow] italovinicius18 commented on issue #28023: Dataproc operators ClusterGenerator can't find parameter of ShieldedInstanceConfig

2022-12-01 Thread GitBox
italovinicius18 commented on issue #28023: URL: https://github.com/apache/airflow/issues/28023#issuecomment-1334184572 I've tried this option but I got a error ValueError: Protocol message GceClusterConfig has no "shieldedInstanceConfig" field. -- This is an automated message from

[GitHub] [airflow] Taragolis commented on a diff in pull request #28039: Migrate amazon provider hooks tests from `unittests` to `pytest`

2022-12-01 Thread GitBox
Taragolis commented on code in PR #28039: URL: https://github.com/apache/airflow/pull/28039#discussion_r1037439960 ## tests/providers/amazon/aws/hooks/test_emr_containers.py: ## @@ -110,7 +109,9 @@ def test_query_status_polling_with_timeout(self, mock_session): mock_se

[GitHub] [airflow] Taragolis opened a new pull request, #28039: Migrate amazon provider hooks tests from `unittests` to `pytest`

2022-12-01 Thread GitBox
Taragolis opened a new pull request, #28039: URL: https://github.com/apache/airflow/pull/28039 related: #27970 Migrate Amazon provider's hooks and utils tests to `pytest`. All changes are more or less straightforward: - Get rid of `unittests.TestCase` class and **TestCase.ass

[GitHub] [airflow] Taragolis commented on pull request #27970: Replace `unittests` in amazon provider tests by pure `pytest`

2022-12-01 Thread GitBox
Taragolis commented on PR #27970: URL: https://github.com/apache/airflow/pull/27970#issuecomment-1334174638 I decide split this changes to multiple separate PR's - Could be really reviewed - Less potential merge conflicts -- This is an automated message from the Apache Git Service. T

[GitHub] [airflow] Taragolis closed pull request #27970: Replace `unittests` in amazon provider tests by pure `pytest`

2022-12-01 Thread GitBox
Taragolis closed pull request #27970: Replace `unittests` in amazon provider tests by pure `pytest` URL: https://github.com/apache/airflow/pull/27970 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [airflow] jrggggg closed pull request #27951: New SambaFileSensor

2022-12-01 Thread GitBox
jrg closed pull request #27951: New SambaFileSensor URL: https://github.com/apache/airflow/pull/27951 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail

[GitHub] [airflow] xlanor commented on pull request #27898: fix: current_state method on TaskInstance doesn't filter by map_index

2022-12-01 Thread GitBox
xlanor commented on PR #27898: URL: https://github.com/apache/airflow/pull/27898#issuecomment-1334118211 > I rerun the test. Lets see seems to be all good now, thank you -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

[GitHub] [airflow] eladkal commented on pull request #27898: fix: current_state method on TaskInstance doesn't filter by map_index

2022-12-01 Thread GitBox
eladkal commented on PR #27898: URL: https://github.com/apache/airflow/pull/27898#issuecomment-1334065847 I rerun the test. Lets see -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

[GitHub] [airflow] mdering commented on issue #28023: Dataproc operators ClusterGenerator can't find parameter of ShieldedInstanceConfig

2022-12-01 Thread GitBox
mdering commented on issue #28023: URL: https://github.com/apache/airflow/issues/28023#issuecomment-1334052073 I would just add it after the call to `make()` ```python CLUSTER_CONFIG['gce_cluster_config']['shielded_instance_config']={...} ``` -- This is an automated message fr

[GitHub] [airflow] xlanor commented on pull request #27898: fix: current_state method on TaskInstance doesn't filter by map_index

2022-12-01 Thread GitBox
xlanor commented on PR #27898: URL: https://github.com/apache/airflow/pull/27898#issuecomment-1334046699 > Assuming CI passes. Posted this issue to slack for sqlite, since it doesnt seem to be running at all. Leaving the slack link here, https://apache-airflow.slack.com/archives/CC

[GitHub] [airflow] yehoshuadimarsky commented on pull request #27947: refactored Amazon Redshift-data functionality into the hook

2022-12-01 Thread GitBox
yehoshuadimarsky commented on PR #27947: URL: https://github.com/apache/airflow/pull/27947#issuecomment-1334045531 > For avoid any further regression error I would suggest move tests this methods from operator `tests/providers/amazon/aws/operators/test_redshift_data.py` to hook `tests/prov

[GitHub] [airflow] yehoshuadimarsky commented on a diff in pull request #27947: refactored Amazon Redshift-data functionality into the hook

2022-12-01 Thread GitBox
yehoshuadimarsky commented on code in PR #27947: URL: https://github.com/apache/airflow/pull/27947#discussion_r1037337955 ## airflow/providers/amazon/aws/operators/redshift_data.py: ## @@ -99,67 +97,29 @@ def __init__( ) self.aws_conn_id = aws_conn_id

[GitHub] [airflow] Taragolis commented on a diff in pull request #28005: Fix GlueCrawlerOperature failure when using tags (apache#27556)

2022-12-01 Thread GitBox
Taragolis commented on code in PR #28005: URL: https://github.com/apache/airflow/pull/28005#discussion_r1037260249 ## airflow/providers/amazon/aws/hooks/glue_crawler.py: ## @@ -78,16 +83,60 @@ def update_crawler(self, **crawler_kwargs) -> bool: crawler_name = crawler_kw

[GitHub] [airflow] jh242 commented on a diff in pull request #27805: Automatically save and allow restore of recent DAG run configs

2022-12-01 Thread GitBox
jh242 commented on code in PR #27805: URL: https://github.com/apache/airflow/pull/27805#discussion_r1037281152 ## airflow/www/static/js/trigger.js: ## @@ -20,13 +20,32 @@ /* global document, CodeMirror, window */ const textArea = document.getElementById('json'); +const recen

[GitHub] [airflow] italovinicius18 commented on issue #28023: Dataproc operators ClusterGenerator can't find parameter of ShieldedInstanceConfig

2022-12-01 Thread GitBox
italovinicius18 commented on issue #28023: URL: https://github.com/apache/airflow/issues/28023#issuecomment-1333966358 @mdering And how I put this using CLUSTER_CONFIG using ClusterGenerator? Like this: ```py CLUSTER_CONFIG = ClusterGenerator( project_id=PROJECT_ID,

[airflow] annotated tag 2.5.0rc3 updated (fa2bec0429 -> 5ee3ab75a3)

2022-12-01 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a change to annotated tag 2.5.0rc3 in repository https://gitbox.apache.org/repos/asf/airflow.git *** WARNING: tag 2.5.0rc3 was modified! *** from fa2bec0429 (commit) to 5ee3ab75a3 (tag) tagging

[GitHub] [airflow] vincbeck commented on a diff in pull request #27892: [AIP-44] Add internal API definition.

2022-12-01 Thread GitBox
vincbeck commented on code in PR #27892: URL: https://github.com/apache/airflow/pull/27892#discussion_r1037258988 ## tests/api_internal/test_internal_api_call.py: ## @@ -0,0 +1,151 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license ag

[GitHub] [airflow] vincbeck commented on a diff in pull request #27892: [AIP-44] Add internal API definition.

2022-12-01 Thread GitBox
vincbeck commented on code in PR #27892: URL: https://github.com/apache/airflow/pull/27892#discussion_r1037262863 ## tests/api_internal/test_internal_api_call.py: ## @@ -0,0 +1,151 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license ag

[airflow] branch constraints-main updated: Updating constraints. Build id:

2022-12-01 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 39b66d5cd0 Updating con

[GitHub] [airflow] vincbeck commented on a diff in pull request #27892: [AIP-44] Add internal API definition.

2022-12-01 Thread GitBox
vincbeck commented on code in PR #27892: URL: https://github.com/apache/airflow/pull/27892#discussion_r1037258988 ## tests/api_internal/test_internal_api_call.py: ## @@ -0,0 +1,151 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license ag

[GitHub] [airflow] vincbeck commented on a diff in pull request #27892: [AIP-44] Add internal API definition.

2022-12-01 Thread GitBox
vincbeck commented on code in PR #27892: URL: https://github.com/apache/airflow/pull/27892#discussion_r1037257842 ## airflow/api_internal/openapi/internal_api_v1.yaml: ## @@ -0,0 +1,92 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor licens

[GitHub] [airflow] potiuk commented on issue #27978: KeyError: 0 error with common-sql version 1.3.0

2022-12-01 Thread GitBox
potiuk commented on issue #27978: URL: https://github.com/apache/airflow/issues/27978#issuecomment-1333943185 @park-peter - can you please test the new snowflake 4.0.2rc1 and report it in the issue here: https://github.com/apache/airflow/issues/27978 -- This is an automated messag

svn commit: r58382 - /dev/airflow/2.5.0rc3/

2022-12-01 Thread ephraimanierobi
Author: ephraimanierobi Date: Thu Dec 1 15:29:18 2022 New Revision: 58382 Log: Add artifacts for Airflow 2.5.0rc3 Added: dev/airflow/2.5.0rc3/ dev/airflow/2.5.0rc3/apache-airflow-2.5.0-source.tar.gz (with props) dev/airflow/2.5.0rc3/apache-airflow-2.5.0-source.tar.gz.asc dev/ai

[GitHub] [airflow] bbovenzi commented on a diff in pull request #27805: Automatically save and allow restore of recent DAG run configs

2022-12-01 Thread GitBox
bbovenzi commented on code in PR #27805: URL: https://github.com/apache/airflow/pull/27805#discussion_r1037236111 ## airflow/www/static/js/trigger.js: ## @@ -20,13 +20,32 @@ /* global document, CodeMirror, window */ const textArea = document.getElementById('json'); +const re

[GitHub] [airflow] mdering commented on issue #28023: Dataproc operators ClusterGenerator can't find parameter of ShieldedInstanceConfig

2022-12-01 Thread GitBox
mdering commented on issue #28023: URL: https://github.com/apache/airflow/issues/28023#issuecomment-1333893667 it looks like this can be specified in the `gce_cluster_config` field. referring to here https://cloud.google.com/python/docs/reference/dataproc/latest/google.cloud.dataproc_v1.typ

[GitHub] [airflow] IAL32 commented on pull request #28005: Fix GlueCrawlerOperature failure when using tags (apache#27556)

2022-12-01 Thread GitBox
IAL32 commented on PR #28005: URL: https://github.com/apache/airflow/pull/28005#issuecomment-1333888171 Changing tags however requires to know the crawler's ARN, which is only possible to obtain via concatenating the current account ID, the region name and the crawler's name. The GlueCra

[GitHub] [airflow] potiuk opened a new issue, #28036: Status of testing Providers that were prepared on December 01, 2022

2022-12-01 Thread GitBox
potiuk opened a new issue, #28036: URL: https://github.com/apache/airflow/issues/28036 ### Body I have a kind request for all the contributors to the latest provider packages release. Could you please help us to test the RC versions of the providers? Let us know in the comme

[GitHub] [airflow] romibuzi commented on pull request #27893: AWSGlueJobHook updates job configuration if it exists

2022-12-01 Thread GitBox
romibuzi commented on PR #27893: URL: https://github.com/apache/airflow/pull/27893#issuecomment-1333885503 @Taragolis and @syedahsn Thanks for your reviews! I have done modifications. Feel free to take a look at them :) -- This is an automated message from the Apache Git Service. To resp

[airflow-site] branch add-documentation-2022-12-01 created (now 37b361a95f)

2022-12-01 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch add-documentation-2022-12-01 in repository https://gitbox.apache.org/repos/asf/airflow-site.git at 37b361a95f Add documentation for packages - 2022-12-01 This branch includes the following

[GitHub] [airflow] IAL32 commented on pull request #28005: Fix GlueCrawlerOperature failure when using tags (apache#27556)

2022-12-01 Thread GitBox
IAL32 commented on PR #28005: URL: https://github.com/apache/airflow/pull/28005#issuecomment-1333883247 @Taragolis I have added several tests, and isolated tag updating by adding a `update_tags` method. This is because when calling `glue_client.get_crawler` the `Tags` is not present, eve

[airflow] annotated tag providers-snowflake/4.0.2rc1 updated (e9a9ae6540 -> 23edcbea85)

2022-12-01 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to annotated tag providers-snowflake/4.0.2rc1 in repository https://gitbox.apache.org/repos/asf/airflow.git *** WARNING: tag providers-snowflake/4.0.2rc1 was modified! *** from e9a9ae6540 (commit)

[airflow] annotated tag providers-exasol/4.1.2rc1 updated (e9a9ae6540 -> 5294e01095)

2022-12-01 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to annotated tag providers-exasol/4.1.2rc1 in repository https://gitbox.apache.org/repos/asf/airflow.git *** WARNING: tag providers-exasol/4.1.2rc1 was modified! *** from e9a9ae6540 (commit) to

[airflow] annotated tag providers-zendesk/4.2.0rc1 updated (e9a9ae6540 -> ea9dd6a613)

2022-12-01 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to annotated tag providers-zendesk/4.2.0rc1 in repository https://gitbox.apache.org/repos/asf/airflow.git *** WARNING: tag providers-zendesk/4.2.0rc1 was modified! *** from e9a9ae6540 (commit) t

[GitHub] [airflow] eladkal commented on issue #26865: logical_date

2022-12-01 Thread GitBox
eladkal commented on issue #26865: URL: https://github.com/apache/airflow/issues/26865#issuecomment-1333856389 No rush :) just checking. if you have questions feel free to join [Slack Chat](https://github.com/apache/airflow#links) there are several support channels for contributors. --

svn commit: r58381 - /dev/airflow/providers/

2022-12-01 Thread potiuk
Author: potiuk Date: Thu Dec 1 14:31:37 2022 New Revision: 58381 Log: Release Airflow Providers on 2022-12-01 - remove old files Removed: dev/airflow/providers/apache-airflow-providers-amazon-6.2.0.tar.gz dev/airflow/providers/apache-airflow-providers-amazon-6.2.0.tar.gz.asc dev/airf

[GitHub] [airflow] RagingPuppies opened a new pull request, #28034: Added custom environment variables to statsd

2022-12-01 Thread GitBox
RagingPuppies opened a new pull request, #28034: URL: https://github.com/apache/airflow/pull/28034 Hi, This is not related to any issue, all airflow components have the option to add custom env to their docker container except statsd, I've just added the same functionality and tested the

svn commit: r58380 - /dev/airflow/providers/

2022-12-01 Thread potiuk
Author: potiuk Date: Thu Dec 1 14:29:57 2022 New Revision: 58380 Log: Add artifacts for Airflow Providers 2022-12-01 Added: dev/airflow/providers/apache-airflow-providers-exasol-4.1.2.tar.gz (with props) dev/airflow/providers/apache-airflow-providers-exasol-4.1.2.tar.gz.asc dev/ai

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #28034: Added custom environment variables to statsd

2022-12-01 Thread GitBox
boring-cyborg[bot] commented on PR #28034: URL: https://github.com/apache/airflow/pull/28034#issuecomment-1333852450 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution

[airflow] branch main updated: Update documentation for ad-hoc provider releases in December (#28031)

2022-12-01 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new e9a9ae6540 Update documentation for ad-hoc provider

[GitHub] [airflow] potiuk merged pull request #28031: Update documentation for ad-hoc provider releases in December

2022-12-01 Thread GitBox
potiuk merged PR #28031: URL: https://github.com/apache/airflow/pull/28031 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[GitHub] [airflow] potiuk commented on pull request #28031: Update documentation for ad-hoc provider releases in December

2022-12-01 Thread GitBox
potiuk commented on PR #28031: URL: https://github.com/apache/airflow/pull/28031#issuecomment-1333847463 Tested docs building locally - merging to speed it up a bit -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

[GitHub] [airflow] mdering commented on pull request #27961: Fix: re-enable use of parameters in gcs_to_bq which had been disabled

2022-12-01 Thread GitBox
mdering commented on PR #27961: URL: https://github.com/apache/airflow/pull/27961#issuecomment-1333843465 since this passed CI and has unit tests, is there anything else I need to do to get this merged? -- This is an automated message from the Apache Git Service. To respond to the message

  1   2   >