[PR] Implements `AwsBaseOperator` and `AwsBaseSensor` [airflow]

2023-10-05 Thread via GitHub
Taragolis opened a new pull request, #34784: URL: https://github.com/apache/airflow/pull/34784 Continuation #24246 This base operators and sensors might help to resolve inconsistency in multiple operators and sensors: 1. Some of current sensors/operators accept only

Re: [PR] Implements `AwsBaseOperator` and `AwsBaseSensor` [airflow]

2023-10-10 Thread via GitHub
Taragolis commented on PR #34784: URL: https://github.com/apache/airflow/pull/34784#issuecomment-1755464358 In [126a1f5](https://github.com/apache/airflow/pull/34784/commits/126a1f50178f1f9d2c603a08a483952bf126520e) split logic into two classes, for reduce repeatable code in `AwsBaseOperato

Re: [PR] Implements `AwsBaseOperator` and `AwsBaseSensor` [airflow]

2023-10-10 Thread via GitHub
o-nikolas commented on code in PR #34784: URL: https://github.com/apache/airflow/pull/34784#discussion_r1353229032 ## airflow/providers/amazon/aws/operators/base_aws.py: ## Review Comment: > It is also might be a good idea to make this classes internal only. +1 --

Re: [PR] Implements `AwsBaseOperator` and `AwsBaseSensor` [airflow]

2023-10-10 Thread via GitHub
o-nikolas commented on code in PR #34784: URL: https://github.com/apache/airflow/pull/34784#discussion_r1353234785 ## airflow/providers/amazon/aws/operators/base_aws.py: ## @@ -0,0 +1,150 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor lic

Re: [PR] Implements `AwsBaseOperator` and `AwsBaseSensor` [airflow]

2023-10-10 Thread via GitHub
Taragolis commented on code in PR #34784: URL: https://github.com/apache/airflow/pull/34784#discussion_r1353460934 ## airflow/providers/amazon/aws/operators/base_aws.py: ## @@ -0,0 +1,150 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor lic

Re: [PR] Implements `AwsBaseOperator` and `AwsBaseSensor` [airflow]

2023-10-10 Thread via GitHub
Taragolis commented on code in PR #34784: URL: https://github.com/apache/airflow/pull/34784#discussion_r1353473369 ## airflow/providers/amazon/aws/operators/base_aws.py: ## @@ -0,0 +1,97 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor lice

Re: [PR] Implements `AwsBaseOperator` and `AwsBaseSensor` [airflow]

2023-10-11 Thread via GitHub
ferruzzi merged PR #34784: URL: https://github.com/apache/airflow/pull/34784 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow

Re: [PR] Implements `AwsBaseOperator` and `AwsBaseSensor` [airflow]

2023-10-05 Thread via GitHub
Taragolis commented on code in PR #34784: URL: https://github.com/apache/airflow/pull/34784#discussion_r1347868314 ## airflow/providers/amazon/aws/operators/base_aws.py: ## Review Comment: It is also might be a good idea to make this classes internal only. -- This is a

Re: [PR] Implements `AwsBaseOperator` and `AwsBaseSensor` [airflow]

2023-10-05 Thread via GitHub
Taragolis commented on code in PR #34784: URL: https://github.com/apache/airflow/pull/34784#discussion_r1347878899 ## airflow/providers/amazon/aws/operators/base_aws.py: ## @@ -0,0 +1,150 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor lic

Re: [PR] Implements `AwsBaseOperator` and `AwsBaseSensor` [airflow]

2023-10-05 Thread via GitHub
ferruzzi commented on code in PR #34784: URL: https://github.com/apache/airflow/pull/34784#discussion_r1347884136 ## airflow/providers/amazon/aws/operators/base_aws.py: ## @@ -0,0 +1,150 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor lice

Re: [PR] Implements `AwsBaseOperator` and `AwsBaseSensor` [airflow]

2023-10-05 Thread via GitHub
ferruzzi commented on code in PR #34784: URL: https://github.com/apache/airflow/pull/34784#discussion_r1347889605 ## airflow/providers/amazon/aws/sensors/base_aws.py: ## @@ -0,0 +1,145 @@ +# Licensed to the Apache Software Foundation (ASF) under one Review Comment: All of th

Re: [PR] Implements `AwsBaseOperator` and `AwsBaseSensor` [airflow]

2023-10-05 Thread via GitHub
ferruzzi commented on code in PR #34784: URL: https://github.com/apache/airflow/pull/34784#discussion_r1347898473 ## airflow/providers/amazon/aws/sensors/base_aws.py: ## @@ -0,0 +1,145 @@ +# Licensed to the Apache Software Foundation (ASF) under one Review Comment: Could we

Re: [PR] Implements `AwsBaseOperator` and `AwsBaseSensor` [airflow]

2023-10-05 Thread via GitHub
ferruzzi commented on code in PR #34784: URL: https://github.com/apache/airflow/pull/34784#discussion_r1347884136 ## airflow/providers/amazon/aws/operators/base_aws.py: ## @@ -0,0 +1,150 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor lice

Re: [PR] Implements `AwsBaseOperator` and `AwsBaseSensor` [airflow]

2023-10-05 Thread via GitHub
Taragolis commented on code in PR #34784: URL: https://github.com/apache/airflow/pull/34784#discussion_r1347948429 ## airflow/providers/amazon/aws/sensors/base_aws.py: ## @@ -0,0 +1,145 @@ +# Licensed to the Apache Software Foundation (ASF) under one Review Comment: This mig

Re: [PR] Implements `AwsBaseOperator` and `AwsBaseSensor` [airflow]

2023-10-05 Thread via GitHub
Taragolis commented on code in PR #34784: URL: https://github.com/apache/airflow/pull/34784#discussion_r1347986606 ## airflow/providers/amazon/aws/sensors/base_aws.py: ## @@ -0,0 +1,145 @@ +# Licensed to the Apache Software Foundation (ASF) under one Review Comment: Just for

Re: [PR] Implements `AwsBaseOperator` and `AwsBaseSensor` [airflow]

2023-10-05 Thread via GitHub
Taragolis commented on code in PR #34784: URL: https://github.com/apache/airflow/pull/34784#discussion_r1347991914 ## airflow/providers/amazon/aws/operators/base_aws.py: ## Review Comment: And wondering about module naming because I a bit confused: Suggestion in old

Re: [PR] Implements `AwsBaseOperator` and `AwsBaseSensor` [airflow]

2023-10-05 Thread via GitHub
Taragolis commented on code in PR #34784: URL: https://github.com/apache/airflow/pull/34784#discussion_r1348066197 ## airflow/providers/amazon/aws/hooks/base_aws.py: ## @@ -463,15 +463,17 @@ def __init__( region_name: str | None = None, client_type: str | None

Re: [PR] Implements `AwsBaseOperator` and `AwsBaseSensor` [airflow]

2023-10-05 Thread via GitHub
ferruzzi commented on code in PR #34784: URL: https://github.com/apache/airflow/pull/34784#discussion_r1348077800 ## airflow/providers/amazon/aws/sensors/base_aws.py: ## @@ -0,0 +1,145 @@ +# Licensed to the Apache Software Foundation (ASF) under one Review Comment: Yeah, I t

Re: [PR] Implements `AwsBaseOperator` and `AwsBaseSensor` [airflow]

2023-10-05 Thread via GitHub
eladkal commented on code in PR #34784: URL: https://github.com/apache/airflow/pull/34784#discussion_r1348316001 ## tests/always/test_project_structure.py: ## @@ -403,6 +403,9 @@ class TestAmazonProviderProjectStructure(ExampleCoverageTest): "airflow.providers.amazon.

Re: [PR] Implements `AwsBaseOperator` and `AwsBaseSensor` [airflow]

2023-10-06 Thread via GitHub
Taragolis commented on code in PR #34784: URL: https://github.com/apache/airflow/pull/34784#discussion_r1348641033 ## tests/always/test_project_structure.py: ## @@ -403,6 +403,9 @@ class TestAmazonProviderProjectStructure(ExampleCoverageTest): "airflow.providers.amazo

Re: [PR] Implements `AwsBaseOperator` and `AwsBaseSensor` [airflow]

2023-10-06 Thread via GitHub
Taragolis commented on PR #34784: URL: https://github.com/apache/airflow/pull/34784#issuecomment-1750665063 >I assume followup PR would be to change AWS operators/hooks to inherit from the base classes? Yep > We need also doc update to explain how to set the botocore parameters

Re: [PR] Implements `AwsBaseOperator` and `AwsBaseSensor` [airflow]

2023-10-06 Thread via GitHub
vincbeck commented on code in PR #34784: URL: https://github.com/apache/airflow/pull/34784#discussion_r1348720686 ## airflow/providers/amazon/aws/operators/base_aws.py: ## @@ -0,0 +1,152 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor lice

Re: [PR] Implements `AwsBaseOperator` and `AwsBaseSensor` [airflow]

2023-10-06 Thread via GitHub
Taragolis commented on code in PR #34784: URL: https://github.com/apache/airflow/pull/34784#discussion_r1348727907 ## airflow/providers/amazon/aws/operators/base_aws.py: ## @@ -0,0 +1,152 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor lic

Re: [PR] Implements `AwsBaseOperator` and `AwsBaseSensor` [airflow]

2023-10-06 Thread via GitHub
Taragolis commented on code in PR #34784: URL: https://github.com/apache/airflow/pull/34784#discussion_r1348730693 ## airflow/providers/amazon/aws/operators/base_aws.py: ## @@ -0,0 +1,152 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor lic

Re: [PR] Implements `AwsBaseOperator` and `AwsBaseSensor` [airflow]

2023-10-06 Thread via GitHub
vincbeck commented on code in PR #34784: URL: https://github.com/apache/airflow/pull/34784#discussion_r1348746185 ## airflow/providers/amazon/aws/operators/base_aws.py: ## @@ -0,0 +1,152 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor lice

Re: [PR] Implements `AwsBaseOperator` and `AwsBaseSensor` [airflow]

2023-10-06 Thread via GitHub
vincbeck commented on code in PR #34784: URL: https://github.com/apache/airflow/pull/34784#discussion_r1348746923 ## airflow/providers/amazon/aws/operators/base_aws.py: ## @@ -0,0 +1,152 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor lice

Re: [PR] Implements `AwsBaseOperator` and `AwsBaseSensor` [airflow]

2023-10-06 Thread via GitHub
ferruzzi commented on code in PR #34784: URL: https://github.com/apache/airflow/pull/34784#discussion_r1348927935 ## airflow/providers/amazon/aws/operators/base_aws.py: ## @@ -0,0 +1,152 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor lice