dazza-codes commented on a change in pull request #6811: [RFC][AIRFLOW-6245] Add custom waiters for AWS batch jobs URL: https://github.com/apache/airflow/pull/6811#discussion_r360728235
########## File path: airflow/providers/amazon/aws/operators/batch.py ########## @@ -19,114 +19,77 @@ # """ -Airflow operator for AWS batch service +An Airflow operator for AWS batch services -.. seealso:: http://boto3.readthedocs.io/en/latest/reference/services/batch.html -""" - -import sys -from random import randint -from time import sleep -from typing import Optional +.. seealso:: -import botocore.exceptions -import botocore.waiter + - http://boto3.readthedocs.io/en/latest/guide/configuration.html + - http://boto3.readthedocs.io/en/latest/reference/services/batch.html + - https://docs.aws.amazon.com/batch/latest/APIReference/Welcome.html +""" -from airflow.contrib.hooks.aws_hook import AwsHook from airflow.exceptions import AirflowException from airflow.models import BaseOperator -from airflow.typing_compat import Protocol +from airflow.providers.amazon.aws.hooks.batch_client import AwsBatchClient from airflow.utils.decorators import apply_defaults -# pylint: disable=invalid-name, unused-argument - -class BatchProtocol(Protocol): Review comment: This PR should have backward-compatible changes, although some code has moved with AIP-21 and this typing-extension protocol was overlooked in those changes. AFAIK, the `BatchProtocol` is only a typing extension that is not actually used in "consumer land", but while I considered this, I changed things up to be consistent with AIP-21 and renamed it, along with updates to the `UPDATING.md`. I hope that resolves this point. In general, most of the refactoring in this PR is essentially `extract-class` and `move-class` refactoring and it applied to previously private-methods that should all continue to behave the same way they did before (only the OOP design and packaging are improved and explicit now). Sorry if that wasn't clear from the description. (Sometimes scrolling through a github web page is not the best way to do a review, because it only offers piecemeal changes while scrolling down the web page; it's a good interface but still leaves a lot to be desired.) ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
