XD-DENG commented on a change in pull request #4919: [AIRFLOW-4093] Throw exception if job failed or cancelled or retry too many times URL: https://github.com/apache/airflow/pull/4919#discussion_r268396887
########## File path: airflow/contrib/operators/aws_athena_operator.py ########## @@ -74,7 +78,17 @@ def execute(self, context): self.result_configuration['OutputLocation'] = self.output_location self.query_execution_id = self.hook.run_query(self.query, self.query_execution_context, self.result_configuration, self.client_request_token) - self.hook.poll_query_status(self.query_execution_id) + query_status = self.hook.poll_query_status(self.query_execution_id, self.max_tries) + + if query_status in AWSAthenaHook.FAILURE_STATES: + raise Exception( + 'Final state of Athena job is {}, query_execution_id is {}.' + .format(query_status, self.query_execution_id)) + elif not query_status or query_status in AWSAthenaHook.INTERMEDIATE_STATES: + raise Exception( + 'Final state of Athena job is {}. \ + Max tries of poll status exceeded, query_execution_id is {}.' Review comment: Kindly allow me a bit more time to have a closer look at your test update before I merge. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services