XD-DENG commented on a change in pull request #4919: [AIRFLOW-4093] Throw
exception if job failed or cancelled or retry too many times
URL: https://github.com/apache/airflow/pull/4919#discussion_r267606803
##########
File path: airflow/contrib/operators/aws_athena_operator.py
##########
@@ -74,7 +78,17 @@ def execute(self, context):
self.result_configuration['OutputLocation'] = self.output_location
self.query_execution_id = self.hook.run_query(self.query,
self.query_execution_context,
self.result_configuration, self.client_request_token)
- self.hook.poll_query_status(self.query_execution_id)
+ query_status = self.hook.poll_query_status(self.query_execution_id,
self.max_tries)
+
+ if query_status in AWSAthenaHook.FAILURE_STATES:
+ raise Exception(
+ 'Final state of Athena job is {}, query_execution_id is {}.'
+ .format(query_status, self.query_execution_id))
+ elif not query_status or query_status in
AWSAthenaHook.INTERMEDIATE_STATES:
+ raise Exception(
+ 'Final state of Athena job is {}. \
+ Max tries of poll status exceeded, query_execution_id is {}.'
Review comment:
Hi @bryanyang0528 This line-breaking here will cause minor display issue.
You can try to run
```python
raise Exception(
'Final state of Athena job is {}. \
Max tries of poll status exceeded, query_execution_id is {}.'
.format("AAA", "BBB"))
```
What you will see is
```
Exception: Final state of Athena job is AAA. Max tries of poll status
exceeded, query_execution_id is **BBB.**
```
in which we have unnecessary extra spaces between the two sentences.
Please change how you break the line. Thanks.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
