jstern commented on a change in pull request #7187: [AIRFLOW-6576] fix
scheduler crash caused by deleted task with sla misses
URL: https://github.com/apache/airflow/pull/7187#discussion_r368011208
##########
File path: airflow/jobs/scheduler_job.py
##########
@@ -429,7 +429,15 @@ def manage_slas(self, dag, session=None):
""".format(task_list=task_list,
blocking_task_list=blocking_task_list,
bug=asciiart.bug)
- tasks_missed_sla = [dag.get_task(sla.task_id) for sla in slas]
+ tasks_missed_sla = []
+ for sla in slas:
+ try:
+ task = dag.get_task(sla.task_id)
+ except AirflowException:
+ # task already deleted from DAG, skip it
Review comment:
I was going to suggest maybe catching a more specific exception subclass
(like `AirflowNotFoundException`, `TaskNotFound`, or `TaskInstanceNotFound` as
appropriate), but it looks like we really do just throw `AirflowException` in
the method:
https://github.com/apache/airflow/blob/master/airflow/models/dag.py#L1214.
Changing the exception that `get_task` method raises might be out of scope for
this PR and have other unanticipated consequences, but it's something to think
about...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services