ephraimbuddy commented on code in PR #30375:
URL: https://github.com/apache/airflow/pull/30375#discussion_r1161087583


##########
airflow/executors/celery_executor.py:
##########
@@ -414,23 +414,23 @@ def try_adopt_task_instances(self, tis: 
Sequence[TaskInstance]) -> Sequence[Task
 
         return not_adopted_tis
 
-    def cleanup_stuck_queued_tasks(self, tis: Sequence[TaskInstance]) -> None:
+    def cleanup_stuck_queued_task(self, ti: TaskInstance) -> None:
         """
         Handle remnants of tasks that were failed because they were stuck in 
queued.
         Tasks can get stuck in queued. If such a task is detected, it will be 
marked
         as `UP_FOR_RETRY` if the task instance has remaining retries or marked 
as `FAILED`
         if it doesn't.
-        :param tis: Task Instances to clean up
+        :param ti: Task Instance to clean up
         """
-        for ti in tis:
-            task_instance_key = ti.key
-            self.running.discard(task_instance_key)
-            celery_async_result = self.tasks.pop(task_instance_key, None)
-            if celery_async_result:
-                try:
-                    app.control.revoke(celery_async_result.task_id)
-                except Exception as ex:
-                    self.log.error("Error revoking task instance %s from 
celery: %s", task_instance_key, ex)
+        print(f"\n\nti: {type(ti)}\n\n")

Review Comment:
   ```suggestion
   ```
   I don't think we need this



##########
airflow/jobs/scheduler_job.py:
##########
@@ -1478,6 +1478,7 @@ def _fail_tasks_stuck_in_queued(self, session: Session = 
NEW_SESSION) -> None:
                             self._task_queued_timeout,
                         )
                         ti.handle_failure(error=msg, session=session)
+                        self.executor.cleanup_stuck_queued_task(ti)

Review Comment:
   Do we really need the ti.handle_failure? Won't it be handled in 
`Scheduler._process_executor_events`?
   
   Also, I think we shouldn't iterate the task instances here but send all to 
the executor, then have the executor iterate and remove them from the running. 
When we iterate here, we call the executor cleanup several times but we could 
call it once with the task instances and then have it remove them from the 
running tasks. Looks more efficient _I think_



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to