karenbraganz commented on issue #51301: URL: https://github.com/apache/airflow/issues/51301#issuecomment-2946132269
@KarthikeyanDevendhiran Are you saying that the tasks remain stuck in the queued state despite workers being available? This should not be a common occurrence with the upgrade to 2.10.5 especially since [PR #43520](https://github.com/apache/airflow/pull/43520) was created to reduce the incidence of this issue. With this PR, tasks are requeued up to two times by default with the same try number once the `task_queued_timeout` is reached. I can look into this if you provide the following information: - Scheduler logs showing the lifecycle of a task instance that got stuck in queued and failed. Please filter the scheduler logs by adding the DAG ID, task ID, and run ID to your search query. - Data from the log table in the Airflow metadata DB for the same task instance that got stuck in queued and failed. You may use this SQL query to get this information: ``` SELECT * FROM log WHERE dag_id = '<fill-in-dag-id>' AND task_id = 'fill-in-task-id' AND run_id = '<fill-in-run-id>' ORDER BY dttm ASC; ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
