Over the last couple weeks I've come across a rather tricky problem a few
times. One DAG run gets "stuck" in the queued state, while subsequent DAG
runs will be stuck running (screenshot below). One of these issues was
caused by `max_active_runs` being met when a task instance from a
previously run DAG was cleared, and one of the tasks had
`depends_on_past=True`. This caused the DAG run to be stuck in queued in
perpetuity until it was realized that the task that wasn't getting
scheduled needed the failed task in the preceding DAG run to be re-run,
which in turn causes the stuck running DAG runs to be stuck in running.
which caused quite a bit of confusion and stress.

Given that Airflow is pretty burnt out on task instance states and colors,
I propose replacing "no_status" with "dependencies_not_met" and surfacing
dependencies in the grid view instead of forcing users to already know
where to look (i.e. "more details" task instance details). Now that I typed
it out, I'm not sure there should be a reason for the "more details" button
and not just laying out all of a task instance's details in the grid view
similar to how the graph and code views are now included in the grid view.

Anyway, I wanted to solicit feedback before I open an issue / start work on
this.

[image: image.png]

Reply via email to