Congrats on landing this, Andrew and Teghveer! Per-instance reporting is a real gap closed.
However, while building the UI for the new fields I hit one case worth fixing before this ships (not yet released) and also worth calling out in this thread. "Degraded means at least one instance up and at least one down" needs a set of instances that *should* be up, and the job table cannot supply one: end_date is only written from inside the replica's own process, so a SIGKILL or OOM kill leaves a row with end_date NULL and state 'running' forever. Kill a scheduler, start a fresh one, and you get permanent "degraded" on a healthy deployment. There is no way to differentiate this row from a replica that genuinely is down. The difference is intent, which the table never records. Configuration does record intent, so I've taken the denominator from there: - Dag processor: the bundles in [dag_processor] dag_bundle_config_list (no --bundle-name covers all of them). Degraded when some bundles have a live processor and some don't. - Triggerer: under [core] multi_team, the team scopes those bundles declare, since a triggerer only gets its own team's triggers. - Scheduler: symmetric, no partitioned work, so healthy or down only. The issue with this solution is it does not take into consideration multi vertical component environment. Let's take an example. If your environment is running 2 schedulers and one is down, the health dashboard will return "healthy" because you have one scheduler which "cover" all your Dags. But again, I could not find a solution to detect a job (scheduler in this example) that is down by accident VS intentionnaly. Though, you can use `instances` for that which list all the schedulers up and running and act on it (expected VS running). PR: https://github.com/apache/airflow/pull/73222 Let me know what you think :) Vincent On 2026/08/30 01:01:08 Jung-Hyun Andrew Kim wrote: > Hey everyone, > > Teghveer (Tegh25) and I are MLH (Major League Hacking) Fellows working on > Airflow, and we just merged the health endpoint improvements discussed in > this thread > <https://lists.apache.org/thread/l1b1wkxs69omjsb8cwdryr2oydjmn3yg>, > implemented in PR #70416 > <https://github.com/apache/airflow/pull/70416#event-30164756803>. > > Previously, the /health endpoint reported a single aggregated status per > component, driven by the health of just one instance, so a distributed > component with a mix of healthy and unhealthy would only show healthy > instead of reporting something more meaningful. This PR replaces that with > per-instance reporting rolled up into a three-tier status: healthy → > degraded → down, where degraded means at least one instance is up and at > least one is down. > > Each instance now reports additional context relevant to its component type > (host_name, team_name, bundle_name, etc.), and we added team_name to > triggerer jobs and bundle_name to DAG processor jobs on the Job table via a > new migration. And test cases were updated! > > Thanks, > > Andrew Kim > > (PS Big shoutout to Dennis for all his review, help, and mentorship > throughout the entire fellowship! He was a huge reason we could make > meaningful contributions this summer and the best, coolest mentor we could > ask for. Thank you Dennis!) > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
