safaehar commented on PR #75216: URL: https://github.com/apache/airflow/pull/75216#issuecomment-6641641030
Thanks for flagging this. I addressed the unfiltered scheduler path separately: - PostgreSQL and SQLite now use the original unjoined `task_instance` aggregation when `Pool.slots_stats()` is called without `pool_names`. - MySQL retains the pool-first join because it produces the better plan there. - Filtered pool-list requests retain the joined query and both pool predicates. I repeated the PostgreSQL query benchmark with 1,000,000 task rows, 102 pools, and the same active states. These are medians from 20 warmed, alternating runs in the same environment. The new commit restores the same aggregation shape used before the PR: | Active task rows | Current PR before fix (joined) | New commit / pre-PR query shape | Improvement | Control | |---:|---:|---:|---:|---:| | 10,000 | 2.725 ms | 2.144 ms | 21.3% | 30.427 ms | | 188,200 | 49.449 ms | 41.185 ms | 16.7% | 30.814 ms | | 901,000 | 81.399 ms | 66.160 ms | 18.7% | 32.096 ms | Absolute timings differ from your environment, but the largest case is close to the reported `82.46 ms` regression and the restored result is close to the reported pre-PR `64.94 ms`. This measures the pool-statistics query rather than the full scheduler queueing section. I also added a regression test that asserts the selected unfiltered query shape for each backend, and verified the affected behavior on SQLite, PostgreSQL, and MySQL. --- Drafted-by: Codex (GPT-5); reviewed by @safaehar before posting -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
