Hello, I've run some benchmarks on the scheduler with my patch and without and shared the metrics on the PR.
https://github.com/apache/airflow/pull/54103#issuecomment-3275278733 The improvements in the scheduler's throughput and performance are quite noticeable. Regards, Christos On Fri, Aug 15, 2025 at 11:07 AM Christos Bisias <[email protected]> wrote: > Hello, > > The patches shared on this thread are trying to address task starvation > issues. I think that my patch isn't comparable to them because it's > addressing how many tasks are picked related to predefined limits and not > which ones or in which order. > > I adjusted the PR and took it out of draft. I made sure that the current > behavior hasn't been changed. All the scheduler tests are passing without > modifications. I've also updated the PR description to summarize the issue > and the solution. > > If you check the unit test that I have added, you will see that we don't > need huge dags in order to observe the behavior that I'm trying to > deal with. > > Please take a look and let me know what you think. > > https://github.com/apache/airflow/pull/54103 > > Thank you, > Christos > > On Fri, Aug 8, 2025 at 10:23 PM Daniel Standish > <[email protected]> wrote: > >> And the diff looks worse than it is because I unindent a big section which >> was previously in the aforementioned loop. >> >
