Asquator commented on PR #54284: URL: https://github.com/apache/airflow/pull/54284#issuecomment-3170774646
> @dstandish wont this basically break priorities? As now lower priority tasks will be queued before higher priority tasks? This could be a great addition if it is the last field of the order by function but not the first. > > Maybe I am wrong but I think that here it will break priorities. If we add it as the last field in the order, https://github.com/apache/airflow/issues/45636 won't be solved because the starved prioritized tasks will ALWAYS win in the order. To avoid this, we have to partition by pool and compute last_scheduling decision for EVERY pool separately. In the same way, we can compute it for every DAG , every DAG run, and every task run (a run of a task in DAG run). After we do this, The query can be a 4-way group by each of these objects, selection of max_tis from every group, and then a global selection of max_tis. I think we don't need WFs here, do we? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
