Thanks for the write-up! I made a couple of comments in the document.
On 24.01.26 01:41, asquator wrote:
Dear Airflow community, Informing you that the AIP has been recently updated to include summary of our research in eliminating scheduler starvation. After initial feedback, more exploration into refactoring priority weights has been done and documented. Some issues pointed out by @Jarek Potiuk and @Ash Berlin-Taylor were addressed. Brief summary of additions: - Clarified terminology - Notion of priority weights use cases - Theoretical foundation of pessimistic/hybrid scheduling in Airflow - Detailed 3 alternatives for refactoring priority weights in the context of scheduling - Addressed migrations At this stage, we're looking to get more feedback and brainstorm from scheduler developers and large-scale users which have immediate impact on the research. https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-100+Eliminate+Scheduler+Starvation+On+Concurrency+Limits --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
--------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
