Hey everyone, I'm trying to get reviewers for the Stage Level Scheduling pull requests. I was hoping to get this into Spark 3.0.The code is mostly complete - its just missing the webUI and final doc changes. If anyone has time, reviews from committers would be appreciated. You can find information about the overall feature and design here:[SPARK-27495] SPIP: Support Stage level resource configuration and scheduling - ASF JIRA it has Stage Level Scheduling SPIP Appendices API/Designattached with a high level overview.
I have a reference pr with most of the code implemented: [WIP][SPARK-27495][Core][YARN][k8s] Stage Level Scheduling code for reference by tgravescs · Pull Request #27053 · apache/spark and I've been trying to break that into smaller pieces for easier review - this is the current one: [SPARK-29306][CORE] Stage Level Sched: Executors need to track what ResourceProfile they are created with by tgravescs · Pull Request #26682 · apache/spark Regards,Tom Graves
