tgravescs commented on issue #27207: [WIP][SPARK-18886][CORE] Make Locality wait time measure resource under utilization due to delay scheduling. URL: https://github.com/apache/spark/pull/27207#issuecomment-576869577 yes its fine not to do any minor changes until we decide on design. I'm not so worried about the 1 executor case if the first taskset takes it because that one should be higher priority and that should work as expected, when there isn't anywhere to put the tasks, then have them wait a bit to try to get locality. Its the case with the fair scheduler where the one that isn't the highest priority that I'm more concerned with. I wonder if we can add in a separate tracking/check in TaskSchedulerImpl that tracks to see if it keeps rejecting it on the non all resource offers but then resets on the all offers, then after some number of those we stop resetting it, thoughts on that?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
