mridulm commented on PR #40307: URL: https://github.com/apache/spark/pull/40307#issuecomment-1458674205
@jerqi Agree that we should have a way to specify locality preference for disaggregated shuffle implementations to spark scheduler - so that shuffle tasks are closer to the data. @otterc is relooking at SPARK-25299, given the platform changes that have gone in since it was proposed - and propose changes in the upcoming months. We can add this to that discussion/jira. Hope that sounds fine. For now, as @pan3793 suggested, turning off reduce locality for Uniffle should remove the need for the code patch (once this PR is merged). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
