mridulm commented on PR #43280: URL: https://github.com/apache/spark/pull/43280#issuecomment-1753930980
Btw, it is not very clear to me how this functionality is going to be leveraged - unlers you are relying on resource profiles to tag which executors should leverage reliable shuffle and which should not. Spark will run tasks across stages on an executor - so there is no expectation of executor id to stage mapping (except via resource profiles). Please add more details in the PR description on how this can be leveraged reliably. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
