[ https://issues.apache.org/jira/browse/SPARK-27651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17052348#comment-17052348 ]
Attila Zsolt Piros commented on SPARK-27651: -------------------------------------------- well in case of dynamic allocation and a recalculation the executors could be already gone. > Avoid the network when block manager fetches shuffle blocks from the same host > ------------------------------------------------------------------------------ > > Key: SPARK-27651 > URL: https://issues.apache.org/jira/browse/SPARK-27651 > Project: Spark > Issue Type: Improvement > Components: Block Manager > Affects Versions: 3.0.0 > Reporter: Attila Zsolt Piros > Assignee: Attila Zsolt Piros > Priority: Major > Fix For: 3.0.0 > > > When a shuffle block (content) is fetched the network is always used even > when it is fetched from the external shuffle service running on the same > host. This can be avoided by getting the local directories of the same host > executors from the external shuffle service and accessing those blocks from > the disk directly. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org