[
https://issues.apache.org/jira/browse/SPARK-27651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Zsolt Piros updated SPARK-27651:
---------------------------------------
Description: When a shuffle block (content) is fetched the network is
always used even when it is fetched from the external shuffle service running
on the same host. This can be avoided by getting the local directories of the
same host executors from the external shuffle service and accessing those
blocks from the disk directly. (was: When a shuffle block (content) is fetched
the network is always used even when it is fetched from an executor (or the
external shuffle service) running on the same host.)
> Avoid the network when block manager fetches shuffle blocks from the same host
> ------------------------------------------------------------------------------
>
> Key: SPARK-27651
> URL: https://issues.apache.org/jira/browse/SPARK-27651
> Project: Spark
> Issue Type: Improvement
> Components: Block Manager
> Affects Versions: 3.0.0
> Reporter: Attila Zsolt Piros
> Assignee: Attila Zsolt Piros
> Priority: Major
> Fix For: 3.0.0
>
>
> When a shuffle block (content) is fetched the network is always used even
> when it is fetched from the external shuffle service running on the same
> host. This can be avoided by getting the local directories of the same host
> executors from the external shuffle service and accessing those blocks from
> the disk directly.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
