Hi, While trying to understand the relationship of BlockManager and ShuffleManager I found that ShuffleManager is used for shuffle block data [1] (and that makes sense).
What I found quite surprising is that BlockManager can call getLocalBytes for non-shuffle blocks that in turn does...fetching shuffle blocks (!) [2]. That begs the question, who could want this? In other words, what other components would want to call BlockManager.getLocalBytes for shuffle blocks? The only caller is TorrentBroadcast which is not for shuffle blocks, is it? In other words, getLocalBytes should NOT be bothering itself with fetching local shuffle blocks as that's a responsibility of someone else, i.e. ShuffleManager. Is this correct? I'd appreciate any help you can provide so I can understand the Spark core better. Thanks! [1] https://github.com/apache/spark/blob/31734399d57f3c128e66b0f97ef83eb4c9165978/core/src/main/scala/org/apache/spark/storage/BlockManager.scala#L382 [2] https://github.com/apache/spark/blob/31734399d57f3c128e66b0f97ef83eb4c9165978/core/src/main/scala/org/apache/spark/storage/BlockManager.scala#L637 Pozdrawiam, Jacek Laskowski ---- https://about.me/JacekLaskowski "The Internals Of" Online Books <https://books.japila.pl/> Follow me on https://twitter.com/jaceklaskowski <https://twitter.com/jaceklaskowski>