jerrypeng commented on code in PR #57286:
URL: https://github.com/apache/spark/pull/57286#discussion_r3590375598
##########
core/src/main/scala/org/apache/spark/storage/BlockManager.scala:
##########
@@ -197,12 +196,16 @@ private[spark] class BlockManager(
// We initialize the ShuffleManager later in SparkContext and Executor, to
allow
// user jars to define custom ShuffleManagers, as such `_shuffleManager`
will be null here
// (except for tests) and we ask for the instance from the SparkEnv.
+ // The default shuffle manager, used here only for block-by-id resolution
+ // (`shuffleBlockResolver`). A pipelined shuffle is served out-of-band by
the incremental manager
+ // and produces no block-manager-addressed blocks, so these paths only ever
resolve regular
+ // shuffles and correctly use the default manager
(SparkEnv.defaultShuffleManager).
private lazy val shuffleManager = {
Option(_shuffleManager).getOrElse {
// Wait for ShuffleManager to be initialized before handling shuffle
operations.
// Exception will be thrown if it is not initialized within the
configured timeout.
waitForShuffleManagerInit()
Review Comment:
How does the blockmanager interact with the shuffle manager?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
