jerrypeng commented on code in PR #57286: URL: https://github.com/apache/spark/pull/57286#discussion_r3590369025
########## core/src/main/scala/org/apache/spark/storage/BlockManagerMasterEndpoint.scala: ########## @@ -63,7 +63,12 @@ class BlockManagerMasterEndpoint( // We initialize the ShuffleManager later in SparkContext and Executor, to allow // user jars to define custom ShuffleManagers, as such `_shuffleManager` will be null here // (except for tests) and we ask for the instance from the SparkEnv. - private lazy val shuffleManager = Option(_shuffleManager).getOrElse(SparkEnv.get.shuffleManager) + // The default shuffle manager, used only for block-by-id resolution during + // external-shuffle-service cleanup. Pipelined shuffles are served out-of-band (no block-manager + // blocks, and not tracked in the MapOutputTracker this loop iterates), so this path only + // resolves regular shuffles. + private lazy val shuffleManager = Review Comment: Why does the BlockManagerMasterEndpoint.scala need the shuffleManager? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
