[GitHub] [spark] zhouyejoe commented on a diff in pull request #35906: [SPARK-33236][shuffle] Enable Push-based shuffle service to store state in NM level DB for work preserving restart

[GitBox]

zhouyejoe commented on code in PR #35906:
URL: https://github.com/apache/spark/pull/35906#discussion_r892921010


##########
common/network-yarn/src/main/java/org/apache/spark/network/yarn/YarnShuffleService.java:
##########
@@ -287,7 +301,13 @@ protected void serviceInit(Configuration externalConf) 
throws Exception {
   }
 
   @VisibleForTesting
-  static MergedShuffleFileManager 
newMergedShuffleFileManagerInstance(TransportConf conf) {
+  public void setShuffleMergeManager(MergedShuffleFileManager mergeManager) {
+    this.shuffleMergeManager = mergeManager;
+  }

Review Comment:
   Added the submitCleanupTask in RemoteBlockPushResolver, but the set method 
has to be added as there is no other way to customize the MergeShuffleManager 
during UT.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org