[ https://issues.apache.org/jira/browse/SPARK-26288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16711068#comment-16711068 ]
Apache Spark commented on SPARK-26288: -------------------------------------- User 'weixiuli' has created a pull request for this issue: https://github.com/apache/spark/pull/23243 > add initRegisteredExecutorsDB in ExternalShuffleService > ------------------------------------------------------- > > Key: SPARK-26288 > URL: https://issues.apache.org/jira/browse/SPARK-26288 > Project: Spark > Issue Type: New Feature > Components: Kubernetes, Shuffle > Affects Versions: 2.4.0 > Reporter: weixiuli > Priority: Major > Fix For: 2.4.0 > > > As we all know that spark on Yarn uses DB to record RegisteredExecutors > information, when the ExternalShuffleService restart and it can be reloaded, > which will be used as well . > While neither spark's standalone nor spark on k8s can record it's > RegisteredExecutors information by db or others ,so when > ExternalShuffleService restart ,which RegisteredExecutors information will be > lost,it is't what we looking forward to . > This commit add initRegisteredExecutorsDB which can be used either spark > standalone or spark on k8s to record RegisteredExecutors information , when > the ExternalShuffleService restart and it can be reloaded, which will be used > as well . -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org