[GitHub] [spark] otterc commented on a diff in pull request #40843: [SPARK-43179][SHUFFLE] Allowing apps to control whether their metadata gets saved in the db by the External Shuffle Service

[via GitHub]

otterc commented on code in PR #40843:
URL: https://github.com/apache/spark/pull/40843#discussion_r1172930741


##########
core/src/main/scala/org/apache/spark/internal/config/package.scala:
##########
@@ -2507,4 +2507,13 @@ package object config {
       .version("3.5.0")
       .intConf
       .createWithDefault(Int.MaxValue)
+
+  private[spark] val SHUFFLE_SERVER_RECOVERY_DISABLED =
+    ConfigBuilder("spark.shuffle.server.recovery.disabled")
+      .internal()
+      .doc("Set to true for applications that prefer to disable recovery when 
the External " +
+        "Shuffle Service restarts.")
+      .version("3.5.0")
+      .booleanConf
+      .createWithDefault(false)

Review Comment:
   - I moved the documentation for this config to `running-on-yarn.md`.
   - In the security doc, I moved the definition under `Yarn` section. 
   PTAL



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org