Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/19631#discussion_r149459107
--- Diff: core/src/main/scala/org/apache/spark/SecurityManager.scala ---
@@ -542,7 +496,55 @@ private[spark] class SecurityManager(
* Gets the secret key.
* @return the secret key as a String if authentication is enabled,
otherwise returns null
*/
- def getSecretKey(): String = secretKey
+ def getSecretKey(): String = {
+ if (isAuthenticationEnabled) {
+ Option(sparkConf.getenv(ENV_AUTH_SECRET))
+ .orElse(sparkConf.getOption(SPARK_AUTH_SECRET_CONF))
+ .getOrElse {
+ throw new IllegalArgumentException(
+ s"A secret key must be specified via the
$SPARK_AUTH_SECRET_CONF config")
+ }
+ } else {
+ null
+ }
+ }
+
+ /**
+ * Initialize the configuration object held by this class for
authentication.
+ *
+ * If authentication is disabled, do nothing.
+ *
+ * In YARN mode, generate a secret key and store it in the configuration
object, setting it up to
+ * also be propagated to executors using an env variable.
+ *
+ * In other modes, assert that the auth secret is set in the
configuration.
+ */
+ def initializeAuth(): Unit = {
+ if (!sparkConf.get(NETWORK_AUTH_ENABLED)) {
+ return
+ }
+
+ if (sparkConf.get(SparkLauncher.SPARK_MASTER, null) != "yarn") {
+ require(sparkConf.contains(SPARK_AUTH_SECRET_CONF),
+ s"A secret key must be specified via the $SPARK_AUTH_SECRET_CONF
config.")
+ return
+ }
+
+ // In YARN, force creation of a new secret if this is client mode.
This ensures each
--- End diff --
I'm not changing the previous behavior for non-YARN. So my change shouldn't
make it easier nor harder to make things work for other cluster managers.
Whether it will work depends on how the auth secret is used in those cases.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
