[
https://issues.apache.org/jira/browse/SPARK-33332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17231010#comment-17231010
]
Surbhi Aggarwal commented on SPARK-33332:
-----------------------------------------
Hey guys,
I tried a temp fix, and it worked. Help me understand if this should actually
be the right fix
1. Populate the key "spark.authenticate.secret.file" value in the environment
variables for the executor here
https://github.com/apache/spark/blob/master/resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/BasicExecutorFeatureStep.scala#L129
2. When getting the token from secret file, also lookup secret file path from
environment variables if not found in the spark conf.
https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/SecurityManager.scala#L355
> Errors in running spark job on K8 with RPC Authentication Secret File
> properties
> --------------------------------------------------------------------------------
>
> Key: SPARK-33332
> URL: https://issues.apache.org/jira/browse/SPARK-33332
> Project: Spark
> Issue Type: Bug
> Components: Kubernetes
> Affects Versions: 3.0.0
> Reporter: Surbhi Aggarwal
> Priority: Major
>
> I am running a spark job on Kubernetes with RPC authentication feature. First
> i am mounting a K8 secret to both driver and executor to a path and providing
> the same path for Auth configuration. Below is the spark-conf i am passing
> related to RPC Auth.
> {code:java}
> --conf spark.authenticate=true
> --conf spark.kubernetes.driver.secrets.spark-secret-sa=/tmp/secrets
> --conf spark.kubernetes.executor.secrets.spark-secret-sa=/tmp/secrets
> --conf spark.authenticate.secret.driver.file=/tmp/secrets/token
> --conf spark.authenticate.secret.executor.file=/tmp/secrets/token
> {code}
>
> Mounting of secret is successfully done on both driver and executor pods and
> i can see the token file written to the containers. Driver is able to load
> the secret as well from the file but executor pods are failing to load the
> secret from file, and fails with the below exception:
> {code:java}
> java.lang.IllegalArgumentException: A secret key must be specified via the
> spark.authenticate.secret config
> at
> org.apache.spark.SecurityManager.$anonfun$getSecretKey$6(SecurityManager.scala:298)
> at scala.Option.getOrElse(Option.scala:189)
> at
> org.apache.spark.SecurityManager.getSecretKey(SecurityManager.scala:297){code}
>
> The value of the spark.authenticate.secret.executor.file property is being
> passed in SPARK_JAVA_OPS
> SPARK_JAVA_OPT_8: -Dspark.authenticate.secret.driver.file=/tmp/secrets/token
> SPARK_JAVA_OPT_9:
> -Dspark.authenticate.secret.executor.file=/tmp/secrets/token
> I am suspecting that values from java options are not being read into the
> spark conf and ultimately leading to failure.
>
> I am not sure if its a bug or a mis-configuration on my end. Amy help is
> greatly appreciated.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
