Hi Sudhir, > On 21 Jan 2021, at 16:24, Sudhir Babu Pothineni <sbpothin...@gmail.com> wrote: > > Any other insights into this issue? I tried multiple way to supply keytab to > executor > > Does spark standalone doesn’t support Kerberos?
Spark standalone mode does not support Kerberos authentication. Related source code: https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala#L346 // Kerberos is not supported in standalone mode, and keytab support is not yet available // in Mesos cluster mode. if (clusterManager != STANDALONE && !isMesosCluster && args.principal != null && args.keytab != null) { // If client mode, make sure the keytab is just a local path. if (deployMode == CLIENT && Utils.isLocalUri(args.keytab)) { args.keytab = new URI(args.keytab).getPath() } If you want to test your application with Kerberos, I recommend for you local mode. https://spark.apache.org/docs/latest/submitting-applications.html#master-urls For example: spark-shell --master local and if you want to access a HDFS filesystem, then you need to add the following parameter as well: spark.kerberos.access.hadoopFileSystems (in old Spark versions this is spark.yarn.access.hadoopFileSystems) spark-shell --master local --conf spark.kerberos.access.hadoopFileSystems=hdfs://namenode.example.com:8020 It will create for you the necessary HDFS delegation token. Here is a very good documentation about the delegation token handling in Spark https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/deploy/security/README.md Best regards, Gabor --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org
