Any update on this? On Tuesday, 16 August 2016, Aneela Saleem <ane...@platalytics.com> wrote:
> Thanks Steve, > > I went through this but still not able to fix the issue > > On Mon, Aug 15, 2016 at 2:01 AM, Steve Loughran <ste...@hortonworks.com > <javascript:_e(%7B%7D,'cvml','ste...@hortonworks.com');>> wrote: > >> Hi, >> >> Just came across this while going through all emails I'd left unread over >> my vacation. >> >> did you manage to fix this? >> >> 1. There's some notes I've taken on this topic: >> https://www.gitbook.com/book/steveloughran/kerberos_and_hadoop/details >> >> -look at "Error messages to fear" to see if this one has surfaced; >> otherwise look at "low level secrets" to see how to start debugging things >> >> >> On 5 Aug 2016, at 14:54, Aneela Saleem <ane...@platalytics.com >> <javascript:_e(%7B%7D,'cvml','ane...@platalytics.com');>> wrote: >> >> Hi all, >> >> I'm trying to connect to Kerberized Hadoop cluster using spark job. I >> have kinit'd from command line. When i run the following job i.e., >> >> *./bin/spark-submit --keytab /etc/hadoop/conf/spark.keytab --principal >> spark/hadoop-master@platalyticsrealm --class >> com.platalytics.example.spark.App --master spark://hadoop-master:7077 >> /home/vm6/project-1-jar-with-dependencies.jar >> hdfs://hadoop-master:8020/text* >> >> I get the error: >> >> Caused by: java.io.IOException: >> org.apache.hadoop.security.AccessControlException: >> Client cannot authenticate via:[TOKEN, KERBEROS] >> at org.apache.hadoop.ipc.Client$Connection$1.run(Client.java:680) >> at java.security.AccessController.doPrivileged(Native Method) >> at javax.security.auth.Subject.doAs(Subject.java:415) >> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGro >> upInformation.java:1628) >> >> Following are the contents of *spark-defaults.conf* file: >> >> spark.master spark://hadoop-master:7077 >> spark.eventLog.enabled true >> spark.eventLog.dir hdfs://hadoop-master:8020/spark/logs >> spark.serializer org.apache.spark.serializer.Kr >> yoSerializer >> spark.yarn.access.namenodes hdfs://hadoop-master:8020/ >> spark.yarn.security.tokens.hbase.enabled true >> spark.yarn.security.tokens.hive.enabled true >> spark.yarn.principal yarn/hadoop-master@platalyticsrealm >> spark.yarn.keytab /etc/hadoop/conf/yarn.keytab >> >> >> Also i have added following in *spark-env.sh* file: >> >> HOSTNAME=`hostname -f` >> export SPARK_HISTORY_OPTS="-Dspark.history.kerberos.enabled=true >> -Dspark.history.kerberos.principal=spark/${HOSTNAME}@platalyticsrealm >> -Dspark.history.kerberos.keytab=/etc/hadoop/conf/spark.keytab" >> >> >> Please guide me, how to trace the issue? >> >> Thanks >> >> >> >
