Did anyone use https://www.codatlas.com/github.com/apache/spark/HEAD/core/src/main/scala/org/apache/spark/deploy/SparkHadoopUtil.scala to interact with secured Hadoop from Spark ?
Thanks, Ajay On Mon, Nov 7, 2016 at 4:37 PM, Ajay Chander <itsche...@gmail.com> wrote: > > Hi Everyone, > > I am trying to develop a simple codebase on my machine to read data from > secured Hadoop cluster. We have a development cluster which is secured > through Kerberos and I want to run a Spark job from my IntelliJ to read > some sample data from the cluster. Has anyone done this before ? Can you > point me to some sample examples? > > I understand that, if we want to talk to secured cluster, we need to have > keytab and principle. I tried using it through > UserGroupInformation.loginUserFromKeytab > and SparkHadoopUtil.get.loginUserFromKeytab but so far no luck. > > I have been trying to do this from quite a while ago. Please let me know > if you need more info. Thanks > > Regards, > Ajay >
