Does YARN provide the token through that env variable you mentioned? Or how does YARN do this?
Tim On Fri, Jun 26, 2015 at 3:51 PM, Marcelo Vanzin <van...@cloudera.com> wrote: > On Fri, Jun 26, 2015 at 3:44 PM, Dave Ariens <dari...@blackberry.com> > wrote: > >> Fair. I will look into an alternative with a generated delegation >> token. However the same issue exists. How can I have the executor run >> some arbitrary code when it gets a task assignment and before it proceeds >> to process it's resources? >> > > Hmm, good question. If it doesn't already, Mesos could have its own > implementation of CoarseGrainedExecutorBackend that provides that > functionality. The only difference is that you'd run something before the > executor starts up, not before each task. > > YARN actually doesn't do it that way; YARN provides the tokens to the > executor before the process starts, so that when you call > "UserGroupInformation.getCurrentUser()" the tokens are already there. > > One way of doing that is by writing the tokens to a file and setting the > KRB5CCNAME env variable when starting the process. You can check the Hadoop > sources for details. Not sure if there's another way. > > > >> >> *From: *Marcelo Vanzin >> *Sent: *Friday, June 26, 2015 6:20 PM >> *To: *Dave Ariens >> *Cc: *Tim Chen; Olivier Girardot; user@spark.apache.org >> *Subject: *Re: Accessing Kerberos Secured HDFS Resources from Spark on >> Mesos >> >> On Fri, Jun 26, 2015 at 3:09 PM, Dave Ariens <dari...@blackberry.com> >> wrote: >> >>> Would there be any way to have the task instances in the slaves call >>> the UGI login with a principal/keytab provided to the driver? >>> >> >> That would only work with a very small number of executors. If you have >> many login requests in a short period of time with the same principal, the >> KDC will start to deny logins. That's why delegation tokens are used >> instead of explicit logins. >> >> -- >> Marcelo >> > > > > -- > Marcelo >
