[ https://issues.apache.org/jira/browse/SPARK-19105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Apache Spark reassigned SPARK-19105: ------------------------------------ Assignee: (was: Apache Spark) > yarn/Client.scala copyToRemote does not include keytab destination name > ----------------------------------------------------------------------- > > Key: SPARK-19105 > URL: https://issues.apache.org/jira/browse/SPARK-19105 > Project: Spark > Issue Type: Bug > Affects Versions: 2.1.0 > Environment: YARN in client mode > Reporter: Peter Parente > > When I specify --principal user@REALM and --keytab /some/path/user.keytab, I > see the following in my app staging directory on HDFS: > {code} > -rw-r--r-- 3 user supergroup 68 2017-01-06 03:59 user.keytab > -rw-r--r-- 3 user supergroup 73502 2017-01-06 03:59 __spark_conf__.zip > -rw-r--r-- 3 user supergroup 189767340 2017-01-06 03:59 > __spark_libs__4440821503780683972.zip > -rw-r--r-- 3 user supergroup 91275 2017-01-06 03:59 py4j-0.10.3-src.zip > -rw-r--r-- 3 user supergroup 440385 2017-01-06 03:59 pyspark.zip > {code} > I also see that my spark.yarn.keytab config value has changed to > user.keytab-54ee5192-43d0-41b5-ba50-1181ece26961 by the yarn client to ensure > the keytab is unique within the app staging directory. However, from the > directory listing above, it's clear that the file written does not match this > new name. As a result, when it comes time to renew the Kerberos ticket, > AMDelegationTokenRenewer fails to find the keytab under the UUID-suffixed > name and also fails to renew the tickets. > The problem looks to be in one call to [copyFileToRemote in > yarn/Client.java|https://github.com/apache/spark/blob/fe1c895e16c475a6f271ce600a42a8d0dc7986e5/resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala#L482] > that leaves off the destination filename param. The other calls in that > object which use copyFileToRemote and have a custom destination name all > provide this parameter (e.g., > https://github.com/apache/spark/blob/fe1c895e16c475a6f271ce600a42a8d0dc7986e5/resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala#L652). -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org