Hi there, I'm using SparkLauncher API from Spark v1.6.1, to submit a Spark job to YARN. The service from where this API will be invoked will need to talk to other services on our cluster via Kerberos (ex. HDFS, YARN etc.). Also, my service expects to impersonate its then logged-in user during job submission to YARN.
Can someone comment on/confirm, the following two observations ? 1. From this Github PR <https://github.com/apache/spark/pull/11358/commits/0159499a55591f25c690bfdfeecfa406142be02b>, it seems that SparkSubmit doesn't support user impersonation when Kerberos credentials of the real user are specified. 2. SPARK-14743 <https://issues.apache.org/jira/browse/SPARK-14743> - It seems that Spark doesn't yet support acquisition of delegation tokens during job submission to YARN irrespective of user impersonation. This Github discussion <https://github.com/apache/spark/pull/11358> is also relevant to this scenario. Thanks, Sudarshan
