[ https://issues.apache.org/jira/browse/SPARK-25732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16651657#comment-16651657 ]
Thomas Graves commented on SPARK-25732: --------------------------------------- So like Marcelo mentioned can't you re-use the keytab/principal option already there? It might need slightly modified to pull from HDFS but that is really what this is doing, its just livy is submitting the job for you. Really the user could specify it when submitting the job as a conf (? I guess depends on who is calling livy, jupyter for instance definitely could as user can pass configs). I would prefer that over adding more configs. There are lots of cases things are in the middle of job submission, livy, oozie, other workflow managers. I don't see that as a reason not to do tokens. User should know they are submitting jobs (especially one that runs for 2 weeks) and until we have a good automated solution, they would have to setup cron or something else to push tokens before they expire. I know the YARN folks were looking at options to help with this but haven't synced with them lately as ideally there would be a way to push the tokens to the RM for it to continue to renew so you would only have to do it before max lifetime. Its easy enough to write a script that runs and does a list of applications running for the user and push tokens to each of those, assuming we had spark-submit option to push tokens. > Allow specifying a keytab/principal for proxy user for token renewal > --------------------------------------------------------------------- > > Key: SPARK-25732 > URL: https://issues.apache.org/jira/browse/SPARK-25732 > Project: Spark > Issue Type: Improvement > Components: Deploy > Affects Versions: 2.4.0 > Reporter: Marco Gaido > Priority: Major > > As of now, application submitted with proxy-user fail after 2 week due to the > lack of token renewal. In order to enable it, we need the the > keytab/principal of the impersonated user to be specified, in order to have > them available for the token renewal. > This JIRA proposes to add two parameters {{--proxy-user-principal}} and > {{--proxy-user-keytab}}, and the last letting a keytab being specified also > in a distributed FS, so that applications can be submitted by servers (eg. > Livy, Zeppelin) without needing all users' principals being on that machine. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org