cxzl25 opened a new pull request, #36808: URL: https://github.com/apache/spark/pull/36808
### What changes were proposed in this pull request? Start `HadoopDelegationTokenManager` when `LocalSchedulerBackend` starts. The behavior is similar to `CoarseGrainedSchedulerBackend` startup, but it does not support local mode. ### Why are the changes needed? After[ SPARK-23639](https://issues.apache.org/jira/browse/SPARK-23639), we can use `spark-sql --master=local --proxy-user xxx` to proxy a user to access the hive meta store in the kerberos environment. But `spark-submit` does not support accessing the hive meta store, we will get The following exception. `No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)` And `spark-sql/spark-submit` will not automatically refresh the token or automatically relogin when using `--keytab` ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Write a simple spark to access the Hive metastore application in the kerberos environment, submit it through `spark-submit --proxy-user`, in this PR, it can work normally. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
