Hi all, I'm trying to understand if the --proxy-user parameter to spark-submit is deprecated, or something similar? The reason I ask is because it's hard to find documentation really talking about it. The Spark Security doc <https://spark.apache.org/docs/latest/security.html> doesn't mention it at all, nor does the YARN doc <https://spark.apache.org/docs/latest/running-on-yarn.html>. It does show up in the source code in this README file <https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/deploy/security/README.md>. Is that supposed to be the definitive documentation for the option?
Looking into vendor specific docs, it disappears between CDH 5.9 <https://docs.cloudera.com/documentation/enterprise/5-9-x/topics/sg_spark_auth.html> and CDH 5.10 <https://docs.cloudera.com/documentation/enterprise/5-10-x/topics/sg_spark_auth.html> docs. I couldn't find it anywhere in the HDP documentation, but I'm not as familiar with navigating there. And perhaps I'm reading a bit too much into the tea leaves here, but the comments on SPARK-13478 <https://issues.apache.org/jira/browse/SPARK-13478> (which removed the ability to specify --proxy-user along with --principal and --keytab) seem to read like "you shouldn't be using it; you should just use principal and keytab". So, am I going crazy or is this really not clearly documented? Is it a deprecated feature or will it continue to be supported long term? Any kind of insight is appreciated.
