Github user yaooqinn commented on the issue:
https://github.com/apache/spark/pull/19121
em..this may also not work for `DefaultContainerExecutor` which launch
containers via admin. might close this pr
---
-
To
Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/19121
Sorry I didn't clearly say the problem. But IMO the changes you made is
really not so necessary.
---
-
To unsubscribe,
Github user yaooqinn commented on the issue:
https://github.com/apache/spark/pull/19121
@jerryshao Yes, you are right. unlike yarn standalone donât switch the
jvm process user. but you said that if a user explicitly set `SPARK_USER`,
there is no mechanism in spark to set this user
Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/19121
No, I don't agree with you.
SPARK_USER is set in SparkContext with driver's current UGI and this env
variable will be propagated to executors to create executor's UGI with the same
user
Github user yaooqinn commented on the issue:
https://github.com/apache/spark/pull/19121
@jerryshao thanks for replying
if a user explicitly set `SPARK_USER` but not wrapping the `SPARK_USER`
UGI doAs for SparkContext initialization, there might be two different users
Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/19121
UGI is only used for security, normally it is used for Spark application to
communicate with Hadoop using correct user.
doAs already wraps the whole `CoarseGrainedExecutorBackend`
Github user yaooqinn commented on the issue:
https://github.com/apache/spark/pull/19121
@jerryshao
1. I didn't meet any problems, these codes are ok to run even if it is
unnecessary.
2. In Standalone mode, if collaborating with a secured hdfs, we might
haven't support yet.
Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/19121
Can you please elaborate the problem you met, did you meet any unexpected
behavior?
The changes here get rid of env variable "SPARK_USER", this might be OK for
yarn application, but what
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/19121
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this
