Re: Vulnerabilities to UserGroupInformation / credentials in a Spark Cluster

Hi Blaze! Thanks for the link, although it did not have anything I didn't already know. I'm afraid I don't quite follow what your concern is here. The files are protected using UNIX permissions on the worker nodes. Is that not what you are seeing? Are you using the LinuxContainerExecutor? Are the

Re: Vulnerabilities to UserGroupInformation / credentials in a Spark Cluster

Ravi, The code and architecture is based on the Hadoop source code submitted through the Yarn Client.This is an issue for map reduce as well. eg: https://pravinchavan.wordpress.com/2013/04/25/223/ On Mon, Oct 30, 2017 at 1:15 PM, Ravi Prakash wrote: > Hi Blaze! > >

Re: Vulnerabilities to UserGroupInformation / credentials in a Spark Cluster

Hi Blaze! Thanks for digging into this. I'm sure security related features could use more attention. Tokens for one user should be isolated from other users. I'm sorry I don't know how spark uses them. Would this question be more appropriate on the spark mailing list?

Re: Vulnerabilities to UserGroupInformation / credentials in a Spark Cluster

I looked at this a bit more and I see a container_tokens file in spark directory. Does this contain the credentials where are added by addCredentials? Is this file accessible to the spark executors? It looks like just a clear text protobuf file.

Vulnerabilities to UserGroupInformation / credentials in a Spark Cluster

Hi, We are submitting critical UserGroupInformation credentials and wanted to know how these are protected in Spark Cluster. Questions: Are the credentials persisted to disk at any point? If so, where? If they are persisted, are they encrypted? Or just obfuscated? is the encryption key