Hi, Spark users.
I wanted to make unknown users create HDFS files, not the OS user who executes the spark application. And I thought it would be possible using UserGroupInformation.createRemoteUser(“other”).doAS(…) However, the files are created by the OS user who launched the spark application in Spark Executors. Although I’ve tested it on Spark Standalone and Yarn, I got the same results. Is it impossible to impersonate a Spark job user using the UserGroupInformation.doAS? PS. In fact, I posted a similar question on the Spark user mailing list, But I didn’t get the answer I wanted. http://apache-spark-user-list.1001560.n3.nabble.com/Is-it-enable-to-use-Multiple-UGIs-in-One-Spark-Context-td39859.html