[GitHub] [spark] yaooqinn commented on pull request #42295: [SPARK-44581][YARN] Fix the bug that ShutdownHookManager get wrong hadoop user group information

yaooqinn commented on PR #42295: URL: https://github.com/apache/spark/pull/42295#issuecomment-1669282650 (I'm not sure but,) You may sync apache/spark master branch with the one in your fork -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] yaooqinn commented on pull request #42295: [SPARK-44581][YARN] Fix the bug that ShutdownHookManager get wrong hadoop user group information

yaooqinn commented on PR #42295: URL: https://github.com/apache/spark/pull/42295#issuecomment-1669192243 It's not OK https://github.com/apache/spark/actions/runs/5794118171/job/15703082268?pr=42295 -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] yaooqinn commented on pull request #42295: [SPARK-44581][YARN] Fix the bug that ShutdownHookManager get wrong hadoop user group information

yaooqinn commented on PR #42295: URL: https://github.com/apache/spark/pull/42295#issuecomment-1668862618 Hi @liangyu-1, please also take care of the CI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] yaooqinn commented on pull request #42295: [SPARK-44581][YARN] Fix the bug that ShutdownHookManager get wrong hadoop user group information

yaooqinn commented on PR #42295: URL: https://github.com/apache/spark/pull/42295#issuecomment-1668845632 LGTM, please update the PR description according the code updates so far -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [spark] yaooqinn commented on pull request #42295: [SPARK-44581][YARN] Fix the bug that ShutdownHookManager get wrong hadoop user group information

yaooqinn commented on PR #42295: URL: https://github.com/apache/spark/pull/42295#issuecomment-1667567285 Hi, @liangyu-1, thanks for the detailed information. Looks like we can either make the SecurityManager instance a lazy value or move the ApplicationMaster instantiating and

[GitHub] [spark] yaooqinn commented on pull request #42295: [SPARK-44581][YARN]Fix the bug that ShutdownHookManager get wrong hadoop user group information

yaooqinn commented on PR #42295: URL: https://github.com/apache/spark/pull/42295#issuecomment-1665171936 If it's not 3.2-specific, and everything is OK on my side by checking the hdfs audit log too, I guess it could be an environment or configuration issue of yours. -- This is an

[GitHub] [spark] yaooqinn commented on pull request #42295: [SPARK-44581][YARN]Fix the bug that ShutdownHookManager get wrong hadoop user group information

yaooqinn commented on PR #42295: URL: https://github.com/apache/spark/pull/42295#issuecomment-1665057223 I see where the hook added. But I have checked some of our online apps versioned 3.1 and 3.3, the stagings are deleted successfully. Is it a 3.2-specific issue? -- This is an

[GitHub] [spark] yaooqinn commented on pull request #42295: [SPARK-44581][YARN]Fix the bug that ShutdownHookManager get wrong hadoop user group information

yaooqinn commented on PR #42295: URL: https://github.com/apache/spark/pull/42295#issuecomment-1664906674 The staging directory is cleaned automatically by Spark, why do you even need this hook？ -- This is an automated message from the Apache Git Service. To respond to the message,