sarutak commented on PR #58574: URL: https://github.com/apache/spark/pull/58574#issuecomment-5582298588
Thanks, @dongjoon-hyun. This all sounds good, and I'll go with (a) for local mode in this PR and file (b) (running `UserCredentialManager` in `LocalSchedulerBackend` for real Kerberos parity) as a follow-up. On the two constructor-window asks: - I'll have the selection phase log one line (INFO/WARN) noting that driver-side access to the wired scheme will fail until credentials are acquired at scheduler start, so the eventual executor-side `ClassNotFoundException` is easier to trace. - I'll document the `SparkSubmit` `FileSystem`-cache case as its own item, calling out that it affects the default K8s cluster-mode path (`--jars s3a://BUCKET/...` with output on the same bucket) and produces a mixed-identity job rather than a failure. The rest is as you listed: I/O-free selection without `init()`, WARN + idempotent fallback in `start()`, a single owner for the loader, `stop()` ordering, scaladoc/description updates, `SparkContext`-level tests, and moving the call earlier. I'll also fix the commit message to `SPARK-59296` on the next push. Will ping you when it's up. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
