sarutak commented on code in PR #58737:
URL: https://github.com/apache/spark/pull/58737#discussion_r3992734351
##########
core/src/main/scala/org/apache/spark/scheduler/local/LocalSchedulerBackend.scala:
##########
@@ -137,6 +143,41 @@ private[spark] class LocalSchedulerBackend(
SparkHadoopUtil.get.addDelegationTokens(tokens, conf)
}
+ /**
+ * Start the UserCredentialManager if OIDC credential propagation is
enabled, mirroring
+ * CoarseGrainedSchedulerBackend. Runs independently of
Kerberos/HadoopDelegationTokenManager.
+ *
+ * In local mode the driver and the single executor share this JVM and the
same
+ * `SparkEnv.get.userCredentials`, so the propagation callback simply
updates that reference
+ * (there is no remote executor to message); the in-JVM Executor picks up
credentials from the
+ * same store via TaskDescription. Driver-side filesystem access uses the
provider wiring that
+ * the selection phase (UserCredentialManager.applyProviderProperties)
already applied to the
+ * driver's Hadoop Configuration.
+ */
+ private def setupUserCredentialManager(): Unit = {
+ // Reuse the loader from SparkContext's selection phase (Some when OIDC is
enabled, None
+ // otherwise). Passing the Option straight through keeps SparkContext as
the single owner of
+ // the loader: create() enforces that an enabled configuration has a
loader rather than
+ // silently allocating one here that no one would close.
+ userCredentialManager = UserCredentialManager.create(conf, { (version,
credentials) =>
+ // No remote executors in local mode; update the shared credential store
directly so that
+ // subsequently dispatched tasks (and driver-side access) observe the
new credentials.
+ VersionedCredentials.updateIfNewer(SparkEnv.get.userCredentials,
version, credentials)
Review Comment:
Thanks, `setupUserCredentialManager()` now captures `val env = SparkEnv.get`
once and uses it in both the callback and the initial store, so a late renewal
can only touch this (stopped) application's store, never a new context's.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
