Github user ArtRand commented on a diff in the pull request:
https://github.com/apache/spark/pull/19272#discussion_r146094019
--- Diff:
resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCoarseGrainedSchedulerBackend.scala
---
@@ -194,6 +198,27 @@ private[spark] class
MesosCoarseGrainedSchedulerBackend(
sc.conf.getOption("spark.mesos.driver.frameworkId").map(_ + suffix)
)
+ // check that the credentials are defined, even though it's likely
that auth would have failed
+ // already if you've made it this far
+ if (principal != null && hadoopDelegationCreds.isDefined) {
+ logDebug(s"Principal found ($principal) starting token renewer")
+ val credentialRenewerThread = new Thread {
+ setName("MesosCredentialRenewer")
+ override def run(): Unit = {
+ val rt =
MesosCredentialRenewer.getTokenRenewalTime(hadoopDelegationCreds.get, conf)
+ val credentialRenewer =
+ new MesosCredentialRenewer(
+ conf,
+ hadoopDelegationTokenManager.get,
+ MesosCredentialRenewer.getNextRenewalTime(rt),
+ driverEndpoint)
+ credentialRenewer.scheduleTokenRenewal()
+ }
+ }
+
+ credentialRenewerThread.start()
+ credentialRenewerThread.join()
--- End diff --
Ok, you're probably right. It appears that the YARN code uses
`setContextClassLoader(userClassLoader)` whereas in Mesos does not has a notion
of `userClassLoader`. Therefore we don't need the separate thread in the Mesos
code. Do I have this correct? Thanks for showing me this!
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
