Github user steveloughran commented on a diff in the pull request:
https://github.com/apache/spark/pull/9182#discussion_r43439082
--- Diff:
yarn/src/main/scala/org/apache/spark/scheduler/cluster/YarnSchedulerBackend.scala
---
@@ -51,6 +51,41 @@ private[spark] abstract class YarnSchedulerBackend(
private implicit val askTimeout = RpcUtils.askRpcTimeout(sc.conf)
+ /** Application ID. Must be set by a subclass before starting the
service */
+ private var appId: ApplicationId = null
+
+ /** Attempt ID. This is unset for client-mode schedulers */
+ private var attemptId: Option[ApplicationAttemptId] = None
+
+ /** Scheduler extension services */
+ private val services: SchedulerExtensionServices = new
SchedulerExtensionServices()
+
+ /**
+ * Bind to YARN. This *must* be done before calling [[start()]].
+ *
+ * @param appId YARN application ID
+ * @param attemptId Optional YARN attempt ID
+ */
+ protected def bindToYarn(appId: ApplicationId, attemptId:
Option[ApplicationAttemptId]): Unit = {
+ this.appId = appId
+ this.attemptId = attemptId
+ }
+
+ override def start() {
+ require(appId != null, "application ID unset")
+ val binding = SchedulerExtensionServiceBinding(sc, appId, attemptId)
+ services.start(binding)
--- End diff --
For the history service: yes. What I didn't want to do was write an
extension service which suited exactly one use case though.
Still, there's not much you can do with an appId that doesn't involve the
YARN client or AM/RPM RPC channels, and if a some extension wants to do that,
it'll probably need more than just the ID structures. For other uses (e.g.
grabbing the app reports, looking things up in the RM web UI, etc, those string
values will work).
I'll go to strings.
W.r.t the existing methods, the application attempt ID that exists today is
the tail of the (parsed) app attempt string, "1", "2", .... For the history
stuff I do need the full attemptID for its uniqueness.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
