You can use the yarn proxy:
http://
<resourceManager>:8088/proxy/<application_ID>/api/v1/applications/<application_name>/executors
I have an scala application that monitor the number of executors of some
spark streamings, and I had a similar problem, where I iterate over the
running jobs and get the number of executors:
val states = EnumSet.of(
YarnApplicationState.RUNNING,
YarnApplicationState.ACCEPTED)
val it = yarnClient.getApplications(states).iterator()
while (it.hasNext()) {
val app = it.next()
val strUrl = appReport.getTrackingUrl + "api/v1/applications/" +
appReport.getName + "/executors"
val url = new URL(strUrl)
val urlCon = url.openConnection()
val content = fromInputStream(urlCon.getInputStream)
.getLines.mkString("\n")
val j = JSON.parseFull(content)
val currentExecutors = j.get.asInstanceOf[List[Map[String,
String]]].filterNot(_("id") == "driver").size
Regards,
Ricardo
On Thu, Feb 18, 2016 at 1:56 PM, alvarobrandon [via Apache Spark User List]
<ml-node+s1001560n26267...@n3.nabble.com> wrote:
> Hello:
>
> I wanted to access the REST API (
> http://spark.apache.org/docs/latest/monitoring.html#rest-api) of Spark to
> monitor my jobs. However I'm running my Spark Apps over YARN. When I try to
> make a request to http://localhost:4040/api/v1 as the documentation says
> I don't get any response. My question is. It is possible to access this
> REST API when you are not using Spark in Standalone mode?
>
> Thanks in advance
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
>
> http://apache-spark-user-list.1001560.n3.nabble.com/SPARK-REST-API-on-YARN-tp26267.html
> To start a new topic under Apache Spark User List, email
> ml-node+s1001560n1...@n3.nabble.com
> To unsubscribe from Apache Spark User List, click here
> <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=1&code=cmljYXJkby5wYWl2YUBjb3JwLmdsb2JvLmNvbXwxfDQ1MDcxMTc2Mw==>
> .
> NAML
> <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
--
Ricardo Paiva
Big Data / Semântica
2483-6432
*globo.com* <http://www.globo.com>
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/SPARK-REST-API-on-YARN-tp26267p26268.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
