[
https://issues.apache.org/jira/browse/SPARK-15999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Faisal reopened SPARK-15999:
----------------------------
> Wrong/Missing information for Spark UI/REST port
> ------------------------------------------------
>
> Key: SPARK-15999
> URL: https://issues.apache.org/jira/browse/SPARK-15999
> Project: Spark
> Issue Type: Bug
> Components: Documentation, Streaming
> Affects Versions: 1.5.0
> Environment: CDH5.5.2, Spark 1.5.0
> Reporter: Faisal
> Priority: Minor
>
> *Spark Monitoring documentation*
> https://spark.apache.org/docs/1.5.0/monitoring.html
> {quote}
> You can access this interface by simply opening http://<driver-node>:4040 in
> a web browser. If multiple SparkContexts are running on the same host, they
> will bind to successive ports beginning with 4040 (4041, 4042, etc).
> {quote}
> This statement is very confusing and doesn't apply at all in spark streaming
> jobs(unless i am missing something)
> Same is the case with REST API calls.
> {quote}
> REST API
> In addition to viewing the metrics in the UI, they are also available as
> JSON. This gives developers an easy way to create new visualizations and
> monitoring tools for Spark. The JSON is available for both running
> applications, and in the history server. The endpoints are mounted at
> /api/v1. Eg., for the history server, they would typically be accessible at
> http://<server-url>:18080/api/v1, and for a running application, at
> http://localhost:4040/api/v1.
> {quote}
> I am running spark streaming job in CDH-5.5.2 Spark version 1.5.0
> and nowhere on driver node, executor node for running/live application i am
> able to call rest service.
> My spark streaming jobs running in yarn cluster mode
> --master yarn-cluster
> However for historyServer
> i am able to call REST service and can pull up json messages
> using the URL
> http://historyServer:18088/api/v1/applications
> {code}
> [ {
> "id" : "application_1463099418950_11465",
> "name" : "PySparkShell",
> "attempts" : [ {
> "startTime" : "2016-06-15T15:28:32.460GMT",
> "endTime" : "2016-06-15T19:01:39.100GMT",
> "sparkUser" : "abc",
> "completed" : true
> } ]
> }, {
> "id" : "application_1463099418950_11635",
> "name" : "DataProcessor-ETL.ETIME",
> "attempts" : [ {
> "attemptId" : "1",
> "startTime" : "2016-06-15T18:56:04.413GMT",
> "endTime" : "2016-06-15T18:58:00.022GMT",
> "sparkUser" : "abc",
> "completed" : true
> } ]
> },
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
