GitHub user nblintao opened a pull request:
https://github.com/apache/spark/pull/14204
[SPARK-16520] [WEBUI] Link executors to corresponding worker pages
## What changes were proposed in this pull request?
Added a new column named "Worker" in the Executor Page of the web UI. For
each executor (except the driver), it will provide a link to the web UI of the
worker it belongs to.
This PR also retrieved worker URLs (besides worker _UI_ URLs) from
`ExecutorRunner` to `ExecutorsPage`. This would help if we want to show then on
the Executor Page later.
Because the data of the URLs had not been retrieved to the executor UI
(even not in the executor), passing them have the following effects:
- The DeveloperApi `ExecutorInfo` is changed. A new argument, `workerUrl:
Map[String, String]`, is added in the constructor of that class. But it is set
to `Map.empty` by default, so calling the old API still works. The executors
for local and Mesos are stilling using call the old API.
- The JSON converter of `ExecutorInfo` is update.
- The REST API for executors has a new item call "worker", it looks like
this:
```
"worker" : {
"url" : "spark://Worker@192.168.1.104:32790",
"ui_url" : "http://192.168.1.104:46445";
}
```
## How was this patch tested?
Run Spark as a "local cluster" by `./bin/spark-shell
--master=local-cluster[2,1,1024]`. Then view the Executor Page on
http://192.168.1.104:4040/executors/.
See the column "Worker" on the right of the Executors Table. For each
executor, click the hypertext link, jump to the web UI of the worker it belongs
to.
Screenshot:
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/nblintao/spark linkExecutorToWorker
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/14204.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #14204
----
commit 21a5abc873c69f306514eaf0130afe96bb8db9d9
Author: Tao Lin <nblin...@gmail.com>
Date: 2016-07-14T15:13:59Z
add links to worker UIs in the executor page
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
