GitHub user nblintao opened a pull request: https://github.com/apache/spark/pull/14204
[SPARK-16520] [WEBUI] Link executors to corresponding worker pages ## What changes were proposed in this pull request? Added a new column named "Worker" in the Executor Page of the web UI. For each executor (except the driver), it will provide a link to the web UI of the worker it belongs to. This PR also retrieved worker URLs (besides worker _UI_ URLs) from `ExecutorRunner` to `ExecutorsPage`. This would help if we want to show then on the Executor Page later. Because the data of the URLs had not been retrieved to the executor UI (even not in the executor), passing them have the following effects: - The DeveloperApi `ExecutorInfo` is changed. A new argument, `workerUrl: Map[String, String]`, is added in the constructor of that class. But it is set to `Map.empty` by default, so calling the old API still works. The executors for local and Mesos are stilling using call the old API. - The JSON converter of `ExecutorInfo` is update. - The REST API for executors has a new item call "worker", it looks like this: ``` "worker" : { "url" : "spark://Worker@192.168.1.104:32790", "ui_url" : "http://192.168.1.104:46445" } ``` ## How was this patch tested? Run Spark as a "local cluster" by `./bin/spark-shell --master=local-cluster[2,1,1024]`. Then view the Executor Page on http://192.168.1.104:4040/executors/. See the column "Worker" on the right of the Executors Table. For each executor, click the hypertext link, jump to the web UI of the worker it belongs to. Screenshot: ![linkworkerui](https://cloud.githubusercontent.com/assets/5558370/16845207/b88d3460-49ad-11e6-9e36-357a25ad60fb.png) You can merge this pull request into a Git repository by running: $ git pull https://github.com/nblintao/spark linkExecutorToWorker Alternatively you can review and apply these changes as the patch at: https://github.com/apache/spark/pull/14204.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #14204 ---- commit 21a5abc873c69f306514eaf0130afe96bb8db9d9 Author: Tao Lin <nblin...@gmail.com> Date: 2016-07-14T15:13:59Z add links to worker UIs in the executor page ---- --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org