Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/22034#discussion_r208552871 --- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala --- @@ -169,6 +171,19 @@ private[spark] class Executor( startDriverHeartbeater() + /** + * We add an empty WebUI in executor to enable Executor MetricsServlet sink if needed. + */ + private val executorWebUiEnabled = conf.getBoolean("spark.executor.ui.enabled", false) + private[executor] var webUi: ExecutorWebUI = _ + if (executorWebUiEnabled && !isLocal) { + webUi = new ExecutorWebUI(conf, env.securityManager, SparkUI.getUIPort(conf)) + webUi.bind() + env.metricsSystem.getServletHandlers.foreach(webUi.attachHandler) + heartbeatReceiverRef.ask[Boolean](ReportExecutorWebUrl(executorId, webUi.webUrl)) + logInfo(s"Starting executor web ui at ${webUi.webUrl}") --- End diff -- It is too overkill to start an jetty server on each executor only for metrics. I believe you have many different ways to collect executor metrics other than servlet. Besides, to get executor URL you add a new heartbeat message, basically I think it is too overkill.
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org