@Yan, the UI should still work. As long as you look into the container that launches the driver, you will find the SparkUI address and port. Note that in yarn-cluster mode the Spark driver doesn't actually run in the Application Manager; just like the executors, it runs in a container that is launched by the Resource Manager after the Application Master requests the container resources. In contrast, in yarn-client mode, your driver is not launched in a container, but in the client process that launched your application (i.e. spark-submit), so the stdout of this program directly contains the SparkUI messages.
@Chester, I'm not sure what has gone wrong as there are many factors at play here. When you go the Resource Manager UI, does the "application URL" link point you to the same SparkUI address as indicated in the logs? If so, this is the correct behavior. However, I believe the redirect error has little to do with Spark itself, but more to do with how you set up the cluster. I have actually run into this myself, but I haven't found a workaround. Let me know if you find anything. 2014-07-07 12:07 GMT-07:00 Chester Chen <ches...@alpinenow.com>: > As Andrew explained, the port is random rather than 4040, as the the spark > driver is started in Application Master and the port is random selected. > > > But I have the similar UI issue. I am running Yarn Cluster mode against my > local CDH5 cluster. > > The log states > "14/07/07 11:59:29 INFO ui.SparkUI: Started SparkUI at > http://10.0.0.63:58750 > > " > > > but when you client the spark UI link (ApplicationMaster or > > http://10.0.0.63:58750), I will got a 404 with the redirect URI > > > http://localhost/proxy/application_1404443455764_0010/ > > > > Looking at the Spark code, notice that the "proxy" is reallya variable to get > the proxy at the yarn-site.xml http address. But when I specified the value > at yarn-site.xml, it still doesn't work for me. > > > > Oddly enough, it works for my co-worker on Pivotal HD cluster, therefore I am > still looking what's the difference in terms of cluster setup or something > else. > > > Chester > > > > > > On Mon, Jul 7, 2014 at 11:42 AM, Andrew Or <and...@databricks.com> wrote: > >> I will assume that you are running in yarn-cluster mode. Because the >> driver is launched in one of the containers, it doesn't make sense to >> expose port 4040 for the node that contains the container. (Imagine if >> multiple driver containers are launched on the same node. This will cause a >> port collision). If you're launching Spark from a gateway node that is >> physically near your worker nodes, then you can just launch your >> application in yarn-client mode, in which case the SparkUI will always be >> started on port 4040 on the node that you ran spark-submit on. The reason >> why sometimes you see the red text is because it appears only on the driver >> containers, not the executor containers. This is because SparkUI belongs to >> the SparkContext, which only exists on the driver. >> >> Andrew >> >> >> 2014-07-07 11:20 GMT-07:00 Yan Fang <yanfang...@gmail.com>: >> >> Hi guys, >>> >>> Not sure if you have similar issues. Did not find relevant tickets in >>> JIRA. When I deploy the Spark Streaming to YARN, I have following two >>> issues: >>> >>> 1. The UI port is random. It is not default 4040. I have to look at the >>> container's log to check the UI port. Is this suppose to be this way? >>> >>> 2. Most of the time, the UI does not work. The difference between logs >>> are (I ran the same program): >>> >>> >>> >>> >>> >>> >>> *14/07/03 11:38:50 INFO spark.HttpServer: Starting HTTP Server14/07/03 >>> 11:38:50 INFO server.Server: jetty-8.y.z-SNAPSHOT 14/07/03 11:38:50 INFO >>> server.AbstractConnector: Started SocketConnector@0.0.0.0:12026 >>> <http://SocketConnector@0.0.0.0:12026>14/07/03 11:38:51 INFO >>> executor.CoarseGrainedExecutorBackend: Got assigned task 0 14/07/03 >>> 11:38:51 INFO executor.Executor: Running task ID 0...* >>> >>> 14/07/02 16:55:32 INFO spark.HttpServer: Starting HTTP Server >>> 14/07/02 16:55:32 INFO server.Server: jetty-8.y.z-SNAPSHOT >>> 14/07/02 16:55:32 INFO server.AbstractConnector: Started >>> SocketConnector@0.0.0.0:14211 >>> >>> >>> >>> >>> *14/07/02 16:55:32 INFO ui.JettyUtils: Adding filter: >>> org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter14/07/02 16:55:32 >>> INFO server.Server: jetty-8.y.z-SNAPSHOT14/07/02 16:55:32 INFO >>> server.AbstractConnector: Started SelectChannelConnector@0.0.0.0:21867 >>> <http://SelectChannelConnector@0.0.0.0:21867> 14/07/02 16:55:32 INFO >>> ui.SparkUI: Started SparkUI at http://myNodeName:21867 >>> <http://myNodeName:21867>14/07/02 16:55:32 INFO >>> cluster.YarnClusterScheduler: Created YarnClusterScheduler* >>> >>> When the red part comes, the UI works sometime. Any ideas? Thank you. >>> >>> Best, >>> >>> Fang, Yan >>> yanfang...@gmail.com >>> +1 (206) 849-4108 >>> >> >> >
