Hi guys, Not sure if you have similar issues. Did not find relevant tickets in JIRA. When I deploy the Spark Streaming to YARN, I have following two issues:
1. The UI port is random. It is not default 4040. I have to look at the container's log to check the UI port. Is this suppose to be this way? 2. Most of the time, the UI does not work. The difference between logs are (I ran the same program): *14/07/03 11:38:50 INFO spark.HttpServer: Starting HTTP Server14/07/03 11:38:50 INFO server.Server: jetty-8.y.z-SNAPSHOT 14/07/03 11:38:50 INFO server.AbstractConnector: Started SocketConnector@0.0.0.0:12026 <http://SocketConnector@0.0.0.0:12026>14/07/03 11:38:51 INFO executor.CoarseGrainedExecutorBackend: Got assigned task 0 14/07/03 11:38:51 INFO executor.Executor: Running task ID 0...* 14/07/02 16:55:32 INFO spark.HttpServer: Starting HTTP Server 14/07/02 16:55:32 INFO server.Server: jetty-8.y.z-SNAPSHOT 14/07/02 16:55:32 INFO server.AbstractConnector: Started SocketConnector@0.0.0.0:14211 *14/07/02 16:55:32 INFO ui.JettyUtils: Adding filter: org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter14/07/02 16:55:32 INFO server.Server: jetty-8.y.z-SNAPSHOT14/07/02 16:55:32 INFO server.AbstractConnector: Started SelectChannelConnector@0.0.0.0:21867 <http://SelectChannelConnector@0.0.0.0:21867> 14/07/02 16:55:32 INFO ui.SparkUI: Started SparkUI at http://myNodeName:21867 <http://myNodeName:21867>14/07/02 16:55:32 INFO cluster.YarnClusterScheduler: Created YarnClusterScheduler* When the red part comes, the UI works sometime. Any ideas? Thank you. Best, Fang, Yan yanfang...@gmail.com +1 (206) 849-4108
