Suraj, Spark History server is running on 18080 (http://spark.apache.org/docs/latest/monitoring.html) which is not going to give you are real-time update on a running Spark application. Given this is Spark on YARN, you will need to view the Spark UI from the Application Master URL which can be found from the YARN Resource Manager UI (master node:8088) and it would be best to use a SOCKS proxy in order nicely resolve the URLs.
Best regards, Christopher From: SURAJ SHETH [mailto:shet...@gmail.com] Sent: Sunday, November 15, 2015 8:19 AM To: user@spark.apache.org Subject: Yarn Spark on EMR Hi, Yarn UI on 18080 stops receiving updates Spark jobs/tasks immediately after it starts. We see only one task completed in the UI while the other hasn't got any resources while in reality, more than 5 tasks would have completed. Hadoop - Amazon 2.6 Spark - 1.5 Thanks and Regards, Suraj Sheth
