Is 4040 port used in your host? It should be default
Example netstat -plten|grep 4040 tcp 0 0 :::4040 :::* LISTEN 1009 42748209 22778/java ps -ef|grep 22778 hduser 22778 22770 0 08:34 pts/1 00:01:18 /usr/java/latest/bin/java -cp /home/hduser/jars/jconn4.jar:/home/hduser/jars/ojdbc6.jar:/usr/lib/spark-1.5 .2-bin-hadoop2.6/conf/:/usr/lib/spark-1.5.2-bin-hadoop2.6/lib/spark-assembly -1.5.2-hadoop2.6.0.jar:/usr/lib/spark-1.5.2-bin-hadoop2.6/lib/datanucleus-co re-3.2.10.jar:/usr/lib/spark-1.5.2-bin-hadoop2.6/lib/datanucleus-api-jdo-3.2 .6.jar:/usr/lib/spark-1.5.2-bin-hadoop2.6/lib/datanucleus-rdbms-3.2.9.jar:/h ome/hduser/hadoop-2.6.0/etc/hadoop/ -Dscala.usejavacp=true -Xms1G -Xmx1G -XX:MaxPermSize=256m org.apache.spark.deploy.SparkSubmit --master spark://50.140.197.217:7077 --class org.apache.spark.repl.Main --name Spark shell spark-shell Dr Mich Talebzadeh LinkedIn <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABU rV8Pw> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUr V8Pw <http://talebzadehmich.wordpress.com/> http://talebzadehmich.wordpress.com NOTE: The information in this email is proprietary and confidential. This message is for the designated recipient only, if you are not the intended recipient, you should destroy it immediately. Any information in this message shall not be understood as given or endorsed by Peridale Technology Ltd, its subsidiaries or their employees, unless expressly so stated. It is the responsibility of the recipient to ensure that this email is virus free, therefore neither Peridale Technology Ltd, its subsidiaries nor their employees accept any responsibility. From: Sutanu Das [mailto:sd2...@att.com] Sent: 18 February 2016 22:58 To: Mich Talebzadeh <m...@peridale.co.uk>; user@spark.apache.org Subject: RE: Spark History Server NOT showing Jobs with Hortonworks Hi Mich, Community - Do I need to specify it in the properties file in my spark-submit ? From: Mich Talebzadeh [mailto:m...@peridale.co.uk] Sent: Thursday, February 18, 2016 4:28 PM To: Sutanu Das; user@spark.apache.org <mailto:user@spark.apache.org> Subject: RE: Spark History Server NOT showing Jobs with Hortonworks The jobs are normally shown under <HOSTNAME>:4040/jobs/ in a normal set up not using any vendor's flavoiur Dr Mich Talebzadeh LinkedIn <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABU rV8Pw> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUr V8Pw <http://talebzadehmich.wordpress.com/> http://talebzadehmich.wordpress.com NOTE: The information in this email is proprietary and confidential. This message is for the designated recipient only, if you are not the intended recipient, you should destroy it immediately. Any information in this message shall not be understood as given or endorsed by Peridale Technology Ltd, its subsidiaries or their employees, unless expressly so stated. It is the responsibility of the recipient to ensure that this email is virus free, therefore neither Peridale Technology Ltd, its subsidiaries nor their employees accept any responsibility. From: Sutanu Das [mailto:sd2...@att.com] Sent: 18 February 2016 22:22 To: user@spark.apache.org <mailto:user@spark.apache.org> Subject: Spark History Server NOT showing Jobs with Hortonworks Hi Community, Challenged with Spark issues with Hortonworks (HDP 2.3.2_Spark 1.4.1) - The Spark History Server is NOT showing the Spark Running Jobs in Local Mode The local-host:4040/app/v1 is ALSO not working How can I look at my local Spark job? # Generated by Apache Ambari. Fri Feb 5 00:37:06 2016 spark.history.kerberos.keytab none spark.history.kerberos.principal none spark.history.provider org.apache.spark.deploy.yarn.history.YarnHistoryProvider spark.history.ui.port 18080 spark.yarn.containerLauncherMaxThreads 25 spark.yarn.driver.memoryOverhead 2048 spark.yarn.executor.memoryOverhead 2048 spark.yarn.historyServer.address has-dal-0001.corp.wayport.net:18080 spark.yarn.max.executor.failures 3 spark.yarn.preserve.staging.files false spark.yarn.queue default spark.yarn.scheduler.heartbeat.interval-ms 5000 spark.yarn.services org.apache.spark.deploy.yarn.history.YarnHistoryService spark.yarn.submit.file.replication 3 History Server * Timeline Service Location: http://has-dal-0002.corp.wayport.net:8188/ * Last Updated: Feb 18, 2016 10:09:12 PM UTC * Service Started: Feb 5, 2016 12:37:15 AM UTC * Current Time: Feb 18, 2016 10:10:46 PM UTC * Timeline Service: Timeline service is enabled * History Provider: Apache Hadoop YARN Timeline Service