Hi Mich, Community - Do I need to specify it in the properties file in my spark-submit ?
From: Mich Talebzadeh [mailto:m...@peridale.co.uk] Sent: Thursday, February 18, 2016 4:28 PM To: Sutanu Das; user@spark.apache.org Subject: RE: Spark History Server NOT showing Jobs with Hortonworks The jobs are normally shown under <HOSTNAME>:4040/jobs/ in a normal set up not using any vendor's flavoiur Dr Mich Talebzadeh LinkedIn https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw http://talebzadehmich.wordpress.com<http://talebzadehmich.wordpress.com/> NOTE: The information in this email is proprietary and confidential. This message is for the designated recipient only, if you are not the intended recipient, you should destroy it immediately. Any information in this message shall not be understood as given or endorsed by Peridale Technology Ltd, its subsidiaries or their employees, unless expressly so stated. It is the responsibility of the recipient to ensure that this email is virus free, therefore neither Peridale Technology Ltd, its subsidiaries nor their employees accept any responsibility. From: Sutanu Das [mailto:sd2...@att.com] Sent: 18 February 2016 22:22 To: user@spark.apache.org<mailto:user@spark.apache.org> Subject: Spark History Server NOT showing Jobs with Hortonworks Hi Community, Challenged with Spark issues with Hortonworks (HDP 2.3.2_Spark 1.4.1) - The Spark History Server is NOT showing the Spark Running Jobs in Local Mode The local-host:4040/app/v1 is ALSO not working How can I look at my local Spark job? # Generated by Apache Ambari. Fri Feb 5 00:37:06 2016 spark.history.kerberos.keytab none spark.history.kerberos.principal none spark.history.provider org.apache.spark.deploy.yarn.history.YarnHistoryProvider spark.history.ui.port 18080 spark.yarn.containerLauncherMaxThreads 25 spark.yarn.driver.memoryOverhead 2048 spark.yarn.executor.memoryOverhead 2048 spark.yarn.historyServer.address has-dal-0001.corp.wayport.net:18080 spark.yarn.max.executor.failures 3 spark.yarn.preserve.staging.files false spark.yarn.queue default spark.yarn.scheduler.heartbeat.interval-ms 5000 spark.yarn.services org.apache.spark.deploy.yarn.history.YarnHistoryService spark.yarn.submit.file.replication 3 History Server * Timeline Service Location: http://has-dal-0002.corp.wayport.net:8188/ * Last Updated: Feb 18, 2016 10:09:12 PM UTC * Service Started: Feb 5, 2016 12:37:15 AM UTC * Current Time: Feb 18, 2016 10:10:46 PM UTC * Timeline Service: Timeline service is enabled * History Provider: Apache Hadoop YARN Timeline Service
