Hi Anfemee, Subject in the email sometimes help ;) Have you seen if the link is sending you to a hostname that is not accessible by your workstation? Sometimes changing the hostname to the ip solve this kind of issue.
2015-10-15 13:34 GMT-03:00 Anfernee Xu <anfernee...@gmail.com>: > Sorry, I have to re-send it again as I did not get the answer. > > Here's the problem I'm facing, I have a standalone java application which > is periodically submit Spark jobs to my yarn cluster, btw I'm not using > 'spark-submit' or 'org.apache.spark.launcher' to submit my jobs. These jobs > are successful and I can see them on Yarn RM webUI, but when I want to > follow the link to the app history on Spark historyserver, I always got > 404(application is not found) from Spark historyserver. > > My code looks likes as below > > > SparkConf conf = new > SparkConf().setAppName("testSpak").setMaster("yarn-client") > .setJars(new String[]{IOUtil.getJar(MySparkApp.class)}); > > conf.set("spark.yarn.historyServer.address", "10.247.44.155:18080"); > conf.set("spark.history.fs.logDirectory", > " > hdfs://myHdfsNameNode:55310/scratch/tie/spark/applicationHistory"); > > JavaSparkContext sc = new JavaSparkContext(conf); > > try { > > ... my application code > > }finally{ > sc.stop(); > > } > > Anything I did wrong or missed? Do I need to configure something on Yarn > side? > > Thanks > > -- > --Anfernee >