[ https://issues.apache.org/jira/browse/SPARK-16595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyukjin Kwon updated SPARK-16595: --------------------------------- Labels: bulk-closed (was: ) > Spark History server Rest Api gives Application not found error for > yarn-cluster mode > ------------------------------------------------------------------------------------- > > Key: SPARK-16595 > URL: https://issues.apache.org/jira/browse/SPARK-16595 > Project: Spark > Issue Type: Bug > Affects Versions: 1.6.1 > Reporter: Yesha Vora > Priority: Major > Labels: bulk-closed > > Scenario: > * Start SparkPi application in Spark1 using yarn-cluster mode > (application_1468686376753_0041) > * After application finishes validate application exists in respective Spark > History server. > {code} > Error loading url > http://xx.xx.xx.xx:18080/api/v1/applications/application_1468686376753_0041/1/executors > HTTP Code: 404 > HTTP Data: no such app: application_1468686376753_0041{code} > {code:title=spark HS log} > 16/07/16 15:55:29 INFO FsHistoryProvider: Replaying log path: > hdfs://xx.xx.xx.xx:8020/spark-history/application_1468678823755_0049.inprogress > 16/07/16 15:56:20 INFO FsHistoryProvider: Replaying log path: > hdfs://xx.xx.xx.xx:8020/spark-history/application_1468678823755_0049 > 16/07/16 16:23:14 INFO FsHistoryProvider: Replaying log path: > hdfs://xx.xx.xx.xx:8020/spark-history/application_1468678823755_0061.inprogress > 16/07/16 16:24:14 INFO FsHistoryProvider: Replaying log path: > hdfs://xx.xx.xx.xx:8020/spark-history/application_1468678823755_0061 > 16/07/16 17:42:32 INFO FsHistoryProvider: Replaying log path: > hdfs://xx.xx.xx.xx:8020/spark-history/local-1468690940553.inprogress > 16/07/16 17:43:22 INFO FsHistoryProvider: Replaying log path: > hdfs://xx.xx.xx.xx:8020/spark-history/local-1468690940553 > 16/07/16 17:43:44 INFO FsHistoryProvider: Replaying log path: > hdfs://xx.xx.xx.xx:8020/spark-history/local-1468691017376.inprogress > 16/07/16 17:44:34 INFO FsHistoryProvider: Replaying log path: > hdfs://xx.xx.xx.xx:8020/spark-history/local-1468691017376 > 16/07/16 18:53:10 INFO FsHistoryProvider: Replaying log path: > hdfs://xx.xx.xx.xx:8020/spark-history/application_1468686376753_0041_1.inprogress > 16/07/16 19:03:26 INFO PackagesResourceConfig: Scanning for root resource and > provider classes in the packages: > org.apache.spark.status.api.v1 > 16/07/16 19:03:35 INFO ScanningResourceConfig: Root resource classes found: > class org.apache.spark.status.api.v1.ApiRootResource > 16/07/16 19:03:35 INFO ScanningResourceConfig: Provider classes found: > class org.apache.spark.status.api.v1.JacksonMessageWriter > 16/07/16 19:03:35 INFO WebApplicationImpl: Initiating Jersey application, > version 'Jersey: 1.9 09/02/2011 11:17 AM' > 16/07/16 19:03:36 INFO SecurityManager: Changing view acls to: spark > 16/07/16 19:03:36 INFO SecurityManager: Changing modify acls to: spark > 16/07/16 19:03:36 INFO SecurityManager: SecurityManager: authentication > disabled; ui acls disabled; users with view permissions: Set(spark); users > with modify permissions: Set(spark) > 16/07/16 19:03:36 INFO ApplicationCache: Failed to load application attempt > application_1468686376753_0041/Some(1) > 16/07/16 19:04:21 INFO FsHistoryProvider: Replaying log path: > hdfs://xx.xx.xx.xx:8020/spark-history/application_1468686376753_0043.inprogress > 16/07/16 19:12:02 INFO FsHistoryProvider: Replaying log path: > hdfs://xx.xx.xx.xx:8020/spark-history/application_1468686376753_0043 > 16/07/16 19:16:11 INFO SecurityManager: Changing view acls to: spark > 16/07/16 19:16:11 INFO SecurityManager: Changing modify acls to: spark > 16/07/16 19:16:11 INFO SecurityManager: SecurityManager: authentication > disabled; ui acls disabled; users with view permissions: Set(spark); users > with modify permissions: Set(spark) > 16/07/16 19:16:11 INFO FsHistoryProvider: Replaying log path: > hdfs://xx.xx.xx.xx:8020/spark-history/application_1468686376753_0043 > 16/07/16 19:16:22 INFO SecurityManager: Changing acls enabled to: false > 16/07/16 19:16:22 INFO SecurityManager: Changing admin acls to: > 16/07/16 19:16:22 INFO SecurityManager: Changing view acls to: hrt_qa{code} > {code} > hdfs@xxx:/var/log/spark$ hdfs dfs -ls /spark-history/ > Found 8 items > -rwxrwx--- 3 hrt_qa hadoop 28793 2016-07-16 15:56 > /spark-history/application_1468678823755_0049 > -rwxrwx--- 3 hrt_qa hadoop 28763 2016-07-16 16:24 > /spark-history/application_1468678823755_0061 > -rwxrwx--- 3 hrt_qa hadoop 58868885 2016-07-16 18:59 > /spark-history/application_1468686376753_0041_1 > -rwxrwx--- 3 hrt_qa hadoop 58841982 2016-07-16 19:11 > /spark-history/application_1468686376753_0043 > -rwxrwx--- 3 hive hadoop 5823 2016-07-16 11:38 > /spark-history/local-1468666932940 > -rwxrwx--- 3 hive hadoop 5757 2016-07-16 22:44 > /spark-history/local-1468669677840.inprogress > -rwxrwx--- 3 hrt_qa hadoop 484113 2016-07-16 17:43 > /spark-history/local-1468690940553 > -rwxrwx--- 3 hrt_qa hadoop 57747 2016-07-16 17:44 > /spark-history/local-1468691017376 > hdfs@xxx:/var/log/spark$ hdfs dfs -ls > /spark-history/application_1468686376753_0041_1 > -rwxrwx--- 3 hrt_qa hadoop 58868885 2016-07-16 18:59 > /spark-history/application_1468686376753_0041_1{code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org