[
https://issues.apache.org/jira/browse/SPARK-16595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15390162#comment-15390162
]
Weiqing Yang commented on SPARK-16595:
--------------------------------------
This issue is not reproduced.
> Spark History server Rest Api gives Application not found error for
> yarn-cluster mode
> -------------------------------------------------------------------------------------
>
> Key: SPARK-16595
> URL: https://issues.apache.org/jira/browse/SPARK-16595
> Project: Spark
> Issue Type: Bug
> Affects Versions: 1.6.1
> Reporter: Yesha Vora
>
> Scenario:
> * Start SparkPi application in Spark1 using yarn-cluster mode
> (application_1468686376753_0041)
> * After application finishes validate application exists in respective Spark
> History server.
> {code}
> Error loading url
> http://xx.xx.xx.xx:18080/api/v1/applications/application_1468686376753_0041/1/executors
> HTTP Code: 404
> HTTP Data: no such app: application_1468686376753_0041{code}
> {code:title=spark HS log}
> 16/07/16 15:55:29 INFO FsHistoryProvider: Replaying log path:
> hdfs://xx.xx.xx.xx:8020/spark-history/application_1468678823755_0049.inprogress
> 16/07/16 15:56:20 INFO FsHistoryProvider: Replaying log path:
> hdfs://xx.xx.xx.xx:8020/spark-history/application_1468678823755_0049
> 16/07/16 16:23:14 INFO FsHistoryProvider: Replaying log path:
> hdfs://xx.xx.xx.xx:8020/spark-history/application_1468678823755_0061.inprogress
> 16/07/16 16:24:14 INFO FsHistoryProvider: Replaying log path:
> hdfs://xx.xx.xx.xx:8020/spark-history/application_1468678823755_0061
> 16/07/16 17:42:32 INFO FsHistoryProvider: Replaying log path:
> hdfs://xx.xx.xx.xx:8020/spark-history/local-1468690940553.inprogress
> 16/07/16 17:43:22 INFO FsHistoryProvider: Replaying log path:
> hdfs://xx.xx.xx.xx:8020/spark-history/local-1468690940553
> 16/07/16 17:43:44 INFO FsHistoryProvider: Replaying log path:
> hdfs://xx.xx.xx.xx:8020/spark-history/local-1468691017376.inprogress
> 16/07/16 17:44:34 INFO FsHistoryProvider: Replaying log path:
> hdfs://xx.xx.xx.xx:8020/spark-history/local-1468691017376
> 16/07/16 18:53:10 INFO FsHistoryProvider: Replaying log path:
> hdfs://xx.xx.xx.xx:8020/spark-history/application_1468686376753_0041_1.inprogress
> 16/07/16 19:03:26 INFO PackagesResourceConfig: Scanning for root resource and
> provider classes in the packages:
> org.apache.spark.status.api.v1
> 16/07/16 19:03:35 INFO ScanningResourceConfig: Root resource classes found:
> class org.apache.spark.status.api.v1.ApiRootResource
> 16/07/16 19:03:35 INFO ScanningResourceConfig: Provider classes found:
> class org.apache.spark.status.api.v1.JacksonMessageWriter
> 16/07/16 19:03:35 INFO WebApplicationImpl: Initiating Jersey application,
> version 'Jersey: 1.9 09/02/2011 11:17 AM'
> 16/07/16 19:03:36 INFO SecurityManager: Changing view acls to: spark
> 16/07/16 19:03:36 INFO SecurityManager: Changing modify acls to: spark
> 16/07/16 19:03:36 INFO SecurityManager: SecurityManager: authentication
> disabled; ui acls disabled; users with view permissions: Set(spark); users
> with modify permissions: Set(spark)
> 16/07/16 19:03:36 INFO ApplicationCache: Failed to load application attempt
> application_1468686376753_0041/Some(1)
> 16/07/16 19:04:21 INFO FsHistoryProvider: Replaying log path:
> hdfs://xx.xx.xx.xx:8020/spark-history/application_1468686376753_0043.inprogress
> 16/07/16 19:12:02 INFO FsHistoryProvider: Replaying log path:
> hdfs://xx.xx.xx.xx:8020/spark-history/application_1468686376753_0043
> 16/07/16 19:16:11 INFO SecurityManager: Changing view acls to: spark
> 16/07/16 19:16:11 INFO SecurityManager: Changing modify acls to: spark
> 16/07/16 19:16:11 INFO SecurityManager: SecurityManager: authentication
> disabled; ui acls disabled; users with view permissions: Set(spark); users
> with modify permissions: Set(spark)
> 16/07/16 19:16:11 INFO FsHistoryProvider: Replaying log path:
> hdfs://xx.xx.xx.xx:8020/spark-history/application_1468686376753_0043
> 16/07/16 19:16:22 INFO SecurityManager: Changing acls enabled to: false
> 16/07/16 19:16:22 INFO SecurityManager: Changing admin acls to:
> 16/07/16 19:16:22 INFO SecurityManager: Changing view acls to: hrt_qa{code}
> {code}
> hdfs@xxx:/var/log/spark$ hdfs dfs -ls /spark-history/
> Found 8 items
> -rwxrwx--- 3 hrt_qa hadoop 28793 2016-07-16 15:56
> /spark-history/application_1468678823755_0049
> -rwxrwx--- 3 hrt_qa hadoop 28763 2016-07-16 16:24
> /spark-history/application_1468678823755_0061
> -rwxrwx--- 3 hrt_qa hadoop 58868885 2016-07-16 18:59
> /spark-history/application_1468686376753_0041_1
> -rwxrwx--- 3 hrt_qa hadoop 58841982 2016-07-16 19:11
> /spark-history/application_1468686376753_0043
> -rwxrwx--- 3 hive hadoop 5823 2016-07-16 11:38
> /spark-history/local-1468666932940
> -rwxrwx--- 3 hive hadoop 5757 2016-07-16 22:44
> /spark-history/local-1468669677840.inprogress
> -rwxrwx--- 3 hrt_qa hadoop 484113 2016-07-16 17:43
> /spark-history/local-1468690940553
> -rwxrwx--- 3 hrt_qa hadoop 57747 2016-07-16 17:44
> /spark-history/local-1468691017376
> hdfs@xxx:/var/log/spark$ hdfs dfs -ls
> /spark-history/application_1468686376753_0041_1
> -rwxrwx--- 3 hrt_qa hadoop 58868885 2016-07-16 18:59
> /spark-history/application_1468686376753_0041_1{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
