[jira] [Commented] (SPARK-8383) Spark History Server shows Last Updated as 1969/12/31 when SparkPI application completed
[ https://issues.apache.org/jira/browse/SPARK-8383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14591478#comment-14591478 ] Saisai Shao commented on SPARK-8383: Hi [~irina.easterl...@teradata.com], I tried to reproduce your issue by following your steps, seems there's no such issue as you mentioned in screenshot, all the application is correctly completed both in RM and spark history server. The only difference is my test Spark version is latest master branch, so this problem might be fixed in the later version, can you please try a newer version? > Spark History Server shows Last Updated as 1969/12/31 when SparkPI > application completed > - > > Key: SPARK-8383 > URL: https://issues.apache.org/jira/browse/SPARK-8383 > Project: Spark > Issue Type: Bug > Components: Spark Core, Web UI >Affects Versions: 1.3.1 > Environment: Spark1.3.1.2.3 >Reporter: Irina Easterling > Attachments: Spark_WrongLastUpdatedDate.png, > YARN_SparkJobCompleted.PNG > > > Spark History Server shows Last Updated as 1969/12/31 when SparkPI > application completed and Started Date is 2015/06/10 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-8383) Spark History Server shows Last Updated as 1969/12/31 when SparkPI application completed
[ https://issues.apache.org/jira/browse/SPARK-8383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14586648#comment-14586648 ] Irina Easterling commented on SPARK-8383: - Spark History Server shows Last Updated as 1969/12/31 when SparkPI application completed Steps to reproduce: 1. Install Spark thru Ambari Wizard 2. After installation run the Spark Pi Example 3. Navigate to your Spark directory: baron1:~ # cd /usr/hdp/current/spark-client/ baron1:/usr/hdp/current/spark-client # su spark spark@baron1:/usr/hdp/current/spark-client> spark-submit --verbose --class org.apache.spark.examples.SparkPi --master yarn-cluster --num-executors 3 --driver-memory 512m --executor-memory 512m --executor-cores 1 lib/spark-examples*.jar 10 4. When job completed. 5. Access to the Ambari > Spark> Spark History Server UI 6. Click on 'Show incomplete applications' link 7. View the result for completed job //Results Last Updated column shows date/time as 1969/12/31 19:00:00 (screenshot attacked) 8. Verify that Spark job completed in YARN. (screenshot attached) There also discrepancy between SparkHistory Server WebUI and YARN/ResourceManager WebUI. Spark job completed and it is shown in the YARN/Resource Manager WebUI. In the SparkHistroyServer WebUI it shows as Uncompleted. See attached screenshots. > Spark History Server shows Last Updated as 1969/12/31 when SparkPI > application completed > - > > Key: SPARK-8383 > URL: https://issues.apache.org/jira/browse/SPARK-8383 > Project: Spark > Issue Type: Bug > Components: Spark Core, Web UI >Affects Versions: 1.3.1 > Environment: Spark1.3.1.2.3 >Reporter: Irina Easterling > Attachments: Spark_WrongLastUpdatedDate.png, > YARN_SparkJobCompleted.PNG > > > Spark History Server shows Last Updated as 1969/12/31 when SparkPI > application completed and Started Date is 2015/06/10 -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
