[ https://issues.apache.org/jira/browse/SPARK-7889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15117288#comment-15117288 ]
Steve Loughran commented on SPARK-7889: --------------------------------------- I'm going to note something something problematic about probing HDFS for changes, HDFS-5478 reports that flushing fs data doesn't immediately update fs length. So detecting changes in log files saved to HDFS isn't likely to be that immediate. the SPARK-1537 timeline binding won't have this problem, nor will logs saved to other filesystems (object stores have different issues) > Jobs progress of apps on complete page of HistoryServer shows uncompleted > ------------------------------------------------------------------------- > > Key: SPARK-7889 > URL: https://issues.apache.org/jira/browse/SPARK-7889 > Project: Spark > Issue Type: Improvement > Components: Spark Core > Reporter: meiyoula > Priority: Minor > > When running a SparkPi with 2000 tasks, cliking into the app on incomplete > page, the job progress shows 400/2000. After the app is completed, the app > goes to complete page from incomplete, and now cliking into the app, the job > progress still shows 400/2000. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org