[
https://issues.apache.org/jira/browse/SPARK-7889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14590215#comment-14590215
]
Steve Loughran commented on SPARK-7889:
---------------------------------------
Is JIRA about
(a) the status on the listing of complete/uncomplete being wrong in some way
(b) the actual job view (history/some-app-id) being stale when a job completes.
(b) is consistent with what I observed in SPARK-8275
Looking at your patch, and comparing it with my proposal, I prefer mine. All
I'm proposing is invalidating the cache on work in progress, so that it is
retrieved again.
Thinking about it some more, we can go one better: rely on the
{{ApplicationHistoryInfo.lastUpdated}} field to tell us when the UI was last
updated. If we cache the update time with the UI, on any GET of an appUI, we
can look to see if the previous UI was not completed and if the lastupdated
time has changed...if so. that triggers a refresh.
with this approach the entry you see will always be the one most recently
published to the history store (of any implementation), and picked up by the
history provider in its getListing()/background refresh operation.
> Jobs progress of apps on complete page of HistoryServer shows uncompleted
> -------------------------------------------------------------------------
>
> Key: SPARK-7889
> URL: https://issues.apache.org/jira/browse/SPARK-7889
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Reporter: meiyoula
> Priority: Minor
>
> When running a SparkPi with 2000 tasks, cliking into the app on incomplete
> page, the job progress shows 400/2000. After the app is completed, the app
> goes to complete page from incomplete, and now cliking into the app, the job
> progress still shows 400/2000.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
