GitHub user zsxwing opened a pull request:
https://github.com/apache/spark/pull/20656
[SPARK-23475][WebUI]Skipped stages should be evicted before completed stages
## What changes were proposed in this pull request?
The root cause of missing completed stages is because `cleanupStages` will
never remove skipped stages.
This PR changes the logic to always remove skipped stage first. This is
safe since the job itself contains enough information to render skipped stages
in the UI.
## How was this patch tested?
The new unit tests.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/zsxwing/spark SPARK-23475
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/20656.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #20656
----
commit 82e7dda1042438392bdc9a294e8a01baebd1e24c
Author: Shixiong Zhu <zsxwing@...>
Date: 2018-02-21T22:39:38Z
skipped stages should be evicted before completed stages
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
