GitHub user jerryshao opened a pull request:
https://github.com/apache/spark/pull/20315
[SPARK-23147][UI] Fix task page table IndexOutOfBound Exception
## What changes were proposed in this pull request?
Stage's task page table will throw an exception when there's no complete
tasks. Furthermore, because the `dataSize` doesn't take running tasks into
account, so sometimes UI cannot show the running tasks. Besides table will only
be displayed when first task is finished according to the default
sortColumn("index").
To reproduce this issue, user could try `sc.parallelize(1 to 20, 20).map {
i => Thread.sleep(10000); i }.collect()` or `sc.parallelize(1 to 20, 20).map {
i => Thread.sleep((20 - i) * 1000); i }.collect` to reproduce the above issue.
Here propose a solution to fix it. Not sure if it is a right fix, please
help to review.
## How was this patch tested?
Manual test.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jerryshao/apache-spark SPARK-23147
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/20315.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #20315
----
commit 568aa847745431f62d79147f80731aa7cb7945b1
Author: jerryshao <sshao@...>
Date: 2018-01-18T13:09:03Z
Fix task page table exception
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
