[
https://issues.apache.org/jira/browse/SPARK-2017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14054307#comment-14054307
]
Reynold Xin commented on SPARK-2017:
------------------------------------
We gotta be careful with pagination because right now the web UI allows sorting
via some client-side JavaScript. If we enable pagination, we need to make sure
the sorting is global across pages (i.e. server side).
This might go overboard, but I think rendering is the bottleneck. If that's
indeed the case, maybe the right solution is for the server to just send all
the data in JSON, and the client side decides what to render (pagination, etc)
...
> web ui stage page becomes unresponsive when the number of tasks is large
> ------------------------------------------------------------------------
>
> Key: SPARK-2017
> URL: https://issues.apache.org/jira/browse/SPARK-2017
> Project: Spark
> Issue Type: Sub-task
> Reporter: Reynold Xin
> Labels: starter
>
> {code}
> sc.parallelize(1 to 1000000, 1000000).count()
> {code}
> The above code creates one million tasks to be executed. The stage detail web
> ui page takes forever to load (if it ever completes).
> There are again a few different alternatives:
> 0. Limit the number of tasks we show.
> 1. Pagination
> 2. By default only show the aggregate metrics and failed tasks, and hide the
> successful ones.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
