[
https://issues.apache.org/jira/browse/SPARK-5899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Owen resolved SPARK-5899.
------------------------------
Resolution: Duplicate
> Viewing specific stage information which contains thousands of tasks will
> freak out the driver and CPU cores from where it runs
> -------------------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-5899
> URL: https://issues.apache.org/jira/browse/SPARK-5899
> Project: Spark
> Issue Type: Bug
> Components: Web UI
> Affects Versions: 1.3.0, 1.2.1
> Environment: CentOS 7, Spark Standalone
> Reporter: Mark Khaitman
> Priority: Critical
>
> If a user ever tries to view specific stage stats, for example, a repartition
> stage which involves 30000 partitions, the Web UI attempts to load every
> single task result onto a single webpage, which is completely destroying CPU
> usage on the driver, which subsequently causes the remaining tasks / jobs to
> be nearly impossible to complete.
> Ideally the task results should be paged (if not too much trouble) to prevent
> this from happening.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
