[ https://issues.apache.org/jira/browse/SPARK-2016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14162405#comment-14162405 ]
Apache Spark commented on SPARK-2016: ------------------------------------- User 'carlosfuertes' has created a pull request for this issue: https://github.com/apache/spark/pull/1682 > rdd in-memory storage UI becomes unresponsive when the number of RDD > partitions is large > ---------------------------------------------------------------------------------------- > > Key: SPARK-2016 > URL: https://issues.apache.org/jira/browse/SPARK-2016 > Project: Spark > Issue Type: Sub-task > Reporter: Reynold Xin > Labels: starter > > Try run > {code} > sc.parallelize(1 to 100, 1000000).cache().count() > {code} > And open the storage UI for this RDD. It takes forever to load the page. > When the number of partitions is very large, I think there are a few > alternatives: > 0. Only show the top 1000. > 1. Pagination > 2. Instead of grouping by RDD blocks, group by executors -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org