[ https://issues.apache.org/jira/browse/SPARK-4845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Owen resolved SPARK-4845. ------------------------------ Resolution: Won't Fix Fix Version/s: (was: 1.3.0) Target Version/s: (was: 1.3.0) See PR discussion. > Adding a parallelismRatio to control the partitions num of shuffledRDD > ---------------------------------------------------------------------- > > Key: SPARK-4845 > URL: https://issues.apache.org/jira/browse/SPARK-4845 > Project: Spark > Issue Type: Improvement > Components: Spark Core > Affects Versions: 1.1.0 > Reporter: Fei Wang > > Adding parallelismRatio to control the partitions num of shuffledRDD, the > rule is: > Math.max(1, parallelismRatio * number of partitions of the largest upstream > RDD) > The ratio is 1.0 by default to make it compatible with the old version. > When we have a good experience on it, we can change this. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org