Repository: spark Updated Branches: refs/heads/branch-1.6 3c4938e26 -> c47a7373a
[SPARK-12090] [PYSPARK] consider shuffle in coalesce() Author: Davies Liu <dav...@databricks.com> Closes #10090 from davies/fix_coalesce. (cherry picked from commit 4375eb3f48fc7ae90caf6c21a0d3ab0b66bf4efa) Signed-off-by: Davies Liu <davies....@gmail.com> Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/c47a7373 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/c47a7373 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/c47a7373 Branch: refs/heads/branch-1.6 Commit: c47a7373a88f49c77b8d65e887cac2ef1ae22eae Parents: 3c4938e Author: Davies Liu <dav...@databricks.com> Authored: Tue Dec 1 22:41:48 2015 -0800 Committer: Davies Liu <davies....@gmail.com> Committed: Tue Dec 1 22:42:03 2015 -0800 ---------------------------------------------------------------------- python/pyspark/rdd.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/c47a7373/python/pyspark/rdd.py ---------------------------------------------------------------------- diff --git a/python/pyspark/rdd.py b/python/pyspark/rdd.py index 4b4d596..00bb9a6 100644 --- a/python/pyspark/rdd.py +++ b/python/pyspark/rdd.py @@ -2015,7 +2015,7 @@ class RDD(object): >>> sc.parallelize([1, 2, 3, 4, 5], 3).coalesce(1).glom().collect() [[1, 2, 3, 4, 5]] """ - jrdd = self._jrdd.coalesce(numPartitions) + jrdd = self._jrdd.coalesce(numPartitions, shuffle) return RDD(jrdd, self.ctx, self._jrdd_deserializer) def zip(self, other): --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org