Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/23228#discussion_r239215561
--- Diff:
core/src/main/scala/org/apache/spark/shuffle/sort/SortShuffleManager.scala ---
@@ -33,10 +33,10 @@ import org.apache.spark.shuffle._
* Sort-based shuffle has two different write paths for producing its map
output files:
*
* - Serialized sorting: used when all three of the following conditions
hold:
- * 1. The shuffle dependency specifies no aggregation or output
ordering.
+ * 1. The shuffle dependency specifies no map-side combine.
--- End diff --
Does this sound right @JoshRosen ?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
