Josh Rosen created SPARK-9702: --------------------------------- Summary: Repartition operator should use Exchange to perform its shuffle Key: SPARK-9702 URL: https://issues.apache.org/jira/browse/SPARK-9702 Project: Spark Issue Type: Improvement Components: SQL Reporter: Josh Rosen
Spark SQL's {{Repartition}} operator is implemented in terms of Spark Core's repartition operator, which means that it has to perform lots of unnecessary row copying and inefficient row serialization. Instead, it would be better if this was implemented using some of Exchange's internals so that it can avoid row format conversions and generic getters / hashcodes. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org