Jackey Lee created SPARK-30036:
----------------------------------
Summary: REPARTITION hint dose not work with order by
Key: SPARK-30036
URL: https://issues.apache.org/jira/browse/SPARK-30036
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 3.0.0
Reporter: Jackey Lee
Example SQL: select /*+ REPARTITION(2) */ * from test order by a
== Physical Plan ==
*(1) Sort [a#0 ASC NULLS FIRST], true, 0
+- Exchange rangepartitioning(a#0 ASC NULLS FIRST, 2), true, [id=#11]
+- Exchange RoundRobinPartitioning(3)
+- Scan hive default.test [a#0, b#1], HiveTableRelation
`default`.`test`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, [a#0,
b#1|#0, b#1]
EnsureRequirements adds ShuffleExchangeExec (RangePartitioning) after Sort if
RoundRobinPartitioning behinds it. This will cause 2 shuffles, and the number
of partitions in the final stage is not the number specified by
RoundRobinPartitioning.
This patch will add a new rule to change RoundRobinPartitioning to
RangePartitioning rather than add new ShuffleExchangeExec.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
