stczwd commented on a change in pull request #26946: [SPARK-30036][SQL] Fix:
REPARTITION hint does not work with order by
URL: https://github.com/apache/spark/pull/26946#discussion_r360788805
##########
File path:
sql/core/src/test/scala/org/apache/spark/sql/ConfigBehaviorSuite.scala
##########
@@ -39,9 +39,8 @@ class ConfigBehaviorSuite extends QueryTest with
SharedSparkSession {
def computeChiSquareTest(): Double = {
val n = 10000
// Trigger a sort
- // Range has range partitioning in its output now. To have a range
shuffle, we
- // need to run a repartition first.
- val data = spark.range(0, n, 1, 1).repartition(10).sort($"id".desc)
+ // Range has range partitioning in its output now.
+ val data = spark.range(0, n, 1, 10).sort($"id".desc)
Review comment:
```
== Optimized Logical Plan ==
Sort [id#8L DESC NULLS LAST], true
+- Range (0, 100, step=1, splits=Some(10))
== Physical Plan ==
*(2) Sort [id#8L DESC NULLS LAST], true, 0
+- Exchange rangepartitioning(id#8L DESC NULLS LAST, 4), true, [id=#33]
+- *(1) Range (0, 100, step=1, splits=10)
```
The sort api will add global=true in SortExec, similar with orderby, which
will cause SortExec's requiredChildDistribution to be parsed into
OrderedDistribution.
EnsureRequirements will add a ShuffleExchangeExec with rangePartitioning
after OrderedDistribution.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
