GitHub user mgaido91 opened a pull request:
https://github.com/apache/spark/pull/21072
[SPARK-23973][SQL] Remove consecutive Sorts
## What changes were proposed in this pull request?
In SPARK-23375 we introduced the ability of removing `Sort` operation
during query optimization if the data is already sorted. In this follow-up we
remove also a `Sort` which is followed by another `Sort`: in this case the
first sort is not needed and can be safely removed.
The PR starts from @henryr's comment:
https://github.com/apache/spark/pull/20560#discussion_r180601594. So credit
should be given to him.
## How was this patch tested?
added UT
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mgaido91/spark SPARK-23973
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/21072.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #21072
----
commit 6ba418619dcfd139b6b3fa81cf8a29d54ca62681
Author: Marco Gaido <marcogaido91@...>
Date: 2018-04-14T10:11:27Z
[SPARK-23973][SQL] Remove consecutive Sorts
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
