[
https://issues.apache.org/jira/browse/SPARK-61296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Utkarsh Bajaj updated SPARK-61296:
----------------------------------
Component/s: SQL
(was: Shuffle)
> Return CollectLimitExec gather results in partition order
> ---------------------------------------------------------
>
> Key: SPARK-61296
> URL: https://issues.apache.org/jira/browse/SPARK-61296
> Project: Spark
> Issue Type: Bug
> Components: Spark Core, SQL
> Affects Versions: 4.0.5
> Reporter: Utkarsh Bajaj
> Priority: Major
>
> CollectLimitExec moves all sorted partitions into one partition/task through
> a shuffle, and then keeps the rows between OFFSET and LIMIT. The shuffle
> hands back the partitions in the order they arrive, not in partition order.
> So the rows can come back out of order, and OFFSET can skip the wrong rows
> for order by queries.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]