Nick Young created SPARK-46693:
----------------------------------
Summary: Inject LocalLimitExec when matching OffsetAndLimit or
LimitAndOffset
Key: SPARK-46693
URL: https://issues.apache.org/jira/browse/SPARK-46693
Project: Spark
Issue Type: Improvement
Components: Optimizer
Affects Versions: 3.5.0
Reporter: Nick Young
For queries containing both a LIMIT and an OFFSET in a subquery, physical
translation will drop the `LocalLimit` planned in the optimizer stage by
mistake; this manifests as larger than necessary shuffle sizes for
`GlobalLimitExec`. Fix to not drop this node.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
