[
https://issues.apache.org/jira/browse/SPARK-38075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Holden Karau updated SPARK-38075:
---------------------------------
Fix Version/s: 3.1.3
(was: 3.1.4)
> Hive script transform with order by and limit will return fake rows
> -------------------------------------------------------------------
>
> Key: SPARK-38075
> URL: https://issues.apache.org/jira/browse/SPARK-38075
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 3.1.2, 3.2.0, 3.1.3, 3.2.1, 3.3.0
> Reporter: Bruce Robbins
> Assignee: Bruce Robbins
> Priority: Major
> Labels: correctness
> Fix For: 3.1.3, 3.3.0, 3.2.2
>
>
> For example:
> {noformat}
> create or replace temp view t as
> select * from values
> (1),
> (2),
> (3)
> as t(a);
> select transform(a)
> USING 'cat' AS (a int)
> FROM t order by a limit 10;
> {noformat}
> This returns:
> {noformat}
> NULL
> NULL
> NULL
> 1
> 2
> 3
> {noformat}
> Without {{order by}} and {{limit}}, the query returns:
> {noformat}
> 1
> 2
> 3
> {noformat}
> Spark script transform does not have this issue. That is, if
> {{spark.sql.catalogImplementation=in-memory}}, Spark does not return fake
> rows.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
