Bruce Robbins created SPARK-38075:
-------------------------------------
Summary: Hive script transform with order by and limit will return
fake rows
Key: SPARK-38075
URL: https://issues.apache.org/jira/browse/SPARK-38075
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 3.3.0
Reporter: Bruce Robbins
For example:
{noformat}
create or replace temp view t as
select * from values
(1),
(2),
(3)
as t(a);
select transform(a)
USING 'cat' AS (a int)
FROM t order by a limit 10;
{noformat}
This returns:
{noformat}
NULL
NULL
NULL
1
2
3
{noformat}
Without {{order by}} and {{limit}}, the query returns:
{noformat}
1
2
3
{noformat}
Spark script transform does not have this issue. That is, if
{{spark.sql.catalogImplementation=in-memory}}, Spark does not return fake rows.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
