Allison Wang created SPARK-41441:
------------------------------------
Summary: Allow Generate with no required child output to host
outer references
Key: SPARK-41441
URL: https://issues.apache.org/jira/browse/SPARK-41441
Project: Spark
Issue Type: Sub-task
Components: SQL
Affects Versions: 3.4.0
Reporter: Allison Wang
Currently, in CheckAnalysis, Spark disallows Generate to host any outer
references when it's required child output is not empty. But when the child
output is empty, it can host outer references, which DecorrelateInnerQuery does
not handle.
For example,
{code:java}
select * from t, lateral (select explode(array(c1, c2))){code}
This throws an internal error :
{code:java}
Caused by: java.lang.AssertionError: assertion failed: Correlated column is not
allowed in Generate explode(array(outer(c1#219), outer(c2#220))), false,
[col#221] +- OneRowRelation{code}
We should support Generate to host outer references when its required child
output is empty.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
