[
https://issues.apache.org/jira/browse/SPARK-52217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mihailo Aleksic updated SPARK-52217:
------------------------------------
Description:
Validation for outer references in subquery expressions is done in the bottom
up manner in single-pass resolver. In other words, we do the validation right
after we resolve the subquery expression and at that moment missing attributes
haven't been pushed down yet.
Because of that, we fail with
UNSUPPORTED_SUBQUERY_EXPRESSION_CATEGORY.MUST_AGGREGATE_CORRELATED_SCALAR_SUBQUERY.
With the validation disabled, we fail with UNRESOLVED_COLUMN.WITH_SUGGESTION
on the unhappy path and pass on the happy path which was intended.
was:Validation for outer references in subquery expressions is done in the
bottom up manner in single-pass resolver. In other words, we do the validation
right after we resolve the subquery expression and at that moment missing
attributes haven't been pushed down yet. Because of that, we fail with
UNSUPPORTED_SUBQUERY_EXPRESSION_CATEGORY.MUST_AGGREGATE_CORRELATED_SCALAR_SUBQUERY.
With the validation disabled, we fail with UNRESOLVED_COLUMN.WITH_SUGGESTION
on the unhappy path and pass on the happy path which was intended.
> Skip outer reference validation under Filter in single-pass resolver
> --------------------------------------------------------------------
>
> Key: SPARK-52217
> URL: https://issues.apache.org/jira/browse/SPARK-52217
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 4.1.0
> Reporter: Mihailo Aleksic
> Priority: Major
>
> Validation for outer references in subquery expressions is done in the bottom
> up manner in single-pass resolver. In other words, we do the validation right
> after we resolve the subquery expression and at that moment missing
> attributes haven't been pushed down yet.
> Because of that, we fail with
> UNSUPPORTED_SUBQUERY_EXPRESSION_CATEGORY.MUST_AGGREGATE_CORRELATED_SCALAR_SUBQUERY.
> With the validation disabled, we fail with UNRESOLVED_COLUMN.WITH_SUGGESTION
> on the unhappy path and pass on the happy path which was intended.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
