[
https://issues.apache.org/jira/browse/SPARK-60876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-60876:
-----------------------------------
Labels: pull-request-available (was: )
> Keep the CHAR/VARCHAR metadata of an aliased outer reference
> ------------------------------------------------------------
>
> Key: SPARK-60876
> URL: https://issues.apache.org/jira/browse/SPARK-60876
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 4.4.0
> Reporter: Stevo Mitric
> Priority: Major
> Labels: pull-request-available
>
> When a LATERAL or correlated subquery returns a CHAR(n) or VARCHAR(n) column
> of the outer query, the subquery output loses the CHAR/VARCHAR type and
> becomes a plain STRING. Comparisons against it are then no longer padded, so
> a filter on a CHAR value silently returns no rows. An inner CHAR column
> returned by the same subquery is not affected.
> {code:sql}
> CREATE TABLE t (c CHAR(5), s STRING) USING parquet;
> INSERT INTO t VALUES ('ab', 'x');
> SELECT t.s FROM t, LATERAL (SELECT t.c AS k) l WHERE l.k = 'ab'; -- returns
> no rows, expected x
> SELECT t.s FROM t, LATERAL (SELECT t.c) l WHERE l.c = 'ab'; -- returns
> no rows, expected x
> SELECT t.s FROM t, LATERAL (SELECT t.*) l WHERE l.c = 'ab'; -- returns
> no rows, expected x
> {code}
> The same happens when a correlated EXISTS, IN or scalar subquery selects an
> outer CHAR column through a derived table, e.g. {{(SELECT t.c AS k FROM t2) x
> WHERE x.k = 'ab'}}. CTAS and views over such a column also store STRING
> instead of CHAR/VARCHAR.
> Cause: the analyzer wraps an outer column returned by a subquery in an
> {{Alias}}, which copies its child's metadata, where Spark keeps the raw
> CHAR/VARCHAR type. {{OuterReference}} reported empty metadata, so the type
> was lost. The explicit-alias form ({{t.c AS k}}) has always been affected.
> Outer star expansion ({{t.*}}) is affected since SPARK-55794 (4.2.0), and the
> bare form ({{t.c}}) since SPARK-58171 (4.3.0), which started aliasing those
> outer references.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
