[
https://issues.apache.org/jira/browse/SPARK-58782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-58782:
-----------------------------------
Labels: pull-request-available (was: )
> DSv2 pushdown serializes a null literal as the string `'null'` instead of SQL
> `NULL`, returning wrong results
> -------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-58782
> URL: https://issues.apache.org/jira/browse/SPARK-58782
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 4.0.0
> Environment: Environment independent
> Reporter: Tim Meehan
> Priority: Major
> Labels: pull-request-available
>
> Problem When a null literal appears in a pushed-down DSv2 expression (JDBC
> aggregate/predicate pushdown), it is serialized into the pushed SQL as the
> 4-character string literal {{'null'}} instead of the SQL keyword
> {{{}NULL{}}}. The external engine then executes a different
> predicate/grouping key than intended and silently returns wrong results.
> Because a connector {{{}Expression{}}}'s {{describe()}} defaults to
> {{{}toString(){}}}, the same wrong text also appears in {{EXPLAIN}} output.
> Reproduction (H2, self-contained — no external DB):
> scala
>
> // table null_literal(s TEXT), rows: 'keep', '', NULL;
> pushDownAggregate=truesql("SELECT NULLIF(s, '') AS g, COUNT(*) FROM
> h2.test.null_literal GROUP BY g")
>
> {{NULLIF(s,'')}} rewrites to {{{}If(s='', null, s){}}}, which is pushed as a
> {{CASE_WHEN}} grouping key. The generated SQL is:
> sql
>
> SELECT CASE WHEN "S" = '' THEN 'null' ELSE "S" END, COUNT(*) FROM ... GROUP
> BY CASE WHEN "S" = '' THEN 'null' ELSE "S" END
> The {{''}} row is bucketed under the string {{"null"}} instead of collapsing
> into the real {{NULL}} group — 3 result rows instead of 2.
> Root cause {{LiteralValue.toString}} matches on {{dataType}} before checking
> the value, so a null {{StringType}} value flows into the {{'…'}} string
> branch and renders as {{{}'null'{}}}. (A null {{BinaryType}} value hits an
> {{assert}} and throws; numeric/boolean nulls render as an unquoted
> {{{}null{}}}.)
> Proposed fix In {{{}LiteralValue.toString{}}}, return the bare keyword
> {{NULL}} for any null value (guard as the first {{{}case{}}}). A null literal
> is SQL {{NULL}} regardless of type, so this is dialect-agnostic and fixes
> every consumer (JDBC dialects, {{{}ToStringSQLBuilder{}}}/EXPLAIN) at once.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
