Aleksandar Trajković created SPARK-59051:
--------------------------------------------
Summary: JDBC partition bounds should be compiled via the dialect
instead of emitted as plain ISO literals
Key: SPARK-59051
URL: https://issues.apache.org/jira/browse/SPARK-59051
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 4.4.0
Reporter: Aleksandar Trajković
Follow-up to SPARK-58825.
JDBCRelation.toBoundValueInWhereClause renders JDBC partition-column bounds
(DateType, TimestampType, TimestampNTZType) as bare quoted ISO-8601 string
literals that are spliced directly into the generated partition WHERE clauses,
bypassing JdbcDialect.compileValue.
This assumes every target database accepts ISO-8601 date/time literals in
predicates, which is not universally true. For example, Oracle's default
NLS_TIMESTAMP_FORMAT / NLS_DATE_FORMAT reject the ISO literal; today this is
only worked around at the test level via a sessionInitStatement that sets an
ISO NLS format. Pushing down raw literal text like this is dialect-dependent
and risky.
Proposal: route partition-bound literals through the dialect (e.g.
JdbcDialect.compileValue, or a dedicated hook) so each connector produces a
valid literal for its SQL dialect. This should cover all temporal
partition-column types (DateType, TimestampType, TimestampNTZType), not just
the TimestampNTZType support added in SPARK-58825.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
