holdenk opened a new pull request, #58393:
URL: https://github.com/apache/spark/pull/58393
### What changes were proposed in this pull request?
Backport of #51533 to `branch-3.5`, plus its comment-only followup #51620.
The original change is SPARK-52832, `2bc575785465de53376b075b35a23a318d0fcd93`;
the followup is `47b08a0e9588955aa389a5f21d0f994765a04fec`.
- `JdbcDialect.quoteIdentifier` escapes a double quote in the identifier by
doubling it, per the ANSI standard.
- `MySQLDialect.quoteIdentifier` escapes a backtick by doubling it, per the
MySQL manual.
The original authors are carried through as authors of this commit. Any
mistake in adapting it to `branch-3.5` is mine, not theirs.
### Backport notes
The cherry-pick does not apply cleanly. Three deviations from a verbatim
pick:
- `DatabricksDialect` does not exist on `branch-3.5` (it arrives in 4.0.0
with SPARK-45139), so its `quoteIdentifier` hunk is dropped.
- The `databricksDialect` assertions come out of the `JDBCSuite` test for
the same reason. They could not simply be left in place: `branch-3.5` registers
no Databricks dialect, so `jdbc:databricks://host/db` resolves to `NoopDialect`
and emits ANSI double quotes, which fails the upstream backtick expectation.
- The `MySQLDialect` comment is the post-followup text, folding in #51620,
which deleted a trailing sentence that did not read sensibly without the
screenshot from the MySQL docs. This makes `branch-3.5` match `master` rather
than `branch-4.0`, which took the fix but not the followup.
The two main-code hunks are otherwise byte-identical to upstream.
### Why are the changes needed?
An identifier that contains the dialect's own quote character terminates the
quoted identifier early, so the remainder of the name is emitted as SQL text
instead of as part of the identifier, and the generated statement is wrong.
`quoteIdentifier` is what every `branch-3.5` dialect uses to quote column,
table, schema and index names in generated SQL. They all inherit the base
implementation except `MySQLDialect`, fixed here too, and `AggregatedDialect`,
which delegates to `dialects.head`. So these two hunks cover the branch.
### Does this PR introduce _any_ user-facing change?
Yes, the same changes upstream made:
- Identifiers containing a double quote, or a backtick under `MySQLDialect`,
are now quoted correctly rather than producing a malformed statement.
- `JDBCRelation.verifyAndGetNormalizedPartitionColumn` matches the
`partitionColumn` option against `dialect.quoteIdentifier(f.name)`, so a caller
who worked around the old behaviour by passing a naively quoted name for a
column that itself contains a quote (field `a"b`, option `"a"b"`) now has to
pass `"a""b"`. No test covers this on any branch.
### How was this patch tested?
`JDBCSuite`, "quote column names by jdbc dialect", which the original commit
extended to cover a double quote and a backtick in the identifier -- here for
the MySQL, Postgres, Derby and Oracle dialects. It fails on `branch-3.5`
without the main-code change:
Expected List("`abc`", "`key`", "`double_quote"`", "`back```"),
but got List("`abc`", "`key`", "`double_quote"`", "`back``")
Green on this branch with the change, JDK 17, Scala 2.12:
JDBCSuite 105 tests
JDBCWriteSuite 32 tests, 1 ignored
JDBCV2Suite 70 tests
The docker integration suites were not run.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Opus 5
Lead-authored-by: Uros Stankovic <[email protected]>
(cherry picked from commit 2bc575785465de53376b075b35a23a318d0fcd93)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
