stevomitric opened a new pull request, #57698:
URL: https://github.com/apache/spark/pull/57698
### What changes were proposed in this pull request?
Adds read and write support for the nanosecond-capable timestamp types
`TIMESTAMP_NTZ(p)` / `TIMESTAMP_LTZ(p)` (`p` in 7-9) to the built-in JDBC
datasource, reaching parity with the microsecond `TimestampType` /
`TimestampNTZType`, and removes the SPARK-57166 JDBC write rejection guardrail.
Specifically:
- `JdbcUtils.getCommonJDBCType`: maps the nanos types to SQL `TIMESTAMP(p)`
(dialects may override, e.g. to emit `TIMESTAMP(p) WITH TIME ZONE`).
- `JdbcUtils.getCatalystType` / `getSchema`: a new `preferTimestampNanos`
JDBC option maps a driver `TIMESTAMP` reporting a sub-microsecond scale (7-9)
to a nanos type, gated by the `spark.sql.timestampNanosTypes.enabled` preview
flag. The default microsecond mapping is preserved.
- `JDBCValueGetter`: `TimestampNTZNanosGetter` reads the wall-clock directly
via `getObject(LocalDateTime)` (mirroring `TimeGetter`, avoiding a zone shift);
`TimestampLTZNanosGetter` mirrors the micro `TimestampGetter` and re-attaches
the sub-microsecond digits from `java.sql.Timestamp.getNanos`. Both floor to
the column precision.
- `JdbcUtils.makeSetter`: NTZ writes the `LocalDateTime` wall-clock via
`setObject`; LTZ writes the micro `java.sql.Timestamp` and restores full nanos.
- `CreatableRelationProvider.supportsDataType` (interfaces.scala):
whitelists the nanos types so JDBC write is no longer rejected.
### Why are the changes needed?
JDBC was the last built-in file/serde datasource still rejecting nanosecond
timestamp types (umbrella SPARK-56822). This extends the feature to JDBC.
### Does this PR introduce any user-facing change?
Yes, only for the preview feature gated by
`spark.sql.timestampNanosTypes.enabled` (disabled by default). Nanos timestamp
columns can now be written to JDBC, and a driver `TIMESTAMP` can be read as a
nanos type via `option("preferTimestampNanos", "true")`.
### How was this patch tested?
Added read (NTZ/LTZ, default-micros-preserved) and write round-trip
(precisions 7-9, NTZ and LTZ) tests against H2 in `JDBCSuite` /
`JDBCWriteSuite`, replacing the SPARK-57166 rejection test. All JDBC suites
pass.
### Was this patch authored or co-authored using generative AI tooling?
Co-authored-by: Claude Code (Claude Opus 4.8)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
