uros-b commented on code in PR #57698:
URL: https://github.com/apache/spark/pull/57698#discussion_r3699759041
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JdbcUtils.scala:
##########
@@ -332,7 +353,8 @@ object JdbcUtils extends Logging with SQLConfHelper {
val columnType =
dialect.getCatalystType(dataType, typeName, fieldSize,
metadata).getOrElse(
- getCatalystType(dataType, typeName, fieldSize, fieldScale, isSigned,
isTimestampNTZ))
+ getCatalystType(dataType, typeName, fieldSize, fieldScale, isSigned,
isTimestampNTZ,
+ preferTimestampNanos))
Review Comment:
preferTimestampNanos is passed as a fallback parameter to getCatalystType
but is not stored in the schema metadata (unlike isTimestampNTZ at line 328,
which calls metadata.putBoolean("isTimestampNTZ", ...)). Any dialect that
overrides getCatalystType will never see preferTimestampNanos and will silently
fall back to the microsecond mapping even when the user set the option. The
asymmetry is real: isTimestampNTZ was deliberately added to metadata so
dialects can honour it; preferTimestampNanos should receive the same treatment
(metadata.putBoolean("preferTimestampNanos", preferTimestampNanos) before the
dialect call). This is not a regression (the flag defaults to false), but the
feature is incompletely integrated for custom or future dialects.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
