codope commented on code in PR #20305:
URL: https://github.com/apache/hudi/pull/20305#discussion_r4266936190
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/cdc/InternalRowToJsonStringConverter.scala:
##########
@@ -95,6 +99,11 @@ class InternalRowToJsonStringConverter(schema: StructType) {
structMap.toMap
case _ => value // fallback
}
+ case _: DecimalType => value.asInstanceOf[Decimal].toJavaBigDecimal
+ case DateType => LocalDate.ofEpochDay(value.asInstanceOf[Int]).toString
+ case TimestampType =>
DateTimeUtils.microsToInstant(value.asInstanceOf[Long]).toString
+ case dt if dt.typeName ==
InternalRowToJsonStringConverter.TIMESTAMP_NTZ_TYPE_NAME =>
+
DateTimeUtils.microsToLocalDateTime(value.asInstanceOf[Long]).format(DateTimeFormatter.ISO_LOCAL_DATE_TIME)
Review Comment:
Thanks. I'd rather keep this one as is. TIMESTAMP_NTZ_TYPE_NAME sits next to
VARIANT_TYPE_NAME, and that constant's scaladoc already explains why this
module matches by type name instead of using the symbol or a SparkAdapter. One
small correction: Spark 3.3 does have TimestampNTZType, it just isn't public
API until 3.4. That's the same reason Spark3_3Adapter.isTimestampNTZType
matches on the class name.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
