hudi-agent commented on code in PR #20305:
URL: https://github.com/apache/hudi/pull/20305#discussion_r4259019443
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/cdc/CDCFileGroupIterator.scala:
##########
@@ -664,7 +667,7 @@ class CDCFileGroupIterator(split: HoodieCDCFileGroupSplit,
if (record == null) {
null
} else {
- convertToUTF8String(HoodieCDCUtils.recordToJson(record))
+
cdcImageConverter.convert(inlineCdcImageDeserializer.deserialize(record).get.asInstanceOf[InternalRow])
Review Comment:
🤖 nit: could you split this line into a local `row` val before calling
`cdcImageConverter.convert`? The `.get.asInstanceOf[InternalRow]` chain is hard
to scan.
<sub><i>⚠️ AI-generated; verify before applying. React 👍/👎 to flag
quality.</i></sub>
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/cdc/InternalRowToJsonStringConverter.scala:
##########
@@ -95,6 +99,11 @@ class InternalRowToJsonStringConverter(schema: StructType) {
structMap.toMap
case _ => value // fallback
}
+ case _: DecimalType => value.asInstanceOf[Decimal].toJavaBigDecimal
+ case DateType => LocalDate.ofEpochDay(value.asInstanceOf[Int]).toString
+ case TimestampType =>
DateTimeUtils.microsToInstant(value.asInstanceOf[Long]).toString
+ case dt if dt.typeName ==
InternalRowToJsonStringConverter.TIMESTAMP_NTZ_TYPE_NAME =>
+
DateTimeUtils.microsToLocalDateTime(value.asInstanceOf[Long]).format(DateTimeFormatter.ISO_LOCAL_DATE_TIME)
Review Comment:
🤖 nit: could you add a short comment on why `timestamp_ntz` is matched by
type name (Spark 3 lacks the type), as the variant branch does?
<sub><i>⚠️ AI-generated; verify before applying. React 👍/👎 to flag
quality.</i></sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
