nsivabalan commented on code in PR #12511:
URL: https://github.com/apache/hudi/pull/12511#discussion_r1902021433
##########
hudi-common/src/main/java/org/apache/hudi/avro/HoodieAvroUtils.java:
##########
@@ -1487,4 +1488,20 @@ public static Comparable<?>
unwrapAvroValueWrapper(Object avroValueWrapper) {
throw new UnsupportedOperationException(String.format("Unsupported type
of the value (%s)", avroValueWrapper.getClass()));
}
}
+
+ public static Comparable<?> unwrapAvroValueWrapper(Object avroValueWrapper,
String wrapperClassName) {
+ if (avroValueWrapper == null) {
+ return null;
+ } else if (DateWrapper.class.getSimpleName().equals(wrapperClassName)) {
+ return LocalDate.ofEpochDay((Integer)((Record) avroValueWrapper).get(0));
+ } else if
(TimestampMicrosWrapper.class.getSimpleName().equals(wrapperClassName)) {
+ Instant instant = microsToInstant((Long)((Record)
avroValueWrapper).get(0));
Review Comment:
here is the reason why we have the special handling of obfuscated detection.
If we take a look at how timestamp is serialized, in L1499, its a long value.
So, the value object itself is of type Long, but we wrap it using
TimestampWrapper object.
but on the reader side, i.e. unwrap method, we only use the object value to
deduce the wrapper class.
```
public static Comparable<?> unwrapAvroValueWrapper(Object avroValueWrapper) {
.
.
} else if (avroValueWrapper instanceof TimestampMicrosWrapper) {
.
.
}
```
So, since the value is Long, (the schema says its TimestampMicrosWrapper,
but the value is of `Long` type), w/o the special handling w/
unwrapAvroValueWrapper(), we might try to deser as LongWrapper instead of
TimestampMicrosWrapper.
Hence, for few data types, where the value type does not match the wrapper
class (DateWrapper, Timestamp), we have to deduce the wrapper class first based
on schema and then call unwrap based on that. I understand, it may not be
apparent, and it took me sometime to get to the bottom of this.
TimestampMicrosWrapper stores Long value.
DateWrapper stores Int value
DecimalWrapper stores ByteBuffer
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
