linliu-code opened a new issue, #19899:
URL: https://github.com/apache/hudi/issues/19899
`HoodieAvroRecord.doGetOrderingValue` returns the payload's ordering value
unconditionally. A
payload built through `HoodieRecordUtils.loadPayload(String,
GenericRecord)`, the overload used
whenever a record is created without an explicit ordering value, defaults to
`OrderingValues.getDefault()`, which is `Integer` 0.
Its sibling `HoodieAvroIndexedRecord.doGetOrderingValue` instead reads the
ordering fields off the
record, so the two Avro record representations disagree on both the ordering
value and its type for
the same input.
When such a record reaches the merge path, its `Integer` 0 is compared
against the base record's
ordering value read from storage, which is typed by the ordering field. For
any ordering column
that is not an `int`, this throws:
```
java.lang.ClassCastException: class java.lang.Long cannot be cast to class
java.lang.Integer
at java.base/java.lang.Integer.compareTo(Integer.java:71)
at
org.apache.hudi.common.table.read.BufferedRecordMergerFactory.shouldKeepNewerRecord(BufferedRecordMergerFactory.java:515)
at
org.apache.hudi.common.table.read.BufferedRecordMergerFactory$EventTimeRecordMerger.finalMerge(BufferedRecordMergerFactory.java:194)
at
org.apache.hudi.common.table.read.buffer.FileGroupRecordBuffer.hasNextBaseRecord(FileGroupRecordBuffer.java:242)
at
org.apache.hudi.io.FileGroupReaderBasedMergeHandle.doMerge(FileGroupReaderBasedMergeHandle.java:274)
```
**How to reproduce.** A copy on write table with a `bigint` precombine
field, updated through a
prepped Spark SQL write (`UPDATE` / `MERGE INTO`, which set
`_hoodie.spark.sql.writes.prepped`).
Those writes leave `shouldCombine` false, so `HoodieCreateRecordUtils` takes
the
`createHoodieRecord` overload that carries no ordering value. Every updated
row fails.
The same is reachable outside the prepped path: `HoodieCreateRecordUtils`
computes the ordering
value with `getNestedFieldVal(..., false, ...)`, which returns `null` for a
null field value, and
`loadPayload(clazz, record, null)` then falls through to the same defaulting
constructor.
`HoodieAvroIndexedRecord` is unaffected because it derives the value from
the record.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
