yihua opened a new pull request, #20658: URL: https://github.com/apache/hudi/pull/20658
### Describe the issue this Pull Request addresses Closes #19636. A table with a nullable event-time ordering field can hold a record whose ordering value is null (for example a base file written before the field was added). Every later merge on that key fails: the file group reader maps the null to the `Integer` 0 default and `Long.compareTo(Integer)` throws a `ClassCastException`, or an NPE where the raw null survives. This breaks COW upserts, MOR snapshot reads and compaction, and the global index merge. #19637 takes a different approach (keeping nulls through the read path); this PR keeps the change at the comparison sites. ### Summary and Changelog A stored null ordering value now ranks below any incoming value, so the incoming record wins, which matches how `DefaultHoodieRecordPayload` already treats a persisted null. - Add `OrderingValues.isBaseOrderingHigher` and use it in `BufferedRecordMergerFactory` (event-time and partial-update mergers), `HoodieRecordMerger.maxOrderingValue`, and the Spark, Flink and Hive event-time mergers. A base value that is null, or the `Integer` 0 default while the incoming value has another class, ranks lowest; multi-field values apply this per field. - Make `OrderingValues.isSameClass` and `ArrayComparable.isValueSameClass` null-safe, so the delete merges in `BufferedRecordMergerFactory` and `HoodieMergedLogRecordScanner` let the delete win instead of throwing an NPE. - A stored `Integer` 0 against an incoming `Integer` still compares as a real value, so int ordering fields behave as before. Two non-null values of different classes, or an incoming null against a real stored value, still fail, now with an `IllegalArgumentException` naming both types. - Tests: unit tests for the helper, the buffered mergers, the delete paths, the log scanner and each engine merger, plus a Spark functional case table (COW, MOR, MOR with `GLOBAL_SIMPLE`, and the partial-update merger) that seeds stored nulls through a row-writer bulk insert and asserts the rows after an upsert with an update and a delete, before and after compaction. Every stored-null case fails without the fix. ### Impact Tables holding a null in a nullable ordering field can be upserted, read and compacted again. No behavior change for non-null ordering values. ### Risk Level low Only inputs that threw a `ClassCastException` or NPE before change behavior; same-class comparisons are untouched. ### Documentation Update none ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
