ad1happy2go opened a new pull request, #20600: URL: https://github.com/apache/hudi/pull/20600
### Describe the issue this Pull Request addresses closes #20599 With `hoodie.datasource.write.keygenerator.consistent.logical.timestamp.enabled=true` and a timestamp ordering field, upserts that update existing keys fail on 1.x. The incoming ordering value becomes a `java.sql.Timestamp`, but the base-file value is a `Long`. COW fails with a `ClassCastException` in `shouldKeepNewerRecord`, MOR fails writing the delete block, and payload comparisons lose sub-millisecond ordering. ### Summary and Changelog Ordering values no longer take the key-generator timestamp conversion and always stay in their stored form, as the Spark record path already does. The flag still applies to record keys and partition paths. - The Spark datasource (`HoodieCreateRecordUtils`), HoodieStreamer (`HoodieStreamerUtils`) and MERGE INTO custom-payload (`ExpressionPayload`) record builders, plus `HoodieAvroIndexedRecord`, `DefaultHoodieRecordPayload` and `PartialUpdateAvroPayload`, read the ordering field without the conversion. - New tests: a COW/MOR upsert in `TestGenericRecordAndRowConsistency` covering newer, older, sub-millisecond and delete cases, plus unit tests for HoodieStreamer, the two payloads and `HoodieAvroIndexedRecord`. Each fails on master and passes with this change. ### Impact Upserts with this flag and a timestamp ordering field work again. With the flag on, payload comparisons on timestamp ordering fields now use the full stored precision (micros) instead of milliseconds, so records less than a millisecond apart no longer tie. ### Risk Level low. Only the ordering value path changes when the flag is on; record keys, partition paths and behavior with the flag off are unchanged. ### Documentation Update none ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
