rangareddy commented on issue #5911: URL: https://github.com/apache/hudi/issues/5911#issuecomment-5102636965
This issue was reviewed as part of the JIRA-migrated backlog triage. Findings: The issue template was submitted unfilled -- the problem description, expected behaviour and stack trace sections still contain the placeholder text -- so there is no reproducible statement of the failure to work from. The most likely cause was identified on the thread and never confirmed: as explained on 2023-01-03, when Hudi sees two records with the same key it orders them by the **preCombine field** and keeps the one with the largest value, and the preCombine field defaults to a column named `ts`. Your 2022-06-24 description -- a record being replaced by one with an *older* `last_update_time` -- is exactly what happens when preCombine is not pointed at `last_update_time`. On the Flink side that is `write.precombine.field`; on the Spark/streamer side `hoodie.datasource.write.precombine.field`. There has been no activity since 2023-01-03. Closing as stale. If bootstrapping with the streamer and then writing incrementally from Flink still loses updates for you, please open a fresh issue on a recent Hudi version with the actual write configs (especially the preCombine field), the two record versions, and the Flink/Hudi versions. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
