abhibhat98 opened a new issue #1676: URL: https://github.com/apache/hudi/issues/1676
For multiple keys in the same update, only the one with higher PRECOMBINE_FIELD_OPT_KEY is considered. The default payload is OverwriteWithLatestAvroPayload which, in preCombine, picks the payload with greatest ordering value. I can change this logic in the custom payload to pick any other value, however is it possible to update a values for same keys, rather than choosing 1? eg. I have this current snapshot of table K1 , V1 If a batch comes with the following values, K1,V2 K1,V3 K1,V4 Assuming V4 > V3,V2, can I still update all of them one by one so that I have the history? For now, only V4 is updated and in case of CDC / events, I lose the history for V2/V3. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
