[GitHub] [hudi] nsivabalan commented on issue #2756: OrderingVal not being honoured for payloads in log files (for MOR table)
nsivabalan commented on issue #2756: URL: https://github.com/apache/hudi/issues/2756#issuecomment-813524391 Have filed a tracking [jira](https://issues.apache.org/jira/browse/HUDI-1763). Feel free to raise a PR :) once again, thanks for raising it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] nsivabalan commented on issue #2756: OrderingVal not being honoured for payloads in log files (for MOR table)
nsivabalan commented on issue #2756: URL: https://github.com/apache/hudi/issues/2756#issuecomment-813522558 yes, you are right. we do have a bug here w/ DefaultHoodieRecordPayload. With OverwriteWithLatestAvroPayload, we always choose the latest and so we didn't have this issue. DefaultHoodieRecordPayload class is something which was added in last few months and hence we didn't catch this. thanks for flagging this. btw, DefaultHoodieRecordPayload uses a diff config named "hoodie.payload.ordering.field" to resolve combineAndGetUpdate. Your approach seems fine to me. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] nsivabalan commented on issue #2756: OrderingVal not being honoured for payloads in log files (for MOR table)
nsivabalan commented on issue #2756: URL: https://github.com/apache/hudi/issues/2756#issuecomment-812703790 CC @n3nash -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hudi] nsivabalan commented on issue #2756: OrderingVal not being honoured for payloads in log files (for MOR table)
nsivabalan commented on issue #2756: URL: https://github.com/apache/hudi/issues/2756#issuecomment-812496813 yes, this is expected. if you are using OverwriteWithLatestAvroPayload as your payload class, combineAndGetUpdateValue does not honor ordering value. And so we added another payload for this purpose. DefaultHoodieRecordPayload. While using this, ensure you set "hoodie.payload.ordering.field" accordingly. Let me know how it goes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
