danny0405 commented on code in PR #17928:
URL: https://github.com/apache/hudi/pull/17928#discussion_r3007023308
##########
hudi-common/src/main/java/org/apache/hudi/common/model/DefaultHoodieRecordPayload.java:
##########
@@ -179,7 +179,31 @@ protected boolean
needUpdatingPersistedRecord(IndexedRecord currentValue,
if (incomingRecord.isEmpty() &&
OrderingValues.isDefault(incomingOrderingVal)) {
return true;
}
- return persistedOrderingVal == null ||
persistedOrderingVal.compareTo(incomingOrderingVal) <= 0;
+ boolean updateOnSameOrderingField =
Boolean.parseBoolean(properties.getProperty(
+ HoodiePayloadProps.UPDATE_ON_SAME_PAYLOAD_ORDERING_FIELD_PROP_KEY,
+
HoodiePayloadProps.DEFAULT_UPDATE_ON_SAME_PAYLOAD_ORDERING_FIELD_PROP_VALUE));
+ return compareOrderingVal(persistedOrderingVal, incomingOrderingVal,
updateOnSameOrderingField);
+ }
+
+ /**
+ * Compares the ordering between persisted entry and input payload.
+ * If updateOnSameOrderingField is true, then incoming record is returned
when payload ordering field is the same.
+ * @param persistedOrderingVal record present in Disk
+ * @param incomingOrderingVal record part of input payload
+ * @return true if the older record(persisted entry) is older than incoming
record.
+ */
+ protected boolean compareOrderingVal(Comparable persistedOrderingVal,
Comparable incomingOrderingVal,
Review Comment:
if `updateOnSameOrderingField` works only for the reverse ordering use
cases, should we limit the scope to just `ReverseOrderHoodieRecordPayload`.
should we also cover `#precombine` to be consistent in the ordering value
comparison.
BTW, the payload class is deprecated, we are not adding new functions to it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
