n3nash commented on issue #1549: URL: https://github.com/apache/incubator-hudi/issues/1549#issuecomment-619420599
@PhatakN1 So this is what is possibly happening : 1) Hard Deletes in Hudi is only supported by following a certain contract with your payload. Your payload implementation should carry an "empty" record value. Something like this -> https://github.com/apache/incubator-hudi/blob/master/hudi-common/src/main/java/org/apache/hudi/common/model/EmptyHoodieRecordPayload.java that is supported out of the box by HoodieWriteClient here -> https://github.com/apache/incubator-hudi/blob/master/hudi-client/src/main/java/org/apache/hudi/client/HoodieWriteClient.java#L307. To simulate this with your own implementation (AwsDMSPayload), you can override the methods `getInsertValue` and `combineAndGetUpdateValue` (take a look at the EmptyPayload above). 2) In your specific use-case, since your payload isn't an "empty" payload, even though the MERGE on the realtime query is happening through the implementation of your payload, Hudi doesn't know whether this is a "hard delete" or a "soft delete" - the only way for Hudi to know it's a hard delete is the way I described above. Let me know if you have further questions ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
