[
https://issues.apache.org/jira/browse/HUDI-6199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HUDI-6199:
---------------------------------
Labels: pull-request-available (was: )
> CDC payload with op field for deletes do not work
> -------------------------------------------------
>
> Key: HUDI-6199
> URL: https://issues.apache.org/jira/browse/HUDI-6199
> Project: Apache Hudi
> Issue Type: Bug
> Reporter: Ethan Guo
> Assignee: Ethan Guo
> Priority: Blocker
> Labels: pull-request-available
> Fix For: 0.13.1
>
>
> Delete operation in custom payload after RFC-46: while looking into a 0.13.1
> release [blocker|https://github.com/apache/hudi/pull/8573], I found that
> custom payload implementation like AWS DMS payload and Debezium payload are
> not properly migrated to the new APIs introduced by RFC-46, causing the
> delete operation to fail. Our tests did not catch this.
>
> It is currently assumed that delete records are marked by
> "_hoodie_is_deleted"; however, custom CDC payloads use op field to mark
> deletes.
>
> Impact:
> OverwriteWithLatest payload(also OverwriteNonDefaultsWithLatestAvroPayload)
> are not affected.
> for any other custom payloads: (AWSDMSAvropayload, All debezium payloads)
> deletes are broken.
> If someone is using "_is_hoodie_deleted" to enforce deletes, there are no
> issues w/ custome payloads.
> COW:
> deleting a non-existant will break if not using "_is_hoodie_deleted" way.
> MOR:
> any deletes will break if not using "_is_hoodie_deleted" way.
> Writer:
> all writers(spark, flink) except spark-sql.
> DefaultHoodieRecordPayload delete marker support in 0.14.0 is also affected.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
