[
https://issues.apache.org/jira/browse/FLINK-37327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17928441#comment-17928441
]
David Radley edited comment on FLINK-37327 at 2/19/25 2:35 PM:
---------------------------------------------------------------
[~klam-shop]Please could you give more detail as to how the downstream sinks
can handle it.
>From
>[https://nightlies.apache.org/flink/flink-docs-release-1.20/docs/connectors/table/formats/debezium/
>
>|https://nightlies.apache.org/flink/flink-docs-release-1.20/docs/connectors/table/formats/debezium/]
> it say "However, currently Flink can’t combine UPDATE_BEFORE and UPDATE_AFTER
>into a single UPDATE message. Therefore, Flink encodes UPDATE_BEFORE and
>UDPATE_AFTER as DELETE and INSERT Debezium messages." I assume if this new
>mode is on then we do not get a DELETE but we only get an INSERT, so you are
>saying the downstream sinks can handle an INSERT when previously it got a
>DELETE first.
Also I wonder how downstream operators would be effected by this mode.
was (Author: davidrad):
Please could you give more detail as to how the downstream sinks can handle it.
>From
>[https://nightlies.apache.org/flink/flink-docs-release-1.20/docs/connectors/table/formats/debezium/
>
>|https://nightlies.apache.org/flink/flink-docs-release-1.20/docs/connectors/table/formats/debezium/]
> it say "However, currently Flink can’t combine UPDATE_BEFORE and UPDATE_AFTER
>into a single UPDATE message. Therefore, Flink encodes UPDATE_BEFORE and
>UDPATE_AFTER as DELETE and INSERT Debezium messages." I assume if this new
>mode is on then we do not get a DELETE but we only get an INSERT, so you are
>saying the downstream sinks can handle an INSERT when previously it got a
>DELETE first.
Also I wonder how downstream operators would be effected by this mode.
> Debezium Avro Format: Add FormatOption to Optionally Skip emitting
> UPDATE_BEFORE Rows
> -------------------------------------------------------------------------------------
>
> Key: FLINK-37327
> URL: https://issues.apache.org/jira/browse/FLINK-37327
> Project: Flink
> Issue Type: Improvement
> Components: Formats (JSON, Avro, Parquet, ORC, SequenceFile)
> Affects Versions: 1.20.1
> Reporter: Kevin Lam
> Priority: Minor
> Labels: pull-request-available
>
> Add a Format Option to the Debezium Format to optionally skip emitting the
> UPDATE_BEFORE Rows when deserializing a Debezium message with op='u'.
> This is helpful for Flink SQL applications that want to operate in UPSERT
> (ChangelogMode=[I,UA,D]) mode and save on processing the UPDATE_BEFORE Rows
> since the downstream sinks can handle it.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
