Runkang He created FLINK-36683:
----------------------------------
Summary: Support metadata 'row_kind' virtual column for Mongo CDC
Connector
Key: FLINK-36683
URL: https://issues.apache.org/jira/browse/FLINK-36683
Project: Flink
Issue Type: Improvement
Components: Flink CDC
Affects Versions: cdc-3.3.0, cdc-3.2.1
Reporter: Runkang He
'row_kind' metadata is very useful in actual user scenarios, the two main
scenarios are below:
1. Save all upstream messages: In this scenario, the downstream will save all
message includes delete messages from upstream. To achieve this requirement, we
should convert all kind of changelogs to append only message, and need to use
metadata row_kind to represent the changelog kind.
2. Ignore upstream delete messages: In this scenario, the upstream cdc source
often deletes historical data regularly to save storage space and only retains
data within seven days. However, the business requires the downstream OLAP
system to retain the full amount of historical data, so it is necessary to
ignore the delete messages from source.
So I think we should support 'row_kind' metadata in Mongo CDC Connector.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
