[
https://issues.apache.org/jira/browse/FLINK-40647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yanquan Lv resolved FLINK-40647.
--------------------------------
Fix Version/s: cdc-3.7.0
Assignee: Mao Jiayi
Resolution: Fixed
Merged in master via eef77bb0610c6af83447527a831b26f56e74b47b.
> Support best-effort schema expansion for existing sink tables during table
> creation
> -----------------------------------------------------------------------------------
>
> Key: FLINK-40647
> URL: https://issues.apache.org/jira/browse/FLINK-40647
> Project: Flink
> Issue Type: New Feature
> Components: Flink CDC
> Reporter: Mao Jiayi
> Assignee: Mao Jiayi
> Priority: Major
> Labels: pull-request-available
> Fix For: cdc-3.7.0
>
>
> When the initial {{CreateTableEvent}} encounters an existing target table,
> missing target columns or narrower target types may cause some table sinks to
> ignore input fields and silently lose data. Similar handling may otherwise
> need to be implemented separately in each connector.
> This issue proposes an opt-in, best-effort schema expansion capability:
> * Add the sink-level option {{{}existing-table.schema-expansion.enabled{}}},
> disabled by default.
> * Extend {{MetadataApplier}} with optional support for querying the existing
> target schema, normalizing target types, and declaring column-name case
> sensitivity.
> * Only run during the initial {{{}CreateTableEvent{}}}.
> * Add missing non-key physical columns as nullable columns.
> * Safely widen non-key column types when the operation is supported.
> * Log derived DDL events and verify successful changes by reading back the
> target schema.
> * Do not introduce new fail-fast behavior. Unsupported, unsafe, or failed
> expansions are delegated to the sink’s existing schema handling.
> Full compatibility validation, key-column modification, and replacing
> connector-specific schema handling are out of scope.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
