[ https://issues.apache.org/jira/browse/FLINK-35256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Qingsheng Ren reassigned FLINK-35256: ------------------------------------- Assignee: yux > Pipeline transform ignores column type nullability > -------------------------------------------------- > > Key: FLINK-35256 > URL: https://issues.apache.org/jira/browse/FLINK-35256 > Project: Flink > Issue Type: Improvement > Components: Flink CDC > Reporter: yux > Assignee: yux > Priority: Major > Labels: pull-request-available > Attachments: log.txt > > > Flink CDC 3.1.0 brought transform feature, allowing column type / value > transformation prior to data routing process. However after the > transformation, column type marked as `NOT NULL` lost their annotation, > causing some downstream sinks to fail since they require primary key to be > NOT NULL. > Here's the minimum reproducible example about this problem: > ```yaml > source: > type: mysql > ... > sink: > type: starrocks > name: StarRocks Sink > ... > pipeline: > name: Sync MySQL Database to StarRocks > parallelism: 4 > transform: > - source-table: reicigo.\.* > projection: ID, UPPER(ID) AS UPID > ``` > In the MySQL source table, primary key column `ID` is marked as `NOT NULL`, > but such information was lost at downstream, causing the following exception > (see attachment). -- This message was sent by Atlassian Jira (v8.20.10#820010)