Sharon Xie created FLINK-35147: ---------------------------------- Summary: SinkMaterializer throws StateMigrationException when widening the field type in the output table Key: FLINK-35147 URL: https://issues.apache.org/jira/browse/FLINK-35147 Project: Flink Issue Type: Bug Components: Table SQL / API Reporter: Sharon Xie Attachments: image-2024-04-17-14-15-21-647.png, image-2024-04-17-14-15-35-297.png
When a field type in the output table is changed from int -> bigint or timestamp(3) -> timestamp(6), SinkMaterializer would fail to restore state. This is unexpected as the change is backward compatible. The new type should be able to "accept" all the old values that had narrower type. Note that the planner works fine and would accept such change. To reproduce ``` CREATE TABLE ltable ( `id` integer primary key, `num` int ) WITH ( 'connector' = 'upsert-kafka', 'properties.bootstrap.servers' = 'kafka.test:9092', 'key.format' = 'json', 'value.format' = 'json', 'topic' = 'test1' ); CREATE TABLE rtable ( `id` integer primary key, `ts` timestamp(3) ) WITH ( 'connector' = 'upsert-kafka', 'properties.bootstrap.servers' = 'kafka.test:9092', 'key.format' = 'json', 'value.format' = 'json', 'topic' = 'test2' ); CREATE TABLE output ( `id` integer primary key, `num` int, `ts` timestamp(3) ) WITH ( 'connector' = 'upsert-kafka', 'properties.bootstrap.servers' = 'kafka.test:9092', 'key.format' = 'json', 'value.format' = 'json', 'topic' = 'test3' ); insert into `output` select ltable.id, num, ts from ltable join rtable on ltable.id = rtable.id ``` Run it, stop with a savepoint, then update output table with ``` CREATE TABLE output ( `id` integer primary key, -- change one of the type below would cause the issue `num` bigint, `ts` timestamp(6) ) WITH ( 'connector' = 'upsert-kafka', 'properties.bootstrap.servers' = 'kafka.test:9092', 'key.format' = 'json', 'value.format' = 'json', 'topic' = 'test3' ); ``` Restart the job with the savepoint created Sample screenshots !image-2024-04-17-14-15-35-297.png! !image-2024-04-17-14-15-21-647.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)