[
https://issues.apache.org/jira/browse/FLINK-38823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18051479#comment-18051479
]
peiyu edited comment on FLINK-38823 at 1/13/26 11:42 AM:
---------------------------------------------------------
!screenshot-1.png!
[~fffy] this is paimon connecotr test
the job config:
{code:yaml}
source:
type: mysql
hostname:
port: 3306
username:
password:
tables: oms_db.tf_b_order,oms_db.tf_b_order_delist
treat-tinyint1-as-boolean.enabled: false
scan.incremental.close-idle-reader.enabled: true
scan.incremental.snapshot.backfill.skip: true
debezium.skipped.operations: d
metadata.list: op_ts
sink:
type: paimon
catalog.properties.uri:
catalog.properties.warehouse:
catalog.properties.metastore: hive
table.properties.deletion-vectors.enabled: true
table.properties.write-buffer-spillable: true
table.properties.precommit-compact: true
transform:
- source-table: oms_db.tf_b_order
projection: ORDER_ID,CREATED_TIME,OLD_ORDER_NUMBER,op_ts as opts_order
primary-keys: ORDER_ID
- source-table: oms_db.tf_b_order_delist
projection: ORDER_ID,DELIST_ID,CARRIER_NAME,op_ts as opts_order_delist
primary-keys: ORDER_ID
route:
- source-table: oms_db.tf_b_order,oms_db.tf_b_order_delist
sink-table: ods_oms_db.tf_b_order_wide
pipeline:
name: mysql_paimon_wide_table
parallelism: 8
{code}
was (Author: JIRAUSER310332):
!screenshot-1.png!
this is paimon connecotr test
the job config:
{code:yaml}
source:
type: mysql
hostname:
port: 3306
username:
password:
tables: oms_db.tf_b_order,oms_db.tf_b_order_delist
treat-tinyint1-as-boolean.enabled: false
scan.incremental.close-idle-reader.enabled: true
scan.incremental.snapshot.backfill.skip: true
debezium.skipped.operations: d
metadata.list: op_ts
sink:
type: paimon
catalog.properties.uri:
catalog.properties.warehouse:
catalog.properties.metastore: hive
table.properties.deletion-vectors.enabled: true
table.properties.write-buffer-spillable: true
table.properties.precommit-compact: true
transform:
- source-table: oms_db.tf_b_order
projection: ORDER_ID,CREATED_TIME,OLD_ORDER_NUMBER,op_ts as opts_order
primary-keys: ORDER_ID
- source-table: oms_db.tf_b_order_delist
projection: ORDER_ID,DELIST_ID,CARRIER_NAME,op_ts as opts_order_delist
primary-keys: ORDER_ID
route:
- source-table: oms_db.tf_b_order,oms_db.tf_b_order_delist
sink-table: ods_oms_db.tf_b_order_wide
pipeline:
name: mysql_paimon_wide_table
parallelism: 8
{code}
> Error running wide table task
> -----------------------------
>
> Key: FLINK-38823
> URL: https://issues.apache.org/jira/browse/FLINK-38823
> Project: Flink
> Issue Type: Bug
> Components: Flink CDC
> Affects Versions: cdc-3.5.0
> Reporter: peiyu
> Priority: Major
> Attachments: image-2025-12-19-16-35-37-842.png, screenshot-1.png
>
>
> pipeline job config :
> {code:yaml}
> source:
> type: mysql
> hostname:
> port: 3306
> username:
> password:
> tables: oms_db.tf_b_order,oms_db.tf_b_order_delist
> treat-tinyint1-as-boolean.enabled: false
> scan.incremental.close-idle-reader.enabled: true
> debezium.skipped.operations: d
> metadata.list: op_ts,table_name
> sink:
> type: values
> transform:
> - source-table: oms_db.tf_b_order
> projection: ORDER_ID,CREATED_TIME,OLD_ORDER_NUMBER,op_ts as opts_order
> primary-keys: ORDER_ID
> - source-table: oms_db.tf_b_order_delist
> projection: ORDER_ID,DELIST_ID,CARRIER_NAME,op_ts as opts_order_delist
> primary-keys: ORDER_ID
> route:
> - source-table: oms_db.tf_b_order,oms_db.tf_b_order_delist
> sink-table: ods.ods_oms_db_tf_b_order_wide
> pipeline:
> name: mysql_paimon_wide_table
> parallelism: 8 {code}
> The initial startup process will encounter an error after about 3 minutes,
> the taskmanager will restart, and then it will run normally. However, after
> the snapshot phase ends, it cannot enter the incremental phase.
> !image-2025-12-19-16-35-37-842.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
