merlinchen opened a new issue, #10163: URL: https://github.com/apache/seatunnel/issues/10163
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happened Hello Seatunnel team, First of all, thank you very much for providing such an outstanding tool that allows us to deploy and configure data synchronization processes easily and efficiently. We are using Seatunnel to synchronize data from MySQL CDC to ClickHouse, but for the past two days we have encountered an issue where the job status shows RUNNING, while the actual synchronization has already stopped working. After analyzing the logs, we found that all problematic jobs had experienced a failure-and-recovery cycle due to various reasons, such as slaveId conflicts or memory_limit_exceed errors on the ClickHouse side preventing data writes. Initially, we suspected that Debezium’s binlog listener malfunctioned after the job restarted. However, after increasing the log level, we confirmed that Debezium was indeed listening and emitting binlog events — but nothing happened afterward. Under normal circumstances, logs from TransformFlowLifeCycle should appear to show data transformation steps, but we did not find any such logs. We also raised the log level to TRACE, but still did not see any valuable information. Once we cancelled the job and restarted it again with the same job configuration file, the synchronization resumed normally. We have attached the complete logs and the job configuration file. [job-1048419550153932801.zip](https://github.com/user-attachments/files/23979008/job-1048419550153932801.zip) [project_contract_payment_sync_job.txt](https://github.com/user-attachments/files/23978979/project_contract_payment_sync_job.txt) Below is the timeline extracted from the logs: 2025-12-03 11:15:08,200 – A slaveId conflict exception occurred 2025-12-03 11:15:11,881 – Logs show the job restarted successfully `turned from state DEPLOYING to RUNNING` 2025-12-04 17:48:49,649 – A row insertion event was detected `[ySqlStreamingChangeEventSource] [blc-38.19.66.8:33001] - Emitted 1 CREATE record(s) for event:` After this insertion event, no logs related to TransformFlowLifeCycle were found. When synchronization works normally, there would be log outputs similar to the following: ``` 2025-12-04 16:11:43,240 DEBUG [ySqlStreamingChangeEventSource] [blc-38.19.66.8:33001] - Emitted 1 CREATE record(s) for event: Event{header=EventHeaderV4{timestamp=1764835904000, eventType=EXT_WRITE_ROWS, serverId=1, headerLength=19, dataLength=438, nextPosition=431955054, flags=0}, data=WriteRowsEventData{tableId=116122, includedColumns={0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29}, rows=[ [[B@2ea81b81, [B@7a67cbf1, [B@4fc3bb2a, [B@2d71d1b, [B@6ccd6ad3, [B@28310cf8, [B@7a79f83f, [B@407c0dae, [B@47606c80, [B@6796ee58, [B@529b5a95, [B@722a2c15, [B@36ea4abc, [B@66de9617, [B@6e9927ab, [B@4df0e4e9, [B@29abf841, [B@3b57ad4d, [B@1c5471f3, [B@5649193c, [B@20d68a44, [B@266d66d6, [B@19895001, 0.00, [B@7172af5, [B@138704f9, [B@5e918b89, [B@dcbd102, [B@333512cc, [B@18be689f] ]}} 2025-12-04 16:11:43,246 DEBUG [ySqlStreamingChangeEventSource] [blc-38.19.66.8:33001] - Received query command: Event{header=EventHeaderV4{timestamp=1764835904000, eventType=QUERY, serverId=1, headerLength=19, dataLength=54, nextPosition=431955223, flags=8}, data=QueryEventData{threadId=187867, executionTime=0, errorCode=0, database='testdb', sql='BEGIN'}} 2025-12-04 16:11:43,247 DEBUG [e.s.t.f.TransformFlowLifeCycle] [BlockingWorker-TaskGroupLocation{jobId=1048434674038210561, pipelineId=1, taskGroupId=2}] - Transform[org.apache.seatunnel.transform.rename.TableRenameMultiCatalogTransform@39e78642] input row SeaTunnelRow{tableId=testdb.general_businesslist, kind=-U, fields=[2e6ae8886f4e46b48bfa6023666e4067, 848484849ae6f22e019ae862a37301e9, , a693c22e7f9a5a27017f9a9230106386, 150913, tmp**tmp****tmptmp, RBTH20251200001, , 2025, , 12, , 507510, ***/******, ***/**/*********, 【*tmp**tmp】，***tmp*tmp*【***】******/**/*********，****：*test**test****，**：1,000.00*，****:2025-12-04, 0, 2025-12-04 16:05:34, ******, a693c22e82bb9f7b0182d2d24a221e80, ***, 2025-12-04 16:03:53, , Q, , , 0, MDSY-FA-2025-M028, **test*test*test**test*******（250kVA）*******, , , 9153010x7194497302, *test**test****, 2025-12-04, ed1911566a47409f82d7918623510ec8, ***, 10501323, **tmp*tmp, ed1911566a47409f82d7918623510ec8, ***, 2025-12-04, , , , , , , , **, , 1000.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, 0.000, , , , , *******************, 53001615x44051000995, ************, ***********, 53001647x36051005881, *************, 09, **, 0, , , , ]} and output row SeaTunnelRow{tableId=TESTDB.GENERAL_BUSINESSLIST, kind=-U, fields=[2e6ae8886f4e46b48bfa6023666e4067, 848484849ae6f22e019ae862a37301e9, , a693c22e7f9a5a27017f9a9230106386, 150913, tmp**tmp****tmptmp, RBTH20251200001, , 2025, , 12, , 507510, ***/******, ***/**/*********, 【*tmp**tmp】，***tmp*tmp*【***】******/**/*********，****：*test**test****，**：1,000.00*，****:2025-12-04, 0, 2025-12-04 16:05:34, ******, a693c22e82bb9f7b0182d2d24a221e80, ***, 2025-12-04 16:03:53, , Q, , , 0, MDSY-FA-2025-M028, **test*test*test**test*******（250kVA）*******, , , 9153010x7194497302, *test**test****, 2025-12-04, ed1911566a47409f82d7918623510ec8, ***, 10501323, **tmp*tmp, ed1911566a47409f82d7918623510ec8, ***, 2025-12-04, , , , , , , , **, , 1000.000, 0.000, 0.000, 0.000, 0.0 00, 0.000, 0.000, 0.000, , , , , *******************, 530016155x4051000995, ************, ***********, 53001647x36051005881, *************, 09, **, 0, , , , ]} ``` We would greatly appreciate any guidance on how to further investigate this issue. If it’s possible to determine the root cause from the provided logs, that would be extremely helpful. Otherwise, could you please point us to the relevant core modules or code areas involved in this process, so that we can continue debugging on our side? Thank you very much. ### SeaTunnel Version 2.3.12 ### SeaTunnel Config ```conf seatunnel: engine: classloader-cache-mode: true history-job-expire-minutes: 1440 backup-count: 1 queue-type: blockingqueue print-execution-info-interval: 60 print-job-metrics-info-interval: 60 slot-service: dynamic-slot: true checkpoint: interval: 10000 timeout: 60000 storage: type: hdfs max-retained: 3 plugin-config: namespace: /home/testuser/seatunnel/checkpoint_snapshot storage.type: hdfs fs.defaultFS: file:/// telemetry: metric: enabled: true logs: scheduled-deletion-enable: true http: enable-http: true port: 8080 enable-dynamic-port: false ``` ### Running Command ```shell ./bin/seatunnel.sh --config ./jobs/project_contract_payment_sync_job.conf -DJvmOption="-Xms256m -Xmx1G" -m cluster --async ``` ### Error Exception ```log Provided the full log file ``` ### Zeta or Flink or Spark Version _No response_ ### Java or Scala Version `1.8.0_472-b08` ### Screenshots _No response_ ### Are you willing to submit PR? - [x] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
