Flink CDC Issue Import created FLINK-34856: ----------------------------------------------
Summary: [Bug] timeout when Invocating Remote RPC. Key: FLINK-34856 URL: https://issues.apache.org/jira/browse/FLINK-34856 Project: Flink Issue Type: Bug Components: Flink CDC Reporter: Flink CDC Issue Import ### Search before asking - [X] I searched in the [issues|https://github.com/ververica/flink-cdc-connectors/issues) and found nothing similar. ### Flink version 1.18.0 ### Flink CDC version 3.0 ### Database and its version Mysql 8.0 ### Minimal reproduce step ``` source: type: mysql hostname: localhost port: 3306 username: root password: 123456 tables: inventory.\.* server-id: 5400-5404 server-time-zone: Asia/Shanghai sink: type: values pipeline: name: Sync MySQL Database to Values parallelism: 2 ``` ### What did you expect to see? running correctly. ### What did you see instead? ``` 2024-01-03 11:20:14 java.lang.IllegalStateException: Failed to send request to coordinator: com.ververica.cdc.runtime.operators.schema.event.SchemaChangeRequest@f75b9e4 at com.ververica.cdc.runtime.operators.schema.SchemaOperator.sendRequestToCoordinator(SchemaOperator.java:126) at com.ververica.cdc.runtime.operators.schema.SchemaOperator.requestSchemaChange(SchemaOperator.java:110) at com.ververica.cdc.runtime.operators.schema.SchemaOperator.handleSchemaChangeEvent(SchemaOperator.java:95) at com.ververica.cdc.runtime.operators.schema.SchemaOperator.processElement(SchemaOperator.java:85) at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.pushToOperator(CopyingChainingOutput.java:75) at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:50) at org.apache.flink.streaming.runtime.tasks.CopyingChainingOutput.collect(CopyingChainingOutput.java:29) at org.apache.flink.streaming.runtime.tasks.SourceOperatorStreamTask$AsyncDataOutputToOutput.emitRecord(SourceOperatorStreamTask.java:309) at org.apache.flink.streaming.api.operators.source.SourceOutputWithWatermarks.collect(SourceOutputWithWatermarks.java:110) at org.apache.flink.streaming.api.operators.source.SourceOutputWithWatermarks.collect(SourceOutputWithWatermarks.java:101) at com.ververica.cdc.connectors.mysql.source.reader.MySqlPipelineRecordEmitter.sendCreateTableEvent(MySqlPipelineRecordEmitter.java:125) at com.ververica.cdc.connectors.mysql.source.reader.MySqlPipelineRecordEmitter.processElement(MySqlPipelineRecordEmitter.java:109) at com.ververica.cdc.connectors.mysql.source.reader.MySqlRecordEmitter.emitRecord(MySqlRecordEmitter.java:82) at com.ververica.cdc.connectors.mysql.source.reader.MySqlRecordEmitter.emitRecord(MySqlRecordEmitter.java:55) at org.apache.flink.connector.base.source.reader.SourceReaderBase.pollNext(SourceReaderBase.java:160) at org.apache.flink.streaming.api.operators.SourceOperator.emitNext(SourceOperator.java:419) at org.apache.flink.streaming.runtime.io.StreamTaskSourceInput.emitNext(StreamTaskSourceInput.java:68) at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65) at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:562) at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:231) at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:858) at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:807) at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:953) at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:932) at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:746) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:562) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: java.util.concurrent.ExecutionException: java.util.concurrent.TimeoutException: Invocation of [RemoteRpcInvocation(JobMasterOperatorEventGateway.sendRequestToCoordinator(OperatorID, SerializedValue)]] at recipient [pekko.tcp://flink@localhost:6123/user/rpc/jobmanager_11] timed out. This is usually caused by: 1) Pekko failed sending the message silently, due to problems like oversized payload or serialization failures. In that case, you should find detailed error information in the logs. 2) The recipient needs more time for responding, due to problems like slow machines or network jitters. In that case, you can try to increase pekko.ask.timeout. at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:395) at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2005) at com.ververica.cdc.runtime.operators.schema.SchemaOperator.sendRequestToCoordinator(SchemaOperator.java:123) ... 26 more ``` ### Anything else? _No response_ ### Are you willing to submit a PR? - [ ] I'm willing to submit a PR! ---------------- Imported from GitHub ---------------- Url: https://github.com/apache/flink-cdc/issues/2956 Created by: [lvyanquan|https://github.com/lvyanquan] Labels: bug, Created at: Wed Jan 03 11:24:39 CST 2024 State: open -- This message was sent by Atlassian Jira (v8.20.10#820010)