感谢Jark! 上次调整了mysql连接参数解决了超时问题,但是目前还是同步这张表的时候,就是在Snapshot快结束阶段卡主,报连接异常,请问这个地方应该排查哪个地方?
一、环境: 1、版本:1.11.2 2、flink CDC 用Stream API 从mysql 同步到kudu 3、*这张表有3400万数据,老是在3340左右就卡住,目前已经把sink 到kudu直接改成 print输出还是会出现一模一样的报错。* 日志如下: ================================== 2020-11-26 14:00:15,293 ERROR *io.debezium.connector.mysql.SnapshotReader* [] - Failed due to error: Aborting snapshot due to error when last running 'SELECT * FROM `uchome`.`forums_post_12`': *Communications link failure* The last packet successfully received from the server was 16 milliseconds ago. The last packet sent successfully to the server was 335,794 milliseconds ago. *org.apache.kafka.connect.errors.ConnectException: Communications link failure* The last packet successfully received from the server was 16 milliseconds ago. The last packet sent successfully to the server was 335,794 milliseconds ago. Error code: 0; SQLSTATE: 08S01. at io.debezium.connector.mysql.AbstractReader.wrap(AbstractReader.java:230) ~[flinkcdc4mysql2kudu.jar:?] at io.debezium.connector.mysql.AbstractReader.failed(AbstractReader.java:207) ~[flinkcdc4mysql2kudu.jar:?] at io.debezium.connector.mysql.SnapshotReader.execute(SnapshotReader.java:831) ~[flinkcdc4mysql2kudu.jar:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_231] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_231] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_231] *Caused by: com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure* The last packet successfully received from the server was 16 milliseconds ago. The last packet sent successfully to the server was 335,794 milliseconds ago. at com.mysql.cj.jdbc.exceptions.SQLError.createCommunicationsException(SQLError.java:174) ~[flinkcdc4mysql2kudu.jar:?] at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:64) ~[flinkcdc4mysql2kudu.jar:?] at com.mysql.cj.jdbc.ConnectionImpl.commit(ConnectionImpl.java:813) ~[flinkcdc4mysql2kudu.jar:?] at io.debezium.connector.mysql.SnapshotReader.execute(SnapshotReader.java:747) ~[flinkcdc4mysql2kudu.jar:?] ... 3 more Caused by: com.mysql.cj.exceptions.CJCommunicationsException: Communications link failure The last packet successfully received from the server was 16 milliseconds ago. The last packet sent successfully to the server was 335,794 milliseconds ago. at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:1.8.0_231] at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[?:1.8.0_231] at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:1.8.0_231] at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[?:1.8.0_231] at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:61) ~[flinkcdc4mysql2kudu.jar:?] at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:105) ~[flinkcdc4mysql2kudu.jar:?] at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:151) ~[flinkcdc4mysql2kudu.jar:?] at com.mysql.cj.exceptions.ExceptionFactory.createCommunicationsException(ExceptionFactory.java:167) ~[flinkcdc4mysql2kudu.jar:?] at com.mysql.cj.protocol.a.NativeProtocol.clearInputStream(NativeProtocol.java:837) ~[flinkcdc4mysql2kudu.jar:?] at com.mysql.cj.protocol.a.NativeProtocol.sendCommand(NativeProtocol.java:652) ~[flinkcdc4mysql2kudu.jar:?] at com.mysql.cj.protocol.a.NativeProtocol.sendQueryPacket(NativeProtocol.java:986) ~[flinkcdc4mysql2kudu.jar:?] at com.mysql.cj.protocol.a.NativeProtocol.sendQueryString(NativeProtocol.java:921) ~[flinkcdc4mysql2kudu.jar:?] at com.mysql.cj.NativeSession.execSQL(NativeSession.java:1165) ~[flinkcdc4mysql2kudu.jar:?] at com.mysql.cj.jdbc.ConnectionImpl.commit(ConnectionImpl.java:801) ~[flinkcdc4mysql2kudu.jar:?] at io.debezium.connector.mysql.SnapshotReader.execute(SnapshotReader.java:747) ~[flinkcdc4mysql2kudu.jar:?] ... 3 more Caused by: java.io.IOException: Socket is closed at com.mysql.cj.protocol.AbstractSocketConnection.getMysqlInput(AbstractSocketConnection.java:72) ~[flinkcdc4mysql2kudu.jar:?] at com.mysql.cj.protocol.a.NativeProtocol.clearInputStream(NativeProtocol.java:833) ~[flinkcdc4mysql2kudu.jar:?] at com.mysql.cj.protocol.a.NativeProtocol.sendCommand(NativeProtocol.java:652) ~[flinkcdc4mysql2kudu.jar:?] at com.mysql.cj.protocol.a.NativeProtocol.sendQueryPacket(NativeProtocol.java:986) ~[flinkcdc4mysql2kudu.jar:?] at com.mysql.cj.protocol.a.NativeProtocol.sendQueryString(NativeProtocol.java:921) ~[flinkcdc4mysql2kudu.jar:?] at com.mysql.cj.NativeSession.execSQL(NativeSession.java:1165) ~[flinkcdc4mysql2kudu.jar:?] at com.mysql.cj.jdbc.ConnectionImpl.commit(ConnectionImpl.java:801) ~[flinkcdc4mysql2kudu.jar:?] at io.debezium.connector.mysql.SnapshotReader.execute(SnapshotReader.java:747) ~[flinkcdc4mysql2kudu.jar:?] ... 3 more 2020-11-26 14:00:15,699 INFO io.debezium.connector.common.BaseSourceTask [] - Stopping down connector 2020-11-26 14:00:15,700 INFO io.debezium.connector.mysql.MySqlConnectorTask [] - Stopping MySQL connector task 2020-11-26 14:00:15,700 INFO io.debezium.connector.mysql.ChainedReader [] - ChainedReader: Stopping the snapshot reader 2020-11-26 14:00:15,700 INFO io.debezium.connector.mysql.SnapshotReader [] - Discarding 1411 unsent record(s) due to the connector shutting down 2020-11-26 14:00:15,700 INFO io.debezium.connector.mysql.SnapshotReader [] - Discarding 0 unsent record(s) due to the connector shutting down 2020-11-26 14:00:15,701 INFO io.debezium.connector.mysql.MySqlConnectorTask [] - Connector task finished all work and is now shutdown ================================== -- Sent from: http://apache-flink.147419.n8.nabble.com/