Hi Hemi, There may be two reasons that I can think of 1. The number of connections exceeds the MySQL limit, you can check the options in my.cnf for your mysql server and increase the max connections. 2. Connection timeout for mysql client, you can try to add 'autoReconnect=true' to the connection url
Best, Shammon FY On Thu, May 25, 2023 at 8:32 AM Hemi Grs <hemi...@gmail.com> wrote: > hey everybody, > > I have a problem with my apache flink, I am synchronizing from MySQL to > Elasticsearch but it seems that I can't run more than 19 tasks. it gave me > this error: > > ------ > Caused by: org.apache.flink.util.FlinkRuntimeException: > org.apache.flink.util.FlinkRuntimeException: > java.sql.SQLTransientConnectionException: connection-pool-10.10.10.111:3306 > - Connection is not available, request timed out after 30000ms. at > com.ververica.cdc.connectors.mysql.debezium.DebeziumUtils.openJdbcConnection(DebeziumUtils.java:64) > at > com.ververica.cdc.connectors.mysql.source.assigners.MySqlSnapshotSplitAssigner.discoveryCaptureTables(MySqlSnapshotSplitAssigner.java:171) > ... 12 more > Caused by: org.apache.flink.util.FlinkRuntimeException: > java.sql.SQLTransientConnectionException: connection-pool-10.10.10.111:3306 > - Connection is not available, request timed out after 30000ms. at > com.ververica.cdc.connectors.mysql.source.connection.JdbcConnectionFactory.connect(JdbcConnectionFactory.java:72) > at io.debezium.jdbc.JdbcConnection.connection(JdbcConnection.java:890) at > io.debezium.jdbc.JdbcConnection.connection(JdbcConnection.java:885) > at io.debezium.jdbc.JdbcConnection.connect(JdbcConnection.java:418) at > com.ververica.cdc.connectors.mysql.debezium.DebeziumUtils.openJdbcConnection(DebeziumUtils.java:61) > ... 13 moreCaused by: java.sql.SQLTransientConnectionException: > connection-pool-10.10.10.111:3306 - Connection is not available, request > timed out after 30000ms. > at > com.ververica.cdc.connectors.shaded.com.zaxxer.hikari.pool.HikariPool.createTimeoutException(HikariPool.java:696) > at > com.ververica.cdc.connectors.shaded.com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:197) > at > com.ververica.cdc.connectors.shaded.com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:162) > at > com.ververica.cdc.connectors.shaded.com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:100) > at > com.ververica.cdc.connectors.mysql.source.connection.JdbcConnectionFactory.connect(JdbcConnectionFactory.java:59) > ... 17 more > ----- > > I have try adding this 2 lines on flink-conf.yaml but doesn't do anything: > ----- > env.java.opts: > "-Dcom.ververica.cdc.connectors.mysql.hikari.maximumPoolSize=100" > flink.connector.mysql-cdc.max-pool-size: 100 > ----- > > does anybody know the solution? > Additional info, my database is doing fine, because I try creating another > apache flink server and it can run another 19 tasks, so total there 38 > tasks running and it's doing fine. So how do I run many tasks on 1 server > and the server still have lots of resources. > > Thanks >
