Flink CDC Issue Import created FLINK-34804:
----------------------------------------------
Summary: [Bug] flink cdc missing step to close mysql connection
Key: FLINK-34804
URL: https://issues.apache.org/jira/browse/FLINK-34804
Project: Flink
Issue Type: Bug
Components: Flink CDC
Reporter: Flink CDC Issue Import
### Search before asking
- [x] I searched in the
[issues|https://github.com/ververica/flink-cdc-connectors/issues] and found
nothing similar.
### Flink version
flink-1.17.0
### Flink CDC version
flink-sql-connector-mysql-cdc-2.4.1
### Database and its version
5.7.23
### Minimal reproduce step
flink cdc missing step to close mysql connection.
mysql> select * from information_schema.PROCESSLIST where user='test__cdc'
order by id;
+----------+-----------+-------------------+----------------+------------------+------+---------------------------------------------------------------+------+
| ID | USER | HOST | DB | COMMAND
| TIME | STATE | INFO |
+----------+-----------+-------------------+----------------+------------------+------+---------------------------------------------------------------+------+
| 73165285 | test__cdc | 88.88.88.88:47874 | test___cdc__db | Sleep
| **2181** | |
NULL |
| 73165286 | test__cdc | 88.88.88.88:47876 | NULL | Sleep
| **2181** | |
NULL |
| 73165293 | test__cdc | 88.88.88.88:47898 | NULL | Sleep
| **2180** | |
NULL |
| 73165294 | test__cdc | 88.88.88.88:47900 | NULL | Binlog Dump GTID
| 2180 | Master has sent all binlog to slave; waiting for more updates | NULL |
+----------+-----------+-------------------+----------------+------------------+------+---------------------------------------------------------------+------+
[thread 73165285]
2023-08-07 16:31:11.564220 88.88.88.88:47874 73165285
test__cdc [SQL [LOGIN] SQL]
2023-08-07 16:31:11.565508 88.88.88.88:47874 73165285
test__cdc [SQL /* mysql-connector-java-8.0.28 (Revision:
[7ff2161da3899f379fb3171b6538b191b1c5c7e2|https://github.com/apache/flink-cdc/commit/7ff2161da3899f379fb3171b6538b191b1c5c7e2])
*/SELECT @@session.auto_increment_increment AS auto_increment_increment,
@@character_set_client AS character_set_client, @@character_set_connection AS
character_set_connection, @@character_set_results AS character_set_results,
@@character_set_server AS character_set_server, @@collation_server AS
collation_server, @@collation_connection AS collation_connection,
@@init_connect AS init_connect, @@interactive_timeout AS interactive_timeout,
@@license AS license, @@lower_case_table_names AS lower_case_table_names,
@@max_allowed_packet AS max_allowed_packet, @@net_write_timeout AS
net_write_timeout, @@performance_schema AS performance_schema,
@@query_cache_size AS query_cache_size, @@query_cache_type AS query_cache_type,
@@sql_mode AS sql_mode, @@system_time_zone AS system_time_zone, @@time_zone AS
time_zone, @@transaction_isolation AS transaction_isolation, @@wait_timeout AS
wait_timeout SQL]
2023-08-07 16:31:11.566643 88.88.88.88:47874 73165285
test__cdc [SQL SHOW WARNINGS SQL]
2023-08-07 16:31:11.567953 88.88.88.88:47874 73165285
test__cdc [SQL SET autocommit=1 SQL]
2023-08-07 16:31:11.569078 88.88.88.88:47874 73165285
test__cdc [SQL SELECT @@session.transaction_read_only SQL]
2023-08-07 16:31:11.571336 88.88.88.88:47874 73165285
test__cdc [SQL SELECT VERSION() SQL]
2023-08-07 16:31:11.573046 88.88.88.88:47874 73165285
test__cdc [SQL SHOW GLOBAL VARIABLES LIKE 'binlog_format' SQL]
2023-08-07 16:31:11.574562 88.88.88.88:47874 73165285
test__cdc [SQL SHOW GLOBAL VARIABLES LIKE 'binlog_row_image' SQL]
2023-08-07 16:31:11.575901 88.88.88.88:47874 73165285
test__cdc [SQL SELECT TIME_TO_SEC(TIMEDIFF(NOW(), UTC_TIMESTAMP())) SQL]
2023-08-07 16:31:11.642133 88.88.88.88:47874 73165285
test__cdc [SQL SHOW CREATE TABLE `test___cdc__db`.`test_cdc_table` SQL]
2023-08-07 16:31:11.654385 88.88.88.88:47874 73165285
test__cdc [SQL SELECT MIN(`machine_name`), MAX(`machine_name`) FROM
`test___cdc__db`.`test_cdc_table` SQL]
2023-08-07 16:31:11.655673 88.88.88.88:47874 73165285
test__cdc [SQL USE `test___cdc__db`; SQL]
2023-08-07 16:31:11.660119 88.88.88.88:47874 73165285
test__cdc [SQL SHOW TABLE STATUS LIKE 'test_cdc_table'; SQL]
2023-08-07 16:31:11.662946 88.88.88.88:47874 73165285
test__cdc [SQL SELECT MAX(`machine_name`) FROM (SELECT `machine_name` FROM
`test___cdc__db`.`test_cdc_table` WHERE `machine_name` >= 'SVR8888' ORDER BY
`machine_name` ASC LIMIT 8096) AS T SQL]
**[missing step to close connection]**
[thread 73165286]
2023-08-07 16:31:11.573639 88.88.88.88:47876 73165286
test__cdc [SQL [LOGIN] SQL]
2023-08-07 16:31:11.574619 88.88.88.88:47876 73165286
test__cdc [SQL /* mysql-connector-java-8.0.28 (Revision:
[7ff2161da3899f379fb3171b6538b191b1c5c7e2|https://github.com/apache/flink-cdc/commit/7ff2161da3899f379fb3171b6538b191b1c5c7e2])
*/SELECT @@session.auto_increment_increment AS auto_increment_increment,
@@character_set_client AS character_set_client, @@character_set_connection AS
character_set_connection, @@character_set_results AS character_set_results,
@@character_set_server AS character_set_server, @@collation_server AS
collation_server, @@collation_connection AS collation_connection,
@@init_connect AS init_connect, @@interactive_timeout AS interactive_timeout,
@@license AS license, @@lower_case_table_names AS lower_case_table_names,
@@max_allowed_packet AS max_allowed_packet, @@net_write_timeout AS
net_write_timeout, @@performance_schema AS performance_schema,
@@query_cache_size AS query_cache_size, @@query_cache_type AS query_cache_type,
@@sql_mode AS sql_mode, @@system_time_zone AS system_time_zone, @@time_zone AS
time_zone, @@transaction_isolation AS transaction_isolation, @@wait_timeout AS
wait_timeout SQL]
2023-08-07 16:31:11.575502 88.88.88.88:47876 73165286
test__cdc [SQL SHOW WARNINGS SQL]
2023-08-07 16:31:11.576469 88.88.88.88:47876 73165286
test__cdc [SQL SET autocommit=1 SQL]
2023-08-07 16:31:11.577338 88.88.88.88:47876 73165286
test__cdc [SQL SELECT @@session.transaction_read_only SQL]
2023-08-07 16:31:11.587311 88.88.88.88:47876 73165286
test__cdc [SQL SHOW DATABASES SQL]
2023-08-07 16:31:11.588581 88.88.88.88:47876 73165286
test__cdc [SQL SHOW FULL TABLES IN `information_schema` where Table_Type =
'BASE TABLE' SQL]
2023-08-07 16:31:11.598959 88.88.88.88:47876 73165286
test__cdc [SQL SHOW FULL TABLES IN `magidb` where Table_Type = 'BASE
TABLE' SQL]
2023-08-07 16:31:11.600502 88.88.88.88:47876 73165286
test__cdc [SQL SHOW FULL TABLES IN `mysql` where Table_Type = 'BASE TABLE'
SQL]
2023-08-07 16:31:11.606279 88.88.88.88:47876 73165286
test__cdc [SQL SHOW FULL TABLES IN `performance_schema` where Table_Type =
'BASE TABLE' SQL]
2023-08-07 16:31:11.630433 88.88.88.88:47876 73165286
test__cdc [SQL SHOW FULL TABLES IN `sys` where Table_Type = 'BASE TABLE'
SQL]
**[missing step to close connection]**
[thread 73165293]
2023-08-07 16:31:12.675640 88.88.88.88:47898 73165293
test__cdc [SQL [LOGIN] SQL]
2023-08-07 16:31:12.676587 88.88.88.88:47898 73165293
test__cdc [SQL /* mysql-connector-java-8.0.28 (Revision:
[7ff2161da3899f379fb3171b6538b191b1c5c7e2|https://github.com/apache/flink-cdc/commit/7ff2161da3899f379fb3171b6538b191b1c5c7e2])
*/SELECT @@session.auto_increment_increment AS auto_increment_increment,
@@character_set_client AS character_set_client, @@character_set_connection AS
character_set_connection, @@character_set_results AS character_set_results,
@@character_set_server AS character_set_server, @@collation_server AS
collation_server, @@collation_connection AS collation_connection,
@@init_connect AS init_connect, @@interactive_timeout AS interactive_timeout,
@@license AS license, @@lower_case_table_names AS lower_case_table_names,
@@max_allowed_packet AS max_allowed_packet, @@net_write_timeout AS
net_write_timeout, @@performance_schema AS performance_schema,
@@query_cache_size AS query_cache_size, @@query_cache_type AS query_cache_type,
@@sql_mode AS sql_mode, @@system_time_zone AS system_time_zone, @@time_zone AS
time_zone, @@transaction_isolation AS transaction_isolation, @@wait_timeout AS
wait_timeout SQL]
2023-08-07 16:31:12.677424 88.88.88.88:47898 73165293
test__cdc [SQL SHOW WARNINGS SQL]
2023-08-07 16:31:12.678321 88.88.88.88:47898 73165293
test__cdc [SQL SET autocommit=1 SQL]
2023-08-07 16:31:12.683095 88.88.88.88:47898 73165293
test__cdc [SQL SHOW MASTER STATUS SQL]
2023-08-07 16:31:12.684154 88.88.88.88:47898 73165293
test__cdc [SQL SELECT
GTID_SUBTRACT('5015a2c3-6a73-11ea-a91b-246e96882760:1-5303826168,b65182c3-b201-11ed-9217-0c42a19a1d62:1-7696392432',
'5015a2c3-6a73-11ea-a91b-246e96882760:1-5303826168,b65182c3-b201-11ed-9217-0c42a19a1d62:1-7696391849')
SQL]
2023-08-07 16:31:12.685018 88.88.88.88:47898 73165293
test__cdc [SQL SELECT @@global.gtid_purged SQL]
2023-08-07 16:31:12.685887 88.88.88.88:47898 73165293
test__cdc [SQL SELECT
GTID_SUBTRACT('b65182c3-b201-11ed-9217-0c42a19a1d62:7696391850-7696392432',
'5015a2c3-6a73-11ea-a91b-246e96882760:1-5303826168,b65182c3-b201-11ed-9217-0c42a19a1d62:1-7694307400')
SQL]
2023-08-07 16:31:12.689066 88.88.88.88:47898 73165293
test__cdc [SQL SHOW GLOBAL VARIABLES LIKE 'GTID_MODE' SQL]
2023-08-07 16:31:12.689930 88.88.88.88:47898 73165293
test__cdc [SQL SHOW MASTER STATUS SQL]
2023-08-07 16:31:12.690788 88.88.88.88:47898 73165293
test__cdc [SQL SELECT @@global.gtid_purged SQL]
**[missing step to close connection]**
[thread 73165294]
2023-08-07 16:31:12.693765 88.88.88.88:47900 73165294
test__cdc [SQL [LOGIN] SQL]
2023-08-07 16:31:12.694955 88.88.88.88:47900 73165294
test__cdc [SQL show global variables like 'binlog_checksum' SQL]
2023-08-07 16:31:12.695767 88.88.88.88:47900 73165294
test__cdc [SQL set @master_binlog_checksum= @@global.binlog_checksum SQL]
2023-08-07 16:31:12.696551 88.88.88.88:47900 73165294
test__cdc [SQL select @@server_id SQL]
2023-08-07 16:31:12.697329 88.88.88.88:47900 73165294
test__cdc [SQL set @master_heartbeat_period=48000000000 SQL]
### What did you expect to see?
After successful initialization, only one dump thread should be retaine.
### What did you see instead?
After successful initialization, have 4 mysql threads.
One is dump thread, and the other three are zombie threads.
### Anything else?
_No response_
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
---------------- Imported from GitHub ----------------
Url: https://github.com/apache/flink-cdc/issues/2371
Created by: [vzong|https://github.com/vzong]
Labels: bug,
Created at: Mon Aug 07 17:06:06 CST 2023
State: open
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
