flybird created KAFKA-14330:
-------------------------------
Summary: The Kafka Connection Is High During Service Startup
Key: KAFKA-14330
URL: https://issues.apache.org/jira/browse/KAFKA-14330
Project: Kafka
Issue Type: Improvement
Components: consumer
Affects Versions: 2.8.1
Reporter: flybird
The Kafka Connection Is High During Service Startup
During service restart, the number of Kafka connections suddenly increases for
a period of time. The actual number of process connections is 328. However, the
number of consumer threads and producer threads of the process is changed by
querying the number of consumer threads and producer threads. The number of
consumer threads x 2 + producer threads = 283. After a while, the number of
connections changes back to 283.
The TCP connection is created when the KafkaConsumer.poll method is invoked.
There are three times to create a TCP connection inside the poll method.
1. Created when the findCoordinator request is initiated to determine the
coordinator and obtain the cluster metadata.
2. When the coordinator is connected, enable normal group coordination
operations.
3. Obtain the actual message when consuming data.
When a Type 3 TCP connection is successfully created, the consumer program
discards the Type 1 TCP connection and then uses the Type 3 TCP connection
instead when periodically requesting metadata. For a consumer program that has
been running for a while, there are only the latter two types of TCP
connections.
In summary, the producer creates one connection, and the consumer creates three
connections. However, the first type of connection is closed after the third
type of connection is created. In the end, there are only two connections.
The first type of connection analysis is a temporary connection, which causes a
high number of connections. Why not disable the third type of connection
immediately after it is created?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
