[
https://issues.apache.org/jira/browse/CAMEL-19285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17719506#comment-17719506
]
Dylan Piergies commented on CAMEL-19285:
----------------------------------------
Have reproduced again on a different system (though I did experience an odd
issue with my docker-compose setup, which is now fixed). Here's a snippet of
the log output from the Kafka broker:
{{kafka-camel-flood-issue-kafka-1 | [2023-05-04 19:12:09,170] INFO
[SocketServer listenerType=ZK_BROKER, nodeId=1001] Failed authentication with
/172.25.0.1 (channelId=172.25.0.3:9093-172.25.0.1:41098-61) (SSL handshake
failed) (org.apache.kafka.common.network.Selector)
kafka-camel-flood-issue-kafka-1 | [2023-05-04 19:12:09,192] INFO
[SocketServer listenerType=ZK_BROKER, nodeId=1001] Failed authentication with
/172.25.0.1 (channelId=172.25.0.3:9093-172.25.0.1:41110-62) (SSL handshake
failed) (org.apache.kafka.common.network.Selector)
kafka-camel-flood-issue-kafka-1 | [2023-05-04 19:12:09,210] INFO
[SocketServer listenerType=ZK_BROKER, nodeId=1001] Failed authentication with
/172.25.0.1 (channelId=172.25.0.3:9093-172.25.0.1:41114-62) (SSL handshake
failed) (org.apache.kafka.common.network.Selector)
kafka-camel-flood-issue-kafka-1 | [2023-05-04 19:12:09,227] INFO
[SocketServer listenerType=ZK_BROKER, nodeId=1001] Failed authentication with
/172.25.0.1 (channelId=172.25.0.3:9093-172.25.0.1:41126-62) (SSL handshake
failed) (org.apache.kafka.common.network.Selector)
kafka-camel-flood-issue-kafka-1 | [2023-05-04 19:12:09,245] INFO
[SocketServer listenerType=ZK_BROKER, nodeId=1001] Failed authentication with
/172.25.0.1 (channelId=172.25.0.3:9093-172.25.0.1:41136-63) (SSL handshake
failed) (org.apache.kafka.common.network.Selector)
kafka-camel-flood-issue-kafka-1 | [2023-05-04 19:12:09,258] INFO
[SocketServer listenerType=ZK_BROKER, nodeId=1001] Failed authentication with
/172.25.0.1 (channelId=172.25.0.3:9093-172.25.0.1:41138-63) (SSL handshake
failed) (org.apache.kafka.common.network.Selector)
kafka-camel-flood-issue-kafka-1 | [2023-05-04 19:12:09,273] INFO
[SocketServer listenerType=ZK_BROKER, nodeId=1001] Failed authentication with
/172.25.0.1 (channelId=172.25.0.3:9093-172.25.0.1:41140-63) (SSL handshake
failed) (org.apache.kafka.common.network.Selector)
kafka-camel-flood-issue-kafka-1 | [2023-05-04 19:12:09,290] INFO
[SocketServer listenerType=ZK_BROKER, nodeId=1001] Failed authentication with
/172.25.0.1 (channelId=172.25.0.3:9093-172.25.0.1:41150-64) (SSL handshake
failed) (org.apache.kafka.common.network.Selector)
kafka-camel-flood-issue-kafka-1 | [2023-05-04 19:12:09,304] INFO
[SocketServer listenerType=ZK_BROKER, nodeId=1001] Failed authentication with
/172.25.0.1 (channelId=172.25.0.3:9093-172.25.0.1:41162-64) (SSL handshake
failed) (org.apache.kafka.common.network.Selector)
kafka-camel-flood-issue-kafka-1 | [2023-05-04 19:12:09,319] INFO
[SocketServer listenerType=ZK_BROKER, nodeId=1001] Failed authentication with
/172.25.0.1 (channelId=172.25.0.3:9093-172.25.0.1:41166-64) (SSL handshake
failed) (org.apache.kafka.common.network.Selector)
kafka-camel-flood-issue-kafka-1 | [2023-05-04 19:12:09,333] INFO
[SocketServer listenerType=ZK_BROKER, nodeId=1001] Failed authentication with
/172.25.0.1 (channelId=172.25.0.3:9093-172.25.0.1:41176-65) (SSL handshake
failed) (org.apache.kafka.common.network.Selector)
kafka-camel-flood-issue-kafka-1 | [2023-05-04 19:12:09,345] INFO
[SocketServer listenerType=ZK_BROKER, nodeId=1001] Failed authentication with
/172.25.0.1 (channelId=172.25.0.3:9093-172.25.0.1:41190-65) (SSL handshake
failed) (org.apache.kafka.common.network.Selector)
kafka-camel-flood-issue-kafka-1 | [2023-05-04 19:12:09,360] INFO
[SocketServer listenerType=ZK_BROKER, nodeId=1001] Failed authentication with
/172.25.0.1 (channelId=172.25.0.3:9093-172.25.0.1:41192-65) (SSL handshake
failed) (org.apache.kafka.common.network.Selector)
kafka-camel-flood-issue-kafka-1 | [2023-05-04 19:12:09,372] INFO
[SocketServer listenerType=ZK_BROKER, nodeId=1001] Failed authentication with
/172.25.0.1 (channelId=172.25.0.3:9093-172.25.0.1:41204-66) (SSL handshake
failed) (org.apache.kafka.common.network.Selector)
kafka-camel-flood-issue-kafka-1 | [2023-05-04 19:12:09,387] INFO
[SocketServer listenerType=ZK_BROKER, nodeId=1001] Failed authentication with
/172.25.0.1 (channelId=172.25.0.3:9093-172.25.0.1:41218-66) (SSL handshake
failed) (org.apache.kafka.common.network.Selector)
kafka-camel-flood-issue-kafka-1 | [2023-05-04 19:12:09,400] INFO
[SocketServer listenerType=ZK_BROKER, nodeId=1001] Failed authentication with
/172.25.0.1 (channelId=172.25.0.3:9093-172.25.0.1:41220-66) (SSL handshake
failed) (org.apache.kafka.common.network.Selector)
kafka-camel-flood-issue-kafka-1 | [2023-05-04 19:12:09,413] INFO
[SocketServer listenerType=ZK_BROKER, nodeId=1001] Failed authentication with
/172.25.0.1 (channelId=172.25.0.3:9093-172.25.0.1:41224-67) (SSL handshake
failed) (org.apache.kafka.common.network.Selector)
kafka-camel-flood-issue-kafka-1 | [2023-05-04 19:12:09,426] INFO
[SocketServer listenerType=ZK_BROKER, nodeId=1001] Failed authentication with
/172.25.0.1 (channelId=172.25.0.3:9093-172.25.0.1:41234-67) (SSL handshake
failed) (org.apache.kafka.common.network.Selector)
kafka-camel-flood-issue-kafka-1 | [2023-05-04 19:12:09,441] INFO
[SocketServer listenerType=ZK_BROKER, nodeId=1001] Failed authentication with
/172.25.0.1 (channelId=172.25.0.3:9093-172.25.0.1:41238-67) (SSL handshake
failed) (org.apache.kafka.common.network.Selector)
kafka-camel-flood-issue-kafka-1 | [2023-05-04 19:12:09,458] INFO
[SocketServer listenerType=ZK_BROKER, nodeId=1001] Failed authentication with
/172.25.0.1 (channelId=172.25.0.3:9093-172.25.0.1:41254-68) (SSL handshake
failed) (org.apache.kafka.common.network.Selector)
kafka-camel-flood-issue-kafka-1 | [2023-05-04 19:12:09,477] INFO
[SocketServer listenerType=ZK_BROKER, nodeId=1001] Failed authentication with
/172.25.0.1 (channelId=172.25.0.3:9093-172.25.0.1:41258-68) (SSL handshake
failed) (org.apache.kafka.common.network.Selector)
kafka-camel-flood-issue-kafka-1 | [2023-05-04 19:12:09,490] INFO
[SocketServer listenerType=ZK_BROKER, nodeId=1001] Failed authentication with
/172.25.0.1 (channelId=172.25.0.3:9093-172.25.0.1:41262-68) (SSL handshake
failed) (org.apache.kafka.common.network.Selector)
kafka-camel-flood-issue-kafka-1 | [2023-05-04 19:12:09,502] INFO
[SocketServer listenerType=ZK_BROKER, nodeId=1001] Failed authentication with
/172.25.0.1 (channelId=172.25.0.3:9093-172.25.0.1:41270-69) (SSL handshake
failed) (org.apache.kafka.common.network.Selector)
kafka-camel-flood-issue-kafka-1 | [2023-05-04 19:12:09,515] INFO
[SocketServer listenerType=ZK_BROKER, nodeId=1001] Failed authentication with
/172.25.0.1 (channelId=172.25.0.3:9093-172.25.0.1:41284-69) (SSL handshake
failed) (org.apache.kafka.common.network.Selector)
kafka-camel-flood-issue-kafka-1 | [2023-05-04 19:12:09,529] INFO
[SocketServer listenerType=ZK_BROKER, nodeId=1001] Failed authentication with
/172.25.0.1 (channelId=172.25.0.3:9093-172.25.0.1:41292-69) (SSL handshake
failed) (org.apache.kafka.common.network.Selector)
kafka-camel-flood-issue-kafka-1 | [2023-05-04 19:12:09,541] INFO
[SocketServer listenerType=ZK_BROKER, nodeId=1001] Failed authentication with
/172.25.0.1 (channelId=172.25.0.3:9093-172.25.0.1:41302-70) (SSL handshake
failed) (org.apache.kafka.common.network.Selector)}}
It is visible from the timestamps that the backoff period is not being
respected.
> Kafka consumer can flood brokers if TLS handshake fails and pollOnError is
> set to RECONNECT
> -------------------------------------------------------------------------------------------
>
> Key: CAMEL-19285
> URL: https://issues.apache.org/jira/browse/CAMEL-19285
> Project: Camel
> Issue Type: Bug
> Components: camel-kafka
> Affects Versions: 3.20.2, 3.20.3
> Reporter: Dylan Piergies
> Priority: Major
> Fix For: 3.20.5, 3.21.0, 4.0-RC1, 4.0
>
>
> The Kafka consumer does not respect reconnect backoff options when a TLS
> handshake fails if the consumer's {{pollOnError}} option is set to
> {{{}RECONNECT{}}}, resulting in reconnection attempts being made in a tight
> loop without delays, meaning that Camel applications consuming from Kafka
> topics can effectively mount a DDoS attack on the Kafka broker. This effect
> is amplified if concurrent consumers are in use, since each consumer thread
> is making its own connection attempts.
> Naturally, we found this out the hard way, in production, when another team
> put in place a firewall rule to allow connections from our consumers. The
> amount of TLS handshake traffic generated was sufficient to overwhelm the
> broker, resulting in an outage.
> I have created a small project to demonstrate the issue against a
> containerised Kafka broker here:
> [https://github.com/dylanpiergies/kafka-camel-flood-issue]
> This issue does not occur when a connection fails for other reasons (e.g.
> connection refused, connection timeout); in these cases the reconnect backoff
> behaves as expected.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
