[
https://issues.apache.org/jira/browse/KAFKA-9239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anders Eknert updated KAFKA-9239:
---------------------------------
Description:
Having experimented some with custom authorization options for Kafka on the
broker side, we have a bunch of clients that are no longer authorized. While
that's expected and fine, we did not anticipate the level of logging that these
unauthorized clients would spew out - putting our whole logging subsystem under
heavy stress.
The message log is similar to the one below:
{code:java}
2019-11-25 10:08:10.262 WARN 1 --- [ntainer#0-0-C-1]
o.a.k.c.consumer.internals.Fetcher : [Consumer clientId=sdp-ee-miami-0,
groupId=sdp-ee-miami] Not authorized to read from topic sdp.ee-miami.
{code}
In just 4 hours this same message was repeated about a hundred million times (
! ) in the worst offending client, 74 million times in the next one and 72
million times in the third.
We will roll out customized burst filters to suppress this on the client
loggers, but it would of course be best if this was fixed in the client.
was:
Having experimented some with custom authorization options for Kafka on the
broker side, we have a bunch of clients that are no longer authorized. While
that's expected and fine, we did not anticipate the level of logging that these
unauthorized clients would spew out - putting our whole logging subsystem under
heavy stress.
The message log is similar to the one below:
{code:java}
2019-11-25 10:08:10.262 WARN 1 --- [ntainer#0-0-C-1]
o.a.k.c.consumer.internals.Fetcher : [Consumer clientId=sdp-ee-miami-0,
groupId=sdp-ee-miami] Not authorized to read from topic sdp.ee-miami.
{code}
In just 4 hours this same message was repeated about a hundred million times(!)
in the worst offending client, 74 million times in the next one and 72 million
times in the third.
We will roll out customized burst filters to suppress this on the client
loggers, but it would of course be best if this was fixed in the client.
> Extreme amounts of logging done by unauthorized Kafka clients
> -------------------------------------------------------------
>
> Key: KAFKA-9239
> URL: https://issues.apache.org/jira/browse/KAFKA-9239
> Project: Kafka
> Issue Type: Bug
> Components: clients
> Reporter: Anders Eknert
> Priority: Major
> Attachments: Screenshot 2019-11-27 at 11.32.38.png
>
>
> Having experimented some with custom authorization options for Kafka on the
> broker side, we have a bunch of clients that are no longer authorized. While
> that's expected and fine, we did not anticipate the level of logging that
> these unauthorized clients would spew out - putting our whole logging
> subsystem under heavy stress.
> The message log is similar to the one below:
> {code:java}
> 2019-11-25 10:08:10.262 WARN 1 --- [ntainer#0-0-C-1]
> o.a.k.c.consumer.internals.Fetcher : [Consumer clientId=sdp-ee-miami-0,
> groupId=sdp-ee-miami] Not authorized to read from topic sdp.ee-miami.
> {code}
> In just 4 hours this same message was repeated about a hundred million times
> ( ! ) in the worst offending client, 74 million times in the next one and 72
> million times in the third.
> We will roll out customized burst filters to suppress this on the client
> loggers, but it would of course be best if this was fixed in the client.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
