[
https://issues.apache.org/jira/browse/KAFKA-20650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vismay reassigned KAFKA-20650:
------------------------------
Assignee: Vismay
> In combined mode, BrokerServer and ControllerServer each independently called
> QuotaFactory.instantiate(), which created a separate ClientQuotaCallback
> instance via reflection for each role.
> ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: KAFKA-20650
> URL: https://issues.apache.org/jira/browse/KAFKA-20650
> Project: Kafka
> Issue Type: Bug
> Components: core
> Affects Versions: 2.8.0, 2.8.1, 2.8.2, 2.8.3, 3.0.0, 3.0.1, 3.0.2, 3.1.0,
> 3.1.1, 3.1.2, 3.2.0, 3.2.1, 3.2.2, 3.2.3, 3.3.0, 3.3.1, 3.3.2, 3.4.0, 3.4.1,
> 3.5.0, 3.5.1, 3.5.2, 3.5.3, 3.6.0, 3.6.1, 3.6.2, 3.7.0, 3.7.1, 3.7.2, 3.8.0,
> 3.8.1, 3.9.0, 3.9.1, 3.9.2, 4.0.0, 4.0.1, 4.1.0, 4.2.0, 4.3.0, 4.0.2, 4.1.1,
> 4.1.2
> Reporter: Patrick Allen
> Assignee: Vismay
> Priority: Major
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> In combined mode,
> [BrokerServer|https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/server/BrokerServer.scala]
> and
> [ControllerServer|https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/server/ControllerServer.scala]
> each independently called
> [QuotaFactory.instantiate()|https://github.com/apache/kafka/blob/trunk/core/src/main/java/kafka/server/QuotaFactory.java],
> which created a separate ClientQuotaCallback instance via reflection for
> each role.
> {code:java}
> // ControllerServer.scala
> quotaManagers = QuotaFactory.instantiate(config,
> metrics,
> time,
> s"controller-${config.nodeId}-",
> ProcessRole.ControllerRole.toString){code}
> {code:java}
> // BrokerServer.scala
> quotaManagers = QuotaFactory.instantiate(config, metrics, time,
> s"broker-${config.nodeId}-", ProcessRole.BrokerRole.toString) {code}
> {code:java}
> // QuotaFactory.java
> Optional<Plugin<ClientQuotaCallback>> clientQuotaCallbackPlugin =
> createClientQuotaCallback(cfg, metrics, role);{code}
>
> This means that two independent callback objects are created with divergent
> state in the same JVM — a regression from the pre-KRaft single-server
> architecture where only one instance existed.
> This only affects combined instances, which in theory should only be used in
> development environments as suggested by Kafka. However, at my company we use
> a custom quota plugin for our Kafka instances, which are often run on
> resource constrained environments where we unfortunately have to run in
> combined mode. We only noticed this issue when migrating to KRaft for version
> 4.0 but I believe it has been present since KRaft's inception (I checked 2.8
> which did indeed have this).
> I have written a simple fix where the quota callback is instead created in
> the shared server, and then passed into the QuotaFactory instantiate as a
> variable so that only a single instance is used across both servers. This is
> in my opinion the quickest fix which I am happy to contribute (waiting on
> developer perms before I can assign this to myself).
> But I do have a wider question of whether Kafka wants to enable custom quota
> management on the controller, and if perhaps we should allow for it also to
> be set in config such as to prevent this issue when in combined:
> {code:java}
> controller.client.quota.callback.class: <package>{code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
