[
https://issues.apache.org/jira/browse/KAFKA-7983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17057633#comment-17057633
]
ASF GitHub Bot commented on KAFKA-7983:
---------------------------------------
d8tltanc commented on pull request #8283: [WIP] KAFKA-7983: supporting
replication.throttled.replicas in dynamic broker configuration
URL: https://github.com/apache/kafka/pull/8283
**More detailed description of your change**
> In KIP-226, we added the support to change broker defaults dynamically.
However, it didn't support changing leader.replication.throttled.replicas and
follower.replication.throttled.replicas. These 2 configs were introduced in
KIP-73 and controls the set of topic partitions on which replication throttling
will be engaged. One useful case is to be able to set a default value for both
configs to * to allow throttling to be engaged for all topic partitions.
Currently, the static default value for both configs are ignored for
replication throttling, it would be useful to fix that as well.
> leader.replication.throttled.replicas and
follower.replication.throttled.replicas are dynamically set through
ReplicationQuotaManager.markThrottled() at the topic level. However, these two
properties don't exist at the broker level config and BrokerConfigHandler
doesn't call ReplicationQuotaManager.markThrottled(). So, currently, we can't
set leader.replication.throttled.replicas and
follower.replication.throttled.replicas at the broker level either statically
or dynamically.
In this patch, we introduced two new dynamic broker configs, both of them
are type of boolean:
"leader.replication.throttled" (default: false)
"follower.replication.throttled" (default: false)
If "leader.replication.throttled" is set to "true", all leader brokers will
be throttled. Similarly, if "follower.replication.throttled" is set to "true",
all follower brokers will be throttled. The throttle mechanism is introduced in
KIP-73.
To implement the broker level throttle, I added a new class variable to
ReplicationQuotaManager. The BrokerConfigHandler will call
updateBrokerThrottle() and update this class variable upon receiving the config
change notification from ZooKeeper. ReplicationQuotaManager::isThrottled()
*Summary of testing strategy (including rationale)
Added
ReplicationQuotaManagerTest::shouldBrokerLevelThrottleAffectAllTopicPartition()
to test if all topic partitions will be throttled when the broker is throttled.
I'm currently working on adding more tests.
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> supporting replication.throttled.replicas in dynamic broker configuration
> -------------------------------------------------------------------------
>
> Key: KAFKA-7983
> URL: https://issues.apache.org/jira/browse/KAFKA-7983
> Project: Kafka
> Issue Type: New Feature
> Components: core
> Reporter: Jun Rao
> Assignee: Cheng Tan
> Priority: Major
>
> In
> [KIP-226|https://cwiki.apache.org/confluence/display/KAFKA/KIP-226+-+Dynamic+Broker+Configuration#KIP-226-DynamicBrokerConfiguration-DefaultTopicconfigs],
> we added the support to change broker defaults dynamically. However, it
> didn't support changing leader.replication.throttled.replicas and
> follower.replication.throttled.replicas. These 2 configs were introduced in
> [KIP-73|https://cwiki.apache.org/confluence/display/KAFKA/KIP-73+Replication+Quotas]
> and controls the set of topic partitions on which replication throttling
> will be engaged. One useful case is to be able to set a default value for
> both configs to * to allow throttling to be engaged for all topic partitions.
> Currently, the static default value for both configs are ignored for
> replication throttling, it would be useful to fix that as well.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
