[
https://issues.apache.org/jira/browse/KAFKA-16344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17823817#comment-17823817
]
Greg Harris commented on KAFKA-16344:
-------------------------------------
Hi [~janardhanag], thanks for the ticket.
At the current time, the offset syncs topic cannot have more than 1 partition.
If more than one partition is present, the MirrorSourceTask will only write to
partition 0, and the MirrorCheckpointTask will only read from partition 0.
Changes to both of these would be necessary to support partitioning that topic,
and would require a KIP.
For a workaround, are you able to increase your `offset.lag.max` from the
default 100? This will make the offset translation less accurate, but can
decrease the throughput on that topic.
Do you have some more statistics that confirm that this topic is too active? It
should have just a fraction of the load of the other topics, so i'd be
interested to know more about the scale you're working at.
Thanks,
Greg
> Internal topic mm2-offset-syncs<clustername>internal created with single
> partition is putting more load on the broker
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: KAFKA-16344
> URL: https://issues.apache.org/jira/browse/KAFKA-16344
> Project: Kafka
> Issue Type: Bug
> Components: connect
> Affects Versions: 3.5.1
> Reporter: Janardhana Gopalachar
> Priority: Major
>
> We are using Kafka 3.5.1 version, we see that the internal topic created by
> mirrormaker
> mm2-offset-syncs<clustername>internal is created with single partition due to
> which the CPU load on the broker which will be leader for this partition is
> increased compared to other brokers. Can multiple partitions be created for
> the topic so that the CPU load would get distributed
>
> Topic: mm2-offset-syncscluster-ainternal TopicId: XRvTDbogT8ytNhqX2YTyrA
> PartitionCount: 1ReplicationFactor: 3 Configs:
> min.insync.replicas=2,cleanup.policy=compact,message.format.version=3.0-IV1
> Topic: mm2-offset-syncscluster-ainternal Partition: 0 Leader: 2
> Replicas: 2,1,0 Isr: 2,1,0
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
