[
https://issues.apache.org/jira/browse/KAFKA-12946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17363496#comment-17363496
]
Ron Dagostino commented on KAFKA-12946:
---------------------------------------
I mean if you take a look at the size on disk, is the size of the log
significantly smaller? Broker 0 might be the leader for partition 0 with 600
GB of size. Maybe broker 1 is a follower with about the same 600 GB size, but
perhaps broker 2 is a follower with just 100 MB. It is unexplained why this
would occur, but it is possible, and if so then you can make 2 the leader, move
1 to 3, move 3 back to 1, move 0 to 3, move 3 back to 0, and then make 0 the
leader again -- now you have the same leadership and followers as before but
100 MB on all 3 replicas.
> __consumer_offsets topic with very big partitions
> -------------------------------------------------
>
> Key: KAFKA-12946
> URL: https://issues.apache.org/jira/browse/KAFKA-12946
> Project: Kafka
> Issue Type: Bug
> Components: log cleaner
> Affects Versions: 2.0.0
> Reporter: Emi
> Priority: Critical
>
> I am using Kafka 2.0.0 with java 8u191
> There is a partitions of the __consumer_offsets topic that is 600 GB with
> 6000 segments older than 4 months. Other partitions of that topic are small:
> 20-30 MB.
> There are 60 consumer groups, 90 topics and 100 partitions per topic.
> There aren't errors in the logs. From the log of the logcleaner, I can see
> that partition is never touched from the logcleaner thread for the
> compaction, but it only add new segments.
> How is this possible?
> There was another partition with the same problem, but after some months it
> has been compacted. Now there is only one partition with this problem, but
> this is bigger and keep growing
> I have used the kafka-dump-log tool to check these old segments and I can see
> many duplicates. So I would assume that is not compacted.
> My settings:
> {{offsets.commit.required.acks = -1}}
> {{[offsets.commit.timeout.ms|http://offsets.commit.timeout.ms/]}} = 5000
> {{offsets.load.buffer.size = 5242880}}
>
> {{[offsets.retention.check.interval.ms|http://offsets.retention.check.interval.ms/]}}
> = 600000
> {{offsets.retention.minutes = 10080}}
> {{offsets.topic.compression.codec = 0}}
> {{offsets.topic.num.partitions = 50}}
> {{offsets.topic.replication.factor = 3}}
> {{offsets.topic.segment.bytes = 104857600}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
