Hi all,
Currently with default settings
min.cleanable.dirty.ratio/log.cleaner.min.cleanable.ratio (0.5)
__consumer_offsets partition size grows up to 80Mb and this seems to slow down
metadata related operations (notably commits / join group).
What are the normal/expected partition avg size for the __consumer_offsets
topic ?
Are there any recommendations to settings
min.cleanable.dirty.ratio/log.cleaner.min.cleanable.ratio for the
__consumer_offsets topic ?
Kafka Cluster
• version 0.10.1
• 6 nodes
• __consumer_offsets #partitions : 50
• Cleaner config :
○ min.cleanable.dirty.ratio : 0.5
○ log.cleaner.enable : True
○ log.cleaner.io.buffer.size : 524288
○ log.cleaner.dedupe.buffer.size : 1073741824
○ log.cleaner.io.max.bytes.per.second: 104857600
Regards,
Amine