Hi, We are experiencing an issue where the RecordAccumulator size is growing beyond the configured buffer.memory size .
Below is the sequence of events when this issue occured - 1. One of the broker out of 10 brokers was brought down at 15:10 PM 2.Till 17:11 PM (same date) the kafka producer instance was in healthy state 3. The same down broker was brought up at ~17:11PM ( same date) 4. Observed UnknownTopicOrPartitionException for some messages 5. HeapDump on 17:14 shows the RecordAccumulator size around ~2.9 GB way above the configured buffer.size =120MB Other Details - kafka version = 0.8.2.1 Below is the kafka producer properties configured - kafka.bootstrap.servers=qa-vip-url kafka.acks=1 kafka.buffer.memory=125829120 kafka.compression.type=none kafka.retries=3 kafka.batch.size=200 kafka.client.id=service-instance-id kafka.linger.ms=0 kafka.max.request.size=5242880 kafka.receive.buffer.bytes=32768 kafka.send.buffer.bytes=131072 kafka.timeout.ms=3000 kafka.block.on.buffer.full=false kafka.metadata.fetch.timeout.ms=60000 kafka.metadata.max.age.ms=300000 kafka.reconnect.backoff.ms=30 kafka.retry.backoff.ms=3000 kafka.key.serializer=org.apache.kafka.common.serialization.ByteArraySerializer kafka.value.serializer=org.apache.kafka.common.serialization.ByteArraySerializer Please help in identifing the root cause which is causing this memory leak for RecordAccumulator. Attached is the HeapDump overview for reference Let me know if you need any other information Regards, Liju John
