[ https://issues.apache.org/jira/browse/KAFKA-7422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16620486#comment-16620486 ]
Shantanu Deshmukh commented on KAFKA-7422: ------------------------------------------ What do you mean by valid SSL store and valid SSL port? > Huge memory consumption (leak?) by Kafka Producer > ------------------------------------------------- > > Key: KAFKA-7422 > URL: https://issues.apache.org/jira/browse/KAFKA-7422 > Project: Kafka > Issue Type: Bug > Components: producer > Affects Versions: 0.10.1.0 > Environment: OS - AIX 7.1, Java - IBM Java 1.8 > Reporter: Shantanu Deshmukh > Priority: Major > Attachments: heapdump.20180918.111423.4587876.0002.phd > > > Hello, > > We have a 3 broker Kafka 0.10.1.0 deployment in production. There are some > applications which have Kafka Producers embedded in them which send > application logs to a topic. This topic has 10 partitions with replication > factor of 3. > > We are observing that memory usage on some of these application servers keep > shooting through the roof intermittently. After taking heapdump we found out > that top suspects were: > *---------------------* > *org.apache.kafka.common.network.Selector -* > occupies *352,519,104 (24.96%)* bytes. The memory is accumulated in one > instance of *"byte[]"* loaded by *"<system class loader>"*. > *org.apache.kafka.common.network.KafkaChannel -* > occupies *352,527,424 (24.96%)* bytes. The memory is accumulated in one > instance of *"byte[]"* loaded by *"<system class loader>"* > *---------------------* > Both of these were holding about 352MB of space. 3 such instances, so they > were consuming about 1.2GB of memory. > Now regarding usage of producers. Not a huge amount of logs are being sent to > Kafka cluster. It is about 200 msgs/sec. Only one producer object is being > used throughout application. Async send function is used. > What could be the cause of such huge memory usage? Is this some sort of > memory leak in this specific Kafka version? > Here's producer config being used at the present. > {{>>>>>>>>>>>>>>>>}} > {{kafka.bootstrap.servers=x.x.x.x:9092,x.x.x.x:9092,x.x.x.x:9092}} > {{ kafka.acks=0}} > {{ > kafka.key.serializer=org.apache.kafka.common.serialization.StringSerializer}} > {{ > kafka.value.serializer=org.apache.kafka.common.serialization.StringSerializer}} > {{[kafka.max.block.ms|http://kafka.max.block.ms/]=1000}} > {{[kafka.request.timeout.ms|http://kafka.request.timeout.ms/]=1000}} > {{ kafka.max.in.flight.requests.per.connection=1}} > {{ kafka.retries=0}} > {{ kafka.compression.type=gzip}} > {{ kafka.security.protocol=SSL}} > {{ kafka.ssl.truststore.location=/data/kafka/kafka-server-truststore.jks}} > {{ kafka.ssl.truststore.password=XXXXXX}} > {{ logger.level=INFO }}{{<<<<<<<<<<<<<<}} > Attaching heapdump for your perusal. -- This message was sent by Atlassian JIRA (v7.6.3#76005)