[
https://issues.apache.org/jira/browse/KAFKA-5734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16132992#comment-16132992
]
jang edited comment on KAFKA-5734 at 8/18/17 1:58 PM:
------------------------------------------------------
I figured out what was problem during jconsole monitoring.
Sending messages program written in java creates many threads those are going
away after its task.
In each thread, kafka producer instance is created at begin of thread and
closed(producer.close(0, TimeUnit.MILLISECONDS) ) at the end of thread task.
When the producer instance is created, MBean "kafka.server:type=Produce"
accumulates with unique client-id.
Due to kafka.server:type=Produce metric, kafka.server:type=kafka-metrics-count
increases gradually.
!attachment-name.jpg|thumbnail!
Should I use only finite number of producer and reuse for threads like kafka
producer pool?
was (Author: jangbora):
I figured out what was problem during jconsole monitoring.
Sending messages program written in java creates many threads those are going
away after its task.
In each thread, kafka producer instance is created at begin of thread and
closed(producer.close(0, TimeUnit.MILLISECONDS) ) at the end of thread task.
When the producer instance is created, MBean "kafka.server:type=Produce"
accumulates with unique client-id.
Due to kafka.server:type=Produce metric, kafka.server:type=kafka-metrics-count
increases gradually.
Should I use only finite number of producer and reuse for threads like kafka
producer pool?
> Heap (Old generation space) gradually increase
> ----------------------------------------------
>
> Key: KAFKA-5734
> URL: https://issues.apache.org/jira/browse/KAFKA-5734
> Project: Kafka
> Issue Type: Bug
> Components: metrics
> Affects Versions: 0.10.2.0
> Environment: ubuntu 14.04 / java 1.7.0
> Reporter: jang
> Attachments: heap-log.xlsx, jconsole.png
>
>
> I set up kafka server on ubuntu with 4GB ram.
> Heap ( Old generation space ) size is increasing gradually like attached
> excel file which recorded gc info in 1 minute interval.
> Finally OU occupies 2.6GB and GC expend too much time ( And out of memory
> exception )
> kafka process argumens are below.
> _java -Xmx3000M -Xms2G -server -XX:+UseG1GC -XX:MaxGCPauseMillis=20
> -XX:InitiatingHeapOccupancyPercent=35 -XX:+DisableExplicitGC
> -Djava.awt.headless=true
> -Xloggc:/usr/local/kafka/bin/../logs/kafkaServer-gc.log -verbose:gc
> -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps
> -Dcom.sun.management.jmxremote
> -Dcom.sun.management.jmxremote.authenticate=false
> -Dcom.sun.management.jmxremote.ssl=false
> -Dkafka.logs.dir=/usr/local/kafka/bin/../logs
> -Dlog4j.configuration=file:/usr/local/kafka/bin/../config/log4j.properties_
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
