[
https://issues.apache.org/jira/browse/KAFKA-1024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13751846#comment-13751846
]
Jason Toffaletti commented on KAFKA-1024:
-----------------------------------------
{noformat}
num #instances #bytes Class description
--------------------------------------------------------------------------
1: 548504 29644120 char[]
2: 14663 14153936 byte[]
3: 547282 13134768 java.lang.String
4: 114927 9202528 java.util.HashMap$Entry[]
5: 64379 8766520 * MethodKlass
6: 64379 8314168 * ConstMethodKlass
7: 127311 6885688 java.lang.Object[]
8: 112352 6291712 java.util.HashMap
9: 5579 5986248 * ConstantPoolKlass
10: 5579 4657080 * InstanceKlassKlass
11: 55875 3582984 java.util.Hashtable$Entry[]
12: 111601 3571232 java.util.Vector
13: 4652 3436992 * ConstantPoolCacheKlass
14: 82004 3280160 java.util.TreeMap$Entry
15: 55819 3125864 java.util.Hashtable
16: 55797 2231880 java.security.ProtectionDomain
17: 16616 2048808 int[]
...
Total : 2632815 148775120
Heap traversal took 51.673 seconds.
{noformat}
The growth is slow but still there. I'm going to try a different garbage
collector to rule that out. This could just be some odd interaction between
G1GC and Scala creating lots of short lived objects. As I understand, G1GC
tries to run collections in a fixed amount of time, so it might just be getting
overwhelmed with garbage. I also haven't forgotten your advice to try a bigger
heap.
> possible memory leak in 0.8 beta1 producer with G1GC
> ----------------------------------------------------
>
> Key: KAFKA-1024
> URL: https://issues.apache.org/jira/browse/KAFKA-1024
> Project: Kafka
> Issue Type: Bug
> Affects Versions: 0.8
> Environment: java version "1.7.0_17"
> Java(TM) SE Runtime Environment (build 1.7.0_17-b02)
> Java HotSpot(TM) 64-Bit Server VM (build 23.7-b01, mixed mode)
> Reporter: Jason Toffaletti
>
> I have this in my pom.xml
> <dependency>
> <groupId>org.apache.kafka</groupId>
> <artifactId>kafka_2.9.2</artifactId>
> <version>0.8.0-beta1</version>
> </dependency>
> <dependency>
> <groupId>org.scala-lang</groupId>
> <artifactId>scala-library</artifactId>
> <version>2.9.2</version>
> </dependency>
> <dependency>
> <groupId>org.xerial.snappy</groupId>
> <artifactId>snappy-java</artifactId>
> <version>1.0.4.1</version>
> </dependency>
> I'm using snappy compression codec and producing about 7k msg/sec on average.
> I'm producing batches up to 10 messages per batch with null keys to a topic
> with 16 partitions, no replication. Xms and Xmx are 64MB and I'm using
> XX:+UseG1GC. After about 12 hours of operation heap usage hits right up
> against the 64MB limit and the producer drops to about 4k msg/sec because of
> the GC pressure. When I restart the process the heap usage goes back to
> normal (around 30MB) and the producer does 7k msg/sec again.
> What else can I provide to help debug this?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
