You can take a look at the GC setting described in https://cwiki.apache.org/confluence/display/KAFKA/Operations
Thanks, Jun On Fri, Jan 31, 2014 at 2:26 AM, Florian Ollech <[email protected]>wrote: > Hi, > > I am currently trying to setup Kafka 0.8 and ran into a problem with the > producer. Every time the server triggers a garbage collection, because > permgen space is full, the server descends into a very unusual memory state > where a lot of garbage collection is suddenly happening (~5-10% gc cpu > time, when normally it is 0-0.5%). Additionally a lot of objects suddenly > get promoted from eden space to survivor space, old gen and perm gen. This > obviously slows down the whole server and turns up the system load by a > lot. > This behaviour only appears when the producer is active during a perm gen > gc and can be remedied by reloading the producer(s). Since reloading the > producer(s) every 5 seconds really is not how it is supposed to work, I am > wondering if any one has any pointers as to where the problem may be? > > My system has the following specs: > - oracle jdk 1.6.0_22-b04 > - sending about 12000 messages per second, about 60 bytes each > - producer counts tried: 1, 10, 25, 50 > - async producer > - ack makes no difference > - batch sizes tried: 100 - 5000 > > I made screenshot of the behaviour: http://i.imgur.com/xtlVNaK.png > at ~10:24:30 the full gc happens, surivor space fills up and old gen > becomes a sawtooth. > at ~10:34 I reloaded the producers and everything was fine again. > > Cheers, > Flo >
