Hi Lawrence, What JVM options were you using? There's a few pages in the confluent docs on JVM tuning iirc. We simply use the G1 and a 4GB Max heap and things work well (running many thousands of clusters).
Thanks Tom Crayford Heroku Kafka On Thursday, 9 June 2016, Lawrence Weikum <lwei...@pandora.com> wrote: > Hello all, > > We’ve been running a benchmark test on a Kafka cluster of ours running > 0.9.0.1 – slamming it with messages to see when/if things might break. > During our test, we caused two brokers to throw OutOfMemory errors (looks > like from the Heap) even though each machine still has 43% of the total > memory unused. > > I’m curious what JVM optimizations are recommended for Kafka brokers? Or > if there aren’t any that are recommended, what are some optimizations > others are using to keep the brokers running smoothly? > > Best, > > Lawrence Weikum > >
