Barry, No, because Kafka also relies heavily on the OS page cache, which uses memory. You'd roughly want to allocate enough page cache to hold all the messages for your consumers for say, 30s.
Kafka also (in our experience on EC2) tends to run out of network far before it runs out of memory or disk bandwidth, so colocating brokers makes that much more likely. Thanks Tom Crayford, Heroku Kafka On Fri, Jun 10, 2016 at 7:02 AM, Barry Kaplan <bkap...@memelet.com> wrote: > If too much heap cause problems, would it make sense to run multiple > brokers on a box with lots memory? For example, an EC2 D2 instance types > has way way more ram than kafka could ever use - -but it has fast connected > disks. > > Would running a broker per disk make sense in this case? > > -barry >
