Re: How to chose the size of a Kafka broker

2016-01-14 Thread Jens Rantil
Hi Vladoiu, I am by no means a Kafka expert, but what are you optimizing for? - Cost could be a variable. - Time to bring on a new broker could be another variable. For large machines that could take longer since they need to stream more data. Cheers, Jens On Wed, Jan 13, 2016 at 1:09

Re: How to chose the size of a Kafka broker

2016-01-13 Thread Stephen Powis
I can't really answer your question, but you don't mention your network layout/hardware. May want to add that as a data point in your decision (wouldn't want to overrun your network device(s) on the brokers). On Wed, Jan 13, 2016 at 7:09 PM, Vladoiu Catalin wrote: > Hi guys, > > I've run into

How to chose the size of a Kafka broker

2016-01-13 Thread Vladoiu Catalin
Hi guys, I've run into a long conversation with my colleagues when we discussed the size of the Brokers for our new Kafka cluster and we still haven't reached a final conclusion. Our main concern is the size of the requests 10-20MB per request (producer will send big requests), maybe more and we