Re: starting of at a small scale, single ec2 instance with 7.5 GB RAM with kafka

2014-05-20 Thread Niek Sanders
If you really only care about small scale (no HA, no horizontal scaling), you could also consider using Redis instead of Kafka for queueing. - Niek On Tue, May 20, 2014 at 2:23 PM, S Ahmed wrote: > Yes agreed, but I have done some load testing before and kafka was doing > 10's of thousands of m

Re: starting of at a small scale, single ec2 instance with 7.5 GB RAM with kafka

2014-05-20 Thread S Ahmed
Yes agreed, but I have done some load testing before and kafka was doing 10's of thousands of messages per second. If I am doing only hundreds, I think it could handle it for now. Like I said this is small scale. On Tue, May 20, 2014 at 2:51 PM, Neha Narkhede wrote: > It is not recommended to

Re: starting of at a small scale, single ec2 instance with 7.5 GB RAM with kafka

2014-05-20 Thread Neha Narkhede
It is not recommended to install both kafka and zookeeper on the same box as both would fight for the available memory and performance will degrade. Thanks Neha On Mon, May 19, 2014 at 7:29 AM, S Ahmed wrote: > Hi, > > I like how kafka operates, but I'm wondering if it is possible to run > eve

RE: starting of at a small scale, single ec2 instance with 7.5 GB RAM with kafka

2014-05-19 Thread S Ahmed
Hi, I like how kafka operates, but I'm wondering if it is possible to run everything on a single ec2 instance with 7.5 GB RAM. So that would be zookeeper and a single kafka broker. I would have a separate server to consume from the broker. Producers would be from my web servers. I don't want