Re: Kafka 0.8 cluster setup?

2013-04-23 Thread Jun Rao
Updated the 0.8 quick start wiki with a link to set up a ZK cluster. Thanks, Jun On Tue, Apr 23, 2013 at 7:42 AM, Chris Curtin wrote: > Beat me to it ;) > > Only caveat is I wouldn't use /tmp for this, since if you're running > tmpwatch the 'myid' file will get removed unexpectedly since it do

Re: Kafka 0.8 cluster setup?

2013-04-23 Thread Jason Huang
Good point. thanks! Jason On Tue, Apr 23, 2013 at 10:42 AM, Chris Curtin wrote: > Beat me to it ;) > > Only caveat is I wouldn't use /tmp for this, since if you're running > tmpwatch the 'myid' file will get removed unexpectedly since it doesn't > seem to be changed at it. That was fun to find

Re: Kafka 0.8 cluster setup?

2013-04-23 Thread Chris Curtin
Beat me to it ;) Only caveat is I wouldn't use /tmp for this, since if you're running tmpwatch the 'myid' file will get removed unexpectedly since it doesn't seem to be changed at it. That was fun to find :) We use /var/zookeeper for our storage. Thanks, Chris On Tue, Apr 23, 2013 at 10:30 AM

Re: Kafka 0.8 cluster setup?

2013-04-23 Thread Jason Huang
Thanks Eric - this helps quite a bit. I will play around with it. Jason On Tue, Apr 23, 2013 at 10:21 AM, Eric Sites wrote: > Jason, > > You need to modify the ZooKeeper config and add the following: > > dataDir=/tmp/zookeeper > > initLimit=50 > syncLimit=2 > > server.1=kafka001.domain.com:2888

Re: Kafka 0.8 cluster setup?

2013-04-23 Thread Eric Sites
Jason, You need to modify the ZooKeeper config and add the following: dataDir=/tmp/zookeeper initLimit=50 syncLimit=2 server.1=kafka001.domain.com:2888:3888 server.2=kafka002.domain.com:2888:3888 server.3=kafka003.domain.com:2888:3888 # Make sure you open those 2 points on each of the serve

Re: Kafka 0.8 cluster setup?

2013-04-23 Thread Jason Huang
Thanks Chris and Neha. Chris - I've been through the link you mentioned before. However, that appears to be using one instance of zookeeper, which makes whichever server that runs zookeeper as the single point of failure? Jason On Tue, Apr 23, 2013 at 8:28 AM, Chris Curtin wrote: > I following

Re: Kafka 0.8 cluster setup?

2013-04-23 Thread Chris Curtin
I following these instructions to get the first 'play' cluster going: https://cwiki.apache.org/KAFKA/kafka-08-quick-start.html Instead of running the 3 brokers on the same machine, I ran on on each machine. Note that you will need to do a little bit of zookeeper setup to get a cluster running, I

Re: Kafka 0.8 cluster setup?

2013-04-22 Thread Neha Narkhede
Hi Jason, We are in the process of updating the documentation. Hoping to finish it by this week. Stay tuned. Thanks, Neha On Mon, Apr 22, 2013 at 2:12 PM, Jason Huang wrote: > Hello, > > We've been playing around with kafka 0.8 for a few months now and decided > to install kafka on a small clus

Kafka 0.8 cluster setup?

2013-04-22 Thread Jason Huang
Hello, We've been playing around with kafka 0.8 for a few months now and decided to install kafka on a small cluster for further testing. I tried to search online but couldn't find any setup documentation for a kafka 0.8 cluster. Does anyone know if such documents exist? If they don't exist, what