Re: Zookeeper setup

2013-08-10 Thread Chris Hogue
Another note I'd suggest for the zookeeper name-spacing doc is that the path must exist in zookeeper before starting the brokers. There is an error message, but it's not always clear what the problem is (I think there is even a Jira issue to make the error more clear). -Chris On Sat, Aug 10, 201

Re: Zookeeper setup

2013-08-10 Thread Jay Kreps
Hey guys, I'm actively working on the docs. Any tips like this are much appreciated especially if you can be specific (as you have) about what is clear and what is unclear. I will attempt to clarify the zookeeper chroot info in the changes I'm making this weekend. Whenever possible if we can c

Re: Zookeeper setup

2013-08-10 Thread Jason Rosenberg
Philip, Agreed vis-a-vis the docs. But I also understand how you can sometimes find yourself with a seemingly basic question that doesn't present itself with an obvious reading list. You just want to get going with the basics, in the interest of learning by doing, and it's usually a huge time sa

Re: Zookeeper setup

2013-08-10 Thread Philip O'Toole
Jason - thanks. I am sure that extra information will be appreciated. Of course, when the list gets a message like this one, it helps everyone if it includes details about what has been already done. E.g. "I have read the docs but it is still not clear to me how X works when...". Perhaps Vadim

Re: Zookeeper setup

2013-08-10 Thread Jason Rosenberg
Yeah, the basics are there, e.g. in the doc for the zookeeper.connect property: "Specifies the zookeeper connection string in the form hostname:port/chroot. Here the chroot is a base directory which is prepended to all path operations (this effectively namespaces all kafka znodes to allow sharing

Re: Zookeeper setup

2013-08-09 Thread Philip O'Toole
Have you read the docs? They are well written. It's all there, including the paths. Philip On Aug 9, 2013, at 3:24 PM, Vadim Keylis wrote: > I am trying to setup kafka service and connect to zookeeper that would be > shared with Other projects. Can someone advice how to configure namespace

Zookeeper setup

2013-08-09 Thread Vadim Keylis
I am trying to setup kafka service and connect to zookeeper that would be shared with Other projects. Can someone advice how to configure namespace in kafka and zookeeper. Thanks so much Sent from my iPad

Re: Clustered Kafka/Zookeeper setup not rebalancing all partitions

2013-04-23 Thread Jun Rao
Could you run the tool kafka.tools.ConsumerOffsetChecker? It will show you which consumer instance owns which partition. Thanks, Jun On Tue, Apr 23, 2013 at 12:12 AM, Karl Kirch wrote: > Our setup is a three node cluster, we have a Kafka 0.7.2 (with 10 > partitions per node) and Zookeeper 3.3

Clustered Kafka/Zookeeper setup not rebalancing all partitions

2013-04-23 Thread Karl Kirch
Our setup is a three node cluster, we have a Kafka 0.7.2 (with 10 partitions per node) and Zookeeper 3.3.4 on each node. When we first start up the cluster, there are no issues and things work just fine, but after a while things start acting a bit haywire. It looks like we start to miss messages