Hi, I am kinda newbie to kafka, I plan to build a cluster with multiple nodes, and multiple brokers on each node, I can find tutorials for set multiple brokers cluster in single node, say http://www.michael-noll.com/blog/2013/03/13/running-a-multi-broker-apache-kafka-cluster-on-a-single-node/ Also I can find some instructions for multiple node setup, but with single broker on each node. I have not seen any documents to teach me how to setup multiple nodes cluster and multiple brokers in each node. I notice some documents points out: we should install kafka on each node which makes sense, and all the brokers in each node should connect to same zookeeper. I am confused since I thought I could setup a zookeeper ensemble cluster separately, and all the brokers connecting to this zookeeper cluster and this zk cluster doesn’t have to be the server hosting the kafka, but some tutorial says I should install zookeeper on each kafka node.
Here is my plan: - I have three nodes: kfServer1, kfserver2, kfserver3, - kfserver1 and kfserver2 are configured as the zookeeper ensemble, which i have done. zk.connect=kfserver1:2181,kfserver2:2181 - broker1, broker2, broker3 are in kfserver1, broker4, broker5, broker6 are on kfserver2, broker7, broker8, broker9 are on kfserver3. When I am configuring, the zk DataDir is in local directory of each node, instead located at the zk ensemble directory, is that correct? So far, I couldnot make above scheme working, anyone have ever made multi-node and multi-broker kafka cluster setup? thanks Alec
