Here is the recommended way to setup a 3-node Kafka cluster. Its always recommended to keep zookeeper nodes on different set of nodes than the one you are running Kafka. To go with your current 3-node installation. 1. Install 3-node zookeeper make sure they are forming the quorum (https://zookeeper.apache.org/doc/r3.3.2/zookeeperAdmin.html) 2. Install apache kafka binaries on all 3 nodes. 3. Make sure you keep the same zookeeper.connect in server.properties on all 3 nodes for your kafka broker. 4. Start Kafka brokers 5. For sanity check, make sure you create a topic with 3-replication factor and see if you can produce & consume messages
Before stepping into security make sure your non-secure Kafka cluster works ok. Once you’ve a stable & working cluster follow instructions in the doc to enable SSL. -Harsha On Mar 1, 2017, 1:08 PM -0800, IT Consultant <0binarybudd...@gmail.com>, wrote: > Hi Harsha , > > Thanks a lot . > > Let me explain where am i stuck , > > i have three machines on which i am running apache kafka with single broker > but zookeeper of each machine is configured with other machine. > > Example : node1=zk1,zk2,zk3 > node2=zk1,zk2,zk3 > node3=zk1,zk2,zk3 > > This is done for HA . > > Now i need to secure this deployment using SSL . > > *Things tried so far :* > > Create a key and certificate for each of these nodes and configure broker > according to the documentation . > > However , i see following error when i run console producer and consumer > with client certificate or client properties file . > > WARN Error while fetching metadata for topic > > > How do i make each broker work with other broker ? > How do i generate and store certificate for this ? because online document > seems to be confusing for me. > How do i make zookeepers sync with each other and behave as earlier ? > > > > On Thu, Mar 2, 2017 at 2:25 AM, Harsha Chintalapani <ka...@harsha.io> wrote: > > > For inter broker communication over SSL all you need is to add > > security.inter.broker.protocol to SSL. > > "How do i make zookeeper talk to each other and brokers?" > > Not sure I understand the question. You need to make sure zookeeper hosts > > and port are reachable from your broker nodes. > > -Harsha > > > > On Wed, Mar 1, 2017 at 12:45 PM IT Consultant <0binarybudd...@gmail.com > > wrote: > > > > > Hi Team , > > > > > > Can you please help me understand , > > > > > > 1. How can i secure multi-node (3 machine) single broker (1 broker ) > > Apache > > > Kafka deployment secure using SSL ? > > > > > > i tried to follow instructions here but found pretty confusing . > > > > > > https://www.confluent.io/blog/apache-kafka-security-authoriz > > > ation-authentication-encryption/ > > > > > > http://docs.confluent.io/2.0.0/kafka/security.html > > > > > > Currently , i have kafka running on 3 different machines . > > > 2. How do i make them talk to each other over SSL ? > > > 3. How do i make zookeeper talk to each other and brokers? > > > > > > Requesting your help . > > > > > > Thanks in advance. > > > > >
