Hi, I am using a single node Kafka V 0.10.2 (16 GB RAM, 8 cores) and a single node zookeeper V 3.4.9 (4 GB RAM, 1 core ). I am having 64 consumer groups and 500 topics each having 250 partitions. I am able to execute the commands which require only Kafka broker and its running fine ex.
> ./kafka-consumer-groups.sh --bootstrap-server localhost:9092 > --describe --group <topic> But when I execute the admin command like create topic, alter topic For example > ./kafka-topics.sh --create --zookeeper <zookeeper>:2181 > --replication-factor 1 --partitions 1 --topic <topic> Following exception is being displayed: > Error while executing topic command : replication factor: 1 larger > than available brokers: 0 [2017-11-16 11:22:13,592] ERROR > org.apache.kafka.common.errors.InvalidReplicationFactorException: > replication factor: 1 larger than available brokers: 0 > (kafka.admin.TopicCommand$) I checked my broker is up. In server.log following warnings are there [2017-11-16 11:14:26,959] WARN Client session timed out, have not heard from server in 15843ms for sessionid 0x15aa7f586e1c061 (org.apache.zookeeper.ClientCnxn) [2017-11-16 11:14:28,795] WARN Unable to reconnect to ZooKeeper service, session 0x15aa7f586e1c061 has expired (org.apache.zookeeper.ClientCnxn) [2017-11-16 11:21:46,055] WARN Unable to reconnect to ZooKeeper service, session 0x15aa7f586e1c067 has expired (org.apache.zookeeper.ClientCnxn) Below mentioned is my Kafka server configuration : broker.id=1 delete.topic.enable=true num.network.threads=3 num.io.threads=8 socket.send.buffer.bytes=102400 socket.receive.buffer.bytes=102400 socket.request.max.bytes=104857600 log.dirs=/kafka/data/logs num.partitions=1 log.segment.bytes=1073741824 log.retention.check.interval.ms=300000 zookeeper.connect=<zookeeperIP>:2181 zookeeper.connection.timeout.ms=6000 Zookeeper Configuration is : # The number of milliseconds of each tick tickTime=2000 # The number of ticks that the initial # synchronization phase can take initLimit=10 # The number of ticks that can pass between # sending a request and getting an acknowledgement syncLimit=5 # the directory where the snapshot is stored. # do not use /tmp for storage, /tmp here is just # example sakes. dataDir=/zookeeper/data # the port at which the clients will connect clientPort=2181 # the maximum number of client connections. # increase this if you need to handle more clients #maxClientCnxns=60 autopurge.snapRetainCount=20 # Purge task interval in hours # Set to "0" to disable auto purge feature autopurge.purgeInterval=48 I am not able to figure out which configuration to tune. What I am missing .Any help will be appreciated. Regards, Abhimanyu