For what it's worth, I used to get similar messages with docker instances on centos.
The way I debugged the problem was by looking at Kafka logs. In that case, it turned out that brokers could not reach zk and this info was in the logs. The logs will list the parameters the broker used at start up and any errors. In my case, the problem was the firewall that blocked access to zk from Kafka. > On Jun 29, 2016, at 6:56 PM, vivek thakre <vivek.tha...@gmail.com> wrote: > > I have Kafka Cluster setup on AWS Public Subnet with all brokers having > elastic IPs > My producers are on private subnet and not able to produce to the kafka on > public subnet. > Both subnets are in same VPC > > I added the private ip/cidr of producer ec2 instance to Public Kafka's > security group. > (I can telnet from private ec2 instance to brokers private ip on 9092 port) > > From the ec2 instance on private subnet, I can list the topics using ZK's > private ip > > [ec2-user@ip-x-x-x-x kafka_2.10-0.9.0.1]$ bin/kafka-topics.sh --zookeeper > <zk_private_ip>:2181 --list > test > > When I try to produce from private ec2 instance using broker's private IP, > I get following error > > [ec2-user@ip-x-x-x-x kafka_2.10-0.9.0.1]$ bin/kafka-console-producer.sh > --broker-list <broker_private_ip>:9092 --topic test > > [2016-06-29 18:47:38,328] ERROR Error when sending message to topic test > with key: null, value: 3 bytes with error: Batch Expired > (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback) > > When I try to produce from private ec2 instance using broker's public IP, I > get following error. > > [2016-06-29 18:53:15,918] ERROR Error when sending message to topic test > with key: null, value: 3 bytes with error: Failed to update metadata after > 60000 ms. (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback) > > Few settings from server.properties > # The id of the broker. This must be set to a unique integer for each > broker. > broker.id=0 > > ############################# Socket Server Settings > ############################# > > listeners=PLAINTEXT://:9092 > > # The port the socket server listens on > #port=9092 > > # Hostname the broker will bind to. If not set, the server will bind to all > interfaces > host.name=<Public IP> > > # Hostname the broker will advertise to producers and consumers. If not > set, it uses the > # value for "host.name" if configured. Otherwise, it will use the value > returned from > # java.net.InetAddress.getCanonicalHostName(). > advertised.host.name=<Public IP> > > Please let me know if I am doing something wrong. > > Thank you > > Vivek
