Can you describe your topic configuration using the below command ? *sh kafka-topics.sh --zookeeper localhost:2181 --topic <topic-name> --describe*
Key for a record is mandatory only for compacted topics. --Kamal On Wed, May 4, 2016 at 2:25 PM, I PVP <i...@hotmail.com> wrote: > HI all, > > What makes a message key mandatory and how to turn it off ? > > I am migrating the messaging piece of a java application from activemq to > kafka. > The application was publishing messages to kafka(0.9.0) with no issues > while running on single broker on my dev machine. > After turning it into multi-broker, by enabling 2 additional brokers, now > I am seeing error messages on all producers saying that "Error when sending > message to topic <topic name> with key: null". The error happens on > producing messages from Java client and for kafka-console-producer.sh too. > > As a newbie I am not doing anything fancy around partitions, key and etc. > > The topics were being create with the following: > /opt/kafka/bin/kafka-topics.sh --zookeeper localhost:2181 --topic <topic > name> --create > > After going multi-broker I was forced to define partitions because the > command above starting saying "Missing required argument "[partitions]” , > so the all topics are being create with the following: > > /opt/kafka/bin/kafka-topics.sh --zookeeper localhost:2181 --topic <topic > name> --create --replication-factor 2 -partitions 1 > > > Messages were being sent by : > ... > ProducerRecord<String, Object> producerRecord = new ProducerRecord<String, > Object>(topicName, message); > producer.send(producerRecord); > ... > > To make things working again after the multi-broker change and the sunden > mandatory message Key requirement I change the code to use a ramdonUUID as > the message key. > ... > ProducerRecord<String, Object> producerRecord = new ProducerRecord<String, > Object>(topicName,UUID.randomUUID().toString(), message); > > producer.send(producerRecord); > ... > > After the change to multi-broker all topics were deleted, all log files > were deleted , zookeeper entries were cleaned and topics were recreated. > > Even now working fine from the java client with the UUID as the message > key it does not work from the command line, it looks like from the > command line there is no way to set the message key and the command line is > critical when needed to do quick tests. > > If none logic at the applications consuming the messages requires a key > why is it forcing to set a key ? The applications consuming the messages > only need to consume the message as they come to the topic. > > Is there a way to turn off the mandatory message key on a multi-broker > situation? > > Thanks > > -- > IPVP > >
