HI all, What makes a message key mandatory and how to turn it off ?
I am migrating the messaging piece of a java application from activemq to kafka. The application was publishing messages to kafka(0.9.0) with no issues while running on single broker on my dev machine. After turning it into multi-broker, by enabling 2 additional brokers, now I am seeing error messages on all producers saying that "Error when sending message to topic <topic name> with key: null". The error happens on producing messages from Java client and for kafka-console-producer.sh too. As a newbie I am not doing anything fancy around partitions, key and etc. The topics were being create with the following: /opt/kafka/bin/kafka-topics.sh --zookeeper localhost:2181 --topic <topic name> --create After going multi-broker I was forced to define partitions because the command above starting saying "Missing required argument "[partitions]” , so the all topics are being create with the following: /opt/kafka/bin/kafka-topics.sh --zookeeper localhost:2181 --topic <topic name> --create --replication-factor 2 -partitions 1 Messages were being sent by : ... ProducerRecord<String, Object> producerRecord = new ProducerRecord<String, Object>(topicName, message); producer.send(producerRecord); ... To make things working again after the multi-broker change and the sunden mandatory message Key requirement I change the code to use a ramdonUUID as the message key. ... ProducerRecord<String, Object> producerRecord = new ProducerRecord<String, Object>(topicName,UUID.randomUUID().toString(), message); producer.send(producerRecord); ... After the change to multi-broker all topics were deleted, all log files were deleted , zookeeper entries were cleaned and topics were recreated. Even now working fine from the java client with the UUID as the message key it does not work from the command line, it looks like from the command line there is no way to set the message key and the command line is critical when needed to do quick tests. If none logic at the applications consuming the messages requires a key why is it forcing to set a key ? The applications consuming the messages only need to consume the message as they come to the topic. Is there a way to turn off the mandatory message key on a multi-broker situation? Thanks -- IPVP
