[ https://issues.apache.org/jira/browse/KAFKA-12159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Victoria Bialas updated KAFKA-12159: ------------------------------------ Description: *BLUF:* The kafka-console-producer should return its prompt after outputting an error message (if it is still running, which in most cases, it is). Current behaviour is it doesn't return a prompt, and hitting return at that point shuts it down. *DETAIL AND EXAMPLE:* The console producer utility behaves in a less than optimal way when you get an error. It doesn’t return the producer prompt even though the producer is still running and accessible. If you hit return after the error, it shuts down the producer, forcing you to restart it to continue sending messages. This can lead you to believe, for example, that you have to shut down the producer after making a config change that takes care of an error, when in fact it isn't necessary to restart. This makes it confusing to demo to users in Docs how to test producers and consumers in scenarios where an error is generated. (I am adding a tip to to be published soon. It will be at the end of step 7 in [Demo: Enabling Schema Validation on a Topic at the Command Line|http://example.com/]. Here is an example from Confluent Platform. The scenario has you try to send a message with schema validation on (which will fail due to the message format), then disable schema validation and resend the message (or a similarly formatted one), which should then succeed. With Confluent "schema validation" on, try to send a message that doesn't conform to the schema defined for a topic. *Producer* {code:java} Last login: Wed Jan 6 17:51:01 on ttys004 Vickys-MacBook-Pro:~ vicky$ kafka-console-producer --broker-list localhost:9092 --topic test-schemas --property parse.key=true --property key.separator=, >1,my first record >2,my second record >[2021-01-06 18:25:08,722] ERROR Error when sending message to topic >test-schemas with key: 1 bytes, value: 16 bytes with error: >(org.apache.kafka.clients.producer.internals.ErrorLoggingCallback) org.apache.kafka.common.InvalidRecordException: This record has failed the validation on broker and hence will be rejected. org.apache.kafka.common.KafkaException: No key found on line 3: at kafka.tools.ConsoleProducer$LineMessageReader.readMessage(ConsoleProducer.scala:290) at kafka.tools.ConsoleProducer$.main(ConsoleProducer.scala:51) at kafka.tools.ConsoleProducer.main(ConsoleProducer.scala) Vickys-MacBook-Pro:~ vicky$ kafka-console-producer --broker-list localhost:9092 --topic test-schemas --property parse.key=true --property key.separator=, >3,my third record > {code} You can see that you lose the producer prompt after the error and get only a blank line, which leads you to believe you've lost the producer (it's actually still running). If you hit return, the producer shuts down, forcing you to restart the producer to continue (when in fact, this isn't necessary.) *Consumer* The consumer still shows only a previous message that was sent when schema validation was disabled earlier in the demo. {code:java} Vickys-MacBook-Pro:~ vicky$ kafka-console-consumer --bootstrap-server localhost:9092 --from-beginning --topic test-schemas --property print.key=trueVickys-MacBook-Pro:~ vicky$ kafka-console-consumer --bootstrap-server localhost:9092 --from-beginning --topic test-schemas --property print.key=true1 my first record {code} If instead, after disabling schema validation in a different shell, you return to the producer window and copy-paste or type the same or similarly formatted message on the blank line following the error, and then hit return, the message will send, and you will see it in the running consumer. *Producer* {code:java} Vickys-MacBook-Pro:~ vicky$ kafka-console-producer --broker-list localhost:9092 --topic test-schemas --property parse.key=true --property key.separator=, >1,my first record >2,my second record >[2021-01-07 11:35:30,443] ERROR Error when sending message to topic >test-schemas with key: 1 bytes, value: 16 bytes with error: >(org.apache.kafka.clients.producer.internals.ErrorLoggingCallback) org.apache.kafka.common.InvalidRecordException: One or more records have been rejected 3,my third record > {code} *Consumer* {code:java} Vickys-MacBook-Pro:~ vicky$ kafka-console-consumer --bootstrap-server localhost:9092 --from-beginning --topic test-schemas --property print.key=trueVickys-MacBook-Pro:~ vicky$ kafka-console-consumer --bootstrap-server localhost:9092 --from-beginning --topic test-schemas --property print.key=true1 my first record3 my third record{code} {color:#ff8b00}_If the prompt was simply shown again after the error, it would solve the usability problem._{color} cc: [~mjsax], [~guozhang], [~gshapira_impala_35cc], [~abhishekd.i...@gmail.com] was: *BLUF:* The kafka-console-producer should return its prompt after outputting an error message (if it is still running, which in most cases, it is). Current behaviour is it doesn't return a prompt, and hitting return at that point shuts it down. *DETAIL AND EXAMPLE:* The console producer utility behaves in a less than optimal way when you get an error. It doesn’t return the producer prompt even though the producer is still running and accessible. If you hit return after the error, it shuts down the producer, forcing you to restart it to continue sending messages. This can lead you to believe, for example, that you have to shut down the producer after making a config change that takes care of an error, when in fact it isn't necessary to restart. This makes it confusing to demo to users in Docs how to test producers and consumers in scenarios where an error is generated. (I am adding a tip o write around this which will be published soon. It will be at the end of step 7 in [Demo: Enabling Schema Validation on a Topic at the Command Line|http://example.com/]. Here is an example from Confluent Platform. The scenario has you try to send a message with schema validation on (which will fail due to the message format), then disable schema validation and resend the message (or a similarly formatted one), which should then succeed. With Confluent "schema validation" on, try to send a message that doesn't conform to the schema defined for a topic. *Producer* {code:java} Last login: Wed Jan 6 17:51:01 on ttys004 Vickys-MacBook-Pro:~ vicky$ kafka-console-producer --broker-list localhost:9092 --topic test-schemas --property parse.key=true --property key.separator=, >1,my first record >2,my second record >[2021-01-06 18:25:08,722] ERROR Error when sending message to topic >test-schemas with key: 1 bytes, value: 16 bytes with error: >(org.apache.kafka.clients.producer.internals.ErrorLoggingCallback) org.apache.kafka.common.InvalidRecordException: This record has failed the validation on broker and hence will be rejected. org.apache.kafka.common.KafkaException: No key found on line 3: at kafka.tools.ConsoleProducer$LineMessageReader.readMessage(ConsoleProducer.scala:290) at kafka.tools.ConsoleProducer$.main(ConsoleProducer.scala:51) at kafka.tools.ConsoleProducer.main(ConsoleProducer.scala) Vickys-MacBook-Pro:~ vicky$ kafka-console-producer --broker-list localhost:9092 --topic test-schemas --property parse.key=true --property key.separator=, >3,my third record > {code} You can see that you lose the producer prompt after the error and get only a blank line, which leads you to believe you've lost the producer (it's actually still running). If you hit return, the producer shuts down, forcing you to restart the producer to continue (when in fact, this isn't necessary.) *Consumer* The consumer still shows only a previous message that was sent when schema validation was disabled earlier in the demo. {code:java} Vickys-MacBook-Pro:~ vicky$ kafka-console-consumer --bootstrap-server localhost:9092 --from-beginning --topic test-schemas --property print.key=trueVickys-MacBook-Pro:~ vicky$ kafka-console-consumer --bootstrap-server localhost:9092 --from-beginning --topic test-schemas --property print.key=true1 my first record {code} If instead, after disabling schema validation in a different shell, you return to the producer window and copy-paste or type the same or similarly formatted message on the blank line following the error, and then hit return, the message will send, and you will see it in the running consumer. *Producer* {code:java} Vickys-MacBook-Pro:~ vicky$ kafka-console-producer --broker-list localhost:9092 --topic test-schemas --property parse.key=true --property key.separator=, >1,my first record >2,my second record >[2021-01-07 11:35:30,443] ERROR Error when sending message to topic >test-schemas with key: 1 bytes, value: 16 bytes with error: >(org.apache.kafka.clients.producer.internals.ErrorLoggingCallback) org.apache.kafka.common.InvalidRecordException: One or more records have been rejected 3,my third record > {code} *Consumer* {code:java} Vickys-MacBook-Pro:~ vicky$ kafka-console-consumer --bootstrap-server localhost:9092 --from-beginning --topic test-schemas --property print.key=trueVickys-MacBook-Pro:~ vicky$ kafka-console-consumer --bootstrap-server localhost:9092 --from-beginning --topic test-schemas --property print.key=true1 my first record3 my third record{code} {color:#ff8b00}_If the prompt was simply shown again after the error, it would solve the usability problem._{color} cc: [~mjsax], [~guozhang], [~gshapira_impala_35cc], [~abhishekd.i...@gmail.com] > kafka-console-producer prompt should redisplay after an error output > -------------------------------------------------------------------- > > Key: KAFKA-12159 > URL: https://issues.apache.org/jira/browse/KAFKA-12159 > Project: Kafka > Issue Type: Bug > Components: producer , tools > Environment: Mac OSX Catalina 10.15.7, iTerm, Linux > Reporter: Victoria Bialas > Priority: Minor > > *BLUF:* The kafka-console-producer should return its prompt after outputting > an error message (if it is still running, which in most cases, it is). > Current behaviour is it doesn't return a prompt, and hitting return at that > point shuts it down. > *DETAIL AND EXAMPLE:* The console producer utility behaves in a less than > optimal way when you get an error. It doesn’t return the producer prompt even > though the producer is still running and accessible. If you hit return after > the error, it shuts down the producer, forcing you to restart it to continue > sending messages. This can lead you to believe, for example, that you have to > shut down the producer after making a config change that takes care of an > error, when in fact it isn't necessary to restart. > This makes it confusing to demo to users in Docs how to test producers and > consumers in scenarios where an error is generated. (I am adding a tip to to > be published soon. It will be at the end of step 7 in [Demo: Enabling Schema > Validation on a Topic at the Command Line|http://example.com/]. > Here is an example from Confluent Platform. The scenario has you try to send > a message with schema validation on (which will fail due to the message > format), then disable schema validation and resend the message (or a > similarly formatted one), which should then succeed. > With Confluent "schema validation" on, try to send a message that doesn't > conform to the schema defined for a topic. > *Producer* > {code:java} > Last login: Wed Jan 6 17:51:01 on ttys004 > Vickys-MacBook-Pro:~ vicky$ kafka-console-producer --broker-list > localhost:9092 --topic test-schemas --property parse.key=true --property > key.separator=, > >1,my first record > >2,my second record > >[2021-01-06 18:25:08,722] ERROR Error when sending message to topic > >test-schemas with key: 1 bytes, value: 16 bytes with error: > >(org.apache.kafka.clients.producer.internals.ErrorLoggingCallback) > org.apache.kafka.common.InvalidRecordException: This record has failed the > validation on broker and hence will be rejected. > org.apache.kafka.common.KafkaException: No key found on line 3: > at > kafka.tools.ConsoleProducer$LineMessageReader.readMessage(ConsoleProducer.scala:290) > at kafka.tools.ConsoleProducer$.main(ConsoleProducer.scala:51) > at kafka.tools.ConsoleProducer.main(ConsoleProducer.scala) > Vickys-MacBook-Pro:~ vicky$ kafka-console-producer --broker-list > localhost:9092 --topic test-schemas --property parse.key=true --property > key.separator=, > >3,my third record > > > {code} > You can see that you lose the producer prompt after the error and get only a > blank line, which leads you to believe you've lost the producer (it's > actually still running). If you hit return, the producer shuts down, forcing > you to restart the producer to continue (when in fact, this isn't necessary.) > *Consumer* > The consumer still shows only a previous message that was sent when schema > validation was disabled earlier in the demo. > {code:java} > Vickys-MacBook-Pro:~ vicky$ kafka-console-consumer --bootstrap-server > localhost:9092 --from-beginning --topic test-schemas --property > print.key=trueVickys-MacBook-Pro:~ vicky$ kafka-console-consumer > --bootstrap-server localhost:9092 --from-beginning --topic test-schemas > --property print.key=true1 > my first record > {code} > If instead, after disabling schema validation in a different shell, you > return to the producer window and copy-paste or type the same or similarly > formatted message on the blank line following the error, and then hit return, > the message will send, and you will see it in the running consumer. > *Producer* > {code:java} > Vickys-MacBook-Pro:~ vicky$ kafka-console-producer --broker-list > localhost:9092 --topic test-schemas --property parse.key=true --property > key.separator=, > >1,my first record > >2,my second record > >[2021-01-07 11:35:30,443] ERROR Error when sending message to topic > >test-schemas with key: 1 bytes, value: 16 bytes with error: > >(org.apache.kafka.clients.producer.internals.ErrorLoggingCallback) > org.apache.kafka.common.InvalidRecordException: One or more records have been > rejected > 3,my third record > > > {code} > *Consumer* > {code:java} > Vickys-MacBook-Pro:~ vicky$ kafka-console-consumer --bootstrap-server > localhost:9092 --from-beginning --topic test-schemas --property > print.key=trueVickys-MacBook-Pro:~ vicky$ kafka-console-consumer > --bootstrap-server localhost:9092 --from-beginning --topic test-schemas > --property print.key=true1 > my first record3 > my third record{code} > {color:#ff8b00}_If the prompt was simply shown again after the error, it > would solve the usability problem._{color} > cc: [~mjsax], [~guozhang], [~gshapira_impala_35cc], > [~abhishekd.i...@gmail.com] > -- This message was sent by Atlassian Jira (v8.3.4#803005)