[
https://issues.apache.org/jira/browse/KAFKA-12159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Victoria Bialas updated KAFKA-12159:
------------------------------------
Description:
*BLUF:* The kafka-console-producer should return its prompt after outputting an
error message (if it is still running, which in most cases, it is). Current
behaviour is it doesn't return a prompt, and hitting return at that point shuts
it down.
*DETAIL AND EXAMPLE:* The console producer utility behaves in a less than
optimal way when you get an error. It doesn’t return the producer prompt even
though the producer is still running and accessible. If you hit return after
the error, it shuts down the producer, forcing you to restart it when in fact
that wasn’t necessary.
This makes it confusing to demo to users in Docs how to test producers and
consumers in scenarios where an error is generated. (I am adding a tip o write
around this which will be published soon. It will be at the end of step 7 in
[Demo: Enabling Schema Validation on a Topic at the Command
Line|http://example.com/].
Here is an example from Confluent Platform. The scenario has you try to send a
message with schema validation on (which will fail due to the message format),
then disable schema validation and resend the message or another in a similar
format, which should then succeed.
With Confluent "schema validation" on, try to send a message that doesn't
conform to the schema defined for a topic.
*Producer*
{code:java}
Last login: Wed Jan 6 17:51:01 on ttys004
Vickys-MacBook-Pro:~ vicky$ kafka-console-producer --broker-list localhost:9092
--topic test-schemas --property parse.key=true --property key.separator=,
>1,my first record
>2,my second record
>[2021-01-06 18:25:08,722] ERROR Error when sending message to topic
>test-schemas with key: 1 bytes, value: 16 bytes with error:
>(org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.InvalidRecordException: This record has failed the
validation on broker and hence will be rejected.
org.apache.kafka.common.KafkaException: No key found on line 3:
at
kafka.tools.ConsoleProducer$LineMessageReader.readMessage(ConsoleProducer.scala:290)
at kafka.tools.ConsoleProducer$.main(ConsoleProducer.scala:51)
at kafka.tools.ConsoleProducer.main(ConsoleProducer.scala)
Vickys-MacBook-Pro:~ vicky$ kafka-console-producer --broker-list localhost:9092
--topic test-schemas --property parse.key=true --property key.separator=,
>3,my third record
>
{code}
You can see that you lose the producer prompt after the error and get only a
blank line, which leads you to believe you've lost the producer (it's actually
still running). If you hit return, the producer shuts down, forcing you to
restart the producer to continue (when in fact, this isn't necessary.)
*Consumer*
The consumer still shows only a previous message that was sent when schema
validation was disabled earlier in the demo.
{code:java}
Vickys-MacBook-Pro:~ vicky$ kafka-console-consumer --bootstrap-server
localhost:9092 --from-beginning --topic test-schemas --property
print.key=trueVickys-MacBook-Pro:~ vicky$ kafka-console-consumer
--bootstrap-server localhost:9092 --from-beginning --topic test-schemas
--property print.key=true1
my first record
{code}
If instead, after disabling schema validation in a different shell, you return
to the producer window and copy-paste or type the message on the blank line
following the error, and then hit return, the message will send, and you will
see it in the running consumer.
*Producer*
{code:java}
Vickys-MacBook-Pro:~ vicky$ kafka-console-producer --broker-list localhost:9092
--topic test-schemas --property parse.key=true --property key.separator=,
>1,my first record
>2,my second record
>[2021-01-07 11:35:30,443] ERROR Error when sending message to topic
>test-schemas with key: 1 bytes, value: 16 bytes with error:
>(org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.InvalidRecordException: One or more records have been
rejected
3,my third record
>
{code}
*Consumer*
{code:java}
Vickys-MacBook-Pro:~ vicky$ kafka-console-consumer --bootstrap-server
localhost:9092 --from-beginning --topic test-schemas --property
print.key=trueVickys-MacBook-Pro:~ vicky$ kafka-console-consumer
--bootstrap-server localhost:9092 --from-beginning --topic test-schemas
--property print.key=true1
my first record3
my third record{code}
If the prompt was simply shown again after the error, it would solve the
usability problem.
cc: [~mjsax], [~guozhang], [~gshapira_impala_35cc], [~abhishekd.i...@gmail.com]
was:
*BLUF:* The kafka-console-producer should return its prompt after outputting an
error message (if it is still running, which in most cases, it is). Current
behaviour is it doesn't return a prompt, and hitting return at that point shuts
it down.
*DETAIL AND EXAMPLE:* The console producer utility behaves in a less than
optimal way when you get an error. It doesn’t return the producer prompt even
though the producer is still running and accessible. If you hit return after
the error, it shuts down the producer, forcing you to restart it when in fact
that wasn’t necessary.
This makes it confusing to demo to users in Docs how to test producers and
consumers in scenarios where an error is generated. (I am adding a tip o write
around this which will be published soon. It will be at the end of step 7 in
[Demo: Enabling Schema Validation on a Topic at the Command
Line|http://example.com].
Here is an example from Confluent Platform. The scenario has you try to send a
message with schema validation on (which will fail due to the message format),
then disable schema validation and resend the message or another in a similar
format, which should then succeed.
# With Confluent "schema validation" on, try to send a message that doesn't
conform to the schema defined for a topic.
*Producer*
{code:java}
Last login: Wed Jan 6 17:51:01 on ttys004
Vickys-MacBook-Pro:~ vicky$ kafka-console-producer --broker-list localhost:9092
--topic test-schemas --property parse.key=true --property key.separator=,
>1,my first record
>2,my second record
>[2021-01-06 18:25:08,722] ERROR Error when sending message to topic
>test-schemas with key: 1 bytes, value: 16 bytes with error:
>(org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.InvalidRecordException: This record has failed the
validation on broker and hence will be rejected.
org.apache.kafka.common.KafkaException: No key found on line 3:
at
kafka.tools.ConsoleProducer$LineMessageReader.readMessage(ConsoleProducer.scala:290)
at kafka.tools.ConsoleProducer$.main(ConsoleProducer.scala:51)
at kafka.tools.ConsoleProducer.main(ConsoleProducer.scala)
Vickys-MacBook-Pro:~ vicky$ kafka-console-producer --broker-list localhost:9092
--topic test-schemas --property parse.key=true --property key.separator=,
>3,my third record
>
{code}
You can see that you lose the producer prompt after the error and get only a
blank line, which leads you to believe you've lost the producer (it's actually
still running). If you hit return, the producer shuts down, forcing you to
restart the producer to continue (when in fact, this isn't necessary.)
*Consumer*
The consumer still shows only a previous message that was sent when schema
validation was disabled earlier in the demo.
{code:java}
Vickys-MacBook-Pro:~ vicky$ kafka-console-consumer --bootstrap-server
localhost:9092 --from-beginning --topic test-schemas --property
print.key=trueVickys-MacBook-Pro:~ vicky$ kafka-console-consumer
--bootstrap-server localhost:9092 --from-beginning --topic test-schemas
--property print.key=true1
my first record
{code}
# If instead, after disabling schema validation in a different shell, you
return to the producer window and copy-paste or type the message on the blank
line following the error, and then hit return, the message will send, and you
will see it in the running consumer.
*Producer*
{code:java}
Vickys-MacBook-Pro:~ vicky$ kafka-console-producer --broker-list localhost:9092
--topic test-schemas --property parse.key=true --property key.separator=,
>1,my first record
>2,my second record
>[2021-01-07 11:35:30,443] ERROR Error when sending message to topic
>test-schemas with key: 1 bytes, value: 16 bytes with error:
>(org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.InvalidRecordException: One or more records have been
rejected
3,my third record
>
{code}
*Consumer*
{code:java}
Vickys-MacBook-Pro:~ vicky$ kafka-console-consumer --bootstrap-server
localhost:9092 --from-beginning --topic test-schemas --property
print.key=trueVickys-MacBook-Pro:~ vicky$ kafka-console-consumer
--bootstrap-server localhost:9092 --from-beginning --topic test-schemas
--property print.key=true1
my first record3
my third record{code}
If the prompt was simply shown again after the error, it would solve the
usability problem.
cc: [~mjsax], [~guozhang], [~gshapira_impala_35cc], [~abhishekd.i...@gmail.com]
> kafka-console-producer prompt should redisplay after an error output
> --------------------------------------------------------------------
>
> Key: KAFKA-12159
> URL: https://issues.apache.org/jira/browse/KAFKA-12159
> Project: Kafka
> Issue Type: Bug
> Components: producer , tools
> Environment: Mac OSX Catalina 10.15.7, iTerm, Linux
> Reporter: Victoria Bialas
> Priority: Minor
>
> *BLUF:* The kafka-console-producer should return its prompt after outputting
> an error message (if it is still running, which in most cases, it is).
> Current behaviour is it doesn't return a prompt, and hitting return at that
> point shuts it down.
> *DETAIL AND EXAMPLE:* The console producer utility behaves in a less than
> optimal way when you get an error. It doesn’t return the producer prompt even
> though the producer is still running and accessible. If you hit return after
> the error, it shuts down the producer, forcing you to restart it when in fact
> that wasn’t necessary.
> This makes it confusing to demo to users in Docs how to test producers and
> consumers in scenarios where an error is generated. (I am adding a tip o
> write around this which will be published soon. It will be at the end of step
> 7 in [Demo: Enabling Schema Validation on a Topic at the Command
> Line|http://example.com/].
> Here is an example from Confluent Platform. The scenario has you try to send
> a message with schema validation on (which will fail due to the message
> format), then disable schema validation and resend the message or another in
> a similar format, which should then succeed.
> With Confluent "schema validation" on, try to send a message that doesn't
> conform to the schema defined for a topic.
> *Producer*
> {code:java}
> Last login: Wed Jan 6 17:51:01 on ttys004
> Vickys-MacBook-Pro:~ vicky$ kafka-console-producer --broker-list
> localhost:9092 --topic test-schemas --property parse.key=true --property
> key.separator=,
> >1,my first record
> >2,my second record
> >[2021-01-06 18:25:08,722] ERROR Error when sending message to topic
> >test-schemas with key: 1 bytes, value: 16 bytes with error:
> >(org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
> org.apache.kafka.common.InvalidRecordException: This record has failed the
> validation on broker and hence will be rejected.
> org.apache.kafka.common.KafkaException: No key found on line 3:
> at
> kafka.tools.ConsoleProducer$LineMessageReader.readMessage(ConsoleProducer.scala:290)
> at kafka.tools.ConsoleProducer$.main(ConsoleProducer.scala:51)
> at kafka.tools.ConsoleProducer.main(ConsoleProducer.scala)
> Vickys-MacBook-Pro:~ vicky$ kafka-console-producer --broker-list
> localhost:9092 --topic test-schemas --property parse.key=true --property
> key.separator=,
> >3,my third record
> >
> {code}
> You can see that you lose the producer prompt after the error and get only a
> blank line, which leads you to believe you've lost the producer (it's
> actually still running). If you hit return, the producer shuts down, forcing
> you to restart the producer to continue (when in fact, this isn't necessary.)
> *Consumer*
> The consumer still shows only a previous message that was sent when schema
> validation was disabled earlier in the demo.
> {code:java}
> Vickys-MacBook-Pro:~ vicky$ kafka-console-consumer --bootstrap-server
> localhost:9092 --from-beginning --topic test-schemas --property
> print.key=trueVickys-MacBook-Pro:~ vicky$ kafka-console-consumer
> --bootstrap-server localhost:9092 --from-beginning --topic test-schemas
> --property print.key=true1
> my first record
> {code}
> If instead, after disabling schema validation in a different shell, you
> return to the producer window and copy-paste or type the message on the blank
> line following the error, and then hit return, the message will send, and you
> will see it in the running consumer.
> *Producer*
> {code:java}
> Vickys-MacBook-Pro:~ vicky$ kafka-console-producer --broker-list
> localhost:9092 --topic test-schemas --property parse.key=true --property
> key.separator=,
> >1,my first record
> >2,my second record
> >[2021-01-07 11:35:30,443] ERROR Error when sending message to topic
> >test-schemas with key: 1 bytes, value: 16 bytes with error:
> >(org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
> org.apache.kafka.common.InvalidRecordException: One or more records have been
> rejected
> 3,my third record
> >
> {code}
> *Consumer*
> {code:java}
> Vickys-MacBook-Pro:~ vicky$ kafka-console-consumer --bootstrap-server
> localhost:9092 --from-beginning --topic test-schemas --property
> print.key=trueVickys-MacBook-Pro:~ vicky$ kafka-console-consumer
> --bootstrap-server localhost:9092 --from-beginning --topic test-schemas
> --property print.key=true1
> my first record3
> my third record{code}
> If the prompt was simply shown again after the error, it would solve the
> usability problem.
> cc: [~mjsax], [~guozhang], [~gshapira_impala_35cc],
> [~abhishekd.i...@gmail.com]
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
