[ https://issues.apache.org/jira/browse/KAFKA-6490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354957#comment-16354957 ]
Prasanna Subburaj commented on KAFKA-6490: ------------------------------------------ [~ewencp]: Thanks for feedback. What you are mentioning makes sense we should users option to chose from because each use case is different. I feel that Discard and log option can be provided to the user and skeptical about the dead letter queue. Also which version should this bug be worked upon ? Can I get please get access to the confluent ([https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals)] as well ? > JSON SerializationException Stops Connect > ----------------------------------------- > > Key: KAFKA-6490 > URL: https://issues.apache.org/jira/browse/KAFKA-6490 > Project: Kafka > Issue Type: Bug > Components: KafkaConnect > Affects Versions: 1.0.0 > Reporter: William R. Speirs > Priority: Major > Attachments: KAFKA-6490_v1.patch > > > If you configure KafkaConnect to parse JSON messages, and you send it a > non-JSON message, the SerializationException message will bubble up to the > top, and stop KafkaConnect. While I understand sending non-JSON to a JSON > serializer is a bad idea, I think that a single malformed message stopping > all of KafkaConnect is even worse. > The data exception is thrown here: > [https://github.com/apache/kafka/blob/trunk/connect/json/src/main/java/org/apache/kafka/connect/json/JsonConverter.java#L305] > > From the call here: > [https://github.com/apache/kafka/blob/trunk/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSinkTask.java#L476] > This bubbles all the way up to the top, and KafkaConnect simply stops with > the message: {{ERROR WorkerSinkTask\{id=elasticsearch-sink-0} Task threw an > uncaught and unrecoverable exception > (org.apache.kafka.connect.runtime.WorkerTask:172)}} > Thoughts on adding a {{try/catch}} around the {{for}} loop in > WorkerSinkTask's {{convertMessages}} so messages that don't properly parse > are logged, but simply ignored? This way KafkaConnect can keep working even > when it encounters a message it cannot decode? -- This message was sent by Atlassian JIRA (v7.6.3#76005)