[ https://issues.apache.org/jira/browse/KAFKA-6738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16445609#comment-16445609 ]
Raul Saez Tapia commented on KAFKA-6738: ---------------------------------------- Hi Konstantine Karantasis, we are in troubles with a connector processing bad messages with AvroConverter. Our trouble is the task in worker is terminated when message is wrong because it is not well formed against a schema. We thought to create an issue for improving Kafka Connect in the same way that proposal from [Ewen Cheslack-Postava|https://plus.google.com/u/0/109815729644713816339?prsrc=4] [Handling of DataException thrown by AvroConverter toConnectData()|[https://groups.google.com/d/msg/confluent-platform/OBuLbVHbuyI/uJIkCliXCAAJ].] After seen this issue we think this is that we need so, do you believe it will resolve our problems? Thanks in advance.- > Kafka Connect handling of bad data > ---------------------------------- > > Key: KAFKA-6738 > URL: https://issues.apache.org/jira/browse/KAFKA-6738 > Project: Kafka > Issue Type: Improvement > Components: KafkaConnect > Affects Versions: 1.1.0 > Reporter: Randall Hauch > Assignee: Konstantine Karantasis > Priority: Critical > > Kafka Connect connectors and tasks fail when they run into an unexpected > situation or error, but the framework should provide more general "bad data > handling" options, including (perhaps among others): > # fail fast, which is what we do today (assuming connector actually fails and > doesn't eat errors) > # retry (possibly with configs to limit) > # drop data and move on > # dead letter queue > This needs to be addressed in a way that handles errors from: > # The connector itself (e.g. connectivity issues to the other system) > # Converters/serializers (bad data, unexpected format, etc) > # SMTs > # Ideally the framework as well, though we obviously want to fix known bugs > anyway -- This message was sent by Atlassian JIRA (v7.6.3#76005)