[
https://issues.apache.org/jira/browse/KAFKA-9740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zihan Li updated KAFKA-9740:
----------------------------
Component/s: KafkaConnect
> Add a "continue" option for Kafka Connect error handling
> --------------------------------------------------------
>
> Key: KAFKA-9740
> URL: https://issues.apache.org/jira/browse/KAFKA-9740
> Project: Kafka
> Issue Type: Improvement
> Components: KafkaConnect
> Reporter: Zihan Li
> Priority: Major
>
> Currently there are two error handling options in Kafka Connect, "none" and
> "all". Option "none" will config the connector to fail fast, and option "all"
> will ignore broken records.
> If users want to store their broken records, they have to config a broken
> record queue, which is too much work for them in some cases.
> Some sink connectors have the ability to deal with broken records, for
> example, a JDBC sink connector can store the broken raw bytes into a separate
> table, a S3 connector can store that in a zipped file.
> Therefore, it would be idea if Kafka Connect provides an additional option
> that sends the broken raw bytes to SinkTask directly.
> SinkTask is then responsible for handling the unparsed bytes input.
> The benefits of having this additional option are:
> * Being user friendly. Connectors can handle broken record and hide that
> from clients.
> * Providing more flexibility to SinkTask in terms of broken record handling.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
