[
https://issues.apache.org/jira/browse/KAFKA-8597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16873387#comment-16873387
]
Randall Hauch commented on KAFKA-8597:
--------------------------------------
Although I'll have to think about this more, my initial thought is that we
won't want to expose the existing DLQ API (which is currently not public) to
the connectors. We may want to consider having a method on the SinkTaskContext
that connectors can use to record a failed record, though the underlying
guarantees provided by that approach (using a producer) complicate the
guarantees provided through the consumer offset management.
A significant disadvantage of any extension of the existing API is that any
connector that depends on that new API would be constrained in the Kafka
Connect versions into which it can be installed. For example, let's imagine
that we add this API in AK 2.4.0; any connector that uses this API would only
be able to be used in Kafka Connect 2.4.0, and would not work in any earlier
versions.
> Give access to the Dead Letter Queue APIs to Kafka Connect Developers
> ---------------------------------------------------------------------
>
> Key: KAFKA-8597
> URL: https://issues.apache.org/jira/browse/KAFKA-8597
> Project: Kafka
> Issue Type: Improvement
> Components: KafkaConnect
> Reporter: Andrea Santurbano
> Priority: Major
> Fix For: 2.4.0
>
>
> Would be cool to have the chance to have access to the DLQ APIs in order to
> enable us (developers) to use that.
> For instance, if someone uses JSON as message format with no schema and it's
> trying to import some data into a table, and the JSON contains a null value
> for a NON-NULL table field, so we want to move that event to the DLQ.
> Thanks a lot!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
