[
https://issues.apache.org/jira/browse/KAFKA-7499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17287091#comment-17287091
]
Aniket Singh commented on KAFKA-7499:
-------------------------------------
Hi [~jbfletch] [~mjsax],
Is this improvement still in progress? I am facing similar issue & have seen
many related question on Stackoverflow as well, where askers need this feature
very badly.
For me the issue is, I am working on a complex xml to avro (schema xsd -> avsc)
conversion and backward compatibility has not been taken care in the pervious
versions of xsd schema (7 years old). So, if an old message arrives which
doesn't contain mandatory fields, it throws SerializationException and Streams
fail.
I do not have control on the incoming source & need to skip similar records or
send them to DLQ also keeping Streams Up & running for processing next set of
records. Currently, ProductionExceptionHandler or
DeserializationExceptionHandler don't cover SerializationException.
Could you please implement this feature soon to help the community?
> Extend ProductionExceptionHandler to cover serialization exceptions
> -------------------------------------------------------------------
>
> Key: KAFKA-7499
> URL: https://issues.apache.org/jira/browse/KAFKA-7499
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Reporter: Matthias J. Sax
> Assignee: jbfletch
> Priority: Major
> Labels: beginner, kip, newbie
>
> In
> [KIP-210|https://cwiki.apache.org/confluence/display/KAFKA/KIP-210+-+Provide+for+custom+error+handling++when+Kafka+Streams+fails+to+produce],
> an exception handler for the write path was introduced. This exception
> handler covers exception that are raised in the producer callback.
> However, serialization happens before the data is handed to the producer with
> Kafka Streams itself and the producer uses `byte[]/byte[]` key-value-pair
> types.
> Thus, we might want to extend the ProductionExceptionHandler to cover
> serialization exception, too, to skip over corrupted output messages. An
> example could be a "String" message that contains invalid JSON and should be
> serialized as JSON.
> KIP-399 (not voted yet; feel free to pick it up):
> [https://cwiki.apache.org/confluence/display/KAFKA/KIP-399%3A+Extend+ProductionExceptionHandler+to+cover+serialization+exceptions]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
