Thanks Mike,

Most of the JIRAs there are internal cleanups, but for the user-facing ones 
we're planning on creating a wiki and collecting feedback like yours, and a 
KIP, so stay tuned (your current feedback already noted, thanks).

Eno
> On 24 May 2017, at 15:35, Mike Gould <mikeyg...@gmail.com> wrote:
> 
> Watching it with interest thanks
> 
> Not sure where appropriate to add suggestions but I'd vote for exceptions
> being passed along the stream in something like a hidden Either wrapper.
> Most of the KStream methods would ignore this but new onException() or
> similar methods would be able to examine the error with the key/value prior
> to the error and handle it - possibly by replacing the message, sending a
> message to a new stream, or even putting it back on the original stream for
> retry.
> 
> Regards
> MikeG
> 
> On Wed, 24 May 2017 at 10:09, Eno Thereska <eno.there...@gmail.com 
> <mailto:eno.there...@gmail.com>> wrote:
> 
>> Just a heads up that we're tracking this and other improvements in
>> exception handling at https://issues.apache.org/jira/browse/KAFKA-5156 
>> <https://issues.apache.org/jira/browse/KAFKA-5156> <
>> https://issues.apache.org/jira/browse/KAFKA-5156 
>> <https://issues.apache.org/jira/browse/KAFKA-5156>>.
>> 
>> Thanks
>> Eno
>>> On 23 May 2017, at 17:31, Mike Gould <mikeyg...@gmail.com> wrote:
>>> 
>>> That's great for the value but not the key
>>> 
>>> On Thu, 13 Apr 2017 at 18:27, Sachin Mittal <sjmit...@gmail.com> wrote:
>>> 
>>>> We are also catching the exception in serde and returning null and then
>>>> filtering out null values downstream so as they are not included.
>>>> 
>>>> Thanks
>>>> Sachin
>>>> 
>>>> 
>>>> On Thu, Apr 13, 2017 at 9:13 PM, Mike Gould <mikeyg...@gmail.com>
>> wrote:
>>>> 
>>>>> Great to know I've not gone off in the wrong direction
>>>>> Thanks
>>>>> 
>>>>> On Thu, 13 Apr 2017 at 16:34, Matthias J. Sax <matth...@confluent.io>
>>>>> wrote:
>>>>> 
>>>>>> Mike,
>>>>>> 
>>>>>> thanks for your feedback. You are absolutely right that Streams API
>>>> does
>>>>>> not have great support for this atm. And it's very valuable that you
>>>>>> report this (you are not the first person). It helps us prioritizing
>> :)
>>>>>> 
>>>>>> For now, there is no better solution as the one you described in your
>>>>>> email, but its on our roadmap to improve the API -- and its priority
>>>> got
>>>>>> just increase by your request.
>>>>>> 
>>>>>> I am sorry, that I can't give you a better answer right now :(
>>>>>> 
>>>>>> 
>>>>>> -Matthias
>>>>>> 
>>>>>> 
>>>>>> On 4/13/17 8:16 AM, Mike Gould wrote:
>>>>>>> Hi
>>>>>>> Are there any better error handling options for Kafka streams in
>>>> java.
>>>>>>> 
>>>>>>> Any errors in the serdes will break the stream.  The suggested
>>>>>>> implementation is to use the byte[] serde and do the deserialisation
>>>>> in a
>>>>>>> map operation.  However this isn't ideal either as there's no great
>>>> way
>>>>>> to
>>>>>>> handle exceptions.
>>>>>>> My current tactics are to use flatMap in place of map everywhere and
>>>>>> return
>>>>>>> empySet on error. Unfortunately this means the error has to be
>>>> handled
>>>>>>> directly in the function where it happened and can only be handled
>>>> as a
>>>>>>> side effect.
>>>>>>> 
>>>>>>> It seems to me that this could be done better. Maybe the *Mapper
>>>>>> interfaces
>>>>>>> could allow specific checked exceptions. These could be handled by
>>>>>> specific
>>>>>>> downstream KStream.mapException() steps which might e.g. Put an error
>>>>>>> response on another stream branch.
>>>>>>> Alternatively could it be made easier to return something like an
>>>>> Either
>>>>>>> from the Mappers with a the addition of few extra mapError or mapLeft
>>>>>>> mapRight methods on KStream?
>>>>>>> 
>>>>>>> Unless there's a better error handling pattern which I've entirely
>>>>>> missed?
>>>>>>> 
>>>>>>> Thanks
>>>>>>> MIkeG
>>>>>>> 
>>>>>> 
>>>>>> --
>>>>> - MikeG
>>>>> http://en.wikipedia.org/wiki/Common_misconceptions
>>>>> <http://en.wikipedia.org/wiki/Special:Random>
>>>>> 
>>>> 
>>> --
>>> - MikeG
>>> http://en.wikipedia.org/wiki/Common_misconceptions
>>> <http://en.wikipedia.org/wiki/Special:Random>
>> 
>> --
> - MikeG
> http://en.wikipedia.org/wiki/Common_misconceptions 
> <http://en.wikipedia.org/wiki/Common_misconceptions>
> <http://en.wikipedia.org/wiki/Special:Random 
> <http://en.wikipedia.org/wiki/Special:Random>>

Reply via email to