Whoops, just noticed there is already a voting thread for this. Hard to
keep track with all the KIPs going on right now!
In that case I'll just wait for the SerializationExceptionOrigin thing to
be added and then I'll vote. Should definitely be able to make 3.8 in this
case :D
On Fri, May 10,
Sounds like we're more or less in agreement here. I think the KIP just
needs one small update still, which is the SerializationExceptionOrigin
enum.
As discussed there are a few options for this and we're all happy to defer
to the preference of the KIP authors, but if we keep the KIP as-is with
Great KIP discussion so far by everyone.
At this point, I'm in agreement with the direction and current state of the
KIP.
As for having two separate callbacks for the ProductionExceptionHandler,
I'm somewhat split in that I agree with points raised by Sophie and
Matthias with my final
position
The type safety issue is definitely not solved by having two separate
callbacks. I just think it gets a bit worse by mashing them into one
method. At least in the plain #handle method you can be sure that the type
is ProducerRecord and in #handleSerialization the type is
some POJO.
And in theory
Thanks Sophie! Makes it much clearer where you are coming from.
About the Type unsafety: isn't this also an issue for the
`handleSerialziationException` case, because the handler is used for all
sink topics, and thus key/value types are not really know w/o taking the
sink topic into account?
Well I definitely don't feel super strongly about it, and more importantly,
I'm not a user. So I will happily defer to the preference of anyone who
will actually be using this feature. But I'll explain my reasoning:
There *is* a relevant distinction between these two callbacks -- because
the
@Loic, yes, what you describe is exactly what I had in mind.
@Sophie, can you elaborate a little bit?
First of all, I agree that it makes sense to maintain the two separate
callbacks for the ProductionExceptionHandler, since one of them is
specifically for serialization exceptions while the
First of all, I agree that it makes sense to maintain the two separate
callbacks for the ProductionExceptionHandler, since one of them is
specifically for serialization exceptions while the other is used for
everything/anything else. I also think we can take advantage of this fact
to simplify
Hi Matthias,
To sum up with the ProductionExceptionHandler callback methods (106) proposed
changes.
A new method ProductionExceptionHandler#handle is added with the following
signature:
> ProductionExceptionHandlerResponse handle(final ErrorHandlerContext context,
> final ProducerRecord
What about (106) to unify both exiting callback methods of
`ProductionExceptionHandler` into a single one, instead of adding two
new ones?
Damien's last reply about it was:
I will think about unifying, I do agree it would be cleaner.
There was not follow up on this question, and the KIP
Hi Bruno,
Good catch, KIP has been updated accordingly.
Regards,
Loïc
US EXTERNAL MESSAGE] Re: [DISCUSS] KIP-1033: Add Kafka Streams
exception handler for exceptions occuring during processing
Warning This might be a fraudulent message! When clicking REPLY, your answers
will NOT go to the sender (andrew_schofi...@live.com). Instead, replies will be
sent to dev@kafka.apach
nks
Sébastien
De : Andrew Schofield
Envoyé : vendredi 3 mai 2024 06:44
À : dev@kafka.apache.org
Objet : [SUSPICIOUS EXTERNAL MESSAGE] Re: [DISCUSS] KIP-1033: Add Kafka Streams
exception handler for exceptions occuring during processing
Warning This mi
rew Schofield
Envoyé : vendredi 3 mai 2024 06:44
À : dev@kafka.apache.org
Objet : [SUSPICIOUS EXTERNAL MESSAGE] Re: [DISCUSS] KIP-1033: Add Kafka Streams
exception handler for exceptions occuring during processing
Warning This might be a fraudulent message! When clicking REPLY, your answers
will NOT
Hi,
I’ve changed my mind on this one having read through the comments.
I don’t think the exception handler should be able to mess with the headers
to the detriment of the code that called the handler.
While I like the hygiene of having an ImmutableHeaders interface,
I feel we can use the
I tend to agree that we should just return a pure Headers instead of
introducing a new class/interface to protect overwriting them. I think a
pretty good case has been made already so I won't add onto it, just wanted
to voice my support.
Is that the only remaining question on this KIP? Might be
Hi all, thanks Damien for the KIP!
After looking into the KIP and comments, my only concern is aligned with
one of Matthias comments, around the ImmutableHeaders introduction, with
the motivation not being clear enough. The existing handlers already expose
the headers (indirectly). Ex.
Thanks for the update.
I am wondering if we should use `ReadOnlyHeaders` instead of
`ImmutableHeaders` as interface name?
Also, the returned `Header` interface is technically not immutable
either, because `Header#key()` returns a mutable byte-array... Would we
need a `ReadOnlyHeader`
Hi Matthias,
I have updated the KIP regarding points 103 and 108.
103.
I have suggested a new `ImmutableHeaders` interface to deal with the
immutability concern of the headers, which is basically the `Headers`
interface without the write accesses.
public interface ImmutableHeaders {
Header
; >>>>>>
> >>>>>> On 4/11/24 12:06 AM, Matthias J. Sax wrote:
> >>>>>>> Thanks for the KIP. Great discussion.
> >>>>>>>
> >>>>>>> I am not sure if I understand the proposal from Bruno
echnical Expert Kafka*
Carmes / Bâtiment A17 4e / 63100 Clermont-Ferrand
*De :* Bruno Cadonna
*Envoyé :* mercredi 10 avril 2024 10:38
*À :* dev@kafka.apache.org
*Objet :* [EXT] Re: [DISCUSS] KIP-1033: Add Kafka Streams
e handler?
> >>>>>> >> Is it the input record to the task? Is it the input record to the
> >>>>>> >> processor node? Is it the input record to the processor?
> >>>>>> >
> >>>>>> > The input
--
*De :* Bruno Cadonna
*Envoyé :* mercredi 10 avril 2024 10:38
*À :* dev@kafka.apache.org
*Objet :* [EXT] Re: [DISCUSS] KIP-1033: Add Kafka Streams exception
handler for exceptions occuring during processing
Warning External sender Do not click on any links or open any
attachments unless
ok with moving forward with
> > > > this KIP as-is, but we should be aware of potential limitations for DSL
> > > > users. We can always do a follow up KIP to close gaps when we
> > understand
> > > > the impact better -- covering the DSL would also ex
; >> > ProcessingExceptionHandler.ProcessingHandlerResponse response =
> > >> this.processingExceptionHandler
> > >> > .handle(internalProcessorContext, (Record) record,
> e);
> > >> >
> > >> > if (response ==
> > >&
t;>
> >> You are right, it will simplify types.
> >>
> >> We update the KIP
> >>
> >> regards
> >>
> >> Sébastien *VIALE***
> >>
> >> *MICHELIN GROUP* - InfORMATION Technology
> >> *Technical Expert Kafka*
&
100 Clermont-Ferrand
*De :* Bruno Cadonna
*Envoyé :* mercredi 10 avril 2024 10:38
*À :* dev@kafka.apache.org
*Objet :* [EXT] Re: [DISCUSS] KIP-1033: Add Kafka Streams exception
handler for exceptions occuring during processing
Warning External sender Do not click on a
100 Clermont-Ferrand
*De :* Bruno Cadonna
*Envoyé :* mercredi 10 avril 2024 10:38
*À :* dev@kafka.apache.org
*Objet :* [EXT] Re: [DISCUSS] KIP-1033: Add Kafka Streams exception
handler for exceptions occuring during processing
War
Cadonna
Envoyé : mercredi 10 avril 2024 10:38
À : dev@kafka.apache.org
Objet : [EXT] Re: [DISCUSS] KIP-1033: Add Kafka Streams exception handler for
exceptions occuring during processing
Warning External sender Do not click on any links or open any attachments
unless you trust the sender and know
Blee-Goldman mailto:sop...@responsive.dev>>
Envoyé : samedi 6 avril 2024 01:08
À : dev@kafka.apache.org<mailto:dev@kafka.apache.org>
mailto:dev@kafka.apache.org>>
Objet : [EXT] Re: [DISCUSS] KIP-1033: Add Kafka Streams exception
handler for exceptions occuring during processing
War
the record be forwarded with
> > >> the ProcessorContext.
> > >>
> > >> The ProcessingContext is sufficient, we do expect that most people
> > >> would need to access the RecordMetadata.
> > >>
> > >>
> > >>
> > &
orwarded with
> > >> the ProcessorContext.
> > >>
> > >> The ProcessingContext is sufficient, we do expect that most people
> > >> would need to access the RecordMetadata.
> > >>
> > >>
> > >>
> > >> 3/ T
;> information about the faulty record.
> >>
> >> Assuming that users want to make some specific treatments to the
> >> record, they can add a try / catch block in the topology.
> >>
> >> It is up to users to cast record value and key
@kafka.apache.org
Objet : [EXT] Re: [DISCUSS] KIP-1033: Add Kafka Streams exception
handler for exceptions occuring during processing
Warning External sender Do not click on any links or open any
attachments unless you trust the sender and know the content is safe.
Hi Damien,
First off thanks
and key in the implementation of the
ProcessorExceptionHandler.
Cheers
Loïc, Damien and Sébastien
De : Sophie Blee-Goldman
Envoyé : samedi 6 avril 2024 01:08
À : dev@kafka.apache.org
Objet : [EXT] Re: [DISCUSS] KIP-1033: Add Kafka Streams exception handler
@Override
public void configure(Map configs) {
}
}
The example will be added to the KIP.
Regards
Sébastien, Damien and Loïc
De : Sebastien Viale
Envoyé : lundi 8 avril 2024 15:49
À : dev@kafka.apache.org
Objet : RE: [EXT] Re: [DISCUSS] KIP-1033: Add Kafka Streams exception handler
for
De : Sophie Blee-Goldman
Envoyé : samedi 6 avril 2024 01:08
À : dev@kafka.apache.org
Objet : [EXT] Re: [DISCUSS] KIP-1033: Add Kafka Streams exception handler for
exceptions occuring during processing
Warning External sender Do not click on any links or open any attachments
Hi Damien,
First off thanks for the KIP, this is definitely a much needed feature. On
the
whole it seems pretty straightforward and I am in favor of the proposal.
Just
a few questions and suggestions here and there:
1. One of the #handle method's parameters is "ProcessorNode node", but
Hi everyone,
After writing quite a few Kafka Streams applications, me and my colleagues
just created KIP-1033 to introduce a new Exception Handler in Kafka Streams
to simplify error handling.
This feature would allow defining an exception handler to automatically
catch exceptions occurring during
39 matches
Mail list logo
