Hi Guozhang,
You are right. I overlooked the fact that skippedDueToDeserializationError
is recorded as DEBUG.
That was it. Now that I got it, it feels like an overkill to set metrics
level to DEBUG just for this!
Thanks for your time!
Srikanth
On Tue, Jan 30, 2018 at 10:56 PM, Guozhang Wang wr
Your code for setting the handler seems right to me.
Another double checking: have you turned on DEBUG level metrics recording
in order for this metric? Note skippedDueToDeserializationError is recorded
as DEBUG level so you need to set metrics.recording.level accordingly
(default is INFO). Lower
Guozhang,
Here is the snippet.
private Properties getProperties() {
Properties p = new Properties();
...
p.put(StreamsConfig.NUM_STREAM_THREADS_CONFIG, kafkaConfig.getString("
streamThreads"));
p.put(StreamsConfig.DEFAULT_DESERIALIZATION_EXCEPTION_HANDLER_CLASS_CONFIG,
LogAndConti
Hi Srikanth,
How did you set the LogAndContinueExceptionHandler in the configs? Could
you copy the code snippet here?
Guozhang
On Sun, Jan 28, 2018 at 11:26 PM, Srikanth wrote:
> Kafka-streams version "1.0.0".
>
> Thanks,
> Srikanth
>
> On Mon, Jan 29, 2018 at 12:23 AM, Guozhang Wang
> wrote
Kafka-streams version "1.0.0".
Thanks,
Srikanth
On Mon, Jan 29, 2018 at 12:23 AM, Guozhang Wang wrote:
> Hello Srikanth,
>
> Which version of Kafka are you using? I'd like to dig for that particular
> branch again.
>
> Guozhang
>
> On Sun, Jan 28, 2018 at 8:54 AM, Srikanth wrote:
>
> > Guozhan
Hello Srikanth,
Which version of Kafka are you using? I'd like to dig for that particular
branch again.
Guozhang
On Sun, Jan 28, 2018 at 8:54 AM, Srikanth wrote:
> Guozhang,
>
> While I understand that this metric is meaningless when handler is set to
> FAIL, in my case I'm actually using LogA
Guozhang,
While I understand that this metric is meaningless when handler is set to
FAIL, in my case I'm actually using LogAndContinueExceptionHandler.
In this case, app needs to report such occurrences. What I noticed is that
only skipped-records is set.
The granularity offered by skippedDueToDes
Guozhang,
I read the comments in KAFKA-6376 and realized that we do something similar
to handle exception in process().
We register a custom metric "process-skipped" and our alerting system looks
for both this and skipped-records.
Srikanth
On Fri, Jan 26, 2018 at 10:26 PM, Guozhang Wang wrote:
Hi Srikanth,
Looked at the source code once again and discussing with other committer I
now remembered why we designed it that way: when you set the
HandlerResponse to FAIL, it means that once a "poison record" is received,
stop the world by throwing this exception all the way up. And hence at tha
Helo Srikanth,
Thanks for reporting this, as I checked the code
skippedDueToDeserializationError is effectively only recorded when the
DeserializationHandlerResponse is not set to FAIL. I agree it is not
exactly matching the documentation's guidance, and will try to file a JIRA
and fix it.
As for
Hello,
As per doc when LogAndContinueExceptionHandler is used it will set
skippedDueToDeserializationError-rate metric to indicate deserialization
error.
I notice that it is never set. Instead skipped-records-rate is set. My
understanding was that skipped-records-rate is set due to timestamp
extra
11 matches
Mail list logo