Re: KStreams.reduceByKey passing nulls to my Deserializer?

2016-06-14 Thread Guozhang Wang
Thanks Avi! On Tue, Jun 14, 2016 at 7:41 AM, Avi Flax wrote: > > > On Jun 10, 2016, at 18:47, Guozhang Wang wrote: > > > > Yes, this is possible > > OK, good to know — thanks! > > I just checked the code in the Deserializers included with Kafka and I see > that they check for null values and si

Re: KStreams.reduceByKey passing nulls to my Deserializer?

2016-06-14 Thread Avi Flax
> On Jun 10, 2016, at 18:47, Guozhang Wang wrote: > > Yes, this is possible OK, good to know — thanks! I just checked the code in the Deserializers included with Kafka and I see that they check for null values and simply pass them through; I guess that’s the correct behavior. I’ve opened a P

Re: KStreams.reduceByKey passing nulls to my Deserializer?

2016-06-10 Thread Guozhang Wang
Hello Avi, Yes, this is possible: although we checked nullable keys when doing reduce / aggregations: https://github.com/apache/kafka/blob/0.10.0/streams/src/main/java/org/apache/kafka/streams/kstream/internals/KStreamReduce.java#L67 We do not check if the there are any values returned from the

Re: KStreams.reduceByKey passing nulls to my Deserializer?

2016-06-10 Thread Avi Flax
> On Jun 10, 2016, at 14:24, Avi Flax wrote: > > Hi, I’m using Kafka Streams (0.10.0) with JRuby, most of my scripts/nodes are > working well at this point, except for one which is using reduceByKey. Whoops, I should probably share my code as well! Here’s the topology: builder.stream(key_se

KStreams.reduceByKey passing nulls to my Deserializer?

2016-06-10 Thread Avi Flax
Hi, I’m using Kafka Streams (0.10.0) with JRuby, most of my scripts/nodes are working well at this point, except for one which is using reduceByKey. This is the first time I’m trying to use the local state store so it’s possible there’s something misconfigured, I’m not sure. My config is pretty