Re: KafkaStream Merging two topics is not working fro custom datatypes

2016-10-12 Thread Michael Noll
Happy to hear it works now for you, Ratha. -Michael On Wed, Oct 12, 2016 at 6:06 AM, Ratha v wrote: > Sorry my fault, In the kafkaConsumer I messed with 'value.deserializer' > property.. > Now things are working fine.. > Thanks a lot. > > On 12 October 2016 at 14:10, Ratha v wrote: > > > HI M

Re: KafkaStream Merging two topics is not working fro custom datatypes

2016-10-11 Thread Ratha v
Sorry my fault, In the kafkaConsumer I messed with 'value.deserializer' property.. Now things are working fine.. Thanks a lot. On 12 October 2016 at 14:10, Ratha v wrote: > HI Michael; > Sorry , after setting "auto.offset.reset" to 'earliest' , I see messages > in my 'targetTopic'. > But still

Re: KafkaStream Merging two topics is not working fro custom datatypes

2016-10-11 Thread Ratha v
HI Michael; Sorry , after setting "auto.offset.reset" to 'earliest' , I see messages in my 'targetTopic'. But still I get my class cast exception issue, when I consume message from the 'targetTopic'. (To consume message I use KafkaConsumer highlevel API) *ConsumerRecords records = consumer.poll(L

Re: KafkaStream Merging two topics is not working fro custom datatypes

2016-10-11 Thread Ratha v
HI Michael; Really appreciate for the clear explanation.. I modified my code as you mentioned. I have written custom, Serde, serializer,deserializer. But now the problem i see is, both topics are not merged. Means, Messages in the 'sourcetopic' not to passed to 'targetTopic' . ('targetTopic has '0

Re: KafkaStream Merging two topics is not working fro custom datatypes

2016-10-11 Thread Michael Noll
When I wrote: "If you haven't changed to default key and value serdes, then `to()` will fail because [...]" it should have read: "If you haven't changed the default key and value serdes, then `to()` will fail because [...]" On Tue, Oct 11, 2016 at 11:12 AM, Michael Noll wrote: > Rat

Re: KafkaStream Merging two topics is not working fro custom datatypes

2016-10-11 Thread Michael Noll
Ratha, if you based your problematic code on the PipeDemo example, then you should have these two lines in your code (which most probably you haven't changed): props.put(StreamsConfig.KEY_SERDE_CLASS_CONFIG, Serdes.String().getClass()); props.put(StreamsConfig.VALUE_SERDE_CLASS_CONFIG, Se

Re: KafkaStream Merging two topics is not working fro custom datatypes

2016-10-10 Thread Ratha v
I checked my target topic and I see few messages than the source topic. (If source topic have 5 messages, I see 2 messages in my target topic) What settings I need to do ? And, when I try to consume message from the target topic, I get ClassCast Exception. java.lang.ClassCastException: java.lang.

KafkaStream Merging two topics is not working fro custom datatypes

2016-10-10 Thread Ratha v
Hi all; I have custom datatype defined (a pojo class). I copy messages from one topic to another topic. I do not see any messages in my target topic. This works fro string messages, but not for my custom message. Waht might be the cause? I followed this sample [1] [1] https://github.com/apache/kaf