Java Kafka Word Count Issue

2015-02-02 Thread Jadhav Shweta
Hi All, I am trying to run Kafka Word Count Program. please find below, the link for the same https://github.com/apache/spark/blob/master/examples/scala-2.10/src/main/java/org/apache/spark/examples/streaming/JavaKafkaWordCount.java I have set spark master to setMaster(local[*]) and I have

Re: Java Kafka Word Count Issue

2015-02-02 Thread Jadhav Shweta
/02/2015 04:13PM Subject: Re: Java Kafka Word Count Issue This is a question about the Kafka producer right? Not Spark On Feb 2, 2015 10:34 AM, Jadhav Shweta jadhav.shw...@tcs.com wrote: Hi All, I am trying to run Kafka Word Count Program. please find below, the link for the same https

Re: Java Kafka Word Count Issue

2015-02-02 Thread Sean Owen
First I would check your code to see how you are pushing records into the topic. Is it reading the whole file each time and resending all of it? Then see if you are using the same consumer.id on the Spark side. Otherwise you are not reading from the same offset when restarting Spark but instead

Re: Java Kafka Word Count Issue

2015-02-02 Thread VISHNU SUBRAMANIAN
and regards Shweta Jadhav -Sean Owen so...@cloudera.com wrote: - To: Jadhav Shweta jadhav.shw...@tcs.com From: Sean Owen so...@cloudera.com Date: 02/02/2015 04:13PM Subject: Re: Java Kafka Word Count Issue This is a question about the Kafka producer right? Not Spark On Feb 2, 2015

Re: Java Kafka Word Count Issue

2015-02-02 Thread Jadhav Shweta
johnfedrickena...@gmail.com Date: 02/02/2015 04:39PM Cc: user@spark.apache.org user@spark.apache.org Subject: Re: Java Kafka Word Count Issue You can use updateStateByKey() to perform the above operation. On Mon, Feb 2, 2015 at 4:29 PM, Jadhav Shweta jadhav.shw...@tcs.com wrote: Hi Sean, Kafka Producer