RE: storm-kafka integration duplication of messages

2014-07-07 Thread Nathan Leung
Are you using all grouping for the deserialization bolt too? My original point is that you should not be using this grouping anywhere in your topology. It will cause the duplication you are seeing unless your parallelism is set to one for your bolts. See the section on groupings here https://stor

RE: storm-kafka integration duplication of messages

2014-07-07 Thread Max Evers
The AllGrouping was intended to duplicate the messages to HDFS and the outbound kafka, what were are seeing is the duplicaiton of messages from the kafkaspout, so the same message is seen multiple times in hdfs AND in the outbound kafka, as well as the logs of the intermediate bolt handling the des

Re: FW: storm-kafka integration duplication of messages

2014-07-07 Thread Nathan Leung
day, July 07, 2014 1:51 PM > > To: 'user@storm.incubator.apache.org' > > Subject: storm-kafka integration duplication of messages > > > > > > > > Working with a kafkaSpout feeding a storm topology, and dumping back out > of the topology to kafka and hdf

Fwd: FW: storm-kafka integration duplication of messages

2014-07-07 Thread Max Evers
> From: Evers, Maxwell C > Sent: Monday, July 07, 2014 1:51 PM > To: 'user@storm.incubator.apache.org' > Subject: storm-kafka integration duplication of messages > > > > Working with a kafkaSpout feeding a storm topology, and dumping back out of the topology to k

Re: Storm-kafka Integration

2014-06-02 Thread Anis Nasir
Thanks and Regards, > > Komal Thombare > > > -Andres Gomez wrote: - > > To: user@storm.incubator.apache.org > From: Andres Gomez > Date: 06/02/2014 04:29PM > > Subject: Re: Storm-kafka Integration > > Hi again, > > I advise you to use the repo

Re: Storm-kafka Integration

2014-06-02 Thread Komal Thombare
Hi Andres, Still i am getting the same error. Thanks and Regards, Komal Thombare -Andres Gomez wrote: - To: user@storm.incubator.apache.org From: Andres Gomez Date: 06/02/2014 04:29PM Subject: Re: Storm-kafka Integration Hi again, I advise you to use the repo kafka happens to you

Re: Storm-kafka Integration

2014-06-02 Thread Andres Gomez
re >> Tata Consultancy Services Limited >> Ph:- 086-55388772 >> Mail-to: komal.thomb...@tcs.com >> Website: http://www.tcs.com >> >> Experience certainty. IT Services >> Business Solutions >> Consulting >&

Re: Storm-kafka Integration

2014-06-02 Thread Komal Thombare
, Komal Thombare -Andres Gomez wrote: - To: user@storm.incubator.apache.org From: Andres Gomez Date: 06/02/2014 02:33PM Subject: Re: Storm-kafka Integration You should make a clone of this project: https://github.com/apache/incubator-storm/tree/master/external/storm-kafka and do “mvn

Re: Storm-kafka Integration

2014-06-02 Thread Andres Gomez
t; Ph:- 086-55388772 > Mail-to: komal.thomb...@tcs.com > Website: http://www.tcs.com > > Experience certainty. IT Services > Business Solutions > Consulting > > > -----Deepak Sharma wrote: - > To: user > From

Re: Storm-kafka Integration

2014-06-02 Thread Joe Stein
b...@tcs.com > Website: http://www.tcs.com > > Experience certainty. IT Services > Business Solutions > Consulting > > > -Deepak Sharma wrote: - > > To: user > From: Deepak Sharma > Date: 06/02/2014 02:2

Re: Storm-kafka Integration

2014-06-02 Thread Komal Thombare
Experience certainty. IT Services Business Solutions Consulting -Deepak Sharma wrote: - To: user From: Deepak Sharma Date: 06/02/2014 02:22PM Subject: Re: Storm-kafka Integration Hi Komal Have you looked at KafkaSpout? Thanks Deepak

Re: Storm-kafka Integration

2014-06-02 Thread Deepak Sharma
Hi Komal Have you looked at KafkaSpout? Thanks Deepak On Mon, Jun 2, 2014 at 2:13 PM, Komal Thombare wrote: > Hi, > > I am trying to integrate storm with kafka, but did not get any proper > documentation to do so. > Can anyone please help me for getting started with the integration. > > Thanks

Re: Storm-kafka Integration

2014-06-02 Thread Komal Thombare
-Andres Gomez wrote: - To: user@storm.incubator.apache.org From: Andres Gomez Date: 06/02/2014 02:17PM Subject: Re: Storm-kafka Integration Do you use trindent or only storm??? Regards, Andres El 02/06/2014, a las 10:43, Komal Thombare escribió: Hi, I am trying to

Re: Storm-kafka Integration

2014-06-02 Thread Andres Gomez
Do you use trindent or only storm??? Regards, Andres El 02/06/2014, a las 10:43, Komal Thombare escribió: > Hi, > > I am trying to integrate storm with kafka, but did not get any proper > documentation to do so. > Can anyone please help me for getting started with the integration. > > Thanks

Storm-kafka Integration

2014-06-02 Thread Komal Thombare
Hi, I am trying to integrate storm with kafka, but did not get any proper documentation to do so. Can anyone please help me for getting started with the integration. Thanks and Regards, Komal Thombare =-=-= Notice: The information contained in this e-mail message and/or at