Fwd: FW: storm-kafka integration duplication of messages

From: Evers, Maxwell C Sent: Monday, July 07, 2014 1:51 PM To: 'user@storm.incubator.apache.org' Subject: storm-kafka integration duplication of messages Working with a kafkaSpout feeding a storm topology, and dumping back out of the topology to kafka and hdfs (duplicating sending

Re: FW: storm-kafka integration duplication of messages

: Monday, July 07, 2014 1:51 PM To: 'user@storm.incubator.apache.org' Subject: storm-kafka integration duplication of messages Working with a kafkaSpout feeding a storm topology, and dumping back out of the topology to kafka and hdfs (duplicating sending to AllGrouping), I am seeing a great

RE: storm-kafka integration duplication of messages

The AllGrouping was intended to duplicate the messages to HDFS and the outbound kafka, what were are seeing is the duplicaiton of messages from the kafkaspout, so the same message is seen multiple times in hdfs AND in the outbound kafka, as well as the logs of the intermediate bolt handling the

RE: storm-kafka integration duplication of messages

Are you using all grouping for the deserialization bolt too? My original point is that you should not be using this grouping anywhere in your topology. It will cause the duplication you are seeing unless your parallelism is set to one for your bolts. See the section on groupings here

Storm-kafka Integration

Hi, I am trying to integrate storm with kafka, but did not get any proper documentation to do so. Can anyone please help me for getting started with the integration. Thanks and Regards, Komal Thombare =-=-= Notice: The information contained in this e-mail message and/or

Re: Storm-kafka Integration

Do you use trindent or only storm??? Regards, Andres El 02/06/2014, a las 10:43, Komal Thombare komal.thomb...@tcs.com escribió: Hi, I am trying to integrate storm with kafka, but did not get any proper documentation to do so. Can anyone please help me for getting started with the

Re: Storm-kafka Integration

-Andres Gomez wrote: - To: user@storm.incubator.apache.org From: Andres Gomez andresgome...@gmail.com Date: 06/02/2014 02:17PM Subject: Re: Storm-kafka Integration Do you use trindent or only storm??? Regards, Andres El 02/06/2014, a las 10:43, Komal Thombare komal.thomb

Re: Storm-kafka Integration

Hi Komal Have you looked at KafkaSpout? Thanks Deepak On Mon, Jun 2, 2014 at 2:13 PM, Komal Thombare komal.thomb...@tcs.com wrote: Hi, I am trying to integrate storm with kafka, but did not get any proper documentation to do so. Can anyone please help me for getting started with the

Re: Storm-kafka Integration

Experience certainty. IT Services Business Solutions Consulting -Deepak Sharma wrote: - To: user user@storm.incubator.apache.org From: Deepak Sharma deepakmc...@gmail.com Date: 06/02/2014 02:22PM Subject: Re: Storm-kafka Integration Hi

Re: Storm-kafka Integration

Experience certainty. IT Services Business Solutions Consulting -Deepak Sharma wrote: - To: user user@storm.incubator.apache.org From: Deepak Sharma deepakmc...@gmail.com Date: 06/02/2014 02:22PM Subject: Re: Storm-kafka

Re: Storm-kafka Integration

-kafka Integration Hi Komal Have you looked at KafkaSpout? Thanks Deepak On Mon, Jun 2, 2014 at 2:13 PM, Komal Thombare komal.thomb...@tcs.com wrote: Hi, I am trying to integrate storm with kafka, but did not get any proper documentation to do so. Can anyone please help me

Re: Storm-kafka Integration

and Regards, Komal Thombare -Andres Gomez wrote: - To: user@storm.incubator.apache.org From: Andres Gomez andresgome...@gmail.com Date: 06/02/2014 04:29PM Subject: Re: Storm-kafka Integration Hi again, I advise you to use the repo kafka happens to you: https://github.com