The AllGrouping was intended to duplicate the messages to HDFS and the outbound kafka, what were are seeing is the duplicaiton of messages from the kafkaspout, so the same message is seen multiple times in hdfs AND in the outbound kafka, as well as the logs of the intermediate bolt handling the deserialization. Presently there is not much more going on in storm that the deserialization. I wanted to get the skeleton of the flow working before I introduced new complications inside of storm.
- Fwd: FW: storm-kafka integration duplication of messages Max Evers
- Re: FW: storm-kafka integration duplication of messages Nathan Leung
- RE: storm-kafka integration duplication of messages Max Evers
- RE: storm-kafka integration duplication of messages Nathan Leung