The spout will reemit some messages if it fails to commit offsets to Kafka. Are these CommitFailedExceptions occuring in your logs normally?
Also since the spout stores offsets in Kafka, you may want to check the replication factor on that topic by running `./kafka-topics.sh --zookeeper localhost:2181 --describe --topic __consumer_offsets` in one of your Kafka /bin directories. 2017-08-21 17:17 GMT+02:00 Elyahou Ittah <elyaho...@fiverr.com>: > The config is the default one, I just set the bootstrap server. > > Kafka version is 0.11 > > Storm-kafka-client is 1.1.0 > > On Mon, Aug 21, 2017 at 5:48 PM, Stig Rohde Døssing <s...@apache.org> > wrote: > >> Hi Elyahou, >> >> Could you post your spout configuration, Kafka version and >> storm-kafka-client version? The logs imply that your spouts are not polling >> often enough. >> >> 2017-08-21 <20%2017%2008%2021> 9:44 GMT+02:00 Elyahou Ittah < >> elyaho...@fiverr.com>: >> >>> I noticed that storm kafka spout reconsume all kafka message after a >>> rolling restart of kafka cluster. >>> >>> This issue occured only with kafkaSpout consumer and not for my other >>> consumers (ruby based using the kafka consumer API like kafkaSpout) >>> >>> Attached logs of the spout. >>> >>> Do you know what can cause this kind of behavior ? >>> >> >> >
