I am facing a very weird issue at the moment.
I have a storm topology which uses Kafka Spout and it stores offsets in ZK.
Kafka topic sometimes do not receive messages for a brief period of time
and the topology catches up with the Kafka offsets.
But after sometime, offset stored in ZK change
Keep in mind that Zookeeper stores the Kafka offsets as they relate to the
consumer group, not overall. Maybe you have another component running under the
same consumer group or you could experiment with running your topology with a
different Kafka consumer group.
Sent from my iPhone
> On
Santosh,
Make sure your SpoutConfig(BrokerHosts hosts, String topic,
String zkRoot, String id); "String id" is static value and
unique across the topologies. If you killing & re-deploying
the topology the value shouldn't be changing. Apart from that
Craig and Harsha: Those were the points I was following while
troubleshooting the issue!
I will check with my colleagues whether someone uses the same zk root.
Spout id is unique across each topology but same across all the deployments
of same topology
I have useStartOffsetTimeIfOffsetOutOfRange
