Yes thats bad approach . Mostly users keep a static string for the “id” part in
the spoutConfig. Whats the need to use randomUUID.
--
Harsha
On March 9, 2015 at 11:09:46 PM, Tousif (tousif.pa...@gmail.com) wrote:
Thanks Harsha,
Does zkRoot in the spoutconfig is used along with random string to create path
in Zookeeper?
SpoutConfig(hosts, PropertyManager.getProperty("kafka.spout.topic").toString(),
"/kafkastormroot", UUID.randomUUID().toString());
zkroot is prepended to the random id we give to create the unique place in ZK
where consumption state is stored. If this is true than every time we deploy
our topology it will create a unique path ?
How will it get previous offset?
On Mon, Mar 9, 2015 at 8:36 PM, Harsha <st...@harsha.io> wrote:
If your topology has saved Kafka offset in your zookeeper it will start
processing from that otherwise It checks spoutConfig.forceFromStart set to true
in this case it will try to fetch data from the beginning of the queue i.e
kafka.api.EarliestTime() . If none of the above matches it will pick users
spoutConfig.startOffsetTime.
"kafka.api.OffsetRequest.EarliestTime() finds the beginning of the data in the
logs and starts streaming from there, kafka.api.OffsetRequest.LatestTime() will
only stream new messages.”
If you stopping and re-deploying the topology make sure you used the same name
as KafkaSpout uses topology name to store and retrieve the offsets from
zookeeper.
--
Harsha
On March 9, 2015 at 7:30:38 AM, Tousif (tousif.pa...@gmail.com) wrote:
If your topology has saved Kafka offset in your zookeeper it will start
processing from that otherwise It checks spoutConfig.forceFromStart set to true
in this case it will try to fetch data from the beginning of the queue i.e
kafka.api.EarliestTime() . If none of the above matches it will pick users
spoutConfig.startOffsetTime.
"kafka.api.OffsetRequest.EarliestTime() finds the beginning of the data in the
logs and starts streaming from there, kafka.api.OffsetRequest.LatestTime() will
only stream new messages.”
If you stopping and re-deploying the topology make sure you used the same name
as KafkaSpout uses topology name to store and retrieve the offsets from
zookeeper.
--
Harsha
--
Regards
Tousif Khazi
