Dear all,
1. For cases when there are multiple topics to read from, do I have to provide
different IDs for each topic?
It seems like only one consumer offset is written in zk if I use same ID for
two topic consumers.
SpoutConfig conf1= new SpoutConfig(zkBrokerHosts, "topic1", zkRoot,
"my-topology-name");
KafkaSpout spout1= new KafkaSpout(conf);
SpoutConfig conf2= new SpoutConfig(zkBrokerHosts, "topic2", zkRoot,
"my-topology-name");KafkaSpout spout2= new KafkaSpout(conf2);
2. In that case, can I set multiple topics in SpoutConfig? such as SpoutConfig
conf= new SpoutConfig(zkBrokerHosts, new String[]{"topic1","topic2"}, zkRoot,
"my-topology-name");
3. how to figure out how far behind is my consumer?kafka provides a way to
check lag
http://community.spiceworks.com/how_to/show/77610-how-far-behind-is-your-kafka-consumer
bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker --group
<consumer_group_name> --zkconnect <zookeeper_host:port> --topic
<topic_name>
But the way kafkaspout writes offset to kafka is different from traditional
consumers so if you run that command, it fails.
Thanks in advance.Sincerely,
