Hi,
I have started to explore the Storm for distributed processing for our use
case which we were earlier fulfilling by JMS based MQ system. Topology
worked after some efforts. It has one spout (KafkaSpout from kafka-storm
project) and 3 bolts. First bolt sets context for other two bolts which in
turn do some processing on the tuples and persist the analyzed results in
some DB (Mongo, Solr, HBase etc).
Recently the topology stopped working. I am able to submit the topology and
it does not throw any error in submitting the topology, however, nimbus.log
or worker-6701.log files are not showing any progress and eventually
topology does not consume any message. I don't have doubt on KafkaSpout
because if it was the culprit, at least some initialization logs of spout
and bolts should have been there in nimbus.log or worker-xxxx.log. Isn't it?
Here is the snippet of nimbus.log after uploading the jar to cluster
Uploading file from client to
/hadoop/storm/nimbus/inbox/stormjar-31fe068b-337b-428f-8ae2-fe13c706b2ab.jar
2014-08-25 07:07:49 b.s.d.nimbus [INFO] Finished uploading file from
client:
/hadoop/storm/nimbus/inbox/stormjar-31fe068b-337b-428f-8ae2-fe13c706b2ab.jar
2014-08-25 07:07:49 b.s.d.nimbus [INFO] Received topology submission for
aleads with conf {"topology.max.task.parallelism" nil,
"topology.acker.executors" nil, "topology.kryo.register" nil,
"topology.kryo.decorators" (), "topology.name" "aleads", "storm.id"
"aleads-3-1408964869", "modelId" "ut", "topology.workers" 1,
"topology.debug" true}
2014-08-25 07:07:50 b.s.d.nimbus [INFO] Activating aleads:
aleads-3-1408964869
2014-08-25 07:07:50 b.s.s.EvenScheduler [INFO] Available slots:
(["e56c2cc7-d35a-4355-9906-506618ff70c5" 6701]
["e56c2cc7-d35a-4355-9906-506618ff70c5" 6700])
2014-08-25 07:07:50 b.s.d.nimbus [INFO] Setting new assignment for topology
id aleads-3-1408964869:
#backtype.storm.daemon.common.Assignment{:master-code-dir
"/hadoop/storm/nimbus/stormdist/aleads-3-1408964869", :node->host
{"e56c2cc7-d35a-4355-9906-506618ff70c5" "hdp.ambari"}, :executor->node+port
{[2 2] ["e56c2cc7-d35a-4355-9906-506618ff70c5" 6701], [3 3]
["e56c2cc7-d35a-4355-9906-506618ff70c5" 6701], [4 4]
["e56c2cc7-d35a-4355-9906-506618ff70c5" 6701], [5 5]
["e56c2cc7-d35a-4355-9906-506618ff70c5" 6701], [6 6]
["e56c2cc7-d35a-4355-9906-506618ff70c5" 6701], [7 7]
["e56c2cc7-d35a-4355-9906-506618ff70c5" 6701], [8 8]
["e56c2cc7-d35a-4355-9906-506618ff70c5" 6701], [9 9]
["e56c2cc7-d35a-4355-9906-506618ff70c5" 6701], [1 1]
["e56c2cc7-d35a-4355-9906-506618ff70c5" 6701]}, :executor->start-time-secs
{[1 1] 1408964870, [9 9] 1408964870, [8 8] 1408964870, [7 7] 1408964870, [6
6] 1408964870, [5 5] 1408964870, [4 4] 1408964870, [3 3] 1408964870, [2 2]
1408964870}}
Can anyone guess what I have done wrong and why Storm is not giving any
error log anywhere.
Storm version is 0.9.1.2.1.3.0-563 (Installed via HortonWorks)
Kafka version is 2.10-0.8.1.1
Storm-Kafka version 0.9.2-incubating
--
Regards,
Vikas Agarwal
91 – 9928301411
InfoObjects, Inc.
Execution Matters
http://www.infoobjects.com
2041 Mission College Boulevard, #280
Santa Clara, CA 95054
+1 (408) 988-2000 Work
+1 (408) 716-2726 Fax
