Hi, I have started to explore the Storm for distributed processing for our use case which we were earlier fulfilling by JMS based MQ system. Topology worked after some efforts. It has one spout (KafkaSpout from kafka-storm project) and 3 bolts. First bolt sets context for other two bolts which in turn do some processing on the tuples and persist the analyzed results in some DB (Mongo, Solr, HBase etc).
Recently the topology stopped working. I am able to submit the topology and it does not throw any error in submitting the topology, however, nimbus.log or worker-6701.log files are not showing any progress and eventually topology does not consume any message. I don't have doubt on KafkaSpout because if it was the culprit, at least some initialization logs of spout and bolts should have been there in nimbus.log or worker-xxxx.log. Isn't it? Here is the snippet of nimbus.log after uploading the jar to cluster Uploading file from client to /hadoop/storm/nimbus/inbox/stormjar-31fe068b-337b-428f-8ae2-fe13c706b2ab.jar 2014-08-25 07:07:49 b.s.d.nimbus [INFO] Finished uploading file from client: /hadoop/storm/nimbus/inbox/stormjar-31fe068b-337b-428f-8ae2-fe13c706b2ab.jar 2014-08-25 07:07:49 b.s.d.nimbus [INFO] Received topology submission for aleads with conf {"topology.max.task.parallelism" nil, "topology.acker.executors" nil, "topology.kryo.register" nil, "topology.kryo.decorators" (), "topology.name" "aleads", "storm.id" "aleads-3-1408964869", "modelId" "ut", "topology.workers" 1, "topology.debug" true} 2014-08-25 07:07:50 b.s.d.nimbus [INFO] Activating aleads: aleads-3-1408964869 2014-08-25 07:07:50 b.s.s.EvenScheduler [INFO] Available slots: (["e56c2cc7-d35a-4355-9906-506618ff70c5" 6701] ["e56c2cc7-d35a-4355-9906-506618ff70c5" 6700]) 2014-08-25 07:07:50 b.s.d.nimbus [INFO] Setting new assignment for topology id aleads-3-1408964869: #backtype.storm.daemon.common.Assignment{:master-code-dir "/hadoop/storm/nimbus/stormdist/aleads-3-1408964869", :node->host {"e56c2cc7-d35a-4355-9906-506618ff70c5" "hdp.ambari"}, :executor->node+port {[2 2] ["e56c2cc7-d35a-4355-9906-506618ff70c5" 6701], [3 3] ["e56c2cc7-d35a-4355-9906-506618ff70c5" 6701], [4 4] ["e56c2cc7-d35a-4355-9906-506618ff70c5" 6701], [5 5] ["e56c2cc7-d35a-4355-9906-506618ff70c5" 6701], [6 6] ["e56c2cc7-d35a-4355-9906-506618ff70c5" 6701], [7 7] ["e56c2cc7-d35a-4355-9906-506618ff70c5" 6701], [8 8] ["e56c2cc7-d35a-4355-9906-506618ff70c5" 6701], [9 9] ["e56c2cc7-d35a-4355-9906-506618ff70c5" 6701], [1 1] ["e56c2cc7-d35a-4355-9906-506618ff70c5" 6701]}, :executor->start-time-secs {[1 1] 1408964870, [9 9] 1408964870, [8 8] 1408964870, [7 7] 1408964870, [6 6] 1408964870, [5 5] 1408964870, [4 4] 1408964870, [3 3] 1408964870, [2 2] 1408964870}} Can anyone guess what I have done wrong and why Storm is not giving any error log anywhere. Storm version is 0.9.1.2.1.3.0-563 (Installed via HortonWorks) Kafka version is 2.10-0.8.1.1 Storm-Kafka version 0.9.2-incubating -- Regards, Vikas Agarwal 91 – 9928301411 InfoObjects, Inc. Execution Matters http://www.infoobjects.com 2041 Mission College Boulevard, #280 Santa Clara, CA 95054 +1 (408) 988-2000 Work +1 (408) 716-2726 Fax