Sorry for the late reply. This exception is not from storm. It is caused by the version issue of Hbase.
Thanks a lot for your help. 2014-03-09 16:35 GMT+08:00 Niels Basjes <ni...@basjes.nl>: > Try putting those static initializations into the prepare. > On Mar 8, 2014 12:20 PM, "Zheng Xue" <xuezhs...@gmail.com> wrote: > >> Thanks for you reply. >> >> I excluded the log4j dependency in the pom file. The topology can run in >> Storm-0.9.0-wip21 now. >> >> But another error appears in the last bolt, which is responsible for >> writing statistics into HBase. Here is the exception information: >> >> >> java.lang.RuntimeException: java.lang.NullPointerException >> at >> backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:87) >> at >> backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:58) >> at >> backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:62) >> at >> backtype.storm.daemon.executor$fn__3483$fn__3495$fn__3542.invoke(executor.clj:715) >> at backtype.storm.util$async_loop$fn__441.invoke(util.clj:396) >> at clojure.lang.AFn.run(AFn.java:24) >> at java.lang.Thread.run(Thread.java:744) >> Caused by: java.lang.NullPointerException >> at >> com.mycompany.app.MobileNetLogThresholdBolt.execute(MobileNetLogThresholdBolt.java:81) >> at >> backtype.storm.daemon.executor$fn__3483$tuple_action_fn__3485.invoke(executor.clj:610) >> at >> backtype.storm.daemon.executor$mk_task_receiver$fn__3406.invoke(executor.clj:381) >> at >> backtype.storm.disruptor$clojure_handler$reify__2948.onEvent(disruptor.clj:43) >> at >> backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:79) >> >> Line 81 in MobileNetLogThresholdBolt.java is "table.put(put)" (see the >> java code of this bolt below). Referred to >> https://github.com/nathanmarz/storm/wiki/Troubleshooting, multiple >> threads using the OutputCollector leads to this exception. But how can I >> fix it? Thanks. >> >> public class MobileNetLogThresholdBolt implements IRichBolt { >> private OutputCollector outputCollector; >> public static Configuration configuration; >> public static String tablename = "t_mobilenet_threshold"; >> public static HTable table; >> static { >> configuration = HBaseConfiguration.create(); >> configuration.set("hbase.zookeeper.property.clientPort","2181"); >> configuration.set("hbase.zookeeper.quorum", "xx.xx.xx.xx"); >> configuration.set("hbase.master", "xx.xx.xx.xx:60000"); >> } >> >> >> private Log log = LogFactory.getLog(MobileNetLogThresholdBolt.class); >> >> @Override >> public void prepare(Map map, TopologyContext topologyContext, >> OutputCollector outputCollector) { >> this.outputCollector = outputCollector; >> try { >> table = new HTable(configuration, tablename); >> } catch (IOException e) { >> // TODO Auto-generated catch block >> e.printStackTrace(); >> } >> >> } >> >> >> @Override >> public void execute(Tuple tuple) { >> >> log.info("deal data " + tuple.getString(0) + "=" + >> tuple.getInteger(1)); >> if (tuple.getInteger(1) > 2) { >> >> Put put = new Put(Bytes.toBytes(tuple.getString(0))); >> put.add(Bytes.toBytes("STAT_INFO"), Bytes.toBytes("COUNT"), >> Bytes.toBytes(String.valueOf(tuple.getInteger(1)))); >> try { >> table.put(put); >> } catch (IOException e) { >> e.printStackTrace(); >> } >> } >> this.outputCollector.emit(tuple, tuple.getValues()); >> this.outputCollector.ack(tuple); >> } >> >> @Override >> public void cleanup() { >> >> } >> >> @Override >> public void declareOutputFields(OutputFieldsDeclarer >> outputFieldsDeclarer) { >> } >> >> @Override >> public Map<String, Object> getComponentConfiguration() { >> return null; >> } >> } >> >> >> 2014-03-08 0:53 GMT+08:00 bijoy deb <bijoy.comput...@gmail.com>: >> >>> Hi Zheng, >>> >>> Did you look at the logs for what exactly the error message is? >>> In case you see any error that says "multiple default.yaml in path >>> ...",try the below: >>> --In the with-dependencies jar,can you check if you have a >>> default.yaml or storm.yaml file.If yes,please delete it and try submitting >>> the resulting jar. >>> >>> Thanks >>> Bijoy >>> >>> >>> On Fri, Mar 7, 2014 at 10:51 AM, Zheng Xue <xuezhs...@gmail.com> wrote: >>> >>>> Hi, all: >>>> >>>> I was trying to build a Storm topology with Hadoop and Hbase >>>> dependencies. And I want to run this topology with Storm-on-Yarn. The >>>> version of Storm in it is 0.9.0-wip21. I created the Jar file with Maven, >>>> and the pom.xml file is attached. >>>> >>>> I submitted the topology (with-dependencies), and there was no >>>> exceptions. But it did't run at all. I checked the supervisor logs (see >>>> bottom), which shows that the workers failed to start. >>>> >>>> To trace the reason of this issue, I add a test topology >>>> (WordCountTopology) in the jar file. The problems remains when trying to >>>> submit the WordCountTopology with dependencies, but it works well when >>>> trying to submit the topology without dependencies. >>>> >>>> To make it clear that where is this issue from, I ran the topology >>>> without Storm-on-Yarn. I tried Storm-0.8.1 and Storm-0.9.0-wip21. The >>>> results show that: >>>> 1) Storm-0.8.1 is okay for the jar topology, no matter with or >>>> without dependencies. >>>> 2) Storm-0.9.0-wip21 is okay for the jar topology without >>>> dependencies, but the same issue (workers failed to start) appears for the >>>> "with-dependencies" jar file. >>>> >>>> >>>> >>>> I am a newer for Storm, and I don't know the differences between >>>> these two Storm version. What will result in the worker failure when trying >>>> to submit the "with-dependencies" topology: Is the "with-dependencies" jar >>>> file too big? or something conflicting? >>>> >>>> Any idea? >>>> Will Storm-0.8.1 work with Storm-on-Yarn? >>>> Thanks~ >>>> >>>> >>>> -- >>>> ------------------ >>>> Best Regards! >>>> >>>> Zheng Xue >>>> *http://netlab.sysu.edu.cn/~zhxue <http://netlab.sysu.edu.cn/~zhxue>* >>>> Computer Science and Technology college,Sun Yat-sen University >>>> E-mail: xuezhs...@gmail.com, xuezh...@acm.org >>>> >>> >>> >> >> >> -- >> ------------------ >> Best Regards! >> >> Zheng Xue >> *http://netlab.sysu.edu.cn/~zhxue <http://netlab.sysu.edu.cn/~zhxue>* >> Computer Science and Technology college,Sun Yat-sen University >> E-mail: xuezhs...@gmail.com, xuezh...@acm.org >> > -- ------------------ Best Regards! Zheng Xue *http://netlab.sysu.edu.cn/~zhxue <http://netlab.sysu.edu.cn/~zhxue>* Computer Science and Technology college,Sun Yat-sen University E-mail: xuezhs...@gmail.com, xuezh...@acm.org
