Yes, set 'nimbus.childopts: "-Xmx?????"' in your storm.yaml, and restart nimbus. If unset, I believe the default is "-Xmx1024m", for a max of 1024 MB heap.
You can set it to "-Xmx2048m", for example, to have a max heap size of 2048 MB. Set this on the node that runs nimbus, not in your topology conf. -- Derek On 3/10/14, 14:19, shahab wrote:
Hi, I am facing "OutOfMemoryError: Java heap space" exception in Nimbus while running in cluster mode. I just wonder what are the possible JVM or Storm options that I can set to overcome this problem? I am running a storm topology in Cluster mode where all servers (zookeeper, nimbus, supervisor and worker) are in one machine. The toplogy that I use is as follows: conf.setMaxSpoutPending(2000); // maximum number of pending messages at spout conf.setNumWorkers(4); conf.put(Config.STORM_ZOOKEEPER_CONNECTION_TIMEOUT, "120000"); conf.setMaxTaskParallelism(2); but I get the following Exception in Nimbus log file: java.lang.OutOfMemoryError: Java heap space at org.apache.commons.io.output.ByteArrayOutputStream.toByteArray(ByteArrayOutputStream.java:271) at org.apache.commons.io.IOUtils.toByteArray(IOUtils.java:219) at org.apache.commons.io.FileUtils.readFileToByteArray(FileUtils.java:1136) at backtype.storm.daemon.nimbus$read_storm_topology.invoke(nimbus.clj:305) at backtype.storm.daemon.nimbus$compute_executors.invoke(nimbus.clj:407) at backtype.storm.daemon.nimbus$compute_executor__GT_component.invoke(nimbus.clj:420) at backtype.storm.daemon.nimbus$read_topology_details.invoke(nimbus.clj:315) at backtype.storm.daemon.nimbus$mk_assignments$iter__3416__3420$fn__3421.invoke(nimbus.clj:636) at clojure.lang.LazySeq.sval(LazySeq.java:42) at clojure.lang.LazySeq.seq(LazySeq.java:60) at clojure.lang.RT.seq(RT.java:473) at clojure.core$seq.invoke(core.clj:133) at clojure.core.protocols$seq_reduce.invoke(protocols.clj:30) at clojure.core.protocols$fn__5875.invoke(protocols.clj:54) at clojure.core.protocols$fn__5828$G__5823__5841.invoke(protocols.clj:13) at clojure.core$reduce.invoke(core.clj:6030) at clojure.core$into.invoke(core.clj:6077) at backtype.storm.daemon.nimbus$mk_assignments.doInvoke(nimbus.clj:635) at clojure.lang.RestFn.invoke(RestFn.java:410) at backtype.storm.daemon.nimbus$fn__3592$exec_fn__1228__auto____3593$fn__3598$fn__3599.invoke(nimbus.clj:872) at backtype.storm.daemon.nimbus$fn__3592$exec_fn__1228__auto____3593$fn__3598.invoke(nimbus.clj:871) at backtype.storm.timer$schedule_recurring$this__1776.invoke(timer.clj:69) at backtype.storm.timer$mk_timer$fn__1759$fn__1760.invoke(timer.clj:33) at backtype.storm.timer$mk_timer$fn__1759.invoke(timer.clj:26) at clojure.lang.AFn.run(AFn.java:24) at java.lang.Thread.run(Thread.java:744) 2014-03-10 20:10:02 NIOServerCnxn [ERROR] Thread Thread[pool-4-thread-40,5,main] died java.lang.OutOfMemoryError: Java heap space at org.apache.commons.io.output.ByteArrayOutputStream.toByteArray(ByteArrayOutputStream.java:271) at org.apache.commons.io.IOUtils.toByteArray(IOUtils.java:219) at org.apache.commons.io.FileUtils.readFileToByteArray(FileUtils.java:1136) at backtype.storm.daemon.nimbus$read_storm_topology.invoke(nimbus.clj:305) at backtype.storm.daemon.nimbus$fn__3592$exec_fn__1228__auto__$reify__3605.getTopologyInfo(nimbus.clj:1066) at backtype.storm.generated.Nimbus$Processor$getTopologyInfo.getResult(Nimbus.java:1481) at backtype.storm.generated.Nimbus$Processor$getTopologyInfo.getResult(Nimbus.java:1469) at org.apache.thrift7.ProcessFunction.process(ProcessFunction.java:32) at org.apache.thrift7.TBaseProcessor.process(TBaseProcessor.java:34) at org.apache.thrift7.server.TNonblockingServer$FrameBuffer.invoke(TNonblockingServer.java:632) at org.apache.thrift7.server.THsHaServer$Invocation.run(THsHaServer.java:201) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:744) 2014-03-10 20:10:02 util [INFO] Halting process: ("Error when processing an event") best , /Shahab