Hello, I'm getting the following error from the storm client when trying to submit a topology: 8637 [main] INFO net.mycompany.storm.topology.TridentTestTopology - Built Topology with 383 bolts and 96 spouts and 0 state spouts 8791 [main] INFO backtype.storm.StormSubmitter - Jar not uploaded to master yet. Submitting jar... 8794 [main] INFO backtype.storm.StormSubmitter - Uploading topology jar /Users/.../myproject-1.0.0-jar-with-dependencies.jar to assigned location: /app/storm/nimbus/inbox/stormjar-71edf0cf-11d0-4c4a-9e1c-2ec3211c06f3.jar 16317 [main] INFO backtype.storm.StormSubmitter - Successfully uploaded topology jar to assigned location: /app/storm/nimbus/inbox/stormjar-71edf0cf-11d0-4c4a-9e1c-2ec3211c06f3.jar 16317 [main] INFO backtype.storm.StormSubmitter - Submitting topology trident-test in distributed mode with conf {"topology.workers":2} Exception in thread "main" java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Arrays.java:2271) at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:113) at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93) at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:140) at org.apache.thrift7.transport.TFramedTransport.write(TFramedTransport.java:146) at org.apache.thrift7.protocol.TBinaryProtocol.writeBinary(TBinaryProtocol.java:196) at backtype.storm.generated.ComponentObject.writeValue(ComponentObject.java:235) at org.apache.thrift7.TUnion.write(TUnion.java:164) at backtype.storm.generated.Bolt.write(Bolt.java:379) at backtype.storm.generated.StormTopology.write(StormTopology.java:589) at backtype.storm.generated.Nimbus$submitTopology_args.write(Nimbus.java:2080) at org.apache.thrift7.TServiceClient.sendBase(TServiceClient.java:63) at backtype.storm.generated.Nimbus$Client.send_submitTopology(Nimbus.java:156) at backtype.storm.generated.Nimbus$Client.submitTopology(Nimbus.java:145) at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:98) at backtype.storm.StormSubmitter.submitTopology(StormSubmitter.java:58) at net.acesinc.storm.topology.TridentTestTopology.main(TridentTestTopology.java:358)
I can see that Nimbus receives the jar file successfully from the logs: 2014-06-17 21:12:53 b.s.d.nimbus [INFO] Uploading file from client to /app/storm/nimbus/inbox/stormjar-71edf0cf-11d0-4c4a-9e1c-2ec3211c06f3.jar 2014-06-17 21:13:00 b.s.d.nimbus [INFO] Finished uploading file from client: /app/storm/nimbus/inbox/stormjar-71edf0cf-11d0-4c4a-9e1c-2ec3211c06f3.jar Then right after the Finished error, I get the OutOfMemoryError on the client side. What I have tried so far: I have increased the memory of the storm client to 1024m. I have increased the memory of nimbus, the supervisors, and the workers to 768m. I have set nimbus.thrift.max_buffer_size: 104857600 (100m) Nothing so far has helped. I can't tell if it is the storm client that is running out of memory or if it is an error coming back from the remote side. A little more on the setup, I'm using Wirbelstrum to run Storm in a clustered configuration. There is a nimbus node and two supervisor nodes. I also have a zookeeper, redis, and a drpc node running, all on a MBP with 16GB of ram. I'm always close to maxing out, so I'm not sure if that is a problem. Any help you could provide would be great. Andrew