[ https://issues.apache.org/jira/browse/STORM-3667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ethan Li updated STORM-3667: ---------------------------- Summary: Exception message on submitting topology due to incorrect worker max heap size should be more straightforward (was: Exception on submitting topology due to incorrect worker max heap size should be more straightforward ) > Exception message on submitting topology due to incorrect worker max heap > size should be more straightforward > -------------------------------------------------------------------------------------------------------------- > > Key: STORM-3667 > URL: https://issues.apache.org/jira/browse/STORM-3667 > Project: Apache Storm > Issue Type: Improvement > Reporter: Ethan Li > Assignee: Ethan Li > Priority: Minor > > Submitting a topology can fail when the largest component heap size is larger > than > {code:java} > topology.worker.max.heap.size.mb > {code} > For example, when you submit a topology with the following command > {code:java} > bin/storm jar /home/y/lib64/jars/storm-starter.jar > org.apache.storm.starter.WordCountTopology -c > topology.worker.max.heap.size.mb=100 -c > topology.component.resources.onheap.memory.mb=120 wc > {code} > While the nimbus log shows > {code:java} > 2020-07-06 21:51:12.026 o.a.s.d.n.Nimbus pool-29-thread-45 [WARN] Topology > submission exception. (topology name='wc') > java.lang.IllegalArgumentException: Topology will not be able to be > successfully scheduled: Config TOPOLOGY_WORKER_MAX_HEAP_SIZE_MB=100.0 < 120.0 > (Largest memory requirement of a component in the topology). Perhaps set > TOPOLOGY_WORKER_MAX_HEAP_SIZE_MB to a larger amount > at > org.apache.storm.daemon.nimbus.Nimbus.validateTopologyWorkerMaxHeapSizeConfigs(Nimbus.java:1650) > ~[storm-server-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at > org.apache.storm.daemon.nimbus.Nimbus.submitTopologyWithOpts(Nimbus.java:3145) > [storm-server-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > at > org.apache.storm.daemon.nimbus.Nimbus.submitTopology(Nimbus.java:3107) > [storm-server-2.3.0-SNAPSHOT.jar:2.3.0-SNAPSHOT] > {code} > the exception returned to users is > {code:java} > 21:51:12.012 [main] INFO o.a.s.StormSubmitter - Submitting topology wc in > distributed mode with conf > {"storm.zookeeper.topology.auth.scheme":"digest","storm.zookeeper.topology.auth.payload":"-8967725949793348049:-5020411463316947973","topology.workers":3,"topology.component.resources.onheap.memory.mb":120,"topology.worker.max.heap.size.mb":100,"topology.debug":true} > java.lang.RuntimeException: org.apache.storm.thrift.TApplicationException: > Internal error processing submitTopology > at > org.apache.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:291) > at > org.apache.storm.StormSubmitter.submitTopology(StormSubmitter.java:210) > at > org.apache.storm.StormSubmitter.submitTopology(StormSubmitter.java:173) > at > org.apache.storm.topology.ConfigurableTopology.submit(ConfigurableTopology.java:119) > at > org.apache.storm.starter.WordCountTopology.run(WordCountTopology.java:58) > at > org.apache.storm.topology.ConfigurableTopology.start(ConfigurableTopology.java:68) > at > org.apache.storm.starter.WordCountTopology.main(WordCountTopology.java:36) > Caused by: org.apache.storm.thrift.TApplicationException: Internal error > processing submitTopology > at > org.apache.storm.thrift.TServiceClient.receiveBase(TServiceClient.java:79) > at > org.apache.storm.generated.Nimbus$Client.recv_submitTopology(Nimbus.java:319) > at > org.apache.storm.generated.Nimbus$Client.submitTopology(Nimbus.java:303) > at > org.apache.storm.StormSubmitter.submitTopologyInDistributeMode(StormSubmitter.java:344) > at > org.apache.storm.StormSubmitter.submitTopologyAs(StormSubmitter.java:279) > ... 6 more > {code} > which is not very helpful. -- This message was sent by Atlassian Jira (v8.3.4#803005)