[ https://issues.apache.org/jira/browse/STORM-2820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
ASF GitHub Bot updated STORM-2820: ---------------------------------- Labels: pull-request-available (was: ) > validateTopologyWorkerMaxHeapSizeConfigs function never picks up the value > set by nimbus > ----------------------------------------------------------------------------------------- > > Key: STORM-2820 > URL: https://issues.apache.org/jira/browse/STORM-2820 > Project: Apache Storm > Issue Type: Bug > Reporter: Ethan Li > Assignee: Ethan Li > Labels: pull-request-available > > https://github.com/apache/storm/blob/master/storm-server/src/main/java/org/apache/storm/daemon/nimbus/Nimbus.java#L2548 > {code:java} > @VisibleForTesting > static void validateTopologyWorkerMaxHeapSizeConfigs( > Map<String, Object> stormConf, StormTopology topology) { > double largestMemReq = getMaxExecutorMemoryUsageForTopo(topology, > stormConf); > double topologyWorkerMaxHeapSize = > > ObjectReader.getDouble(stormConf.get(Config.TOPOLOGY_WORKER_MAX_HEAP_SIZE_MB), > 768.0); > if (topologyWorkerMaxHeapSize < largestMemReq) { > throw new IllegalArgumentException( > "Topology will not be able to be successfully scheduled: > Config " > + "TOPOLOGY_WORKER_MAX_HEAP_SIZE_MB=" > + topologyWorkerMaxHeapSize > + " < " + largestMemReq + " (Largest memory requirement > of a component in the topology)." > + " Perhaps set TOPOLOGY_WORKER_MAX_HEAP_SIZE_MB to a > larger amount"); > } > } > {code} > The topologyWorkerMaxHeapSize in the above code is either the value from > topology configuration (set by the topology) or 768.0. It never picks up the > value set by nimbus . > A test: > I set > {code:java} > topology.worker.max.heap.size.mb: 2000.0 > {code} > in storm.yaml. > And have the WordCountTopology to have the WordCount bolt with memory load of > 1024MB. > {code:java} > builder.setBolt("count", new WordCount(), 12).fieldsGrouping("split", new > Fields("word")).setMemoryLoad(1024); > {code} > I got an error when submitting this topology. The nimbus log shows > {code:java} > 2017-11-15 19:46:43.085 o.a.s.d.n.Nimbus pool-14-thread-2 [WARN] Topology > submission exception. (topology name='wc') > java.lang.IllegalArgumentException: Topology will not be able to be > successfully scheduled: Config TOPOLOGY_WORKER_MAX_HEAP_SIZE_MB=768.0 < > 1024.0 (Largest memory requirement of a component in the topology). Perhaps > set TOPOLOGY_WORKER_MAX_HEAP_SIZE_MB to a larger amount > {code} > -- This message was sent by Atlassian JIRA (v6.4.14#64029)