[
https://issues.apache.org/jira/browse/STORM-893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14935874#comment-14935874
]
ASF GitHub Bot commented on STORM-893:
--------------------------------------
Github user revans2 commented on a diff in the pull request:
https://github.com/apache/storm/pull/746#discussion_r40730552
--- Diff: storm-core/src/jvm/backtype/storm/StormSubmitter.java ---
@@ -442,4 +444,36 @@ public static String submitJar(Map conf, String
localJar, ProgressListener liste
*/
public void onCompleted(String srcFile, String targetFile, long
totalBytes);
}
+
+
+ private static void validateConfs(Map stormConf, StormTopology
topology) throws IllegalArgumentException {
+ LOG.info("Validating storm Confs...");
+ double largestMemReq = getMaxExecutorMemoryUsageForTopo(topology,
stormConf);
+ Double topologyWorkerMaxHeapSize =
Utils.getDouble(stormConf.get(Config.TOPOLOGY_WORKER_MAX_HEAP_SIZE_MB));
+ if(topologyWorkerMaxHeapSize < largestMemReq) {
+ throw new IllegalArgumentException("Topology will not be able
to be successfully scheduled: Config TOPOLOGY_WORKER_MAX_HEAP_SIZE_MB="
+
+Utils.getDouble(stormConf.get(Config.TOPOLOGY_WORKER_MAX_HEAP_SIZE_MB)) + " <
"
+ + largestMemReq + " (Largest memory
requirement of a component in the topology). Perhaps set
TOPOLOGY_WORKER_MAX_HEAP_SIZE_MB to a larger amount");
+ }
+ }
+
+
+ private static double getMaxExecutorMemoryUsageForTopo(StormTopology
topology, Map topologyConf) {
+ double largestMemoryOperator = 0.0;
+ for(Map<String, Double> entry :
backtype.storm.scheduler.resource.Utils.getBoltsResources(topology,
topologyConf).values()) {
--- End diff --
Can we rename backtype.storm.scheduler.resource.Utils to be something like
ResourceUtils instead. Just so there is no conflict with imports. I think it
will make the code cleaner.
> Resource Aware Scheduling
> -------------------------
>
> Key: STORM-893
> URL: https://issues.apache.org/jira/browse/STORM-893
> Project: Apache Storm
> Issue Type: Umbrella
> Reporter: Robert Joseph Evans
> Assignee: Boyang Jerry Peng
> Attachments: resource_aware_scheduler_api.pdf
>
>
> At Yahoo we have been working on resource aware scheduling in storm, based
> off of some work done in academia. This rollup ticket is to track the
> complete project. With several sub tasks. Some that are already done and
> need to be pushed back, and others that we have not started on yet.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
