[
https://issues.apache.org/jira/browse/STORM-1093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14963410#comment-14963410
]
ASF GitHub Bot commented on STORM-1093:
---------------------------------------
Github user revans2 commented on a diff in the pull request:
https://github.com/apache/storm/pull/790#discussion_r42380191
--- Diff: conf/defaults.yaml ---
@@ -140,7 +140,8 @@ supervisor.memory.capacity.mb: 3072.0
supervisor.cpu.capacity: 400.0
### worker.* configs are for task workers
-worker.childopts: "-Xmx768m"
+worker.heap.memory.mb: 768
+worker.childopts: "-Xmx%HEAP-MEM%m"
--- End diff --
The reason for this is backwards compatibility + flexibility. If an admin
has set their own worker.childopts for a given supervisor to say 2GB, we want
that to have higher priority over what we calculate. Similarly if someone sets
topology.worker.childopts it needs to have higher priority over the other two.
But what if the topology wants to test out the resource aware scheduler on just
their topology? With this they can set topology.worker.childopts to
"-Xmx%HEAP-MEM%m" and override the hard coded 2GB that was there before.
> Launching Workers with resources specified in resource-aware schedulers
> -----------------------------------------------------------------------
>
> Key: STORM-1093
> URL: https://issues.apache.org/jira/browse/STORM-1093
> Project: Apache Storm
> Issue Type: Story
> Components: storm-core
> Reporter: Zhuo Liu
> Assignee: Zhuo Liu
>
> Currently, we have Resource Aware Scheduler (STORM-894) in nimbus, which can
> allocate different types of resource (CPU, onheap-memory, offheap-memory) to
> the workers assigned to each topology's tasks.
> However, such resources are not visible to the supervisor, therefore,
> supervisor will still launch workers with fixed amount of heap size memory
> (e.g., -Xmx=768M).
> Therefore, we need a whole set of schemes that allow nimbus to put different
> types of resources to each worker slot, then push the resources with
> assignment to ZooKeepers; also, at the supervisor side, such resources in
> each worker slot should be used by supervisor for launching a worker's JVM
> (initially, the JVM heap size).
> This scheme can be used not only by RAS scheduler (STORM-893), but also by
> any customized scheduler for conducting mem/cpu/network resource
> specified-scheduling.
> In the future, the resources of memory, CPU and network can also be used by
> supervisor to launch a worker in a resource-segregated container, such as a
> CGroup or Docker, with isolated Memory/CPU/Network resources.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
