[GitHub] storm pull request: [STORM-1165] - normalize the scales of CPU/Mem...
Github user asfgit closed the pull request at: https://github.com/apache/storm/pull/856 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request: [STORM-1165] - normalize the scales of CPU/Mem...
Github user redsanket commented on a diff in the pull request:
https://github.com/apache/storm/pull/856#discussion_r43921033
--- Diff:
storm-core/src/jvm/backtype/storm/scheduler/resource/strategies/ResourceAwareStrategy.java
---
@@ -209,18 +209,18 @@ private WorkerSlot getBestWorker(ExecutorDetails
exec, TopologyDetails td, Strin
for (RAS_Node n : nodes) {
if(n.getFreeSlots().size()>0) {
if (n.getAvailableMemoryResources() >= taskMem
- && n.getAvailableCpuResources() >= taskCPU) {
- double a = Math.pow((taskCPU -
n.getAvailableCpuResources())
- * this.CPU_WEIGHT, 2);
- double b = Math.pow((taskMem -
n.getAvailableMemoryResources())
- * this.MEM_WEIGHT, 2);
- double c = 0.0;
- if(this.refNode != null) {
- c = Math.pow(this.distToNode(this.refNode, n)
- * this.NETWORK_WEIGHT, 2);
- }
- double distance = Math.sqrt(a + b + c);
- nodeRankMap.put(distance, n);
+&& n.getAvailableCpuResources() >= taskCPU) {
+double a = Math.pow(((taskCPU -
n.getAvailableCpuResources())/(n.getAvailableCpuResources() + 1))
+* this.CPU_WEIGHT, 2);
+double b = Math.pow(((taskMem -
n.getAvailableMemoryResources())/(n.getAvailableMemoryResources() + 1))
+* this.MEM_WEIGHT, 2);
+double c = 0.0;
+if(this.refNode != null) {
+c = Math.pow(this.distToNode(this.refNode, n)
+* this.NETWORK_WEIGHT, 2);
+}
+double distance = Math.sqrt(a + b + c);
+nodeRankMap.put(distance, n);
--- End diff --
indentation?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: [STORM-1165] - normalize the scales of CPU/Mem...
Github user redsanket commented on a diff in the pull request:
https://github.com/apache/storm/pull/856#discussion_r43921298
--- Diff:
storm-core/src/jvm/backtype/storm/scheduler/resource/strategies/ResourceAwareStrategy.java
---
@@ -209,18 +209,18 @@ private WorkerSlot getBestWorker(ExecutorDetails
exec, TopologyDetails td, Strin
for (RAS_Node n : nodes) {
if(n.getFreeSlots().size()>0) {
if (n.getAvailableMemoryResources() >= taskMem
- && n.getAvailableCpuResources() >= taskCPU) {
- double a = Math.pow((taskCPU -
n.getAvailableCpuResources())
- * this.CPU_WEIGHT, 2);
- double b = Math.pow((taskMem -
n.getAvailableMemoryResources())
- * this.MEM_WEIGHT, 2);
- double c = 0.0;
- if(this.refNode != null) {
- c = Math.pow(this.distToNode(this.refNode, n)
- * this.NETWORK_WEIGHT, 2);
- }
- double distance = Math.sqrt(a + b + c);
- nodeRankMap.put(distance, n);
+&& n.getAvailableCpuResources() >= taskCPU) {
+double a = Math.pow(((taskCPU -
n.getAvailableCpuResources())/(n.getAvailableCpuResources() + 1))
+* this.CPU_WEIGHT, 2);
+double b = Math.pow(((taskMem -
n.getAvailableMemoryResources())/(n.getAvailableMemoryResources() + 1))
+* this.MEM_WEIGHT, 2);
+double c = 0.0;
+if(this.refNode != null) {
+c = Math.pow(this.distToNode(this.refNode, n)
+* this.NETWORK_WEIGHT, 2);
+}
+double distance = Math.sqrt(a + b + c);
+nodeRankMap.put(distance, n);
--- End diff --
sorry never mind +1 NB
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: [STORM-1165] - normalize the scales of CPU/Mem...
Github user jerrypeng commented on the pull request: https://github.com/apache/storm/pull/856#issuecomment-153797453 @zhuoliu thanks for your thorough review! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request: [STORM-1165] - normalize the scales of CPU/Mem...
Github user zhuoliu commented on the pull request: https://github.com/apache/storm/pull/856#issuecomment-153797173 Looks good. +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request: [STORM-1165] - normalize the scales of CPU/Mem...
Github user jerrypeng commented on a diff in the pull request:
https://github.com/apache/storm/pull/856#discussion_r43907913
--- Diff:
storm-core/src/jvm/backtype/storm/scheduler/resource/strategies/ResourceAwareStrategy.java
---
@@ -209,18 +209,18 @@ private WorkerSlot getBestWorker(ExecutorDetails
exec, TopologyDetails td, Strin
for (RAS_Node n : nodes) {
if(n.getFreeSlots().size()>0) {
if (n.getAvailableMemoryResources() >= taskMem
- && n.getAvailableCpuResources() >= taskCPU) {
- double a = Math.pow((taskCPU -
n.getAvailableCpuResources())
- * this.CPU_WEIGHT, 2);
- double b = Math.pow((taskMem -
n.getAvailableMemoryResources())
- * this.MEM_WEIGHT, 2);
- double c = 0.0;
- if(this.refNode != null) {
- c = Math.pow(this.distToNode(this.refNode, n)
- * this.NETWORK_WEIGHT, 2);
- }
- double distance = Math.sqrt(a + b + c);
- nodeRankMap.put(distance, n);
+&& n.getAvailableCpuResources() >= taskCPU) {
+double a = Math.pow(((taskCPU -
n.getAvailableCpuResources())/n.getAvailableCpuResources() + 1)
+* this.CPU_WEIGHT, 2);
+double b = Math.pow(((taskMem -
n.getAvailableMemoryResources())/n.getAvailableMemoryResources() + 1)
--- End diff --
Ya you are right forgot the parentheses! good catch!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: [STORM-1165] - normalize the scales of CPU/Mem...
Github user zhuoliu commented on a diff in the pull request:
https://github.com/apache/storm/pull/856#discussion_r43907626
--- Diff:
storm-core/src/jvm/backtype/storm/scheduler/resource/strategies/ResourceAwareStrategy.java
---
@@ -209,18 +209,18 @@ private WorkerSlot getBestWorker(ExecutorDetails
exec, TopologyDetails td, Strin
for (RAS_Node n : nodes) {
if(n.getFreeSlots().size()>0) {
if (n.getAvailableMemoryResources() >= taskMem
- && n.getAvailableCpuResources() >= taskCPU) {
- double a = Math.pow((taskCPU -
n.getAvailableCpuResources())
- * this.CPU_WEIGHT, 2);
- double b = Math.pow((taskMem -
n.getAvailableMemoryResources())
- * this.MEM_WEIGHT, 2);
- double c = 0.0;
- if(this.refNode != null) {
- c = Math.pow(this.distToNode(this.refNode, n)
- * this.NETWORK_WEIGHT, 2);
- }
- double distance = Math.sqrt(a + b + c);
- nodeRankMap.put(distance, n);
+&& n.getAvailableCpuResources() >= taskCPU) {
+double a = Math.pow(((taskCPU -
n.getAvailableCpuResources())/n.getAvailableCpuResources() + 1)
+* this.CPU_WEIGHT, 2);
+double b = Math.pow(((taskMem -
n.getAvailableMemoryResources())/n.getAvailableMemoryResources() + 1)
--- End diff --
Should this be (n.getAvailableMemoryResources()+1) to avoid "divide by 0"?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] storm pull request: [STORM-1165] - normalize the scales of CPU/Mem...
Github user revans2 commented on the pull request: https://github.com/apache/storm/pull/856#issuecomment-153786537 +1 looks good. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] storm pull request: [STORM-1165] - normalize the scales of CPU/Mem...
GitHub user jerrypeng opened a pull request: https://github.com/apache/storm/pull/856 [STORM-1165] - normalize the scales of CPU/Mem/Net when choosing the best node for Resource Aware Scheduler You can merge this pull request into a Git repository by running: $ git pull https://github.com/jerrypeng/storm STORM-1165 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/storm/pull/856.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #856 commit 780a472ad2dc4e838b86f65f0c40eb9443046038 Author: Boyang Jerry Peng Date: 2015-11-03T23:13:08Z [YSTORM-1433] - normalize the scales of CPU/Mem/Net when choosing the best node commit e638d9a20fac44844f59ee911d3d15bc726d0f5d Author: Boyang Jerry Peng Date: 2015-11-04T15:57:59Z add a +1 in the denominator to deal with the case if available resources is zero --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
