sodonnel commented on a change in pull request #668: HDDS-3139 Pipeline placement should max out pipeline usage URL: https://github.com/apache/hadoop-ozone/pull/668#discussion_r395613532
########## File path: hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/PipelinePlacementPolicy.java ########## @@ -315,6 +314,50 @@ DatanodeDetails fallBackPickNodes( return results; } + private DatanodeDetails randomPick(List<DatanodeDetails> healthyNodes) { + DatanodeDetails datanodeDetails; + int firstNodeNdx = getRand().nextInt(healthyNodes.size()); + int secondNodeNdx = getRand().nextInt(healthyNodes.size()); + + // There is a possibility that both numbers will be same. + // if that is so, we just return the node. + if (firstNodeNdx == secondNodeNdx) { + datanodeDetails = healthyNodes.get(firstNodeNdx); + } else { + DatanodeDetails firstNodeDetails = healthyNodes.get(firstNodeNdx); + DatanodeDetails secondNodeDetails = healthyNodes.get(secondNodeNdx); + datanodeDetails = nodeManager.getPipelinesCount(firstNodeDetails) + >= nodeManager.getPipelinesCount(secondNodeDetails) + ? secondNodeDetails : firstNodeDetails; + } + return datanodeDetails; + } + + private List<DatanodeDetails> getLowerLoadNodes( + List<DatanodeDetails> nodes, int num) { + int maxPipelineUsage = nodes.size() * heavyNodeCriteria / + HddsProtos.ReplicationFactor.THREE.getNumber(); + return nodes.stream() + // Skip the nodes which exceeds the load limit. + .filter(p -> nodeManager.getPipelinesCount(p) < num - maxPipelineUsage) Review comment: I think there is a bug here. I put some debug in and ran the test you added as part of this change, and this method always returned an empty list. maxPipelineUsage starts at 13, so we have: ``` "0 < 3 - 13" -> "0 < -10 "-> false and all the nodes are filtered out. Should this be: ``` .filter(p -> nodeManager.getPipelinesCount(p) < maxPipelineUsage - num) ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org