[jira] [Commented] (HDFS-9044) Give Priority to FavouredNodes , before selecting nodes from FavouredNode's Node Group
[ https://issues.apache.org/jira/browse/HDFS-9044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980438#comment-14980438 ] Hudson commented on HDFS-9044: -- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #550 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/550/]) HDFS-9044. Give Priority to FavouredNodes , before selecting nodes from (vinayakumarb: rev 588baab160e7c328dca1c45cf3541e79218406e8) * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithNodeGroup.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithNodeGroup.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java > Give Priority to FavouredNodes , before selecting nodes from FavouredNode's > Node Group > -- > > Key: HDFS-9044 > URL: https://issues.apache.org/jira/browse/HDFS-9044 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: J.Andreina >Assignee: J.Andreina > Fix For: 2.8.0 > > Attachments: HDFS-9044.1.patch, HDFS-9044.2.patch, HDFS-9044.3.patch, > HDFS-9044.4.patch, HDFS-9044.5.patch, HDFS-9044.6.patch > > > Passing Favored nodes intention is to place replica among the favored node > Current behavior in Node group is > If favored node is not available it goes to one among favored > nodegroup. > {noformat} > Say for example: > 1)I need 3 replicas and passed 5 favored nodes. > 2)Out of 5 favored nodes 3 favored nodes are not good. > 3)Then based on BlockPlacementPolicyWithNodeGroup out of 5 targets node > returned , 3 will be random node from 3 bad FavoredNode's nodegroup. > 4)Then there is a probability that all my 3 replicas are placed on > Random node from FavoredNodes's nodegroup , instead of giving priority to 2 > favored nodes returned as target. > {noformat} > *Instead of returning 5 targets on 3rd step above , we can return 2 good > favored nodes as target* > *And remaining 1 needed replica can be chosen from Random node of bad > FavoredNodes's nodegroup.* > This will make sure that the FavoredNodes are given priority. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9044) Give Priority to FavouredNodes , before selecting nodes from FavouredNode's Node Group
[ https://issues.apache.org/jira/browse/HDFS-9044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14979986#comment-14979986 ] Hudson commented on HDFS-9044: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #599 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/599/]) HDFS-9044. Give Priority to FavouredNodes , before selecting nodes from (vinayakumarb: rev 588baab160e7c328dca1c45cf3541e79218406e8) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithNodeGroup.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithNodeGroup.java > Give Priority to FavouredNodes , before selecting nodes from FavouredNode's > Node Group > -- > > Key: HDFS-9044 > URL: https://issues.apache.org/jira/browse/HDFS-9044 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: J.Andreina >Assignee: J.Andreina > Fix For: 2.8.0 > > Attachments: HDFS-9044.1.patch, HDFS-9044.2.patch, HDFS-9044.3.patch, > HDFS-9044.4.patch, HDFS-9044.5.patch, HDFS-9044.6.patch > > > Passing Favored nodes intention is to place replica among the favored node > Current behavior in Node group is > If favored node is not available it goes to one among favored > nodegroup. > {noformat} > Say for example: > 1)I need 3 replicas and passed 5 favored nodes. > 2)Out of 5 favored nodes 3 favored nodes are not good. > 3)Then based on BlockPlacementPolicyWithNodeGroup out of 5 targets node > returned , 3 will be random node from 3 bad FavoredNode's nodegroup. > 4)Then there is a probability that all my 3 replicas are placed on > Random node from FavoredNodes's nodegroup , instead of giving priority to 2 > favored nodes returned as target. > {noformat} > *Instead of returning 5 targets on 3rd step above , we can return 2 good > favored nodes as target* > *And remaining 1 needed replica can be chosen from Random node of bad > FavoredNodes's nodegroup.* > This will make sure that the FavoredNodes are given priority. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9044) Give Priority to FavouredNodes , before selecting nodes from FavouredNode's Node Group
[ https://issues.apache.org/jira/browse/HDFS-9044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980010#comment-14980010 ] Hudson commented on HDFS-9044: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #2542 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2542/]) HDFS-9044. Give Priority to FavouredNodes , before selecting nodes from (vinayakumarb: rev 588baab160e7c328dca1c45cf3541e79218406e8) * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithNodeGroup.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithNodeGroup.java > Give Priority to FavouredNodes , before selecting nodes from FavouredNode's > Node Group > -- > > Key: HDFS-9044 > URL: https://issues.apache.org/jira/browse/HDFS-9044 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: J.Andreina >Assignee: J.Andreina > Fix For: 2.8.0 > > Attachments: HDFS-9044.1.patch, HDFS-9044.2.patch, HDFS-9044.3.patch, > HDFS-9044.4.patch, HDFS-9044.5.patch, HDFS-9044.6.patch > > > Passing Favored nodes intention is to place replica among the favored node > Current behavior in Node group is > If favored node is not available it goes to one among favored > nodegroup. > {noformat} > Say for example: > 1)I need 3 replicas and passed 5 favored nodes. > 2)Out of 5 favored nodes 3 favored nodes are not good. > 3)Then based on BlockPlacementPolicyWithNodeGroup out of 5 targets node > returned , 3 will be random node from 3 bad FavoredNode's nodegroup. > 4)Then there is a probability that all my 3 replicas are placed on > Random node from FavoredNodes's nodegroup , instead of giving priority to 2 > favored nodes returned as target. > {noformat} > *Instead of returning 5 targets on 3rd step above , we can return 2 good > favored nodes as target* > *And remaining 1 needed replica can be chosen from Random node of bad > FavoredNodes's nodegroup.* > This will make sure that the FavoredNodes are given priority. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9044) Give Priority to FavouredNodes , before selecting nodes from FavouredNode's Node Group
[ https://issues.apache.org/jira/browse/HDFS-9044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14979893#comment-14979893 ] Vinayakumar B commented on HDFS-9044: - +1 for latest patch. Will commit shortly > Give Priority to FavouredNodes , before selecting nodes from FavouredNode's > Node Group > -- > > Key: HDFS-9044 > URL: https://issues.apache.org/jira/browse/HDFS-9044 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: J.Andreina >Assignee: J.Andreina > Attachments: HDFS-9044.1.patch, HDFS-9044.2.patch, HDFS-9044.3.patch, > HDFS-9044.4.patch, HDFS-9044.5.patch, HDFS-9044.6.patch > > > Passing Favored nodes intention is to place replica among the favored node > Current behavior in Node group is > If favored node is not available it goes to one among favored > nodegroup. > {noformat} > Say for example: > 1)I need 3 replicas and passed 5 favored nodes. > 2)Out of 5 favored nodes 3 favored nodes are not good. > 3)Then based on BlockPlacementPolicyWithNodeGroup out of 5 targets node > returned , 3 will be random node from 3 bad FavoredNode's nodegroup. > 4)Then there is a probability that all my 3 replicas are placed on > Random node from FavoredNodes's nodegroup , instead of giving priority to 2 > favored nodes returned as target. > {noformat} > *Instead of returning 5 targets on 3rd step above , we can return 2 good > favored nodes as target* > *And remaining 1 needed replica can be chosen from Random node of bad > FavoredNodes's nodegroup.* > This will make sure that the FavoredNodes are given priority. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9044) Give Priority to FavouredNodes , before selecting nodes from FavouredNode's Node Group
[ https://issues.apache.org/jira/browse/HDFS-9044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14979961#comment-14979961 ] Hudson commented on HDFS-9044: -- FAILURE: Integrated in Hadoop-trunk-Commit #8723 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/8723/]) HDFS-9044. Give Priority to FavouredNodes , before selecting nodes from (vinayakumarb: rev 588baab160e7c328dca1c45cf3541e79218406e8) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithNodeGroup.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithNodeGroup.java > Give Priority to FavouredNodes , before selecting nodes from FavouredNode's > Node Group > -- > > Key: HDFS-9044 > URL: https://issues.apache.org/jira/browse/HDFS-9044 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: J.Andreina >Assignee: J.Andreina > Fix For: 2.8.0 > > Attachments: HDFS-9044.1.patch, HDFS-9044.2.patch, HDFS-9044.3.patch, > HDFS-9044.4.patch, HDFS-9044.5.patch, HDFS-9044.6.patch > > > Passing Favored nodes intention is to place replica among the favored node > Current behavior in Node group is > If favored node is not available it goes to one among favored > nodegroup. > {noformat} > Say for example: > 1)I need 3 replicas and passed 5 favored nodes. > 2)Out of 5 favored nodes 3 favored nodes are not good. > 3)Then based on BlockPlacementPolicyWithNodeGroup out of 5 targets node > returned , 3 will be random node from 3 bad FavoredNode's nodegroup. > 4)Then there is a probability that all my 3 replicas are placed on > Random node from FavoredNodes's nodegroup , instead of giving priority to 2 > favored nodes returned as target. > {noformat} > *Instead of returning 5 targets on 3rd step above , we can return 2 good > favored nodes as target* > *And remaining 1 needed replica can be chosen from Random node of bad > FavoredNodes's nodegroup.* > This will make sure that the FavoredNodes are given priority. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9044) Give Priority to FavouredNodes , before selecting nodes from FavouredNode's Node Group
[ https://issues.apache.org/jira/browse/HDFS-9044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980152#comment-14980152 ] Hudson commented on HDFS-9044: -- FAILURE: Integrated in Hadoop-Yarn-trunk #1335 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/1335/]) HDFS-9044. Give Priority to FavouredNodes , before selecting nodes from (vinayakumarb: rev 588baab160e7c328dca1c45cf3541e79218406e8) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithNodeGroup.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithNodeGroup.java > Give Priority to FavouredNodes , before selecting nodes from FavouredNode's > Node Group > -- > > Key: HDFS-9044 > URL: https://issues.apache.org/jira/browse/HDFS-9044 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: J.Andreina >Assignee: J.Andreina > Fix For: 2.8.0 > > Attachments: HDFS-9044.1.patch, HDFS-9044.2.patch, HDFS-9044.3.patch, > HDFS-9044.4.patch, HDFS-9044.5.patch, HDFS-9044.6.patch > > > Passing Favored nodes intention is to place replica among the favored node > Current behavior in Node group is > If favored node is not available it goes to one among favored > nodegroup. > {noformat} > Say for example: > 1)I need 3 replicas and passed 5 favored nodes. > 2)Out of 5 favored nodes 3 favored nodes are not good. > 3)Then based on BlockPlacementPolicyWithNodeGroup out of 5 targets node > returned , 3 will be random node from 3 bad FavoredNode's nodegroup. > 4)Then there is a probability that all my 3 replicas are placed on > Random node from FavoredNodes's nodegroup , instead of giving priority to 2 > favored nodes returned as target. > {noformat} > *Instead of returning 5 targets on 3rd step above , we can return 2 good > favored nodes as target* > *And remaining 1 needed replica can be chosen from Random node of bad > FavoredNodes's nodegroup.* > This will make sure that the FavoredNodes are given priority. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9044) Give Priority to FavouredNodes , before selecting nodes from FavouredNode's Node Group
[ https://issues.apache.org/jira/browse/HDFS-9044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980209#comment-14980209 ] Hudson commented on HDFS-9044: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #2488 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2488/]) HDFS-9044. Give Priority to FavouredNodes , before selecting nodes from (vinayakumarb: rev 588baab160e7c328dca1c45cf3541e79218406e8) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithNodeGroup.java * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithNodeGroup.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt > Give Priority to FavouredNodes , before selecting nodes from FavouredNode's > Node Group > -- > > Key: HDFS-9044 > URL: https://issues.apache.org/jira/browse/HDFS-9044 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: J.Andreina >Assignee: J.Andreina > Fix For: 2.8.0 > > Attachments: HDFS-9044.1.patch, HDFS-9044.2.patch, HDFS-9044.3.patch, > HDFS-9044.4.patch, HDFS-9044.5.patch, HDFS-9044.6.patch > > > Passing Favored nodes intention is to place replica among the favored node > Current behavior in Node group is > If favored node is not available it goes to one among favored > nodegroup. > {noformat} > Say for example: > 1)I need 3 replicas and passed 5 favored nodes. > 2)Out of 5 favored nodes 3 favored nodes are not good. > 3)Then based on BlockPlacementPolicyWithNodeGroup out of 5 targets node > returned , 3 will be random node from 3 bad FavoredNode's nodegroup. > 4)Then there is a probability that all my 3 replicas are placed on > Random node from FavoredNodes's nodegroup , instead of giving priority to 2 > favored nodes returned as target. > {noformat} > *Instead of returning 5 targets on 3rd step above , we can return 2 good > favored nodes as target* > *And remaining 1 needed replica can be chosen from Random node of bad > FavoredNodes's nodegroup.* > This will make sure that the FavoredNodes are given priority. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9044) Give Priority to FavouredNodes , before selecting nodes from FavouredNode's Node Group
[ https://issues.apache.org/jira/browse/HDFS-9044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980160#comment-14980160 ] Hudson commented on HDFS-9044: -- FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #612 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/612/]) HDFS-9044. Give Priority to FavouredNodes , before selecting nodes from (vinayakumarb: rev 588baab160e7c328dca1c45cf3541e79218406e8) * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java * hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestReplicationPolicyWithNodeGroup.java * hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt * hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyWithNodeGroup.java > Give Priority to FavouredNodes , before selecting nodes from FavouredNode's > Node Group > -- > > Key: HDFS-9044 > URL: https://issues.apache.org/jira/browse/HDFS-9044 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: J.Andreina >Assignee: J.Andreina > Fix For: 2.8.0 > > Attachments: HDFS-9044.1.patch, HDFS-9044.2.patch, HDFS-9044.3.patch, > HDFS-9044.4.patch, HDFS-9044.5.patch, HDFS-9044.6.patch > > > Passing Favored nodes intention is to place replica among the favored node > Current behavior in Node group is > If favored node is not available it goes to one among favored > nodegroup. > {noformat} > Say for example: > 1)I need 3 replicas and passed 5 favored nodes. > 2)Out of 5 favored nodes 3 favored nodes are not good. > 3)Then based on BlockPlacementPolicyWithNodeGroup out of 5 targets node > returned , 3 will be random node from 3 bad FavoredNode's nodegroup. > 4)Then there is a probability that all my 3 replicas are placed on > Random node from FavoredNodes's nodegroup , instead of giving priority to 2 > favored nodes returned as target. > {noformat} > *Instead of returning 5 targets on 3rd step above , we can return 2 good > favored nodes as target* > *And remaining 1 needed replica can be chosen from Random node of bad > FavoredNodes's nodegroup.* > This will make sure that the FavoredNodes are given priority. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9044) Give Priority to FavouredNodes , before selecting nodes from FavouredNode's Node Group
[ https://issues.apache.org/jira/browse/HDFS-9044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14977941#comment-14977941 ] Vinayakumar B commented on HDFS-9044: - Patch looks good. checkstyle comments can be cleared by formatting below line {code}super.chooseFavouredNodes(src, numOfReplicas, favoredNodes, favoriteAndExcludedNodes, blocksize, maxNodesPerRack, results, avoidStaleNodes, storageTypes);{code} Whitespaces line endigs also could be removed. +1 once addresed. > Give Priority to FavouredNodes , before selecting nodes from FavouredNode's > Node Group > -- > > Key: HDFS-9044 > URL: https://issues.apache.org/jira/browse/HDFS-9044 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: J.Andreina >Assignee: J.Andreina > Attachments: HDFS-9044.1.patch, HDFS-9044.2.patch, HDFS-9044.3.patch, > HDFS-9044.4.patch, HDFS-9044.5.patch > > > Passing Favored nodes intention is to place replica among the favored node > Current behavior in Node group is > If favored node is not available it goes to one among favored > nodegroup. > {noformat} > Say for example: > 1)I need 3 replicas and passed 5 favored nodes. > 2)Out of 5 favored nodes 3 favored nodes are not good. > 3)Then based on BlockPlacementPolicyWithNodeGroup out of 5 targets node > returned , 3 will be random node from 3 bad FavoredNode's nodegroup. > 4)Then there is a probability that all my 3 replicas are placed on > Random node from FavoredNodes's nodegroup , instead of giving priority to 2 > favored nodes returned as target. > {noformat} > *Instead of returning 5 targets on 3rd step above , we can return 2 good > favored nodes as target* > *And remaining 1 needed replica can be chosen from Random node of bad > FavoredNodes's nodegroup.* > This will make sure that the FavoredNodes are given priority. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9044) Give Priority to FavouredNodes , before selecting nodes from FavouredNode's Node Group
[ https://issues.apache.org/jira/browse/HDFS-9044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14978094#comment-14978094 ] Vinayakumar B commented on HDFS-9044: - +1, pending jenkins > Give Priority to FavouredNodes , before selecting nodes from FavouredNode's > Node Group > -- > > Key: HDFS-9044 > URL: https://issues.apache.org/jira/browse/HDFS-9044 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: J.Andreina >Assignee: J.Andreina > Attachments: HDFS-9044.1.patch, HDFS-9044.2.patch, HDFS-9044.3.patch, > HDFS-9044.4.patch, HDFS-9044.5.patch, HDFS-9044.6.patch > > > Passing Favored nodes intention is to place replica among the favored node > Current behavior in Node group is > If favored node is not available it goes to one among favored > nodegroup. > {noformat} > Say for example: > 1)I need 3 replicas and passed 5 favored nodes. > 2)Out of 5 favored nodes 3 favored nodes are not good. > 3)Then based on BlockPlacementPolicyWithNodeGroup out of 5 targets node > returned , 3 will be random node from 3 bad FavoredNode's nodegroup. > 4)Then there is a probability that all my 3 replicas are placed on > Random node from FavoredNodes's nodegroup , instead of giving priority to 2 > favored nodes returned as target. > {noformat} > *Instead of returning 5 targets on 3rd step above , we can return 2 good > favored nodes as target* > *And remaining 1 needed replica can be chosen from Random node of bad > FavoredNodes's nodegroup.* > This will make sure that the FavoredNodes are given priority. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9044) Give Priority to FavouredNodes , before selecting nodes from FavouredNode's Node Group
[ https://issues.apache.org/jira/browse/HDFS-9044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14979726#comment-14979726 ] Hadoop QA commented on HDFS-9044: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 9s {color} | {color:blue} docker + precommit patch detected. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color} | {color:green} The patch appears to include 1 new or modified test files. {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m 57s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 38s {color} | {color:green} trunk passed with JDK v1.8.0_60 {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 41s {color} | {color:green} trunk passed with JDK v1.7.0_79 {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 18s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 16s {color} | {color:green} trunk passed {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m 4s {color} | {color:red} hadoop-hdfs-project/hadoop-hdfs in trunk cannot run convertXmlToText from findbugs {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 13s {color} | {color:green} trunk passed with JDK v1.8.0_60 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 57s {color} | {color:green} trunk passed with JDK v1.7.0_79 {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 40s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 40s {color} | {color:green} the patch passed with JDK v1.8.0_60 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 40s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s {color} | {color:green} the patch passed with JDK v1.7.0_79 {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 34s {color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 18s {color} | {color:red} Patch generated 2 new checkstyle issues in hadoop-hdfs-project/hadoop-hdfs (total was 51, now 52). {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 14s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color} | {color:green} Patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 13s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 14s {color} | {color:green} the patch passed with JDK v1.8.0_60 {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 58s {color} | {color:green} the patch passed with JDK v1.7.0_79 {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 56m 31s {color} | {color:red} hadoop-hdfs in the patch failed with JDK v1.8.0_60. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 49m 27s {color} | {color:red} hadoop-hdfs in the patch failed with JDK v1.7.0_79. {color} | | {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 20s {color} | {color:red} Patch generated 58 ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 128m 11s {color} | {color:black} {color} | \\ \\ || Reason || Tests || | JDK v1.7.0_79 Failed junit tests | hadoop.hdfs.server.blockmanagement.TestNodeCount | | | hadoop.hdfs.server.datanode.TestBlockScanner | | | hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot | | | hadoop.hdfs.TestRecoverStripedFile | \\ \\ || Subsystem || Report/Notes || | Docker | Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-10-29 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12769230/HDFS-9044.6.patch | | JIRA Issue | HDFS-9044 | | Optional Tests | asflicense javac javadoc mvninstall unit findbugs checkstyle compile | | uname | Linux 322d50b7733c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality |
[jira] [Commented] (HDFS-9044) Give Priority to FavouredNodes , before selecting nodes from FavouredNode's Node Group
[ https://issues.apache.org/jira/browse/HDFS-9044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14979772#comment-14979772 ] J.Andreina commented on HDFS-9044: -- Failures are unrelated to this patch. Please review. > Give Priority to FavouredNodes , before selecting nodes from FavouredNode's > Node Group > -- > > Key: HDFS-9044 > URL: https://issues.apache.org/jira/browse/HDFS-9044 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: J.Andreina >Assignee: J.Andreina > Attachments: HDFS-9044.1.patch, HDFS-9044.2.patch, HDFS-9044.3.patch, > HDFS-9044.4.patch, HDFS-9044.5.patch, HDFS-9044.6.patch > > > Passing Favored nodes intention is to place replica among the favored node > Current behavior in Node group is > If favored node is not available it goes to one among favored > nodegroup. > {noformat} > Say for example: > 1)I need 3 replicas and passed 5 favored nodes. > 2)Out of 5 favored nodes 3 favored nodes are not good. > 3)Then based on BlockPlacementPolicyWithNodeGroup out of 5 targets node > returned , 3 will be random node from 3 bad FavoredNode's nodegroup. > 4)Then there is a probability that all my 3 replicas are placed on > Random node from FavoredNodes's nodegroup , instead of giving priority to 2 > favored nodes returned as target. > {noformat} > *Instead of returning 5 targets on 3rd step above , we can return 2 good > favored nodes as target* > *And remaining 1 needed replica can be chosen from Random node of bad > FavoredNodes's nodegroup.* > This will make sure that the FavoredNodes are given priority. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9044) Give Priority to FavouredNodes , before selecting nodes from FavouredNode's Node Group
[ https://issues.apache.org/jira/browse/HDFS-9044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14958440#comment-14958440 ] Hadoop QA commented on HDFS-9044: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:red}-1{color} | pre-patch | 18m 53s | Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 1 new or modified test files. | | {color:green}+1{color} | javac | 7m 57s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 10m 22s | There were no new javadoc warning messages. | | {color:red}-1{color} | release audit | 0m 20s | The applied patch generated 1 release audit warnings. | | {color:red}-1{color} | checkstyle | 1m 29s | The applied patch generated 4 new checkstyle issues (total was 49, now 52). | | {color:red}-1{color} | whitespace | 0m 0s | The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. | | {color:green}+1{color} | install | 1m 34s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 35s | The patch built with eclipse:eclipse. | | {color:green}+1{color} | findbugs | 2m 28s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. | | {color:green}+1{color} | native | 3m 12s | Pre-build of native portion | | {color:red}-1{color} | hdfs tests | 50m 18s | Tests failed in hadoop-hdfs. | | | | 97m 11s | | \\ \\ || Reason || Tests || | Failed unit tests | hadoop.hdfs.TestDFSClientRetries | | | hadoop.hdfs.TestLeaseRecovery2 | | | hadoop.fs.TestGlobPaths | | Timed out tests | org.apache.hadoop.hdfs.tools.TestDFSAdmin | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12766714/HDFS-9044.5.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | trunk / c80b3a8 | | Pre-patch Findbugs warnings | https://builds.apache.org/job/PreCommit-HDFS-Build/12999/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html | | Release Audit | https://builds.apache.org/job/PreCommit-HDFS-Build/12999/artifact/patchprocess/patchReleaseAuditProblems.txt | | checkstyle | https://builds.apache.org/job/PreCommit-HDFS-Build/12999/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt | | whitespace | https://builds.apache.org/job/PreCommit-HDFS-Build/12999/artifact/patchprocess/whitespace.txt | | hadoop-hdfs test log | https://builds.apache.org/job/PreCommit-HDFS-Build/12999/artifact/patchprocess/testrun_hadoop-hdfs.txt | | Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/12999/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf902.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/12999/console | This message was automatically generated. > Give Priority to FavouredNodes , before selecting nodes from FavouredNode's > Node Group > -- > > Key: HDFS-9044 > URL: https://issues.apache.org/jira/browse/HDFS-9044 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: J.Andreina >Assignee: J.Andreina > Attachments: HDFS-9044.1.patch, HDFS-9044.2.patch, HDFS-9044.3.patch, > HDFS-9044.4.patch, HDFS-9044.5.patch > > > Passing Favored nodes intention is to place replica among the favored node > Current behavior in Node group is > If favored node is not available it goes to one among favored > nodegroup. > {noformat} > Say for example: > 1)I need 3 replicas and passed 5 favored nodes. > 2)Out of 5 favored nodes 3 favored nodes are not good. > 3)Then based on BlockPlacementPolicyWithNodeGroup out of 5 targets node > returned , 3 will be random node from 3 bad FavoredNode's nodegroup. > 4)Then there is a probability that all my 3 replicas are placed on > Random node from FavoredNodes's nodegroup , instead of giving priority to 2 > favored nodes returned as target. > {noformat} > *Instead of returning 5 targets on 3rd step above , we can return 2 good > favored nodes as target* > *And remaining 1 needed replica can be chosen from Random node of bad > FavoredNodes's nodegroup.* > This will make sure that the FavoredNodes are given priority. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9044) Give Priority to FavouredNodes , before selecting nodes from FavouredNode's Node Group
[ https://issues.apache.org/jira/browse/HDFS-9044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14954827#comment-14954827 ] Vinayakumar B commented on HDFS-9044: - Hi [~andreina], thanks for the patch, Patch looks almost good. Here are few minor comments, 1. Below lines from {{BlockPlacementPolicyWithNodeGroup#chooseFavouredNodes(..)}} could be just replaced with {{super.chooseFavouredNodes(..)}} {code}+for (int i = 0; i < favoredNodes.size() && results.size() < numOfReplicas; +i++) { + DatanodeDescriptor favoredNode = favoredNodes.get(i); + // Choose a single node which is local to favoredNode. + // 'results' is updated within chooseLocalNode + DatanodeStorageInfo target = null; + try { +target = +chooseLocalStorage(favoredNode, favoriteAndExcludedNodes, + blocksize, maxNodesPerRack, results, avoidStaleNodes, + storageTypes, false); + } catch (NotEnoughReplicasException e) { +// catch Exception and continue with other favored nodes +continue; + } + if (target == null) { +LOG.warn("Could not find a target for file " + src ++ " with favored node " + favoredNode); +continue; + } + favoriteAndExcludedNodes.add(target.getDatanodeDescriptor()); +}{code} 2. {{checkFavoredNodePartOfTarget(..)}} could be renamed to {{isNodeChosen(..)}} 3. {{clusterMapNodeGroup.getNodeGroup(favoredNode.getNetworkLocation())}} can be extracted to a variable {{scope}} for better readability 4. I think its better to update javadoc for {{BlockPlacementPolicyWithNodeGroup#chooseLocalStorage(..)}} by mentioning, fallback to nodegroup/rack will happen if flag {{fallbackToNodeGroupAndLocalRack}} is set. > Give Priority to FavouredNodes , before selecting nodes from FavouredNode's > Node Group > -- > > Key: HDFS-9044 > URL: https://issues.apache.org/jira/browse/HDFS-9044 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: J.Andreina >Assignee: J.Andreina > Attachments: HDFS-9044.1.patch, HDFS-9044.2.patch, HDFS-9044.3.patch, > HDFS-9044.4.patch > > > Passing Favored nodes intention is to place replica among the favored node > Current behavior in Node group is > If favored node is not available it goes to one among favored > nodegroup. > {noformat} > Say for example: > 1)I need 3 replicas and passed 5 favored nodes. > 2)Out of 5 favored nodes 3 favored nodes are not good. > 3)Then based on BlockPlacementPolicyWithNodeGroup out of 5 targets node > returned , 3 will be random node from 3 bad FavoredNode's nodegroup. > 4)Then there is a probability that all my 3 replicas are placed on > Random node from FavoredNodes's nodegroup , instead of giving priority to 2 > favored nodes returned as target. > {noformat} > *Instead of returning 5 targets on 3rd step above , we can return 2 good > favored nodes as target* > *And remaining 1 needed replica can be chosen from Random node of bad > FavoredNodes's nodegroup.* > This will make sure that the FavoredNodes are given priority. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9044) Give Priority to FavouredNodes , before selecting nodes from FavouredNode's Node Group
[ https://issues.apache.org/jira/browse/HDFS-9044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14908123#comment-14908123 ] J.Andreina commented on HDFS-9044: -- [~vinayrpet] and [~szetszwo] , can you please have a look at this jira and provide your feedback > Give Priority to FavouredNodes , before selecting nodes from FavouredNode's > Node Group > -- > > Key: HDFS-9044 > URL: https://issues.apache.org/jira/browse/HDFS-9044 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: J.Andreina >Assignee: J.Andreina > Attachments: HDFS-9044.1.patch, HDFS-9044.2.patch, HDFS-9044.3.patch, > HDFS-9044.4.patch > > > Passing Favored nodes intention is to place replica among the favored node > Current behavior in Node group is > If favored node is not available it goes to one among favored > nodegroup. > {noformat} > Say for example: > 1)I need 3 replicas and passed 5 favored nodes. > 2)Out of 5 favored nodes 3 favored nodes are not good. > 3)Then based on BlockPlacementPolicyWithNodeGroup out of 5 targets node > returned , 3 will be random node from 3 bad FavoredNode's nodegroup. > 4)Then there is a probability that all my 3 replicas are placed on > Random node from FavoredNodes's nodegroup , instead of giving priority to 2 > favored nodes returned as target. > {noformat} > *Instead of returning 5 targets on 3rd step above , we can return 2 good > favored nodes as target* > *And remaining 1 needed replica can be chosen from Random node of bad > FavoredNodes's nodegroup.* > This will make sure that the FavoredNodes are given priority. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9044) Give Priority to FavouredNodes , before selecting nodes from FavouredNode's Node Group
[ https://issues.apache.org/jira/browse/HDFS-9044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14904337#comment-14904337 ] Hadoop QA commented on HDFS-9044: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | pre-patch | 18m 2s | Pre-patch trunk compilation is healthy. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 1 new or modified test files. | | {color:green}+1{color} | javac | 8m 0s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 10m 6s | There were no new javadoc warning messages. | | {color:green}+1{color} | release audit | 0m 24s | The applied patch does not increase the total number of release audit warnings. | | {color:red}-1{color} | checkstyle | 1m 24s | The applied patch generated 2 new checkstyle issues (total was 50, now 51). | | {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that end in whitespace. | | {color:green}+1{color} | install | 1m 32s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with eclipse:eclipse. | | {color:green}+1{color} | findbugs | 2m 33s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. | | {color:green}+1{color} | native | 3m 13s | Pre-build of native portion | | {color:green}+1{color} | hdfs tests | 164m 10s | Tests passed in hadoop-hdfs. | | | | 210m 0s | | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12761822/HDFS-9044.4.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | trunk / 7c5c099 | | checkstyle | https://builds.apache.org/job/PreCommit-HDFS-Build/12618/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt | | hadoop-hdfs test log | https://builds.apache.org/job/PreCommit-HDFS-Build/12618/artifact/patchprocess/testrun_hadoop-hdfs.txt | | Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/12618/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/12618/console | This message was automatically generated. > Give Priority to FavouredNodes , before selecting nodes from FavouredNode's > Node Group > -- > > Key: HDFS-9044 > URL: https://issues.apache.org/jira/browse/HDFS-9044 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: J.Andreina >Assignee: J.Andreina > Attachments: HDFS-9044.1.patch, HDFS-9044.2.patch, HDFS-9044.3.patch, > HDFS-9044.4.patch > > > Passing Favored nodes intention is to place replica among the favored node > Current behavior in Node group is > If favored node is not available it goes to one among favored > nodegroup. > {noformat} > Say for example: > 1)I need 3 replicas and passed 5 favored nodes. > 2)Out of 5 favored nodes 3 favored nodes are not good. > 3)Then based on BlockPlacementPolicyWithNodeGroup out of 5 targets node > returned , 3 will be random node from 3 bad FavoredNode's nodegroup. > 4)Then there is a probability that all my 3 replicas are placed on > Random node from FavoredNodes's nodegroup , instead of giving priority to 2 > favored nodes returned as target. > {noformat} > *Instead of returning 5 targets on 3rd step above , we can return 2 good > favored nodes as target* > *And remaining 1 needed replica can be chosen from Random node of bad > FavoredNodes's nodegroup.* > This will make sure that the FavoredNodes are given priority. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9044) Give Priority to FavouredNodes , before selecting nodes from FavouredNode's Node Group
[ https://issues.apache.org/jira/browse/HDFS-9044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14904366#comment-14904366 ] J.Andreina commented on HDFS-9044: -- Checkstyle comments can be ignored , as all the 9 parameters of the method are been used. Please review. > Give Priority to FavouredNodes , before selecting nodes from FavouredNode's > Node Group > -- > > Key: HDFS-9044 > URL: https://issues.apache.org/jira/browse/HDFS-9044 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: J.Andreina >Assignee: J.Andreina > Attachments: HDFS-9044.1.patch, HDFS-9044.2.patch, HDFS-9044.3.patch, > HDFS-9044.4.patch > > > Passing Favored nodes intention is to place replica among the favored node > Current behavior in Node group is > If favored node is not available it goes to one among favored > nodegroup. > {noformat} > Say for example: > 1)I need 3 replicas and passed 5 favored nodes. > 2)Out of 5 favored nodes 3 favored nodes are not good. > 3)Then based on BlockPlacementPolicyWithNodeGroup out of 5 targets node > returned , 3 will be random node from 3 bad FavoredNode's nodegroup. > 4)Then there is a probability that all my 3 replicas are placed on > Random node from FavoredNodes's nodegroup , instead of giving priority to 2 > favored nodes returned as target. > {noformat} > *Instead of returning 5 targets on 3rd step above , we can return 2 good > favored nodes as target* > *And remaining 1 needed replica can be chosen from Random node of bad > FavoredNodes's nodegroup.* > This will make sure that the FavoredNodes are given priority. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9044) Give Priority to FavouredNodes , before selecting nodes from FavouredNode's Node Group
[ https://issues.apache.org/jira/browse/HDFS-9044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14902436#comment-14902436 ] Hadoop QA commented on HDFS-9044: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | pre-patch | 17m 33s | Pre-patch trunk compilation is healthy. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 1 new or modified test files. | | {color:green}+1{color} | javac | 7m 46s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 10m 1s | There were no new javadoc warning messages. | | {color:green}+1{color} | release audit | 0m 23s | The applied patch does not increase the total number of release audit warnings. | | {color:red}-1{color} | checkstyle | 1m 19s | The applied patch generated 2 new checkstyle issues (total was 50, now 51). | | {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that end in whitespace. | | {color:green}+1{color} | install | 1m 28s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 34s | The patch built with eclipse:eclipse. | | {color:green}+1{color} | findbugs | 2m 26s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. | | {color:green}+1{color} | native | 3m 15s | Pre-build of native portion | | {color:red}-1{color} | hdfs tests | 163m 11s | Tests failed in hadoop-hdfs. | | | | 208m 0s | | \\ \\ || Reason || Tests || | Failed unit tests | hadoop.hdfs.TestDFSClientRetries | | | hadoop.hdfs.server.namenode.TestFSNamesystem | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12761595/HDFS-9044.3.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | trunk / 10ab7d5 | | checkstyle | https://builds.apache.org/job/PreCommit-HDFS-Build/12584/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt | | hadoop-hdfs test log | https://builds.apache.org/job/PreCommit-HDFS-Build/12584/artifact/patchprocess/testrun_hadoop-hdfs.txt | | Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/12584/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf903.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/12584/console | This message was automatically generated. > Give Priority to FavouredNodes , before selecting nodes from FavouredNode's > Node Group > -- > > Key: HDFS-9044 > URL: https://issues.apache.org/jira/browse/HDFS-9044 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: J.Andreina >Assignee: J.Andreina > Attachments: HDFS-9044.1.patch, HDFS-9044.2.patch, HDFS-9044.3.patch > > > Passing Favored nodes intention is to place replica among the favored node > Current behavior in Node group is > If favored node is not available it goes to one among favored > nodegroup. > {noformat} > Say for example: > 1)I need 3 replicas and passed 5 favored nodes. > 2)Out of 5 favored nodes 3 favored nodes are not good. > 3)Then based on BlockPlacementPolicyWithNodeGroup out of 5 targets node > returned , 3 will be random node from 3 bad FavoredNode's nodegroup. > 4)Then there is a probability that all my 3 replicas are placed on > Random node from FavoredNodes's nodegroup , instead of giving priority to 2 > favored nodes returned as target. > {noformat} > *Instead of returning 5 targets on 3rd step above , we can return 2 good > favored nodes as target* > *And remaining 1 needed replica can be chosen from Random node of bad > FavoredNodes's nodegroup.* > This will make sure that the FavoredNodes are given priority. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9044) Give Priority to FavouredNodes , before selecting nodes from FavouredNode's Node Group
[ https://issues.apache.org/jira/browse/HDFS-9044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14902475#comment-14902475 ] J.Andreina commented on HDFS-9044: -- Testcase failures are unrelated. Checkstyle comments can be ignored , as all the 9 parameters of the method are been used. Please review. > Give Priority to FavouredNodes , before selecting nodes from FavouredNode's > Node Group > -- > > Key: HDFS-9044 > URL: https://issues.apache.org/jira/browse/HDFS-9044 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: J.Andreina >Assignee: J.Andreina > Attachments: HDFS-9044.1.patch, HDFS-9044.2.patch, HDFS-9044.3.patch > > > Passing Favored nodes intention is to place replica among the favored node > Current behavior in Node group is > If favored node is not available it goes to one among favored > nodegroup. > {noformat} > Say for example: > 1)I need 3 replicas and passed 5 favored nodes. > 2)Out of 5 favored nodes 3 favored nodes are not good. > 3)Then based on BlockPlacementPolicyWithNodeGroup out of 5 targets node > returned , 3 will be random node from 3 bad FavoredNode's nodegroup. > 4)Then there is a probability that all my 3 replicas are placed on > Random node from FavoredNodes's nodegroup , instead of giving priority to 2 > favored nodes returned as target. > {noformat} > *Instead of returning 5 targets on 3rd step above , we can return 2 good > favored nodes as target* > *And remaining 1 needed replica can be chosen from Random node of bad > FavoredNodes's nodegroup.* > This will make sure that the FavoredNodes are given priority. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9044) Give Priority to FavouredNodes , before selecting nodes from FavouredNode's Node Group
[ https://issues.apache.org/jira/browse/HDFS-9044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14900780#comment-14900780 ] Hadoop QA commented on HDFS-9044: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | pre-patch | 17m 56s | Pre-patch trunk compilation is healthy. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 1 new or modified test files. | | {color:green}+1{color} | javac | 8m 8s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 10m 26s | There were no new javadoc warning messages. | | {color:green}+1{color} | release audit | 0m 25s | The applied patch does not increase the total number of release audit warnings. | | {color:red}-1{color} | checkstyle | 1m 20s | The applied patch generated 6 new checkstyle issues (total was 50, now 55). | | {color:green}+1{color} | whitespace | 0m 1s | The patch has no lines that end in whitespace. | | {color:green}+1{color} | install | 1m 31s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 34s | The patch built with eclipse:eclipse. | | {color:red}-1{color} | findbugs | 2m 36s | The patch appears to introduce 1 new Findbugs (version 3.0.0) warnings. | | {color:green}+1{color} | native | 3m 29s | Pre-build of native portion | | {color:green}+1{color} | hdfs tests | 189m 50s | Tests passed in hadoop-hdfs. | | | | 236m 19s | | \\ \\ || Reason || Tests || | FindBugs | module:hadoop-hdfs | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12761401/HDFS-9044.2.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | trunk / c9cb6a5 | | checkstyle | https://builds.apache.org/job/PreCommit-HDFS-Build/12566/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt | | Findbugs warnings | https://builds.apache.org/job/PreCommit-HDFS-Build/12566/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html | | hadoop-hdfs test log | https://builds.apache.org/job/PreCommit-HDFS-Build/12566/artifact/patchprocess/testrun_hadoop-hdfs.txt | | Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/12566/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/12566/console | This message was automatically generated. > Give Priority to FavouredNodes , before selecting nodes from FavouredNode's > Node Group > -- > > Key: HDFS-9044 > URL: https://issues.apache.org/jira/browse/HDFS-9044 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: J.Andreina >Assignee: J.Andreina > Attachments: HDFS-9044.1.patch, HDFS-9044.2.patch > > > Passing Favored nodes intention is to place replica among the favored node > Current behavior in Node group is > If favored node is not available it goes to one among favored > nodegroup. > {noformat} > Say for example: > 1)I need 3 replicas and passed 5 favored nodes. > 2)Out of 5 favored nodes 3 favored nodes are not good. > 3)Then based on BlockPlacementPolicyWithNodeGroup out of 5 targets node > returned , 3 will be random node from 3 bad FavoredNode's nodegroup. > 4)Then there is a probability that all my 3 replicas are placed on > Random node from FavoredNodes's nodegroup , instead of giving priority to 2 > favored nodes returned as target. > {noformat} > *Instead of returning 5 targets on 3rd step above , we can return 2 good > favored nodes as target* > *And remaining 1 needed replica can be chosen from Random node of bad > FavoredNodes's nodegroup.* > This will make sure that the FavoredNodes are given priority. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-9044) Give Priority to FavouredNodes , before selecting nodes from FavouredNode's Node Group
[ https://issues.apache.org/jira/browse/HDFS-9044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14900405#comment-14900405 ] Hadoop QA commented on HDFS-9044: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | pre-patch | 18m 0s | Pre-patch trunk compilation is healthy. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 1 new or modified test files. | | {color:green}+1{color} | javac | 8m 1s | There were no new javac warning messages. | | {color:red}-1{color} | javadoc | 10m 19s | The applied patch generated 1 additional warning messages. | | {color:green}+1{color} | release audit | 0m 23s | The applied patch does not increase the total number of release audit warnings. | | {color:red}-1{color} | checkstyle | 1m 24s | The applied patch generated 10 new checkstyle issues (total was 50, now 59). | | {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that end in whitespace. | | {color:green}+1{color} | install | 1m 28s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 36s | The patch built with eclipse:eclipse. | | {color:red}-1{color} | findbugs | 2m 35s | The patch appears to introduce 1 new Findbugs (version 3.0.0) warnings. | | {color:green}+1{color} | native | 3m 18s | Pre-build of native portion | | {color:red}-1{color} | hdfs tests | 193m 1s | Tests failed in hadoop-hdfs. | | | | 239m 8s | | \\ \\ || Reason || Tests || | FindBugs | module:hadoop-hdfs | | Failed unit tests | hadoop.hdfs.server.namenode.TestFileTruncate | | | hadoop.hdfs.TestEncryptionZonesWithKMS | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12761367/HDFS-9044.1.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | trunk / c9cb6a5 | | javadoc | https://builds.apache.org/job/PreCommit-HDFS-Build/12564/artifact/patchprocess/diffJavadocWarnings.txt | | checkstyle | https://builds.apache.org/job/PreCommit-HDFS-Build/12564/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt | | Findbugs warnings | https://builds.apache.org/job/PreCommit-HDFS-Build/12564/artifact/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html | | hadoop-hdfs test log | https://builds.apache.org/job/PreCommit-HDFS-Build/12564/artifact/patchprocess/testrun_hadoop-hdfs.txt | | Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/12564/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/12564/console | This message was automatically generated. > Give Priority to FavouredNodes , before selecting nodes from FavouredNode's > Node Group > -- > > Key: HDFS-9044 > URL: https://issues.apache.org/jira/browse/HDFS-9044 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: J.Andreina >Assignee: J.Andreina > Attachments: HDFS-9044.1.patch > > > Passing Favored nodes intention is to place replica among the favored node > Current behavior in Node group is > If favored node is not available it goes to one among favored > nodegroup. > {noformat} > Say for example: > 1)I need 3 replicas and passed 5 favored nodes. > 2)Out of 5 favored nodes 3 favored nodes are not good. > 3)Then based on BlockPlacementPolicyWithNodeGroup out of 5 targets node > returned , 3 will be random node from 3 bad FavoredNode's nodegroup. > 4)Then there is a probability that all my 3 replicas are placed on > Random node from FavoredNodes's nodegroup , instead of giving priority to 2 > favored nodes returned as target. > {noformat} > *Instead of returning 5 targets on 3rd step above , we can return 2 good > favored nodes as target* > *And remaining 1 needed replica can be chosen from Random node of bad > FavoredNodes's nodegroup.* > This will make sure that the FavoredNodes are given priority. -- This message was sent by Atlassian JIRA (v6.3.4#6332)