[jira] [Commented] (HDDS-228) Add the ReplicaMaps to ContainerStateManager
[ https://issues.apache.org/jira/browse/HDDS-228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16542129#comment-16542129 ] Ajay Kumar commented on HDDS-228: - [~anu] thanks for review and commit. [~nandakumar131] thanks for review and discussion. > Add the ReplicaMaps to ContainerStateManager > > > Key: HDDS-228 > URL: https://issues.apache.org/jira/browse/HDDS-228 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: SCM >Reporter: Anu Engineer >Assignee: Ajay Kumar >Priority: Major > Fix For: 0.2.1 > > Attachments: HDDS-228.00.patch, HDDS-228.01.patch, HDDS-228.02.patch, > HDDS-228.03.patch, HDDS-228.04.patch, HDDS-228.05.patch, HDDS-228.06.patch > > > We need to maintain a list of data nodes in the SCM that tells us where a > container is located. This created from the container reports. The HDDS-175 > refactored the class to make this separation easy and this JIRA is a followup > that keeps a hash table to track this information. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-228) Add the ReplicaMaps to ContainerStateManager
[ https://issues.apache.org/jira/browse/HDDS-228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16541088#comment-16541088 ] genericqa commented on HDDS-228: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 11m 17s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 6m 5s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 29m 53s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 36m 43s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 29s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 20s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 13m 52s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Skipped patched modules with no Java source: hadoop-ozone/integration-test {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 50s{color} | {color:red} hadoop-hdds/server-scm in trunk has 1 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 0s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 25s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 56s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 35m 15s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 35m 15s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 26s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 15s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 11m 37s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Skipped patched modules with no Java source: hadoop-ozone/integration-test {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 54s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 56s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 29s{color} | {color:green} server-scm in the patch passed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 13m 27s{color} | {color:red} integration-test in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 44s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}170m 7s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:abb62dd | | JIRA Issue | HDDS-228 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12931251/HDDS-228.06.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 0618d4e20267 3.13.0-143-generic #192-Ubuntu SMP Tue Feb 27 10:45:36 UTC 2018 x86_64 x86_64 x86_64 GNU/L
[jira] [Commented] (HDDS-228) Add the ReplicaMaps to ContainerStateManager
[ https://issues.apache.org/jira/browse/HDDS-228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16541052#comment-16541052 ] genericqa commented on HDDS-228: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 24s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m 51s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 27m 35s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 28m 37s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 28s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 15s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 12m 30s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Skipped patched modules with no Java source: hadoop-ozone/integration-test {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 44s{color} | {color:red} hadoop-hdds/server-scm in trunk has 1 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 58s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 22s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 51s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 28m 40s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 28m 40s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 25s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 2s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 10m 0s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Skipped patched modules with no Java source: hadoop-ozone/integration-test {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 46s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 41s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 24s{color} | {color:green} server-scm in the patch passed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 19m 32s{color} | {color:red} integration-test in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 34s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}139m 38s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.ozone.TestStorageContainerManager | | | hadoop.ozone.container.common.statemachine.commandhandler.TestCloseContainerByPipeline | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:abb62dd | | JIRA Issue | HDDS-228 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12931249/HDDS-228.05.patch | | Optional Tests | asflicense compile javac java
[jira] [Commented] (HDDS-228) Add the ReplicaMaps to ContainerStateManager
[ https://issues.apache.org/jira/browse/HDDS-228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16540944#comment-16540944 ] Ajay Kumar commented on HDDS-228: - [~anu] as discussed moved the lock in {{ContainerStateMap#getContainerReplicas}} before if condition in patch v6. Also fixed remaining {{ResultCodes.IO_EXCEPTION}} > Add the ReplicaMaps to ContainerStateManager > > > Key: HDDS-228 > URL: https://issues.apache.org/jira/browse/HDDS-228 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: SCM >Reporter: Anu Engineer >Assignee: Ajay Kumar >Priority: Major > Fix For: 0.2.1 > > Attachments: HDDS-228.00.patch, HDDS-228.01.patch, HDDS-228.02.patch, > HDDS-228.03.patch, HDDS-228.04.patch, HDDS-228.05.patch, HDDS-228.06.patch > > > We need to maintain a list of data nodes in the SCM that tells us where a > container is located. This created from the container reports. The HDDS-175 > refactored the class to make this separation easy and this JIRA is a followup > that keeps a hash table to track this information. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-228) Add the ReplicaMaps to ContainerStateManager
[ https://issues.apache.org/jira/browse/HDDS-228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16540946#comment-16540946 ] Anu Engineer commented on HDDS-228: --- Thank you for updating the patch. +1, v6, Pending Jenkins. > Add the ReplicaMaps to ContainerStateManager > > > Key: HDDS-228 > URL: https://issues.apache.org/jira/browse/HDDS-228 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: SCM >Reporter: Anu Engineer >Assignee: Ajay Kumar >Priority: Major > Fix For: 0.2.1 > > Attachments: HDDS-228.00.patch, HDDS-228.01.patch, HDDS-228.02.patch, > HDDS-228.03.patch, HDDS-228.04.patch, HDDS-228.05.patch, HDDS-228.06.patch > > > We need to maintain a list of data nodes in the SCM that tells us where a > container is located. This created from the container reports. The HDDS-175 > refactored the class to make this separation easy and this JIRA is a followup > that keeps a hash table to track this information. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-228) Add the ReplicaMaps to ContainerStateManager
[ https://issues.apache.org/jira/browse/HDDS-228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16540933#comment-16540933 ] Anu Engineer commented on HDDS-228: --- +1, pending Jenkins. There is small issue that I can fix while committing. {code:java} removeContainerReplica: This was in 2 places -- I will fix this while committing. throw new SCMException( "No entry exist for containerId: " + containerID + " in replica map.", ResultCodes.IO_EXCEPTION);{code} ==> *FAILED_TO_FIND_CONTAINER* > Add the ReplicaMaps to ContainerStateManager > > > Key: HDDS-228 > URL: https://issues.apache.org/jira/browse/HDDS-228 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: SCM >Reporter: Anu Engineer >Assignee: Ajay Kumar >Priority: Major > Fix For: 0.2.1 > > Attachments: HDDS-228.00.patch, HDDS-228.01.patch, HDDS-228.02.patch, > HDDS-228.03.patch, HDDS-228.04.patch, HDDS-228.05.patch > > > We need to maintain a list of data nodes in the SCM that tells us where a > container is located. This created from the container reports. The HDDS-175 > refactored the class to make this separation easy and this JIRA is a followup > that keeps a hash table to track this information. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-228) Add the ReplicaMaps to ContainerStateManager
[ https://issues.apache.org/jira/browse/HDDS-228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16540918#comment-16540918 ] Ajay Kumar commented on HDDS-228: - [~anu] thanks for review. Addressed all comments in patch v5. > Add the ReplicaMaps to ContainerStateManager > > > Key: HDDS-228 > URL: https://issues.apache.org/jira/browse/HDDS-228 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: SCM >Reporter: Anu Engineer >Assignee: Ajay Kumar >Priority: Major > Fix For: 0.2.1 > > Attachments: HDDS-228.00.patch, HDDS-228.01.patch, HDDS-228.02.patch, > HDDS-228.03.patch, HDDS-228.04.patch, HDDS-228.05.patch > > > We need to maintain a list of data nodes in the SCM that tells us where a > container is located. This created from the container reports. The HDDS-175 > refactored the class to make this separation easy and this JIRA is a followup > that keeps a hash table to track this information. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-228) Add the ReplicaMaps to ContainerStateManager
[ https://issues.apache.org/jira/browse/HDDS-228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16540725#comment-16540725 ] Anu Engineer commented on HDDS-228: --- Thank you for the patch. It looks good overall. Some very minor comments. * *ContainerStateMap.java*: {code:java} throw new SCMException( "No entry exist for containerId: " + containerID + " in replica map.", ResultCodes.IO_EXCEPTION); {code} Replace ResultCodes.IO_EXCEPTION ---> FAILED_TO_FIND_CONTAINER * *getContainerReplicas*(): {code:java} if (contReplicaMap.containsKey(containerID)) { return Collections .unmodifiableSet(contReplicaMap.get(containerID)); } {code} I think you need to lock here too, since you are locking in the add and remove. * *addContainerReplica*(): Please correct me if I am wrong, We seem to be locking and releasing too many times: {code:java} for (DatanodeDetails dn : dnList) { Preconditions.checkNotNull(dn); // Take lock to avoid race condition around insertion. try (AutoCloseableLock lock = autoLock.acquire()) { if (contReplicaMap.containsKey(containerID)) { contReplicaMap.get(containerID).add(dn); } else { Set dnSet = new HashSet<>(); dnSet.add(dn); contReplicaMap.put(containerID, dnSet); } } } {code} For each DN we are locking and releasing. Assuming that we are only going to add handleful of replicas at the max (say 3 or less than 5), it might be cheaper to do the locking before for loop. * There is probably a latent bug in the {{contReplicaMap.get(containerID).add(dn);}}, it is possible the user might add a DNDetail which already part of this set. That error code – which is false – is ignored instead of propagating it back to user. if we don't want to propagate it back, we should at least log that info. * *ContainerStateMap.java:364*: Spurious Edit? * *testReplicaMap*(): It might be a good idea to re-insert a node and add a test case, as well as define that behavior. > Add the ReplicaMaps to ContainerStateManager > > > Key: HDDS-228 > URL: https://issues.apache.org/jira/browse/HDDS-228 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: SCM >Reporter: Anu Engineer >Assignee: Ajay Kumar >Priority: Major > Fix For: 0.2.1 > > Attachments: HDDS-228.00.patch, HDDS-228.01.patch, HDDS-228.02.patch, > HDDS-228.03.patch, HDDS-228.04.patch > > > We need to maintain a list of data nodes in the SCM that tells us where a > container is located. This created from the container reports. The HDDS-175 > refactored the class to make this separation easy and this JIRA is a followup > that keeps a hash table to track this information. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-228) Add the ReplicaMaps to ContainerStateManager
[ https://issues.apache.org/jira/browse/HDDS-228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16539276#comment-16539276 ] genericqa commented on HDDS-228: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 28s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m 5s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 26m 18s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 28m 32s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 18s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 55s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 10m 47s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Skipped patched modules with no Java source: hadoop-ozone/integration-test {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 35s{color} | {color:red} hadoop-hdds/server-scm in trunk has 3 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 39s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 21s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 49s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 28m 12s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 28m 12s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 23s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 4s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 10m 26s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Skipped patched modules with no Java source: hadoop-ozone/integration-test {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 42s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 41s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 20s{color} | {color:green} server-scm in the patch passed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 20m 18s{color} | {color:red} integration-test in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 43s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}136m 54s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.ozone.TestStorageContainerManager | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:abb62dd | | JIRA Issue | HDDS-228 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12931048/HDDS-228.04.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 213b261fce
[jira] [Commented] (HDDS-228) Add the ReplicaMaps to ContainerStateManager
[ https://issues.apache.org/jira/browse/HDDS-228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16539160#comment-16539160 ] Ajay Kumar commented on HDDS-228: - [~nandakumar131] thanks for review. Addressed most of you suggestions, except one. {quote}ContainerStateManager#getContainerReplicas javadoc is incorrect. ContainerStateMap#getContainerReplicas javadoc is incorrect ContainerStateMap#removeContainerReplica: For the third case, the call should throw Exception.{quote} Done {quote}ContainerStateMap#addContainerReplica: If the container is not already present, this method adds the container to the contReplicaMap and updates the set with the given datanode. It would be easy to detect inconsistency if we throw Exception in case if the container is not present in the contReplicaMap. A container should be added to the contReplicaMap when addContainer is called. This way we can maintain consistency between containerMap and contReplicaMap.{quote} IMO adding a new replica even if one doesn't exist do not violate any condition. So we should add it silently. Also there is little value in adding open containers to replicaMap as there replication will be handled by RATIS. So only contrainer report handler should add or remove entries from this instead of adding it via addContainer. > Add the ReplicaMaps to ContainerStateManager > > > Key: HDDS-228 > URL: https://issues.apache.org/jira/browse/HDDS-228 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: SCM >Reporter: Anu Engineer >Assignee: Ajay Kumar >Priority: Major > Fix For: 0.2.1 > > Attachments: HDDS-228.00.patch, HDDS-228.01.patch, HDDS-228.02.patch, > HDDS-228.03.patch, HDDS-228.04.patch > > > We need to maintain a list of data nodes in the SCM that tells us where a > container is located. This created from the container reports. The HDDS-175 > refactored the class to make this separation easy and this JIRA is a followup > that keeps a hash table to track this information. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-228) Add the ReplicaMaps to ContainerStateManager
[ https://issues.apache.org/jira/browse/HDDS-228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16538921#comment-16538921 ] Nanda kumar commented on HDDS-228: -- Thanks [~ajayydv] for updating the patch. {{ContainerStateManager#getContainerReplicas}} javadoc is incorrect. {{ContainerStateMap#getContainerReplicas}} javadoc is incorrect {{ContainerStateMap#addContainerReplica}}: If the container is not already present, this method adds the container to the contReplicaMap and updates the set with the given datanode. It would be easy to detect inconsistency if we throw {{Exception}} in case if the container is not present in the contReplicaMap. A container should be added to the contReplicaMap when {{addContainer}} is called. This way we can maintain consistency between {{containerMap}} and {{contReplicaMap}}. {{ContainerStateMap#removeContainerReplica}}: There are three possible cases that has to be handled here. 1. Container exists in {{contReplicaMap}} and it has an entry for the given datanode in its set. 2. Container exists in {{contReplicaMap}} but it doesn't have the datanode in its set. 3. Container doesn't exist in {{contReplicaMap}}. Current behavior: For the first call will return true. For the second and third case, the call will false. We can change the behavior to: For the first case, the call will return true. For the second case, the call will return false. For the third case, the call should throw Exception. This way, we will consistently throw an exception in all the related calls, if we don't have the container in our map. > Add the ReplicaMaps to ContainerStateManager > > > Key: HDDS-228 > URL: https://issues.apache.org/jira/browse/HDDS-228 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: SCM >Reporter: Anu Engineer >Assignee: Ajay Kumar >Priority: Major > Fix For: 0.2.1 > > Attachments: HDDS-228.00.patch, HDDS-228.01.patch, HDDS-228.02.patch, > HDDS-228.03.patch > > > We need to maintain a list of data nodes in the SCM that tells us where a > container is located. This created from the container reports. The HDDS-175 > refactored the class to make this separation easy and this JIRA is a followup > that keeps a hash table to track this information. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-228) Add the ReplicaMaps to ContainerStateManager
[ https://issues.apache.org/jira/browse/HDDS-228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16537503#comment-16537503 ] Ajay Kumar commented on HDDS-228: - failed tests are unrelated. > Add the ReplicaMaps to ContainerStateManager > > > Key: HDDS-228 > URL: https://issues.apache.org/jira/browse/HDDS-228 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: SCM >Reporter: Anu Engineer >Assignee: Ajay Kumar >Priority: Major > Fix For: 0.2.1 > > Attachments: HDDS-228.00.patch, HDDS-228.01.patch, HDDS-228.02.patch, > HDDS-228.03.patch > > > We need to maintain a list of data nodes in the SCM that tells us where a > container is located. This created from the container reports. The HDDS-175 > refactored the class to make this separation easy and this JIRA is a followup > that keeps a hash table to track this information. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-228) Add the ReplicaMaps to ContainerStateManager
[ https://issues.apache.org/jira/browse/HDDS-228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16537434#comment-16537434 ] genericqa commented on HDDS-228: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 24s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 32s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 27m 8s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 28m 35s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 27s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 15s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 12m 25s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Skipped patched modules with no Java source: hadoop-ozone/integration-test {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 45s{color} | {color:red} hadoop-hdds/server-scm in trunk has 3 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 39s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 21s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 49s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 28m 3s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 28m 3s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 28s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 14s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 10m 22s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Skipped patched modules with no Java source: hadoop-ozone/integration-test {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 46s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 41s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 17s{color} | {color:green} server-scm in the patch passed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 25m 45s{color} | {color:red} integration-test in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 35s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}143m 22s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.ozone.ozShell.TestOzoneShell | | | hadoop.ozone.container.common.statemachine.commandhandler.TestCloseContainerByPipeline | | | hadoop.ozone.TestStorageContainerManager | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:abb62dd | | JIRA Issue | HDDS-228 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12930872/HDDS-228.03.patch | | Option
[jira] [Commented] (HDDS-228) Add the ReplicaMaps to ContainerStateManager
[ https://issues.apache.org/jira/browse/HDDS-228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16537239#comment-16537239 ] Ajay Kumar commented on HDDS-228: - [~nandakumar131] As discussed responsibility to put open or close containers lies with ContainerReportHandler and ReplicationManager. Updated patch v3 to throw an SCM exception if requested containerId doesn't exist in replicaMap. > Add the ReplicaMaps to ContainerStateManager > > > Key: HDDS-228 > URL: https://issues.apache.org/jira/browse/HDDS-228 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: SCM >Reporter: Anu Engineer >Assignee: Ajay Kumar >Priority: Major > Fix For: 0.2.1 > > Attachments: HDDS-228.00.patch, HDDS-228.01.patch, HDDS-228.02.patch, > HDDS-228.03.patch > > > We need to maintain a list of data nodes in the SCM that tells us where a > container is located. This created from the container reports. The HDDS-175 > refactored the class to make this separation easy and this JIRA is a followup > that keeps a hash table to track this information. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-228) Add the ReplicaMaps to ContainerStateManager
[ https://issues.apache.org/jira/browse/HDDS-228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16536621#comment-16536621 ] Nanda kumar commented on HDDS-228: -- Thanks [~ajayydv] for updating the patch and for the explanation. The patch looks good to me. With this behavior how are we going to differentiate an invalid containerId and an open containerId when we make {{getContainerReplicas}} call? (Note: Open containers are also maintained by ContainerStateManager) > Add the ReplicaMaps to ContainerStateManager > > > Key: HDDS-228 > URL: https://issues.apache.org/jira/browse/HDDS-228 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: SCM >Reporter: Anu Engineer >Assignee: Ajay Kumar >Priority: Major > Fix For: 0.2.1 > > Attachments: HDDS-228.00.patch, HDDS-228.01.patch, HDDS-228.02.patch > > > We need to maintain a list of data nodes in the SCM that tells us where a > container is located. This created from the container reports. The HDDS-175 > refactored the class to make this separation easy and this JIRA is a followup > that keeps a hash table to track this information. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-228) Add the ReplicaMaps to ContainerStateManager
[ https://issues.apache.org/jira/browse/HDDS-228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16534364#comment-16534364 ] genericqa commented on HDDS-228: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 26s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 20s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 25m 36s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 27m 40s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 17s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 14s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 11m 33s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Skipped patched modules with no Java source: hadoop-ozone/integration-test {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 32s{color} | {color:red} hadoop-hdds/server-scm in trunk has 3 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 35s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 52s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 43s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 26m 14s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 26m 14s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 15s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 52s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 9m 4s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Skipped patched modules with no Java source: hadoop-ozone/integration-test {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 39s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 38s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 16s{color} | {color:green} server-scm in the patch passed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 23m 12s{color} | {color:red} integration-test in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 29s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}133m 12s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.ozone.TestStorageContainerManager | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:abb62dd | | JIRA Issue | HDDS-228 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12930468/HDDS-228.02.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 63bc021ed5
[jira] [Commented] (HDDS-228) Add the ReplicaMaps to ContainerStateManager
[ https://issues.apache.org/jira/browse/HDDS-228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16534283#comment-16534283 ] Ajay Kumar commented on HDDS-228: - [~nandakumar131] thanks for review. {quote}Instead of adding container entry in contReplicaMap inside addContainerReplica call, can we do it in addContainer call. This way containerMap and contReplicaMap will be consistent. {quote} addContainer is called from ContainerStateManager#allocateContainer as well. This means Container is probably in open state and we don't want replicationManager to handle it until it is closed. Instead of adding an entry right away we can wait for DataNodes to report this container via ContainerReport and then add it. Having separate API will allow ContainerReportHandler to do this mutation and send a message to ReplicationManager accordingly. {quote}It would be nice to have DatanodeDetails as varargs in addContainerReplica, so that it will be easy to update/add multiple datanodes at the same time.{quote} Done {quote}Inside ContainerStateManager#allocateContainer method we should update the container replica after we add the container to ContainerStateMap.{quote} same as above. > Add the ReplicaMaps to ContainerStateManager > > > Key: HDDS-228 > URL: https://issues.apache.org/jira/browse/HDDS-228 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: SCM >Reporter: Anu Engineer >Assignee: Ajay Kumar >Priority: Major > Attachments: HDDS-228.00.patch, HDDS-228.01.patch > > > We need to maintain a list of data nodes in the SCM that tells us where a > container is located. This created from the container reports. The HDDS-175 > refactored the class to make this separation easy and this JIRA is a followup > that keeps a hash table to track this information. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-228) Add the ReplicaMaps to ContainerStateManager
[ https://issues.apache.org/jira/browse/HDDS-228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16534179#comment-16534179 ] Nanda kumar commented on HDDS-228: -- Thanks [~ajayydv] for working on this. Overall the patch looks good to me. Instead of adding container entry in {{contReplicaMap}} inside {{addContainerReplica}} call, can we do it in {{addContainer}} call. This way {{containerMap}} and {{contReplicaMap}} will be consistent. We can add a empty set here, later on the set can be updated via {{addContainerReplica}}. We should throw {{SCMException("No such container.", FAILED_TO_FIND_CONTAINER)}} exception if we are not able to find the container in {{contReplicaMap}}, this way we can detect inconsistency in ContainerStateMap, if any. It would be nice to have DatanodeDetails as varargs in {{addContainerReplica}}, so that it will be easy to update/add multiple datanodes at the same time. Inside {{ContainerStateManager#allocateContainer}} method we should update the container replica after we add the container to ContainerStateMap (line: 314) > Add the ReplicaMaps to ContainerStateManager > > > Key: HDDS-228 > URL: https://issues.apache.org/jira/browse/HDDS-228 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: SCM >Reporter: Anu Engineer >Assignee: Ajay Kumar >Priority: Major > Attachments: HDDS-228.00.patch, HDDS-228.01.patch > > > We need to maintain a list of data nodes in the SCM that tells us where a > container is located. This created from the container reports. The HDDS-175 > refactored the class to make this separation easy and this JIRA is a followup > that keeps a hash table to track this information. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-228) Add the ReplicaMaps to ContainerStateManager
[ https://issues.apache.org/jira/browse/HDDS-228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16534178#comment-16534178 ] genericqa commented on HDDS-228: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 22s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m 6s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 25m 41s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 28m 53s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 17s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 1s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 11m 3s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Skipped patched modules with no Java source: hadoop-ozone/integration-test {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 40s{color} | {color:red} hadoop-hdds/server-scm in trunk has 3 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 53s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 21s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 47s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 26m 56s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 26m 56s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 25s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 9s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 9m 49s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Skipped patched modules with no Java source: hadoop-ozone/integration-test {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 47s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 54s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 24s{color} | {color:green} server-scm in the patch passed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 20m 19s{color} | {color:red} integration-test in the patch failed. {color} | | {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 40s{color} | {color:red} The patch generated 1 ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}135m 58s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.ozone.TestStorageContainerManager | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:abb62dd | | JIRA Issue | HDDS-228 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12930426/HDDS-228.01.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux aa44f718ee22 4.4.0-64-gen
[jira] [Commented] (HDDS-228) Add the ReplicaMaps to ContainerStateManager
[ https://issues.apache.org/jira/browse/HDDS-228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16534143#comment-16534143 ] genericqa commented on HDDS-228: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 15s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 20s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 28m 21s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 29m 33s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 21s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 7s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 12m 29s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Skipped patched modules with no Java source: hadoop-ozone/integration-test {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 41s{color} | {color:red} hadoop-hdds/server-scm in trunk has 3 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 50s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 21s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 52s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 34m 38s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 34m 38s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 24s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 14s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 11m 25s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Skipped patched modules with no Java source: hadoop-ozone/integration-test {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 54s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 58s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 23s{color} | {color:green} server-scm in the patch passed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red} 18m 3s{color} | {color:red} integration-test in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 44s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}146m 17s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.ozone.TestStorageContainerManager | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:abb62dd | | JIRA Issue | HDDS-228 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12930418/HDDS-228.00.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 4edb97155a
[jira] [Commented] (HDDS-228) Add the ReplicaMaps to ContainerStateManager
[ https://issues.apache.org/jira/browse/HDDS-228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16534031#comment-16534031 ] Ajay Kumar commented on HDDS-228: - patch v1 with following 2 changes: * getContainerReplicas returns UnmodifiableSet to prevent any client from accidentally changing actual replica set. Any mutation should happen via api. * changed contReplicaMap to HashMap. > Add the ReplicaMaps to ContainerStateManager > > > Key: HDDS-228 > URL: https://issues.apache.org/jira/browse/HDDS-228 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: SCM >Reporter: Anu Engineer >Assignee: Ajay Kumar >Priority: Major > Attachments: HDDS-228.00.patch, HDDS-228.01.patch > > > We need to maintain a list of data nodes in the SCM that tells us where a > container is located. This created from the container reports. The HDDS-175 > refactored the class to make this separation easy and this JIRA is a followup > that keeps a hash table to track this information. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org