[jira] [Commented] (HDDS-853) Option to force close a container in Datanode
[ https://issues.apache.org/jira/browse/HDDS-853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16695169#comment-16695169 ] Hudson commented on HDDS-853: - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #15488 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/15488/]) HDDS-853. Option to force close a container in Datanode. Contributed by (nanda: rev ebb9245366d211db6c4505106135e80ad87ba160) * (edit) hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/protocol/commands/CloseContainerCommand.java * (edit) hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/common/statemachine/commandhandler/CloseContainerCommandHandler.java * (edit) hadoop-hdds/container-service/src/main/proto/StorageContainerDatanodeProtocol.proto * (edit) hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/common/statemachine/commandhandler/TestCloseContainerCommandHandler.java > Option to force close a container in Datanode > - > > Key: HDDS-853 > URL: https://issues.apache.org/jira/browse/HDDS-853 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: Ozone Datanode >Reporter: Nanda kumar >Assignee: Nanda kumar >Priority: Major > Fix For: 0.4.0 > > Attachments: HDDS-853.000.patch, HDDS-853.001.patch > > > We need an option to force close a container in Datanode. When the container > is marked as QuasiClosed, based on the blockCommitSequenceId SCM will decide > the latest container replica and it will close try to close the QuasiClosed > container. > For this, we need force close support in Datanode which will close the > QuasiClosed container. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-853) Option to force close a container in Datanode
[ https://issues.apache.org/jira/browse/HDDS-853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16695154#comment-16695154 ] Nanda kumar commented on HDDS-853: -- Committed this to trunk. > Option to force close a container in Datanode > - > > Key: HDDS-853 > URL: https://issues.apache.org/jira/browse/HDDS-853 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: Ozone Datanode >Reporter: Nanda kumar >Assignee: Nanda kumar >Priority: Major > Fix For: 0.4.0 > > Attachments: HDDS-853.000.patch, HDDS-853.001.patch > > > We need an option to force close a container in Datanode. When the container > is marked as QuasiClosed, based on the blockCommitSequenceId SCM will decide > the latest container replica and it will close try to close the QuasiClosed > container. > For this, we need force close support in Datanode which will close the > QuasiClosed container. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-853) Option to force close a container in Datanode
[ https://issues.apache.org/jira/browse/HDDS-853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16695087#comment-16695087 ] Anu Engineer commented on HDDS-853: --- bq. Will fix this while committing. +1, Please go ahead. > Option to force close a container in Datanode > - > > Key: HDDS-853 > URL: https://issues.apache.org/jira/browse/HDDS-853 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: Ozone Datanode >Reporter: Nanda kumar >Assignee: Nanda kumar >Priority: Major > Attachments: HDDS-853.000.patch, HDDS-853.001.patch > > > We need an option to force close a container in Datanode. When the container > is marked as QuasiClosed, based on the blockCommitSequenceId SCM will decide > the latest container replica and it will close try to close the QuasiClosed > container. > For this, we need force close support in Datanode which will close the > QuasiClosed container. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-853) Option to force close a container in Datanode
[ https://issues.apache.org/jira/browse/HDDS-853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16695082#comment-16695082 ] Nanda kumar commented on HDDS-853: -- Thanks [~msingh] and [~anu] for the review. bq. Please move the newly added variable to use a new proto ID, instead of reusing an older value. Will fix this while committing. > Option to force close a container in Datanode > - > > Key: HDDS-853 > URL: https://issues.apache.org/jira/browse/HDDS-853 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: Ozone Datanode >Reporter: Nanda kumar >Assignee: Nanda kumar >Priority: Major > Attachments: HDDS-853.000.patch, HDDS-853.001.patch > > > We need an option to force close a container in Datanode. When the container > is marked as QuasiClosed, based on the blockCommitSequenceId SCM will decide > the latest container replica and it will close try to close the QuasiClosed > container. > For this, we need force close support in Datanode which will close the > QuasiClosed container. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-853) Option to force close a container in Datanode
[ https://issues.apache.org/jira/browse/HDDS-853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16695079#comment-16695079 ] Anu Engineer commented on HDDS-853: --- Please move the newly added variable to use a new proto ID, instead of reusing an older value. There is no benefit in renumbering field 3 to 4 other than breaking backward compatibility. > Option to force close a container in Datanode > - > > Key: HDDS-853 > URL: https://issues.apache.org/jira/browse/HDDS-853 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: Ozone Datanode >Reporter: Nanda kumar >Assignee: Nanda kumar >Priority: Major > Attachments: HDDS-853.000.patch, HDDS-853.001.patch > > > We need an option to force close a container in Datanode. When the container > is marked as QuasiClosed, based on the blockCommitSequenceId SCM will decide > the latest container replica and it will close try to close the QuasiClosed > container. > For this, we need force close support in Datanode which will close the > QuasiClosed container. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-853) Option to force close a container in Datanode
[
https://issues.apache.org/jira/browse/HDDS-853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16694599#comment-16694599
]
Hadoop QA commented on HDDS-853:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
35s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 26m
10s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
33s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
21s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
35s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 21s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
51s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 50s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 30s{color}
| {color:red} container-service in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
27s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 61m 10s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | HDDS-853 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12949021/HDDS-853.001.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle cc |
| uname | Linux 05b0d512249e 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14
08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / f63e4e4 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-HDDS-Build/1782/artifact/out/patch-unit-hadoop-hdds_container-service.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDDS-Build/1782/testReport/ |
| Max. process+thread count | 337 (vs. ulimit of 1) |
| modules | C: hadoop-hdds/container-service U: hadoop-hdds/container-service |
| Console output |
https://builds.apache.org/job/PreCommit-HDDS-Build/1782/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT
[jira] [Commented] (HDDS-853) Option to force close a container in Datanode
[ https://issues.apache.org/jira/browse/HDDS-853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16694502#comment-16694502 ] Nanda kumar commented on HDDS-853: -- Thanks [~msingh] for the review. Addressed the comments in patch v001. > Option to force close a container in Datanode > - > > Key: HDDS-853 > URL: https://issues.apache.org/jira/browse/HDDS-853 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: Ozone Datanode >Reporter: Nanda kumar >Assignee: Nanda kumar >Priority: Major > Attachments: HDDS-853.000.patch, HDDS-853.001.patch > > > We need an option to force close a container in Datanode. When the container > is marked as QuasiClosed, based on the blockCommitSequenceId SCM will decide > the latest container replica and it will close try to close the QuasiClosed > container. > For this, we need force close support in Datanode which will close the > QuasiClosed container. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-853) Option to force close a container in Datanode
[ https://issues.apache.org/jira/browse/HDDS-853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16694505#comment-16694505 ] Mukul Kumar Singh commented on HDDS-853: Thanks for updating the patch [~nandakumar131]. +1, the v1 patch looks good to me, pending jenkins. > Option to force close a container in Datanode > - > > Key: HDDS-853 > URL: https://issues.apache.org/jira/browse/HDDS-853 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: Ozone Datanode >Reporter: Nanda kumar >Assignee: Nanda kumar >Priority: Major > Attachments: HDDS-853.000.patch, HDDS-853.001.patch > > > We need an option to force close a container in Datanode. When the container > is marked as QuasiClosed, based on the blockCommitSequenceId SCM will decide > the latest container replica and it will close try to close the QuasiClosed > container. > For this, we need force close support in Datanode which will close the > QuasiClosed container. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-853) Option to force close a container in Datanode
[ https://issues.apache.org/jira/browse/HDDS-853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16693166#comment-16693166 ] Mukul Kumar Singh commented on HDDS-853: Thanks for working on this [~nandakumar131]. The patch looks good to me. Some minor comments. 1) CloseContainerCommandHandler:96, lets add a check that force option is false if the pipeline exists. 2) StorageContainerDatanodeProtocolProtos:294, Foce->Force 3) CloseContainerCommand:51, lets remove this function, and lets reconstruct the request with the force option for force option. > Option to force close a container in Datanode > - > > Key: HDDS-853 > URL: https://issues.apache.org/jira/browse/HDDS-853 > Project: Hadoop Distributed Data Store > Issue Type: Improvement > Components: Ozone Datanode >Reporter: Nanda kumar >Assignee: Nanda kumar >Priority: Major > Attachments: HDDS-853.000.patch > > > We need an option to force close a container in Datanode. When the container > is marked as QuasiClosed, based on the blockCommitSequenceId SCM will decide > the latest container replica and it will close try to close the QuasiClosed > container. > For this, we need force close support in Datanode which will close the > QuasiClosed container. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-853) Option to force close a container in Datanode
[
https://issues.apache.org/jira/browse/HDDS-853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16692029#comment-16692029
]
Hadoop QA commented on HDDS-853:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 22m
52s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 8s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
46s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 15s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 29s{color}
| {color:red} container-service in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
22s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 53m 1s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | HDDS-853 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12948738/HDDS-853.000.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle cc |
| uname | Linux 33221380922e 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 9366608 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-HDDS-Build/1759/artifact/out/patch-unit-hadoop-hdds_container-service.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDDS-Build/1759/testReport/ |
| Max. process+thread count | 417 (vs. ulimit of 1) |
| modules | C: hadoop-hdds/container-service U: hadoop-hdds/container-service |
| Console output |
https://builds.apache.org/job/PreCommit-HDDS-Build/1759/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT ht
