[jira] [Updated] (HDFS-14064) WEBHDFS: Support Enable/Disable EC Policy
[ https://issues.apache.org/jira/browse/HDFS-14064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated HDFS-14064: Status: Patch Available (was: Open) > WEBHDFS: Support Enable/Disable EC Policy > - > > Key: HDFS-14064 > URL: https://issues.apache.org/jira/browse/HDFS-14064 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: Ayush Saxena >Assignee: Ayush Saxena >Priority: Major > Attachments: HDFS-14064-01.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-14064) WEBHDFS: Support Enable/Disable EC Policy
Ayush Saxena created HDFS-14064: --- Summary: WEBHDFS: Support Enable/Disable EC Policy Key: HDFS-14064 URL: https://issues.apache.org/jira/browse/HDFS-14064 Project: Hadoop HDFS Issue Type: Improvement Reporter: Ayush Saxena Assignee: Ayush Saxena -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-539) ozone datanode ignores the invalid options
[
https://issues.apache.org/jira/browse/HDDS-539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16682734#comment-16682734
]
Hadoop QA commented on HDDS-539:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s{color}
| {color:red} HDDS-539 does not apply to trunk. Rebase required? Wrong Branch?
See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HDDS-539 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12947724/HDDS-539.003.patch |
| Console output |
https://builds.apache.org/job/PreCommit-HDDS-Build/1677/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> ozone datanode ignores the invalid options
> --
>
> Key: HDDS-539
> URL: https://issues.apache.org/jira/browse/HDDS-539
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
>Reporter: Namit Maheshwari
>Assignee: Vinicius Higa Murakami
>Priority: Major
> Labels: newbie
> Attachments: HDDS-539.003.patch, HDDS-539.patch
>
>
> ozone datanode command starts datanode and ignores the invalid option, apart
> from help
> {code:java}
> [root@ctr-e138-1518143905142-481027-01-02 bin]# ./ozone datanode -help
> Starts HDDS Datanode
> {code}
> For all the other invalid options, it just ignores and starts the DN like
> below:
> {code:java}
> [root@ctr-e138-1518143905142-481027-01-02 bin]# ./ozone datanode -ABC
> 2018-09-22 00:59:34,462 [main] INFO - STARTUP_MSG:
> /
> STARTUP_MSG: Starting HddsDatanodeService
> STARTUP_MSG: host =
> ctr-e138-1518143905142-481027-01-02.hwx.site/172.27.54.20
> STARTUP_MSG: args = [-ABC]
> STARTUP_MSG: version = 3.2.0-SNAPSHOT
> STARTUP_MSG: classpath =
> /root/ozone-0.3.0-SNAPSHOT/etc/hadoop:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/commons-cli-1.2.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/kerb-crypto-1.0.1.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/guava-11.0.2.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/hadoop-auth-3.2.0-SNAPSHOT.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jcip-annotations-1.0-1.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jsr305-3.0.0.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/commons-compress-1.4.1.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/commons-beanutils-1.9.3.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/commons-collections-3.2.2.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jsp-api-2.1.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/kerb-simplekdc-1.0.1.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/htrace-core4-4.1.0-incubating.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/zookeeper-3.4.9.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/gson-2.2.4.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/token-provider-1.0.1.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/dnsjava-2.1.7.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/avro-1.7.7.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jsr311-api-1.1.1.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jersey-json-1.19.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/stax2-api-3.1.4.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/log4j-1.2.17.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/accessors-smart-1.2.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/commons-lang3-3.7.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jersey-server-1.19.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/netty-3.10.5.Final.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/snappy-java-1.0.5.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/kerby-config-1.0.1.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/kerby-util-1.0.1.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/httpclient-4.5.2.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jetty-security-9.3.19.v20170502.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/hadoop-annotations-3.2.0-SNAPSHOT.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/re2j-1.1.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jackson-databind-2.9.5.jar:/root/ozo
[jira] [Updated] (HDDS-539) ozone datanode ignores the invalid options
[
https://issues.apache.org/jira/browse/HDDS-539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinicius Higa Murakami updated HDDS-539:
Attachment: (was: HDDS-539_2.patch)
> ozone datanode ignores the invalid options
> --
>
> Key: HDDS-539
> URL: https://issues.apache.org/jira/browse/HDDS-539
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
>Reporter: Namit Maheshwari
>Assignee: Vinicius Higa Murakami
>Priority: Major
> Labels: newbie
> Attachments: HDDS-539.003.patch, HDDS-539.patch
>
>
> ozone datanode command starts datanode and ignores the invalid option, apart
> from help
> {code:java}
> [root@ctr-e138-1518143905142-481027-01-02 bin]# ./ozone datanode -help
> Starts HDDS Datanode
> {code}
> For all the other invalid options, it just ignores and starts the DN like
> below:
> {code:java}
> [root@ctr-e138-1518143905142-481027-01-02 bin]# ./ozone datanode -ABC
> 2018-09-22 00:59:34,462 [main] INFO - STARTUP_MSG:
> /
> STARTUP_MSG: Starting HddsDatanodeService
> STARTUP_MSG: host =
> ctr-e138-1518143905142-481027-01-02.hwx.site/172.27.54.20
> STARTUP_MSG: args = [-ABC]
> STARTUP_MSG: version = 3.2.0-SNAPSHOT
> STARTUP_MSG: classpath =
> /root/ozone-0.3.0-SNAPSHOT/etc/hadoop:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/commons-cli-1.2.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/kerb-crypto-1.0.1.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/guava-11.0.2.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/hadoop-auth-3.2.0-SNAPSHOT.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jcip-annotations-1.0-1.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jsr305-3.0.0.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/commons-compress-1.4.1.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/commons-beanutils-1.9.3.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/commons-collections-3.2.2.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jsp-api-2.1.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/kerb-simplekdc-1.0.1.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/htrace-core4-4.1.0-incubating.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/zookeeper-3.4.9.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/gson-2.2.4.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/token-provider-1.0.1.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/dnsjava-2.1.7.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/avro-1.7.7.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jsr311-api-1.1.1.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jersey-json-1.19.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/stax2-api-3.1.4.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/log4j-1.2.17.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/accessors-smart-1.2.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/commons-lang3-3.7.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jersey-server-1.19.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/netty-3.10.5.Final.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/snappy-java-1.0.5.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/kerby-config-1.0.1.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/kerby-util-1.0.1.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/httpclient-4.5.2.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jetty-security-9.3.19.v20170502.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/hadoop-annotations-3.2.0-SNAPSHOT.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/re2j-1.1.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jackson-databind-2.9.5.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/commons-math3-3.1.1.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/commons-logging-1.1.3.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jersey-core-1.19.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/kerb-client-1.0.1.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jsch-0.1.54.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jersey-servlet-1.19.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/asm-5.0.4.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jackson-core-2.9.5.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jetty-util-9.3.19.v20170502.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/nimbus-jose-jwt-4.41.1.jar:/ro
[jira] [Updated] (HDDS-539) ozone datanode ignores the invalid options
[
https://issues.apache.org/jira/browse/HDDS-539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinicius Higa Murakami updated HDDS-539:
Attachment: HDDS-539.003.patch
> ozone datanode ignores the invalid options
> --
>
> Key: HDDS-539
> URL: https://issues.apache.org/jira/browse/HDDS-539
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
>Reporter: Namit Maheshwari
>Assignee: Vinicius Higa Murakami
>Priority: Major
> Labels: newbie
> Attachments: HDDS-539.003.patch, HDDS-539.patch
>
>
> ozone datanode command starts datanode and ignores the invalid option, apart
> from help
> {code:java}
> [root@ctr-e138-1518143905142-481027-01-02 bin]# ./ozone datanode -help
> Starts HDDS Datanode
> {code}
> For all the other invalid options, it just ignores and starts the DN like
> below:
> {code:java}
> [root@ctr-e138-1518143905142-481027-01-02 bin]# ./ozone datanode -ABC
> 2018-09-22 00:59:34,462 [main] INFO - STARTUP_MSG:
> /
> STARTUP_MSG: Starting HddsDatanodeService
> STARTUP_MSG: host =
> ctr-e138-1518143905142-481027-01-02.hwx.site/172.27.54.20
> STARTUP_MSG: args = [-ABC]
> STARTUP_MSG: version = 3.2.0-SNAPSHOT
> STARTUP_MSG: classpath =
> /root/ozone-0.3.0-SNAPSHOT/etc/hadoop:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/commons-cli-1.2.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/kerb-crypto-1.0.1.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/guava-11.0.2.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/hadoop-auth-3.2.0-SNAPSHOT.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jcip-annotations-1.0-1.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jsr305-3.0.0.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/commons-compress-1.4.1.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/commons-beanutils-1.9.3.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/commons-collections-3.2.2.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jsp-api-2.1.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/kerb-simplekdc-1.0.1.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/htrace-core4-4.1.0-incubating.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/zookeeper-3.4.9.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/gson-2.2.4.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/token-provider-1.0.1.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/dnsjava-2.1.7.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/avro-1.7.7.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jsr311-api-1.1.1.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jersey-json-1.19.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/stax2-api-3.1.4.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/log4j-1.2.17.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/accessors-smart-1.2.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/commons-lang3-3.7.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jersey-server-1.19.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/netty-3.10.5.Final.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/snappy-java-1.0.5.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/kerby-config-1.0.1.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/kerby-util-1.0.1.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/httpclient-4.5.2.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jetty-security-9.3.19.v20170502.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/hadoop-annotations-3.2.0-SNAPSHOT.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/re2j-1.1.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jackson-databind-2.9.5.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/commons-math3-3.1.1.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/commons-logging-1.1.3.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jersey-core-1.19.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/kerb-client-1.0.1.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jsch-0.1.54.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jersey-servlet-1.19.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/asm-5.0.4.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jackson-core-2.9.5.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/jetty-util-9.3.19.v20170502.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar:/root/ozone-0.3.0-SNAPSHOT/share/hadoop/common/lib/nimbus-jose-jwt-4.41.1.jar:/root/ozone-
[jira] [Commented] (HDFS-14045) Use different metrics in DataNode to better measure latency of heartbeat/blockReports/incrementalBlockReports of Active/Standby NN
[ https://issues.apache.org/jira/browse/HDFS-14045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16682726#comment-16682726 ] Jiandan Yang commented on HDFS-14045: -- TestNameNodeMXBean running failed is not related with this patch, I run successfully in my local machine. > Use different metrics in DataNode to better measure latency of > heartbeat/blockReports/incrementalBlockReports of Active/Standby NN > -- > > Key: HDFS-14045 > URL: https://issues.apache.org/jira/browse/HDFS-14045 > Project: Hadoop HDFS > Issue Type: Improvement > Components: datanode >Reporter: Jiandan Yang >Assignee: Jiandan Yang >Priority: Major > Attachments: HDFS-14045.001.patch, HDFS-14045.002.patch, > HDFS-14045.003.patch, HDFS-14045.004.patch, HDFS-14045.005.patch, > HDFS-14045.006.patch, HDFS-14045.007.patch, HDFS-14045.008.patch > > > Currently DataNode uses same metrics to measure rpc latency of NameNode, but > Active and Standby usually have different performance at the same time, > especially in large cluster. For example, rpc latency of Standby is very long > when Standby is catching up editlog. We may misunderstand the state of HDFS. > Using different metrics for Active and standby can help us obtain more > precise metric data. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-709) Modify Close Container handling sequence on datanodes
[
https://issues.apache.org/jira/browse/HDDS-709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16682668#comment-16682668
]
Hadoop QA commented on HDDS-709:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 21m
52s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
47s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
18m 14s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-ozone/integration-test {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
50s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
28s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
24s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 15m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 15m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 24s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-ozone/integration-test {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
27s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 45s{color}
| {color:red} common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 42s{color}
| {color:red} container-service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 37s{color}
| {color:red} client in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 42s{color}
| {color:red} integration-test in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
46s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}110m 48s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | HDDS-7
[jira] [Commented] (HDDS-709) Modify Close Container handling sequence on datanodes
[ https://issues.apache.org/jira/browse/HDDS-709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16682623#comment-16682623 ] Jitendra Nath Pandey commented on HDDS-709: --- Thanks for addressing the comments [~shashikant]. +1, LGTM. I have re-triggered jenkins because surefire plugin issue has been fixed. > Modify Close Container handling sequence on datanodes > - > > Key: HDDS-709 > URL: https://issues.apache.org/jira/browse/HDDS-709 > Project: Hadoop Distributed Data Store > Issue Type: Bug > Components: Ozone Datanode >Reporter: Shashikant Banerjee >Assignee: Shashikant Banerjee >Priority: Major > Attachments: HDDS-709.000.patch, HDDS-709.001.patch, > HDDS-709.002.patch, HDDS-709.003.patch, HDDS-709.004.patch, > HDDS-709.005.patch, HDDS-709.006.patch > > > With quasi closed container state for handling majority node failures, the > close container handling sequence in Datanodes need to change. Once the > datanodes receive a close container command from SCM, the open container > replicas individually be marked in the closing state. In a closing state, > only the transactions coming from the Ratis leader are allowed , all other > write transaction will fail. A close container transaction will be queued via > Ratis on the leader which will be replayed to the followers which makes it > transition to CLOSED/QUASI CLOSED state. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14017) ObserverReadProxyProviderWithIPFailover should work with HA configuration
[
https://issues.apache.org/jira/browse/HDFS-14017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16682588#comment-16682588
]
Konstantin Shvachko commented on HDFS-14017:
I meant nameserviceID, rather than namespace. Here is an example:
{code:java}
dfs.nameservices = virtual-address-nn.g.li.com
dfs.ha.namenodes.virtual-address-nn.g.li.com = nn1, nn2
dfs.namenode.rpc-address.virtual-address-nn.g.li.com.nn1 =
physical-address-ha1.g.li.com
dfs.namenode.rpc-address.virtual-address-nn.g.li.com.nn2 =
physical-address-ha2.g.li.com
{code}
Here {{virtual-address-nn.g.li.com}} plays the role of the nameserviceID, which
turns out to be the virtual address of NN. Does your patch works with that
configuration? We should really unit test this.
> ObserverReadProxyProviderWithIPFailover should work with HA configuration
> -
>
> Key: HDFS-14017
> URL: https://issues.apache.org/jira/browse/HDFS-14017
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Chen Liang
>Assignee: Chen Liang
>Priority: Major
> Attachments: HDFS-14017-HDFS-12943.001.patch,
> HDFS-14017-HDFS-12943.002.patch, HDFS-14017-HDFS-12943.003.patch,
> HDFS-14017-HDFS-12943.004.patch, HDFS-14017-HDFS-12943.005.patch,
> HDFS-14017-HDFS-12943.006.patch, HDFS-14017-HDFS-12943.008.patch
>
>
> Currently {{ObserverReadProxyProviderWithIPFailover}} extends
> {{ObserverReadProxyProvider}}, and the only difference is changing the proxy
> factory to use {{IPFailoverProxyProvider}}. However this is not enough
> because when calling constructor of {{ObserverReadProxyProvider}} in
> super(...), the follow line:
> {code:java}
> nameNodeProxies = getProxyAddresses(uri,
> HdfsClientConfigKeys.DFS_NAMENODE_RPC_ADDRESS_KEY);
> {code}
> will try to resolve the all configured NN addresses to do configured
> failover. But in the case of IPFailover, this does not really apply.
>
> A second issue closely related is about delegation token. For example, in
> current IPFailover setup, say we have a virtual host nn.xyz.com, which points
> to either of two physical nodes nn1.xyz.com or nn2.xyz.com. In current HDFS,
> there is always only one DT being exchanged, which has hostname nn.xyz.com.
> Server only issues this DT, and client only knows the host nn.xyz.com, so all
> is good. But in Observer read, even with IPFailover, the client will no
> longer contacting nn.xyz.com, but will actively reaching to nn1.xyz.com and
> nn2.xyz.com. During this process, current code will look for DT associated
> with hostname nn1.xyz.com or nn2.xyz.com, which is different from the DT
> given by NN. causing Token authentication to fail. This happens in
> {{AbstractDelegationTokenSelector#selectToken}}. New IPFailover proxy
> provider will need to resolve this as well.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-14017) ObserverReadProxyProviderWithIPFailover should work with HA configuration
[
https://issues.apache.org/jira/browse/HDFS-14017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16682588#comment-16682588
]
Konstantin Shvachko edited comment on HDFS-14017 at 11/10/18 9:08 PM:
--
I meant nameserviceID, rather than namespace. Here is an example:
{code:java}
dfs.nameservices = virtual-address-nn.g.li.com
dfs.ha.namenodes.virtual-address-nn.g.li.com = nn1, nn2
dfs.namenode.rpc-address.virtual-address-nn.g.li.com.nn1 =
physical-address-ha1.g.li.com
dfs.namenode.rpc-address.virtual-address-nn.g.li.com.nn2 =
physical-address-ha2.g.li.com
{code}
Here {{virtual-address-nn.g.li.com}} plays the role of the nameserviceID, which
turns out to be the virtual address of NN. Does your patch works with that
configuration? We should really unit test this.
So for ORPPwithIPF, the IPF will use {{virtual-address-nn.g.li.com}} as the
NN's virtual address, while the ORPP part will work with the physical addresses
{{physical-address-ha*.g.li.com}}.
was (Author: shv):
I meant nameserviceID, rather than namespace. Here is an example:
{code:java}
dfs.nameservices = virtual-address-nn.g.li.com
dfs.ha.namenodes.virtual-address-nn.g.li.com = nn1, nn2
dfs.namenode.rpc-address.virtual-address-nn.g.li.com.nn1 =
physical-address-ha1.g.li.com
dfs.namenode.rpc-address.virtual-address-nn.g.li.com.nn2 =
physical-address-ha2.g.li.com
{code}
Here {{virtual-address-nn.g.li.com}} plays the role of the nameserviceID, which
turns out to be the virtual address of NN. Does your patch works with that
configuration? We should really unit test this.
> ObserverReadProxyProviderWithIPFailover should work with HA configuration
> -
>
> Key: HDFS-14017
> URL: https://issues.apache.org/jira/browse/HDFS-14017
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Chen Liang
>Assignee: Chen Liang
>Priority: Major
> Attachments: HDFS-14017-HDFS-12943.001.patch,
> HDFS-14017-HDFS-12943.002.patch, HDFS-14017-HDFS-12943.003.patch,
> HDFS-14017-HDFS-12943.004.patch, HDFS-14017-HDFS-12943.005.patch,
> HDFS-14017-HDFS-12943.006.patch, HDFS-14017-HDFS-12943.008.patch
>
>
> Currently {{ObserverReadProxyProviderWithIPFailover}} extends
> {{ObserverReadProxyProvider}}, and the only difference is changing the proxy
> factory to use {{IPFailoverProxyProvider}}. However this is not enough
> because when calling constructor of {{ObserverReadProxyProvider}} in
> super(...), the follow line:
> {code:java}
> nameNodeProxies = getProxyAddresses(uri,
> HdfsClientConfigKeys.DFS_NAMENODE_RPC_ADDRESS_KEY);
> {code}
> will try to resolve the all configured NN addresses to do configured
> failover. But in the case of IPFailover, this does not really apply.
>
> A second issue closely related is about delegation token. For example, in
> current IPFailover setup, say we have a virtual host nn.xyz.com, which points
> to either of two physical nodes nn1.xyz.com or nn2.xyz.com. In current HDFS,
> there is always only one DT being exchanged, which has hostname nn.xyz.com.
> Server only issues this DT, and client only knows the host nn.xyz.com, so all
> is good. But in Observer read, even with IPFailover, the client will no
> longer contacting nn.xyz.com, but will actively reaching to nn1.xyz.com and
> nn2.xyz.com. During this process, current code will look for DT associated
> with hostname nn1.xyz.com or nn2.xyz.com, which is different from the DT
> given by NN. causing Token authentication to fail. This happens in
> {{AbstractDelegationTokenSelector#selectToken}}. New IPFailover proxy
> provider will need to resolve this as well.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-14017) ObserverReadProxyProviderWithIPFailover should work with HA configuration
[
https://issues.apache.org/jira/browse/HDFS-14017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16682168#comment-16682168
]
Konstantin Shvachko edited comment on HDFS-14017 at 11/10/18 8:52 PM:
--
Thanks for sharing [~shv].
What did you mean by "the virtual address of the NameName in current IFPP comes
from the _nameserviceID_"? How's that different from the current patch? maybe
an example? And yes, current IPProxy does not care about physical addresses,
that's the whole point of it. But in observer read, then are to we able to talk
to different NNs without knowing physical addresses, or are we only talking
about the failover part here?
cc. [~xkrogen]
was (Author: vagarychen):
Thanks for sharing [~shv].
What did you mean by "the virtual address of the NameName in current IFPP comes
from the namespaceID"? How's that different from the current patch? maybe an
example? And yes, current IPProxy does not care about physical addresses,
that's the whole point of it. But in observer read, then are to we able to talk
to different NNs without knowing physical addresses, or are we only talking
about the failover part here?
cc. [~xkrogen]
> ObserverReadProxyProviderWithIPFailover should work with HA configuration
> -
>
> Key: HDFS-14017
> URL: https://issues.apache.org/jira/browse/HDFS-14017
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Chen Liang
>Assignee: Chen Liang
>Priority: Major
> Attachments: HDFS-14017-HDFS-12943.001.patch,
> HDFS-14017-HDFS-12943.002.patch, HDFS-14017-HDFS-12943.003.patch,
> HDFS-14017-HDFS-12943.004.patch, HDFS-14017-HDFS-12943.005.patch,
> HDFS-14017-HDFS-12943.006.patch, HDFS-14017-HDFS-12943.008.patch
>
>
> Currently {{ObserverReadProxyProviderWithIPFailover}} extends
> {{ObserverReadProxyProvider}}, and the only difference is changing the proxy
> factory to use {{IPFailoverProxyProvider}}. However this is not enough
> because when calling constructor of {{ObserverReadProxyProvider}} in
> super(...), the follow line:
> {code:java}
> nameNodeProxies = getProxyAddresses(uri,
> HdfsClientConfigKeys.DFS_NAMENODE_RPC_ADDRESS_KEY);
> {code}
> will try to resolve the all configured NN addresses to do configured
> failover. But in the case of IPFailover, this does not really apply.
>
> A second issue closely related is about delegation token. For example, in
> current IPFailover setup, say we have a virtual host nn.xyz.com, which points
> to either of two physical nodes nn1.xyz.com or nn2.xyz.com. In current HDFS,
> there is always only one DT being exchanged, which has hostname nn.xyz.com.
> Server only issues this DT, and client only knows the host nn.xyz.com, so all
> is good. But in Observer read, even with IPFailover, the client will no
> longer contacting nn.xyz.com, but will actively reaching to nn1.xyz.com and
> nn2.xyz.com. During this process, current code will look for DT associated
> with hostname nn1.xyz.com or nn2.xyz.com, which is different from the DT
> given by NN. causing Token authentication to fail. This happens in
> {{AbstractDelegationTokenSelector#selectToken}}. New IPFailover proxy
> provider will need to resolve this as well.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-14017) ObserverReadProxyProviderWithIPFailover should work with HA configuration
[
https://issues.apache.org/jira/browse/HDFS-14017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16682145#comment-16682145
]
Konstantin Shvachko edited comment on HDFS-14017 at 11/10/18 8:51 PM:
--
Had an offline discussion with Erik. The gist of the problem is that the
virtual address of the NameName in current IFPP comes from the _nameserviceID_,
and it doesn't look further into physical addresses of the NameNodes. So if we
want to keep this behavior, we should do the same for ORPPWithIPF.
I suggest that we double-check this is the case. If so, Erik says it is, let's
go this route, even though it seems hacky, but let's finally document this
properly.
was (Author: shv):
Had an offline discussion with Erik. The gist of the problem is that the
virtual address of the NameName in current IFPP comes from the namespaceID, and
it doesn't look further into physical addresses of the NameNodes. So if we want
to keep this behavior, we should do the same for ORPPWithIPF.
I suggest that we double-check this is the case. If so, Erik says it is, let's
go this route, even though it seems hacky, but let's finally document this
properly.
> ObserverReadProxyProviderWithIPFailover should work with HA configuration
> -
>
> Key: HDFS-14017
> URL: https://issues.apache.org/jira/browse/HDFS-14017
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Chen Liang
>Assignee: Chen Liang
>Priority: Major
> Attachments: HDFS-14017-HDFS-12943.001.patch,
> HDFS-14017-HDFS-12943.002.patch, HDFS-14017-HDFS-12943.003.patch,
> HDFS-14017-HDFS-12943.004.patch, HDFS-14017-HDFS-12943.005.patch,
> HDFS-14017-HDFS-12943.006.patch, HDFS-14017-HDFS-12943.008.patch
>
>
> Currently {{ObserverReadProxyProviderWithIPFailover}} extends
> {{ObserverReadProxyProvider}}, and the only difference is changing the proxy
> factory to use {{IPFailoverProxyProvider}}. However this is not enough
> because when calling constructor of {{ObserverReadProxyProvider}} in
> super(...), the follow line:
> {code:java}
> nameNodeProxies = getProxyAddresses(uri,
> HdfsClientConfigKeys.DFS_NAMENODE_RPC_ADDRESS_KEY);
> {code}
> will try to resolve the all configured NN addresses to do configured
> failover. But in the case of IPFailover, this does not really apply.
>
> A second issue closely related is about delegation token. For example, in
> current IPFailover setup, say we have a virtual host nn.xyz.com, which points
> to either of two physical nodes nn1.xyz.com or nn2.xyz.com. In current HDFS,
> there is always only one DT being exchanged, which has hostname nn.xyz.com.
> Server only issues this DT, and client only knows the host nn.xyz.com, so all
> is good. But in Observer read, even with IPFailover, the client will no
> longer contacting nn.xyz.com, but will actively reaching to nn1.xyz.com and
> nn2.xyz.com. During this process, current code will look for DT associated
> with hostname nn1.xyz.com or nn2.xyz.com, which is different from the DT
> given by NN. causing Token authentication to fail. This happens in
> {{AbstractDelegationTokenSelector#selectToken}}. New IPFailover proxy
> provider will need to resolve this as well.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14045) Use different metrics in DataNode to better measure latency of heartbeat/blockReports/incrementalBlockReports of Active/Standby NN
[
https://issues.apache.org/jira/browse/HDFS-14045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16682524#comment-16682524
]
Hadoop QA commented on HDFS-14045:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
22s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
4s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 22m
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 26m
35s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 4m
0s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
55s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
21m 33s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
43s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
25s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 24m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 24m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 1s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
26s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 11m
7s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}100m 58s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
48s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}251m 56s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.server.namenode.TestNameNodeMXBean |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | HDFS-14045 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12947700/HDFS-14045.008.patch |
| Optional Tests | dupname asflicense mvnsite compile javac javadoc
mvninstall unit shadedclient findbugs checkstyle |
| uname | Linux 4d0040dc557a 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14
08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 2664248 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/2547
[jira] [Commented] (HDFS-13998) ECAdmin NPE with -setPolicy -replicate
[
https://issues.apache.org/jira/browse/HDFS-13998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16682523#comment-16682523
]
Hadoop QA commented on HDFS-13998:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 14m
22s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 21m
3s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
57s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
2s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 42s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
52s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
47s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 2s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 93m
31s{color} | {color:green} hadoop-hdfs in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
33s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}167m 50s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | HDFS-13998 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12947706/HDFS-13998.03.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle xml |
| uname | Linux 77066ff784cc 3.13.0-143-generic #192-Ubuntu SMP Tue Feb 27
10:45:36 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 2664248 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/25480/testReport/ |
| Max. process+thread count | 2748 (vs. ulimit of 1) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/25480/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> ECAdmin
[jira] [Updated] (HDDS-692) Use the ProgressBar class in the RandomKeyGenerator freon test
[ https://issues.apache.org/jira/browse/HDDS-692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zsolt Horvath updated HDDS-692: --- Status: Open (was: Patch Available) > Use the ProgressBar class in the RandomKeyGenerator freon test > -- > > Key: HDDS-692 > URL: https://issues.apache.org/jira/browse/HDDS-692 > Project: Hadoop Distributed Data Store > Issue Type: Bug > Components: Tools >Reporter: Elek, Marton >Assignee: Zsolt Horvath >Priority: Major > Attachments: HDDS-692.001.patch, HDDS-692.002.patch, > HDDS-692.003.patch, HDDS-692.004.patch > > > HDDS-443 provides a reusable progress bar to make it easier to add more freon > tests, but the existing RandomKeyGenerator test > (hadoop-ozone/tools/src/main/java/org/apache/hadoop/ozone/freon/RandomKeyGenerator.java) > still doesn't use it. > It would be good to switch to use the new progress bar there. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-164) Add unit test for HddsDatanodeService
[
https://issues.apache.org/jira/browse/HDDS-164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16682460#comment-16682460
]
Hadoop QA commented on HDDS-164:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
24s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 23m
5s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
31s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
32s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 48s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 21s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 46s{color}
| {color:red} container-service in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
27s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 56m 36s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.ozone.TestHddsDatanodeService |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | HDDS-164 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12947707/HDDS-164.001.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 2086e337cdf1 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14
08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 2664248 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_181 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-HDDS-Build/1675/artifact/out/patch-unit-hadoop-hdds_container-service.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDDS-Build/1675/testReport/ |
| Max. process+thread count | 336 (vs. ulimit of 1) |
| modules | C: hadoop-hdds/container-service U: hadoop-hdds/container-service |
| Console output |
https://builds.apache.org/job/PreCommit-HDDS-Build/1675/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message
[jira] [Updated] (HDDS-164) Add unit test for HddsDatanodeService
[
https://issues.apache.org/jira/browse/HDDS-164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiqun Lin updated HDDS-164:
---
Status: Patch Available (was: Open)
Attach the patch for adding the UT.
[~nandakumar131], Please have a look.
> Add unit test for HddsDatanodeService
> -
>
> Key: HDDS-164
> URL: https://issues.apache.org/jira/browse/HDDS-164
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
> Components: Ozone Datanode
>Reporter: Nanda kumar
>Assignee: Yiqun Lin
>Priority: Major
> Labels: newbie, test
> Attachments: HDDS-164.001.patch
>
>
> We have to add unit-test for {{HddsDatanodeService}} class.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDDS-164) Add unit test for HddsDatanodeService
[
https://issues.apache.org/jira/browse/HDDS-164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiqun Lin reassigned HDDS-164:
--
Assignee: Yiqun Lin
> Add unit test for HddsDatanodeService
> -
>
> Key: HDDS-164
> URL: https://issues.apache.org/jira/browse/HDDS-164
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
> Components: Ozone Datanode
>Reporter: Nanda kumar
>Assignee: Yiqun Lin
>Priority: Major
> Labels: newbie, test
>
> We have to add unit-test for {{HddsDatanodeService}} class.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDDS-164) Add unit test for HddsDatanodeService
[
https://issues.apache.org/jira/browse/HDDS-164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiqun Lin updated HDDS-164:
---
Attachment: HDDS-164.001.patch
> Add unit test for HddsDatanodeService
> -
>
> Key: HDDS-164
> URL: https://issues.apache.org/jira/browse/HDDS-164
> Project: Hadoop Distributed Data Store
> Issue Type: Sub-task
> Components: Ozone Datanode
>Reporter: Nanda kumar
>Assignee: Yiqun Lin
>Priority: Major
> Labels: newbie, test
> Attachments: HDDS-164.001.patch
>
>
> We have to add unit-test for {{HddsDatanodeService}} class.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13998) ECAdmin NPE with -setPolicy -replicate
[
https://issues.apache.org/jira/browse/HDFS-13998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16682428#comment-16682428
]
Zsolt Venczel commented on HDFS-13998:
--
Thanks for the review [~templedf]!
As I can see the extraneous " " was there for some time now and it was added
due to the line length checkstyle limitation.
I reshuffled this code section a bit to look prettier.
> ECAdmin NPE with -setPolicy -replicate
> --
>
> Key: HDFS-13998
> URL: https://issues.apache.org/jira/browse/HDFS-13998
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: erasure-coding
>Affects Versions: 3.2.0, 3.1.2
>Reporter: Xiao Chen
>Assignee: Zsolt Venczel
>Priority: Major
> Attachments: HDFS-13998.01.patch, HDFS-13998.02.patch,
> HDFS-13998.03.patch
>
>
> HDFS-13732 tried to improve the output of the console tool. But we missed the
> fact that for replication, {{getErasureCodingPolicy}} would return null.
> This jira is to fix it in ECAdmin, and add a unit test.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-13998) ECAdmin NPE with -setPolicy -replicate
[
https://issues.apache.org/jira/browse/HDFS-13998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zsolt Venczel updated HDFS-13998:
-
Attachment: HDFS-13998.03.patch
> ECAdmin NPE with -setPolicy -replicate
> --
>
> Key: HDFS-13998
> URL: https://issues.apache.org/jira/browse/HDFS-13998
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: erasure-coding
>Affects Versions: 3.2.0, 3.1.2
>Reporter: Xiao Chen
>Assignee: Zsolt Venczel
>Priority: Major
> Attachments: HDFS-13998.01.patch, HDFS-13998.02.patch,
> HDFS-13998.03.patch
>
>
> HDFS-13732 tried to improve the output of the console tool. But we missed the
> fact that for replication, {{getErasureCodingPolicy}} would return null.
> This jira is to fix it in ECAdmin, and add a unit test.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-14045) Use different metrics in DataNode to better measure latency of heartbeat/blockReports/incrementalBlockReports of Active/Standby NN
[ https://issues.apache.org/jira/browse/HDFS-14045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jiandan Yang updated HDFS-14045: - Attachment: HDFS-14045.008.patch > Use different metrics in DataNode to better measure latency of > heartbeat/blockReports/incrementalBlockReports of Active/Standby NN > -- > > Key: HDFS-14045 > URL: https://issues.apache.org/jira/browse/HDFS-14045 > Project: Hadoop HDFS > Issue Type: Improvement > Components: datanode >Reporter: Jiandan Yang >Assignee: Jiandan Yang >Priority: Major > Attachments: HDFS-14045.001.patch, HDFS-14045.002.patch, > HDFS-14045.003.patch, HDFS-14045.004.patch, HDFS-14045.005.patch, > HDFS-14045.006.patch, HDFS-14045.007.patch, HDFS-14045.008.patch > > > Currently DataNode uses same metrics to measure rpc latency of NameNode, but > Active and Standby usually have different performance at the same time, > especially in large cluster. For example, rpc latency of Standby is very long > when Standby is catching up editlog. We may misunderstand the state of HDFS. > Using different metrics for Active and standby can help us obtain more > precise metric data. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14045) Use different metrics in DataNode to better measure latency of heartbeat/blockReports/incrementalBlockReports of Active/Standby NN
[
https://issues.apache.org/jira/browse/HDFS-14045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16682395#comment-16682395
]
Jiandan Yang commented on HDFS-14045:
--
Thanks [~xkrogen] for your comments very much.
{quote}
Can we change the same of the method/parameter to something indicating it is
for metrics only, maybe like nnLatencyMetricsSuffix? It looks particularly odd
to me in IncrementalBlockReportManager right now.
{quote}
I rename {{nnLatencyMetricsSuffix}} into {{rpcMetricSuffix}}, what do you
think of this name?
{quote}
I think I would prefer to see the existing methods in DataNodeMetrics changed
to update both metrics, rather than the caller having to remember to call both
methods. It introduces less possibility for the two metrics to get out of sync
later.
{quote}
Very good suggestion, I have changed to update both metrics at one method in
patch008, but serviceId-nnId is needed when updating metric, so there is need
to add a parameter as suffix of metrics in the existing methods.
{quote}
I'm not sure if you should re-use the same MutableRatesWithAggregation for all
of the metrics. It seems cleaner to me to have one per metric type, e.g. one
for heartbeats, one for lifeline, and so on, but let me know if you disagree. I
think this may even make it so that, if you set up the names correctly, the
MutableRatesWithAggregation can replace the existing MutableRate while
maintaining the name of the metric. Not 100% sure on this.
{quote}
I prefer to re-use MutableRatesWithAggregation for simplicity, it does not need
to add fields when adding new metrics.
{quote}
You should update Metrics.md documenting these new metrics
{quote}
Thanks for reminding to modify Metrics.md, and newly added metrics have been
written to Metrics.md in patch008
> Use different metrics in DataNode to better measure latency of
> heartbeat/blockReports/incrementalBlockReports of Active/Standby NN
> --
>
> Key: HDFS-14045
> URL: https://issues.apache.org/jira/browse/HDFS-14045
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
>Reporter: Jiandan Yang
>Assignee: Jiandan Yang
>Priority: Major
> Attachments: HDFS-14045.001.patch, HDFS-14045.002.patch,
> HDFS-14045.003.patch, HDFS-14045.004.patch, HDFS-14045.005.patch,
> HDFS-14045.006.patch, HDFS-14045.007.patch
>
>
> Currently DataNode uses same metrics to measure rpc latency of NameNode, but
> Active and Standby usually have different performance at the same time,
> especially in large cluster. For example, rpc latency of Standby is very long
> when Standby is catching up editlog. We may misunderstand the state of HDFS.
> Using different metrics for Active and standby can help us obtain more
> precise metric data.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-13101) Yet another fsimage corruption related to snapshot
[ https://issues.apache.org/jira/browse/HDFS-13101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16682387#comment-16682387 ] Wei-Chiu Chuang commented on HDFS-13101: A little bit of updates: After HDFS-13314 was applied, we were finally able to identify a set of fsimages + edits that reproduced corruption reliably. Spoiler: the fsimage was already partially corrupted before HDFS-13314 aborted check point. So it took us a little while to understand other sequences of events leading to the null pointer. The gist of the issue is a delete trash operation followed by delete snapshots. The fix for this bug is still undergoing. [~sodonnell], [~smeng] and [~adam.antal] have put in tremendous number of hours on it, and they've been actively working on the reproduction in the past few weeks. [~yzhangal] mind if I reassign this jira to [~smeng] for tracking purposes? > Yet another fsimage corruption related to snapshot > -- > > Key: HDFS-13101 > URL: https://issues.apache.org/jira/browse/HDFS-13101 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Yongjun Zhang >Assignee: Yongjun Zhang >Priority: Major > > Lately we saw case similar to HDFS-9406, even though HDFS-9406 fix is > present, so it's likely another case not covered by the fix. We are currently > trying to collect good fsimage + editlogs to replay to reproduce it and > investigate. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-709) Modify Close Container handling sequence on datanodes
[
https://issues.apache.org/jira/browse/HDDS-709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16682285#comment-16682285
]
Hadoop QA commented on HDDS-709:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
15s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 21m
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
3s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
16m 26s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-ozone/integration-test {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
47s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
23s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 15m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 15m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 15m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 30s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-ozone/integration-test {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
23s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 46s{color}
| {color:red} common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 42s{color}
| {color:red} container-service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 37s{color}
| {color:red} client in the patch failed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 41s{color}
| {color:red} integration-test in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
49s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}106m 56s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | HDDS-7
