[jira] [Commented] (HDFS-12180) Ozone: Corona: Add stats and progress bar to corona
[
https://issues.apache.org/jira/browse/HDFS-12180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097464#comment-16097464
]
Hadoop QA commented on HDFS-12180:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
28s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} HDFS-7240 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
40s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
53s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
57s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
55s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} HDFS-7240 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 66m 55s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 94m 55s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.ozone.scm.TestContainerSQLCli |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure150 |
| | hadoop.hdfs.server.namenode.TestDecommissioningStatus |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HDFS-12180 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12878518/HDFS-12180-HDFS-7240.001.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux e0301a7de26f 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | HDFS-7240 / c539095 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/20392/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/20392/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/20392/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Ozone: Corona: Add stats and progress bar to corona
> ---
>
> Key: HDFS-12180
> URL: https://issues.apache.org/jira/browse/HDFS-12180
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>
[jira] [Commented] (HDFS-12180) Ozone: Corona: Add stats and progress bar to corona
[ https://issues.apache.org/jira/browse/HDFS-12180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097439#comment-16097439 ] Nandakumar commented on HDFS-12180: --- patch v1 with checkstyle fixes. > Ozone: Corona: Add stats and progress bar to corona > --- > > Key: HDFS-12180 > URL: https://issues.apache.org/jira/browse/HDFS-12180 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Reporter: Nandakumar >Assignee: Nandakumar > Attachments: HDFS-12180-HDFS-7240.000.patch, > HDFS-12180-HDFS-7240.001.patch > > > More statistics to be added to corona run summary and also progress bar which > shows the status of run. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12180) Ozone: Corona: Add stats and progress bar to corona
[ https://issues.apache.org/jira/browse/HDFS-12180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandakumar updated HDFS-12180: -- Attachment: HDFS-12180-HDFS-7240.001.patch > Ozone: Corona: Add stats and progress bar to corona > --- > > Key: HDFS-12180 > URL: https://issues.apache.org/jira/browse/HDFS-12180 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Reporter: Nandakumar >Assignee: Nandakumar > Attachments: HDFS-12180-HDFS-7240.000.patch, > HDFS-12180-HDFS-7240.001.patch > > > More statistics to be added to corona run summary and also progress bar which > shows the status of run. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12180) Ozone: Corona: Add stats and progress bar to corona
[
https://issues.apache.org/jira/browse/HDFS-12180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097421#comment-16097421
]
Hadoop QA commented on HDFS-12180:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
14s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} HDFS-7240 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
14s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
53s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
58s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
51s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} HDFS-7240 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 34s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs: The patch
generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 67m 19s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 95m 35s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080 |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure150 |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HDFS-12180 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12878512/HDFS-12180-HDFS-7240.000.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 201e9f1b4ff4 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | HDFS-7240 / c539095 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/20391/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/20391/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/20391/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/20391/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Ozone: Corona: Add stats and progress bar to corona
> ---
>
>
[jira] [Updated] (HDFS-12180) Ozone: Corona: Add stats and progress bar to corona
[ https://issues.apache.org/jira/browse/HDFS-12180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandakumar updated HDFS-12180: -- Status: Patch Available (was: Open) > Ozone: Corona: Add stats and progress bar to corona > --- > > Key: HDFS-12180 > URL: https://issues.apache.org/jira/browse/HDFS-12180 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Reporter: Nandakumar >Assignee: Nandakumar > Attachments: HDFS-12180-HDFS-7240.000.patch > > > More statistics to be added to corona run summary and also progress bar which > shows the status of run. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12180) Ozone: Corona: Add stats and progress bar to corona
[ https://issues.apache.org/jira/browse/HDFS-12180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandakumar updated HDFS-12180: -- Attachment: HDFS-12180-HDFS-7240.000.patch > Ozone: Corona: Add stats and progress bar to corona > --- > > Key: HDFS-12180 > URL: https://issues.apache.org/jira/browse/HDFS-12180 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Reporter: Nandakumar >Assignee: Nandakumar > Attachments: HDFS-12180-HDFS-7240.000.patch > > > More statistics to be added to corona run summary and also progress bar which > shows the status of run. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12167) Ozone: Intermittent failure TestContainerPersistence#testListKey
[ https://issues.apache.org/jira/browse/HDFS-12167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097393#comment-16097393 ] Anu Engineer commented on HDFS-12167: - [~surendrasingh] Welcome to Ozone !, Thank you for the contribution. > Ozone: Intermittent failure TestContainerPersistence#testListKey > > > Key: HDFS-12167 > URL: https://issues.apache.org/jira/browse/HDFS-12167 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone, test >Affects Versions: HDFS-7240 >Reporter: Weiwei Yang >Assignee: Surendra Singh Lilhore >Priority: Minor > Fix For: HDFS-7240 > > Attachments: HDFS-12167-HDFS-7240.001.patch > > > TestContainerPersistence#listKeys seems to fail intermittently. It looks > like it was failing because some unexpected format of container name. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Resolved] (HDFS-11176) httpfs.sh run fails due to missing temp dir
[
https://issues.apache.org/jira/browse/HDFS-11176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Zhuge resolved HDFS-11176.
---
Resolution: Duplicate
Fix Version/s: 3.0.0-alpha4
Fixed by HDFS-10860:
https://github.com/apache/hadoop/blob/69b23632c48297ac844c58fd3e21aad10c093cc8/hadoop-hdfs-project/hadoop-hdfs-httpfs/src/main/libexec/shellprofile.d/hadoop-httpfs.sh#L65.
> httpfs.sh run fails due to missing temp dir
> ---
>
> Key: HDFS-11176
> URL: https://issues.apache.org/jira/browse/HDFS-11176
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: httpfs
>Affects Versions: 3.0.0-alpha2
>Reporter: John Zhuge
>Assignee: John Zhuge
>Priority: Minor
> Fix For: 3.0.0-alpha4
>
>
> Got this error when running {{httpfs.sh run}} in Pseudo-Distributed mode:
> {noformat}
> Nov 25, 2016 1:27:33 PM org.apache.catalina.core.ApplicationContext log
> INFO: ERROR: S01: Dir
> [/Users/jzhuge/hadoop2/hadoop-dist/target/hadoop-3.0.0-alpha2-SNAPSHOT/temp]
> does not exist
> Nov 25, 2016 1:27:33 PM org.apache.catalina.core.StandardContext listenerStart
> SEVERE: Exception sending context initialized event to listener instance of
> class org.apache.hadoop.fs.http.server.HttpFSServerWebApp
> java.lang.RuntimeException: org.apache.hadoop.lib.server.ServerException:
> S01: Dir
> [/Users/jzhuge/hadoop2/hadoop-dist/target/hadoop-3.0.0-alpha2-SNAPSHOT/temp]
> does not exist
> at
> org.apache.hadoop.lib.servlet.ServerWebApp.contextInitialized(ServerWebApp.java:161)
> at
> org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4276)
> at
> org.apache.catalina.core.StandardContext.start(StandardContext.java:4779)
> at
> org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:803)
> at
> org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:780)
> at
> org.apache.catalina.core.StandardHost.addChild(StandardHost.java:583)
> at
> org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:1080)
> at
> org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:1003)
> at
> org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:507)
> at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1322)
> at
> org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:325)
> at
> org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:142)
> at
> org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1069)
> at org.apache.catalina.core.StandardHost.start(StandardHost.java:822)
> at
> org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1061)
> at
> org.apache.catalina.core.StandardEngine.start(StandardEngine.java:463)
> at
> org.apache.catalina.core.StandardService.start(StandardService.java:525)
> at
> org.apache.catalina.core.StandardServer.start(StandardServer.java:761)
> at org.apache.catalina.startup.Catalina.start(Catalina.java:595)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:289)
> at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:414)
> Caused by: org.apache.hadoop.lib.server.ServerException: S01: Dir
> [/Users/jzhuge/hadoop2/hadoop-dist/target/hadoop-3.0.0-alpha2-SNAPSHOT/temp]
> does not exist
> at org.apache.hadoop.lib.server.Server.verifyDir(Server.java:400)
> at org.apache.hadoop.lib.server.Server.init(Server.java:349)
> at
> org.apache.hadoop.fs.http.server.HttpFSServerWebApp.init(HttpFSServerWebApp.java:100)
> at
> org.apache.hadoop.lib.servlet.ServerWebApp.contextInitialized(ServerWebApp.java:158)
> ... 24 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12176) dfsadmin shows DFS Used%: NaN% if the cluster has zero block.
[
https://issues.apache.org/jira/browse/HDFS-12176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097380#comment-16097380
]
Wei-Chiu Chuang commented on HDFS-12176:
Wanted to commit the patch, but there's a mvn compilation issue (looks like
HADOOP-14677).
I will be off the following few days so I'll commit the patch when I am back on
line (or someone else can commit for me:) )
> dfsadmin shows DFS Used%: NaN% if the cluster has zero block.
> -
>
> Key: HDFS-12176
> URL: https://issues.apache.org/jira/browse/HDFS-12176
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Wei-Chiu Chuang
>Assignee: Weiwei Yang
>Priority: Trivial
> Attachments: HDFS-12176.001.patch
>
>
> This is rather a non-issue, but thought I should file it anyway.
> I have a test cluster with just NN fsimage, no DN, no blocks, and dfsadmin
> shows:
> {noformat}
> $ hdfs dfsadmin -report
> Configured Capacity: 0 (0 B)
> Present Capacity: 0 (0 B)
> DFS Remaining: 0 (0 B)
> DFS Used: 0 (0 B)
> DFS Used%: NaN%
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12176) dfsadmin shows DFS Used%: NaN% if the cluster has zero block.
[
https://issues.apache.org/jira/browse/HDFS-12176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097364#comment-16097364
]
Wei-Chiu Chuang commented on HDFS-12176:
+1
> dfsadmin shows DFS Used%: NaN% if the cluster has zero block.
> -
>
> Key: HDFS-12176
> URL: https://issues.apache.org/jira/browse/HDFS-12176
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Wei-Chiu Chuang
>Assignee: Weiwei Yang
>Priority: Trivial
> Attachments: HDFS-12176.001.patch
>
>
> This is rather a non-issue, but thought I should file it anyway.
> I have a test cluster with just NN fsimage, no DN, no blocks, and dfsadmin
> shows:
> {noformat}
> $ hdfs dfsadmin -report
> Configured Capacity: 0 (0 B)
> Present Capacity: 0 (0 B)
> DFS Remaining: 0 (0 B)
> DFS Used: 0 (0 B)
> DFS Used%: NaN%
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12145) Ozone: OzoneFileSystem: Ozone & KSM should support "/" delimited key names
[
https://issues.apache.org/jira/browse/HDFS-12145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097350#comment-16097350
]
Weiwei Yang commented on HDFS-12145:
Hi [~msingh]
Thanks for the update, but I was meant to say that we could add an argument for
{{PutHelper}} so it can put a specific key name, such as
{code}
PutHelper(OzoneRestClient client, String dir, String keyName);
{code}
Then in test {{testPutKey()}}, we could add
{code}
runTestPutKey(new PutHelper(ozoneRestClient,
path, OzoneUtils.getRequestID().toLowerCase()));
runTestPutKey(new PutHelper(ozoneRestClient,
path, getRandomKeyDelimitedBy("/")));
{code}
so that we can test both delimited and non-delimited keys.
Here I think we can rename {{getMultiPartKey()}} to
{{getRandomKeyDelimitedBy()}}, this method allows you to get a key name with a
specific delimiter.
{code}
String getRandomKeyDelimitedBy(String delimiter);
{code}
Does that make sense to you? Please let me know.
Thanks a lot.
> Ozone: OzoneFileSystem: Ozone & KSM should support "/" delimited key names
> --
>
> Key: HDFS-12145
> URL: https://issues.apache.org/jira/browse/HDFS-12145
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
>Affects Versions: HDFS-7240
>Reporter: Mukul Kumar Singh
>Assignee: Mukul Kumar Singh
> Fix For: HDFS-7240
>
> Attachments: HDFS-12145-HDFS-7240.001.patch,
> HDFS-12145-HDFS-7240.002.patch, HDFS-12145-HDFS-7240.003.patch,
> HDFS-12145-HDFS-7240.004.patch
>
>
> With OzoneFileSystem, key names will be delimited by "/" which is used as the
> path separator.
> Support should be added in KSM and Ozone to support key name with "/"
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12167) Ozone: Intermittent failure TestContainerPersistence#testListKey
[ https://issues.apache.org/jira/browse/HDFS-12167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097342#comment-16097342 ] Weiwei Yang commented on HDFS-12167: I have committed this to the feature branch, thanks [~surendrasingh] for the contribution! > Ozone: Intermittent failure TestContainerPersistence#testListKey > > > Key: HDFS-12167 > URL: https://issues.apache.org/jira/browse/HDFS-12167 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone, test >Affects Versions: HDFS-7240 >Reporter: Weiwei Yang >Assignee: Surendra Singh Lilhore >Priority: Minor > Fix For: HDFS-7240 > > Attachments: HDFS-12167-HDFS-7240.001.patch > > > TestContainerPersistence#listKeys seems to fail intermittently. It looks > like it was failing because some unexpected format of container name. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12167) Ozone: Intermittent failure TestContainerPersistence#testListKey
[ https://issues.apache.org/jira/browse/HDFS-12167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang updated HDFS-12167: --- Affects Version/s: HDFS-7240 > Ozone: Intermittent failure TestContainerPersistence#testListKey > > > Key: HDFS-12167 > URL: https://issues.apache.org/jira/browse/HDFS-12167 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone, test >Affects Versions: HDFS-7240 >Reporter: Weiwei Yang >Assignee: Surendra Singh Lilhore >Priority: Minor > Fix For: HDFS-7240 > > Attachments: HDFS-12167-HDFS-7240.001.patch > > > TestContainerPersistence#listKeys seems to fail intermittently. It looks > like it was failing because some unexpected format of container name. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12167) Ozone: Intermittent failure TestContainerPersistence#testListKey
[ https://issues.apache.org/jira/browse/HDFS-12167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang updated HDFS-12167: --- Resolution: Fixed Fix Version/s: HDFS-7240 Tags: test Status: Resolved (was: Patch Available) > Ozone: Intermittent failure TestContainerPersistence#testListKey > > > Key: HDFS-12167 > URL: https://issues.apache.org/jira/browse/HDFS-12167 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone, test >Affects Versions: HDFS-7240 >Reporter: Weiwei Yang >Assignee: Surendra Singh Lilhore >Priority: Minor > Fix For: HDFS-7240 > > Attachments: HDFS-12167-HDFS-7240.001.patch > > > TestContainerPersistence#listKeys seems to fail intermittently. It looks > like it was failing because some unexpected format of container name. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12167) Ozone: Intermittent failure TestContainerPersistence#testListKey
[ https://issues.apache.org/jira/browse/HDFS-12167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang updated HDFS-12167: --- Tags: (was: test) > Ozone: Intermittent failure TestContainerPersistence#testListKey > > > Key: HDFS-12167 > URL: https://issues.apache.org/jira/browse/HDFS-12167 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone, test >Affects Versions: HDFS-7240 >Reporter: Weiwei Yang >Assignee: Surendra Singh Lilhore >Priority: Minor > Fix For: HDFS-7240 > > Attachments: HDFS-12167-HDFS-7240.001.patch > > > TestContainerPersistence#listKeys seems to fail intermittently. It looks > like it was failing because some unexpected format of container name. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12176) dfsadmin shows DFS Used%: NaN% if the cluster has zero block.
[
https://issues.apache.org/jira/browse/HDFS-12176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097337#comment-16097337
]
Weiwei Yang commented on HDFS-12176:
Thanks [~hkoneru] for the review.
Hi [~jojochuang], does the change look good to you?
> dfsadmin shows DFS Used%: NaN% if the cluster has zero block.
> -
>
> Key: HDFS-12176
> URL: https://issues.apache.org/jira/browse/HDFS-12176
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: Wei-Chiu Chuang
>Assignee: Weiwei Yang
>Priority: Trivial
> Attachments: HDFS-12176.001.patch
>
>
> This is rather a non-issue, but thought I should file it anyway.
> I have a test cluster with just NN fsimage, no DN, no blocks, and dfsadmin
> shows:
> {noformat}
> $ hdfs dfsadmin -report
> Configured Capacity: 0 (0 B)
> Present Capacity: 0 (0 B)
> DFS Remaining: 0 (0 B)
> DFS Used: 0 (0 B)
> DFS Used%: NaN%
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12167) Ozone: Intermittent failure TestContainerPersistence#testListKey
[ https://issues.apache.org/jira/browse/HDFS-12167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097336#comment-16097336 ] Weiwei Yang commented on HDFS-12167: Thanks [~surendrasingh] for fixing this issue, the change looks good to me, I am going to commit this shortly. > Ozone: Intermittent failure TestContainerPersistence#testListKey > > > Key: HDFS-12167 > URL: https://issues.apache.org/jira/browse/HDFS-12167 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone, test >Reporter: Weiwei Yang >Assignee: Surendra Singh Lilhore >Priority: Minor > Attachments: HDFS-12167-HDFS-7240.001.patch > > > TestContainerPersistence#listKeys seems to fail intermittently. It looks > like it was failing because some unexpected format of container name. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12167) Ozone: Intermittent failure TestContainerPersistence#testListKey
[
https://issues.apache.org/jira/browse/HDFS-12167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097224#comment-16097224
]
Hadoop QA commented on HDFS-12167:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
10s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} HDFS-7240 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
4s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
15s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
13s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} HDFS-7240 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 78m 41s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}108m 29s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080 |
| | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting |
| | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure |
| | hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery
|
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HDFS-12167 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12878462/HDFS-12167-HDFS-7240.001.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 4b5b91f3bb80 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26
18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | HDFS-7240 / 63e211f |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/20390/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/20390/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://builds.apache.org/job/PreCommit-HDFS-Build/20390/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Ozone: Intermittent failure TestContainerPersistence#testListKey
>
>
> Key: HDFS-12167
> URL:
[jira] [Commented] (HDFS-6984) In Hadoop 3, make FileStatus serialize itself via protobuf
[
https://issues.apache.org/jira/browse/HDFS-6984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097205#comment-16097205
]
Hadoop QA commented on HDFS-6984:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 10 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
44s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 15m
6s{color} | {color:red} root in trunk failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m
54s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
1s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 4m
24s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
28s{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs-client in trunk has 2
extant Findbugs warnings. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
47s{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs in trunk has 10 extant
Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
22s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 11m
20s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 11m 20s{color}
| {color:red} root generated 80 new + 1313 unchanged - 0 fixed = 1393 total
(was 1313) {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
2m 4s{color} | {color:orange} root: The patch generated 12 new + 824 unchanged
- 22 fixed = 836 total (was 846) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 4m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
4s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
38s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
51s{color} | {color:green} hadoop-hdfs-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
15s{color} | {color:green} hadoop-hdfs-project/hadoop-hdfs generated 0 new + 9
unchanged - 1 fixed = 9 total (was 10) {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
41s{color} | {color:green} hadoop-hdfs-httpfs in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
39s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
34s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
30s{color} | {color:green} hadoop-hdfs-client in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 76m 1s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
37s{color} | {color:green} hadoop-hdfs-httpfs in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
[jira] [Commented] (HDFS-12178) Ozone: OzoneClient: Handling SCM container creationFlag at client side
[
https://issues.apache.org/jira/browse/HDFS-12178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097203#comment-16097203
]
Hadoop QA commented on HDFS-12178:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} HDFS-7240 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 16m
16s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
12s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
7s{color} | {color:green} HDFS-7240 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} HDFS-7240 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 40s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs: The patch
generated 2 new + 0 unchanged - 0 fixed = 2 total (was 0) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 80m 27s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
25s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}111m 42s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks |
| | hadoop.hdfs.server.namenode.ha.TestPipelinesFailover |
| | hadoop.ozone.container.common.TestDatanodeStateMachine |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure080 |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HDFS-12178 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12878460/HDFS-12178-HDFS-7240.000.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux f3755dee1624 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31
14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | HDFS-7240 / 63e211f |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-HDFS-Build/20389/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
|
| unit |
https://builds.apache.org/job/PreCommit-HDFS-Build/20389/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HDFS-Build/20389/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
[jira] [Commented] (HDFS-12136) BlockSender performance regression due to volume scanner edge case
[
https://issues.apache.org/jira/browse/HDFS-12136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097190#comment-16097190
]
Wei-Chiu Chuang commented on HDFS-12136:
HDFS-6804 is another facet of the same race condition.
[~brahmareddy] made a unit test that reliably reproduces the bug if HDFS-11160
is reverted.
> BlockSender performance regression due to volume scanner edge case
> --
>
> Key: HDFS-12136
> URL: https://issues.apache.org/jira/browse/HDFS-12136
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
>Affects Versions: 2.8.0
>Reporter: Daryn Sharp
>Assignee: Daryn Sharp
>Priority: Critical
> Attachments: HDFS-12136.branch-2.patch, HDFS-12136.trunk.patch
>
>
> HDFS-11160 attempted to fix a volume scan race for a file appended mid-scan
> by reading the last checksum of finalized blocks within the {{BlockSender}}
> ctor. Unfortunately it's holding the exclusive dataset lock to open and read
> the metafile multiple times Block sender instantiation becomes serialized.
> Performance completely collapses under heavy disk i/o utilization or high
> xceiver activity. Ex. lost node replication, balancing, or decommissioning.
> The xceiver threads congest creating block senders and impair the heartbeat
> processing that is contending for the same lock. Combined with other lock
> contention issues, pipelines break and nodes sporadically go dead.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12190) Enable 'hdfs dfs -stat' to display access time
[
https://issues.apache.org/jira/browse/HDFS-12190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097167#comment-16097167
]
Hadoop QA commented on HDFS-12190:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
25s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 14m
59s{color} | {color:red} root in trunk failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m
2s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
57s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
26s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
45s{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs in trunk has 10 extant
Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
36s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 10m
10s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 54s{color} | {color:orange} root: The patch generated 9 new + 199 unchanged
- 2 fixed = 208 total (was 201) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
36s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m
28s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 65m 30s{color}
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
40s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}136m 50s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks |
| | hadoop.hdfs.TestDFSStripedInputStreamWithRandomECPolicy |
| | hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics |
| | hadoop.hdfs.server.namenode.ha.TestPipelinesFailover |
| | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure150 |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | HDFS-12190 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12878454/HDFS-12190.001.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle xml |
| uname | Linux a0db36731684 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 465c213 |
| Default Java | 1.8.0_131 |
| mvninstall |
[jira] [Updated] (HDFS-12167) Ozone: Intermittent failure TestContainerPersistence#testListKey
[ https://issues.apache.org/jira/browse/HDFS-12167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Surendra Singh Lilhore updated HDFS-12167: -- Status: Patch Available (was: Open) > Ozone: Intermittent failure TestContainerPersistence#testListKey > > > Key: HDFS-12167 > URL: https://issues.apache.org/jira/browse/HDFS-12167 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone, test >Reporter: Weiwei Yang >Assignee: Surendra Singh Lilhore >Priority: Minor > Attachments: HDFS-12167-HDFS-7240.001.patch > > > TestContainerPersistence#listKeys seems to fail intermittently. It looks > like it was failing because some unexpected format of container name. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12167) Ozone: Intermittent failure TestContainerPersistence#testListKey
[
https://issues.apache.org/jira/browse/HDFS-12167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Surendra Singh Lilhore updated HDFS-12167:
--
Attachment: HDFS-12167-HDFS-7240.001.patch
This test case failed because of random generated containerName, which contain
file separator ("/"). {{ContainerManagerImpl#readContainerInfo()}} failed to
parse container keyname from the container file path because of file separator.
It is giving keyname {{~C"8}} instead of {{c0@]=[3/~C"8}}.
Attached initial patch, please review.
> Ozone: Intermittent failure TestContainerPersistence#testListKey
>
>
> Key: HDFS-12167
> URL: https://issues.apache.org/jira/browse/HDFS-12167
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone, test
>Reporter: Weiwei Yang
>Assignee: Surendra Singh Lilhore
>Priority: Minor
> Attachments: HDFS-12167-HDFS-7240.001.patch
>
>
> TestContainerPersistence#listKeys seems to fail intermittently. It looks
> like it was failing because some unexpected format of container name.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12178) Ozone: OzoneClient: Handling SCM container creationFlag at client side
[ https://issues.apache.org/jira/browse/HDFS-12178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandakumar updated HDFS-12178: -- Status: Patch Available (was: Open) > Ozone: OzoneClient: Handling SCM container creationFlag at client side > -- > > Key: HDFS-12178 > URL: https://issues.apache.org/jira/browse/HDFS-12178 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Reporter: Nandakumar >Assignee: Nandakumar > Attachments: HDFS-12178-HDFS-7240.000.patch > > > SCM BlockManager provisions a pool of containers upon block creation request, > only one container is returned with creationFlag to the client. The other > containers provisioned in the same batch will not have this flag. This jira > is to handle that scenario at client side, until HDFS-11888 is fixed. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12178) Ozone: OzoneClient: Handling SCM container creationFlag at client side
[ https://issues.apache.org/jira/browse/HDFS-12178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097158#comment-16097158 ] Nandakumar commented on HDFS-12178: --- The workaround is to try and create container every time. There is a local cache maintained for created containers, if the container name is not present in that cache we try to create the container. > Ozone: OzoneClient: Handling SCM container creationFlag at client side > -- > > Key: HDFS-12178 > URL: https://issues.apache.org/jira/browse/HDFS-12178 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Reporter: Nandakumar >Assignee: Nandakumar > Attachments: HDFS-12178-HDFS-7240.000.patch > > > SCM BlockManager provisions a pool of containers upon block creation request, > only one container is returned with creationFlag to the client. The other > containers provisioned in the same batch will not have this flag. This jira > is to handle that scenario at client side, until HDFS-11888 is fixed. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-12178) Ozone: OzoneClient: Handling SCM container creationFlag at client side
[ https://issues.apache.org/jira/browse/HDFS-12178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nandakumar updated HDFS-12178: -- Attachment: HDFS-12178-HDFS-7240.000.patch > Ozone: OzoneClient: Handling SCM container creationFlag at client side > -- > > Key: HDFS-12178 > URL: https://issues.apache.org/jira/browse/HDFS-12178 > Project: Hadoop HDFS > Issue Type: Sub-task > Components: ozone >Reporter: Nandakumar >Assignee: Nandakumar > Attachments: HDFS-12178-HDFS-7240.000.patch > > > SCM BlockManager provisions a pool of containers upon block creation request, > only one container is returned with creationFlag to the client. The other > containers provisioned in the same batch will not have this flag. This jira > is to handle that scenario at client side, until HDFS-11888 is fixed. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-6984) In Hadoop 3, make FileStatus serialize itself via protobuf
[
https://issues.apache.org/jira/browse/HDFS-6984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Douglas updated HDFS-6984:
Attachment: HDFS-6984.015.patch
Rebased patch.
> In Hadoop 3, make FileStatus serialize itself via protobuf
> --
>
> Key: HDFS-6984
> URL: https://issues.apache.org/jira/browse/HDFS-6984
> Project: Hadoop HDFS
> Issue Type: Improvement
>Affects Versions: 3.0.0-alpha1
>Reporter: Colin P. McCabe
>Assignee: Colin P. McCabe
> Labels: BB2015-05-TBR
> Attachments: HDFS-6984.001.patch, HDFS-6984.002.patch,
> HDFS-6984.003.patch, HDFS-6984.004.patch, HDFS-6984.005.patch,
> HDFS-6984.006.patch, HDFS-6984.007.patch, HDFS-6984.008.patch,
> HDFS-6984.009.patch, HDFS-6984.010.patch, HDFS-6984.011.patch,
> HDFS-6984.012.patch, HDFS-6984.013.patch, HDFS-6984.014.patch,
> HDFS-6984.015.patch, HDFS-6984.nowritable.patch
>
>
> FileStatus was a Writable in Hadoop 2 and earlier. Originally, we used this
> to serialize it and send it over the wire. But in Hadoop 2 and later, we
> have the protobuf {{HdfsFileStatusProto}} which serves to serialize this
> information. The protobuf form is preferable, since it allows us to add new
> fields in a backwards-compatible way. Another issue is that already a lot of
> subclasses of FileStatus don't override the Writable methods of the
> superclass, breaking the interface contract that read(status.write) should be
> equal to the original status.
> In Hadoop 3, we should just make FileStatus serialize itself via protobuf so
> that we don't have to deal with these issues. It's probably too late to do
> this in Hadoop 2, since user code may be relying on the existing FileStatus
> serialization there.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-11896) Non-dfsUsed will be doubled on dead node re-registration
[
https://issues.apache.org/jira/browse/HDFS-11896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16097132#comment-16097132
]
Konstantin Shvachko commented on HDFS-11896:
Let's make sure {{TestDeadDatanode.testNonDFSUsedONDeadNodeReReg()}} is not
failing.
> Non-dfsUsed will be doubled on dead node re-registration
>
>
> Key: HDFS-11896
> URL: https://issues.apache.org/jira/browse/HDFS-11896
> Project: Hadoop HDFS
> Issue Type: Bug
>Affects Versions: 2.7.3
>Reporter: Brahma Reddy Battula
>Assignee: Brahma Reddy Battula
>Priority: Blocker
> Labels: release-blocker
> Attachments: HDFS-11896-002.patch, HDFS-11896-003.patch,
> HDFS-11896-004.patch, HDFS-11896-005.patch, HDFS-11896-branch-2.7-001.patch,
> HDFS-11896-branch-2.7-002.patch, HDFS-11896-branch-2.7-003.patch,
> HDFS-11896-branch-2.7-004.patch, HDFS-11896.patch
>
>
> *Scenario:*
> i)Make you sure you've non-dfs data.
> ii) Stop Datanode
> iii) wait it becomes dead
> iv) now restart and check the non-dfs data
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
