subject:"\[jira\] \[Commented\] \(HDFS\-10976\) Fsck should mark EC files explicitly"

[jira] [Commented] (HDFS-10976) Fsck should mark EC files explicitly

2016-10-19 Thread Takanobu Asanuma (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-10976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15590457#comment-15590457
 ] 

Takanobu Asanuma commented on HDFS-10976:
-

+1 (non-binding). Thanks again.

> Fsck should mark EC files explicitly
> 
>
> Key: HDFS-10976
> URL: https://issues.apache.org/jira/browse/HDFS-10976
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>  Components: namenode
>Affects Versions: 3.0.0-alpha2
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>  Labels: supportability
> Attachments: HDFS-10976.001.patch, HDFS-10976.002.patch
>
>
> Currently fsck reports corrupt EC files as follows: it does not distinguish 
> erasure coded files versus replicated files. In addition, it would be nice to 
> print out the ec policy of the corrupt EC file.
> {quote}
> /striped 
> /striped/corrupted 393216 bytes, 1 block(s): 
> /striped/corrupted: CORRUPT blockpool BP-1564681138-127.0.0.1-1475793860787 
> block blk_-9223372036854775792
>  Under replicated 
> BP-1564681138-127.0.0.1-1475793860787:blk_-9223372036854775792_1001. Target 
> Replicas is 9 but found 5 live replica(s), 0 decommissioned replica(s) and 0 
> decommissioning replica(s).
>  CORRUPT 1 blocks of total size 393216 B
> 0. BP-1564681138-127.0.0.1-1475793860787:blk_-9223372036854775792_1001 
> len=393216 Live_repl=5  
> [DatanodeInfoWithStorage[127.0.0.1:62192,DS-81dcbc38-755e-446e-a028-71a79e4de6d9,DISK](CORRUPT),
>  
> DatanodeInfoWithStorage[127.0.0.1:62180,DS-98fe193d-6342-4b2c-ad61-4586e2530b1e,DISK](CORRUPT),
>  
> DatanodeInfoWithStorage[127.0.0.1:62167,DS-53031f88-0c63-4839-ab18-efea2f1bb063,DISK](CORRUPT),
>  
> DatanodeInfoWithStorage[127.0.0.1:62162,DS-e8b418fd-165d-4d6f-886b-f75c21be096d,DISK](CORRUPT),
>  
> DatanodeInfoWithStorage[127.0.0.1:62176,DS-03e51584-5b33-4bb6-89b5-f519cda57429,DISK](LIVE),
>  
> DatanodeInfoWithStorage[127.0.0.1:62158,DS-ce9ca7b3-5b00-4351-8537-822eed532b46,DISK](LIVE),
>  
> DatanodeInfoWithStorage[127.0.0.1:62184,DS-668c500e-eb3d-4d40-b900-814076d5e160,DISK](LIVE),
>  
> DatanodeInfoWithStorage[127.0.0.1:62171,DS-763a3961-b214-4601-81c0-abdaecf539c4,DISK](LIVE),
>  
> DatanodeInfoWithStorage[127.0.0.1:62188,DS-d4ea6399-bead-452e-8dd5-bc8c5ebd4f45,DISK](LIVE)]
> {quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-10976) Fsck should mark EC files explicitly

2016-10-19 Thread Wei-Chiu Chuang (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-10976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15589977#comment-15589977
 ] 

Wei-Chiu Chuang commented on HDFS-10976:


Thanks [~andrew.wang]. The failed tests does not fail in my local tree. I'll 
commit v02 patch.

> Fsck should mark EC files explicitly
> 
>
> Key: HDFS-10976
> URL: https://issues.apache.org/jira/browse/HDFS-10976
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>  Components: namenode
>Affects Versions: 3.0.0-alpha2
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>  Labels: supportability
> Attachments: HDFS-10976.001.patch, HDFS-10976.002.patch
>
>
> Currently fsck reports corrupt EC files as follows: it does not distinguish 
> erasure coded files versus replicated files. In addition, it would be nice to 
> print out the ec policy of the corrupt EC file.
> {quote}
> /striped 
> /striped/corrupted 393216 bytes, 1 block(s): 
> /striped/corrupted: CORRUPT blockpool BP-1564681138-127.0.0.1-1475793860787 
> block blk_-9223372036854775792
>  Under replicated 
> BP-1564681138-127.0.0.1-1475793860787:blk_-9223372036854775792_1001. Target 
> Replicas is 9 but found 5 live replica(s), 0 decommissioned replica(s) and 0 
> decommissioning replica(s).
>  CORRUPT 1 blocks of total size 393216 B
> 0. BP-1564681138-127.0.0.1-1475793860787:blk_-9223372036854775792_1001 
> len=393216 Live_repl=5  
> [DatanodeInfoWithStorage[127.0.0.1:62192,DS-81dcbc38-755e-446e-a028-71a79e4de6d9,DISK](CORRUPT),
>  
> DatanodeInfoWithStorage[127.0.0.1:62180,DS-98fe193d-6342-4b2c-ad61-4586e2530b1e,DISK](CORRUPT),
>  
> DatanodeInfoWithStorage[127.0.0.1:62167,DS-53031f88-0c63-4839-ab18-efea2f1bb063,DISK](CORRUPT),
>  
> DatanodeInfoWithStorage[127.0.0.1:62162,DS-e8b418fd-165d-4d6f-886b-f75c21be096d,DISK](CORRUPT),
>  
> DatanodeInfoWithStorage[127.0.0.1:62176,DS-03e51584-5b33-4bb6-89b5-f519cda57429,DISK](LIVE),
>  
> DatanodeInfoWithStorage[127.0.0.1:62158,DS-ce9ca7b3-5b00-4351-8537-822eed532b46,DISK](LIVE),
>  
> DatanodeInfoWithStorage[127.0.0.1:62184,DS-668c500e-eb3d-4d40-b900-814076d5e160,DISK](LIVE),
>  
> DatanodeInfoWithStorage[127.0.0.1:62171,DS-763a3961-b214-4601-81c0-abdaecf539c4,DISK](LIVE),
>  
> DatanodeInfoWithStorage[127.0.0.1:62188,DS-d4ea6399-bead-452e-8dd5-bc8c5ebd4f45,DISK](LIVE)]
> {quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-10976) Fsck should mark EC files explicitly

2016-10-19 Thread Andrew Wang (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-10976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15589817#comment-15589817
 ] 

Andrew Wang commented on HDFS-10976:


LGTM +1

> Fsck should mark EC files explicitly
> 
>
> Key: HDFS-10976
> URL: https://issues.apache.org/jira/browse/HDFS-10976
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>  Components: namenode
>Affects Versions: 3.0.0-alpha2
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>  Labels: supportability
> Attachments: HDFS-10976.001.patch, HDFS-10976.002.patch
>
>
> Currently fsck reports corrupt EC files as follows: it does not distinguish 
> erasure coded files versus replicated files. In addition, it would be nice to 
> print out the ec policy of the corrupt EC file.
> {quote}
> /striped 
> /striped/corrupted 393216 bytes, 1 block(s): 
> /striped/corrupted: CORRUPT blockpool BP-1564681138-127.0.0.1-1475793860787 
> block blk_-9223372036854775792
>  Under replicated 
> BP-1564681138-127.0.0.1-1475793860787:blk_-9223372036854775792_1001. Target 
> Replicas is 9 but found 5 live replica(s), 0 decommissioned replica(s) and 0 
> decommissioning replica(s).
>  CORRUPT 1 blocks of total size 393216 B
> 0. BP-1564681138-127.0.0.1-1475793860787:blk_-9223372036854775792_1001 
> len=393216 Live_repl=5  
> [DatanodeInfoWithStorage[127.0.0.1:62192,DS-81dcbc38-755e-446e-a028-71a79e4de6d9,DISK](CORRUPT),
>  
> DatanodeInfoWithStorage[127.0.0.1:62180,DS-98fe193d-6342-4b2c-ad61-4586e2530b1e,DISK](CORRUPT),
>  
> DatanodeInfoWithStorage[127.0.0.1:62167,DS-53031f88-0c63-4839-ab18-efea2f1bb063,DISK](CORRUPT),
>  
> DatanodeInfoWithStorage[127.0.0.1:62162,DS-e8b418fd-165d-4d6f-886b-f75c21be096d,DISK](CORRUPT),
>  
> DatanodeInfoWithStorage[127.0.0.1:62176,DS-03e51584-5b33-4bb6-89b5-f519cda57429,DISK](LIVE),
>  
> DatanodeInfoWithStorage[127.0.0.1:62158,DS-ce9ca7b3-5b00-4351-8537-822eed532b46,DISK](LIVE),
>  
> DatanodeInfoWithStorage[127.0.0.1:62184,DS-668c500e-eb3d-4d40-b900-814076d5e160,DISK](LIVE),
>  
> DatanodeInfoWithStorage[127.0.0.1:62171,DS-763a3961-b214-4601-81c0-abdaecf539c4,DISK](LIVE),
>  
> DatanodeInfoWithStorage[127.0.0.1:62188,DS-d4ea6399-bead-452e-8dd5-bc8c5ebd4f45,DISK](LIVE)]
> {quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-10976) Fsck should mark EC files explicitly

2016-10-19 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-10976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15589785#comment-15589785
 ] 

Hadoop QA commented on HDFS-10976:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
12s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
46s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
52s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
12s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
42s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
39s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
 9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
37s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 68m  2s{color} 
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
17s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 87m 24s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.server.namenode.TestDiskspaceQuotaUpdate |
|   | hadoop.hdfs.TestPersistBlocks |
\\
\\
|| Subsystem || Report/Notes ||
| Docker |  Image:yetus/hadoop:9560f25 |
| JIRA Issue | HDFS-10976 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12834222/HDFS-10976.002.patch |
| Optional Tests |  asflicense  compile  javac  javadoc  mvninstall  mvnsite  
unit  findbugs  checkstyle  |
| uname | Linux 6f500f1e8f33 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 
21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh 
|
| git revision | trunk / e9c4616 |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| unit | 
https://builds.apache.org/job/PreCommit-HDFS-Build/17221/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HDFS-Build/17221/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U: 
hadoop-hdfs-project/hadoop-hdfs |
| Console output | 
https://builds.apache.org/job/PreCommit-HDFS-Build/17221/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT   http://yetus.apache.org |


This message was automatically generated.



> Fsck should mark EC files explicitly
> 
>
> Key: HDFS-10976
> URL: https://issues.apache.org/jira/browse/HDFS-10976
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>  Components: namenode
>Affects Versions: 3.0.0-alpha2
>Reporter: Wei-Chiu Chuang
>

[jira] [Commented] (HDFS-10976) Fsck should mark EC files explicitly

2016-10-18 Thread Takanobu Asanuma (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-10976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15587417#comment-15587417
 ] 

Takanobu Asanuma commented on HDFS-10976:
-

Thanks for the patch, [~jojochuang]! It mostly looks good to me(non-binding). 
One minor comment is that we can reuse the variable.

{code:java}
ErasureCodingPolicy ecPolicy = file.getErasureCodingPolicy();
if (ecPolicy == null) { // a replicated file
{code}

> Fsck should mark EC files explicitly
> 
>
> Key: HDFS-10976
> URL: https://issues.apache.org/jira/browse/HDFS-10976
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>  Components: namenode
>Affects Versions: 3.0.0-alpha2
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>  Labels: supportability
> Attachments: HDFS-10976.001.patch
>
>
> Currently fsck reports corrupt EC files as follows: it does not distinguish 
> erasure coded files versus replicated files. In addition, it would be nice to 
> print out the ec policy of the corrupt EC file.
> {quote}
> /striped 
> /striped/corrupted 393216 bytes, 1 block(s): 
> /striped/corrupted: CORRUPT blockpool BP-1564681138-127.0.0.1-1475793860787 
> block blk_-9223372036854775792
>  Under replicated 
> BP-1564681138-127.0.0.1-1475793860787:blk_-9223372036854775792_1001. Target 
> Replicas is 9 but found 5 live replica(s), 0 decommissioned replica(s) and 0 
> decommissioning replica(s).
>  CORRUPT 1 blocks of total size 393216 B
> 0. BP-1564681138-127.0.0.1-1475793860787:blk_-9223372036854775792_1001 
> len=393216 Live_repl=5  
> [DatanodeInfoWithStorage[127.0.0.1:62192,DS-81dcbc38-755e-446e-a028-71a79e4de6d9,DISK](CORRUPT),
>  
> DatanodeInfoWithStorage[127.0.0.1:62180,DS-98fe193d-6342-4b2c-ad61-4586e2530b1e,DISK](CORRUPT),
>  
> DatanodeInfoWithStorage[127.0.0.1:62167,DS-53031f88-0c63-4839-ab18-efea2f1bb063,DISK](CORRUPT),
>  
> DatanodeInfoWithStorage[127.0.0.1:62162,DS-e8b418fd-165d-4d6f-886b-f75c21be096d,DISK](CORRUPT),
>  
> DatanodeInfoWithStorage[127.0.0.1:62176,DS-03e51584-5b33-4bb6-89b5-f519cda57429,DISK](LIVE),
>  
> DatanodeInfoWithStorage[127.0.0.1:62158,DS-ce9ca7b3-5b00-4351-8537-822eed532b46,DISK](LIVE),
>  
> DatanodeInfoWithStorage[127.0.0.1:62184,DS-668c500e-eb3d-4d40-b900-814076d5e160,DISK](LIVE),
>  
> DatanodeInfoWithStorage[127.0.0.1:62171,DS-763a3961-b214-4601-81c0-abdaecf539c4,DISK](LIVE),
>  
> DatanodeInfoWithStorage[127.0.0.1:62188,DS-d4ea6399-bead-452e-8dd5-bc8c5ebd4f45,DISK](LIVE)]
> {quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-10976) Fsck should mark EC files explicitly

2016-10-18 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-10976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15586645#comment-15586645
 ] 

Hadoop QA commented on HDFS-10976:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
12s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  8m 
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
28s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
52s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
14s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
51s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
45s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
48s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}  
0m 25s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs: The patch 
generated 1 new + 100 unchanged - 0 fixed = 101 total (was 100) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  2m  
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
43s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 67m  6s{color} 
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
23s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 88m 29s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.server.datanode.TestDirectoryScanner |
\\
\\
|| Subsystem || Report/Notes ||
| Docker |  Image:yetus/hadoop:9560f25 |
| JIRA Issue | HDFS-10976 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12834018/HDFS-10976.001.patch |
| Optional Tests |  asflicense  compile  javac  javadoc  mvninstall  mvnsite  
unit  findbugs  checkstyle  |
| uname | Linux 852aafaa85a2 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 
17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh 
|
| git revision | trunk / b733a6f |
| Default Java | 1.8.0_101 |
| findbugs | v3.0.0 |
| checkstyle | 
https://builds.apache.org/job/PreCommit-HDFS-Build/17207/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
 |
| unit | 
https://builds.apache.org/job/PreCommit-HDFS-Build/17207/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HDFS-Build/17207/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U: 
hadoop-hdfs-project/hadoop-hdfs |
| Console output | 
https://builds.apache.org/job/PreCommit-HDFS-Build/17207/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT   http://yetus.apache.org |


This message was automatically generated.



> Fsck should mark EC files explicitly
> 
>
> Key: HDFS-10976
> URL:

[jira] [Commented] (HDFS-10976) Fsck should mark EC files explicitly

2016-10-18 Thread Wei-Chiu Chuang (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-10976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15586414#comment-15586414
 ] 

Wei-Chiu Chuang commented on HDFS-10976:


The output of fsck with -files parameter becomes as follows:

{noformat}
FSCK started by weichiu (auth:SIMPLE) from /127.0.0.1 for path / at Tue Oct 18 
12:15:15 PDT 2016
/ 
/replicated 
/replicated/replfile 1024 bytes, replicated: replication=3, 1 block(s):  
MISSING 1 blocks of total size 1024 B
0. BP-1938052379-172.16.1.88-1476818109173:blk_1073741825_1001 len=1024 MISSING!

/striped 
/striped/largeFile 786432 bytes, erasure-coded: policy=RS-DEFAULT-6-3-64k, 1 
block(s):  MISSING 1 blocks of total size 786432 B
0. BP-1938052379-172.16.1.88-1476818109173:blk_-9223372036854775792_1002 
len=786432 MISSING! Live_repl=0

/striped/smallFile 12 bytes, erasure-coded: policy=RS-DEFAULT-6-3-64k, 1 
block(s):  MISSING 1 blocks of total size 12 B
0. BP-1938052379-172.16.1.88-1476818109173:blk_-9223372036854775776_1003 len=12 
MISSING! Live_repl=0

{noformat}

> Fsck should mark EC files explicitly
> 
>
> Key: HDFS-10976
> URL: https://issues.apache.org/jira/browse/HDFS-10976
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>  Components: namenode
>Affects Versions: 3.0.0-alpha2
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>  Labels: supportability
> Attachments: HDFS-10976.001.patch
>
>
> Currently fsck reports corrupt EC files as follows: it does not distinguish 
> erasure coded files versus replicated files. In addition, it would be nice to 
> print out the ec policy of the corrupt EC file.
> {quote}
> /striped 
> /striped/corrupted 393216 bytes, 1 block(s): 
> /striped/corrupted: CORRUPT blockpool BP-1564681138-127.0.0.1-1475793860787 
> block blk_-9223372036854775792
>  Under replicated 
> BP-1564681138-127.0.0.1-1475793860787:blk_-9223372036854775792_1001. Target 
> Replicas is 9 but found 5 live replica(s), 0 decommissioned replica(s) and 0 
> decommissioning replica(s).
>  CORRUPT 1 blocks of total size 393216 B
> 0. BP-1564681138-127.0.0.1-1475793860787:blk_-9223372036854775792_1001 
> len=393216 Live_repl=5  
> [DatanodeInfoWithStorage[127.0.0.1:62192,DS-81dcbc38-755e-446e-a028-71a79e4de6d9,DISK](CORRUPT),
>  
> DatanodeInfoWithStorage[127.0.0.1:62180,DS-98fe193d-6342-4b2c-ad61-4586e2530b1e,DISK](CORRUPT),
>  
> DatanodeInfoWithStorage[127.0.0.1:62167,DS-53031f88-0c63-4839-ab18-efea2f1bb063,DISK](CORRUPT),
>  
> DatanodeInfoWithStorage[127.0.0.1:62162,DS-e8b418fd-165d-4d6f-886b-f75c21be096d,DISK](CORRUPT),
>  
> DatanodeInfoWithStorage[127.0.0.1:62176,DS-03e51584-5b33-4bb6-89b5-f519cda57429,DISK](LIVE),
>  
> DatanodeInfoWithStorage[127.0.0.1:62158,DS-ce9ca7b3-5b00-4351-8537-822eed532b46,DISK](LIVE),
>  
> DatanodeInfoWithStorage[127.0.0.1:62184,DS-668c500e-eb3d-4d40-b900-814076d5e160,DISK](LIVE),
>  
> DatanodeInfoWithStorage[127.0.0.1:62171,DS-763a3961-b214-4601-81c0-abdaecf539c4,DISK](LIVE),
>  
> DatanodeInfoWithStorage[127.0.0.1:62188,DS-d4ea6399-bead-452e-8dd5-bc8c5ebd4f45,DISK](LIVE)]
> {quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-10976) Fsck should mark EC files explicitly

2016-10-18 Thread Wei-Chiu Chuang (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-10976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15586107#comment-15586107
 ] 

Wei-Chiu Chuang commented on HDFS-10976:


Assign this jira to myself. I want to take this opportunity to understand fsck 
implementation.

> Fsck should mark EC files explicitly
> 
>
> Key: HDFS-10976
> URL: https://issues.apache.org/jira/browse/HDFS-10976
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>  Components: namenode
>Affects Versions: 3.0.0-alpha2
>Reporter: Wei-Chiu Chuang
>Assignee: Wei-Chiu Chuang
>
> Currently fsck reports corrupt EC files as follows: it does not distinguish 
> erasure coded files versus replicated files. In addition, it would be nice to 
> print out the ec policy of the corrupt EC file.
> {quote}
> /striped 
> /striped/corrupted 393216 bytes, 1 block(s): 
> /striped/corrupted: CORRUPT blockpool BP-1564681138-127.0.0.1-1475793860787 
> block blk_-9223372036854775792
>  Under replicated 
> BP-1564681138-127.0.0.1-1475793860787:blk_-9223372036854775792_1001. Target 
> Replicas is 9 but found 5 live replica(s), 0 decommissioned replica(s) and 0 
> decommissioning replica(s).
>  CORRUPT 1 blocks of total size 393216 B
> 0. BP-1564681138-127.0.0.1-1475793860787:blk_-9223372036854775792_1001 
> len=393216 Live_repl=5  
> [DatanodeInfoWithStorage[127.0.0.1:62192,DS-81dcbc38-755e-446e-a028-71a79e4de6d9,DISK](CORRUPT),
>  
> DatanodeInfoWithStorage[127.0.0.1:62180,DS-98fe193d-6342-4b2c-ad61-4586e2530b1e,DISK](CORRUPT),
>  
> DatanodeInfoWithStorage[127.0.0.1:62167,DS-53031f88-0c63-4839-ab18-efea2f1bb063,DISK](CORRUPT),
>  
> DatanodeInfoWithStorage[127.0.0.1:62162,DS-e8b418fd-165d-4d6f-886b-f75c21be096d,DISK](CORRUPT),
>  
> DatanodeInfoWithStorage[127.0.0.1:62176,DS-03e51584-5b33-4bb6-89b5-f519cda57429,DISK](LIVE),
>  
> DatanodeInfoWithStorage[127.0.0.1:62158,DS-ce9ca7b3-5b00-4351-8537-822eed532b46,DISK](LIVE),
>  
> DatanodeInfoWithStorage[127.0.0.1:62184,DS-668c500e-eb3d-4d40-b900-814076d5e160,DISK](LIVE),
>  
> DatanodeInfoWithStorage[127.0.0.1:62171,DS-763a3961-b214-4601-81c0-abdaecf539c4,DISK](LIVE),
>  
> DatanodeInfoWithStorage[127.0.0.1:62188,DS-d4ea6399-bead-452e-8dd5-bc8c5ebd4f45,DISK](LIVE)]
> {quote}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-10976) Fsck should mark EC files explicitly

[jira] [Commented] (HDFS-10976) Fsck should mark EC files explicitly

[jira] [Commented] (HDFS-10976) Fsck should mark EC files explicitly

[jira] [Commented] (HDFS-10976) Fsck should mark EC files explicitly

[jira] [Commented] (HDFS-10976) Fsck should mark EC files explicitly

[jira] [Commented] (HDFS-10976) Fsck should mark EC files explicitly

[jira] [Commented] (HDFS-10976) Fsck should mark EC files explicitly

[jira] [Commented] (HDFS-10976) Fsck should mark EC files explicitly

8 matches

Site Navigation

Mail list logo

Footer information