[jira] [Commented] (HDFS-8694) Expose the stats of IOErrors on each FsVolume through JMX
[ https://issues.apache.org/jira/browse/HDFS-8694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15806019#comment-15806019 ] Hadoop QA commented on HDFS-8694: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s{color} | {color:blue} Docker mode activated. {color} | | {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 4s{color} | {color:red} HDFS-8694 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} | \\ \\ || Subsystem || Report/Notes || | JIRA Issue | HDFS-8694 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12743409/HDFS-8694.001.patch | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/18085/console | | Powered by | Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org | This message was automatically generated. > Expose the stats of IOErrors on each FsVolume through JMX > - > > Key: HDFS-8694 > URL: https://issues.apache.org/jira/browse/HDFS-8694 > Project: Hadoop HDFS > Issue Type: Improvement > Components: datanode >Affects Versions: 2.7.0 >Reporter: Lei (Eddy) Xu >Assignee: Lei (Eddy) Xu > Attachments: HDFS-8694.000.patch, HDFS-8694.001.patch > > > Currently, once DataNode hits an {{IOError}} when writing / reading block > files, it starts a background {{DiskChecker.checkDirs()}} thread. But if this > thread successfully finishes, DN does not record this {{IOError}}. > We need one measurement to count all {{IOErrors}} for each volume. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-8694) Expose the stats of IOErrors on each FsVolume through JMX
[ https://issues.apache.org/jira/browse/HDFS-8694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15025274#comment-15025274 ] Hadoop QA commented on HDFS-8694: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 4s {color} | {color:red} HDFS-8694 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} | \\ \\ || Subsystem || Report/Notes || | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12743409/HDFS-8694.001.patch | | JIRA Issue | HDFS-8694 | | Powered by | Apache Yetus http://yetus.apache.org | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/13642/console | This message was automatically generated. > Expose the stats of IOErrors on each FsVolume through JMX > - > > Key: HDFS-8694 > URL: https://issues.apache.org/jira/browse/HDFS-8694 > Project: Hadoop HDFS > Issue Type: Improvement > Components: datanode >Affects Versions: 2.7.0 >Reporter: Lei (Eddy) Xu >Assignee: Lei (Eddy) Xu > Attachments: HDFS-8694.000.patch, HDFS-8694.001.patch > > > Currently, once DataNode hits an {{IOError}} when writing / reading block > files, it starts a background {{DiskChecker.checkDirs()}} thread. But if this > thread successfully finishes, DN does not record this {{IOError}}. > We need one measurement to count all {{IOErrors}} for each volume. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8694) Expose the stats of IOErrors on each FsVolume through JMX
[ https://issues.apache.org/jira/browse/HDFS-8694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14628371#comment-14628371 ] Lei (Eddy) Xu commented on HDFS-8694: - Thanks for the reviews, [~andrew.wang] bq. I have a hard time understanding when we should call handle the disk error vs. just bubbling up, since it bubbles there seems like a danger of handling the same root IOE more than once. What's the methodology here? Is it possible to move handling to the top-level somewhere? I can manually examine all the current callsites and callers, but that's not very future-proof. The reason that call {{volume#handleIOErrors()}} is that when the {{IOE}} pops up to the place we used to call {{DataNode#checkDiskErrorAsync()}}, the context (IOs on which volume) is usually missing. My intention was to call {{volume#handleIOErrors()}} at the highest level that manages {{volume}} object lifetime. I will try to get rid of {{DataNode#checkDiskErrorAsync()}} call in a following JIRA. bq. Since we now have the volume as context, we should really move the disk checker to be per-volume rather than DN wide. One volume throwing an error is no reason to check all of them. This can be deferred to a follow-up; I think it's a slam dunk. Yes. It is the reason to put {{hadnleIOErrors()}} in to {{FsVolumeSpi}}. I was thinking to use a per-volume thread to do {{checkDirs()}} and also use {{numOfErrors()}} as trigger. I will do it in a following JIRA as well. Working on the rest of comments. Thanks a lot for these great comments. > Expose the stats of IOErrors on each FsVolume through JMX > - > > Key: HDFS-8694 > URL: https://issues.apache.org/jira/browse/HDFS-8694 > Project: Hadoop HDFS > Issue Type: Improvement > Components: datanode, HDFS >Affects Versions: 2.7.0 >Reporter: Lei (Eddy) Xu >Assignee: Lei (Eddy) Xu > Attachments: HDFS-8694.000.patch, HDFS-8694.001.patch > > > Currently, once DataNode hits an {{IOError}} when writing / reading block > files, it starts a background {{DiskChecker.checkDirs()}} thread. But if this > thread successfully finishes, DN does not record this {{IOError}}. > We need one measurement to count all {{IOErrors}} for each volume. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8694) Expose the stats of IOErrors on each FsVolume through JMX
[ https://issues.apache.org/jira/browse/HDFS-8694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14626770#comment-14626770 ] Andrew Wang commented on HDFS-8694: --- Hi Eddy, overall this patch looks great, thanks for working on it. Some review comments: High-level: * I have a hard time understanding when we should call handle the disk error vs. just bubbling up, since it bubbles there seems like a danger of handling the same root IOE more than once. What's the methodology here? Is it possible to move handling to the top-level somewhere? I can manually examine all the current callsites and callers, but that's not very future-proof. * Related to the above, our unit tests do not cover anywhere close to all the locations that handle an IOError. Adding tests for all of these would be onerous, so very interested in a solution to the above. * Since we now have the volume as context, we should really move the disk checker to be per-volume rather than DN wide. One volume throwing an error is no reason to check all of them. This can be deferred to a follow-up; I think it's a slam dunk. Nits: * FsDatasetImpl#moveBlockAcrossStorage, can we get rid of volume and move targetVolume's declaration outside of the try? Looks equal to volume. * In places like BlockSender#close we actually could IOE multiple times but only increment once. Thoughts? * Extra debug print in TestDataTransferKeepalive#testSlowReader * Linebreak here is kind of awkward, move the end parens or the try up? {code} try (ReplicaHandler replica = dataset.append( block, block.getGenerationStamp() + 1, block.getNumBytes()) ) { {code} * TestMover adds a test timeout, looks unrelated to this patch? > Expose the stats of IOErrors on each FsVolume through JMX > - > > Key: HDFS-8694 > URL: https://issues.apache.org/jira/browse/HDFS-8694 > Project: Hadoop HDFS > Issue Type: Improvement > Components: datanode, HDFS >Affects Versions: 2.7.0 >Reporter: Lei (Eddy) Xu >Assignee: Lei (Eddy) Xu > Attachments: HDFS-8694.000.patch, HDFS-8694.001.patch > > > Currently, once DataNode hits an {{IOError}} when writing / reading block > files, it starts a background {{DiskChecker.checkDirs()}} thread. But if this > thread successfully finishes, DN does not record this {{IOError}}. > We need one measurement to count all {{IOErrors}} for each volume. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8694) Expose the stats of IOErrors on each FsVolume through JMX
[ https://issues.apache.org/jira/browse/HDFS-8694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14623649#comment-14623649 ] Lei (Eddy) Xu commented on HDFS-8694: - The checkstyle warnings are false alarm. They complained about the length of the files. The failure test {{TestStandbyIsHot}} is not relevent. I can not produce it on my dev machine, and it seems to appear in other recent JIRAs. > Expose the stats of IOErrors on each FsVolume through JMX > - > > Key: HDFS-8694 > URL: https://issues.apache.org/jira/browse/HDFS-8694 > Project: Hadoop HDFS > Issue Type: Improvement > Components: datanode, HDFS >Affects Versions: 2.7.0 >Reporter: Lei (Eddy) Xu >Assignee: Lei (Eddy) Xu > Attachments: HDFS-8694.000.patch, HDFS-8694.001.patch > > > Currently, once DataNode hits an {{IOError}} when writing / reading block > files, it starts a background {{DiskChecker.checkDirs()}} thread. But if this > thread successfully finishes, DN does not record this {{IOError}}. > We need one measurement to count all {{IOErrors}} for each volume. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8694) Expose the stats of IOErrors on each FsVolume through JMX
[ https://issues.apache.org/jira/browse/HDFS-8694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14623085#comment-14623085 ] Hadoop QA commented on HDFS-8694: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | pre-patch | 17m 15s | Pre-patch trunk compilation is healthy. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 8 new or modified test files. | | {color:green}+1{color} | javac | 7m 37s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 9m 37s | There were no new javadoc warning messages. | | {color:green}+1{color} | release audit | 0m 24s | The applied patch does not increase the total number of release audit warnings. | | {color:red}-1{color} | checkstyle | 1m 20s | The applied patch generated 2 new checkstyle issues (total was 424, now 418). | | {color:green}+1{color} | whitespace | 0m 4s | The patch has no lines that end in whitespace. | | {color:green}+1{color} | install | 1m 20s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 34s | The patch built with eclipse:eclipse. | | {color:green}+1{color} | findbugs | 2m 30s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. | | {color:green}+1{color} | native | 3m 2s | Pre-build of native portion | | {color:red}-1{color} | hdfs tests | 162m 5s | Tests failed in hadoop-hdfs. | | | | 205m 54s | | \\ \\ || Reason || Tests || | Failed unit tests | hadoop.hdfs.server.namenode.ha.TestStandbyIsHot | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12743409/HDFS-8694.001.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | trunk / 0824426 | | checkstyle | https://builds.apache.org/job/PreCommit-HDFS-Build/11667/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt | | hadoop-hdfs test log | https://builds.apache.org/job/PreCommit-HDFS-Build/11667/artifact/patchprocess/testrun_hadoop-hdfs.txt | | Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/11667/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/11667/console | This message was automatically generated. > Expose the stats of IOErrors on each FsVolume through JMX > - > > Key: HDFS-8694 > URL: https://issues.apache.org/jira/browse/HDFS-8694 > Project: Hadoop HDFS > Issue Type: Improvement > Components: datanode, HDFS >Affects Versions: 2.7.0 >Reporter: Lei (Eddy) Xu >Assignee: Lei (Eddy) Xu > Attachments: HDFS-8694.000.patch, HDFS-8694.001.patch > > > Currently, once DataNode hits an {{IOError}} when writing / reading block > files, it starts a background {{DiskChecker.checkDirs()}} thread. But if this > thread successfully finishes, DN does not record this {{IOError}}. > We need one measurement to count all {{IOErrors}} for each volume. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8694) Expose the stats of IOErrors on each FsVolume through JMX
[ https://issues.apache.org/jira/browse/HDFS-8694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14614369#comment-14614369 ] Lei (Eddy) Xu commented on HDFS-8694: - The failure test {{TestStandbyIsHot}} passed on my local machine. And the checkstyle warnings are false alarm. > Expose the stats of IOErrors on each FsVolume through JMX > - > > Key: HDFS-8694 > URL: https://issues.apache.org/jira/browse/HDFS-8694 > Project: Hadoop HDFS > Issue Type: Improvement > Components: datanode, HDFS >Affects Versions: 2.7.0 >Reporter: Lei (Eddy) Xu >Assignee: Lei (Eddy) Xu > Attachments: HDFS-8694.000.patch, HDFS-8694.001.patch > > > Currently, once DataNode hits an {{IOError}} when writing / reading block > files, it starts a background {{DiskChecker.checkDirs()}} thread. But if this > thread successfully finishes, DN does not record this {{IOError}}. > We need one measurement to count all {{IOErrors}} for each volume. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8694) Expose the stats of IOErrors on each FsVolume through JMX
[ https://issues.apache.org/jira/browse/HDFS-8694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14614177#comment-14614177 ] Hadoop QA commented on HDFS-8694: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:red}-1{color} | pre-patch | 17m 47s | Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 8 new or modified test files. | | {color:green}+1{color} | javac | 7m 31s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 9m 38s | There were no new javadoc warning messages. | | {color:green}+1{color} | release audit | 0m 23s | The applied patch does not increase the total number of release audit warnings. | | {color:red}-1{color} | checkstyle | 2m 12s | The applied patch generated 2 new checkstyle issues (total was 446, now 440). | | {color:green}+1{color} | whitespace | 0m 4s | The patch has no lines that end in whitespace. | | {color:green}+1{color} | install | 1m 35s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with eclipse:eclipse. | | {color:green}+1{color} | findbugs | 3m 16s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. | | {color:green}+1{color} | native | 3m 16s | Pre-build of native portion | | {color:red}-1{color} | hdfs tests | 159m 34s | Tests failed in hadoop-hdfs. | | | | 205m 52s | | \\ \\ || Reason || Tests || | Failed unit tests | hadoop.hdfs.server.namenode.ha.TestStandbyIsHot | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12743409/HDFS-8694.001.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | trunk / 688617d | | Pre-patch Findbugs warnings | https://builds.apache.org/job/PreCommit-HDFS-Build/11575/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html | | checkstyle | https://builds.apache.org/job/PreCommit-HDFS-Build/11575/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt | | hadoop-hdfs test log | https://builds.apache.org/job/PreCommit-HDFS-Build/11575/artifact/patchprocess/testrun_hadoop-hdfs.txt | | Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/11575/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/11575/console | This message was automatically generated. > Expose the stats of IOErrors on each FsVolume through JMX > - > > Key: HDFS-8694 > URL: https://issues.apache.org/jira/browse/HDFS-8694 > Project: Hadoop HDFS > Issue Type: Improvement > Components: datanode, HDFS >Affects Versions: 2.7.0 >Reporter: Lei (Eddy) Xu >Assignee: Lei (Eddy) Xu > Attachments: HDFS-8694.000.patch, HDFS-8694.001.patch > > > Currently, once DataNode hits an {{IOError}} when writing / reading block > files, it starts a background {{DiskChecker.checkDirs()}} thread. But if this > thread successfully finishes, DN does not record this {{IOError}}. > We need one measurement to count all {{IOErrors}} for each volume. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HDFS-8694) Expose the stats of IOErrors on each FsVolume through JMX
[ https://issues.apache.org/jira/browse/HDFS-8694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14614132#comment-14614132 ] Hadoop QA commented on HDFS-8694: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:red}-1{color} | pre-patch | 18m 5s | Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 8 new or modified test files. | | {color:green}+1{color} | javac | 7m 32s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 9m 40s | There were no new javadoc warning messages. | | {color:green}+1{color} | release audit | 0m 23s | The applied patch does not increase the total number of release audit warnings. | | {color:red}-1{color} | checkstyle | 2m 16s | The applied patch generated 2 new checkstyle issues (total was 446, now 440). | | {color:green}+1{color} | whitespace | 0m 5s | The patch has no lines that end in whitespace. | | {color:green}+1{color} | install | 1m 36s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 34s | The patch built with eclipse:eclipse. | | {color:green}+1{color} | findbugs | 3m 17s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. | | {color:green}+1{color} | native | 3m 16s | Pre-build of native portion | | {color:red}-1{color} | hdfs tests | 158m 48s | Tests failed in hadoop-hdfs. | | | | 205m 35s | | \\ \\ || Reason || Tests || | Failed unit tests | hadoop.hdfs.server.namenode.ha.TestStandbyIsHot | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12743409/HDFS-8694.001.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | trunk / 2eae130 | | Pre-patch Findbugs warnings | https://builds.apache.org/job/PreCommit-HDFS-Build/11574/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html | | checkstyle | https://builds.apache.org/job/PreCommit-HDFS-Build/11574/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt | | hadoop-hdfs test log | https://builds.apache.org/job/PreCommit-HDFS-Build/11574/artifact/patchprocess/testrun_hadoop-hdfs.txt | | Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/11574/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/11574/console | This message was automatically generated. > Expose the stats of IOErrors on each FsVolume through JMX > - > > Key: HDFS-8694 > URL: https://issues.apache.org/jira/browse/HDFS-8694 > Project: Hadoop HDFS > Issue Type: Improvement > Components: datanode, HDFS >Affects Versions: 2.7.0 >Reporter: Lei (Eddy) Xu >Assignee: Lei (Eddy) Xu > Attachments: HDFS-8694.000.patch, HDFS-8694.001.patch > > > Currently, once DataNode hits an {{IOError}} when writing / reading block > files, it starts a background {{DiskChecker.checkDirs()}} thread. But if this > thread successfully finishes, DN does not record this {{IOError}}. > We need one measurement to count all {{IOErrors}} for each volume. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
Auto-Re: [jira] [Commented] (HDFS-8694) Expose the stats of IOErrors on each FsVolume through JMX
您的邮件已收到!谢谢!
[jira] [Commented] (HDFS-8694) Expose the stats of IOErrors on each FsVolume through JMX
[ https://issues.apache.org/jira/browse/HDFS-8694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606992#comment-14606992 ] Hadoop QA commented on HDFS-8694: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:red}-1{color} | pre-patch | 22m 35s | Pre-patch trunk has 1 extant Findbugs (version 3.0.0) warnings. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 6 new or modified test files. | | {color:red}-1{color} | javac | 9m 3s | The applied patch generated 2 additional warning messages. | | {color:green}+1{color} | javadoc | 11m 22s | There were no new javadoc warning messages. | | {color:green}+1{color} | release audit | 0m 24s | The applied patch does not increase the total number of release audit warnings. | | {color:red}-1{color} | checkstyle | 2m 27s | The applied patch generated 4 new checkstyle issues (total was 445, now 441). | | {color:green}+1{color} | whitespace | 0m 3s | The patch has no lines that end in whitespace. | | {color:green}+1{color} | install | 1m 33s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with eclipse:eclipse. | | {color:green}+1{color} | findbugs | 3m 12s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. | | {color:green}+1{color} | native | 3m 16s | Pre-build of native portion | | {color:red}-1{color} | hdfs tests | 177m 31s | Tests failed in hadoop-hdfs. | | | | 232m 4s | | \\ \\ || Reason || Tests || | Failed unit tests | hadoop.hdfs.TestEncryptedTransfer | | | hadoop.hdfs.web.TestWebHdfsFileSystemContract | | | hadoop.hdfs.TestHDFSFileSystemContract | | | hadoop.hdfs.TestDataTransferKeepalive | | Timed out tests | org.apache.hadoop.hdfs.server.namenode.TestNameEditsConfigs | | | org.apache.hadoop.hdfs.server.mover.TestMover | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12742648/HDFS-8694.000.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | trunk / d3797f9 | | Pre-patch Findbugs warnings | https://builds.apache.org/job/PreCommit-HDFS-Build/11538/artifact/patchprocess/trunkFindbugsWarningshadoop-hdfs.html | | javac | https://builds.apache.org/job/PreCommit-HDFS-Build/11538/artifact/patchprocess/diffJavacWarnings.txt | | checkstyle | https://builds.apache.org/job/PreCommit-HDFS-Build/11538/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt | | hadoop-hdfs test log | https://builds.apache.org/job/PreCommit-HDFS-Build/11538/artifact/patchprocess/testrun_hadoop-hdfs.txt | | Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/11538/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/11538/console | This message was automatically generated. > Expose the stats of IOErrors on each FsVolume through JMX > - > > Key: HDFS-8694 > URL: https://issues.apache.org/jira/browse/HDFS-8694 > Project: Hadoop HDFS > Issue Type: Improvement > Components: datanode, HDFS >Affects Versions: 2.7.0 >Reporter: Lei (Eddy) Xu >Assignee: Lei (Eddy) Xu > Attachments: HDFS-8694.000.patch > > > Currently, once DataNode hits an {{IOError}} when writing / reading block > files, it starts a background {{DiskChecker.checkDirs()}} thread. But if this > thread successfully finishes, DN does not record this {{IOError}}. > We need one measurement to count all {{IOErrors}} for each volume. -- This message was sent by Atlassian JIRA (v6.3.4#6332)