[jira] [Updated] (HDFS-8694) Expose the stats of IOErrors on each FsVolume through JMX
[
https://issues.apache.org/jira/browse/HDFS-8694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Junping Du updated HDFS-8694:
-
Target Version/s: (was: 2.8.0)
> Expose the stats of IOErrors on each FsVolume through JMX
> -
>
> Key: HDFS-8694
> URL: https://issues.apache.org/jira/browse/HDFS-8694
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
>Affects Versions: 2.7.0
>Reporter: Lei (Eddy) Xu
>Assignee: Lei (Eddy) Xu
> Attachments: HDFS-8694.000.patch, HDFS-8694.001.patch
>
>
> Currently, once DataNode hits an {{IOError}} when writing / reading block
> files, it starts a background {{DiskChecker.checkDirs()}} thread. But if this
> thread successfully finishes, DN does not record this {{IOError}}.
> We need one measurement to count all {{IOErrors}} for each volume.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-8694) Expose the stats of IOErrors on each FsVolume through JMX
[
https://issues.apache.org/jira/browse/HDFS-8694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsz Wo Nicholas Sze updated HDFS-8694:
--
Component/s: (was: HDFS)
> Expose the stats of IOErrors on each FsVolume through JMX
> -
>
> Key: HDFS-8694
> URL: https://issues.apache.org/jira/browse/HDFS-8694
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
>Affects Versions: 2.7.0
>Reporter: Lei (Eddy) Xu
>Assignee: Lei (Eddy) Xu
> Attachments: HDFS-8694.000.patch, HDFS-8694.001.patch
>
>
> Currently, once DataNode hits an {{IOError}} when writing / reading block
> files, it starts a background {{DiskChecker.checkDirs()}} thread. But if this
> thread successfully finishes, DN does not record this {{IOError}}.
> We need one measurement to count all {{IOErrors}} for each volume.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HDFS-8694) Expose the stats of IOErrors on each FsVolume through JMX
[
https://issues.apache.org/jira/browse/HDFS-8694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei (Eddy) Xu updated HDFS-8694:
Attachment: HDFS-8694.001.patch
Updated the patch to fix {{TestDataTransferKeepalive}}, the other failed unit
tests were passed locally on my machine.
Not sure about the timed out tests. So upload this patch to test again.
Expose the stats of IOErrors on each FsVolume through JMX
-
Key: HDFS-8694
URL: https://issues.apache.org/jira/browse/HDFS-8694
Project: Hadoop HDFS
Issue Type: Improvement
Components: datanode, HDFS
Affects Versions: 2.7.0
Reporter: Lei (Eddy) Xu
Assignee: Lei (Eddy) Xu
Attachments: HDFS-8694.000.patch, HDFS-8694.001.patch
Currently, once DataNode hits an {{IOError}} when writing / reading block
files, it starts a background {{DiskChecker.checkDirs()}} thread. But if this
thread successfully finishes, DN does not record this {{IOError}}.
We need one measurement to count all {{IOErrors}} for each volume.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HDFS-8694) Expose the stats of IOErrors on each FsVolume through JMX
[
https://issues.apache.org/jira/browse/HDFS-8694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei (Eddy) Xu updated HDFS-8694:
Attachment: (was: HDFS-8694.001.patch)
Expose the stats of IOErrors on each FsVolume through JMX
-
Key: HDFS-8694
URL: https://issues.apache.org/jira/browse/HDFS-8694
Project: Hadoop HDFS
Issue Type: Improvement
Components: datanode, HDFS
Affects Versions: 2.7.0
Reporter: Lei (Eddy) Xu
Assignee: Lei (Eddy) Xu
Attachments: HDFS-8694.000.patch
Currently, once DataNode hits an {{IOError}} when writing / reading block
files, it starts a background {{DiskChecker.checkDirs()}} thread. But if this
thread successfully finishes, DN does not record this {{IOError}}.
We need one measurement to count all {{IOErrors}} for each volume.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HDFS-8694) Expose the stats of IOErrors on each FsVolume through JMX
[
https://issues.apache.org/jira/browse/HDFS-8694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei (Eddy) Xu updated HDFS-8694:
Attachment: HDFS-8694.001.patch
Re-upload to fix {{TestMover}} failure. Passed all known failures in the
previous Jenkins run.
Expose the stats of IOErrors on each FsVolume through JMX
-
Key: HDFS-8694
URL: https://issues.apache.org/jira/browse/HDFS-8694
Project: Hadoop HDFS
Issue Type: Improvement
Components: datanode, HDFS
Affects Versions: 2.7.0
Reporter: Lei (Eddy) Xu
Assignee: Lei (Eddy) Xu
Attachments: HDFS-8694.000.patch, HDFS-8694.001.patch
Currently, once DataNode hits an {{IOError}} when writing / reading block
files, it starts a background {{DiskChecker.checkDirs()}} thread. But if this
thread successfully finishes, DN does not record this {{IOError}}.
We need one measurement to count all {{IOErrors}} for each volume.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Auto-Re: [jira] [Updated] (HDFS-8694) Expose the stats of IOErrors on each FsVolume through JMX
您的邮件已收到！谢谢！
[jira] [Updated] (HDFS-8694) Expose the stats of IOErrors on each FsVolume through JMX
[
https://issues.apache.org/jira/browse/HDFS-8694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei (Eddy) Xu updated HDFS-8694:
Status: Patch Available (was: Open)
Expose the stats of IOErrors on each FsVolume through JMX
-
Key: HDFS-8694
URL: https://issues.apache.org/jira/browse/HDFS-8694
Project: Hadoop HDFS
Issue Type: Improvement
Components: datanode, HDFS
Affects Versions: 2.7.0
Reporter: Lei (Eddy) Xu
Assignee: Lei (Eddy) Xu
Attachments: HDFS-8694.000.patch
Currently, once DataNode hits an {{IOError}} when writing / reading block
files, it starts a background {{DiskChecker.checkDirs()}} thread. But if this
thread successfully finishes, DN does not record this {{IOError}}.
We need one measurement to count all {{IOErrors}} for each volume.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HDFS-8694) Expose the stats of IOErrors on each FsVolume through JMX
[
https://issues.apache.org/jira/browse/HDFS-8694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei (Eddy) Xu updated HDFS-8694:
Attachment: HDFS-8694.000.patch
The # of IOerror is kept in {{FsVolumes}} and exposes it as
{{VolumeInfo}}:{{errors}} in {{JMX}}:
{code}
{
name : Hadoop:service=DataNode,name=DataNodeInfo,
modelerType : org.apache.hadoop.hdfs.server.datanode.DataNode,
XceiverCount : 1,
DatanodeNetworkCounts : [ ],
Version : 3.0.0-SNAPSHOT,
RpcPort : 50002,
HttpPort : null,
NamenodeAddresses :
{\localhost\:\BP-1143576736-127.0.0.1-1435617411301\},
VolumeInfo :
{\/foo/hadoop/test/data/dn0/vol0/current\:{\freeSpace\:100856463360,\errors\:0,\usedSpace\:8192,\reservedSpaceForRBW\:0,\reservedSpace\:0},\/foo/hadoop/test/data/dn0/vol1/current\:{\freeSpace\:100856463360,\errors\:0,\usedSpace\:8192,\reservedSpaceForRBW\:0,\reservedSpace\:0}},
ClusterId : CID-022d733d-9061-4f14-a990-8a93316ca57d
},
{code}
Expose the stats of IOErrors on each FsVolume through JMX
-
Key: HDFS-8694
URL: https://issues.apache.org/jira/browse/HDFS-8694
Project: Hadoop HDFS
Issue Type: Improvement
Components: datanode, HDFS
Affects Versions: 2.7.0
Reporter: Lei (Eddy) Xu
Assignee: Lei (Eddy) Xu
Attachments: HDFS-8694.000.patch
Currently, once DataNode hits an {{IOError}} when writing / reading block
files, it starts a background {{DiskChecker.checkDirs()}} thread. But if this
thread successfully finishes, DN does not record this {{IOError}}.
We need one measurement to count all {{IOErrors}} for each volume.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Auto-Re: [jira] [Updated] (HDFS-8694) Expose the stats of IOErrors on each FsVolume through JMX
您的邮件已收到！谢谢！
