HDFS-11282. Document the missing metrics of DataNode Volume IO operations. Contributed by Yiqun Lin.
Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/4a659ff4 Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/4a659ff4 Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/4a659ff4 Branch: refs/heads/HADOOP-13345 Commit: 4a659ff40fca7c263d62ac7514afc100a4dbb1ed Parents: 0b8a7c1 Author: Arpit Agarwal <a...@apache.org> Authored: Thu Jan 5 19:34:32 2017 -0800 Committer: Arpit Agarwal <a...@apache.org> Committed: Thu Jan 5 19:34:32 2017 -0800 ---------------------------------------------------------------------- .../hadoop-common/src/site/markdown/Metrics.md | 31 ++++++++++++++++++++ 1 file changed, 31 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/hadoop/blob/4a659ff4/hadoop-common-project/hadoop-common/src/site/markdown/Metrics.md ---------------------------------------------------------------------- diff --git a/hadoop-common-project/hadoop-common/src/site/markdown/Metrics.md b/hadoop-common-project/hadoop-common/src/site/markdown/Metrics.md index 1d5ea5e..541f5dc 100644 --- a/hadoop-common-project/hadoop-common/src/site/markdown/Metrics.md +++ b/hadoop-common-project/hadoop-common/src/site/markdown/Metrics.md @@ -323,6 +323,37 @@ Each metrics record contains tags such as SessionId and Hostname as additional i | `RemoteBytesWritten` | Number of bytes written by remote clients | | `BPServiceActorInfo` | The information about a block pool service actor | +FsVolume +-------- + +Per-volume metrics contain Datanode Volume IO related statistics. Per-volume metrics are off by default. They can be enbabled by setting `dfs.datanode.fileio.events.class` to **org.apache.hadoop.hdfs.server.datanode.ProfilingFileIoEvents**, but enabling per-volume metrics may have a performance impact. Each metrics record contains tags such as Hostname as additional information along with metrics. + +| Name | Description | +|:---- |:---- | +| `TotalMetadataOperations` | Total number (monotonically increasing) of metadata operations. Metadata operations include stat, list, mkdir, delete, move, open and posix_fadvise. | +| `MetadataOperationRateNumOps` | The number of metadata operations within an interval time of metric | +| `MetadataOperationRateAvgTime` | Mean time of metadata operations in milliseconds | +| `MetadataOperationLatency`*num*`s(50|75|90|95|99)thPercentileLatency` | The 50/75/90/95/99th percentile of metadata operations latency in milliseconds. Percentile measurement is off by default, by watching no intervals. The intervals are specified by `dfs.metrics.percentiles.intervals`. | +| `TotalDataFileIos` | Total number (monotonically increasing) of data file io operations | +| `DataFileIoRateNumOps` | The number of data file io operations within an interval time of metric | +| `DataFileIoRateAvgTime` | Mean time of data file io operations in milliseconds | +| `DataFileIoLatency`*num*`s(50|75|90|95|99)thPercentileLatency` | The 50/75/90/95/99th percentile of data file io operations latency in milliseconds. Percentile measurement is off by default, by watching no intervals. The intervals are specified by `dfs.metrics.percentiles.intervals`. | +| `FlushIoRateNumOps` | The number of file flush io operations within an interval time of metric | +| `FlushIoRateAvgTime` | Mean time of file flush io operations in milliseconds | +| `FlushIoLatency`*num*`s(50|75|90|95|99)thPercentileLatency` | The 50/75/90/95/99th percentile of file flush io operations latency in milliseconds. Percentile measurement is off by default, by watching no intervals. The intervals are specified by `dfs.metrics.percentiles.intervals`. | +| `SyncIoRateNumOps` | The number of file sync io operations within an interval time of metric | +| `SyncIoRateAvgTime` | Mean time of file sync io operations in milliseconds | +| `SyncIoLatency`*num*`s(50|75|90|95|99)thPercentileLatency` | The 50/75/90/95/99th percentile of file sync io operations latency in milliseconds. Percentile measurement is off by default, by watching no intervals. The intervals are specified by `dfs.metrics.percentiles.intervals`. | +| `ReadIoRateNumOps` | The number of file read io operations within an interval time of metric | +| `ReadIoRateAvgTime` | Mean time of file read io operations in milliseconds | +| `ReadIoLatency`*num*`s(50|75|90|95|99)thPercentileLatency` | The 50/75/90/95/99th percentile of file read io operations latency in milliseconds. Percentile measurement is off by default, by watching no intervals. The intervals are specified by `dfs.metrics.percentiles.intervals`. | +| `WriteIoRateNumOps` | The number of file write io operations within an interval time of metric | +| `WriteIoRateAvgTime` | Mean time of file write io operations in milliseconds | +| `WriteIoLatency`*num*`s(50|75|90|95|99)thPercentileLatency` | The 50/75/90/95/99th percentile of file write io operations latency in milliseconds. Percentile measurement is off by default, by watching no intervals. The intervals are specified by `dfs.metrics.percentiles.intervals`. | +| `TotalFileIoErrors` | Total number (monotonically increasing) of file io error operations | +| `FileIoErrorRateNumOps` | The number of file io error operations within an interval time of metric | +| `FileIoErrorRateAvgTime` | It measures the mean time in milliseconds from the start of an operation to hitting a failure | + yarn context ============ --------------------------------------------------------------------- To unsubscribe, e-mail: common-commits-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-commits-h...@hadoop.apache.org
