[
https://issues.apache.org/jira/browse/HDFS-7129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14152017#comment-14152017
]
Arpit Agarwal commented on HDFS-7129:
-------------------------------------
Hi Xiaoyu, thanks for taking up this task! Nice updates to the test cases.
Few comments below:
# We should not increment addRamDiskBytesWrite by block length in createRbw
since at this point we are just reserving space for the block. No data has been
written. This metric can be incremented in finalizeReplica when the final block
length is known.
# Similarly for addRamDiskBytesWriteFallback. This metric can be hard to track
since we'd need to keep state until finalize that the write was originally
requested on RAM Disk. We can skip this metric and just have
incrRamDiskBlocksWriteFallback.
# When you call incrRamDiskBytesLazyPersisted, you can avoid the system call to
get the file length. It is available as {{replicaInfo#getNumBytes}}
# RamDiskReplica#numReads could be renamed to numAccesses.
# ramDiskBlocksEvictionWindow and ramDiskBlocksLazyPersistWindow should be
MutableQuantile.
Looks good otherwise.
> Metrics to track usage of memory for writes
> -------------------------------------------
>
> Key: HDFS-7129
> URL: https://issues.apache.org/jira/browse/HDFS-7129
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode
> Affects Versions: HDFS-6581
> Reporter: Arpit Agarwal
> Assignee: Xiaoyu Yao
> Attachments: HDFS-7129.0.patch, HDFS-7129.1.patch
>
>
> A few metrics to evaluate feature usage and suggest improvements. Thanks to
> [~sureshms] for some of these suggestions.
> # Number of times a block in memory was read (before being ejected)
> # Average block size for data written to memory tier
> # Time the block was in memory before being ejected
> # Number of blocks written to memory
> # Number of memory writes requested but not satisfied (failed-over to disk)
> # Number of blocks evicted without ever being read from memory
> # Average delay between memory write and disk write (window where a node
> restart could cause data loss).
> # Replicas written to disk by lazy writer
> # Bytes written to disk by lazy writer
> # Replicas deleted by application before being persisted to disk
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
