[
https://issues.apache.org/jira/browse/HDFS-6930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14118657#comment-14118657
]
Colin Patrick McCabe commented on HDFS-6930:
--------------------------------------------
bq. Eviction is done when we have Less than 10% free space or Insufficient
space for 3 default length blocks.
One thing that might be suboptimal here is that we're using the
{{dfs.blocksize}} configuration key on the DataNode and assuming that will be
the same value used by the client. Clearly, the client could use 256 MB
blocks, whereas the DN could use 128 MB blocks. Etc.
Also, we don't really know how big the ramdisks are going to be. I can easily
see a 300 GB ramdisk being used in a few years. Just defaulting to keeping 10%
free seems like too much.
So, why not just have a minimum free space configuration key. It could be
specified as a number of bytes, rather than as a percentage. So we could
default it to 128 MB * 3 to get your current default of leaving space for 3
blocks. This would work better for bigger ramdisks (unlike a percentage-based
scheme) and wouldn't make assumptions about the client's and DN's block size
configuration being the same.
> Improve replica eviction from RAM disk
> --------------------------------------
>
> Key: HDFS-6930
> URL: https://issues.apache.org/jira/browse/HDFS-6930
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode
> Affects Versions: HDFS-6581
> Reporter: Arpit Agarwal
> Assignee: Arpit Agarwal
> Attachments: HDFS-6930.01.patch
>
>
> The current replica eviction scheme is inefficient since it performs multiple
> file operations in the context of block allocation.
> A better implementation would be asynchronous eviction when free space on RAM
> disk falls below a low watermark to make block allocation faster.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
