[
https://issues.apache.org/jira/browse/HADOOP-2816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12628160#action_12628160
]
Suresh Srinivas commented on HADOOP-2816:
-----------------------------------------
For reporting the following info needs to be considered:
Total capacity - Capacity of all the data directories
Reserved space - Space reserved for non DFS usage
dfs.datanode.du.pct - When calculating DFS remaining space, only use this
percentage of the real available space
Here is how DFS remaining space is calculated:
Available space is Minimum of (Available space on local file system) or (Total
capacity - DFS used space - Reserved space)
DFS remaining = (dfs.datanode.du.pct) * Available space
Current proposal does not consider the factor dfs.datanode.du.pct. I am not
sure why du.pct is being used. If it is to reduce available disk space for DFS,
to consider factors such as disk fragmentation - it is not serving the purpose.
Available space keeps on decreasing. The percentage is applied to the shrinking
available space. Eventually the DFS ends up using all the available space any
way (in theory) and the du.pct will not serve any purpose.
My proposal:
1) Remove du.pct configuration option
or
2) If du.pct is used, it is calculated on Total capacity and not on available
space. This helps set aside a percentage of total capacity.
> Cluster summary at name node web has confusing report for space utilization
> ---------------------------------------------------------------------------
>
> Key: HADOOP-2816
> URL: https://issues.apache.org/jira/browse/HADOOP-2816
> Project: Hadoop Core
> Issue Type: Improvement
> Components: dfs
> Affects Versions: 0.15.3
> Reporter: Robert Chansler
> Assignee: Suresh Srinivas
>
> In one example:
> Cluster Summary
> Capacity : 1.15 PB
> DFS Remaining : 192 TB
> DFS Used : 717 TB
> DFS Used% : 62 %
> Why is Capacity not equal Used plus Remaining?
> (The answer is that there is an estimated reserve for local files.)
> The presentation should be easily understood by the user.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
