[
https://issues.apache.org/jira/browse/HADOOP-2816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Chansler updated HADOOP-2816:
------------------------------------
Release Note: Improved space reporting for NameNode Web UI. Applications
that parse the Web UI output should be reviewed. (was: This fix changes the
following:
1) Datanode heartbeat reported Capacity information is changed. Earlier the
Capacity was sum of all the diskspace of data directories. With this change, it
is sum of all the diskspace of data directories minus the reserved space
configured using dfs.datanode.du.reserved config param. This change is
reflected by changing the protocol version from 17 to 18.
2) The Namenode Web UI is changed accordingly as detailed below...
Cluster Summary
Capacity : Currently, this is sum of the file system capacity of all the data
directories. This is changed to Sum of the file system capacity of all the data
directories minus Reserved space. The name is changed to "Configured Capacity".
Present Capacity: This is newly added and represents the present capacity
available for DFS use. This is sum of DFS Remaining and DFS Used given below
DFS Remaining : This will remain as it is
DFS Used : This will remain as it is
DFS Used% : This is changed. It is calculated based on Present Capacity and not
Configured Capacity.
Live Nodes : This will remain as it is
Dead Nodes : This will remain as it is
Node data prints currently:
Node Last Contact Admin State Size (TB) Used (%) Used (%) Remaining (TB) Blocks
It will be change to:
Node Last Contact Admin State Capacity (TB) Present Capacity (TB) Used (%) Used
(%) Remaining (TB) Blocks
Size column is renamed as Total Capacity. Previously this was calculated as sum
of file system capacity of all the data directories. It is changed to exclude
reserved space and will be calculated as (sum of file system capacity of all
the data directories - reserved space)
)
Hadoop Flags: [Incompatible change, Reviewed] (was: [Reviewed,
Incompatible change])
This fix changes the following:
1) Datanode heartbeat reported Capacity information is changed. Earlier the
Capacity was sum of all the diskspace of data directories. With this change, it
is sum of all the diskspace of data directories minus the reserved space
configured using dfs.datanode.du.reserved config param. This change is
reflected by changing the protocol version from 17 to 18.
2) The Namenode Web UI is changed accordingly as detailed below...
Cluster Summary
Capacity : Currently, this is sum of the file system capacity of all the data
directories. This is changed to Sum of the file system capacity of all the data
directories minus Reserved space. The name is changed to "Configured Capacity".
Present Capacity: This is newly added and represents the present capacity
available for DFS use. This is sum of DFS Remaining and DFS Used given below
DFS Remaining : This will remain as it is
DFS Used : This will remain as it is
DFS Used% : This is changed. It is calculated based on Present Capacity and not
Configured Capacity.
Live Nodes : This will remain as it is
Dead Nodes : This will remain as it is
Node data prints currently:
Node Last Contact Admin State Size (TB) Used (%) Used (%) Remaining (TB) Blocks
It will be change to:
Node Last Contact Admin State Capacity (TB) Present Capacity (TB) Used (%) Used
(%) Remaining (TB) Blocks
Size column is renamed as Total Capacity. Previously this was calculated as sum
of file system capacity of all the data directories. It is changed to exclude
reserved space and will be calculated as (sum of file system capacity of all
the data directories - reserved space)
> Cluster summary at name node web has confusing report for space utilization
> ---------------------------------------------------------------------------
>
> Key: HADOOP-2816
> URL: https://issues.apache.org/jira/browse/HADOOP-2816
> Project: Hadoop Core
> Issue Type: Improvement
> Components: dfs
> Reporter: Robert Chansler
> Assignee: Suresh Srinivas
> Fix For: 0.19.0
>
> Attachments: HADOOP-2816.patch, HADOOP-2816.patch, HADOOP-2816.patch
>
>
> In one example:
> Cluster Summary
> Capacity : 1.15 PB
> DFS Remaining : 192 TB
> DFS Used : 717 TB
> DFS Used% : 62 %
> Why is Capacity not equal Used plus Remaining?
> (The answer is that there is an estimated reserve for local files.)
> The presentation should be easily understood by the user.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
