Hi,
Inline explanation:
On Tue, Nov 6, 2012 at 1:41 PM, Elaine Gan <elaine-...@gmo.jp> wrote:
> Hi,
>
> I'm a bit confused on the local disk and the hdfs disk.
> Through my admin page, i see information like the following.
>
> Configured Capacity : 254.05 GB
Configured Capacity = Sum(Configured DN Capacity)
= 64*4 = 256.
> DFS Used : 207.09 GB
HDFS files are cumulatively using 207 GB of data. This includes
replicated blocks (default replication factor is 3).
> Non DFS Used : 16.87 GB
This is Sum(Configured DN Space) that lies outside the HDFS files.
HDFS runs utilities such as "du"/"df" on disks and can pick up space
that is being utilized by non HDFS, local files such as log files,
etc. lying in the same disk as the one used for DNs.
> DFS Remaining : 30.09 GB
Your DFS has only 30 GB more to spare, which with a replication factor
of 3, is less than 10 GB of true file length space.
> DFS Used% : 81.52 %
> DFS Remaining% : 11.84 %
These should be self-explainatory.
> I only made configuration on the heap memory and not the disk capacity.
> Where does "Configured Capacity : 254.05 GB " this comes from?
> I have 4 nodes, each with 64GB of disk (local).
> Will HDFS use it fully ?
Yes it would, unless you configure the "dfs.datanode.du.reserved"
config param at each DN to a space value in bytes that must be left
free on all configured volumes.
> I still need some place for local files.
>
> Thank you.
>
Hope this helps!
--
Harsh J
