[
https://issues.apache.org/jira/browse/HADOOP-803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12469918
]
Hairong Kuang commented on HADOOP-803:
--------------------------------------
> If the blocks were created after Namenode is started, one in inode will share
> the object with blockMap. When name nodes starts up, it initially creates all
> the blocks in Indode while reading the image file.. does not seem easy to
> share that reference.
Even after a name node starts up, a block in inode does not share the object
with blockMap. So a name node contains two Block instantiations per block.
To reduce the number of block instantiations to 1, I think we can create a
TreeSet of blocks which maps a block id to its block object. When a block is
intially created, simply add the block to this block set. When later a
reference to the block is needed, we can get it from the block set.
We can furthur remove the blockMap and activeBlock list by adding two
non-persistent fields to the Block data structures. One is a reference to all
its containing data nodes and one to the inode representing the file that the
block belongs to.
> Reducing memory consumption on Namenode : Part 1
> ------------------------------------------------
>
> Key: HADOOP-803
> URL: https://issues.apache.org/jira/browse/HADOOP-803
> Project: Hadoop
> Issue Type: Bug
> Components: dfs
> Reporter: Raghu Angadi
> Assigned To: Raghu Angadi
> Fix For: 0.12.0
>
> Attachments: block-refs-2.patch, block-refs-3.patch,
> block-refs-5.patch, HADOOP-803-2.patch, HADOOP-803.patch, HADOOP-803_3.patch,
> HADOOP-803_4.patch, NameNodeMemoryHogs.txt
>
>
> There appears to be some places in Namenode that allow reducing memory
> consumption without intrusive code or feature changes. This bug is an initial
> attempt making those changes. Please include your thoughts as well.
> One change I am planning to make :
> Currently one copy of each block exists for each of the replicas and one copy
> for blockMap. I think they are all supposed to be same.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
