[
https://issues.apache.org/jira/browse/HDFS-1110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12865889#action_12865889
]
dhruba borthakur commented on HDFS-1110:
----------------------------------------
Thanks Suresh for the numbers.
> agree with Dhruba that we need to optimize only for the top ten (or so) file
> names, which will give us 5% saving in the meta
One fear I have is that the regex matching inside the fsnamesystem lock could
increase CPU increase in such a way that the 5% gain in memory might not be a
good tradeoff. any thoughts on this?
> Namenode heap optimization - reuse objects for commonly used file names
> -----------------------------------------------------------------------
>
> Key: HDFS-1110
> URL: https://issues.apache.org/jira/browse/HDFS-1110
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: Suresh Srinivas
> Assignee: Suresh Srinivas
> Fix For: 0.22.0
>
> Attachments: hdfs-1110.2.patch, hdfs-1110.patch
>
>
> There are a lot of common file names used in HDFS, mainly created by
> mapreduce, such as file names starting with "part". Reusing byte[]
> corresponding to these recurring file names will save significant heap space
> used for storing the file names in millions of INodeFile objects.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
