[
https://issues.apache.org/jira/browse/HDFS-6709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14073677#comment-14073677
]
Andrew Purtell commented on HDFS-6709:
--------------------------------------
bq. No, because every modern GC uses "generational collection." This means that
short-lived instances are quickly cleaned up, without any pauses.
... and modern JVM versions have escape analysis enabled by default. Although
there are limitations, simple objects that don't escape the local block (like
Iterators) or the method can be allocated on the stack once native code is
emitted by the server compiler. No heap allocation happens at all. You can use
fastdebug JVM builds during dev to learn explicitly what your code is doing in
this regard.
> Implement off-heap data structures for NameNode and other HDFS memory
> optimization
> ----------------------------------------------------------------------------------
>
> Key: HDFS-6709
> URL: https://issues.apache.org/jira/browse/HDFS-6709
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Attachments: HDFS-6709.001.patch
>
>
> We should investigate implementing off-heap data structures for NameNode and
> other HDFS memory optimization. These data structures could reduce latency
> by avoiding the long GC times that occur with large Java heaps. We could
> also avoid per-object memory overheads and control memory layout a little bit
> better. This also would allow us to use the JVM's "compressed oops"
> optimization even with really large namespaces, if we could get the Java heap
> below 32 GB for those cases. This would provide another performance and
> memory efficiency boost.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
