[
https://issues.apache.org/jira/browse/HDFS-5957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13918720#comment-13918720
]
Colin Patrick McCabe commented on HDFS-5957:
--------------------------------------------
bq. This doesn't make sense to me – direct buffers are off-heap and allocating
one is really just a wrapper around Unsafe.allocateMemory(), which itself is
just a regular old malloc.
{{ByteBuffer#allocateDirect}} is known to be a lot slower than
{{ByteBuffer#allocate}}. One of the reasons is that if the constructor of
{{DirectByteBuffer}} calls {{Bits.reserveMemory(cap)}}, which sometimes does a
{{System.gc}} and {{sleep(100ms)}}.
The code is here:
http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/6-b14/java/nio/DirectByteBuffer.java
It does come down to a {{malloc}} at the end of the day, but it seems like the
JDK (well, openJDK 6 at least) makes some effort to share a single pool of
space between the direct and normal java memory, even going so far as to
trigger a GC to keep things tidy.
bq. That's correct. Zero is interpreted as a special value meaning "don't do
any mmaps".
Yeah. I wonder if we should revisit that and create a separate 'enable' flag
for the client, before the configuration here gets set in stone.
> Provide support for different mmap cache retention policies in
> ShortCircuitCache.
> ---------------------------------------------------------------------------------
>
> Key: HDFS-5957
> URL: https://issues.apache.org/jira/browse/HDFS-5957
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: hdfs-client
> Affects Versions: 2.3.0
> Reporter: Chris Nauroth
>
> Currently, the {{ShortCircuitCache}} retains {{mmap}} regions for reuse by
> multiple reads of the same block or by multiple threads. The eventual
> {{munmap}} executes on a background thread after an expiration period. Some
> client usage patterns would prefer strict bounds on this cache and
> deterministic cleanup by calling {{munmap}}. This issue proposes additional
> support for different caching policies that better fit these usage patterns.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
