[
https://issues.apache.org/jira/browse/DIRECTMEMORY-63?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13211302#comment-13211302
]
Michael André Pearce commented on DIRECTMEMORY-63:
--------------------------------------------------
If we do go down this route, we can then start looking at a few options of
getting the pointers off heap.
1) using the hash like a disk does, and using paging of the hashes against
pointers which each page can be essentially be a form of buffer (fully off heap)
2) or a simpler solution for now would be using the something like what youve
already looked into http://code.google.com/p/vanilla-java/wiki/HugeCollections
where we simply store in our on heap map, the hash and the location in the
arraylist (both primitive's very low gc overhead). e.g. hash 19283746464 maps
to position 4 and then we goto the offheap list, at that position to get the
main off heap pointer. This would allow then an option that we could store the
underlying key object in a key offheap list , which then allows us to return
the list current keys if required (many uses cases dont need this though in my
caching experience and as such we could make this an optional config switch to
use or not)
> Investigate removing the need for an on heap map to store keys/pointers.
> ------------------------------------------------------------------------
>
> Key: DIRECTMEMORY-63
> URL: https://issues.apache.org/jira/browse/DIRECTMEMORY-63
> Project: Apache DirectMemory
> Issue Type: Improvement
> Reporter: Michael André Pearce
>
> On performance testing and monitoring jvm heap allocation, it became clear
> that as the off heap elements increased so did the on heap used memory. This
> is because DirectMemory is storing keys and pointers in the heap meaning that
> the cache isn't fully off heap, and on heap memory will increase linearly
> with elements in the cache.
> An idea maybe is to look into using a key's hash for location on a separate
> single off heap buffer to store the main heaps pointers.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
