[
https://issues.apache.org/jira/browse/PHOENIX-2405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15201908#comment-15201908
]
stack commented on PHOENIX-2405:
--------------------------------
bq. Add means we can have a switch to use or not use Mnemonic way to deal with
memory allocation.
So, phoenix would have to provide an alternate for case where mnemonic is not
available or not properly installed or is missing native, supporting libs? Or,
does mnemonic do a noop or a fallback if a support is missing; e.g. you ask for
nvram but none installed, what does mnemonic do? Does it fail to start or crash
out or go a different route? I see I can ask to build without native lib
support when I build it but am wondering if at runtime, you can ask the lib if
it is good-to-go, coherent.. These questions belong better on the mnemonic
mailing lists (when they get set up). Sounds like mnemonic would be good fit
here (or phoenix runs inside the hbase process; the hbase process already
sports a chunking/block store; the supporting environment could provide phoenix
with mnemonic interface?) but phoenix would still need an alternative and it
might be a while yet before mnemonic a viable option? Thanks [~ywang261]
> Improve performance and stability of server side sort for ORDER BY
> ------------------------------------------------------------------
>
> Key: PHOENIX-2405
> URL: https://issues.apache.org/jira/browse/PHOENIX-2405
> Project: Phoenix
> Issue Type: Bug
> Reporter: James Taylor
> Assignee: Maryann Xue
> Labels: gsoc2016
> Fix For: 4.8.0
>
>
> We currently use memory mapped files to buffer data as it's being sorted in
> an ORDER BY (see MappedByteBufferQueue). The following types of exceptions
> have been seen to occur:
> {code}
> Caused by: java.lang.OutOfMemoryError: Map failed
> at sun.nio.ch.FileChannelImpl.map0(Native Method)
> at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:904)
> {code}
> [~apurtell] has read that memory mapped files are not cleaned up after very
> well in Java:
> {quote}
> "Map failed" means the JVM ran out of virtual address space. If you search
> around stack overflow for suggestions on what to do when your app (in this
> case Phoenix) encounters this issue when using mapped buffers, the answers
> tend toward manually cleaning up the mapped buffers or explicitly triggering
> a full GC. See
> http://stackoverflow.com/questions/8553158/prevent-outofmemory-when-using-java-nio-mappedbytebuffer
> for example. There are apparently long standing JVM/JRE problems with
> reclamation of mapped buffers. I think we may want to explore in Phoenix a
> different way to achieve what the current code is doing.
> {quote}
> Instead of using memory mapped files, we could use heap memory, or perhaps
> there are other mechanisms too.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
