[
https://issues.apache.org/jira/browse/ARROW-7610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17019056#comment-17019056
]
Jacques Nadeau commented on ARROW-7610:
---------------------------------------
Per comments on the mailing list,
I think we should use the existing netty facades for this.
[PlatformDependent.allocateMemory(long)|https://github.com/netty/netty/blob/4.1/common/src/main/java/io/netty/util/internal/PlatformDependent.java#L383]
[PlatformDependent.freeMemory(long)|https://github.com/netty/netty/blob/4.1/common/src/main/java/io/netty/util/internal/PlatformDependent.java#L387]
These delegate to unsafe which also ensures that -XX:MaxDirectMemorySize is
respected.
I don't think we need a new allocator. We can simply check the size and use
this path if beyond an int in size within the existing allocator.
> [Java] Finish support for 64 bit int allocations
> -------------------------------------------------
>
> Key: ARROW-7610
> URL: https://issues.apache.org/jira/browse/ARROW-7610
> Project: Apache Arrow
> Issue Type: Bug
> Components: Java
> Reporter: Micah Kornfield
> Assignee: Liya Fan
> Priority: Major
>
> 1. Add an allocator capable of allocating larger then 2GB of data.
> 2. Do end-to-end round trip trip on a larger vector/record batch size.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
