With the recently merged changes to the underlying ArrowBuf APIs to allow 64-bit memory address spaces there is some follow-up work to actually confirm it works. I opened a JIRA [1] to track this work.
The main question is how to provide an allocator that supports larger memory chunks. It appears the Netty API only takes an 32-bit integer for array sizes. Doing a little bit of investigation it seems like JNA [2] exposes a direct call to malloc of 64-bit integers [3]. The other options would seem to be rolling our own allocator via JNI. Is there anybody worked with JNA and can share experiences? Is anyone familiar with other options? Thanks, Micah [1] https://issues.apache.org/jira/browse/ARROW-7606 [2] https://github.com/java-native-access/jna [3] https://github.com/java-native-access/jna/blob/master/src/com/sun/jna/Native.java#L2265
