zhztheplayer commented on pull request #7030: URL: https://github.com/apache/arrow/pull/7030#issuecomment-658605439
> I was suggesting a third approach: C++ memory pool that simply updates the direct memory info via Bits in java. It should be done chunk-wise to avoid excessive JNI boundary calls. It looks a little bit more complicated: IIUC Netty doesn't seem to reserve from Bits internally, it has a individual counter that is limited to max direct mem amount. If we use Bits then we'll be able to exceed the overall usage by using up to 2x amount. My suggestion is to maintain our own mechanism to reserve Bits for every byte Java allocator managed, then ignore Netty's own direct mem counter. This way no JNI overhead will be brought. Would you think of this a feasible direction? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
