zhztheplayer commented on PR #33744: URL: https://github.com/apache/arrow/pull/33744#issuecomment-1387359104
Thanks and would you suggest to add a new implementation of interface `BufferAllocator` (e.g. `AutoCleanBufferAllocator`)? If yes I can try start from there. > the GC does not see into native allocations and you can run out of memory despite having 'free' memory because the GC has not collected the small Java-side objects that hold the large native buffers. Yes I can image something like that too. What I can try is to reuse the JVM's direct memory counter which can trigger GC when reaching its limit, or rebuild a similar counter like that. Not sure if there are some better solutions but this one is probably OK since it was at least from JVM itself. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
