jonded94 commented on issue #44599: URL: https://github.com/apache/arrow/issues/44599#issuecomment-2479330073
Our use case actually would requires that the workloads require as little memory as possible and also hand it back over to the system as soon as possible. If reading in a big file suddenly keeps all processes at ~2GiB more memory than they ideally would need, this is somewhat of a dealbreaker. That all memory pools per default *don't* hand memory which is no longer required back to the system certainly could be considered unexpected by some. That for some reason, only the `system` memory pool seems to react to `.release_unused()` calls, could be even considered a bug? It's unclear for me as a user how I would make sure to explicitly free that unused memory. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
