Dandandan commented on PR #23492: URL: https://github.com/apache/datafusion/pull/23492#issuecomment-5175612108
> What can we expect for memory consumption? Will we use more memory because we prefetch? Yes. The extra memory consumption is pretty low (and can be configured) - it is roughly max the IO budget x partitions (probably slightly higher because of overhead / Tokio tasks running, but generally much lower). Thus 16 (partitions/cores) * 8 MiB = ~128 MiB -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
