pitrou commented on pull request #11588: URL: https://github.com/apache/arrow/pull/11588#issuecomment-960612064
> The problem seems to be the default prefetch. Is it? I'm skeptical that the kernel prefetches an entire 1GB file if you don't ask it to. > While WILLNEED and DONTNEED are specific prefetching instructions (and only apply to a range of data) the advice RANDOM, SEQUENTIAL, and NORMAL are file-wide instructions to control the default behavior. That's only for `posix_fadvise`, not `posix_madvise`. And it's Linux-specific... > Another way to phrase it would be "can we modify the file-wide default and still provide instructions for specific regions." What would that change? The file-wide default is probably not to prefetch. It doesn't seem to me that you are addressing the original complaint. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
