shubhamsrkdev commented on PR #15556: URL: https://github.com/apache/lucene/pull/15556#issuecomment-3739716938
> Why isn't this something that is done at the directory level? It's a a good idea. On a cursory look is `IOContext`a good candidate to add this? > If the idea is that prefetching is too costly, why don't we have something at the directory level that skips prefetching if the directory was opened on a particular kind of hardware? Or, we force users to do this? I am not a huge fan of yet another top level thing that spiders everywhere :/ it seems like a big change. So currently we sample pages to check if they're already loaded, and track by incrementing `hotCounter` when prefetching is skipped. If `hotCounter < 1000`, we consider the index cold and continue prefetching. I am not sure how/if this approach would transfer to hardware memory decisions? What do you think? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
