Jon Kay wrote: > Nick Lewycky wrote: > >>Finally, does anyone have suggestions for how to test for performance >>improvement due to prefetching? > > A good way to test how your algorithms are working is to get a nice, long > actual Squid workload -eg, URLs fetched, and compare how long it takes > to execute the whole thing with and without prefetching.
That's a very good plan. Does anyone have recent logs publicly available? I have some IRCache logs for the day of May 31, 2004 -- but when I tried the first 5,000 entries, I found that 87% of the prefetches weren't fetched later in the log. I think this is mostly because the pages changed after that date and also because of filtering effects from client caching. What I'd really like to have is a way to look at the page load times instead of running through individual URLs. > Note that you generally have to prefetch a LOT of stuff to get much > improvement, > because web cache fetch popularity follows zipf's law and decays slowly. I hadn't heard of Zipf's law. It's interesting, thank you for introducing me to it! Just to make certain I understand what you're saying ... you're noting that I need a lot of log data to test with because most fetches enter the working set where prefetching won't help and so I need a large number of cache misses? > Good luck with your work. Thank you! Nick Lewycky