On December 14, 2001 12:59 pm, Dave Rolsky wrote: > On Fri, 14 Dec 2001, Perrin Harkins wrote: > > The thing you were missing is that on an OS with an aggressively caching > > filesystem (like Linux), frequently read files will end up cached in RAM > > anyway. The kernel can usually do a better job of managing an efficient > > cache than your program can. > > Plus for some reason IPC overhead seems to seriously degrade as the size > of the overall shared memory segment increases. I'm not sure why this is > but I think the docs for IPC::Shareable mention this. Maybe for _very_ > small amounts of data, shared memory might still be a win. >
Very interesting. We at UF were using IPC caching, in earlier versions of our comment system. We turfed it for several reasons. - Locking and storage were taking appreciable wall clock time. - We had a large memory footprint anyway, and a per-process hash did not actually add all that much. - IPC was persistent over server instances, but we already had a dbi connection, and that is persistent over system reboots as well. The earlier version was topping out at about 8 hits per second. So our solution was caching in-process with just a hash, and using a DBI/mysql persistent store. in pseudo code sub get_stuff { if (! $cache{$whatever} ) { if !( $cache{whatever} = dbi_lookup()) { $cache{$whatever}=derive_data_from_original_source($whatever); dbi_save($cache_whatever); } } $cache{$whatever} } Thanks to that and another local content cache implemented with the file system with time based aging, UF's latest version of the system can do over 100 page views per second on a relatively modest machine (P3 700, 256M IDE, linux 2.2.x) The filesystem based / time sensitive aging is a nice little thing we should put up in CPAN. We've just not done so yet. -- Jay "yohimbe" Thorne [EMAIL PROTECTED] Mgr Sys & Tech, Userfriendly.org