> The thing you were missing is that on an OS with an aggressively caching
> filesystem (like Linux), frequently read files will end up cached in RAM
> anyway. The kernel can usually do a better job of managing an efficient
> cache than your program can.
>
> For what it's worth, DeWitt Clinton accompanied his first release of
> File::Cache (the precursor to Cache::FileCache) with a benchmark showing
> this same thing. That was the reason File::Cache was created.
While that's true, there are still some problems with a file cache. Namely
to get reasonably performance you have to do the directory hashing structure
so that you don't end up with too many files (one for each key) in one
directory. Thus for every add to the cache you have to:
* stat each directory in the hash path and create it if it doesn't exist
* open and create the file and write to it, close the file
A similar method is required for reading. All that still takes a bit of
time. This is where having some shared memory representation can be a really
help since you don't have to traverse, open, read/write, close a file every
time. Witness the performance of IPC::MM which seems to be mostly limited by
the performance of the Storable module. I'm planning on doing another test
which just stores some data without the 'streaming' to see which examples
are really limited by Storable and which by their implementation, this might
be useful for some people.
> And ++ on Paul's comments about Devel::DProf and other profilers.
Ditto again. I've been using Apache::DProf recently and it's been great at
tracking down exactly where time is spent in my program. If you have any
performance problems, definitely use it first before making any assumptions.
One question though, I have a call tree that looks a bit like:
main
-> calls f1
-> calls f2
-> calls f3
-> calls f2
The two calls to f2 may take completely different times. Using 'dprofpp -I',
I can see what percentage of overall time is spent in 'f1 and children', 'f3
and children' and 'f2 and children'. But I can't see an easy way to tell
'time in f2 and children when f2 was called from f1' (or ditto for f3). Does
that make sense?
Rob
