Comparison of different caching schemes

[Rob Mueller (fastmail)]

Just thought people might be interested...   I sat down the other day and wrote a test script to try out various caching implementations. The script is pretty basic at the moment, I just wanted to get an idea of the performance of different methods.   The basic scenario is the common mod_perl situation: * Multiple processes * You need to share perl structures between processes * You want to index the data structure on some key * Same key is written and read multiple times   I tried out the following systems. * Null reference case (just store in 'in process' hash) * Storable reference case (just store in 'in process' hash after 'freeze') * Cache::Mmap (uses Storable) * Cache::FileCache (uses Storable) * DBI (I used InnoDB), use Storable, always do 'delete' then 'insert' * DBI, use Storable, do 'select' then 'insert' or 'update' * MLDBM::Sync::SDBM_File (uses Storable) * Cache::SharedMemoryCache (uses Storable)   For systems like Cache::* which can automatically delete items after an amount of time or size, I left these options off or made the cache big enough that this wouldn't happen.   I've included the script for people to try out if they like and add other test cases.   Now to the results, here they are.   Package C0 - In process hash Sets per sec = 147116 Gets per sec = 81597 Mixes per sec = 124120 Package C1 - Storable freeze/thaw Sets per sec = 2665 Gets per sec = 6653 Mixes per sec = 3880 Package C2 - Cache::Mmap Sets per sec = 809 Gets per sec = 3235 Mixes per sec = 1261 Package C3 - Cache::FileCache Sets per sec = 393 Gets per sec = 831 Mixes per sec = 401 Package C4 - DBI with freeze/thaw Sets per sec = 651 Gets per sec = 1648 Mixes per sec = 816 Package C5 - DBI (use updates with dup) with freeze/thaw Sets per sec = 657 Gets per sec = 1994 Mixes per sec = 944 Package C6 - MLDBM::Sync::SDBM_File Sets per sec = 334 Gets per sec = 1279 Mixes per sec = 524 Package C7 - Cache::SharedMemoryCache Sets per sec = 42 Gets per sec = 29 Mixes per sec = 32   Notes: * System = Pentium III 866, Linux 2.4.16-0.6, Ext3 (no special file filesystem flags), MySQL (with InnoDB tables) * Null reference hash is slower reading because it does a very basic check to see that the retrieved hash has the same number of keys as the stored hash * Approximate performance order (best to worst) = Cache::Mmap, DBI, MLDBM::Sync::SDBM_File, Cache::FileCache, Cache::SharedMemoryCache * Remember what Knuth said, "Premature optimisation is the root of all evil." This data won't help you if something else in your application is the bottleneck... * The code is available at: http://fastmail.fm/users/robm/perl/cacheperltest.pl   Have I missed something obvious?   Rob . <- Grain of salt to be taken with this post