We use a very simple data retrieval method for smallish datasets. The
data is just stored in memory or as a memory mapped file and a
sequential search used. It sounds crude but when you use a fast search
algorithm like Boyer-Moore it outperforms index methods up to a
surprisingly large number of records. As you can imagine the code
footprint is miniscule and if you add regular expression logic you can
realize very intricate search patterns.
We use the method in conjunction with a database to achieve an enormous
speed increase on "LIKE" type searches. Grep a few files to get a feel
for the performance.
Another method which works well for memory resident storage is to
implement self balancing AVL trees. The code is simple and the
performance lightning fast. With a little ingenuity you can use disk
storage. Mini Sql (MSql) is a good example of how this can be effective.
As Einstein said - "Make it as simple as possible, but not too simple".
Applying Occam's Razor can turn bloated solutions into more
effective lean ones. Typical solutions come in two sizes just like Army
boots - too big and too small.
Lloyd wrote:
Would anybody suggest a good tool for performance measurement (on
Linux) ?
On Wed, 2007-04-11 at 10:35 -0500, John Stanton wrote:
You might discover that you can craft a very effective memory
resident
storage system using a compression system like Huffman Encoding and
an
index method appropriate to the key you are using for retrieval.
That
could work very well in an embedded system, have a small footprint in
data and code and be very fast.
______________________________________
Scanned and protected by Email scanner
-----------------------------------------------------------------------------
To unsubscribe, send email to [EMAIL PROTECTED]
-----------------------------------------------------------------------------
-----------------------------------------------------------------------------
To unsubscribe, send email to [EMAIL PROTECTED]
-----------------------------------------------------------------------------
