If you want to get as much as possible of the Sqlite database into
physical memory on the computer, read all of it. If you just want to
preload the cache and VM so that the first user gets faster response
execute a dummy query like one you would expect the first user to run.
Since the VM logic will work on a least recently used algorithm the
preload using a read of the file will not be very successful if you have
a huge database and not much physical memory. You will only have the
tail of the DB in physical memory.
My preference would be to execute a dummy query as part of the DB open.
That would leave the cache primed with a working set of pages likely
to satisfy the first user query. After that the cache takes care of
itself. This method not only primes the Sqlite cache but also makes the
pages resident in physical memory. Reading the file alone does not
prime the Sqlite cache.
Kees Nuyt wrote:
On Mon, 11 Dec 2006 19:50:52 -0600, John Stanton wrote:
Use a Unix read or a Windows API ReadFile and read the whole file
in one call.
Find the size of it using a Unix lseek or
Win32 API GetFileSize call. You will certainly
pull the whole file into VM that way.
Would that be wise for a database larger than a few tens of
MegaBytes? I guess it could cause some serious swapping,
invalidate hardware disk caches again, and make the rest of the
system slow...
The large block (does Visual Basic support such a large 'blob'
variable?) you'd read would be in Virtual Memory all right, but
as there is no relation between a large variable in memory and
the sqlite database, any database access would cause pages to be
read from disk again, the memory would not be used, only
hardware- and software diskcaches.
Read/discard in 64 kByte chunks seems just a little better, that
could still load the database in the OS disk cache, depending on
the disk cache size, and VM strategy settings of the system.
Defining a large sqlite database page cache and prefilling it
with the most frequently used pages by executing a carefully
crafted query would be a better answer, but that only helps when
the database is kept open all the time.
Frankly, I wouldn't bother; I'd just define a large sqlite cache
and a suitable database page size and have the first user /
process digest the initial delay. If the database isn't used for
a while on an otherwise busy system, disk caches would quickly
be reused by something else anyway.
Jay Sprenkle wrote:
On 12/11/06, Dennis Cote <[EMAIL PROTECTED]> wrote:
Simply read the entire file once using fread when you open the database.
That will load the entire file in to the OS cache so that when SQLite
starts reading the file, the required disk blocks are already in memory.
The result is the same fast lookups that you see in SQLite the second
time you run a query. The additional read is also quite fast since it
reads the file sequentially from start to finish with no seeking,
whereas SQLite will seek back and forth as it reads database pages into
memory.
You probably can't fseek() to the end and get the same effect?
You could do this in vb6 by writing a loop and using "input #1"
and discarding what was read. In vb.net it's much simpler using
a file stream reader.
--
The JS Image Collector suite:
http://groups-beta.google.com/group/js-image-collector?hl=en
SqliteImporter and SqliteReplicator: Command line utilities for Sqlite
http://www.reddawn.net/~jsprenkl/Sqlite
Cthulhu Bucks!
http://www.cthulhubucks.com
-----------------------------------------------------------------------------
To unsubscribe, send email to [EMAIL PROTECTED]
-----------------------------------------------------------------------------
