Yes the timestamp method is one we have done from within our own code and it
may very well be the operating system catching up to where it should be and not
the database's fault at all.
It hangs on a read after the inserts--- ourDB.Exec("Select x From Y where
x.HashCode = z LIMIT 1;").
(where x.HashCode is unique.)
Originally we were just storing all hash codes in memory but on our text box
(512MB RAM/4 GIG VRam/40G HD) after about 250,000 hashcodes (32 bit signed
integers) we encountered wicked memory thrashing (physical / virtual) to the
point where the system would (appear to) lock up and had to abandon the idea
for the database.
It eventually does complete, the first time it takes about 30 seconds and does
not generate any errors. After that one time we can hit the database 10-20
times a second with no problems for over 12 hours (length of our test).
Again it may very well be the operating system catching up to where it should
be and not the database's fault at all.
Thanks for all your good advice.
Tom
----------------------------------------
From: Dennis Jenkins <[EMAIL PROTECTED]>
Sent: Monday, December 19, 2005 11:25 AM
To: sqlite-users@sqlite.org
Subject: Re: [sqlite] multiple Db's and journal file time hit?
[EMAIL PROTECTED] wrote:
>The delay seems to coincide with the journal file creation-- it happens after
>our first (committed but not yet written to disk) write attempt to the
>database, the journal file does not at-the-time exist, and there are 500+
>inserts pending with reads rapidly being added after that.
>
>Although I too have seen anti-virus programs hose things up, we don't have any
>running. It was a good thing to check for though.
>
>We can live with this minor start-up penalty because our users will oftentimes
>run this app 12-36 hours straight and could end up hitting the database
>1,000,000 times in that period.
>
>It does not appear to be any kind of error as no exceptions are thrown. We can
>create a minor user-appreciated perception of busyness on-screen. :-)
>
>Tom
>
>----------------------------------------
> From: [EMAIL PROTECTED]
>Sent: Monday, December 19, 2005 8:59 AM
>To: sqlite-users@sqlite.org
>Subject: Re: [sqlite] multiple Db's and journal file time hit?
>
>[EMAIL PROTECTED] wrote:
>
>
>>We just noticed a 30-40 second hit at early on in our program running. >
>>
>>
>
>Others have reported things like this caused by anti-virus
>software running on the same machine and insisting on doing
>some kind of virus scan the first time the journal file is
>created.
>--
>D. Richard Hipp
>
>
A few suggestions:
White box it:
Compile SQLITE yourself with debugging symbols. Run your code in a
profiler. Then run it again in a debugger examine the regions of code
that the profiler flagged as "taking a long time".
Black box it:
Use "filemon" (win32/64, from www.sysinternals.com) or "strace" on Linux
or the equivilent on any other platform. They will timestamp and record
all disk access. Have your own program display the exact system
timestamps before and after the lengthy operation. Examine the logs to
see what is going on.
