Hi,
For the last month I've been troubled by an apparent bug in SQLite 3.3.5
which has been very hard to track down. After extensive debugging and
bug-tracking it seems there must be a bug/feature somewhere in SQLite -
propably in sqlite3_prepare because this is where the symptom of the bug
appears every time. It escalated when I updated the SQLite from 3.3.4 to
3.3.5 - although I'm not sure how often the problem occured in 3.3.4.
I'm using SQLite 3.3.5 in a threaded .NET environment - although all
communication with SQLite happens synchronized manner (eg. only one
thread at a time).
In one particular function I do the following:
1) Set binary semaphore to lock access to database
2) Open connection
3) Begin transaction
4) UPDATE table
5) - if 0 rows affected in (4) - do INSERT
6) If more data - return to (4)
7) UPDATE table #2
8) - if 0 rows affected in (7) - do INSERT
9) Commit transaction (or rollback if error has occured)
10) Close connection
All calls to database is done by calling sqlite3_prepare() and step() -
through the use of the Finisar.SQLite wrapper.
This all works very well for some time - then suddenly after 250 - 10000
calls to my function sqlite3_prepare returns "Object reference not set
to an instance of an object" - which is a .NET exception I interpretate
as a Null-pointer-exception / malloc failure. This in turn results in my
rollback returning "library routine called out of sequence" - which
means the rollback hasn't been exectuted which in turn locks the entire
database, and the process needs to be restarted to unlock the database.
>From previous communication on this list I know "library routine called
out of sequence" means that sqlite3_prepare() is called on a closed
connection - but this is _not_ the case (although because of the
null-pointer exception / malloc error the connection might indeed have
been closed at the time I try to run the "rollback"). Of course I
suspected some error in my program - but there aren't any. From the
error I suspected a corruct heap from a memory leak or something - but
there aren't any, which I have concluded by going over my code by hand -
and testing with both BoundsChecker and SciTech Memory Profiler.
I then found out, that the problem primarily occurs when the #2 table is
updated (7+8 in the above). If I only update 1 table in the transaction
the error doesn't occur.
The two tables are very simple - but with indexes on them. Maybe this is
the cause?
I'm sorry I cannot be more specific - but this is because of the weird
nature of the problem. I'm not able to debug the actual source code of
SQLite, because the problem only occurs at random interval and stepping
through thousands of calls to SQLite is just to big of a job.
I have run SQLite in both runtime and debug - and know that the
sqlite3_prepare has an assert that checks for "mallocHasFailed", but I
haven't been able to verify the value of this because of the nature of
the bug. The assert isn't fired - although this might be because it
resides in a separate DLL and VS.NET doesn't catch it, although I debug
in both native and managed code and is able to step through the
sqlite3_prepare(). The machine has plenty of RAM and CPU and no other
processes that consume noticable resources.
The only way I can reproduce it in my test environment is by calling the
SQLite between 250 and 10000 times - and doing a heap collection in
SciTech Memory Profiler at various intervals. In my live environment the
error occurs every 1 - 5 days.
My main problem is that the process needs to be restarted to unlock the
tables.
Does anybody know of this problem?
Best regards,
Michael Hansen
