Found it. I had opened the databse with SQLITE_OPEN_NOMUTEX.
For some reason it didn't bite me with 3.7.3 but the timing changed just enough
on 3.7.5 to make it break.
I was originally running with SQLITE_THREADSAFE=1 but had experimented with
changing it...and surprise, surprise...it didn't matter
Back to running as usual....so 3.7.5 is in the clear in my app so far.
P.S. If I hadn't started writing another app to test this I don't know when I
would've caught the open flag. I set that up so long ago I completely forgot
about it.
Michael D. Black
Senior Scientist
NG Information Systems
Advanced Analytics Directorate
________________________________________
From: sqlite-users-boun...@sqlite.org [sqlite-users-boun...@sqlite.org] on
behalf of Richard Hipp [d...@sqlite.org]
Sent: Tuesday, January 25, 2011 2:28 PM
To: General Discussion of SQLite Database; Black, Michael A (TS)
Subject: EXTERNAL:Re: [sqlite] :Re: SQLite version 3.7.5 - code freeze
On Tue, Jan 25, 2011 at 3:17 PM, Black, Michael (IS) <michael.bla...@ngc.com
> wrote:
> Ok...good news for 3.7.5 is that once I increased my commit interval I
> started seeing problems with 3.7.3 -- so I think it's me causing the
> problem.
>
> What I'm doing is a begin/commit in a thread while the main thread is doing
> a lot of inserts (almost as fast as possible...doing some other math in
> there too).
>
> threadsafe=3
>
Are you using the same database connection for both threads? (I'm guessing
you are.) SQLite requires SQLITE_THREADSAFE=1 if you want to do that.
Please recompile and rerun with SQLITE_THREADSAFE=1 and let me know if that
clears the problem. Either that, or create a separate database connection
(using independent calls to sqlite3_open()) for each thread.
SQLite requires SQLITE_THREADSAFE to be one of 0, 1, or 2. 3 is not a valid
option. On the other hand, no compile-time warnings are issued if you use
3. I'll look into fixing that.
Also please note that SQLITE_THREADSAFE=1 is the strictest, not 2 as you
would expect. This is in order to maintain backwards compatibility. (There
was originally only THREADSAFE 0 and 1. 2, which is an intermediate state,
was a latter addition.)
Thanks.
>
> I either get an error on insert "data disk image is malformed" from the
> main process, or
> SQlite commit err=5
> SQlite begin err=1
>
> Here' s my thread code...any observations about "bad idea" here that I'm
> doing?
>
> dbCommit() {
> sqlite3_stmt *pStmt;
> const char *sql="COMMIT";
> int err = sqlite3_prepare_v2(pDb,sql,strlen(sql),&pStmt,NULL);
> if (err != SQLITE_OK) {
> cerr << "Error preparing '" << sql << "':" << sqlite3_errmsg(pDb);
> }
> do {
> err = sqlite3_step(pStmt);
> if (err != SQLITE_DONE) {
> cerr << "SQlite commit err=" << sqlite3_reset(pStmt) << endl;
> Sleep(1000);
> }
> } while (err!=SQLITE_DONE);
> sqlite3_finalize(pStmt);
> dbBegin(); // we'll crank up another begin to keep it going
> }
>
> dbBegin() {
> sqlite3_stmt *pStmt;
> const char *sql="BEGIN";
> int err = sqlite3_prepare_v2(pDb,sql,strlen(sql),&pStmt,NULL);
> if (err != SQLITE_OK) {
> cerr << "Error preparing '" << sql << "':" << sqlite3_errmsg(pDb);
> }
> do {
> err = sqlite3_step(pStmt);
> if (err != SQLITE_DONE) {
> cerr << "SQlite begin err=" << sqlite3_reset(pStmt) << endl;
> Sleep(1000);
> }
> } while (err!=SQLITE_DONE);
> sqlite3_finalize(pStmt);
> }
>
> And the insert inside the main proces
> do {
> err = sqlite3_step(pStmt);
> if (err != SQLITE_DONE) {
> cerr << "Error on insert with '" << mysql << "'=" <<
> sqlite3_reset(pStmt) << endl;
> Sleep(1000);
> }
> } while (err != SQLITE_DONE);
> sqlite3_finalize(pStmt);
>
>
> Michael D. Black
> Senior Scientist
> NG Information Systems
> Advanced Analytics Directorate
>
>
>
> ________________________________________
> From: sqlite-users-boun...@sqlite.org [sqlite-users-boun...@sqlite.org] on
> behalf of Richard Hipp [d...@sqlite.org]
> Sent: Monday, January 24, 2011 4:29 PM
> To: General Discussion of SQLite Database
> Subject: EXTERNAL:Re: [sqlite] :Re: SQLite version 3.7.5 - code freeze
>
> On Mon, Jan 24, 2011 at 5:23 PM, Black, Michael (IS) <
> michael.bla...@ngc.com
> > wrote:
>
> > I plugged 3.7.5 code freeze into my app and fairly quickly hit a
> potential
> > bug.
> >
> > I'm using a separate thread to commit my transactions and
> > SQLITE_THREADSAFE=2
> >
> > I never saw a problem in my testing so far with 3.7.3 -- but a few runs
> > after putting 3.7.5 in I started getting an insert error followed by
> > "database disk image is malformed". It's probalistic.
> >
> > I never saw any errors or with 3.7.3.
> >
> > I should be able to write a standalone app that simulates this and
> > increases the likelihood of failure if it'd help (no doubt it would so
> I'll
> > start).
> >
>
> That would be great. Thanks.
>
>
> >
> > Michael D. Black
> > Senior Scientist
> > NG Information Systems
> > Advanced Analytics Directorate
> > _______________________________________________
> > sqlite-users mailing list
> > sqlite-users@sqlite.org
> > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
> >
>
>
>
> --
> D. Richard Hipp
> d...@sqlite.org
> _______________________________________________
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
> _______________________________________________
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
--
D. Richard Hipp
d...@sqlite.org
_______________________________________________
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
_______________________________________________
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
