Re: [sqlite] occasional SQLITE_PROTOCOL on synchronous=normal with WAL
On Tue, Dec 7, 2010 at 6:51 PM, Richard Hipp d...@sqlite.org wrote: You have to have multiple processes really hammering away at the database in order to get this to happen, which is apparently what you are doing, huh. And we've only seen this on variations of unix, not on windows. Does this jive with what you are seeing? Sounds like it - I am running on Linux and have a bunch of worker processes contending to open and update databases. Adding the sched_yield may have reduced the number of errors. It's sort of difficult to gauge the frequency, but there's not an order of magnitude difference or anything. -- Mike Barton ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
[sqlite] occasional SQLITE_PROTOCOL on synchronous=normal with WAL
We've been testing switching our app to use WAL journaling mode. We're using a snapshot of trunk from last week sometime. We have a sort of weird occasional problem where we get SQLITE_PROTOCOL when setting pragma synchronous=normal. Our app has a lot of concurrent connections to the database (within the same process and by other processes), but I can't figure out anything else that would be touching, much less locking, the db file. Does anyone have suggestions for digging a little deeper on this problem? -- Michael Barton ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] occasional SQLITE_PROTOCOL on synchronous=normal with WAL
On Tue, Dec 7, 2010 at 7:30 PM, Michael Barton m...@weirdlooking.com wrote: We've been testing switching our app to use WAL journaling mode. We're using a snapshot of trunk from last week sometime. We have a sort of weird occasional problem where we get SQLITE_PROTOCOL when setting pragma synchronous=normal. Our app has a lot of concurrent connections to the database (within the same process and by other processes), but I can't figure out anything else that would be touching, much less locking, the db file. Does anyone have suggestions for digging a little deeper on this problem? -- Michael Barton ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users -- D. Richard Hipp d...@sqlite.org ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] occasional SQLITE_PROTOCOL on synchronous=normal with WAL
On Tue, Dec 7, 2010 at 7:30 PM, Michael Barton m...@weirdlooking.com wrote: We've been testing switching our app to use WAL journaling mode. We're using a snapshot of trunk from last week sometime. We have a sort of weird occasional problem where we get SQLITE_PROTOCOL when setting pragma synchronous=normal. Our app has a lot of concurrent connections to the database (within the same process and by other processes), but I can't figure out anything else that would be touching, much less locking, the db file. Does anyone have suggestions for digging a little deeper on this problem? This is due to a race condition that doesn't go away like we thought it would. SQLite tries and tries to get a lock it needs, and which should only be transiently held, but gives up after 100 attempts. There is a 1 microsecond delay between each attempt following the 5th. This was first seen on Android a couple of weeks ago, and then late last week on a Mac. Dan was experimenting with some fixes just this morning, and he was able to make the situation much harder to reach, but was not able to eliminate it. If you are willing to hack the code, I believe Dan saw that the incidences of SQLITE_PROTOCOL were reduced but not completely eliminated if you add a call to sched_yield() right before or right after the call to usleep() in the unixSleep() function. You have to have multiple processes really hammering away at the database in order to get this to happen, which is apparently what you are doing, huh. And we've only seen this on variations of unix, not on windows. Does this jive with what you are seeing? -- Michael Barton ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users -- D. Richard Hipp d...@sqlite.org ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
