Re: [sqlite] Database locking Error
On 30 Aug 2013, at 5:36am, techi eth techi...@gmail.com wrote: Thanks for suggestion. I will take it forward. locking will happen with two simultaneous processes accessing for read also? I don't think it can happen if they're both reading. But a process which is reading could block a process which is writing, and a process which is writing could block a process which is reading. The easiest way to cope with it is to always set a timeout and handle the error correctly. Simon. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] Database locking Error
On 29 Aug 2013, at 6:48am, techi eth techi...@gmail.com wrote: I am checking for all the function.As of now i am not using sqlite3 time out but testing application will take decision accordingly to recall the operation based on type of error. If you have two simultaneous processes accessing the same database and have not set a timeout, that explains your locking errors. You will get constant collisions as both processes try to access the database at the same time and if you haven't set a timeout every one of these will instantly be reported as a locking error. I suggest you set a timeout in both processes to, perhaps, 5 seconds. This means that the processes will each retry for anything up to 5 seconds before reporting locking errors. Simon. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] Database locking Error
Thanks for suggestion. I will take it forward. locking will happen with two simultaneous processes accessing for read also? Thanks On 8/29/13, Simon Slavin slav...@bigfraud.org wrote: On 29 Aug 2013, at 6:48am, techi eth techi...@gmail.com wrote: I am checking for all the function.As of now i am not using sqlite3 time out but testing application will take decision accordingly to recall the operation based on type of error. If you have two simultaneous processes accessing the same database and have not set a timeout, that explains your locking errors. You will get constant collisions as both processes try to access the database at the same time and if you haven't set a timeout every one of these will instantly be reported as a locking error. I suggest you set a timeout in both processes to, perhaps, 5 seconds. This means that the processes will each retry for anything up to 5 seconds before reporting locking errors. Simon. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] Database locking Error
Yes, i am checking the return code. On Tue, Aug 27, 2013 at 5:09 PM, Simon Slavin slav...@bigfraud.org wrote: On 27 Aug 2013, at 5:15am, techi eth techi...@gmail.com wrote: For read operation i am doing _prepare(), _step(), _finalize(). For all other operation i am doing _exec(). Do you see any issue ? Nothing obvious from what I already know apart from the fact that you don't mention setting a timeout: http://www.sqlite.org/c3ref/busy_timeout.html http://www.sqlite.org/pragma.html#pragma_busy_timeout Without that, a process getting a lock won't back off and retry, it'll just immediately return an error. Which is almost never what people want. You would probably be better off setting this to 5 seconds or something. I suspect that the process which is getting the lock is fine and that it's the other process which is keeping the file locked. Are you checking the return codes from /all/ your SQLite calls in other processes ? The code you included doesn't seem to do that but I thought that might be pseudocode. Simon. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] Database locking Error
On 28 Aug 2013, at 9:24am, techi eth techi...@gmail.com wrote: Yes, i am checking the return code. Just for the function that gives the error, or for the calls before that too ? And are you setting a timeout ? If so, for how long ? Simon. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] Database locking Error
I am checking for all the function.As of now i am not using sqlite3 time out but testing application will take decision accordingly to recall the operation based on type of error. On Wed, Aug 28, 2013 at 3:03 PM, Simon Slavin slav...@bigfraud.org wrote: On 28 Aug 2013, at 9:24am, techi eth techi...@gmail.com wrote: Yes, i am checking the return code. Just for the function that gives the error, or for the calls before that too ? And are you setting a timeout ? If so, for how long ? Simon. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] Database locking Error
On 27 Aug 2013, at 5:15am, techi eth techi...@gmail.com wrote: For read operation i am doing _prepare(), _step(), _finalize(). For all other operation i am doing _exec(). Do you see any issue ? Nothing obvious from what I already know apart from the fact that you don't mention setting a timeout: http://www.sqlite.org/c3ref/busy_timeout.html http://www.sqlite.org/pragma.html#pragma_busy_timeout Without that, a process getting a lock won't back off and retry, it'll just immediately return an error. Which is almost never what people want. You would probably be better off setting this to 5 seconds or something. I suspect that the process which is getting the lock is fine and that it's the other process which is keeping the file locked. Are you checking the return codes from /all/ your SQLite calls in other processes ? The code you included doesn't seem to do that but I thought that might be pseudocode. Simon. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
[sqlite] Database locking Error
Hi, I come across database file locking error while trying below case with Update. Case: One process is updating a Colum periodically other process is selecting same Colum or other colum in table periodically for read. What is the best way to handle so that Update will not get locking error? Cheers - Techi ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] Database locking Error
On 08/26/2013 01:37 PM, techi eth wrote: Hi, I come across database file locking error while trying below case with Update. Case: One process is updating a Colum periodically other process is selecting same Colum or other colum in table periodically for read. What is the best way to handle so that Update will not get locking error? Using WAL mode. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] Database locking Error
Thanks. I tried using PRAGMA jouranl_mode = WAL but still it is getting failed for some time. Find below more details. Process 1: Fun_Open() sqlite3_open(DATABASE, dbUpdate); sqlite3_exec(dbUpdate, PRAGMA jouranl_mode = WAL, NULL, NULL, sErrMsg); Fun_Update() on TblTest /* Do the periodic update operation by using db handler return from above*/ Process 2: Fun_Open() sqlite3_open(DATABASE, dbRead); sqlite3_exec(dbRead, PRAGMA jouranl_mode = WAL, NULL, NULL, sErrMsg); Fun_Read() on TblTest** /* Do the periodic read operation by using db handler return from above*/ Cheers - Techi On Mon, Aug 26, 2013 at 12:14 PM, Dan Kennedy danielk1...@gmail.com wrote: On 08/26/2013 01:37 PM, techi eth wrote: Hi, I come across database file locking error while trying below case with Update. Case: One process is updating a Colum periodically other process is selecting same Colum or other colum in table periodically for read. What is the best way to handle so that Update will not get locking error? Using WAL mode. __**_ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-**bin/mailman/listinfo/sqlite-**usershttp://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] Database locking Error
On Mon, 26 Aug 2013 13:32:42 +0530, techi eth techi...@gmail.com wrote: PRAGMA jouranl_mode = WAL That's misspelled, both in your text and in your code. Try: PRAGMA journal_mode=WAL; You only have to do this once, e.g. at database creation. Journal mode WAL is a persistent property of the database file. Every connection will respect it. -- Groet, Cordialement, Pozdrawiam, Regards, Kees Nuyt ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] Database locking Error
Oppps. Apologies for my mistake. My test is working correctly now. Thanks a lot. You only have to do this once, e.g. at database creation. Journal mode WAL is a persistent property of the database file. Every connection will respect it. All PRAGMA Option is attached to database as a persistent property? Is their any issue if we set all option again with every new connection to same file? On Mon, Aug 26, 2013 at 2:40 PM, Kees Nuyt k.n...@zonnet.nl wrote: On Mon, 26 Aug 2013 13:32:42 +0530, techi eth techi...@gmail.com wrote: PRAGMA jouranl_mode = WAL That's misspelled, both in your text and in your code. Try: PRAGMA journal_mode=WAL; You only have to do this once, e.g. at database creation. Journal mode WAL is a persistent property of the database file. Every connection will respect it. -- Groet, Cordialement, Pozdrawiam, Regards, Kees Nuyt ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] Database locking Error
You only have to do this once, e.g. at database creation. Journal mode WAL is a persistent property of the database file. Every connection will respect it. All PRAGMA Option is attached to database as a persistent property? No, only the ones which are persistent. Journal_mode is persistent. Is their any issue if we set all option again with every new connection to same file? No. You can set them every time if you wish without harm. In fact if you are using a shared sqlite3 engine that you did not compile this may be a good idea as otherwise you have no way to know what defaults/settings are in place (other than prayer). ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] Database locking Error
On 26 Aug 2013, at 9:02am, techi eth techi...@gmail.com wrote: Fun_Read() on TblTest** /* Do the periodic read operation by using db handler return from above*/ How is your read done ? Do you use _exec() like you do for the PRAGMA, or _prepare(), _step(), _finalize(), or _query() or something else ? Simon. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] Database locking Error
For read operation i am doing _prepare(), _step(), _finalize(). For all other operation i am doing _exec(). Do you see any issue ? Cheers - Techi On Mon, Aug 26, 2013 at 9:22 PM, Simon Slavin slav...@bigfraud.org wrote: On 26 Aug 2013, at 9:02am, techi eth techi...@gmail.com wrote: Fun_Read() on TblTest** /* Do the periodic read operation by using db handler return from above*/ How is your read done ? Do you use _exec() like you do for the PRAGMA, or _prepare(), _step(), _finalize(), or _query() or something else ? Simon. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
[sqlite] Database locking without locking database file
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, the FAQ of qemu describes Sqlite to be threadsafe [1]. The threadsafeness can e.g. be chosen at compiletime [2]. If several Sqlite instances try to access the same database file on a system, the database file gets locked via simple file locks [3]. Those files can also be located on network filesystem like NFS or CIFS. [4] describes, that there might be some broken NFS implementations, which have apparently issues with file locking I'm working on a project where it is necessary that the Sqlite database is stored on a networking filesystem and several other clients have access to that file. I first use sshfs-fuse when i realized, that sshfs-fuse is not able to forward file locks. So I got (of course) malformed databases. Now my Question: According to [4], it seems that NFS also has problems with sharing locks. So why does Sqlite make use of file locks instead of writing those locks (including a timestamp for expiration) inside the database file or by touching dotfiles for locking? [1] : http://www.sqlite.org/faq.html#q6 [2] : http://www.sqlite.org/compile.html#threadsafe [3] : http://www.sqlite.org/howtocorrupt.html Chapter 2.1 [4] : http://www.sqlite.org/faq.html#q5 Regards, - -- Ralf Ramsauer PGP: 0x8F10049B -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.20 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJSClJeAAoJEPovtFKPEASbWbMQAJPhmoCxoWWbDDFxAtNVSVAo Jd9HmuUsPxGtKUVW+5yUvyHbtUcKaL9dRLq5uzswN3PiWfa0ro0h6FcWXGpwZYwG +wVopcPHbCNxnbLwCQ05xdgtrJ7eLgvD5zj6/Y4ud+9OTHwLm27hjXorZX9Lt9qG aq6sEMynuVGv136xWuJTDxPjJdgCtsQSaT2VMeArIA+Mn92CkqhK5YHYZ37RVMRh l6wmX7U30d6cr7WqkV/wfVR7xjvQruum+4kMCAovpnNQJOTMa7m4Ky6SiclVN4bu 6AxEjhX7vk565PpWLm4UoMUma7TXHvd7FFCCKdagEe28ZF8Q+Bl2I1Lyn0+AD715 THCVgbpXms6IYptBHRQxuCC4/saGOMICdyn2ZBHDj6XmTYhR2d+3Reu5Men42QWI QHY2AEcmQANN+dRWvUtwfKW738nGGqgDsB97cz8iUI3eWfY/VlKe5ZbPPB2RM/mc w+yb1YxKFpAWXyVBPUC8HrT+AB2fWzqG+fijsaAeWfNxGt8cQivnBThmyf+JrJOC A+lQAwHiADY7lyTG3q8VgwmznRCPGLUd2M246h2hvSBfyG+/OLd+iH6Kk5E9Grnl jK6gRotY7A7T6of+sOwwcFbAtAZ0r8DGflGw0BoIglJjecstp3XdkBgxByXPABQs ohEq/wiS/PDKaMp2rP95 =MNHI -END PGP SIGNATURE- ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] Database locking without locking database file
On Tue, Aug 13, 2013 at 11:35 AM, Ralf Ramsauer r...@ramses-pyramidenbau.de wrote: Now my Question: According to [4], it seems that NFS also has problems with sharing locks. So why does Sqlite make use of file locks instead of writing those locks (including a timestamp for expiration) inside the database file or by touching dotfiles for locking? You cannot write into the database file to create a lock because that presents a race condition if two processes are trying to lock at the same time. If you open using the unix-dotfile VFS, it will use dotfile locking. However, you give up a lot of concurrency with this (since dot-files are always exclusive locks and never reader/writer locks) and it is slower and it does not work with WAL mode (which requires multiple reader/writer locks per file) and if your process dies unexpectedly it will leave stale locks that have to be removed by hand. -- D. Richard Hipp d...@sqlite.org ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] Database locking without locking database file
Hi, thanks a lot for your answer. On 14.08.2013 13:52, Richard Hipp wrote Now my Question: According to [4], it seems that NFS also has problems with sharing locks. So why does Sqlite make use of file locks instead of writing those locks (including a timestamp for expiration) inside the database file or by touching dotfiles for locking? You cannot write into the database file to create a lock because that presents a race condition if two processes are trying to lock at the same time. Ah ok, point taken. I thought as much. If you open using the unix-dotfile VFS, it will use dotfile locking. Great! As I see, Sqlite supports unix-dotfile locking [1], I haven't seen this article before. Perhaps this solves my problem However, you give up a lot of concurrency with this (since dot-files are always exclusive locks and never reader/writer locks) and it is slower and it does not work with WAL mode (which requires multiple reader/writer locks per file) and if your process dies unexpectedly it will leave stale locks that have to be removed by hand. I can accept that :-) One further question: Why is it proposed not to use NFS? Why is it so risky? I can hardly believe that NFS locking is that broken... Neither cifs. [1] : http://www.sqlite.org/vfs.html Thanks in advance -- Ralf Ramsauer PGP: 0x8F10049B ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] Database locking without locking database file
On Wed, Aug 14, 2013 at 3:06 PM, Ralf Ramsauer ralf+sql...@ramses-pyramidenbau.de wrote: Why is it proposed not to use NFS? Why is it so risky? I can hardly believe that NFS locking is that broken... Few of us can believe it, but many of us have had horrible experiences (regardless of sqlite) with locking on network drives, either corruption or 100x slower performance when file locking is used. Try it if you must, but don't say you weren't warned. Neither cifs. ha. Ha. HAHAHAHA!!! That made my day :). -- - stephan beal http://wanderinghorse.net/home/stephan/ http://gplus.to/sgbeal ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] Database locking without locking database file
On 14 Aug 2013, at 5:36pm, Stephan Beal sgb...@googlemail.com wrote: On Wed, Aug 14, 2013 at 3:06 PM, Ralf Ramsauer ralf+sql...@ramses-pyramidenbau.de wrote: Why is it proposed not to use NFS? Why is it so risky? I can hardly believe that NFS locking is that broken... Few of us can believe it, but many of us have had horrible experiences (regardless of sqlite) with locking on network drives, either corruption or 100x slower performance when file locking is used. Try it if you must, but don't say you weren't warned. I can add from personal experience that various implementations of NFS over the years have been terrible. From sync() and fsync() calls which did nothing at all, to unlock calls which unlock all files when you're asking only for one file, there has been some truly terrible source code. Section 2.1 in 'howtocorrupt' is dead right. Simon. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] Database locking without locking database file
On 14.08.2013 18:36, Stephan Beal wrote: On Wed, Aug 14, 2013 at 3:06 PM, Ralf Ramsauer ralf+sql...@ramses-pyramidenbau.de wrote: Why is it proposed not to use NFS? Why is it so risky? I can hardly believe that NFS locking is that broken... Few of us can believe it, but many of us have had horrible experiences (regardless of sqlite) with locking on network drives, either corruption or 100x slower performance when file locking is used. Try it if you must, but don't say you weren't warned. Okay, I'll try it out and stress test it. I never had such problems with NFS before. I'll want to be able to reconstruct it... Neither cifs. ha. Ha. HAHAHAHA!!! That made my day :). Cheers ;-) Of course i prefer NFS but in some cases one doesn't get out of Cifs. Regards, -- Ralf Ramsauer PGP: 0x8F10049B ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] Database locking without locking database file
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 14/08/13 06:06, Ralf Ramsauer wrote: Neither cifs. I worked on a CIFS server (visionfs)[1]. They are a convoluted complicated mess. During the OLE2 era, Microsoft's apps abused locking as a means of inter-process communication. It got very complicated, with Excel especially bad/adventurous in this respect. CIFS locks are also completely different in virtually every way from Unix ones. You cannot map between them sensibly. Both VisionFS and Samba ended up having a database of the lock information and essentially ignored what Unix said. What this means is you cannot use CIFS from a Unix box and get correct semantics because the Unix API does not expose anything close to cifs reality. Windows machines can in theory do the right thing. It has been a while, but back in those days there would be calls over the wire that I couldn't find any API to request. For well over a decade the protocol has roughly consisted of NT kernel data structures being sent over the wire. The server side gets adjusted to detect and work around bugs in the various client versions. Because of the asynchronous nature of networked file systems, race conditions, and the possibilities of running out of disk space, most apps have done updates by writing to a new file and only when everything is successful overwriting the original. Updating a (possibly shared) file in-place which is what SQLite does is fraught with peril, which is why it turns out to be the least exercised part of protocols and where bugs/quirks lie. [1] http://www.rogerbinns.com/visionfs.html Roger -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) iEYEARECAAYFAlIMB3wACgkQmOOfHg372QTyGgCgxB/7XxUibw+p0T/aI0cNecE4 ZegAoMDoh7RKb6hZmXsWTbreWhXpSYWP =0UbL -END PGP SIGNATURE- ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
[sqlite] database locking
Hi List
I have a python script that runs 3 instances simultaneously on a
quadcore x86 system. I'm not using the multiprocessing libraries. The
programs are just run in parallel. They all access (for reading only)
a single sqlite3 database. I'm setting the following pragma commands:
offset_db.execute('pragma synchronous = 0')
offset_db.execute('pragma temp_store = 2')
offset_db.execute('pragma read_uncommitted = true')
offset_db.execute('pragma cache_size = 2000')
offset_db.execute('pragma default_cache_size = 2000')
offset_db.execute('pragma journal_mode = off')
Nine times out of ten I get the following error when I run my code:
offset_db.execute('pragma synchronous = 0')
sqlite3.OperationalError: database is locked
If I simply run the script again, it will always work the second time.
The script only make select statements. There is no writing
whatsoever. Any ideas what's going on here? I should that there is
another program that runs before these that inserts data into the
database. Could it be that something is not closing correctly?
Thanks in advance for any guidance.
-Sean
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] Database locking issue
Hi, now I have tried the actual version 3.6.3 as of sqlite- amalgamation-3.6.3.tgz, but still get database locks I cannot explain. Brief information: I use two database files. One (lbDMF.db3) for system informations such as what my application should show in forms of a database (Template Designer.db3). The system database therefore is required to store sql queries of the application database (Template Designer.db3). Looking at the files after I have the lock situation (after a successfull close I also get a lock when reopened) I have determined content in Template Designer.db3-journal that shows words (of columns or tables from lbDMF.db3. How could it be, that there is content in that file that fas nothing to do with that database ? Here is the content: Ÿ’˘ °c◊ öà ts/CPP/Test/GUI/wxWrapper/Template Designer .db3 ö Ö V 8 ¸ › æ ü Å c E . ‡ ¿ †åxdJ0 Ú Œ ™ ã l M 3 IdAnwendungsparameter ! 3 idAnwendungsparameter 3 IDAnwendungsparameter = IdFormularaktionenzuordnen = idFormularaktionenzuordnen = IDFormularaktionenzuordnen ) IdUebersetzungen ) idUebersetzungen ) IDUebersetzungen IdAktionen idAktionen IDAktionen 5 IdAnwendungenFormulare ú Ïÿƒ∞ú LiliLili Lili LuluLulu Lulu LoloLolo Lolo LalaLala Lala TestTest Test öà The words Anwendungsparameter, Formularaktionenzuordnen, Uebersetzungen, Aktionen and AnwendungenFormulare are columns or tables from lbDMF. I am confused !! Also I could rename these files while the application claims they are locked. (This happens while an update attempt I'll give a retry with a shortly closed database to get rid of the lock, but the lock remains) Any hints are welcome. Thanks, Lothar Am 19.09.2008 um 17:37 schrieb Lothar Behrens: Am 19.09.2008 um 17:03 schrieb Ken: Try it with the latest full build say version 3.6.2 and see what happens instead of a patched I'll give that a try. Could the current code be compiled with Open Watcom as A DLL ? (I haven't seen these __declspec(dllexport) and the opposite stuff in the actual code) Lothar -- | Rapid Prototyping | XSLT Codegeneration | http://www.lollisoft.de Lothar Behrens Heinrich-Scheufelen-Platz 2 73252 Lenningen ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users -- | Rapid Prototyping | XSLT Codegeneration | http://www.lollisoft.de Lothar Behrens Heinrich-Scheufelen-Platz 2 73252 Lenningen ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
[sqlite] Database locking issue
Hi,
I am using Version 3.5.2 of Sqlite with the changes of the following
CVS checkin numbers: 4543 and 5243 to get the sqlite3_sql and
sqlite3_next_stmt functions
into my API.
I have patched my files due to the need to figuring out what
statements were unfinalized. But now I get 'database is locked' errors
right after reopening the
database (the closing of the database works as I try to keep my
statements open as short as possible thus closing now works).
Due to this new error ('database is locked') I added reporting
statemens in use. The result of this is a report of the insert
statement itself that causes the error for
this report.
So I am a little confused.
If a database is locked, is there always a statement that causes this
lock, or is the lock keeping even the statement gets finalized ?
Or did I have to apply other patches related to them above ?
Thanks
Lothar
-- | Rapid Prototyping | XSLT Codegeneration | http://www.lollisoft.de
Lothar Behrens
Heinrich-Scheufelen-Platz 2
73252 Lenningen
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] Database locking issue
Try it with the latest full build say version 3.6.2 and see what happens
instead of a patched
--- On Fri, 9/19/08, Lothar Behrens [EMAIL PROTECTED] wrote:
From: Lothar Behrens [EMAIL PROTECTED]
Subject: [sqlite] Database locking issue
To: General Discussion of SQLite Database sqlite-users@sqlite.org
Date: Friday, September 19, 2008, 9:02 AM
Hi,
I am using Version 3.5.2 of Sqlite with the changes of the following
CVS checkin numbers: 4543 and 5243 to get the sqlite3_sql and
sqlite3_next_stmt functions
into my API.
I have patched my files due to the need to figuring out what
statements were unfinalized. But now I get 'database is locked' errors
right after reopening the
database (the closing of the database works as I try to keep my
statements open as short as possible thus closing now works).
Due to this new error ('database is locked') I added reporting
statemens in use. The result of this is a report of the insert
statement itself that causes the error for
this report.
So I am a little confused.
If a database is locked, is there always a statement that causes this
lock, or is the lock keeping even the statement gets finalized ?
Or did I have to apply other patches related to them above ?
Thanks
Lothar
-- | Rapid Prototyping | XSLT Codegeneration | http://www.lollisoft.de
Lothar Behrens
Heinrich-Scheufelen-Platz 2
73252 Lenningen
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] Database locking issue
Am 19.09.2008 um 17:03 schrieb Ken: Try it with the latest full build say version 3.6.2 and see what happens instead of a patched I'll give that a try. Could the current code be compiled with Open Watcom as A DLL ? (I haven't seen these __declspec(dllexport) and the opposite stuff in the actual code) Lothar -- | Rapid Prototyping | XSLT Codegeneration | http://www.lollisoft.de Lothar Behrens Heinrich-Scheufelen-Platz 2 73252 Lenningen ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
[sqlite] Database locking and blocking
When a process gets SQLITE_BUSY, it needs to wait for the blocking operation to finish before it can successfully retry (whether it's just retrying a COMMIT or redoing a whole transaction). How do people typically handle this? If a different process is the one holding the lock, then you need to sleep for a while, or sleep(0) to yield the scheduler. (This seems to be the case sqlite3_busy_* has in mind.) However, in a state-engine application where multiple independent tasks in one thread might have interleaved, simultaneous access to the database, then you need to stash away what you're doing and let the other tasks run, so the one blocking you can finish. If you're in a state engine (or Lua coroutines, etc.), it's hard to tell which of these cases you're in. If you guess incorrectly, you'll either chew CPU by not yielding the scheduler, or you'll yield the scheduler when the blocking task is in the same process. Both of these can be pretty bad failure modes. I may end up just never leaving a transaction unfinished between state engine updates, so there are never half-finished transactions sitting around holding locks (guaranteeing that the correct thing to do is sleep). That's not good for large, expensive transactions, though, where other tasks should be given a chance to run. -- Glenn Maynard ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
[sqlite] Database Locking
I'm using SQLite on a Windows XP system with 1GB of RAM. I'm writing a server which spawns multiple threads for it's connections. It is querying and writing to an SQLite database within each thread. Whenever a query or insert/update statement is issued, the SQLite database is opened, queried/written to, and closed again. When a bunch of queries come in (about 10 in a second, for instance), SQLite reports that the database is locked despite my setting the sqlite_busy_timeout to 150 or so. Certain database writes thus do not take place, which is a major problem. If I detect when my call returns SQLITE_BUSY and sleep for a while (about 100 ms) and re-try the call, it continuously says that the database is locked. Any help would be appreciated. Thank you. Richard Boehme - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [sqlite] Database Locking
In a message dated 6/4/2004 11:53:05 AM Eastern Daylight Time, [EMAIL PROTECTED] writes: I'm using SQLite on a Windows XP system with 1GB of RAM. I'm writing a server which spawns multiple threads for it's connections. It is querying and writing to an SQLite database within each thread. Whenever a query or insert/update statement is issued, the SQLite database is opened, queried/written to, and closed again. When a bunch of queries come in (about 10 in a second, for instance), SQLite reports that the database is locked despite my setting the sqlite_busy_timeout to 150 or so. Certain database writes thus do not take place, which is a major problem. If I detect when my call returns SQLITE_BUSY and sleep for a while (about 100 ms) and re-try the call, it continuously says that the database is locked. I have an application basically does the same thing and retry after SQLITE_BUSY works like a champ. So I would say the problem is not sqlite but your implementation. Maybe you can post your busy-retry code so I can take a look and make suggestion? Wei
Re: [sqlite] Database Locking
I'm using SQLite on a Windows XP system with 1GB of RAM. I'm writing a server which spawns multiple threads for it's connections. It is querying and writing to an SQLite database within each thread. Whenever a query or insert/update statement is issued, the SQLite database is opened, queried/written to, and closed again. When a bunch of queries come in (about 10 in a second, for instance), SQLite reports that the database is locked despite my setting the sqlite_busy_timeout to 150 or so. Certain database writes thus do not take place, which is a major problem. If I detect when my call returns SQLITE_BUSY and sleep for a while (about 100 ms) and re-try the call, it continuously says that the database is locked. Try setting a bigger busy_timeout... I think that 150 is too small... maybe about 1000 it is better to set a bigger timeout than set a small timeout and call many Sleep(). Are you sure you finalize your queries? I'm using the CppSQLite Wrapper in an application where I've many threads accessing the DB but I never had your problem... I think that the problem is in your implementation... Paolo - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
