Re: [sqlite] Slow commits

2012-01-14 Thread Dimiter 'malkia' Stanev
Hi Normand, Just for sanity, check whether Windows is not creating previous versions for you. Right click on the file in Explorer -> Properties -> Previous Versions. Also check whether Caching is enabled on your HDD: Control Panel -> Device Manager -> Disk Drives -> (Click on your HDD) The

Re: [sqlite] Slow commits

2012-01-13 Thread Normand Mongeau
ipp Envoyé : 13 janvier 2012 19:35 À : General Discussion of SQLite Database Objet : Re: [sqlite] Slow commits On Fri, Jan 13, 2012 at 6:49 PM, Normand Mongeau wrote: > > >>> Begin by doing: >> >> PRAGMA synchronous=OFF; >> > > With the above,

Re: [sqlite] Slow commits

2012-01-13 Thread Richard Hipp
On Fri, Jan 13, 2012 at 6:49 PM, Normand Mongeau wrote: > > >>> Begin by doing: >> >> PRAGMA synchronous=OFF; >> > > With the above, the total commitTransaction time goes down to 385 > milliseconds... Impressive. > > The "PRAGMA synchronous=OFF" command turns of syncing of content to the disk

Re: [sqlite] Slow commits

2012-01-13 Thread Normand Mongeau
On 2012-01-13 18:10, Roger Binns wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 13/01/12 14:35, Normand Mongeau wrote: It gets worse. On a clean empty database, the same 534 transactions take 140 seconds. That's a not very impressive rate of 3.8 inserts per second. The FAQ says that

Re: [sqlite] Slow commits

2012-01-13 Thread Normand Mongeau
On 2012-01-13 17:45, Richard Hipp wrote: On Fri, Jan 13, 2012 at 5:35 PM, Normand Mongeauwrote: It gets worse. On a clean empty database, the same 534 transactions take 140 seconds. That's a not very impressive rate of 3.8 inserts per second. The FAQ says that SQLite should be able to do a "

Re: [sqlite] Slow commits

2012-01-13 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 13/01/12 14:35, Normand Mongeau wrote: > > It gets worse. On a clean empty database, the same 534 transactions > take 140 seconds. That's a not very impressive rate of 3.8 inserts > per second. The FAQ says that SQLite should be able to do a "few

Re: [sqlite] Slow commits

2012-01-13 Thread Simon Slavin
On 13 Jan 2012, at 10:35pm, Normand Mongeau wrote: > It gets worse. On a clean empty database, the same 534 transactions take 140 > seconds. That's a not very impressive rate of 3.8 inserts per second. The > FAQ says that SQLite should be able to do a "few dozen transactions per > second". I'

Re: [sqlite] Slow commits

2012-01-13 Thread Richard Hipp
On Fri, Jan 13, 2012 at 5:35 PM, Normand Mongeau wrote: > > It gets worse. On a clean empty database, the same 534 transactions take > 140 seconds. That's a not very impressive rate of 3.8 inserts per second. > The FAQ says that SQLite should be able to do a "few dozen transactions per > second".

Re: [sqlite] Slow commits

2012-01-13 Thread Normand Mongeau
It gets worse. On a clean empty database, the same 534 transactions take 140 seconds. That's a not very impressive rate of 3.8 inserts per second. The FAQ says that SQLite should be able to do a "few dozen transactions per second". I'd be happy to see that. I don't have much experience with

Re: [sqlite] Slow commits

2012-01-13 Thread Normand Mongeau
On 2012-01-13 15:35, Richard Hipp wrote: On Fri, Jan 13, 2012 at 3:34 PM, Normand Mongeauwrote: On 2012-01-13 15:23, Richard Hipp wrote: On Fri, Jan 13, 2012 at 3:19 PM, Normand Mongeau>wrote: not really, no. This is a server that receives files, and the transaction below means a file h

Re: [sqlite] Slow commits

2012-01-13 Thread Richard Hipp
On Fri, Jan 13, 2012 at 3:34 PM, Normand Mongeau wrote: > > > On 2012-01-13 15:23, Richard Hipp wrote: > >> On Fri, Jan 13, 2012 at 3:19 PM, Normand Mongeau> com >wrote: >> >> not really, no. This is a server that receives files, and the transaction >>> below means a file has arrived. >>> >>> Do

Re: [sqlite] Slow commits

2012-01-13 Thread Normand Mongeau
On 2012-01-13 15:23, Richard Hipp wrote: On Fri, Jan 13, 2012 at 3:19 PM, Normand Mongeauwrote: not really, no. This is a server that receives files, and the transaction below means a file has arrived. Does your server have a really, really slow disk drive? Transaction commit normally take

Re: [sqlite] Slow commits

2012-01-13 Thread Normand Mongeau
Record sizes are approx 480 bytes for tableA, 380 bytes for tableB and 800 bytes for tableC. Storage is my hard drive, which is a normal SATA disk. On 2012-01-13 15:23, Stephan Beal wrote: On Fri, Jan 13, 2012 at 9:19 PM, Normand Mongeauwrote: not really, no. This is a server that receives

Re: [sqlite] Slow commits

2012-01-13 Thread Richard Hipp
On Fri, Jan 13, 2012 at 3:19 PM, Normand Mongeau wrote: > not really, no. This is a server that receives files, and the transaction > below means a file has arrived. > Does your server have a really, really slow disk drive? Transaction commit normally takes milliseconds. I'm not sure why you ar

Re: [sqlite] Slow commits

2012-01-13 Thread Stephan Beal
On Fri, Jan 13, 2012 at 9:19 PM, Normand Mongeau wrote: > not really, no. This is a server that receives files, and the transaction > below means a file has arrived. > > You haven't told us how big the records are. If you are storing, e.g., 2GB file uploads in each transaction then of course it wi

Re: [sqlite] Slow commits

2012-01-13 Thread Normand Mongeau
not really, no. This is a server that receives files, and the transaction below means a file has arrived. Normand On 2012-01-13 15:16, Simon Slavin wrote: On 13 Jan 2012, at 7:57pm, Normand Mongeau wrote: begin immediate transaction insert 1 record in tableA insert 1 record in tableB insert

Re: [sqlite] Slow commits

2012-01-13 Thread Simon Slavin
On 13 Jan 2012, at 7:57pm, Normand Mongeau wrote: > begin immediate transaction > insert 1 record in tableA > insert 1 record in tableB > insert 1 record in tableC > commit transaction > > Inserting 534 records takes about 75 seconds. Most of the time (about 71 > seconds) is spent on the commit

[sqlite] Slow commits

2012-01-13 Thread Normand Mongeau
Hi, I have an app using sqlite, and the main insertion point is very slow on commitTransaction. This is what I do: begin immediate transaction insert 1 record in tableA insert 1 record in tableB insert 1 record in tableC commit transaction Inserting 534 records takes about 75 seconds. Most o