Scott, The whole job is wrapped in an explicit transaction. Variables are bound and statements prepared only once, using reset. This is a write only app. 100% insert. Ken
Scott Hess <[EMAIL PROTECTED]> wrote: Are you using explicit transactions at all? If not, as a quick test, put the _entire_ job in a transaction and see what happens. -scott On 3/15/07, Ken wrote: > To answer your question: > Yes I can use a flat file at this stage, but eventually it needs to be > imported into some type of structure. So to that end I decided early on to > use sqlite to write the data out. > > I was hoping for better performance. The raw I/O to read the data and process > is around .75 seconds (no write i/o).. So using a flat file output costs > about .7 seconds. > > Using sqlite to do the output costs about 2.25 seconds. My question is why? > And what can be done to improve this performance? > > > > John Stanton wrote: Ken wrote: > > I'm looking for suggestions on improving performance of my sqlite > > application. > > > > Here are system timings for a run where the sqlite db has been replaced > > with a flat file output. > > real 0m1.459s > > user 0m0.276s > > sys 0m0.252s > > > > This is a run when using sqlite as the output format. > > real 0m3.095s > > user 0m1.956s > > sys 0m0.160s > > > > As you can see sqlite takes twice as long and almost 8 times the user time. > > > > Output size for flat file: 13, 360, 504 flatfile.dat > > Output size fo sqlit file: 11,042,816 sqlt.db f > > > > Slite db has the following pragmas set. > > > > PRAGMA default_synchronous=FULL > > PRAGMA temp_store=memory > > PRAGMA page_size=4096 > > PRAGMA cache_size=2000 > > > > Any ideas how to get the sqlite output timings to a more respectable level > > would be appreciated. > > > > Thanks > > Ken > > > If you want flat file performance, use a flat file. Sqlite is built on > top of a flat file and cannot be faster or even as fast. If your > application can use a flat file, why use anything more complex? > > ----------------------------------------------------------------------------- > To unsubscribe, send email to [EMAIL PROTECTED] > ----------------------------------------------------------------------------- > > > ----------------------------------------------------------------------------- To unsubscribe, send email to [EMAIL PROTECTED] -----------------------------------------------------------------------------
