Mike said: >>According to my math your final database size should be on the order of 100Meg? >> >>That means at 200 minutes and 1,000,000 records: >>83 inserts per second >>8333 bytes per second
>>Both of these values are terrible. >>#1 What kind of network connection do you have? 100BaseT? >>#2 What kind of server are you writing to? >>#3 How fast does this run if you write to your local machine? >>Michael D. Black >It DOES sound terrible since 90%? of the time is spend in retrieving >data from a remote oracle server over a slow line. The problem is that >the time spend saving to sqlite is still more than I used to spend on >saving to my previous file system. >Can you answer #3 though? It's nearly 50% faster. >Does anybody know how to make the journal file go to a different location than the database? Apprarently it's not treated as a "temporary" file. Perhaps it should be?? Mike said: Now I'm confused...how can you be 50% faster if 90% of the time is in retrieving from Oracle? Werner with a list of processing speeds to clarify matters. Date Strt End Minutes recs Version Filename 2010-05-03 15:42 16:49 66.7 F 421710 4.4.0.6 Due 2010-05-24 01:12 01:30 17.9 F 419604 CLAR Due 2010-05-27 11:07 13:01 113.5 F 419604 4.4.0.7 Due 2010-06-03 18:54 19:42 47.8 F 419604 4.4.0.9 Due 2010-06-04 17:24 18:09 45.5 B 419604 4.4.0.9 Due 2010-06-07 11:08 13:59 170.5 U 419604 4.4.0.8 Due 2010-06-17 09:13 09:29 16.2 B 419604 CLAR Due 2010-06-28 13:51 14:07 15.9 F 419604 CLAR Due 2010-06-28 15:12 15:37 25.4 B 419604 CLAR Due 2010-06-29 15:57 16:21 24.1 B 419604 CLAR Due 2010-06-30 11:39 12:17 37.8 F 418620 4.4.1.2 Due 2010-07-08 14:38 15:28 49.4 F 418620 4.4.1.2 Due 2010-07-13 13:09 13:52 42.2 U 418620 4.4.1.2 Due 2010-07-13 18:02 18:31 28.6 F 418620 4.4.1.2 Due (local drive) 2010-07-14 13:27 14:15 47.8 B 418620 4.4.1.2 Due 2010-07-14 15:28 16:03 34.6 F 418620 4.4.1.2 Due I hope above lines stay in correct columns. As you can see I started with building these records in 66 minutes. Where previous version "CLAR" takes between 15 and 24 minutes After taking out count(*) and adding a few pragma's and saving 6000 records rather than 500 at a time I've got it down to 34 minutes. If I build in on local drive it takes 28 minutes.(with chunks of 500) I went with a finecomb thru the code and could not find any other showstoppers like "count" Maybe I should dump to a txt file and see what the speed is like? ps. The last entry (34 minutes) was done with blocks of 6000 rather than blocks of 500. DISCLAIMER: Everything in this email and its attachments relating to the official business of Face Technologies is proprietary to Face Technologies. It is confidential, legally privileged and protected by law. The person addressed in the email is the sole authorised recipient. Any unauthorized dissemination or copying of this e-mail (or any attachment to this e-mail) or the wrongful disclosure of the information herein contained is prohibited. _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
