Dear Olaf, On May 22, 2009, at 3:21 PM, Olaf Schmidt wrote:
> > "Rosemary Alles" <al...@ipac.caltech.edu> schrieb im > Newsbeitrag > news:f113017d-8851-476d-8e36-56b2c4165...@ipac.caltech.edu... > >> I have a database (simple schema) with two tables on which I >> perform "concurrent" udpates over NFS ... >> ... >> Large updates, distributed over several cores over NFS - >> supposedly concurrent "but not really"? > Could you give some more detailed background-info about > your current scenario? > > What do you mean with "cores"? > Are these multiple client *machines* - or do we talk about > a sinlge client-machine with multiple (cpu-)cores here? > Multiple machines with multiple cpus. > In case you mean only "several cores" on a single client-machine - > why is your DB "behind" NFS at all (and not on a local disk)? > N/A - see above - several machines. > In either case (be it multiple client-machines which talk to your > DB - or just multiple cores on a (capable but) single client- > machine - you will not achieve faster inserts against your DB > by working concurrently (regarding the write-direction to the > SQLite-DB). > SQLite can profit from multiple cores (threads) only in the > Read-Direction. We achieve neither, they are both (read and write - not done simultaneously, i.e. never read when writing and vice versa) god awful slow. > > > That does not mean, that your Inserts will be slow (if only > one process/thread or core will handle them) - after all you > perform your DB-writes against a single resource (your disk). > And whilst working against a local disk, SQLite can achieve > ca. 50000-200000 inserts per second, depending of course > on the Column-Count and if the underlying table has indexes > defined on its columns. In my tests I can achieve ca. > 120000 inserts per second on a table with 8 "mixed-type" > Columns (having no indexes defined on it - normal 7200rpm > SATA-HardDisk). > Obviously, we are doing something weird, or our NFS configuration is strange or whatever. The disk is obviously not local The insert speeds you specify are more than likely heavily dependent on configuration/ setup. Are you also working on multiple machines with multiple cpus over NFS? > So, what timings do you currently get, in case you perform > your updates only running on one single client (or core)? In this case, on a local disk, much less than a second per update. However, scaled over the previously described "concurrent" scenario with several identical copies of the process (program) attempting to access the database over NFS from several computers with several cpus - up to a minute per update - It's ridiculous. > > And could you please check the DB-Size before and after > such a typical "update-job" (so that we get an impression > about the transferred byte-volume - maybe you could also > give the count of new records in case of insert-jobs)? > Don't have those numbers handy, but will soon. The total size of current DB is up to 70mb. > And do you work over GBit-Ethernet (or an even faster > "NFS-channel")? > The latter. My primary concern now is to prevent a dead-lock. Many thanks, rosemary. > Regards, > > Olaf Schmidt > > > > _______________________________________________ > sqlite-users mailing list > sqlite-users@sqlite.org > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
