This helps with the locking issues. I believe the slowness was actually related to the index on the 'name' column. If I remove the index from the text field, then the inserts are much faster.
If I keep the index, the inserts eventually come to a crawl. Is 50-100 million with index on a text column an unusual setup? I've read some articles of folks storing hashkeys of the text values in a separate indexed integer column to speed up the indexing process. Thanks, --erik On Sat, Sep 18, 2010 at 9:18 PM, Roger Binns <[email protected]> wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On 09/18/2010 01:45 PM, Erik Fears wrote: > > but I've found that during a write transaction readers are locked from > > reading the database. > > I suggest reading the following two pages: > > http://www.sqlite.org/lockingv3.html > http://www.sqlite.org/wal.html > > Roger > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.10 (GNU/Linux) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ > > iEYEARECAAYFAkyVjxEACgkQmOOfHg372QThYwCff5Cv9kMNpA3DgoDdpIV3rHb7 > S+cAnj+1Gi2r7vB9coOLuY/ypKxYq6Cl > =28W8 > -----END PGP SIGNATURE----- > _______________________________________________ > sqlite-users mailing list > [email protected] > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users > _______________________________________________ sqlite-users mailing list [email protected] http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
