What is your page size?   Make sure it matches the sector size of your
device.   SSDs can be picky about write sizes.  As an experiment, you might
try using larger page sizes that are multiples of the sector size.

Try to reduce the size of the records you are writing.  Ie. can you map any
string data to an integer enumeration?    Perhaps you can "normalize" some
of the columns to de-duplicate the amount of data stored.   Can you
compress your blob data?  In general, the less bytes you have to write the
better.

If possible, make sure your records fit "nicely" into your page size as
this should make writes more efficient.  The sqlite-analyzer might give you
some clues about the "shape" of your database.
http://www.sqlite.org/download.html

Do you have a non-integer column defined as a "PRIMARY KEY"?  Perhaps the
"WITHOUT ROWID" optimization will help.
http://www.sqlite.org/withoutrowid.html

You mentioned that this work might get moved to a "computing cluster".  As
a caution, your tuning for your mac laptop/ssd combination might make
little difference (or even make it worse) on a machine with different
cpu/memory/io characteristics.  Ie. you might be severely CPU bound there.

Are you really going to be doing bulk loads like this all the time?
Usually data is loaded once, then read many times.  If not, perhaps you
should be focused on improving your query performance.

HTH.
-Shane





_______________________________________________
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Reply via email to