But I thought he said he dropped the indexes (meaning they aren't there during inserts). That should make sorting irrelevant.
3 Things. #1 Test with :memory: database and see what the speed is. That tells you if it's SQLite or disk I/O as the bottleneck. #2 Try WAL mode "pragma journal_mode=WAL" -- sometimes helps a lot. #3 Create your indexes after you're done with all the inserts. Michael D. Black Senior Scientist NG Information Systems Advanced Analytics Directorate ________________________________________ From: [email protected] [[email protected]] on behalf of Simon Slavin [[email protected]] Sent: Monday, March 21, 2011 9:00 PM To: General Discussion of SQLite Database Subject: EXT :Re: [sqlite] Bulk loading large dataset; looking for speedups. On 22 Mar 2011, at 1:27am, Douglas Eck wrote: > I fixed the slowness by sorting the sql statements in ascending key > order and dumping to a new text file, then running sqlite on the new > text file. That *really* helped :-) Neat. Must remember that sorting improves speed that much. Simon. _______________________________________________ sqlite-users mailing list [email protected] http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users _______________________________________________ sqlite-users mailing list [email protected] http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
