There was a significant increase in throughput converting the sqlite3_exec inserts into prepared statements:
I suspect every previous insert was compiling versus now only 1 of every 10,000 inserts. I'll bump this up to 100K next revision I've changed many variables so its hard to base line the differential: PRAGMA synchronous=OFF" PRAGMA count_changes=OFF" PRAGMA page_size=32768" PRAGMA temp_store=MEMORY" PRAGMA journal_mode=OFF" Library compiled with # define SQLITE_THREADSAFE 0 The CPU is still pegged at 100% but the extraction databases appear to be generating much faster. Currently testing 30 extractors running on a 4 core VM. <<< Reply to Simon >>> There's nothing wrong with sqlite3_exec that will causes excessive CPU usage, but the above usage of it suggests someone put that together quickly. The last three parameters should be more like NULL, 0, &zErrMsg showing no callback, a dummy argument for the non-existant callback, and somewhere for SQLite to put an error message if there is one. Even if you burst the _exec() into _prepare(), _step(), _finalize(), you'll still have to provide the equivalent parameters somewhere. I don't know whether using 0, 0, 0 when there is no error will cause any problems. I suspect not. Simon. _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
