I should have clarified: the problematic locking is happening in the OS layer. I've completely disable SQLite (thread) locking by building with SQLITE_THREADSAFE=0. (And, yes, I'm only using SQLite from a single thread!)
Regarding transactions, I'm bundling write operations into transactions, but not optimally. If, say, I do a huge write as a single transaction, will that cause SQLite to keep everything in memory until I COMMIT? Perhaps that's the right strategy for me to pursue. If there is a document (or even section of the SQLite source) that I could read to fully understand where the transition from memory (page cache, etc.) to disk occurs, that would probably get me above n00b level of understanding, which would help. Dave Sent with inky<http://inky.com?kme=signature> "Simon Slavin" <slavins at bigfraud.org> wrote: On 17 Feb 2016, at 2:37pm, Dave Baggett <dmb at inky.com> wrote: > I'm seeking specific advice on how to tune SQLite for this application when > deployed on a target with extremely poor write performance. On this target > writing in many small chunks is much more expensive than writing in a single > big sequential chunk. In particular, the write syscall is very expensive, > because frequent writes cause the host to spend huge amounts of time in lock > contention, because the locking strategy is very coarse. The vast majority of time spent writing is not in the change you asked to make (INSERT, UPDATE, etc.) but in the measures made to handle the transaction: locking, ensuring ACID, etc.. Are you doing multiple write commands in one chunk of time ? If so, are you enclosing them in a single transaction ? Simon. _______________________________________________ sqlite-users mailing list sqlite-users at mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
