Nicolas, From: <[EMAIL PROTECTED]> > On the other hand, I tried to make better use of the cache: if I run my 1M inserts in 10 transactions of 100,000, things get a bit slower than 100 transactions of 10,000 inserts. > I tried one transaction of 1,000,000 inserts and the test app hangs at 100% cpu for over 30 minutes now, not sure about what is going on here. >
I think I just recently bumped into something similar. Doing 8000 inserts in a transaction and I got an out-of-memory error before the transaction could complete - so now I limit it to around 1000 inserts at a time without problem. So disk cache on the sqlite database is one issue, but if you're needing a large amount of temporarily allocated memory in some operation then the O/S may need to be swapping chunks of memory back and forth to disk - which could mean that the memory manager disk accesses are fighting with the SQLite disk accesses - which could explain the catastrophic slow down that you observed for 1,000,000 inserts. In my insert I was using a lot of sqlite3_bind_text with the final value set to SQLITE_TRANSIENT - I'm sure sure at what point that "TRANSIENT" memory gets freed. Regards, Carl.
