It also helps to put the database file(s) on a different physical hard drive than the hard drive of the operating system and swap.
--- Carl Jacobs <[EMAIL PROTECTED]> wrote: > Nicolas, > > From: <[EMAIL PROTECTED]> > > On the other hand, I tried to make better use of the cache: if I run my 1M > inserts in 10 transactions of 100,000, things get a bit slower than 100 > transactions of 10,000 inserts. > > I tried one transaction of 1,000,000 inserts and the test app hangs at > 100% cpu for over 30 minutes now, not sure about what is going on here. > > > > I think I just recently bumped into something similar. Doing 8000 inserts in > a transaction and I got an out-of-memory error before the transaction could > complete - so now I limit it to around 1000 inserts at a time without > problem. So disk cache on the sqlite database is one issue, but if you're > needing a large amount of temporarily allocated memory in some operation > then the O/S may need to be swapping chunks of memory back and forth to > disk - which could mean that the memory manager disk accesses are fighting > with the SQLite disk accesses - which could explain the catastrophic slow > down that you observed for 1,000,000 inserts. In my insert I was using a lot > of sqlite3_bind_text with the final value set to SQLITE_TRANSIENT - I'm sure > sure at what point that "TRANSIENT" memory gets freed. > > Regards, > Carl. > > > __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
