On Wed, Nov 2, 2011 at 11:41 AM, Fabian <fabianpi...@gmail.com> wrote: > 2011/11/2 Mr. Puneet Kishor <punk.k...@gmail.com> >> ahh, so you *are* getting expected behavior, just not what *you* expected. >> Did you have a different number in mind instead of a factor of 300? And, if >> so, why? > > To read an (un-cached) 150mb file from disk, doesn't take 30 seconds. And
But that's NOT what SQLite3 is doing. SQLite3 is doing random I/O. And the OS sees the random I/O pattern and concludes it's better to not read the whole file in. So for those 10K inserts you pay -worst case- 10K I/Os. At ~12ms per random I/O (likely the seek times for your disks) you're talking 120s, so you're actually far from the worst case -- even at 7ms seek time you're talking about twice the time you've seen in the worst case. What you should do, given that this one file is critical to your app, is read the whole file into memory (if it were stored on contiguous blocks, which it won't be, that'd make for about one second to read it in). Actually, it might be nice if SQLite3 had a function or open flag by which to request that the whole thing be read into memory, because the OS certainly won't know to do it. Nico -- _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
