On Wed, Nov 2, 2011 at 1:20 PM, Fabian <[email protected]> wrote: > Linux will not read the whole file in, but Windows eventually does. The > inserts go progressively faster when they are reaching halfway, and Windows > reads very large pages from disk, even if you request only 10 bytes. So in
The largest read will depend on how many contiguous blocks are on disk. For a file with lots of random writes that could be very low. You'd need to get down and dirty with the filesystem to find out for sure. Or use DTrace (ah, but there's no DTrace on Windows). You're left to make assumptions, and you want to make the most optimistic ones and feel disappointed when it turns out that those assumptions were wrong :) Even if the file was all contiguous on disk and the OS was smart enough to realize that reading the whole thing in is the right thing to do, there's a limit to how far you can take this since the file could be too large to fit in RAM. This isn't SQLite3's fault... > reality a very large percentage of these 10K I/O's will come from a buffer > (either Windows one or your harddrive's buffer), and will not result in any > physical reads from disk. Ofcourse you're right that these random reads > will be slower than a sequential file-copy, because they are random, and > not large, continous blocks. The I/Os will be blocksize I/Os, not application write size. Incidentally, it pays to make the SQLite3 page size match the filesystem preferred block size. >>Actually, it might be nice if SQLite3 had a function or open >>flag by which to request that the whole thing be read into memory, >>because the OS certainly won't know to do it. > > I completely agree, because all the current methods (copy the file to > 'null', etc.) didn't work well. But note that this can still fail you when the file is larger than available RAM. In that case such a flag would be very bad. And SQLite3 can't know how much RAM is available. The OS can know (sortof) and the user can know, but SQLite3 can't. So I take the above back -- such a flag would probably result in posts about how SQLite3 startup causes thrashing... Nico -- _______________________________________________ sqlite-users mailing list [email protected] http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
