> The same with STL vectors: initializing it with a size is faster than > growing it element by element.
That's pretty bad comparison. Initializing of vector with size is faster because it doesn't have to copy all elements later when it reallocates its memory. File system doesn't work that way, it doesn't change location of already written file data when you grow the file. Probably more appropriate comparison will be with STL deque. And depending on its internal implementation you can see very little improvement of reserving some size in deque compared to growing element by element. > Here I (or we) think of the cycles the system needs when the small niche > of the initial database is exhausted and it has to look for another free > block on the filesystem. If you can tell the system in advance, how big > the niche has to be, it saves some time. Did you measure that or know of someone who measured? I guess that can save some time (depending on file system implementation) if your disk is not fragmented at all and all free space is in one big chunk. And even in this case I doubt that speed up will be very big. But if your disk is heavily fragmented algorithm for allocating big file in the file system can be pretty much the same: find first free block, use it, if it's not enough find next one, use it etc. So it could be the same performance as when you grow your file block by block. Pavel On Thu, Aug 12, 2010 at 7:19 AM, TeDe <tede_1...@gmx.de> wrote: > Hello Pawel, > > you made some good points. I'm still in the stage of evaluation, I don't > claim to know, its faster. But I saw that behavior on a filemanger: when > you copy a large file, it immediately reseveres the whole space. The > same with STL vectors: initializing it with a size is faster than > growing it element by element. > Therefor my question, if there is such a possibility. > >> I wouldn't be so sure about that. Did anybody make any measurements? >> 1) I don't know where do you think CPU cycles are saved but you >> definitely get some more CPU cycles in maintaining free-list of pages >> which will never be there if database grows page-by-page. > Here I (or we) think of the cycles the system needs when the small niche > of the initial database is exhausted and it has to look for another free > block on the filesystem. If you can tell the system in advance, how big > the niche has to be, it saves some time. >> 2) Even if you use Martin's technique by creating some big blob why do >> you think that SQLite will grow database file by necessary amount of >> pages at once instead of page-by-page? And is there something in >> SQLite's code that writes several sequential pages in one OS call >> instead of writing them page-by-page? > Thats indeed very uncertain. Here we have to look, how SQLite handles > this. If you have a transaction, SQLite could look, how much more space > is needed and preallocate this. >> I can agree that making file grow in one big piece instead of many >> small ones seems to compact most IO into one call instead of many. But >> will it be somehow faster? I doubt it. And bear in mind that all your >> efforts can be immediately trashed away by another process reading >> some big file(s) which will consume all OS file cache, so OS will have >> to re-read your database file later when you actually need it. This >> way I guess overall number of IO operations on the system will only >> increase... > That might be. We just can prove it one way: measure, measure, measure. > > I also don't expect huge performance increases, but if it is some > percent with little effort.. It could be worth it. > > Thomas > _______________________________________________ > sqlite-users mailing list > sqlite-users@sqlite.org > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users > _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
