On Saturday, 19 October, 2019 18:55, Peng Yu <pengyu...@gmail.com> wrote:
>I have never seen such a dramatic difference in non-sqlite3 >operations. For example, cat has some difference but is ~13%. Have you been looking? >$ time cat file.sqa > /dev/null > >real 0m7.282s >user 0m0.067s >sys 0m2.371s >$ time cat file.sqa > /dev/null > >real 0m6.316s >user 0m0.062s >sys 0m2.319s That must be a really huge file to take so long to spool to the bitbucket. I'd need at least a 20 GB file to take that long. Even then the whole thing would be in memory after the first go, and the second go would take only milliseconds. You must have less memory than the size of that file. (and no, there is no real difference between those two timings). >How to prove the large time difference of sqlite3 is indeed due to >cache and where is the cache? You can't prove anything. The disk cache is part of the Operating System. >Why the caching used by sqlite3 can not persist for a longer time? The page cache used by sqlite3 only exists while the program is running. It releases the memory it is using when the program terminates (all programs do this). The Operating System, however, may "help" you by storing files in RAM assuming that you have enough RAM and that it is not needed for something else. >On Sat, Oct 19, 2019 at 7:29 PM Keith Medcalf <kmedc...@dessus.com> >wrote: >> >> >> On Saturday, 19 October, 2019 18:05, Peng Yu <pengyu...@gmail.com> >wrote: >> >> Looks like the difference between reading from disk and reading from >cache. >> >> >I see that sqlite3 can be very different in terms of run time. >> >> >$ time sqlite3 file.sqa -Atv > /dev/null >> > >> >real 0m3.259s >> >user 0m0.193s >> >sys 0m0.704s >> >$ time sqlite3 file.sqa -Atv > /dev/null >> > >> >real 0m0.148s >> >user 0m0.086s >> >sys 0m0.057s >> > >> ># Then do some else briefly >> > >> >$ time sqlite3 file.sqa -Atv > /dev/null >> > >> >real 0m3.853s >> >user 0m0.200s >> >sys 0m0.828s >> > >> >$ time sqlite3 gncdhgncsym2htmlsqa.sqa -Atv > /dev/null >> > >> >real 0m0.172s >> >user 0m0.094s >> >sys 0m0.065s >> > >> >file.sqa is of size 4.1G and 42282 rows. >> > >> >Does anybody know why there can be such big differences in speed? >> > >> >My testing machine is a MacbookAir. -- The fact that there's a Highway to Hell but only a Stairway to Heaven says a lot about anticipated traffic volume. _______________________________________________ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
