[Monotone-devel] ideas to speed up monotone SQLite database access

2006-02-05 Thread Joe Wilson
I noticed a recent post on the SQLite mailing list (http://www.mail-archive.com/sqlite-users%40sqlite.org/msg12839.html) asking how monotone might speed up its database access. Two simple optimizations that come to mind are: (1) Use a larger default SQLite page size and/or cache size:

Re: [Monotone-devel] ideas to speed up monotone SQLite database access

2006-02-05 Thread Daniel Carosone
On Sun, Feb 05, 2006 at 02:39:34PM -0800, Joe Wilson wrote: (1) Use a larger default SQLite page size and/or cache size: As well as the other reasons, another advantage of this is that it reduces the IO per-transfer overhead from everything between the syscall layer and the disk, and probably

Re: [Monotone-devel] ideas to speed up monotone SQLite database access

2006-02-05 Thread Timothy Brownawell
On Sun, 2006-02-05 at 14:39 -0800, Joe Wilson wrote: [...] (2) Use a :memory: database for the initial monotone pull and then dump the resultant database to file. This would eliminate the disk latency entirely. Since most monotone databases I've seen are generally under 100 megs, this is a

Re: [Monotone-devel] ideas to speed up monotone SQLite database access

2006-02-05 Thread Joe Wilson
A ^C handler could be created to flush the contents to disk. As for crashes, c'est la vie. ;-) Even if this idea is not pursued, getting timings using a :memory: db would give you a theoretical ideal database time. The remaining time would be mostly attributed to monotone code itself and netsync

[Monotone-devel] Re: ideas to speed up monotone SQLite database access

2006-02-05 Thread Joe Wilson
Some more data on various SQLite page sizes' effect on database size, this time on the monotone repository itself: $ ls -l *.db -rwxr-xr-x 1 Administrator None 50421760 Feb 5 21:56 mt.db -rwxr-xr-x 1 Administrator None 49896448 Feb 5 22:04 mt1024.db -rwxr-xr-x 1 Administrator None 37847040 Feb