On 10/27/11 13:47, Mark Phippard wrote:
On Thu, Oct 27, 2011 at 4:47 AM, Attila Nagy <b...@fsn.hu
<mailto:b...@fsn.hu>> wrote:
On 10/26/11 15:37, Philip Martin wrote:
Attila Nagy<b...@fsn.hu> <mailto:b...@fsn.hu> writes:
I'm trying to update a working copy of some tens of GBs with svn 1.7.1
Did you upgrade with 1.7.0 or 1.7.1?
I've upgraded the WC with 1.7.0, then switched to 1.7.1, which I'm
currently using.
The upgrade took nearly one week (I can't afford a clean checkout,
because I have to preserve the files' inode numbers), at the start
it was running very fast, and at the end of the week it could
print a new directory in every 8-10 seconds, while the svn
processes CPU usage was 100%.
The disks weren't touched, it seems that even with a database
completely in the OS's buffer cache the SQL operations are pretty
slow with a lot of files.
Could it be because some indexes are missing (or not optimized for
this amount of files), or it is what we could get from SQLite?
I can see where having all that RAM could help greatly when the
database is being read, but I do not see how it would help when we are
writing to the database. I would imagine the database does stuff to
flush itself to disk whenever a transaction is committed. I thought
this was why we see 1.7 slower than other versions via NFS, even on
small working copies where the database would fit in anyones RAM.
I would imagine svn upgrade is almost entirely writes and I recall it
does quite a few transactions. So couldn't the linear slow down just
be based on the growth in the amount of bytes that are written to disk
each time?
I've checked the disk IO stat and I couldn't see any bottlenecks there.
In fact while upgrade-ing, I barely saw any disk IO.
Checking the process revealed a lot of random reads from the wc.db,
which the OS could serve out of memory.
BTW, I don't have any transaction-related files next to my wc.db (I
remember about wal, or journal named files, but I'm not a regular user
of SQLite), and definitely couldn't see too much fsyncs.
