On Sun, 17 Mar 2013 02:14:30 +1100 (EST)
Ian Smith <smi...@nimnet.asn.au> wrote:
On Wed, 13 Mar 2013 21:11:28 -0500, John Mehr wrote:
> On Wed, 13 Mar 2013 16:29:45 +1100 (EST)
> Ian Smith <smi...@nimnet.asn.au> wrote:
svn0.us-west.freebsd.org as per example, but it's the
closest to here
anyway; pings avg 217.5ms stddev 0.4ms, us-east avg
269.3ms sd 9.5ms.
fastest_cvsup used to find cvs mirrors in Australia at
around 55-70ms,
so for another while I thought RTT might be the issue -
but it's not.
I'm seeing ~110ms pings to svn0.us-west.freebsd.org.
> I'll definitely put in a verbosity level that does
exactly what you suggest.
> Sorry about that.
Not at all, and thanks! But neither, I now find, is
logging the extra
time issue; as mentioned elsewhere, running at -v0 makes
next to no
difference, 12:48 west vs 12:54 east on 4 do-nothing
runs, 700KB/s link,
not running in script(1) - though I think that matters
not, or little.
So I began watching top more closely, also running
iostat -w10, noticing
that svnup was pegging CPU at 100% with ~75% _system_?
time for nearly 9
minutes of the run, doing little or no disk I/O and a
max of 130KB/s in,
22.6KB/s out, no stress there, growing to ~20MB
resident, then doing a
flurry of disk I/O at about 3-5MB/s for the last 3m40s
odd. IOW, svnup
here is mostly heavily CPU-bound, which I don't
understand, but then I
know nothing about what level of computation the
protocol demands.
Except for the time duration of the run, the behavior
you're describing is normal. Using the svn protocol, the
program runs in three phases:
1) It issues a series of get-dir commands to the server to
get a list of all the files/directories in the repository
and stores the structure in memory. There is some disk
activity during this as it creates the directory tree (it
uses one lstat per directory and creates the directory if
it doesn't exist). On my development machine, this phase
takes about 25% of the total time of a do-nothing run.
2) Because the svn protocol doesn't include MD5 signatures
and the last-author/committed date/committed revision
fields in the results of the get-dir commands
<gripe>although they *are* included in the http protocol
responses</gripe> it has to issue a get-file request for
each file to get this information. This phase takes about
50% of the total time of a do-nothing run.
3) The downloaded MD5 signatures are for versions of the
files that do not have their $FreeBSD$ tags expanded, so
each file on the local machine must be loaded into memory
and have their $FreeBSD: ... $ tags collapsed and MD5
summed to see if there's a match. If there's no match,
the file on the server is downloaded. This phase takes up
the remaining 25% and is I/O intensive.
This three step process is further complicated because of
the huge time penalty that occurs when sending commands
one-at-a-time. To get around this, as many commands that
can fit in 32KB are grouped together and sent in bulk.
I've done some more controlled tests since, running both
iostat -w10 and
netstat -w10 -dn before, during and until a bit after
the run, attached,
hoping these might help pinpoint any bottlenecks? This
was running -v1
default verbosity, hence the (not at all heavy) tty
output numbers.
Just sometimes, running slower hardware can have
advantages! On your
times, yours likely has more than 500% of my single-core
P3-M's 1133MHz
performance, which would tend to mask this issue pretty
well for you :)
I do all of my development and test runs on a nettop with
an AMD E-350 (dual core, 1.6GHz) processor, and with the
program only utilizing one core I'd be willing to bet the
two chips are in roughly the same league. Memory and hard
drives probably aren't, and with 512MB of memory, how does
your swap utilization look?
As soon as I can get a replacement CMOS battery, I'll be
dusting off my old Athlon 2700+ system to run tests on it.
John Baldwin has best described when c{,v}sup deletes
(only in-tree)
files, covered in section 'delete' in csup(1). I think
this should
probably be default behaviour, even if svn proper would
do otherwise.
Will do. Setting up both a default of deleting only files
that appear in previous checkouts plus giving each user
the option of skipping specific directories from any
pruning is on the to-do list. Thanks!
cheers, Ian
_______________________________________________
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"