On 2016-03-14 00:08, Thor Lancelot Simon wrote:
I had occasion a couple of days ago to try to block-copy a very large
filesystem from a xen dom0 to another machine across a fast local network.
I tried this:
sysctl -w kern.sbmax=10000000
sysctl -w net.inet.tcp.sendbuf_auto=0
dd if=/dev/rsd0g bs=1048576 | ttcp -t -b 2097152 -l 131072 -fm -p 9001 172.0.0.1
On the far end, I had simply the corresponding ttcp -r job (with corresponding
socket/tcp sysctls in place) redirected to a file.
The underlying hardware is a RAID5 of 300GB SAS drives on an Areca 1680.
The Ethernet controller in use is a 'wm'. All supported offload functionality
is turned on (turning it off just slows things down).
Shockingly, I saw the system go to 0% idle time, with 45-55% "Interrupt"
and the rest "System". Interrupts per second were a comparatively low 1500,
about 500 disk and 1000 network. Throughput was horrible -- about 35MB/sec.
This might not be related, or useful, but it do sound similar to what
I've been seeing on VAXen for a few years now. I get unexpectedly high
interrupt rates, but most significantly, I can get ridiculous system
time when doing something like a "cvs update". I have never had the time
to try and figure it out, but I normally have over 50% (usually around
70%) in system when doing this. Which does not seem reasonable.
But once more, this might be a red herring, so feel free to ignore.
Johnny