On Jan 17, 2007, at 2:09 PM, Robert Latham wrote:
On Wed, Jan 17, 2007 at 11:47:37AM -0800, Chris Halstead wrote:
Copy tests were done by simply timing 'cp' from the mounted PVFS
volume to
local /dev/shm (to avoid filesystem latency).
When I strace the pvfs server process, I see the server writing to
the TCP
socket in chunks no larger than ~4k:
writev(11, [{"\277\312\0\0\4\0\0\0\213^\0\0\0\0\0\0(\20\0\0\0\0\0
\0", 24},
{"\274\v\0\0\2\0\0\0!\0\0\0\0\0\0\0\1\0\0\0\0\0\0\0\0\0>"...,
4136}], 2) =
4160
What you are seeing there is that the cp(1) program uses the blocksize
of the destination file system when deciding how big a buffer to use.
About a year ago we got the coreutils maintainers to take into account
the source and destination file systems, but that's only available in
coreutils-6.0 and newer (so it might take a while longer to show up in
distributions).
I've tried tuning kernel-level TCP settings and
TCPBufferSend/TCPBufferReceive in the pvfs config, but I never see
a change
in the size of the writes to the TCP socket. Is this normal? Is
there a
tuning parameter that I'm missing that will influence the size of the
writes?
Two things to try: 1: you can upgrade your coreutils or 2: you can use
pvfs2-cp, which will bypass the kernel VFS and use a 4 MB buffer size
by default.
If you wanted to go through the VFS, you could also use dd and
specify the blocksize you want.
-sam
==rob
--
Rob Latham
Mathematics and Computer Science Division A215 0178 EA2D B059 8CDF
Argonne National Lab, IL USA B29D F333 664A 4280 315B
_______________________________________________
Pvfs2-users mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
_______________________________________________
Pvfs2-users mailing list
[email protected]
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-users
