Re: Fast network file copy; "recvfile()" ?

2008-01-21 Thread dean gaudet
On Thu, 17 Jan 2008, Patrick J. LoPresti wrote: > I need to copy large (> 100GB) files between machines on a fast > network. Both machines have reasonably fast disk subsystems, with > read/write performance benchmarked at > 800 MB/sec. Using 10GigE cards > and the usual tweaks to tcp_rmem etc.,

Re: Fast network file copy; recvfile() ?

2008-01-21 Thread dean gaudet
On Thu, 17 Jan 2008, Patrick J. LoPresti wrote: I need to copy large ( 100GB) files between machines on a fast network. Both machines have reasonably fast disk subsystems, with read/write performance benchmarked at 800 MB/sec. Using 10GigE cards and the usual tweaks to tcp_rmem etc., I am

Re: Fast network file copy; "recvfile()" ?

2008-01-17 Thread Jan Engelhardt
On Jan 17 2008 12:53, Patrick J. LoPresti wrote: >Using 10GigE cards and the usual tweaks to tcp_rmem etc., I am >getting single-stream TCP throughput better than 600 MB/sec. Hm, be aware not to hit the sequence wrap :) >1) O_DIRECT read() + send() >2) mmap() + madvise(WILLNEED) + send() >3)

Fast network file copy; "recvfile()" ?

2008-01-17 Thread Patrick J. LoPresti
I need to copy large (> 100GB) files between machines on a fast network. Both machines have reasonably fast disk subsystems, with read/write performance benchmarked at > 800 MB/sec. Using 10GigE cards and the usual tweaks to tcp_rmem etc., I am getting single-stream TCP throughput better than 600

Fast network file copy; recvfile() ?

2008-01-17 Thread Patrick J. LoPresti
I need to copy large ( 100GB) files between machines on a fast network. Both machines have reasonably fast disk subsystems, with read/write performance benchmarked at 800 MB/sec. Using 10GigE cards and the usual tweaks to tcp_rmem etc., I am getting single-stream TCP throughput better than 600

Re: Fast network file copy; recvfile() ?

2008-01-17 Thread Jan Engelhardt
On Jan 17 2008 12:53, Patrick J. LoPresti wrote: Using 10GigE cards and the usual tweaks to tcp_rmem etc., I am getting single-stream TCP throughput better than 600 MB/sec. Hm, be aware not to hit the sequence wrap :) 1) O_DIRECT read() + send() 2) mmap() + madvise(WILLNEED) + send() 3)