Brian, > The fastest data rate you're seeing here is 64MiB/14.80s, so about > 4.47MB/s or roughly 35-40Mbps, which is probably about the middle of > what you'd expect out of a 100Mbps ethernet (maybe a bit on the low > side, but not by much). Was the client CPU pegged during the upload? I > suspect you're CPU bound, but that overall your network is pretty close > to being saturated too.
Yeah, I think we're approximately saturating the network during the large file transfers. But for the small files, both network and CPU load are very low (under 10%). I used systrace to count kernel calls during the large and small file transfers. The heavy end of the histograms are: tahoe backup large files: native-recvfrom(29),: 1593 native-fstat(292),: 2542 native-mmap(197),: 2668 native-sigprocmask(48),: 3183 native-stat(291),: 3367 native-gettimeofday(116),: 4122 native-mprotect(74),: 4167 native-fcntl(92),: 4646 native-read(3),: 5162 native-poll(252),: 8206 native-open(5),: 11813 native-sendto(133): 12296 tahoe backup small files: native-gettimeofday(116),: 5700 native-mprotect(74),: 6205 native-poll(252),: 7344 native-stat(291),: 7443 native-read(3),: 9235 native-fstat(292),: 11692 native-sendto(133): 11922 native-open(5),: 14857 native-fcntl(92),: 34077 native-write(4),: 38708 native-lseek(199),: 45015 native-recvfrom(29),: 186860 daemon process, combined for both transfers: native-recvfrom(29),: 9260 native-getpid(20),: 10350 native-open(5),: 27059 native-fcntl(92),: 27200 native-mmap(197),: 29567 native-read(3),: 30886 native-munmap(73),: 47465 native-write(4),: 76731 native-poll(252),: 124552 native-gettimeofday(116),: 291893 The large number of recvfrom calls on the tahoe backup of small files looks odd because it's a huge mismatch against the sendto calls, but nothing else stands out. I wonder whether temporary file creation for the small file transfers might be part of the problem. I know that temporary files are created on occasion; could someone explain precisely when? > The pipeline will really become important when your client-to-server > latency is more than the fraction of a millisecond that you probably get > on your LAN. How do ping times compare between the wired and the > wireless connections? On this network I'd expect them to be too close to > have a huge effect on tahoe's performance, although clearly there *some* > kind of effect. The one thing that I know wireless networks are more > susceptible to is up/down interference: the ACK coming back from the > server will contend with the next data packet coming from the client > (it'd be worse if both client and server are on wireless connections). Ping to the storage node through the wired interface is about 0.66ms. Ping to the storage node through the wireless interface is about 1.25ms. -- Kyle Markley _______________________________________________ tahoe-dev mailing list tahoe-dev@tahoe-lafs.org http://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev
