Re: [Samba] SMB/CIFS seq. transfers top out at 30MiB/s (NFSv4 and HTTP: 100MiB/s+)
On Thu, 21 Jan 2010 00:10:07 +0100 Johannes Truschnigg johan...@truschnigg.info wrote: On Wednesday 20 January 2010 20:55:16 Jeff Layton wrote: […] Most likely, you're running into the lack of parallelism in Linux' CIFS client. Writes are done in in turn currently and not in parallel as they should be. […] Very interesting; but that wouldn't explain why it's slow for Windows XP clients, would it? Is there any SMB/CIFS client implementation that's known for its speed so I could test if it's really a client issue? Oops, missed that point in the initial email. You're correct -- it wouldn't explain why windows is slow. I *think* smbclient is actually reasonably fast and does parallel reads/writes. You may want to try it. If it's also slow, I'd probably do some analysis of the traffic on the wire and see if you can determine the cause that way. -- Jeff Layton jlay...@samba.org signature.asc Description: PGP signature -- To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/options/samba
Re: [Samba] SMB/CIFS seq. transfers top out at 30MiB/s (NFSv4 and HTTP: 100MiB/s+)
Hi Matt, thanks very much for your input! On Thursday 21 January 2010 15:02:09 M D wrote: […] socket options = IPTOS_LOWDELAY TCP_NODELAY After this, I'm seeing significantly quicker transfers to windows 7 clients - at almost line speed […] I had those options enabled a while back, and tried once again today - to no positive effect whatsoever. Performance has in fact been going down, albeit very slightly, with those settings by about 5-10%. I top out at 27MiB/s with all my clients. Thanks for your insights regarding kernel tunables - I already tweaked the kernel's TCP parameters (initial window size, rx buffer memory, etc.) to allow for better transfer speeds on my GBit link, and other file transfer protocols easily hit the speeds I'd like to see with CIFS - that is, more than 100MiB/s. Right now, even OpenSSH's sftp-server using sftp is faster for me (with considerably more load on the transfer's endpoints, of course) than smbd serving up CIFS. -- with best regards: - Johannes Truschnigg ( johan...@truschnigg.info ) www: http://johannes.truschnigg.info/ phone: +43 650 2 17 xmpp: johan...@truschnigg.info Please do not bother me with HTML-eMail or attachments. Thank you. signature.asc Description: This is a digitally signed message part. -- To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/options/samba
Re: [Samba] SMB/CIFS seq. transfers top out at 30MiB/s (NFSv4 and HTTP: 100MiB/s+)
On Thursday 21 January 2010 14:31:24 Jeff Layton wrote: […] I *think* smbclient is actually reasonably fast and does parallel reads/writes. You may want to try it. If it's also slow, I'd probably do some analysis of the traffic on the wire and see if you can determine the cause that way. As stated in my initial mail, I already tried using `smbclient`, and it basically hits the very same speed-limit that the kernel's CIFS-implementation cannot exceed. Is there any canonical or recommended way to debug/profile smbd's performance characteristica at runtime? I'm not sure how looking into the transferred packets themselves could help me, since the reason for the bad performance is either on the server (I'm quite adamantly sure by now that the server, and smbd specifically, is the root of the problem at hand), or the clients - not the wire or networking equipment in between. If you can elaborate on the subject though, I'd be delighted to listen and learn :) Thanks very much for contributing! -- with best regards: - Johannes Truschnigg ( johan...@truschnigg.info ) www: http://johannes.truschnigg.info/ phone: +43 650 2 17 xmpp: johan...@truschnigg.info Please do not bother me with HTML-eMail or attachments. Thank you. signature.asc Description: This is a digitally signed message part. -- To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/options/samba
Re: [Samba] SMB/CIFS seq. transfers top out at 30MiB/s (NFSv4 and HTTP: 100MiB/s+)
On Thu, Jan 21, 2010 at 06:05:23PM +0100, Johannes Truschnigg wrote: As stated in my initial mail, I already tried using `smbclient`, and it basically hits the very same speed-limit that the kernel's CIFS-implementation cannot exceed. Which version of smbclient? Please use 3.2 at least. Volker signature.asc Description: Digital signature -- To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/options/samba
Re: [Samba] SMB/CIFS seq. transfers top out at 30MiB/s (NFSv4 and HTTP: 100MiB/s+)
On Wed, 20 Jan 2010 19:41:24 +0100 Johannes Truschnigg johan...@truschnigg.info wrote: Hello list, I'm using Samba 3.4.5 on a home-hosted fileserver of mine to easily share files with both GNU/Linux and Windows XP (Professional 32Bit SP3) clients. The machines are connected to each other via a switched GBit ethernet network, the actual available bandwidth between the server and the nodes over TCP amounts to about 940-980MBit (according to iperf). The server's storage backend is aquite potent, and achieves sequential read and write speeds well over the network's linespeed (somewhere around 180MiB/s read and 130MiB/s write at worst and well over 200MiB/s and 160MiB/s, resp., at best). When serving files from the very same filesystem as with Samba, Apache 2.2 delivers about 110MiB/s on average, and NFSv4-transfers match that number in terms of speed. With Samba and SMB/CIFS, however, the transfer speed tops out at rather disappointing 28-30MiB/s, serving both Windows and GNU/Linux clients. I'm using the in-kernel CIFS support on the GNU/Linux machines, and the default Attach Network Drive-feature with Windows. Using the (afaik pure userspace) `smbclient` implementation on the GNU machine doesn't change anything to the better, the speed remains at the aforementioned ~30MiB/s. The server system isn't really loaded while serving files over CIFS (its cores don't even clock to higher frequencies, but remain at a comfortable 1GHz), and smbd never consumes substantially more than ~15% CPUtime while reading from disk and delivering to the clients. I tried adopting advice from the Samba manual's performance tuning section (http://www.samba.org/samba/docs/man/Samba-HOWTO-Collection/speed.html), but that didn't improve the situation at all - transfer speed remained roughly the same, compared to these settings' default values. Playing with the clients' mount options (rsize and wsize, specifically) didn't yield any noticeable improvements, either. I continue to see other peoples' reports on the web that they manage to squeeze much higher transfer rates (70MiB/s+) out of dedicated NAS appliances via SMB/CIFS (most of which supposedly also run Samba for providing that service), and I fail to see why my platform won't deliver similar results, as the hardware should easily match whatever those NAS-devices offer. Below I will list what I think might be relevant information to track down what's wrong; in case I'm missing something that'd be of use, please, let me know! SNIP START: egrep -v '^[[:space:]]*[;#]|^$' /etc/samba/smb.conf [global] workgroup = ARBEITSGRUPPE security = share load printers = no guest account = nobody dns proxy = no syslog = 1 syslog only = yes socket options = SO_RCVBUF=8192 SO_SNDBUF=8192 disable netbios = yes unix extensions = yes unix charset = UTF-8 display charset = UTF-8 use mmap = yes use sendfile = yes wins support = no [files] path = /srv/files/pub/ public = yes only guest = yes writable = yes printable = no SNIP END: egrep -v '^[[:space:]]*[;#]|^$' /etc/samba/smb.conf My GNU/Linux client mounts the share with the following options: SNIP START: /etc/fstab //virtue.local/files /media/network cifs \ auto,user,pass=,rw,uid=1000,gid=100,noexec,nolock,\ file_mode=0664,dir_mode=0775,iocharset=utf8,\ wsize=57344,rsize=57344 0 0 SNIP END: /etc/fstab The output of `smbd -d` is recorded here: http://pasted.at/fb1889588d_nl.html All GNU/Linux clients use recent (2.6.32+) kernels and are x86_64 machines. All clients are able to get the HTTP transfer speeds described above. I'm still using the very same kernel that was used to build Samba/smbd on. There's nothing out of the ordinary recorded in the server's logs. `nmbd` isn't running on the server, as I don't need that kind of name resolution support. Samba is Version 3.4.5, running on Gentoo GNU/Linux ~amd64. If anyone spots something obvious that might limit transfer speeds in the way I described, please leave a comment. Thanks very much in advance for your time and effort! Most likely, you're running into the lack of parallelism in Linux' CIFS client. Writes are done in in turn currently and not in parallel as they should be. Fixing it is something we'd like to do, but it's a non-trivial amount of work and it'll probably be a while before it's fixed. -- Jeff Layton jlay...@samba.org -- To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/options/samba
Re: [Samba] SMB/CIFS seq. transfers top out at 30MiB/s (NFSv4 and HTTP: 100MiB/s+)
On Wednesday 20 January 2010 20:55:16 Jeff Layton wrote: […] Most likely, you're running into the lack of parallelism in Linux' CIFS client. Writes are done in in turn currently and not in parallel as they should be. […] Very interesting; but that wouldn't explain why it's slow for Windows XP clients, would it? Is there any SMB/CIFS client implementation that's known for its speed so I could test if it's really a client issue? -- with best regards: - Johannes Truschnigg ( johan...@truschnigg.info ) www: http://johannes.truschnigg.info/ phone: +43 650 2 17 xmpp: johan...@truschnigg.info Please do not bother me with HTML-eMail or attachments. Thank you. signature.asc Description: This is a digitally signed message part. -- To unsubscribe from this list go to the following URL and read the instructions: https://lists.samba.org/mailman/options/samba
