Re: [osol-discuss] CIFS vs NFS performance
Operation: dd if=/dev/zero of=test bs=16384 count=512 Results: (CIFS) 8388608 bytes (8.4 MB) copied, 1.7191 s, 4.9 MB/s (NFS) 8388608 bytes (8.4 MB) copied, 0.852603 s, 9.8 MB/s what's the problem? what is the expected result? what caught your attention? cifs and nfs are very different protocols, so it's perfectly reasonable for them to differ performance wise. Also dd is hardly a benchmarking tool I gotta disagree Ignacio. In the end, there may be a fundamental difference in SMB that makes it slower, but until such a fundamental characteristic is identified I fully agree with Yannis that the SMB should be approx. 2x faster than measured (approx matching the NFS). It's not acceptable to simply say so what. It sucks, that's interesting, oh well. It warrants deeper investigation. Also dd is a perfect benchmarking tool. I use it for this sort of operation all the time. It's simple and effective for a lot of situations. Yannis, I would make the following suggestions: Your benchmark is only 1-2 seconds long. You may see skewed results due to system caching. Since you're going across a 100Mb line, you will not max out your disks; the network is definitely the bottleneck. I would suggest something like this: time dd if=/dev/zero of=. bs=1G count=1 This should take about 2 minutes (maybe 4 minutes on the SMB, if SMB is performing poorly) Also, in your NFS setup, check for sync and async options. These too may be skewing your results. I know that one of my colleagues has had great success tuning NFS performance by changing the udp/tcp options of the server, and block sizes. Perhaps the same can be done here? ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: [osol-discuss] CIFS vs NFS performance
I would expect both protocols to saturate the network. What are your performance expectations for cifs? CIFS performance is not limited by network BW or CPU performance (at this BW level). Something else is causing the degradation to 50% of available network BW. Do you have any ideas? I think if you do a lower level network analysis, you will see that both protocols are saturating the network. I think you will see that your SMB implementation is folding in higher overhead, such as, maybe using much smaller packet sizes, or using much more encoding, or both. This would still make the network the bottleneck, while delivering different (in this case 50%) a different level of end user performance. ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: [osol-discuss] CIFS vs NFS performance
use filebench, it has a cifs plugin I think Or iozone, or any other benchmarking utility, which, behind the scenes, will write to disk and measure the throughput of each one, exactly like dd does, except the benchmarking tools will repeat and change parameters such as block size and filesize on each subsequent test. Point is: don't dismiss the dd results. Any other benchmarking tool does the same thing in a more elaborate form. If dd models the behavior he expects to use, then dd measures the performance he cares about. Additional information coming from a different benchmark tool is nice, but that's not to say the present information is unimportant or invalid. Also, you might want to ask cifs related questions in cifs-discuss This is not a purely cifs discussion, and although he might get more results asking in different places too, this group is a perfectly acceptable place to have this discussion. ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: [osol-discuss] CIFS vs NFS performance
After looking around, it appears it is an inherent limitation of the SMB protocol. It doesn't pipeline requests which means it experiences the full 6 msecs latency introduced by MOCA. SMB2 should fix this when it is available in CIFS. In my test, just now (using 1Gb interconnect) I got the following results. I'll point out that with a wire 10x faster than Yannis's wire, I got CIFS performance which is higher than 10x his NFS performance. So I don't think it's fair to conclude it's inherent to CIFS. Also, my NFS performance was acceptable, but not as good as the CIFS performance. CIFS time dd if=/dev/zero of=junk.file bs=10M count=1000 (10 GB) copied, 103.609 s, 101 MB/s real1m43.719s (I calculate 98 MB/s) NFS time dd if=/dev/zero of=junk.file bs=10M count=1000 real2m21.163s (I calculate 73 MB/s) ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: [osol-discuss] CIFS vs NFS performance
My perfomance for my home file server is similar, with actual file transfers, from the windows pc to the opensolaris (snv125) server via CIFS I get in upload, around 100MB/s in burts of 15 seconds, average around 80MB/s (burst probably due to zfs flushing) and read speed is 100MB/s flat line, until the windows pc's cache fills and then its limited by the local drive and that around 75-80MB/s -- This message posted from opensolaris.org ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
[osol-discuss] CIFS vs NFS performance
Setup: Linux client (Atom 330, 1GB d...@533) connects to Solaris server (Atom 330, 2GB d...@533) Network connectivity through 100 Mbps MOCA link (3 msecs latency each way)Solaris server exports the same filesystem through cifs and nfs Operation: dd if=/dev/zero of=test bs=16384 count=512 Results: (CIFS) porta-deb:/safe/storage# dd if=/dev/zero of=test bs=16384 count=512 8388608 bytes (8.4 MB) copied, 1.7191 s, 4.9 MB/s (NFS) porta-deb:/safe-nfs/storage# dd if=/dev/zero of=test bs=16384 count=512 8388608 bytes (8.4 MB) copied, 0.852603 s, 9.8 MB/s Any ideas? Yannis -- This message posted from opensolaris.org ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: [osol-discuss] CIFS vs NFS performance
On Wed, Nov 18, 2009 at 4:55 PM, Yannis Schoinas yan...@schoinas.net wrote: Setup: Linux client (Atom 330, 1GB d...@533) connects to Solaris server (Atom 330, 2GB d...@533) Network connectivity through 100 Mbps MOCA link (3 msecs latency each way)Solaris server exports the same filesystem through cifs and nfs Operation: dd if=/dev/zero of=test bs=16384 count=512 Results: (CIFS) porta-deb:/safe/storage# dd if=/dev/zero of=test bs=16384 count=512 8388608 bytes (8.4 MB) copied, 1.7191 s, 4.9 MB/s (NFS) porta-deb:/safe-nfs/storage# dd if=/dev/zero of=test bs=16384 count=512 8388608 bytes (8.4 MB) copied, 0.852603 s, 9.8 MB/s what's the problem? what is the expected result? what caught your attention? cifs and nfs are very different protocols, so it's perfectly reasonable for them to differ performance wise. Also dd is hardly a benchmarking tool Any ideas? Yannis -- This message posted from opensolaris.org ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: [osol-discuss] CIFS vs NFS performance
I would expect both protocols to saturate the network. What are your performance expectations for cifs? CIFS performance is not limited by network BW or CPU performance (at this BW level). Something else is causing the degradation to 50% of available network BW. Do you have any ideas? What's wrong with dd? What would you use to test sequential write performance? Yannis -- This message posted from opensolaris.org ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: [osol-discuss] CIFS vs NFS performance
On Wed, Nov 18, 2009 at 5:54 PM, Yannis Schoinas yan...@schoinas.net wrote: I would expect both protocols to saturate the network. What are your performance expectations for cifs? CIFS performance is not limited by network BW or CPU performance (at this BW level). Something else is causing the degradation to 50% of available network BW. Do you have any ideas? What's wrong with dd? What would you use to test sequential write performance? use filebench, it has a cifs plugin I think how are you measuring network bandwidth? both cifs and nfs have some protocol related overhead. Also, you might want to ask cifs related questions in cifs-discuss nacho ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: [osol-discuss] CIFS vs NFS performance
After looking around, it appears it is an inherent limitation of the SMB protocol. It doesn't pipeline requests which means it experiences the full 6 msecs latency introduced by MOCA. SMB2 should fix this when it is available in CIFS. Yannis -- This message posted from opensolaris.org ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
Re: [osol-discuss] CIFS vs NFS performance
You can also try much larger sample sets. Something like iometer allows you to specify queue depth to really push things along as well. -- This message posted from opensolaris.org ___ opensolaris-discuss mailing list opensolaris-discuss@opensolaris.org
