Re: [Samba] single stream performance issue, Win2K, WinXP, Samba 3.2.5-4lenny7 (Debian Lenny)

2010-02-02 Thread Stan Hoeppner
Stan Hoeppner put forth on 1/27/2010 4:37 PM:
 Stan Hoeppner put forth on 1/25/2010 5:30 PM:
 Volker Lendecke put forth on 1/25/2010 1:28 AM:
 
 The dual-stream one is kindof limited help. The interesting
 piece is how Win-Win does its thing faster, so we need to
 see that one.

 I've been busting my but trying to get you something meaningful.  This dump 
 is
 less than optimal for two reasons, but it's the best I can get you thus far.

 1.  Running tshark on Win2K creates a huge network performance hit and thus 
 b/w
 numbers for small file (250MB) transfers don't come close to accurately
 describing the real world.  With tshark running the b/w is less than half of
 normal with small files.

 2.  Because of this I had to do a huge file copy to allow time for the 
 client to
 level off at peak performance, which is still ~500KB/s lower than normal due 
 to
 tshark overhead.

 Anyway, the file is over 400MB.  It'll take quite a while to grab off my 
 server.

 http://www.hardwarefreak.com/smb-winwin-single-stream

 Hope you are able to glean something meaningful from it.
 
 Were you able to grab this trace file yet Volker?  If so, have you found
 anything interesting yet when comparing it to the previous Samba-Win2K trace
 file?  Any clues yet as to why the win-win throughput is almost 3MB/s better
 than Samba-Win?  If you haven't dug into it yet, as a reminder, this last 
 trace
 capture was done with tshark on windows.  The previous trace file was captured
 on the Linux machine with tcpdump.

Some additional data points to add to this performance issue.  Using smbclient
3.2.5 on the samba server box, with no tweak options, I ran get/put tests to
each of the Windows workstations.  The results are interesting to say the least.
 The get performance maxes the fast ethernet link at ~11MB/s.  If you recall,
the workstations max out their upload to smbd at ~8MB/s, with the same for
download.  If smbclient on the smbd box can pull from a workstation at 11MB/s,
using the same TCP/IP stack and SO settings smbd should be able to absorb an
upload at 11MB/s because it's the same path.  But it doesn't, it caps the
workstation uploads (and downloads) at ~8MB/s, no matter what options I try.

Windows XP Home machine:
smb: \ get src.exe
getting file \src.exe of size 121983488 as src.exe (11276.5 kb/s) (average
11276.5 kb/s)
smb: \ put src.exe
putting file src.exe as \src.exe (6149.3 kb/s) (average 6149.3 kb/s)

Windows 2000 Pro machine:
smb: \ get src.exe
getting file \src.exe of size 121983488 as src.exe (11195.9 kb/s) (average
11195.9 kb/s)
smb: \ put src.exe
putting file src.exe as \src.exe (6362.5 kb/s) (average 6362.5 kb/s)

-- 
Stan
-- 
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba


Re: [Samba] single stream performance issue, Win2K, WinXP, Samba 3.2.5-4lenny7 (Debian Lenny)

2010-01-27 Thread Stan Hoeppner
Stan Hoeppner put forth on 1/25/2010 5:30 PM:
 Volker Lendecke put forth on 1/25/2010 1:28 AM:

 The dual-stream one is kindof limited help. The interesting
 piece is how Win-Win does its thing faster, so we need to
 see that one.
 
 I've been busting my but trying to get you something meaningful.  This dump is
 less than optimal for two reasons, but it's the best I can get you thus far.
 
 1.  Running tshark on Win2K creates a huge network performance hit and thus 
 b/w
 numbers for small file (250MB) transfers don't come close to accurately
 describing the real world.  With tshark running the b/w is less than half of
 normal with small files.
 
 2.  Because of this I had to do a huge file copy to allow time for the client 
 to
 level off at peak performance, which is still ~500KB/s lower than normal due 
 to
 tshark overhead.
 
 Anyway, the file is over 400MB.  It'll take quite a while to grab off my 
 server.
 
 http://www.hardwarefreak.com/smb-winwin-single-stream
 
 Hope you are able to glean something meaningful from it.

Were you able to grab this trace file yet Volker?  If so, have you found
anything interesting yet when comparing it to the previous Samba-Win2K trace
file?  Any clues yet as to why the win-win throughput is almost 3MB/s better
than Samba-Win?  If you haven't dug into it yet, as a reminder, this last trace
capture was done with tshark on windows.  The previous trace file was captured
on the Linux machine with tcpdump.

-- 
Stan
-- 
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba


Re: [Samba] single stream performance issue, Win2K, WinXP, Samba 3.2.5-4lenny7 (Debian Lenny)

2010-01-25 Thread Stan Hoeppner
Volker Lendecke put forth on 1/25/2010 1:28 AM:
 On Mon, Jan 25, 2010 at 12:14:36AM -0600, Stan Hoeppner wrote:
 Volker Lendecke put forth on 1/24/2010 6:51 AM:
 On Sun, Jan 24, 2010 at 02:09:51PM +0200, Michael Wood wrote:
 Except that he said I can copy files between the Win2K and WinXP
 machines at just over 10MB/s in a single stream and max out the 11MB/s
 with two streams.  I am assuming he used the same client in that test
 as he did with the test against Samba.  So from what he's said it
 seems that he gets more speed with a Windows server than with Samba
 for the same client.

 So what we need is a full network trace of both cases.

 Actually I'll give you something slightly different, and more to the original
 question.  I've taken two tcp captures on the Samba server machine.  Both
 transfers were performed using the Windows 2000 cli copy command pulling a
 36MB avi file from a share on the Samba server.  The first test was a single
 stream copy.  The second test was a dual stream copy of the same file
 concurrently to two different destination directories.  I also had iftop 
 running
 during the tests.  The single stream transfer maxed out at just over 64Mb/s.
 The dual stream test maxed out at 92Mb/s.  Following are the two tcpdump 
 output
 files using tcpdump -p -s 0 -w FILE port 445:

 http://www.hardwarefreak.com/smb_single_stream
 http://www.hardwarefreak.com/smb_dual_stream
 
 The dual-stream one is kindof limited help. The interesting
 piece is how Win-Win does its thing faster, so we need to
 see that one.

I think something is wrong.  I downloaded Wireshark Win32.  When running

tshark -p -w smb-winwin-single-stream port 445

the transfer rate is half what it is without Wireshark running.  What am I doing
wrong?

Thanks.

-- 
Stan
-- 
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba


Re: [Samba] single stream performance issue, Win2K, WinXP, Samba 3.2.5-4lenny7 (Debian Lenny)

2010-01-25 Thread Stan Hoeppner
Stan Hoeppner put forth on 1/25/2010 12:07 PM:
 Volker Lendecke put forth on 1/25/2010 1:28 AM:
 On Mon, Jan 25, 2010 at 12:14:36AM -0600, Stan Hoeppner wrote:
 Volker Lendecke put forth on 1/24/2010 6:51 AM:
 On Sun, Jan 24, 2010 at 02:09:51PM +0200, Michael Wood wrote:
 Except that he said I can copy files between the Win2K and WinXP
 machines at just over 10MB/s in a single stream and max out the 11MB/s
 with two streams.  I am assuming he used the same client in that test
 as he did with the test against Samba.  So from what he's said it
 seems that he gets more speed with a Windows server than with Samba
 for the same client.

 So what we need is a full network trace of both cases.

 Actually I'll give you something slightly different, and more to the 
 original
 question.  I've taken two tcp captures on the Samba server machine.  Both
 transfers were performed using the Windows 2000 cli copy command pulling a
 36MB avi file from a share on the Samba server.  The first test was a single
 stream copy.  The second test was a dual stream copy of the same file
 concurrently to two different destination directories.  I also had iftop 
 running
 during the tests.  The single stream transfer maxed out at just over 64Mb/s.
 The dual stream test maxed out at 92Mb/s.  Following are the two tcpdump 
 output
 files using tcpdump -p -s 0 -w FILE port 445:

 http://www.hardwarefreak.com/smb_single_stream
 http://www.hardwarefreak.com/smb_dual_stream

 The dual-stream one is kindof limited help. The interesting
 piece is how Win-Win does its thing faster, so we need to
 see that one.
 
 I think something is wrong.  I downloaded Wireshark Win32.  When running
 
 tshark -p -w smb-winwin-single-stream port 445
 
 the transfer rate is half what it is without Wireshark running.  What am I 
 doing
 wrong?

This is rather interesting, and disheartening.  I've just spent 30 minutes
playing with tshark and windump.  For small file transfers, the presence of the
capture tools running cuts the network interface performance in half.  If I copy
a 600MB file, the rate gradually increases to 10MB/s but only after about 45
seconds.  Given my limited outbound, I doubt anyone wishes to try to download a
600MB file from my server, nor analyze the contents of such a behemoth.

What Windows capture tool is available that does not itself *cause* a further
performance problem in the act of capturing the data to solve one?  This is a
ridiculous situation.  This machine has a 2GHz AthlonXP CPU, 1GB RAM, and a
120GB 7200RPM IDE disk.  CPU for tshark or windump never exceeds 25%.  Why are
these capture tools doing this?  They've created a catch 22.  I can't report the
data without the capture, but the capture ruins the data.

This is very, very frustrating.  tcpdump on Debian has no such problems, and
that machine is a lowly dual 550 with only 384MB of PC100.  However, it's Linux
instead of Windows, which helps tremendously.  And, it's got an Intel Pro 100
server adapter in it whereas the workstation has an integrated nVidia nForce2
MCP 10/100 motherboard down NIC.

Please help alleviate the frustration here and get me back on the path to
solving this performance issue.

Thanks.

-- 
Stan
-- 
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba


Re: [Samba] single stream performance issue, Win2K, WinXP, Samba 3.2.5-4lenny7 (Debian Lenny)

2010-01-25 Thread Michael Wood
2010/1/25 Stan Hoeppner s...@hardwarefreak.com:
[...]
 This is rather interesting, and disheartening.  I've just spent 30 minutes
 playing with tshark and windump.  For small file transfers, the presence of 
 the
 capture tools running cuts the network interface performance in half.  If I 
 copy
 a 600MB file, the rate gradually increases to 10MB/s but only after about 45
 seconds.  Given my limited outbound, I doubt anyone wishes to try to download 
 a
 600MB file from my server, nor analyze the contents of such a behemoth.

 What Windows capture tool is available that does not itself *cause* a further
 performance problem in the act of capturing the data to solve one?  This is a
 ridiculous situation.  This machine has a 2GHz AthlonXP CPU, 1GB RAM, and a
 120GB 7200RPM IDE disk.  CPU for tshark or windump never exceeds 25%.  Why are
 these capture tools doing this?  They've created a catch 22.  I can't report 
 the
 data without the capture, but the capture ruins the data.
[...]

If you can find a spare box with two NICs in it, you could set up a
Linux box as a bridge (even running from a live CD) and run tcpdump on
that.

Otherwise, maybe this helps:
http://support.microsoft.com/kb/812953

Wireshark seems to be able to load Microsoft NetMon captures, so I
think that should work too and might not cause the performance drop
that tshark/windump (winpcap) do.

-- 
Michael Wood esiot...@gmail.com
-- 
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba


Re: [Samba] single stream performance issue, Win2K, WinXP, Samba 3.2.5-4lenny7 (Debian Lenny)

2010-01-25 Thread Stan Hoeppner
Volker Lendecke put forth on 1/25/2010 1:28 AM:
 On Mon, Jan 25, 2010 at 12:14:36AM -0600, Stan Hoeppner wrote:
 Volker Lendecke put forth on 1/24/2010 6:51 AM:
 On Sun, Jan 24, 2010 at 02:09:51PM +0200, Michael Wood wrote:
 Except that he said I can copy files between the Win2K and WinXP
 machines at just over 10MB/s in a single stream and max out the 11MB/s
 with two streams.  I am assuming he used the same client in that test
 as he did with the test against Samba.  So from what he's said it
 seems that he gets more speed with a Windows server than with Samba
 for the same client.

 So what we need is a full network trace of both cases.

 Actually I'll give you something slightly different, and more to the original
 question.  I've taken two tcp captures on the Samba server machine.  Both
 transfers were performed using the Windows 2000 cli copy command pulling a
 36MB avi file from a share on the Samba server.  The first test was a single
 stream copy.  The second test was a dual stream copy of the same file
 concurrently to two different destination directories.  I also had iftop 
 running
 during the tests.  The single stream transfer maxed out at just over 64Mb/s.
 The dual stream test maxed out at 92Mb/s.  Following are the two tcpdump 
 output
 files using tcpdump -p -s 0 -w FILE port 445:

 http://www.hardwarefreak.com/smb_single_stream
 http://www.hardwarefreak.com/smb_dual_stream
 
 The dual-stream one is kindof limited help. The interesting
 piece is how Win-Win does its thing faster, so we need to
 see that one.

I've been busting my but trying to get you something meaningful.  This dump is
less than optimal for two reasons, but it's the best I can get you thus far.

1.  Running tshark on Win2K creates a huge network performance hit and thus b/w
numbers for small file (250MB) transfers don't come close to accurately
describing the real world.  With tshark running the b/w is less than half of
normal with small files.

2.  Because of this I had to do a huge file copy to allow time for the client to
level off at peak performance, which is still ~500KB/s lower than normal due to
tshark overhead.

Anyway, the file is over 400MB.  It'll take quite a while to grab off my server.

http://www.hardwarefreak.com/smb-winwin-single-stream

Hope you are able to glean something meaningful from it.

-- 
Stan
-- 
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba


Re: [Samba] single stream performance issue, Win2K, WinXP, Samba 3.2.5-4lenny7 (Debian Lenny)

2010-01-25 Thread Linda Walsh

Stan Hoeppner wrote:

For raw bandwidth maximization, what port and protocol are used won't
make much difference, if any.  In fact it shouldn't make _any_
difference in raw b/w.  Communications between the Samba server and
Win2K client appear to be exclusively over TCP 139 at this point
according to netstat, instead I'm misreading or looking in the wrong
place.


--- I haven't read the rest of the thread yet, so forgive me if I am
  covering things that have already been covered.

  139 AFAIK, uses UDP, that means one packet up, it gets ACKED, (packet
send back to sender) then another packet goes up.

  445 uses TCP, which can have multiple packets sent without waiting for
an ACK.  Suppose round trip for an 'empty packet is 2 ms.  For round
numbers use 1000B/packet. So you send 1000B on a 1MB/s line (yeah, it's
an odd flaver of ethernet).  But for each 1000 bytes sent, it takes
1000/10^6(B/s)  = 1ms.  So it would take 2 seconds to send.  Now the
other side could wait for the response to come back and that would take
another 1ms for an empty packet (which can include an 'ACK'.  So round
trip time for 1000 bytes would be 3m.  Now your 1MB line has dropped to
1000B / 3ms.

Instead of nearly 1000 packets/second, you only see a throughput of 
300k on our 1MB line : 33%.  Yuck!


Now tcp doesn't require nearly the overhead for single packets.  Opening
the TCP connection takes extra long -- maybe in our example it would take
5ms.  But then further packets can be sent with .05ms overhead instead of
1ms. (these figures are illustrative, not accurate!) But now you send 30
packets at 1ms+.5 each, and they all travel and are received in 30.30 ms.
The ack back takes another .5 (as it's within the TCP stream, where you
only need send packet# and ack -- no addressing or port or security info.
That 'intro stuff' is only done once at the begining of each stream open
(which in Samba is only once/ session -- not once/connection).
Additonally, the Ack back takes place AS the next packet is being sent.
Most implementations will allow the next one-to-several packets to be
sent WITHOUT having heard back.  That's important. 


So the total wait time -- is 1.5*30 or 45ms+ + the last ack has to waited
for -- so 45.5 ms. to send your 30,000 bytes.  Now we're talking 659k on
our 1MB line.  Not perfect, but maybe as perfect as less than ideal
hardware allows due to overhead (or maybe OS overhead/packet...whatever).
But in this *bogus*, (but representative in a relative sense) example TCP
bought over 100% more throughput.  In real life, might add 10-30%.
Depends on hardward and OS implementation.

Do you see why TCP=better? (for large packet sizes).  For small, sparse
amounts of data, UDP might be better.

The penalty of per-packet overhead RTT times goes *up* with the faster
networking equipment you use.  At 1GB, 1ms is a loss of a million bits!

That make sense?

So a UDP connection is much more inefficient and may show as busy but
some of that is spent constructing/sending headers while other parts are
waiting on ACKS.

-linda

--
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba


Re: [Samba] single stream performance issue, Win2K, WinXP, Samba 3.2.5-4lenny7 (Debian Lenny)

2010-01-25 Thread Michael Wood
Hi

2010/1/26 Linda Walsh sa...@tlinx.org:
 Stan Hoeppner wrote:

 For raw bandwidth maximization, what port and protocol are used won't
 make much difference, if any.  In fact it shouldn't make _any_
 difference in raw b/w.  Communications between the Samba server and
 Win2K client appear to be exclusively over TCP 139 at this point
 according to netstat, instead I'm misreading or looking in the wrong
 place.

 --- I haven't read the rest of the thread yet, so forgive me if I am
  covering things that have already been covered.

  139 AFAIK, uses UDP, that means one packet up, it gets ACKED, (packet
 send back to sender) then another packet goes up.

I'm pretty sure you're wrong about port 139 necessarily using UDP, and
Stan said further up the thread that he was using TCP on port 139.  He
later changed to port 445 anyway and was still having the same
problem.

-- 
Michael Wood esiot...@gmail.com
-- 
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba


Re: [Samba] single stream performance issue, Win2K, WinXP, Samba 3.2.5-4lenny7 (Debian Lenny)

2010-01-24 Thread Volker Lendecke
On Sat, Jan 23, 2010 at 02:11:04PM -0600, Stan Hoeppner wrote:
 The 11MB/s was a different test, which I clearly stated.
 It consisted of two concurrent single stream file copies
 _from_ the Samba server _to_ a Win2K workstation using
 standard Windows Explorer as the file copy program.  This
 test saturated one leg of the 100FDX ethernet connection
 at ~11.5MB/s.

Just a quick hint: Single stream performance really heavily
depends on the concrete client behaviour. smbclient from 3.2
and higher should give you good performance. And watch out
which program on the Windows client you use to do the copy.
xcopy, robocopy and the Windows explorer on some OS version
give dramatically different results. The difference comes
from overlapping requests or their absence.

Volker


signature.asc
Description: Digital signature
-- 
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba

Re: [Samba] single stream performance issue, Win2K, WinXP, Samba 3.2.5-4lenny7 (Debian Lenny)

2010-01-24 Thread Michael Wood
2010/1/24 Volker Lendecke volker.lende...@sernet.de:
 On Sat, Jan 23, 2010 at 02:11:04PM -0600, Stan Hoeppner wrote:
 The 11MB/s was a different test, which I clearly stated.
 It consisted of two concurrent single stream file copies
 _from_ the Samba server _to_ a Win2K workstation using
 standard Windows Explorer as the file copy program.  This
 test saturated one leg of the 100FDX ethernet connection
 at ~11.5MB/s.

 Just a quick hint: Single stream performance really heavily
 depends on the concrete client behaviour. smbclient from 3.2
 and higher should give you good performance. And watch out
 which program on the Windows client you use to do the copy.
 xcopy, robocopy and the Windows explorer on some OS version
 give dramatically different results. The difference comes
 from overlapping requests or their absence.

Except that he said I can copy files between the Win2K and WinXP
machines at just over 10MB/s in a single stream and max out the 11MB/s
with two streams.  I am assuming he used the same client in that test
as he did with the test against Samba.  So from what he's said it
seems that he gets more speed with a Windows server than with Samba
for the same client.

-- 
Michael Wood esiot...@gmail.com
-- 
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba


Re: [Samba] single stream performance issue, Win2K, WinXP, Samba 3.2.5-4lenny7 (Debian Lenny)

2010-01-24 Thread Volker Lendecke
On Sun, Jan 24, 2010 at 02:09:51PM +0200, Michael Wood wrote:
 Except that he said I can copy files between the Win2K and WinXP
 machines at just over 10MB/s in a single stream and max out the 11MB/s
 with two streams.  I am assuming he used the same client in that test
 as he did with the test against Samba.  So from what he's said it
 seems that he gets more speed with a Windows server than with Samba
 for the same client.

So what we need is a full network trace of both cases.

Volker
-- 
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba


Re: [Samba] single stream performance issue, Win2K, WinXP, Samba 3.2.5-4lenny7 (Debian Lenny)

2010-01-24 Thread Stan Hoeppner
Volker Lendecke put forth on 1/24/2010 5:04 AM:
 On Sat, Jan 23, 2010 at 02:11:04PM -0600, Stan Hoeppner wrote:
 The 11MB/s was a different test, which I clearly stated.
 It consisted of two concurrent single stream file copies
 _from_ the Samba server _to_ a Win2K workstation using
 standard Windows Explorer as the file copy program.  This
 test saturated one leg of the 100FDX ethernet connection
 at ~11.5MB/s.
 
 Just a quick hint: Single stream performance really heavily
 depends on the concrete client behaviour. smbclient from 3.2
 and higher should give you good performance. And watch out
 which program on the Windows client you use to do the copy.
 xcopy, robocopy and the Windows explorer on some OS version
 give dramatically different results. The difference comes
 from overlapping requests or their absence.

I just tested xcopy with the previously mentioned 600MB+ file between my Win2K
Pro workstation and the WinXP workstation, single stream, both directions.  I
also did the same xcopy tests between the Win2K Pro workstation and the Samba
server.  The b/w results are identical to the previous Windows Explorer file
copy tests, 8MB/s up/down to Samba, and a little over 10MB/s up/down to the
WinXP machine.

Are overlapping requests the cause of the single stream performance issue here?
 If so, are overlapping requests something that is negotiated in the SMB
protocol between the hosts or is it statically configured, or something compiled
into the client code?  I.e. if overlapping requests is the issue, why do the two
Windows machines seem to do it correctly between themselves, but the
Windows/Samba combination does not?

Is there something I can manually configure on Win2K Pro to overlap requests
to/from Samba?

Thanks for the assistance, insight, and education.

-- 
Stan
-- 
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba


Re: [Samba] single stream performance issue, Win2K, WinXP, Samba 3.2.5-4lenny7 (Debian Lenny)

2010-01-24 Thread Stan Hoeppner
Michael Wood put forth on 1/24/2010 6:09 AM:
 2010/1/24 Volker Lendecke volker.lende...@sernet.de:
 On Sat, Jan 23, 2010 at 02:11:04PM -0600, Stan Hoeppner wrote:
 The 11MB/s was a different test, which I clearly stated.
 It consisted of two concurrent single stream file copies
 _from_ the Samba server _to_ a Win2K workstation using
 standard Windows Explorer as the file copy program.  This
 test saturated one leg of the 100FDX ethernet connection
 at ~11.5MB/s.

 Just a quick hint: Single stream performance really heavily
 depends on the concrete client behaviour. smbclient from 3.2
 and higher should give you good performance. And watch out
 which program on the Windows client you use to do the copy.
 xcopy, robocopy and the Windows explorer on some OS version
 give dramatically different results. The difference comes
 from overlapping requests or their absence.
 
 Except that he said I can copy files between the Win2K and WinXP
 machines at just over 10MB/s in a single stream and max out the 11MB/s
 with two streams.  I am assuming he used the same client in that test
 as he did with the test against Samba.  So from what he's said it
 seems that he gets more speed with a Windows server than with Samba
 for the same client.

This is correct.  Except, just to be clear, the two Windows machines are both
_client_ versions of MS Windows, not Windows Server.  I eliminated my only
remaining Windows server box a short wile ago, replacing it with the Samba
server on much newer faster hardware.  So, the environment consists of two
Windows workstations and one Linux/Samba server (although it serves much more
than just Samba).

Almost all of my testing has been performed at the console of the Windows 2000
Professional workstation.  The two remote test systems are Samba 3.2.5 on Debian
Lenny and a Windows XP Home machine.  I'm sure when you said Windows server
you meant server in the SMB client/server relationship sense, but just in case I
thought I'd clarify exactly what systems are involved to prevent possible
confusion.  So, ironically, these two Windows clients serve single stream SMB to
one another faster than either of them to/from Samba.

I'm glad more people are getting involved in this thread because it's making me
look at things I previously had not (though should have).  I just checked and
found that the two Windows clients are talking to each other over TCP 445 which
is SMB over TCP/IP without NETBIOS.  Until just a few minutes ago, my Windows
2000 Pro machine had NETBIOS over TCP/IP enabled, and was only talking to the
Samba server over TCP 139.  Thinking this might make a slight difference, and to
make my testing as close to apples-apples as I can, I disabled NETBIOS over
TCP/IP, and now the Win2K workstation is connecting to Samba over only TCP 445.
 However, I ran the file transfer tests again, and there was no change in single
stream performance--still 8MB/s up and down.

I hope that whatever the cause of this 3MB/s performance loss is that the
solution is user configurable, in the Windows registry or in smb.conf.  I've
been trying to wean myself off Windows for years but so far only for server
duties.  I can't stand the thought that two Windows workstations will be faster
xferring files between each other than to my sleek Debian Samba file server.
I've spent hundreds of hours tweaking the performance of my Debian server, and
it's fast as greased lightning for most everything except single stream Samba
performance.  I really want to remedy this situation. ;)

-- 
Stan
-- 
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba


Re: [Samba] single stream performance issue, Win2K, WinXP, Samba 3.2.5-4lenny7 (Debian Lenny)

2010-01-24 Thread Stan Hoeppner
Volker Lendecke put forth on 1/24/2010 6:51 AM:
 On Sun, Jan 24, 2010 at 02:09:51PM +0200, Michael Wood wrote:
 Except that he said I can copy files between the Win2K and WinXP
 machines at just over 10MB/s in a single stream and max out the 11MB/s
 with two streams.  I am assuming he used the same client in that test
 as he did with the test against Samba.  So from what he's said it
 seems that he gets more speed with a Windows server than with Samba
 for the same client.
 
 So what we need is a full network trace of both cases.

Exactly how would I perform such a task?  With what utilities?  Do you mean
something like tcpdump on the Linux side?  I'm not familiar with a Windows tool
for the same.  Can the Windows network monitor do this?  I've never really used 
it.

-- 
Stan
-- 
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba


Re: [Samba] single stream performance issue, Win2K, WinXP, Samba 3.2.5-4lenny7 (Debian Lenny)

2010-01-24 Thread Michael Wood
2010/1/24 Stan Hoeppner s...@hardwarefreak.com:
 Michael Wood put forth on 1/24/2010 6:09 AM:
[...]
 Except that he said I can copy files between the Win2K and WinXP
 machines at just over 10MB/s in a single stream and max out the 11MB/s
 with two streams.  I am assuming he used the same client in that test
 as he did with the test against Samba.  So from what he's said it
 seems that he gets more speed with a Windows server than with Samba
 for the same client.

 This is correct.  Except, just to be clear, the two Windows machines are both
 _client_ versions of MS Windows, not Windows Server.  I eliminated my only

I meant server in the sense that you connect to it from a client
rather in the Windows Server 200x sense.

 remaining Windows server box a short wile ago, replacing it with the Samba
 server on much newer faster hardware.  So, the environment consists of two
 Windows workstations and one Linux/Samba server (although it serves much more
 than just Samba).

 Almost all of my testing has been performed at the console of the Windows 2000
 Professional workstation.  The two remote test systems are Samba 3.2.5 on 
 Debian
 Lenny and a Windows XP Home machine.  I'm sure when you said Windows server
 you meant server in the SMB client/server relationship sense, but just in 
 case I

Yes, that's what I meant :)

 thought I'd clarify exactly what systems are involved to prevent possible
 confusion.  So, ironically, these two Windows clients serve single stream SMB 
 to
 one another faster than either of them to/from Samba.
[...]

-- 
Michael Wood esiot...@gmail.com
-- 
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba


Re: [Samba] single stream performance issue, Win2K, WinXP, Samba 3.2.5-4lenny7 (Debian Lenny)

2010-01-24 Thread Michael Wood
2010/1/24 Stan Hoeppner s...@hardwarefreak.com:
 Volker Lendecke put forth on 1/24/2010 6:51 AM:
 On Sun, Jan 24, 2010 at 02:09:51PM +0200, Michael Wood wrote:
 Except that he said I can copy files between the Win2K and WinXP
 machines at just over 10MB/s in a single stream and max out the 11MB/s
 with two streams.  I am assuming he used the same client in that test
 as he did with the test against Samba.  So from what he's said it
 seems that he gets more speed with a Windows server than with Samba
 for the same client.

 So what we need is a full network trace of both cases.

 Exactly how would I perform such a task?  With what utilities?  Do you mean
 something like tcpdump on the Linux side?  I'm not familiar with a Windows 
 tool
 for the same.  Can the Windows network monitor do this?  I've never really 
 used it.

Start here:
http://wiki.samba.org/index.php/Capture_Packets

wireshark runs on Windows too or you can use windump which is the
equivalent of tcpdump.

-- 
Michael Wood esiot...@gmail.com
-- 
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba


Re: [Samba] single stream performance issue, Win2K, WinXP, Samba 3.2.5-4lenny7 (Debian Lenny)

2010-01-24 Thread Stan Hoeppner
Volker Lendecke put forth on 1/24/2010 6:51 AM:
 On Sun, Jan 24, 2010 at 02:09:51PM +0200, Michael Wood wrote:
 Except that he said I can copy files between the Win2K and WinXP
 machines at just over 10MB/s in a single stream and max out the 11MB/s
 with two streams.  I am assuming he used the same client in that test
 as he did with the test against Samba.  So from what he's said it
 seems that he gets more speed with a Windows server than with Samba
 for the same client.
 
 So what we need is a full network trace of both cases.

Actually I'll give you something slightly different, and more to the original
question.  I've taken two tcp captures on the Samba server machine.  Both
transfers were performed using the Windows 2000 cli copy command pulling a
36MB avi file from a share on the Samba server.  The first test was a single
stream copy.  The second test was a dual stream copy of the same file
concurrently to two different destination directories.  I also had iftop running
during the tests.  The single stream transfer maxed out at just over 64Mb/s.
The dual stream test maxed out at 92Mb/s.  Following are the two tcpdump output
files using tcpdump -p -s 0 -w FILE port 445:

http://www.hardwarefreak.com/smb_single_stream
http://www.hardwarefreak.com/smb_dual_stream

The file sizes are 38MB and 76MB respectively.  The raw outbound link speed
behind which my web server sits is only 512Kb/s so it'll take a few minutes to
pull the files, probably about 30-35 minutes or so for both files.  My apologies
for any inconvenience this may cause.

-- 
Stan
-- 
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba


Re: [Samba] single stream performance issue, Win2K, WinXP, Samba 3.2.5-4lenny7 (Debian Lenny)

2010-01-24 Thread Volker Lendecke
On Mon, Jan 25, 2010 at 12:14:36AM -0600, Stan Hoeppner wrote:
 Volker Lendecke put forth on 1/24/2010 6:51 AM:
  On Sun, Jan 24, 2010 at 02:09:51PM +0200, Michael Wood wrote:
  Except that he said I can copy files between the Win2K and WinXP
  machines at just over 10MB/s in a single stream and max out the 11MB/s
  with two streams.  I am assuming he used the same client in that test
  as he did with the test against Samba.  So from what he's said it
  seems that he gets more speed with a Windows server than with Samba
  for the same client.
  
  So what we need is a full network trace of both cases.
 
 Actually I'll give you something slightly different, and more to the original
 question.  I've taken two tcp captures on the Samba server machine.  Both
 transfers were performed using the Windows 2000 cli copy command pulling a
 36MB avi file from a share on the Samba server.  The first test was a single
 stream copy.  The second test was a dual stream copy of the same file
 concurrently to two different destination directories.  I also had iftop 
 running
 during the tests.  The single stream transfer maxed out at just over 64Mb/s.
 The dual stream test maxed out at 92Mb/s.  Following are the two tcpdump 
 output
 files using tcpdump -p -s 0 -w FILE port 445:
 
 http://www.hardwarefreak.com/smb_single_stream
 http://www.hardwarefreak.com/smb_dual_stream

The dual-stream one is kindof limited help. The interesting
piece is how Win-Win does its thing faster, so we need to
see that one.

Volker


signature.asc
Description: Digital signature
-- 
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba

Re: [Samba] single stream performance issue, Win2K, WinXP, Samba 3.2.5-4lenny7 (Debian Lenny)

2010-01-23 Thread Linda Walsh

Igor wrote:

 I don't find it strange at all. Your computer is acting as a traffic
 proxy between two samba servers. If you have 100Mb network interface
 your bandwidth should split exactly in two.
 
	But he said he doesn't get a split in two when a win2k server

is used (he gets 11Mbps).I.e. Two network streams in two different
directions should NOT halve throughput, _unless_ something is operating
in half-duplex mode.   100Mbps, full duplex should, _easily_,
allow two 8 MBps streams if they are going in opposite directions.  



Stan wrote:

Interestingly, if I launch a file copy with the SH source file being
on one smb share on the server, and the destination being SH another
smb share (separate filesystem) on the server, the combined throughput
SH is also 8MB/s, 4 up and 4 down, which is very strange as this
should be two SH distinct streams.

---
	I agree.  Is it possible your network device isn't running 
in FULL duplex?   


Other things to check (to optimize speed compared to ftp):

1) Ensure your communications are using TCP (port 445) and not
UDP (port 139).

	2) Ensure encryption (Sealing) is off.  

	3) Ensure packet Signing is off. 


The overhead of 2  3 contribute to around a 15% performance hit according
to 1 MS source.  (Obviously turning such things off presumes you are on
a 'safe' network consistent with FTP usage, vs. SCP/SSH).

	You need to make sure that, at least, one side has each of 
Sign and Seal turned off and the other side has it set to 'no' or 'auto'.
If one side has 'require' set for the feature, and the other has the 
same feature turned off, it will prohibit communications.


Linda
(who's been bummed by the huge drop in networking and disk performance
in windows 7).

--
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba


Re: [Samba] single stream performance issue, Win2K, WinXP, Samba 3.2.5-4lenny7 (Debian Lenny)

2010-01-23 Thread Learner Study
Hi Linda:

Looking at some internet resources, it appears that both encryption
and packet signing are off by default. Can u pls let me know how to
disable these on samba server side (on 3.0.x)

Thanks.

On Sat, Jan 23, 2010 at 12:48 AM, Linda Walsh sa...@tlinx.org wrote:
 Igor wrote:

  I don't find it strange at all. Your computer is acting as a traffic
  proxy between two samba servers. If you have 100Mb network interface
  your bandwidth should split exactly in two.

 But he said he doesn't get a split in two when a win2k server
 is used (he gets 11Mbps).I.e. Two network streams in two different
 directions should NOT halve throughput, _unless_ something is operating
 in half-duplex mode.   100Mbps, full duplex should, _easily_,
 allow two 8 MBps streams if they are going in opposite directions.

 Stan wrote:

 Interestingly, if I launch a file copy with the SH source file being
 on one smb share on the server, and the destination being SH another
 smb share (separate filesystem) on the server, the combined throughput
 SH is also 8MB/s, 4 up and 4 down, which is very strange as this
 should be two SH distinct streams.

 ---
I agree.  Is it possible your network device isn't running in FULL
 duplex?
Other things to check (to optimize speed compared to ftp):

1) Ensure your communications are using TCP (port 445) and not
 UDP (port 139).

2) Ensure encryption (Sealing) is off.
3) Ensure packet Signing is off.
 The overhead of 2  3 contribute to around a 15% performance hit according
 to 1 MS source.  (Obviously turning such things off presumes you are on
 a 'safe' network consistent with FTP usage, vs. SCP/SSH).

You need to make sure that, at least, one side has each of Sign and
 Seal turned off and the other side has it set to 'no' or 'auto'.
 If one side has 'require' set for the feature, and the other has the same
 feature turned off, it will prohibit communications.

 Linda
 (who's been bummed by the huge drop in networking and disk performance
 in windows 7).

 --
 To unsubscribe from this list go to the following URL and read the
 instructions:  https://lists.samba.org/mailman/options/samba

-- 
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba


Re: [Samba] single stream performance issue, Win2K, WinXP, Samba 3.2.5-4lenny7 (Debian Lenny)

2010-01-23 Thread Stan Hoeppner
Learner Study put forth on 1/23/2010 3:31 AM:
 Hi Linda:
 
 Looking at some internet resources, it appears that both encryption
 and packet signing are off by default. Can u pls let me know how to
 disable these on samba server side (on 3.0.x)

Pretty sure they are both off in my case.  I did not enable them in smb.conf.

 On Sat, Jan 23, 2010 at 12:48 AM, Linda Walsh sa...@tlinx.org wrote:
 Igor wrote:

  I don't find it strange at all. Your computer is acting as a traffic
  proxy between two samba servers. If you have 100Mb network interface
  your bandwidth should split exactly in two.

 But he said he doesn't get a split in two when a win2k server
 is used (he gets 11Mbps).I.e. Two network streams in two different
 directions should NOT halve throughput, _unless_ something is operating
 in half-duplex mode.   100Mbps, full duplex should, _easily_,
 allow two 8 MBps streams if they are going in opposite directions.

The 11MB/s was a different test, which I clearly stated.   It consisted of two
concurrent single stream file copies _from_ the Samba server _to_ a Win2K
workstation using standard Windows Explorer as the file copy program.  This test
saturated one leg of the 100FDX ethernet connection at ~11.5MB/s.

 Stan wrote:

 Interestingly, if I launch a file copy with the SH source file being
 on one smb share on the server, and the destination being SH another
 smb share (separate filesystem) on the server, the combined throughput
 SH is also 8MB/s, 4 up and 4 down, which is very strange as this
 should be two SH distinct streams.

 ---
I agree.  Is it possible your network device isn't running in FULL
 duplex?

Absolutely not.  Both interfaces (Samba server and Win2K workstation) are
configured and confirmed to be operating in full duplex mode.  I confirmed this
by forcing the Win2k box to 100FDX.  This broke the switch which wants full
autonegotiation, forcing the link to half duplex.  It dropped performance by
over 60%.  I reenabled full autonegotiation, and performed a test which I had
not previously.  I launched two copy operations of the same ~600MB file, one up,
one down, and according to NetMeter, was running ~7.5MB/s up to the Samba server
and 6.5MB/s down to the workstation.  Combined this is 14MB/s, more than a HDX
link can provide.  I'm ashamed I can't get that close to the ideal 22MB/s.
There are two possibilities for this that I can think of:

1.  The two switches involved are soho/consumer class, both unmanaged.  One of
the two is a $10 Rosewill (no name Chinese) 8 port desktop jobby.  The other is
a circa 2003/4 SMC rack mount combo 8port 100FDX switch and firewall router,
which is a much better piece of gear, ran me about $100 in '03, but still pretty
low end.  The cost/quality of these two may or may not be a factor, but at this
point I assume it might be.

2.  The machines themselves may not be up to it, although I would think they
should be given their specs, and that we're talking about merely 100FDX with a
theoretical max of 12.5MB/s.

This is something I'll investigate further, after I figure out the single stream
problem.

Other things to check (to optimize speed compared to ftp):

1) Ensure your communications are using TCP (port 445) and not
 UDP (port 139).

For raw bandwidth maximization, what port and protocol are used won't make much
difference, if any.  In fact it shouldn't make _any_ difference in raw b/w.
Communications between the Samba server and Win2K client appear to be
exclusively over TCP 139 at this point according to netstat, instead I'm
misreading or looking in the wrong place.

Proto Recv-Q Send-Q Local Address   Foreign Address State
tcp0  0 0.0.0.0:139 0.0.0.0:*   LISTEN
tcp0  0 0.0.0.0:445 0.0.0.0:*   LISTEN
tcp0  0 192.168.100.9:139   192.168.100.53:1128 ESTABLISHED
udp0  0 192.168.100.9:137   0.0.0.0:*
udp0  0 0.0.0.0:137 0.0.0.0:*
udp0  0 192.168.100.9:138   0.0.0.0:*
udp0  0 0.0.0.0:138 0.0.0.0:*

2) Ensure encryption (Sealing) is off.
3) Ensure packet Signing is off.

I assume these are off by default.  I didn't enable them in smb.conf.

 The overhead of 2  3 contribute to around a 15% performance hit according
 to 1 MS source.  (Obviously turning such things off presumes you are on
 a 'safe' network consistent with FTP usage, vs. SCP/SSH).

The network is private, thus safe in this context.  I'm pretty sure both of my
measuring tools are reporting raw bandwidth, iftop on Linux and NetMeter on
Windows 2000, so even if there is SMB overhead in the mix, it's irrelevant at
this point.  My problem is I can't max out single stream _raw_ bandwidth to/from
the Samba server.  I'm only getting 65Mb/s raw with a single file copy.  I get
92Mb/s with two concurrent operations going the same direction, same as with 
FTP.

  

Re: [Samba] single stream performance issue, Win2K, WinXP, Samba 3.2.5-4lenny7 (Debian Lenny)

2010-01-23 Thread Stan Hoeppner
Stan Hoeppner put forth on 1/23/2010 2:11 PM:

 Absolutely not.  Both interfaces (Samba server and Win2K workstation) are
 configured and confirmed to be operating in full duplex mode.  I confirmed 
 this
 by forcing the Win2k box to 100FDX.  This broke the switch which wants full
 autonegotiation, forcing the link to half duplex.  It dropped performance by
 over 60%.  I reenabled full autonegotiation, and performed a test which I had
 not previously.  I launched two copy operations of the same ~600MB file, one 
 up,
 one down, and according to NetMeter, was running ~7.5MB/s up to the Samba 
 server
 and 6.5MB/s down to the workstation.  Combined this is 14MB/s, more than a HDX
 link can provide.  I'm ashamed I can't get that close to the ideal 22MB/s.
 There are two possibilities for this that I can think of:

I just did some additional testing to see how FTP would perform with full duplex
put/get to/from the xfs filesystem backing the Samba share for comparison to
Samba performance.  I used two sessions of the Windows 2000 inbuilt FTP command
line client with default settings.  I used the same 600MB+ file up/down, two
copies, different names.

full duplex get/put:
get ftp: 678624350 bytes received in 65.92Seconds 10294.35Kbytes/sec.
put ftp: 678624350 bytes sent in 89.88Seconds 7550.76Kbytes/sec.

one way get:
get ftp: 678624350 bytes received in 57.84Seconds 11731.97Kbytes/sec.

The up/down is very uneven with the concurrent ftp transfers, downloads
receiving more b/w than up.  ProFTPD doesn't balance or shape traffic by
default.  I looked into using mod_shaper but it's not compiled into the Debian
ProFTP daemon, and frankly I have no use for the traffic shaper beyond this
testing.  The headache of installing it or another ftp daemon which does shaping
is more hassle than it's worth at this point.

Anyway, as you can see from the numbers above, one transfer ran considerably
longer than the other as it received less b/w during concurrency, about 25
seconds, which artificially inflates its transfer rate a bit as it gets all
11MB/s of the b/w for the remainder of its transfer after the other transfer has
completed.  With that caveat mentioned, I'll point out that watching the output
of NetMeter during concurrency clearly showed a combined total of 16MB/s
up+down.  I ran the same test, same 600MB+ file, and traffic to/from Samba
averages only 13MB/s.  Remember, I'm reporting raw numbers, so protocol overhead
is irrelevant.

This testing demonstrates that due to one or multiple factors, hardware and/or
software, my combined maximum full duplex throughput between my Win2K
workstation and the Debian Lenny Samba server is approximately 16MB/s with my
fastest applications tested to date, those being FTP client and server.  Best
one-way performance achieved to date is the maximum for switched fast ethernet,
just over 11MB/s, obtained with FTP.

Full duplex performance maximization would be great, but frankly it doesn't
concern me as it is not one of my needs.  What is a need is maximizing one-way
single stream performance between the Samba server and my workstation.

So, again, what can I do to bring my single stream raw transfer Win2K-Debian
Samba server performance up near the level of my FTP performance.  FTP
performance peaks at 11MB/s with a single stream yet SMB performance peaks at
only 8MB/s single stream.  Again, this is measuring raw packet performance on
the interface, so protocol overhead is already in the numbers.  Samba 3.2.5 or
Win2K, or the combination of the two, simply will not saturate the interface
with a single stream.  With two streams going the same direction, they _do_
saturate the interface at 11MB/s.

What's the solution to getting that last 3MB/s out of a single stream?  Or, put
a better way, that last 30% that's being left on the table.  From other posts
I've read here, people using GigE are also seeing something similar.  They can't
get that last 30% or so into the interface with a single stream.  I don't think
this problem is merely affecting me on Fast Ethernet.  I guess I really needed
the extra performance I could go buy some GigE cards (if they make them in
regular PCI) and a GigE switch.  However, I'd really just like to maximize what
I already have, and not go spending money I don't want to. ;)

-- 
Stan
-- 
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba


[Samba] single stream performance issue, Win2K, WinXP, Samba 3.2.5-4lenny7 (Debian Lenny)

2010-01-21 Thread Stan Hoeppner
Hello fellow Samba users and devs.  This is my first post.  I've searched
documentation far and wide for Windows, Linux, and Samba, and have not been able
to shed any light on this issue.

I can't get more than 8MB/s during a single file copy stream out of my Samba
server over my 100FDX switched network either from Win2K or WinXP (I don't have
a *nix client to test with).  The network is idle during testing.  Via FTP on
these Win machines to/from the same filesystem (100GB XFS) as the Samba share I
consistently get just a shade over 11MB/s.  However, if I launch two
simultaneous file copy streams from Windows Explorer or from the command line, I
hit the 11MB/s I see via FTP.  Interestingly, if I launch a file copy with the
source file being on one smb share on the server, and the destination being
another smb share (separate filesystem) on the server, the combined throughput
is also 8MB/s, 4 up and 4 down, which is very strange as this should be two
distinct streams.  I can copy files between the Win2K and WinXP machines at just
over 10MB/s in a single stream and max out the 11MB/s with two streams.

I've tweaked every relevant Windows registry setting I can identify, and I've
tried all combination of the following smb.conf settings with various buffer 
sizes

max xmit = 65535
socket options = TCP_NODELAY
socket options = SO_SNDBUF=262144 SO_RCVBUF=262144
socket options = IPTOS_LOWDELAY

and none of these tweaks make a difference.  Still only 8MB/s with a single 
stream.

I've eliminated the network hardware and any CPU/mem/disk bottlenecks on the
server and workstations as possible causes.  The machines are all much more
powerful than the minimums required to fully saturate a 100FDX network.

I don't know if the problem lies with the Windows clients or with smbd.  The one
thing that is certain is that this is a single stream performance issue.
Launching multiple copy streams maxes the network just as FTP does.  Why is
3MB/s of free bandwidth being left on the table for single stream operations to
from smbd?

Any/all hints comments are welcome.  I've burned many hours on trying to figure
this out to no avail, and if I had any hair I'm sure I'd have pulled much of it
out troubleshooting this. ;)  I'd really like to max out that single stream
performance.

Thanks.

-- 
Stan
-- 
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba


Re: [Samba] single stream performance issue, Win2K, WinXP, Samba 3.2.5-4lenny7 (Debian Lenny)

2010-01-21 Thread Igor
Hello Stan,

Friday, January 22, 2010, 2:26:41 AM, you wrote:

 I don't find it strange at all. Your computer is acting as a traffic
 proxy between two samba servers. If you have 100Mb network interface
 your bandwidth should split exactly in two.

 FTP is a different protocol. You might find the answer if you look at
 the percentage the carrying protocol like SAMBA consumes out of the
 traffic to support protocol integrity. You may find out that the rest 2Mb
 (you should actually have no more than 10Mb/s at 100Mb interface) are
 used by the carrying protocol itself. That somehow though the protocol
 itself allows but for the sake of connectivity the maximum size of
 the packet is set to 64К. Which is not surprising as far as Microsoft goes.

 I'm sure people around here might provide you with some data
 about SAMBA efficiency, but just remembering what a big difference
 1000Mb Ethernet produces over 100Mb as far as SAMBA is concerned -
 well my best guess it's not more that 80% of all traffic. 20% goes
 for protocol support.

SH hit the 11MB/s I see via FTP.  Interestingly, if I launch a file copy with 
the
SH source file being on one smb share on the server, and the destination being
SH another smb share (separate filesystem) on the server, the combined 
throughput
SH is also 8MB/s, 4 up and 4 down, which is very strange as this should be two
SH distinct streams.  I can copy files between the Win2K and WinXP machines at 
just
SH over 10MB/s in a single stream and max out the 11MB/s with two streams.


-- 
www.rol.ru
Best regards,
 Igormailto:sp...@online.ru

-- 
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba

Re: [Samba] single stream performance issue, Win2K, WinXP, Samba 3.2.5-4lenny7 (Debian Lenny)

2010-01-21 Thread Igor
Hello Stan,

Friday, January 22, 2010, 2:26:41 AM, you wrote:


Check it out, I found it with google:

http://oreilly.com/catalog/samba/chapter/book/appb.pdf

You see out of the box there is about 20% difference between SMB and
FTP performance which corresponds with your experience.



SH Hello fellow Samba users and devs.  This is my first post.  I've searched
SH documentation far and wide for Windows, Linux, and Samba, and have not been 
able
SH to shed any light on this issue.

SH I can't get more than 8MB/s during a single file copy stream out of my Samba
SH server over my 100FDX switched network either from Win2K or WinXP (I don't 
have
SH a *nix client to test with).  The network is idle during testing.  Via FTP 
on
SH these Win machines to/from the same filesystem (100GB XFS) as the Samba 
share I
SH consistently get just a shade over 11MB/s.  However, if I launch two
SH simultaneous file copy streams from Windows Explorer or from the command 
line, I
SH hit the 11MB/s I see via FTP.  Interestingly, if I launch a file copy with 
the
SH source file being on one smb share on the server, and the destination being
SH another smb share (separate filesystem) on the server, the combined 
throughput
SH is also 8MB/s, 4 up and 4 down, which is very strange as this should be two
SH distinct streams.  I can copy files between the Win2K and WinXP machines at 
just
SH over 10MB/s in a single stream and max out the 11MB/s with two streams.

SH I've tweaked every relevant Windows registry setting I can identify, and 
I've
SH tried all combination of the following smb.conf settings with various 
buffer sizes

SH max xmit = 65535
SH socket options = TCP_NODELAY
SH socket options = SO_SNDBUF=262144 SO_RCVBUF=262144
SH socket options = IPTOS_LOWDELAY

SH and none of these tweaks make a difference.  Still only 8MB/s with a single 
stream.

SH I've eliminated the network hardware and any CPU/mem/disk bottlenecks on the
SH server and workstations as possible causes.  The machines are all much more
SH powerful than the minimums required to fully saturate a 100FDX network.

SH I don't know if the problem lies with the Windows clients or with smbd.  
The one
SH thing that is certain is that this is a single stream performance issue.
SH Launching multiple copy streams maxes the network just as FTP does.  Why is
SH 3MB/s of free bandwidth being left on the table for single stream 
operations to
SH from smbd?

SH Any/all hints comments are welcome.  I've burned many hours on trying to 
figure
SH this out to no avail, and if I had any hair I'm sure I'd have pulled much 
of it
SH out troubleshooting this. ;)  I'd really like to max out that single stream
SH performance.

SH Thanks.

SH -- 
SH Stan



-- 
www.rol.ru
Best regards,
 Igormailto:sp...@online.ru

-- 
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba


Re: [Samba] single stream performance issue, Win2K, WinXP, Samba 3.2.5-4lenny7 (Debian Lenny)

2010-01-21 Thread Stan Hoeppner
Igor put forth on 1/21/2010 6:04 PM:
 Hello Stan,

Hello Igor,

  I don't find it strange at all. Your computer is acting as a traffic
  proxy between two samba servers. If you have 100Mb network interface
  your bandwidth should split exactly in two.

Which should be 5.5MB/s instead of 4MB/s for each stream.  I included this
example for comparison, as an additional data point.  It is not the thrust of my
post, merely supporting data, as this should in theory be _two_ streams.

  FTP is a different protocol. You might find the answer if you look at
  the percentage the carrying protocol like SAMBA consumes out of the
  traffic to support protocol integrity. You may find out that the rest 2Mb
  (you should actually have no more than 10Mb/s at 100Mb interface) are
  used by the carrying protocol itself. That somehow though the protocol
  itself allows but for the sake of connectivity the maximum size of
  the packet is set to 64К. Which is not surprising as far as Microsoft goes.

I don't think you read my post thoroughly, or possibly I didn't state my case
thoroughly or eloquently.  _Two_ concurrent Windows file copy streams to/from
the Samba share peak iftop at 92Mb/s.  A single FTP get/put to the same share
filesystem peaks iftop at 92Mb/s.  A _single_ Windows file copy peaks iftop at
65Mb/s.  I've performed these tests over 10 times with the same results.

Why will _one_ Windows file copy stream to/from the Samba share not peak the
interface?  Why do _two_ concurrent streams peak the interface but not _one_?

  I'm sure people around here might provide you with some data
  about SAMBA efficiency, but just remembering what a big difference
  1000Mb Ethernet produces over 100Mb as far as SAMBA is concerned -
  well my best guess it's not more that 80% of all traffic. 20% goes
  for protocol support.

Protocol efficiency doesn't even come into play here AFAICT.  A single stream
should easily max a 100Mbit pipe.  I'm only able to max the pipe using two or
more concurrent streams from the same host.  One stream won't do it.  I don't
know if the problem is with Windows or with Samba, but I'd sure like to find out
and fix it.  My testing has eliminated both operating systems, their storage,
and the raw network performance as degrading factors.  The only thing left if
the SMB/CFS engines on both systems, which seem to have no trouble at all
clogging the pipe with multiple streams, but run along lazily at 65% of peak
when only one stream is present.

-- 
Stan
-- 
To unsubscribe from this list go to the following URL and read the
instructions:  https://lists.samba.org/mailman/options/samba