Re: tuning for samba
On Thu, Jul 11, 2002 at 11:20:51AM +0930, Richard Sharpe wrote: On Wed, 10 Jul 2002, Chad David wrote: A local company has been having issues with samba for some time (it kills an e250, and has seriously stressed an e5000) and I've been telling the admin (half seriously) that he should just toss it on a PC with FreeBSD. Well they finally got tired of hearing FreeBSD this and FreeBSD that and asked me to bring a box in if I was so confident... tomorrow morning at 9am. So, I'm building a new box tonight and was wondering if anybody has any tried and true tuning parameters for samba on -stable. They currently have ~700 users attached. The load per user is pretty low but just rebooting and handling the reconnects has killed small boxes. As a follow up to my post a few weeks back... The box is currently easily handling an average of 650 connections with 80% cpu free and lots of memory. The only thing I had to change was kern.ipc.nmbclusters=32768. As others have said, memory is an issue. Memory hasn't actually been a problem. The vast majority is inactive, and the scan rate is pretty close to 0. In some 'benchmark' testing, I have noticed that FreeBSD holds up pretty well to large numbers of connects coming in at one time, say compared to Linux. Starting up 100 clients during about two or three seconds (as long as it takes to fork 100 processes on the driver) does not kill a FreeBSD Samba server as much as it does a Linux server running Linux 2.4.x. All of the load is actually redirected through a firewall, and when we changed the mapping from the e250 over to FreeBSD about 350 clients connected all at once. The cpu dropped to about 50% for 30 seconds and then things settled down. We did see 3 seg faults in smbd during the initial rush, but have been unable to reproduce or to get a core file. As a side note, the data being served will be attached to the samba server via NFS. Hmmm, some of the locking stuff might be an issue then ... We haven't noticed any problems with samba that could be traced to NFS. I have found that the NFS performance is actually worse for bulk copies when nfsiod is running, so we just do not run it. Thanks to everybody for their input. Due to the success of our tests FreeBSD will now be replacing Solaris 9 in this environment. -- Chad David[EMAIL PROTECTED] www.FreeBSD.org [EMAIL PROTECTED] ACNS Inc. Calgary, Alberta Canada To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: tuning for samba
Chad David wrote: So, I'm building a new box tonight and was wondering if anybody has any tried and true tuning parameters for samba on -stable. Since you never got any actual answers to your question, I offer the following. The only samba tuning option I've ever seen make a difference is enabling socket options = TCP_NODELAY. Also, make sure that newreno is turned off on the samba host. As for nfs mount options, I have found that -3cisl works best for me, when the servers are sun, or netapp boxes. HTH, Doug -- We have known freedom's price. We have shown freedom's power. And in this great conflict, ... we will see freedom's victory. - George W. Bush, President of the United States State of the Union, January 28, 2002 Do YOU Yahoo!? To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: tuning for samba
On Sun, 14 Jul 2002, Doug Barton wrote: Chad David wrote: So, I'm building a new box tonight and was wondering if anybody has any tried and true tuning parameters for samba on -stable. Since you never got any actual answers to your question, I offer the following. The only samba tuning option I've ever seen make a difference is enabling socket options = TCP_NODELAY. Also, make sure that newreno Default: socket options = TCP_NODELAY is turned off on the samba host. As for nfs mount options, I have found that -3cisl works best for me, when the servers are sun, or netapp boxes. How does turning off newreno help? We think we are seeing fast retransmit get confused in the presence of dropped packets. Is this possibly related? Regards - Richard Sharpe, [EMAIL PROTECTED], [EMAIL PROTECTED], [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: tuning for samba
Chad David wrote:
A local company has been having issues with samba for some time (it kills
an e250, and has seriously stressed an e5000) and I've been telling the
admin (half seriously) that he should just toss it on a PC with FreeBSD.
Well they finally got tired of hearing FreeBSD this and FreeBSD that and
asked me to bring a box in if I was so confident... tomorrow morning at
9am. So, I'm building a new box tonight and was wondering if anybody
has any tried and true tuning parameters for samba on -stable. They
currently have ~700 users attached. The load per user is pretty low
but just rebooting and handling the reconnects has killed small boxes.
As a side note, the data being served will be attached to the samba server
via NFS.
The one thing I've seen kill a box besides the reboot-reconnect blast
is content searches by the Windows Find dialog. All it takes is one
user on a fast machine and network link doing the Windows equivalent
of find / -name * -exec grep foo \{\} \; to run you out of file
descriptors in a matter of seconds.
Samba uses a seperate process for each connection, and Windows opens
one connection per share. Most Windows users only work on one share
at a time, so with two open shares on ~700 machines that means ~1400
connections with roughly half of them idle. That's a lot of freeable
RAM should you suddenly need it.
To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-hackers in the body of the message
Re: tuning for samba
On Wed, 10 Jul 2002, Darren Pilgrim wrote:
Chad David wrote:
A local company has been having issues with samba for some time (it kills
an e250, and has seriously stressed an e5000) and I've been telling the
admin (half seriously) that he should just toss it on a PC with FreeBSD.
Well they finally got tired of hearing FreeBSD this and FreeBSD that and
asked me to bring a box in if I was so confident... tomorrow morning at
9am. So, I'm building a new box tonight and was wondering if anybody
has any tried and true tuning parameters for samba on -stable. They
currently have ~700 users attached. The load per user is pretty low
but just rebooting and handling the reconnects has killed small boxes.
As a side note, the data being served will be attached to the samba server
via NFS.
The one thing I've seen kill a box besides the reboot-reconnect blast
is content searches by the Windows Find dialog. All it takes is one
user on a fast machine and network link doing the Windows equivalent
of find / -name * -exec grep foo \{\} \; to run you out of file
descriptors in a matter of seconds.
Yes, Samba has to do readdir scans to simulate a case-insensitive file
system on a case-sensitive file system.
Samba uses a seperate process for each connection, and Windows opens
one connection per share.
Yes to the first claim, no to the second. Most definitely not. For a
single client, windows puts all share access (net use, mounting, whatever
you want to call it) over the single TCP connection to the server.
The only time Windows will create a new connection is if you have given
the server multiple NetBIOS names, and you use different NetBIOS names to
access the share. For example, even if the NetBIOS names NB1 and NB2
translate to the same IP (10.10.10.10), if you do the following:
net use f: \\nb1\share1
net use f: \\nb2\share1
the client will establish two different connections. However, that is the
only way I know to get multiple connections from a client to a server.
Even Terminal Server multiplexes multiple users over the one TCP
connection.
Most Windows users only work on one share
at a time, so with two open shares on ~700 machines that means ~1400
connections with roughly half of them idle. That's a lot of freeable
RAM should you suddenly need it.
Nope, ~700 connections!
Regards
-
Richard Sharpe, [EMAIL PROTECTED], [EMAIL PROTECTED],
[EMAIL PROTECTED]
To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-hackers in the body of the message
Re: tuning for samba
Richard Sharpe wrote: On Wed, 10 Jul 2002, Darren Pilgrim wrote: Samba uses a seperate process for each connection, and Windows opens one connection per share. Yes to the first claim, no to the second. Most definitely not. For a single client, windows puts all share access (net use, mounting, whatever you want to call it) over the single TCP connection to the server. You're right, sorry. I had gotten mixed up on the multiple connection issue because of my own configuration that results in one share per connection. Nope, ~700 connections! Even with just one connection per machine, though, you're still going to have a significant amount of swappable memory in idle smbd processes. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: tuning for samba
On Thu, 11 Jul 2002, Darren Pilgrim wrote: Richard Sharpe wrote: On Wed, 10 Jul 2002, Darren Pilgrim wrote: Samba uses a seperate process for each connection, and Windows opens one connection per share. Yes to the first claim, no to the second. Most definitely not. For a single client, windows puts all share access (net use, mounting, whatever you want to call it) over the single TCP connection to the server. You're right, sorry. I had gotten mixed up on the multiple connection issue because of my own configuration that results in one share per connection. Nope, ~700 connections! Even with just one connection per machine, though, you're still going to have a significant amount of swappable memory in idle smbd processes. Yes, I agree. Something that I would like to do more about by making sure that as much as possible is shared. Regards - Richard Sharpe, [EMAIL PROTECTED], [EMAIL PROTECTED], [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: tuning for samba
Richard Sharpe wrote: On Thu, 11 Jul 2002, Darren Pilgrim wrote: Richard Sharpe wrote: On Wed, 10 Jul 2002, Darren Pilgrim wrote: Samba uses a seperate process for each connection, and Windows opens one connection per share. Yes to the first claim, no to the second. Most definitely not. For a single client, windows puts all share access (net use, mounting, whatever you want to call it) over the single TCP connection to the server. You're right, sorry. I had gotten mixed up on the multiple connection issue because of my own configuration that results in one share per connection. Nope, ~700 connections! Even with just one connection per machine, though, you're still going to have a significant amount of swappable memory in idle smbd processes. Yes, I agree. Something that I would like to do more about by making sure that as much as possible is shared. At over 4MB per process (4252K each on my server), I should hope that most of it is already shared. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: tuning for samba
On Thu, Jul 11, 2002 at 12:33:30AM -0700, Darren Pilgrim wrote: Richard Sharpe wrote: On Thu, 11 Jul 2002, Darren Pilgrim wrote: Richard Sharpe wrote: On Wed, 10 Jul 2002, Darren Pilgrim wrote: ... Even with just one connection per machine, though, you're still going to have a significant amount of swappable memory in idle smbd processes. Yes, I agree. Something that I would like to do more about by making sure that as much as possible is shared. At over 4MB per process (4252K each on my server), I should hope that most of it is already shared. With my testing last night, 350 clients each writing used ~700M of cache (with was the data being writen) and only ~100M of active memory. There was only a nominal amount swapped (probably getty and friends), so the number of shared pages is actually quite high with ~2.1M of resident mem showing for each process. If it were otherwise I would have quickly burned the 1G in the test server. The only thing I managed to exhaust was mbuf clusters, and that was on the clients first and finally on the server after a bit. Thanks to everybody for their input and suggestions, and I'll let you know how it works in the wild :). -- Chad David[EMAIL PROTECTED] www.FreeBSD.org [EMAIL PROTECTED] ACNS Inc. Calgary, Alberta Canada To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
tuning for samba
A local company has been having issues with samba for some time (it kills an e250, and has seriously stressed an e5000) and I've been telling the admin (half seriously) that he should just toss it on a PC with FreeBSD. Well they finally got tired of hearing FreeBSD this and FreeBSD that and asked me to bring a box in if I was so confident... tomorrow morning at 9am. So, I'm building a new box tonight and was wondering if anybody has any tried and true tuning parameters for samba on -stable. They currently have ~700 users attached. The load per user is pretty low but just rebooting and handling the reconnects has killed small boxes. As a side note, the data being served will be attached to the samba server via NFS. Thanks. -- Chad David[EMAIL PROTECTED] www.FreeBSD.org [EMAIL PROTECTED] ACNS Inc. Calgary, Alberta Canada To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: tuning for samba
On Thu, 2002-07-11 at 09:37, Chad David wrote: A local company has been having issues with samba for some time (it kills an e250, and has seriously stressed an e5000) and I've been telling the admin (half seriously) that he should just toss it on a PC with FreeBSD. Well they finally got tired of hearing FreeBSD this and FreeBSD that and asked me to bring a box in if I was so confident... tomorrow morning at 9am. So, I'm building a new box tonight and was wondering if anybody has any tried and true tuning parameters for samba on -stable. They currently have ~700 users attached. The load per user is pretty low but just rebooting and handling the reconnects has killed small boxes. As a side note, the data being served will be attached to the samba server via NFS. Thanks. -- Chad David[EMAIL PROTECTED] www.FreeBSD.org [EMAIL PROTECTED] ACNS Inc. Calgary, Alberta Canada To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message -- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au The nice thing about standards is that there are so many of them to choose from. -- Andrew Tanenbaum GPG Fingerprint - 9A8C 569F 685A D928 5140 AE4B 319B 41F4 5D17 FDD5 To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: tuning for samba
On Thu, 2002-07-11 at 09:37, Chad David wrote: A local company has been having issues with samba for some time (it kills an e250, and has seriously stressed an e5000) and I've been telling the admin (half seriously) that he should just toss it on a PC with FreeBSD. Well they finally got tired of hearing FreeBSD this and FreeBSD that and asked me to bring a box in if I was so confident... tomorrow morning at 9am. So, I'm building a new box tonight and was wondering if anybody has any tried and true tuning parameters for samba on -stable. They currently have ~700 users attached. The load per user is pretty low but just rebooting and handling the reconnects has killed small boxes. Heheh.. As a side note, the data being served will be attached to the samba server via NFS. Lots and lots of RAM and mbufs? It is basically going to be doing zero local disk access ya? Got any 1Gb DIMMs handy? :) -- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au The nice thing about standards is that there are so many of them to choose from. -- Andrew Tanenbaum GPG Fingerprint - 9A8C 569F 685A D928 5140 AE4B 319B 41F4 5D17 FDD5 To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: tuning for samba
On Wed, 10 Jul 2002, Chad David wrote: A local company has been having issues with samba for some time (it kills an e250, and has seriously stressed an e5000) and I've been telling the admin (half seriously) that he should just toss it on a PC with FreeBSD. Well they finally got tired of hearing FreeBSD this and FreeBSD that and asked me to bring a box in if I was so confident... tomorrow morning at 9am. So, I'm building a new box tonight and was wondering if anybody has any tried and true tuning parameters for samba on -stable. They currently have ~700 users attached. The load per user is pretty low but just rebooting and handling the reconnects has killed small boxes. As others have said, memory is an issue. In some 'benchmark' testing, I have noticed that FreeBSD holds up pretty well to large numbers of connects coming in at one time, say compared to Linux. Starting up 100 clients during about two or three seconds (as long as it takes to fork 100 processes on the driver) does not kill a FreeBSD Samba server as much as it does a Linux server running Linux 2.4.x. Certainly, a 2GB machine that I regularly test against does not notice the smbds start up all that much. As a side note, the data being served will be attached to the samba server via NFS. Hmmm, some of the locking stuff might be an issue then ... Regards - Richard Sharpe, [EMAIL PROTECTED], [EMAIL PROTECTED], [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: tuning for samba
On Thu, Jul 11, 2002 at 09:45:46AM +0930, Daniel O'Connor wrote: On Thu, 2002-07-11 at 09:37, Chad David wrote: A local company has been having issues with samba for some time (it kills an e250, and has seriously stressed an e5000) and I've been telling the admin (half seriously) that he should just toss it on a PC with FreeBSD. Well they finally got tired of hearing FreeBSD this and FreeBSD that and asked me to bring a box in if I was so confident... tomorrow morning at 9am. So, I'm building a new box tonight and was wondering if anybody has any tried and true tuning parameters for samba on -stable. They currently have ~700 users attached. The load per user is pretty low but just rebooting and handling the reconnects has killed small boxes. Heheh.. As a side note, the data being served will be attached to the samba server via NFS. Lots and lots of RAM and mbufs? It is basically going to be doing zero local disk access ya? Got any 1Gb DIMMs handy? :) So you don't think 32M will do it eh? ;). I have a pretty good handle on the basic issues, what I was hoping for was somebody to step forward and say we do this, and this is what has worked best for us :). -- Chad David[EMAIL PROTECTED] www.FreeBSD.org [EMAIL PROTECTED] ACNS Inc. Calgary, Alberta Canada To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: tuning for samba
On Thu, Jul 11, 2002 at 11:20:51AM +0930, Richard Sharpe wrote: Certainly, a 2GB machine that I regularly test against does not notice the smbds start up all that much. I have no real way of testing this type of load here, but first thing tomorrow morning I'll know.. As a side note, the data being served will be attached to the samba server via NFS. Hmmm, some of the locking stuff might be an issue then ... This is my biggest concern. I just don't know what to tune here since the data just basically passes straight through the box, and the with about of data being served and the access patterns buffering is pointless. One thing I failed to mention, none of the clients ever write; the system is completely read only. Thanks. -- Chad David[EMAIL PROTECTED] www.FreeBSD.org [EMAIL PROTECTED] ACNS Inc. Calgary, Alberta Canada To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: tuning for samba
On Wed, 10 Jul 2002, Chad David wrote: On Thu, Jul 11, 2002 at 11:20:51AM +0930, Richard Sharpe wrote: Certainly, a 2GB machine that I regularly test against does not notice the smbds start up all that much. I have no real way of testing this type of load here, but first thing tomorrow morning I'll know.. Up on samba.org in CVS under cifs-load-gen is a tool that can simulate clients. Simulating the startup of 100's of clients and then watching what happens to the server is not too hard, as long as you have a driver that can withstand the load of that many driver processes starting :-) As a side note, the data being served will be attached to the samba server via NFS. Hmmm, some of the locking stuff might be an issue then ... This is my biggest concern. I just don't know what to tune here since the data just basically passes straight through the box, and the with about of data being served and the access patterns buffering is pointless. One thing I failed to mention, none of the clients ever write; the system is completely read only. Thanks. -- Regards - Richard Sharpe, [EMAIL PROTECTED], [EMAIL PROTECTED], [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: tuning for samba
On Thu, 2002-07-11 at 10:14, Chad David wrote: This is my biggest concern. I just don't know what to tune here since the data just basically passes straight through the box, and the with about of data being served and the access patterns buffering is pointless. I disagree.. Buffering is probably going to help - even just a little. One thing I failed to mention, none of the clients ever write; the system is completely read only. Ahh.. well you can throw any type of 'real' locking away and tell samba to fake it all I guess. You should also look at the acregmin/acregmax/acdirmin/acdirmax options - if the store is static then you could probably increase them quite a lot which would reduce NFS traffic. Also don't forget to run nfsiod. -- Daniel O'Connor software and network engineer for Genesis Software - http://www.gsoft.com.au The nice thing about standards is that there are so many of them to choose from. -- Andrew Tanenbaum GPG Fingerprint - 9A8C 569F 685A D928 5140 AE4B 319B 41F4 5D17 FDD5 To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: tuning for samba
In the last episode (Jul 10), Chad David said: As a side note, the data being served will be attached to the samba server via NFS. Wouldn't it be better to run samba directly on the server that's providing the data? Why force it over the network twice? -- Dan Nelson [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: tuning for samba
On Wed, Jul 10, 2002 at 09:36:10PM -0500, Dan Nelson wrote: In the last episode (Jul 10), Chad David said: As a side note, the data being served will be attached to the samba server via NFS. Wouldn't it be better to run samba directly on the server that's providing the data? They have a huge amount of data (seismic, well log etc.) and getting access to the actual data servers is just not that simple; as well, copying a subset for this test would take days and a lot of disk. -- Chad David[EMAIL PROTECTED] www.FreeBSD.org [EMAIL PROTECTED] ACNS Inc. Calgary, Alberta Canada To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
