Re: Obtaining 75k (active) concurrent tcp sessions..
On Fri, 13 Feb 2004 16:48:59 -0600 Matt Freitag <[EMAIL PROTECTED]> wrote: > No, > kern.ipc.nmbclusters is read-only. > > Setting this in /etc/sysctl.conf is futile, it'll never actually be > set in the kernel this way, the change will just get a read-only > error, just like you would once the machine is booted. As long as I > can remember it's been like this. > > sysctl: oid 'kern.ipc.nmbclusters' is a read only tunable > sysctl: Tunable values are set in /boot/loader.conf > hehe, yeah, just checkted, was getting it confused with another... ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Obtaining 75k (active) concurrent tcp sessions..
No, kern.ipc.nmbclusters is read-only. Setting this in /etc/sysctl.conf is futile, it'll never actually be set in the kernel this way, the change will just get a read-only error, just like you would once the machine is booted. As long as I can remember it's been like this. sysctl: oid 'kern.ipc.nmbclusters' is a read only tunable sysctl: Tunable values are set in /boot/loader.conf -mpf --- Only dimly aware of a certain unease in the air --- Vulpes Velox wrote: On Fri, 13 Feb 2004 14:20:43 -0600 Matt Freitag <[EMAIL PROTECTED]> wrote: For FreeBSD to support that many concurrent connections some kernel values must be tweaked. Namely, you'll need to set "kern.ipc.nmbclusters=81920" in loader.conf as it's a read-only oid. Is this something that has changed since 4x? AFAIK it should be settable in /etc/sysctl.conf... atleast it is on 4x, not sure about 5x ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]" ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Obtaining 75k (active) concurrent tcp sessions..
On Fri, 13 Feb 2004 14:20:43 -0600 Matt Freitag <[EMAIL PROTECTED]> wrote: > For FreeBSD to support that many concurrent connections some kernel > values must be tweaked. Namely, you'll need to set > "kern.ipc.nmbclusters=81920" in loader.conf as it's a read-only oid. Is this something that has changed since 4x? AFAIK it should be settable in /etc/sysctl.conf... atleast it is on 4x, not sure about 5x ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Obtaining 75k (active) concurrent tcp sessions..
For FreeBSD to support that many concurrent connections some kernel values must be tweaked. Namely, you'll need to set "kern.ipc.nmbclusters=81920" in loader.conf as it's a read-only oid. Another route is to add "options NMBCLUSTERS=81920" into your kernel and compile/install (if it's too high/too low you'll have to compile and install again, much more cumbersome than setting it in loader.conf). Also you'll need to set "kern.maxfilesperproc=81920" if you'll be having the same daemon receive each of those tcp connections, or else you'll run out of fd's fairly quick. I don't think it'll hurt to bump kern.maxfiles either. Now, 81920 might not be the integral value you want, you can obviously make it more or less Personally I've had a mixed experience with that, as I've had kernel panics when setting nmbclusters above that threshold. Your own mileage may vary. Another variable to keep an eye is kern.ipc.somaxconn as this will dictate your connection queue. If there's going to be a flood of connections, the default value will be seriously anemic in this respect. If you run a stateful ipfw firewall on this machine as well, and use keep-alive connections, then be sure your "net.inet.ip.fw.dyn_max" is high enough to allocate dynamic rulesets for them all. Personally when dealing with machines handling connection loads like that, I have a tendancy to turn down "net.inet.tcp.sendspace" so it uses less memory per tcp connection. If you're handling this many connections, turning this up isn't a good idea. The "net.inet.tcp.recvspace" default should be ok for this, though if you really do push >75k connections concurrent, turning it down wouldn't be such a bad idea imho. One last word of advice, "man 7 tuning". -mpf --- Only dimly aware of a certain unease in the air --- Bill wrote: What steps would I need to take in order to obtain 75,000 concurrent TCP sessions on a FreeBSD 5.2 system running on the following hardware: dual xenon 3ghz 1mb cache processors 2 gigs of memory two dual port fibre gigabit nic's 1 onboard copper 10/100 nic I read a post that was sent to freebsd-hackers, which mentioned an individual was able to obtain 1.6 million concurrent tcp sessions, so I assume it's possible. My goal is to setup a server, which is capable of accepting at least 75k tcp connections to perform some firewall stress tests at work. Given that information on this subject is quite scarce, I thought I'd post this question and see what type of response I get back. Any assistance or suggestions would be greatly appreciated, Thanks in advance, -=-Bill-=- ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]" ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Obtaining 75k (active) concurrent tcp sessions..
On Thu, Feb 12, 2004 at 10:48:13PM -0600, Mike Silbersack wrote: > That was with a heavily modified version of FreeBSD, you wouldn't be able > to hit 1.6 million out of the box. What's more, the amount of content you'd be able to shift over that number of connections and the overall performance would be pretty shocking. If you're dealing with that many connections, you should be clustering. In fact, even for 75k, I would suggest clustering your the app over a couple of servers anyway. That reminds me - it's a long time since I last looked into -cluster. :-) -- Paul Robinson ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Obtaining 75k (active) concurrent tcp sessions..
On Thu, 12 Feb 2004, Bill wrote: > I read a post that was sent to freebsd-hackers, which mentioned an > individual was able to obtain 1.6 million concurrent tcp sessions, so I > assume it's possible. That was with a heavily modified version of FreeBSD, you wouldn't be able to hit 1.6 million out of the box. > My goal is to setup a server, which is capable of accepting at least 75k tcp > connections to perform some firewall stress tests at work. Given that > information on this subject is quite scarce, I thought I'd post this > question and see what type of response I get back. > > Any assistance or suggestions would be greatly appreciated, > > Thanks in advance, > > -=-Bill-=- I've run tests up to a few thousand connections, I believe that 75000 should be doable, but it will take tuning. Start with 5000 and keep increasing in increments of 5000 from there, upping values for various resource limits as you hit them. I think that maxsockets, mbuf clusters, and maxfiles will be your main limitations... they should all scale to 75000, but I'm not sure how many people have done so. Now, if you want good performance... that could be another story. However, if all you're doing is having a sample app which accepts connections and holds them open until the client hangs up, then you should be able to do it. (If you were sending real data, then the amount of memory being used for socket buffers might become a problem.) Note that for the client side of those connections, you may need more than one machine; with only ~65535 possible ephemeral ports available per IP (and it being tough to use the same ephemeral port on different IPs with the BSD network stack), it'd be best to just do two client machines with 75000/2 connections each. (There is no such limit for the server side, of course.) Post to this list if you run into any problems, or find any specific issues that prevent you from reaching the goal. Mike "Silby" Silbersack ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "[EMAIL PROTECTED]"
