I have forgotten and am too lazy to go through all my emails again to read, but have you tried standard intel server NIC's for this?
Curtis LaMasters http://www.curtis-lamasters.com http://www.builtnetworks.com On Wed, Jul 29, 2009 at 10:30 AM, Lenny<five2one.le...@gmail.com> wrote: > Hi guys, > > I know how sick of me you are by now, but I've had some developments here > and now I'm stuck again. > > So, FINALLY I convinced the management to buy a new server. We bought an IBM > x3550 with 2 Quad Core CPUs E5420 and 2GB RAM PC2-5300 667MHz. Not just > that, we bought 2 of them ( we need the second one for the identical > project). > > I really wanted to try the Yandex em driver, so I installed the "1.2.3-RC2 > built on Wed Jun 24 10:37:51 EDT 2009 version. > These are the things I've changed: > /etc/sysctl.conf > added: > dev.em.0.rx_processing_limit=1000 > dev.em.1.rx_processing_limit=1000 > > kern.ipc.somaxconn=1024 > > dev.em.0.rx_kthreads=2 > dev.em.1.rx_kthreads=2 > > /boot/loader.conf > added: > hw.em.rxd="4096" > hw.em.txd="4096" > > I wasn't sure about the kthreads, so I put "2" as a value, but I later read > that people use 6. > Now, for the problem: > It's the same! I had 50kpps, which was 180Mbps and I had 2 CPUs go up to > 100%. > How is that even possible? > > Should I give up on RC version and go back to stable, which also means > giving up on the em driver? > Any other things I can adjust? > > By the way, I checked sysctl net.inet.ip.intr_queue_drops and it's "0". > On the interfaces I see that em0(outside) has 0 errors, but on the > em1(inside) there are 3666587/0. > 6 of the CPUs(cores) are usually 100% idle, while the other 2 are actually > stuck with emX taskq. > In other words - nothing's changed. Except for 1 thing - the management > spent the money and now they wanna see results. > Please help! > > Lenny. > > > changed: > net.inet.ip.intr_queue_maxlen=4096 > (old value=1000) > On Thu, May 14, 2009 at 11:26 AM, Lenny <five2one.le...@gmail.com> wrote: >> >> Thanks for all the suggestions, guys. >> Anyway, I found it very interesting that the new snapshots have yandex >> driver in them, so I decided to try it. >> Of course, as I don't have the new server yet, I had to try on my old IBM >> x335. >> >> But here are a couple of things that wouldn't let me try it: >> with 2 SCSI hard drives in RAID 1 (LSI controller), it would always give >> an error and reboot in 15 sec. If I take out 1 drive - it would boot, but >> then get stuck on "configuring wan interface" for a long time and then >> reboot. It's not connected, but previous pfSense versions had no problems >> with it. >> >> Any suggestions? >> >> thanks, >> Lenny. >> >> P.S. we don't have eval-systems, so I'll have to take the risk and buy a >> cat in the sack. >> >> On Wed, May 13, 2009 at 6:13 PM, Scott Ullrich <sullr...@gmail.com> wrote: >>> >>> On Wed, May 13, 2009 at 10:21 AM, Rainer Duffner <rai...@ultra-secure.de> >>> wrote: >>> > AFAIK, SUN still provides eval-systems for free. >>> > I would evaluate one of the new X2270 with the Nehalem Xeons. >>> > This should provide a 50% boost even on 5400-series Xeons. >>> > Also, they use Intel NICs, IIRC. >>> > >>> > The smallest test-system already has 6 GB of RAM and costs 2000 USD, >>> > which you have to pay only after 60 days. >>> >>> All good advice here in the last couple messages. Wanted to add that >>> I would suggest trying a recent 1.2 / FreeBSD 7 pfSense snapshot from >>> snapshots.pfsense.org as we added the high performance yandex driver >>> located at >>> http://people.yandex-team.ru/~wawa/em-6.9.6-RELENG7-yandex-1.36.2.10.tar.gz >>> >>> The README file that was included: >>> >>> Main features >>> ------------- >>> >>> RX queue is being processed w/more than one thread. Use "sysctl >>> dev.em.X.rx_kthreads" to alter number of threads. >>> >>> TX interrupts has been removed because it's not neccessary actually. >>> That's why interrupt rate has been reduced twice at least. >>> >>> TX queue cleaning moved to seperate kthread. em_start uses mtx_trylock >>> instean of mtx_lock. That's why em_start locks less. >>> >>> + RX queues' priority may be altered thru sysctl. System seems to be >>> more stable if RX scheduled w/less priority. >>> >>> + RX interrupt stay masked if there is no thread ready to catch >>> interrupt. The hint reduces context switching under load. >>> >>> You will want to experiment with 1 thread per proc and 2 threads per >>> proc by setting "sysctl dev.em.X.rx_kthreads" (I think) >>> >>> Scott >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: support-unsubscr...@pfsense.com >>> For additional commands, e-mail: support-h...@pfsense.com >>> >>> Commercial support available - https://portal.pfsense.org >>> >> > > --------------------------------------------------------------------- To unsubscribe, e-mail: support-unsubscr...@pfsense.com For additional commands, e-mail: support-h...@pfsense.com Commercial support available - https://portal.pfsense.org
