Tom Preston-Werner <[email protected]> wrote: > On Thu, Sep 17, 2009 at 11:48 PM, Eric Wong <[email protected]> wrote: > > At what request rates were you running into this? ??Also how large are > > your responses? ??It could be the listen() backlog overflowing if Unicorn > > isn't logging anything. > > I was hitting the 502s at about 1300 req/sec and 80% CPU utilization. > Response size was only a few bytes + headers. I was just testing a > very simple string response from our Rails app to make sure our setup > could tolerate very high request rates.
Yup, as I suspected: your UNIX socket setup was maxing out right around where your TCP setup was maxing out. TCP is just better at handling/recovering from errors. > > Does increasing the listen :backlog parameter work? ??Default is 1024 > > (which is pretty high already), maybe try a higher number along with the > > net.core.netdev_max_backlog sysctl. > > This was the first thing I tried after getting your response, and it > seems that upping the :backlog to 2048 solves the 502 problem! I'm now > able to get 1500 req/sec out of Unicorn/UNIX (as opposed to 1350 > req/sec with the TCP/HAProxy setup). I'm quite satisfied with this > result, and I think this is how we'll end up deploying the app. Good to know it worked! However, I do hesitate to recommend a large listen() backlog for production. It can impede with monitoring/failover/load-balancing in multi-server setups even if it looks good on benchmarks. I'll make a separate call-for-testing mailing list related to this subject in a bit... > Thanks for your help, and I'll try to keep you updated on how our > installation performs and if I see any strange behavior under normal > traffic. No problem, thanks for the feedback! It's great to know people actually use it. -- Eric Wong _______________________________________________ mongrel-unicorn mailing list [email protected] http://rubyforge.org/mailman/listinfo/mongrel-unicorn
