64-per isn't *that* many if you are using multi-CPU web servers and
they're spending most of their time waiting for the database or
memcached. We've run with higher limits than that in the past. Right now
I think our limit is around 50 Apache processes per host.
We used to try to figure out our total maximum number of incoming
connections and specify the corresponding value on memcached's command
line, but after one too many instances of, "We just added another 300
web servers, forgot to bump up the limit, and now we're getting
connection failures," we decided to run memcached with "-c 500000" so as
to avoid running into any connection limits in the foreseeable future.
It stinks to have to restart memcached just to bring a bunch of new web
hosts online. Running with that high limit has not caused any problems.
The growth beyond the memory limit can be more than just a wee bit, BTW;
for servers with lots of non-UDP clients, we set aside a couple
gigabytes for per-connection memory. It's not just the
memcached-initiated memory allocation you have to worry about, but also
the kernel-managed TCP buffers. And even inside memcached, the amount of
memory used per connection can be variable; memcached allocates enough
memory to buffer up an entire incoming command, so if you do a huge
multiget it will (temporarily) eat a lot of memory internally.
-Steve
dormando wrote:
64-per is a lot :)
Anyway, it's fine, go nuts. There's a comment next to the maxfiles
section:
/* to limit connections-related memory to about 5MB */
... I have no idea how they arrived at this conclusion but it's not
hard to figure out. The only things to keep in mind:
- There are small memory allocations per connection.
- They're not presently tracked by the global memory limit, so
memcached will grow beyond that limit a wee bit based on your
connections (and I'm not man enough to fix it instead of just
repeating this).
-Dormando
Robin Schuil wrote:
Hi all,
I'm trying to run memcached but had some trouble with the connection
limit. Already found that I need to use the '-c <num>' option to
increase the # of maximum simultaneous connections. However, I have
200 webservers each with ~64 apache childs running (there may be
peaks). If each process would create a persistent connection that
would mean that each of my 10 memcached nodes have to handle 12,800
simultaneous connections.
I guess that shouldn't be a problem, but the low default maximum of
1024 somewhat bothers me. Is it sane to start memcached with a limit
of 40K connections, or is it likely that I'll run into other problems
then?
Your input is greatly appreciated.
Thanks,
Robin
