David Favor wrote:
Pratyoosh Sharma wrote:
We have an external facing portal that serves multiple applications
via single sign on, to bypass cross domain limitations & simplifying
support we proxy all the requests through apache httpd, there are
about 100 independent applications that we proxy using mod_proxy
In case of slow responses (due to hung threads) unforeseen situations,
we cause saturation of resources on apache & cause it to die.
We want to introduce a circuit breaker/bulkhead (timeouts are not good
enough) so that outages in few application do not impact the rest.
Anyone else having the same issue? Ideas for solution?
Many times the underlying OS setup (where Apache is running)
can be tuned to circumvent this.
The speeds listed below are fairly simple to attain. The
following are required for this...
1) Dedicated hardware
Cheapest quad core + hyperthreading is sufficient for
even the largest of projects
2) Following the tuning steps below
3) Usually takes only 3-4 hours to tune for these speeds.
Start by testing bare metal speed of OS + Apache via this...
cd some-domain-root (we'll call it foo.com)
touch foo.html
ab -k -t 10 -n 10000000 -c 5 http://foo.com/foo.html
A correctly tuned server should return huge numbers like...
30,000-50,000+ requests/second
Next test your CMS system... as in...
ab -k -t 10 -n 10000000 -c 5 http://foo.com/ (notice no foo.html)
A correctly tuned WordPress system should return numbers like...
3,000-5,000 requests/second
Until you get these numbers fixed, any other attempt at tuning will fail
miserably.
To get these huge WordPress speeds requires tuning ever subsystem, bare
metal upwards...
1) Raw disk devices (filesystem + mount options)
2) /tmp must be put in tmpfs (/var/tmp must stay on disk)
3) Use MariaDB (MySQL that works + works fast) + tune it well
4) Tune the Apache MPM you're using...
Till you get the 30,000-50,000+ requests/second static file speed.
5) Tune PHP Opcache till you see 99% hits out of cache, with
lots of free memory left over.
Keep in mind, since PHP-5.5 APC is no longer used, Opcache
replaces APC. This bites many people as they update Apache/PHP.
6) Tune WordPress theme + plugins + caching
Till you get the 3,000-5,000 requests/second
PHP/CMS/WordPress/Whatever speed.
7) Track Opcache usage + increase memory as required.
8) Track top/swapping + add more RAM as required.
In most cases, it's far cheaper + easier to manage large sites by...
1) Doing the tuning above
2) Changing Apache ServerLimit till all Apache crashes + weirdness
disappears
3) Adding memory to Opcache + System RAM produces far better results
than
complex frontends.
The big winner is making sure Opcache has high hit rate + extra memory...
And there is no swapping.
One other thing to avoid. Apache suexec appears to degrade performance in some
cases. Haven't tracked this down yet + the easy work around is this...
1) set mount options on disk to add 'acl'
2) set all Apache directories to default Apache user:group ownership +
permissions
3) use setfacl to add handle sandboxing + allowing access to different
Apache installation data files, like with ftp or sftp.
Using extended ACLs seems to produce no measurable performance reduction + it's
way easier to manage than suexec setups.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@httpd.apache.org
For additional commands, e-mail: users-h...@httpd.apache.org
