Hello there,

Let me start off with the obvious *I do know that ElastiCache is a hosted 
service and is not the same thing as memcached.* That said Amazon's support 
is beyond useless so here is the next best place to ask.

See below for a diagram of this.
The application stack consists of:


   - Apache2 with mod_php5 for the web server layer
   - Joomla 2.5.11 with a bunch of custom extensions for the application
   - MySQL 5.5 on Amazon RDS for Database
   - Apache Solr 4.0 for search indexing
   - ElastiCache for caching (obviously)

After pouring over some staticstics and application traces and it appears 
that over 50% of the application time is spent interacting with memcache
Here's a sample graph of what I mean:

<https://lh4.googleusercontent.com/-Zibq65hrVno/UiSDKy1YpjI/AAAAAAAAB5o/5R-JF8bQ_ns/s1600/2013-09-02-132254_833x384_scrot.png>

Graph of Application Response Times

It's clear that an usually long time is being spent in memcached, I've also 
taken a look at some statistics directly from the memcached server:


<https://lh6.googleusercontent.com/-5xs60-65Nu8/UiSD4SGMqPI/AAAAAAAAB5w/oG_MzK5ZGok/s1600/2013-09-02-132431_1066x530_scrot.png>

Memcache Statistics


As you can see there are only around 7MiB of objects in memcache, yet still 
a memcache REPLACE always takes 100ms, and a memcache ADD often takes over 
50ms

The set rate is unusually high (seen here is 1500 SET/s) which is crazy for 
an application which processes around 50 requests per minute. The 
performance of the memcache server degrades as the number of items grows, 
as such we have to issue a FLUSH_ALL command every 30 minutes in order to 
keep the number of items low enough that the site doesn't completely fail 
to load.

We have tried both using Amazon ElastiCache and running our own memcache 
server, both of which have given us the same result, leading me to believe 
that the server implimentation is not the root cause of the issue.


Thanks in advance!


Application Stack:

<https://lh5.googleusercontent.com/-gZYiEsTVq-k/UiSEyaX3AkI/AAAAAAAAB58/LX-LixvUh0I/s1600/Application+Stack+-+New+Page.png>

The Application Stack



-- 

--- 
You received this message because you are subscribed to the Google Groups 
"memcached" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to memcached+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Reply via email to