Hi all, I've been working on implementing a ESI (Edge Side Include) Proxy caching server. The idea is very similar to SSI (Server Side Include), except the includes are remote instead of local.
There's a very nice write up of using SSI in this context with nginx and memcached here => http://blog.kovyrin.net/2007/08/05/using-nginx-ssi-and-memcache-to-make-your-web-applications-faster/ My project is mongrel-esi (http://code.google.com/p/mongrel-esi) and I'd like to use it as an example of how one can use memcache and still invalidate URLs using regular expressions. The basic idea is to store the regular expressions that have been picked up by processing an invalidation rule in memcache. Before retrieving a key from memcache checking if the key matches any of the regular expressions in the invalidation-key and that the object being cached is older then the invalidation rule that matched. We can probably be clever about how we name the invalidation-key so that it too can expire. So long as the invalidation-key doesn't expire before a predetermined max-expiration time that all keys get. Then we can be certain that either before we request a key, or the keys expiration time the objects marked to expire by the regex will expire as requested by surrogate. For me this realization is great because it means I can finish implementing the proxy server using memcached as the cache storage. Hopefully, it also will be a helpful solution for others looking to access all the keys. And of course, I may be missing something important, so please correct me if I have. With this solution I think it's important to get the clever part about making sure the expire rules have an expire time on them that is greater then or equal to the expire time of all keys. There may also be some ways to name the keys based on the specific domain to have more then one invalidation key to avoid letting it grow into a large dataset that needs to be retrieved everytime. It also means you have two hits on memcache per key lookup instead of the one before... There are probably other techniques for solving this problem... Just hoping it makes sense and is useful to others... -Todd
