On 2012-4-9 18:28, Michael Bayer wrote:
On Apr 9, 2012, at 11:49 AM, Wichert Akkerman wrote:
On 2012-4-9 17:28, Michael Bayer wrote:
There's a decent README up now at http://pypi.python.org/pypi/dogpile.cache and
you can read all the docs at http://dogpilecache.readthedocs.org/. I'm hoping
to get some testers and initial feedback.
Can you shed some light on how this differs from retools
(http://readthedocs.org/docs/retools/en/latest/), other than that dogpile does
not support redis and retools only supports redis?
Basically the caching and distributed locking features of retools should be a dogpile
backend, and in fact they can be if retools wants to publish a dogpile.cache backend.
Though looking at the source he'd have to rip out some more rudimental functions out of
CacheRegion.load(), which seems to be totally inlined right now. The redis lock itself
could be used but needs a "wait" flag.
Would you also be willing to accept a pull request (assuming you use
git, otherwise a patch?) that adds a redis backend to dogpile directly?
It seems like we'd almost be better off taking the "stats" logic wired into
"load" and just making that an optional feature of dogpile, so that all the backends
could get at hit/miss stats equally. The get/set/lock things i see in retools would only give us
like a dozen lines of code that can actually be reused, and putting keys into redis and locking are
obviously almost the same as a memcached backend in any case. It's just a question of, which
project wants to maintain the redis backend.
The statistics are certainly very useful.
He appears to have a "region invalidate" feature taking advantage of being able
to query across a range in redis, that's not something we can generalize across backends
so I try not to rely on things like that, but dogpile can expose this feature via the
backend directly.
Region invalidate is very very useful in my experience: for us it allows
us to have a management-system invalidate caches for a running website
without having to make it aware of all the implementation details of the
site. We can now simple say 'invalidate everything related to magazines'
instead of invalidating 15 different functions separately (which will
get out of sync as well).
The function decorators and the dogpile integration in retools are of course
derived from Beaker the same way dogpile's are and work similarly. Dogpile's
schemes are generalized and pluggable.
A problem I have with the Beaker and retools decorators is that they
make it very hard to include context from a view into a cache key. For
example for complex views it is very common that you want to cache a
helper method for the view class, but you want the context and things
like request.application_url to be part of the cache key, but those are
never passed to the method. That leads to code like this:
class MyView:
@some_cache_decorator
def _slow_task(self, context_id):
# Do something
def slow_task(self):
return self._slow_task(self.context.id)
one approach I used to take was to use a decorator which could take a
function parameter which returned extra cache keys. You could use that
like this:
class MyView:
def _cachekey(self, *a, **kw):
return (self.request.application_url, self.context.id)
@some_cache_decorator(extra_keys=_cachekey)
def slow_task(self):
# Do things here
It seems like Ben stuck with the @decorate("short-term", "namespace") model we first did
in Beaker, whereas with dogpile.cache though the API looks more like flask-cache
(http://packages.python.org/Flask-Cache/), where you have a "cache" object that provides the
decorator.
That sounds like the dogpile approach does not supported environments
where you have multiple copies of the same application in the same
process space but using different configurations? That's a rare
situation, but to some people appears to be to important.
Queue/job/etc appears to be something else entirely, I'd ask how that compares
to celery-redis and other redis-queue solutions.
I can already answer that one :)
Wichert.
--
Wichert Akkerman <wich...@wiggy.net> It is simple to make things.
http://www.wiggy.net/ It is hard to make things simple.
--
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To post to this group, send email to sqlalchemy@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
