On Mon, Nov 17, 2008 at 5:52 PM, Anders <[EMAIL PROTECTED]> wrote: > > On Nov 17, 6:30 pm, Jon McAlister <[EMAIL PROTECTED]> wrote: >> >> For any particular "key", all instances will talk to the same memcache >> backend. Note that we can easily have different keys hosted on >> different backends, though, thanks to the simplicity of the memcache >> API (i.e. lack of transactions). This is how we can shard one app's >> memcache data on to multiple machines. > > Ah! That explains it. This also means that an idea I had about > sharding the Memcache will work. Let's say that we have a key named > 'indexpage'. We can then shard the Memcache by adding an index, say > 0..99 to the key, so instead of just accessing a single key > 'indexpage' we can randomly access keys with the index added to it, > such as: 'indexpage_32', 'indexpage_7', 'indexpage_85' etc.
Before you go and implement that, do you have any evidence that memcache could be a bottle neck? Otherwise it sounds like a case of possible premeture optimization. Not withstanding the fact that as I understand memcache 'shards' by hashing the key - but that leads no garenteers that your keys will end up on seperate instances. And that also leads to more work as you now have to generate your page 100 times (which if you using the cache right is probably expensive) >From my experience of Memcache (not on AppEngine) - its very quick at dealing out the same result multiple times. And if memcache is truly distributed on AppEngine - and it doesnt do it already, then there is always the possiblility of edge caching really hot items (say on the machine itself) - which your sharding would instantly make less effective. (memcache - can itself be cached - which as Jon points out the datastore cant) I guess what trying to say is if at all possible you should leave the 'scaling' to the platform, its only where that is not possible (like counters) that you should consider it yourself, (like you say in your opening post!) > > > -- Barry - www.nearby.org.uk - www.geograph.org.uk - --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~----------~----~----~----~------~----~------~--~---
