While only somewhat related, I put in a request for a memcache feature that allows data to prioritised, with higher priority data never evicted by lower priority data. Doesn't solve the entire problem of eviction, but at least lets you happily cache as much as you want at a lower priority and not care at all that you might evict something more important. As noted, eviction for reasons other than LRU is the biggest problem.
http://code.google.com/p/googleappengine/issues/detail?id=1797 On Apr 15, 7:01 pm, peterk <peter.ke...@gmail.com> wrote: > >> If you can tolerate additional latency, why not store data into the > >> datastore? > > Because it's expensive? :) There's a big difference between making a > 100ms put to datastore every request and making one every n requests. > Requests with that higher latency may be tolerable from a end-user's > POV but may not be tolerable from my wallet's perspective... :) > > >> We exposed a "grabTail" method for Memcache but removed it due to > >> namespace bugs. If we release this in the future, grabTail will return the > >> next item to be evicted by LRU, so you can write an asynchronous task to > >> flush this to the datastore. > > This would be good :) I'd no idea the grabTail method did that. The > next question then would be what proportion of evictions are cause by > LRU over-writes...I'm sure evictions sometimes happen for other > reasons (like hardware failure), but if those other reasons cause only > a tiny proportion of evictions, then this regime might make memcache > reliable 'enough'. > > If grabtail allows this, please bring it back soon! :D > > On Apr 15, 6:29 pm, "Ikai L (Google)" <ika...@google.com> wrote: > > > > > We're moving into a middle ground between Memcache and the datastore. If you > > can tolerate additional latency, why not store data into the datastore? You > > should not design assuming memcache will preserve critical data. Don't store > > anything in memcache that you cannot recover if evicted, even if your > > application runs in a degraded mode. > > > We exposed a "grabTail" method for Memcache but removed it due to namespace > > bugs. If we release this in the future, grabTail will return the next item > > to be evicted by LRU, so you can write an asynchronous task to flush this to > > the datastore. There are no plans to create triggers for eviction. > > > On Thu, Apr 15, 2010 at 8:15 AM, peterk <peter.ke...@gmail.com> wrote: > > > It would, but depending on the extent of that latency it might be a > > > reasonable trade off for reliability. I've an app that could > > > definitely tolerate some % of memcache saves being higher latency if > > > it guaranteed I didn't lose data. > > > > The option of a silent background put() to the datastore upon eviction > > > also wouldn't necessarily incur extra latency for your memcache save, > > > but would instead 'just' cost you cpu_ms for those ds puts (i'm sure > > > those DS puts could be asynchronous and your memcache operation could > > > return without waiting for them to finish). > > > > Again, I don't know if it's practical on the backend, feasible to > > > implement. But I believe other database systems offer that kind of > > > write-behind cache. > > > > On Apr 15, 3:45 pm, Wooble <geoffsp...@gmail.com> wrote: > > > > On Apr 15, 9:48 am, peterk <peter.ke...@gmail.com> wrote: > > > > > > It'd be great if memcache could throw a warning or error upon a put to > > > > > memcache, if that put will cause an eviction of another value, and ask > > > > > you to retry your put with a token to confirm you're OK with the > > > > > eviction. > > > > > It's a cache. That would add annoying latency. > > > > -- > > > You received this message because you are subscribed to the Google Groups > > > "Google App Engine" group. > > > To post to this group, send email to google-appeng...@googlegroups.com. > > > To unsubscribe from this group, send email to > > > google-appengine+unsubscr...@googlegroups.com<google-appengine%2Bunsubscrib > > > e...@googlegroups.com> > > > . > > > For more options, visit this group at > > >http://groups.google.com/group/google-appengine?hl=en. > > > -- > > Ikai Lan > > Developer Programs Engineer, Google App > > Enginehttp://googleappengine.blogspot.com|http://twitter.com/app_engine -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
