Hi Ikai, If the grab tail do intend to only return objects belong to a specific namespace, it will be nice if the memchace "clear all" function only delete objects in a specific namespace as well.
At first I thought the "clear all don't respect namespace" is due to memcache implementation constraint. But if grab tail can respect namespace, there's no reason why clear all can't respect namespace right? It will be very helpful if we can flush a specific type of objects (that belong to same namespace) in memcache. Right now we either flush everything or maintain a set of keys and do a delete all. Thanks. On Feb 5, 12:04 pm, "Ikai L (Google)" <ika...@google.com> wrote: > Hey Viktor, > > I've been doing a bunch of research on this and it turns out my original > post was incorrect. Unlike the version of Memcached that is out in the wild, > the version of Memcached being used for App Engine *does* provide namespaced > partitioning and nested namespace partitioning, with LRU expiration > implemented both globally and per namespace (not 100% sure about namespaces > deeper than root + 1). What this allows is constant time namespace flushing, > among other security features. This is the mechanism that allows App Engine > developers to all use Memcache without trampling over each others' data and > is more sophisticated than prefixing a key - allowing us to track usage per > user without have to iterate over a key range. > > I'm going to file a bug internally against this mirroring your bug report. > It's possible we just haven't exposed the interface for nested LRU grabTail > correctly. I'll update the external bug if I hear anything. > > > > On Thu, Feb 4, 2010 at 1:17 AM, phraktle <phrak...@gmail.com> wrote: > > Hi Ikai, > > > Any updates? (I see there's a mention of adding grabTail in the 1.3.1 > > release notes - though it has been there in 1.3.0 already) > > > In the meantime, I have implemented the queue with a head/tail > > pointer, using the increment call... This works, but if grabTail > > worked as advertised, that would be more elegant. > > > Regards, > > Viktor > > > On Feb 2, 7:28 pm, "Ikai L (Google)" <ika...@google.com> wrote: > > > That's interesting. Let me follow up with the team to try to understand > > how > > > this is supposed to work. > > > > On Tue, Feb 2, 2010 at 12:05 AM, phraktle <phrak...@gmail.com> wrote: > > > > > Plus, it's working okay in the development server :) > > > > > V. > > > > > On Feb 2, 9:03 am, phraktle <phrak...@gmail.com> wrote: > > > > > Hi, > > > > > > In this case grabTail would be quite useless, as one cannot really > > > > > construct queues with it (ie. it would just force removing items from > > > > > all namespaces that wouldn't even expire otherwise). The > > documentation > > > > > also implies that there's a separate LRU list per namespace, which > > > > > does make sense, but is not what's happening in production: > > > > > >http://code.google.com/appengine/docs/java/javadoc/com/google/appengi > > .. > > > > .) > > > > > > "Grabs (atomically get and delete) items off the tail of LRU list. > > > > > This can be used to implement queue system with high throughput and > > > > > low latency, but low reliability. Current namespace should be set and > > > > > not empty for the service. For each namespace memcache maintains a > > > > > separate LRU list." > > > > > > Regards, > > > > > Viktor > > > > > > On Feb 1, 9:22 pm, "Ikai L (Google)" <ika...@google.com> wrote: > > > > > > > I'll raise the issue with some other members of the team, but it > > seems > > > > like > > > > > > this is working as expected. A memcache namespace is nothing more > > than > > > > a > > > > > > prefix applied to a memcache key. There's no true partitioning > > > > mechanism > > > > > > within memcache. grabTail simply returns the item that would be > > expired > > > > by > > > > > > the LRU mechanism if memcache needed more space. There isn't a > > > > different > > > > > > "queue" per namespace, only a "queue" for global expirations. > > > > > > > On Mon, Feb 1, 2010 at 4:29 AM, phraktle <phrak...@gmail.com> > > wrote: > > > > > > > Hi, > > > > > > > > On production, grabTail returns objects from other namespaces. > > This > > > > is > > > > > > > a significant problem that > > > > > > > makes grabTail (thus queue-like usage) unusable. > > > > > > > > I filed this as a bug, with a very simple example here: > > > > > > >http://code.google.com/p/googleappengine/issues/detail?id=2706 > > > > > > > > Can you please look into this? I don't even see a workaround that > > I > > > > > > > can implement in the meantime... > > > > > > > > Thanks, > > > > > > > Viktor > > > > > > > > -- > > > > > > > You received this message because you are subscribed to the > > Google > > > > Groups > > > > > > > "Google App Engine for Java" group. > > > > > > > To post to this group, send email to > > > > > > > google-appengine-j...@googlegroups.com. > > > > > > > To unsubscribe from this group, send email to > > > > > > > google-appengine-java+unsubscr...@googlegroups.com<google-appengine-java%2bunsubscr...@googlegroups.com><google-appengine-java%2B > > unsubscr...@googlegroups.com><google-appengine-java%2B > > > > unsubscr...@googlegroups.com> > > > > > > > . > > > > > > > For more options, visit this group at > > > > > > >http://groups.google.com/group/google-appengine-java?hl=en. > > > > > > > -- > > > > > > Ikai Lan > > > > > > Developer Programs Engineer, Google App Enginehttp:// > > > > googleappengine.blogspot.com|http://twitter.com/app_engine > > > > > -- > > > > You received this message because you are subscribed to the Google > > Groups > > > > "Google App Engine for Java" group. > > > > To post to this group, send email to > > > > google-appengine-j...@googlegroups.com. > > > > To unsubscribe from this group, send email to > > > > google-appengine-java+unsubscr...@googlegroups.com<google-appengine-java%2bunsubscr...@googlegroups.com><google-appengine-java%2B > > unsubscr...@googlegroups.com> > > > > . > > > > For more options, visit this group at > > > >http://groups.google.com/group/google-appengine-java?hl=en. > > > > -- > > > Ikai Lan > > > Developer Programs Engineer, Google App Enginehttp:// > > googleappengine.blogspot.com|http://twitter.com/app_engine > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine for Java" group. > > To post to this group, send email to > > google-appengine-j...@googlegroups.com. > > To unsubscribe from this group, send email to > > google-appengine-java+unsubscr...@googlegroups.com<google-appengine-java%2bunsubscr...@googlegroups.com> > > . > > For more options, visit this group at > >http://groups.google.com/group/google-appengine-java?hl=en. > > -- > Ikai Lan > Developer Programs Engineer, Google App > Enginehttp://googleappengine.blogspot.com|http://twitter.com/app_engine -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
