If you have low traffic, you're best off writing to a sharded counter. Here are a few strategies I've considered or used in the past and their tradeoffs:
- Periodic flushing. Has it passed some time since I've last flushed the value in this key? Yes? Persist this. You can also set this to be "generational" - that is, if it's rare this is flushed, almost always flush it. If it's a number that grows frequently, flush less. This strategy works best if you have a few objects with a low of growing counts. - Maintain an index of Memcache values to flush. You may want to do this in Memcache. Note that there are race conditions here, but if you're doing flushing, you're likely willing to absorb a few lost counts. The nice thing is that this can be done by a cron job, and you're less likely to miss values that are infrequently updated. This doesn't scale too well if you have a LOT of values getting updated frequently, but I suppose you could shard the indexes. This strategy is better if you have many objects with low counts. For anything in between, Memcache flushing is a premature optimization. If you have a moderate number of objects that are updated occasionally, you're just giving yourself unnecessary work. Even if they are updated a bit more frequently, you will likely be best with a sharded counter strategy. On Thu, Apr 8, 2010 at 11:42 PM, Tero Nurminen <tero.nurmi...@gmail.com>wrote: > Hi John, > > you mentioned about persisting the memcache counts periodically into > datastore. What's the best practice to do these periodical updates > let's say on a situation when the traffic is very volatile? I've > implemented Twig-persist into my project, so would you like to provide > some additional code for the example based on the Cookbook strategy, > http://code.google.com/p/twig-persist/wiki/Cookbook? > > On Apr 8, 12:33 pm, John Patterson <jdpatter...@gmail.com> wrote: > > You will need to read and update the object in a transaction which > > will throw an exception on commit if another thread updates the same > > entity. You can then refresh the object and try again. Also look > > into memcache which has an atomic counter increment method that is > > much faster than reading and writing to the datastore. The downside > > is that the counts could be reset so you may need to periodically > > store them in the datastore. > > > > On 8 Apr 2010, at 16:17, mscwd01 wrote: > > > > > Hey, > > > > > How do you ensure, once you have queried an object from the datastore, > > > that another call to that object cannot be made? > > > I basically have a bunch of objects with counters which get > > > incremented during each call to that object. > > > > > What I need to ensure is the object doesn't get called by two threads, > > > each of which increment the counter by 1 and persist the object back > > > to the datastore, only for the counter to be incremented by 1 and not > > > by 2. > > > > > Thanks > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "Google App Engine for Java" group. > > > To post to this group, send email to > google-appengine-java@googlegroups.com > > > . > > > To unsubscribe from this group, send email to > google-appengine-java+unsubscr...@googlegroups.com<google-appengine-java%2bunsubscr...@googlegroups.com> > > > . > > > For more options, visit this group athttp:// > groups.google.com/group/google-appengine-java?hl=en > > > . > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine for Java" group. > To post to this group, send email to > google-appengine-j...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine-java+unsubscr...@googlegroups.com<google-appengine-java%2bunsubscr...@googlegroups.com> > . > For more options, visit this group at > http://groups.google.com/group/google-appengine-java?hl=en. > > -- Ikai Lan Developer Programs Engineer, Google App Engine http://googleappengine.blogspot.com | http://twitter.com/app_engine -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
