Hey all, I've been looking at the Task Queue API and counter example. In my app, each user will have a couple of counters maintained for them, counting various things.
Thing is, these counters need to be accurate. So I'm not sure if the example given for the Task Queue API using memcache would be appropriate for me - it would not be good, really, if my counters were to be inaccurate. My users would expect accurate counts here. So I was thinking about a sort of modified version whereby each change to the counter would be stored in the DS in its own entity. E.g. an entity called 'counter_delta' or some such, which holds the delta to apply to a counter, and the key to the counter that the delta is to be applied to. Then, using the Task Queue I guess I could hoover up all these delta entities, aggregate them, and apply them in one go (or in batches) to the counter. And then delete the delta entries. Thus the task queue is the only thing accessing the counter entity, and it does so in a controllable fashion - so no real contention. Each change to the counter gets written to the store in its own counter_delta entity, so no contention there either. And because the deltas are stored in DS and not in memcache, it should be much more reliable. However, I'm not entirely sure how I should actually go about implementing this, or specifically, the task queue end of things. I'm thinking if there is a change to a counter to be made, I should check if there's a task already running for this counter, and if so, not to do insert any new task, and let the currently running task take care of it. If there is no running task for this counter, I would instead create one, and set it to run in - say - 60 seconds, allowing time for further deltas for this counter to accumulate so the task can take care of more than just one delta. This would mean the counter might be inaccurate for up to 60 seconds, but I can live with that. But what I'm wondering is, how can I implement this 'don't insert a new task if one for this counter is already in the queue or running' behaviour? I was thinking initially that I could give the task a name based on the counter, so that only one such task can exist at any one time. However, I believe we have no control over when that name is freed up - it isn't necessarily freed up when the task ends, I believe names can be reserved for up to 7 days (?) So that wouldn't work. If a name could be freed up when a task was really finished then this could work, I think. I was thinking also I could store a flag so that when a counter_delta is created, I'd look to see if a flag for this counter was present, and if so, do nothing. If not, create the task, and create the flag. Then when the task was all done and didn't see any more counter_deltas, it'd delete the flag. But I'm worried that there could be race conditions here, and some deltas might get overlooked as a result? And if I were to use transactions on such a flag, would I not fall into the same contention trap I'm trying to avoid in the first place? Help? :| Thanks for any advice/insight... -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
