I did not - i'm definitely going to add that, thank Dale. I re- enabled the queue the other day and it's been behaving as expected, so it was definitely some external factor that made it spin out of control.
My project is a cloud based data historian on www.nimbits.com where users attach arduino devices to the services and record second by second data, so when they delete a data point I need to chew through the obsolete values that were recorded to that point. I ran out of quota on a day where a user deleted a point with millions of recorded values on it. It was supposed to just be a small background task. On Mar 6, 11:44 am, Dale <dalehu...@gmail.com> wrote: > Did you set max_concurrent_requests for that queue? Perhaps there was > some latency on the datastore and your queue was adding 5 tasks each > second but all the tasks were still running until the 10 s datastore > timeout and then retrying. In that case you would get 50 tasks > executing in parallel all querying 1000 records. Could explain the > high CPU usage. > > Dale > > On Mar 5, 5:33 pm, Benjamin <bsaut...@gmail.com> wrote: > > > > > > > > > Hi Nick, > > > The app id is nimbits1 (nimbits1.appspot.com) > > > The spike occurred around 12pm EST on March 3rd. 95% of the cpu being > > used at the time was > > Task Queues > deletedata which is set to 5/sec with a bucket size of > > 5. > > > Thanks! > > > On Mar 4, 4:21 pm, Nicholas Verne <nve...@google.com> wrote: > > > > Could you give us your app id and the approximate time of the gobbling > > > of CPU so we can investigate? > > > > Thanks, > > > > Nick > > > > On Sat, Mar 5, 2011 at 5:49 AM, Benjamin <bsaut...@gmail.com> wrote: > > > > I have a Task that's purpose is to chew away at data that needs to be > > > > deleted. I have millions of records that need to go, so the task uses > > > > the low level api and key only queries to grab 1000 delete them and > > > > restart the task until the remaining count is zero. > > > > > The task is set to 5/sec with a bucket size of 5. The other day it > > > > skyrocketed my CPU usage and maxed out my 5$ quota (400 CPU seconds / > > > > second, i think the lights in New York dimmed). Am i missing some > > > > setting here? How do i keep a task queue chugging along at a fixed > > > > rate without a burst like that, regardless of how many tasks are in > > > > the queue? > > > > > Thanks! > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups "Google App Engine" group. > > > > To post to this group, send email to google-appengine@googlegroups.com. > > > > To unsubscribe from this group, send email to > > > > google-appengine+unsubscr...@googlegroups.com. > > > > For more options, visit this group > > > > athttp://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
