For those of you who might run into this in the future.. I've gleaned that task ETA is the "soonest" a task will execute. Under load those times start to slip. Even after jacking queue processing rates way up I'm finding huge delays build up over time. I'm going to need a new scheduling strategy. For high throughput workloads, be warned: executing tasks with an ETA is not reliable. Perhaps there is a way to spin up additional appengine instances... but for my purposes I'm going to write some code to determine if a task is "late" when I execute it and re-schedule it appropriately so my tasks remain spaced at a proper distance.
On Thursday, June 8, 2017 at 11:42:53 AM UTC-4, Robert _ wrote: > > As a quick follow-on I should note that this is not an issue with my local > machine or development environment. Tasks are being enqueued *from the > exact same Compute Engine machine* in all scenarios. I have done the work > to isolate this issue. The timing error is either introduced by the > mechanism that pushes the task into the queue or by the queue mechanism > itself. > > > On Thursday, June 8, 2017 at 11:34:35 AM UTC-4, Robert _ wrote: >> >> >> I'm experiencing a very tricky bug or "feature" in Taskqueues regarding >> tasks set with an ETA. Tasks are being executed 1 hour later than expected >> *in some instances* (inconsistently). I think this behavior started after >> DST. >> >> In my experience with a handful of scheduled tasks (e.g. development), it >> works fine. I'm using UTC timestamps in my Tasks set with the >> "etaMillis()" method in the Taskqueue API. Things execute just when >> expected. >> >> Each night, however, I have thousands of tasks that queue up. This >> causes multiple AppEngine instances to fire up (and I think this is >> important--the root cause of the inconsistent behavior). Only during load >> am I seeing this issue. *Some* tasks--are executing exactly 1 hour behind >> when they are expected to. My AppEngine region is us-central. >> >> Task queues are a wonderful feature of AppEngine. Anyone else experience >> this with high volumes of scheduled tasks? What's going on? Different app >> engine instances with different time zone settings? >> >> I'm thinking a potential alternative will be to specify countdown >> instead... maybe this will resolve the issue? >> > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at https://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/a652965b-0afd-4a7c-885d-431db911041b%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
