Slow tasks don't get ran nearly as aggressively, at least not in my experience. I've found that I can pump far, far more tasks through if I keep tasks fast (sub 1000ms).
Are you seeing pending ms values in your logs? Robert On Jun 18, 2011, at 22:52, Waleed Abdulla <wal...@ninua.com> wrote: > I think I narrowed the issue a little bit. Added a lot of logging.info() > statements and I noticed the following: > > 1. The time between inserting a task and when it starts running is too long: > 5 - 20 seconds. Typically tasks should execute right away after inserting if > you're below the queue rate. That's issue #1 > > 2. I put a timer in my code to calculate the time it takes to execute each > task. When I compare that time with the time app engine reports in the logs, > there is a big difference. 5 - 20 seconds. For example, processing a task > takes 25 seconds, of which only 10 seconds is actually spent inside my code. > So the infrastructure is keeping requests on hold for a while before they get > executed. That's issue #2 > > When I compare the delay in #1 to the time difference calculated in #2, > they're very close and they correlate (i.e. they go up and down together). So > that means that issue #1 is probably caused by issue #2. (i.e. tasks do get > picked for execution right away, but the actual execution doesn't start until > 5 - 20 seconds later). > > There seems to be an infrastructure problem in GAE. Maybe GAE is > experiencing high load, too much network traffic, ..etc. Whatever the reason, > it seems that requests are waiting in a queue for too long before they run. > > Does anyone notice any issues that confirm or deny the above? Any ideas > how to work around it? > > Waleed > > > > > > > On Sat, Jun 18, 2011 at 7:45 PM, Chiguireitor <johnvillarzava...@gmail.com> > wrote: > The other issue is that each task spawns a new instance... could you be > hitting a max instance limit there? i really don't know if there's an > instance limit... but could happen > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/google-appengine/-/nsxW4XGycnwJ. > > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
