Re: [google-appengine] Re: Task queue can't keep up with load. Suggestions?
I'm noticing a big improvement today. After having a task queue backlog of 5 days and growing yesterday, today the tasks are running faster and the backlog is quickly going down (4 days as of this moment) and at this rate it should be down to 0 by the end of the day. It seems that someone fixed something. Thank you :) On Sun, May 23, 2010 at 12:20 PM, Wesley C (Google) wesc+...@google.comwesc%2b...@google.com wrote: thanks for all your feedback on the the current task queue execution (or lack thereof). we have reported this to the engineering team, but to help other users, can one of you file a new issue at http://code.google.com/p/googleappengine/issues/entry and everyone affected star it for us? please post the issue link to this thread as a trackback. thanks! -- wesley - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Core Python Programming, Prentice Hall, (c)2007,2001 Python Fundamentals, Prentice Hall, (c)2009 http://corepython.com wesley.j.chun :: wesc+api-at-google-dot-com developer relations :: google app engine @app_engine :: googleappengine.blogspot.com -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Task queue can't keep up with load. Suggestions?
Just an update. The problem is back, and my task queues are back to their very slow execution mode :( My tasks backlog now stands at 4 days :( On Mon, May 24, 2010 at 10:57 AM, Waleed Abdulla wal...@ninua.com wrote: I'm noticing a big improvement today. After having a task queue backlog of 5 days and growing yesterday, today the tasks are running faster and the backlog is quickly going down (4 days as of this moment) and at this rate it should be down to 0 by the end of the day. It seems that someone fixed something. Thank you :) On Sun, May 23, 2010 at 12:20 PM, Wesley C (Google) wesc+...@google.comwesc%2b...@google.com wrote: thanks for all your feedback on the the current task queue execution (or lack thereof). we have reported this to the engineering team, but to help other users, can one of you file a new issue at http://code.google.com/p/googleappengine/issues/entry and everyone affected star it for us? please post the issue link to this thread as a trackback. thanks! -- wesley - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Core Python Programming, Prentice Hall, (c)2007,2001 Python Fundamentals, Prentice Hall, (c)2009 http://corepython.com wesley.j.chun :: wesc+api-at-google-dot-com developer relations :: google app engine @app_engine :: googleappengine.blogspot.com -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Task queue can't keep up with load. Suggestions?
thanks for all your feedback on the the current task queue execution (or lack thereof). we have reported this to the engineering team, but to help other users, can one of you file a new issue at http://code.google.com/p/googleappengine/issues/entry and everyone affected star it for us? please post the issue link to this thread as a trackback. thanks! -- wesley - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Core Python Programming, Prentice Hall, (c)2007,2001 Python Fundamentals, Prentice Hall, (c)2009 http://corepython.com wesley.j.chun :: wesc+api-at-google-dot-com developer relations :: google app engine @app_engine :: googleappengine.blogspot.com -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Task queue can't keep up with load. Suggestions?
1500 tasks per minute would be 2 160 000 tasks per day which exceeds the billing quota of 1 000 000. I had my quota raised to 5M tasks per day as the original quota was not enough. Maybe you can aggregate your tasks for better results. That actually goes against the advice I get to keep the tasks as small as possible. Plus, I already get a lot of DeadlinceExceeded errors even though my tasks should not get close to 30 seconds, but the datastore gets too slow sometimes. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Task queue can't keep up with load. Suggestions?
Google told me to just make sure all tasks ran below 1 second but this doesn't even really help and IMO, defeats one of the main goals of the Task Queue itself (longer running processing, albeit below 30 secs) brian I agree, Brian. My tasks cannot be done in less than a second because they are mainly pulling RSS feeds from other sites and that usually takes close to a second (sometimes much more), although most of it is actually just waiting for the other site to reply. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
