Re: [appengine-java] Re: problems with SDK 1.2.8 and (local) task queues timing out...
Please re-read the thread leading up to your post. The timeout shouldn't impact the completion of the request on the server. If you're seeing something different please let me know. Thanks, Max On Thu, Dec 10, 2009 at 10:26 PM, Heyali heyal...@gmail.com wrote: I am also facing simmilar problem. Any clue? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: problems with SDK 1.2.8 and (local) task queues timing out...
Local task execution is using the local UrlFetchService implementation with its default timeout: 5 seconds. So, if the url that your task hits takes more than 5 seconds to respond you'll see this exception. However, this exception is from the client of the task url, not the task execution itself. The client will timeout after 5 seconds but the task should still execute to completion - we don't interrupt it. Please let me know if that's not the case. Now, 5 seconds is clearly not enough time since the request limit for apps is 30 seconds. This is a bug, and it's already fixed. It should go out with the next SDK. In the meantime just try to ignore the timeouts. As for retries, this is a known limitation: http://code.google.com/appengine/docs/java/taskqueue/overview.html#Task_Queues_and_the_Development_Server We'll get this brought in line with production shortly. Thanks, and sorry for the trouble. Max On Sun, Dec 6, 2009 at 9:57 PM, Tristan sp00s...@gmail.com wrote: I've seen this as well. I've also noticed that if I purposefully make a task fail (by returning any HTTP status code outside of the range 200-299), it doesn't get added back to queue for retrying. This used to work in 1.2.6. -Tristan On Dec 6, 4:18 pm, Larry Cable larry.ca...@gmail.com wrote: I just updated from 1.2.6 to 1.2.8 and my application code has started failing locally. I am using task queue's in order to process uploaded files, this worked in 1.2.6 (although you had to manually fire the task queue from the _ah admin console) Now, it is failing under 1.2.8 (as it seems as though it is now automatically starting the queue) with the following exception: Dec 7, 2009 12:08:57 AM org.quartz.core.JobRunShell run SEVERE: Job default.a7f0abc6-ed81-4fa4-96f1-c09048a99883 threw an unhandled Exception: com.google.apphosting.api.ApiProxy$ApplicationException: ApplicationError: 5: http method POST against URLhttp://localhost:8080/admin/gae-tasks/upload/csv/irs/processor.task timed out. at com.google.appengine.api.urlfetch.dev.LocalURLFetchService.fetch (LocalURLFetchService.java:214) at com.google.appengine.api.labs.taskqueue.dev.UrlFetchJob.execute (UrlFetchJob.java:53) at com.google.appengine.api.labs.taskqueue.dev.UrlFetchJob.execute (UrlFetchJob.java:46) at org.quartz.core.JobRunShell.run(JobRunShell.java:195) at org.quartz.simpl.SimpleThreadPool$WorkerThread.run (SimpleThreadPool.java:520) Dec 7, 2009 12:08:57 AM org.quartz.core.ErrorLogger schedulerError SEVERE: Job (default.a7f0abc6-ed81-4fa4-96f1-c09048a99883 threw an exception. org.quartz.SchedulerException: Job threw an unhandled exception. [See nested exception: com.google.apphosting.api.ApiProxy $ApplicationException: ApplicationError: 5: http method POST against URLhttp://localhost:8080/admin/gae-tasks/upload/csv/irs/processor.task timed out.] at org.quartz.core.JobRunShell.run(JobRunShell.java:206) at org.quartz.simpl.SimpleThreadPool$WorkerThread.run (SimpleThreadPool.java:520) * Nested Exception (Underlying Cause) --- com.google.apphosting.api.ApiProxy$ApplicationException: ApplicationError: 5: http method POST against URLhttp://localhost:8080/admin/gae-tasks/upload/csv/irs/processor.task timed out. at com.google.appengine.api.urlfetch.dev.LocalURLFetchService.fetch (LocalURLFetchService.java:214) at com.google.appengine.api.labs.taskqueue.dev.UrlFetchJob.execute (UrlFetchJob.java:53) at com.google.appengine.api.labs.taskqueue.dev.UrlFetchJob.execute (UrlFetchJob.java:46) at org.quartz.core.JobRunShell.run(JobRunShell.java:195) at org.quartz.simpl.SimpleThreadPool$WorkerThread.run (SimpleThreadPool.java:520) it implies that there is some form of timeout occuring ... since the app code has not changed and is not apparently getting invoked at all, I am somewhat at loss to know how to resolve this! Has anyone else seen these problems? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: problems with SDK 1.2.8 and (local) task queues timing out...
Regarding the per-request quota Are you worried about the number of requests or the duration of those requests? On Mon, Dec 7, 2009 at 11:17 AM, Larry Cable larry.ca...@gmail.com wrote: Hi Max, after sleeping on it, it became apparent that as you describe the timeout is in fact in the client and not the execution of the (servlet) task itself ... On Dec 7, 10:54 am, Max Ross (Google) maxr+appeng...@google.commaxr%2bappeng...@google.com wrote: Local task execution is using the local UrlFetchService implementation with its default timeout: 5 seconds. So, if the url that your task hits takes more than 5 seconds to respond you'll see this exception. However, this exception is from the client of the task url, not the task execution itself. The client will timeout after 5 seconds but the task should still execute to completion - we don't interrupt it. Please let me know if that's not the case. so I concur with your assertion above ... Now, 5 seconds is clearly not enough time since the request limit for apps is 30 seconds. This is a bug, and it's already fixed. It should go out with the next SDK. In the meantime just try to ignore the timeouts. great thanks ... Since I am using GAE-VFS and tasks in order to upload, parse and persist data via JDO into the DataStore, I anticipate that the Servlet/Task will potentially exceed the per-request quota and throw ... I was planning on catching this and scheduling additional tasks until all the file processing is complete ... any thoughts on this ... do you see any problems in doing so? As for retries, this is a known limitation: http://code.google.com/appengine/docs/java/taskqueue/overview.html#Ta... We'll get this brought in line with production shortly. Thanks, and sorry for the trouble. no problem! ... Thanks for the response! Max On Sun, Dec 6, 2009 at 9:57 PM, Tristan sp00s...@gmail.com wrote: I've seen this as well. I've also noticed that if I purposefully make a task fail (by returning any HTTP status code outside of the range 200-299), it doesn't get added back to queue for retrying. This used to work in 1.2.6. -Tristan On Dec 6, 4:18 pm, Larry Cable larry.ca...@gmail.com wrote: I just updated from 1.2.6 to 1.2.8 and my application code has started failing locally. I am using task queue's in order to process uploaded files, this worked in 1.2.6 (although you had to manually fire the task queue from the _ah admin console) Now, it is failing under 1.2.8 (as it seems as though it is now automatically starting the queue) with the following exception: Dec 7, 2009 12:08:57 AM org.quartz.core.JobRunShell run SEVERE: Job default.a7f0abc6-ed81-4fa4-96f1-c09048a99883 threw an unhandled Exception: com.google.apphosting.api.ApiProxy$ApplicationException: ApplicationError: 5: http method POST against URLhttp://localhost:8080/admin/gae-tasks/upload/csv/irs/processor.task timed out. at com.google.appengine.api.urlfetch.dev.LocalURLFetchService.fetch (LocalURLFetchService.java:214) at com.google.appengine.api.labs.taskqueue.dev.UrlFetchJob.execute (UrlFetchJob.java:53) at com.google.appengine.api.labs.taskqueue.dev.UrlFetchJob.execute (UrlFetchJob.java:46) at org.quartz.core.JobRunShell.run(JobRunShell.java:195) at org.quartz.simpl.SimpleThreadPool$WorkerThread.run (SimpleThreadPool.java:520) Dec 7, 2009 12:08:57 AM org.quartz.core.ErrorLogger schedulerError SEVERE: Job (default.a7f0abc6-ed81-4fa4-96f1-c09048a99883 threw an exception. org.quartz.SchedulerException: Job threw an unhandled exception. [See nested exception: com.google.apphosting.api.ApiProxy $ApplicationException: ApplicationError: 5: http method POST against URLhttp://localhost:8080/admin/gae-tasks/upload/csv/irs/processor.task timed out.] at org.quartz.core.JobRunShell.run(JobRunShell.java:206) at org.quartz.simpl.SimpleThreadPool$WorkerThread.run (SimpleThreadPool.java:520) * Nested Exception (Underlying Cause) --- com.google.apphosting.api.ApiProxy$ApplicationException: ApplicationError: 5: http method POST against URLhttp://localhost:8080/admin/gae-tasks/upload/csv/irs/processor.task timed out. at com.google.appengine.api.urlfetch.dev.LocalURLFetchService.fetch (LocalURLFetchService.java:214) at com.google.appengine.api.labs.taskqueue.dev.UrlFetchJob.execute (UrlFetchJob.java:53) at com.google.appengine.api.labs.taskqueue.dev.UrlFetchJob.execute (UrlFetchJob.java:46) at org.quartz.core.JobRunShell.run(JobRunShell.java:195) at org.quartz.simpl.SimpleThreadPool$WorkerThread.run (SimpleThreadPool.java:520) it implies that there is some form of timeout occuring ... since
Re: [appengine-java] Re: problems with SDK 1.2.8 and (local) task queues timing out...
Your approach sounds fine. As an alternative you could schedule the next task up front, then check to see if there is any work to be done, then proceed. That way you'll always have the next task scheduled no matter what might go wrong during processing. On Mon, Dec 7, 2009 at 2:19 PM, Larry Cable larry.ca...@gmail.com wrote: sorry I should have been more precise, essentially I am concerned about blowing the 30sec per request processing/duration limit previously I was bulk uploading csv files and parsing them into POJOs then persisting them via JDO to pre-populate my app data ... I was encountering both the 5sec DS operation and 30sec processing quota exceptions with this model ... So, all I do initially is upload the CSV in Gae VFS ... then I schedule a task the task then batches lines from the file, parses, then pesists them ... if it gets an exception then it quickly (I hope) (repeatedly) reschedules another task to upload ... Thanks - Larry On Dec 7, 11:22 am, Max Ross (Google) maxr+appeng...@google.commaxr%2bappeng...@google.com wrote: Regarding the per-request quota Are you worried about the number of requests or the duration of those requests? On Mon, Dec 7, 2009 at 11:17 AM, Larry Cable larry.ca...@gmail.com wrote: Hi Max, after sleeping on it, it became apparent that as you describe the timeout is in fact in the client and not the execution of the (servlet) task itself ... On Dec 7, 10:54 am, Max Ross (Google) maxr+appeng...@google.commaxr%2bappeng...@google.com maxr%2bappeng...@google.com maxr%252bappeng...@google.com wrote: Local task execution is using the local UrlFetchService implementation with its default timeout: 5 seconds. So, if the url that your task hits takes more than 5 seconds to respond you'll see this exception. However, this exception is from the client of the task url, not the task execution itself. The client will timeout after 5 seconds but the task should still execute to completion - we don't interrupt it. Please let me know if that's not the case. so I concur with your assertion above ... Now, 5 seconds is clearly not enough time since the request limit for apps is 30 seconds. This is a bug, and it's already fixed. It should go out with the next SDK. In the meantime just try to ignore the timeouts. great thanks ... Since I am using GAE-VFS and tasks in order to upload, parse and persist data via JDO into the DataStore, I anticipate that the Servlet/Task will potentially exceed the per-request quota and throw ... I was planning on catching this and scheduling additional tasks until all the file processing is complete ... any thoughts on this ... do you see any problems in doing so? As for retries, this is a known limitation: http://code.google.com/appengine/docs/java/taskqueue/overview.html#Ta. .. We'll get this brought in line with production shortly. Thanks, and sorry for the trouble. no problem! ... Thanks for the response! Max On Sun, Dec 6, 2009 at 9:57 PM, Tristan sp00s...@gmail.com wrote: I've seen this as well. I've also noticed that if I purposefully make a task fail (by returning any HTTP status code outside of the range 200-299), it doesn't get added back to queue for retrying. This used to work in 1.2.6. -Tristan On Dec 6, 4:18 pm, Larry Cable larry.ca...@gmail.com wrote: I just updated from 1.2.6 to 1.2.8 and my application code has started failing locally. I am using task queue's in order to process uploaded files, this worked in 1.2.6 (although you had to manually fire the task queue from the _ah admin console) Now, it is failing under 1.2.8 (as it seems as though it is now automatically starting the queue) with the following exception: Dec 7, 2009 12:08:57 AM org.quartz.core.JobRunShell run SEVERE: Job default.a7f0abc6-ed81-4fa4-96f1-c09048a99883 threw an unhandled Exception: com.google.apphosting.api.ApiProxy$ApplicationException: ApplicationError: 5: http method POST against URLhttp://localhost:8080/admin/gae-tasks/upload/csv/irs/processor.task timed out. at com.google.appengine.api.urlfetch.dev.LocalURLFetchService.fetch (LocalURLFetchService.java:214) at com.google.appengine.api.labs.taskqueue.dev.UrlFetchJob.execute (UrlFetchJob.java:53) at com.google.appengine.api.labs.taskqueue.dev.UrlFetchJob.execute (UrlFetchJob.java:46) at org.quartz.core.JobRunShell.run(JobRunShell.java:195) at org.quartz.simpl.SimpleThreadPool$WorkerThread.run (SimpleThreadPool.java:520) Dec 7, 2009 12:08:57 AM org.quartz.core.ErrorLogger schedulerError SEVERE: Job (default.a7f0abc6-ed81-4fa4-96f1-c09048a99883
