Hi, Using tasks to iterate over a big amount data is the way to go:
a) a first task or interactive request splits the amount of data in chunks defined for example by key range or by range of another property of the object. b) then it launches a task per range (i.e the range is a param of the task) to do what has to be done on each object. Each task has 10 min to complete. If it gets near the limit of 10 min, it launches a new task to contnue the job with the initial range minus what has already been done. This until end of range. What very interesting with tasks is the parallelismn, if your parametrize queue.xml accordingly, you will see many jvms launched by GAE to accommodate in terms of CPU power the parallelism you want to reach (I have sometimes 12 - 15 jvms in my own case for such cases). So, instead of lasting hours, your job can be run in minutes. Important point: the ranges have to be defined in relation with entity groups (if you have any), else you may have heavy datastore contention issues. regards didier On Jan 21, 7:14 pm, "ss.require" <ss.requ...@gmail.com> wrote: > I also have a similar task. > Please, could you clarify in more details how to use Task Queue API > and datastore cursors? it is also good that task execution's limit is > 10 minute now. > > On 21 Січ, 20:03, mge...@gmail.com wrote: > > > For sure Mapper API is a really good choice. Another option would be using > > Task Queue API and datastore cursors. Both options will solve your problem. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
