Here's a link: http://code.google.com/p/appengine-mapreduce/
On Fri, Sep 10, 2010 at 8:58 AM, Robert Kluin <robert.kl...@gmail.com>wrote: > You might want to look into the mapper api. It is designed to work > with large datasets, so it may be a good fit. > > > Robert > > > > > On Fri, Sep 10, 2010 at 07:11, Sümer Cip <sum...@gmail.com> wrote: > > I have 1 million recordes. I am doing statitical work with the data > field. > > What I want to do is to work in batches(with crons) with my data. That's > > all. > > > > On Fri, Sep 10, 2010 at 6:56 AM, Robert Kluin <robert.kl...@gmail.com> > > wrote: > >> > >> What is "many entities," is it 100, 1,000, 1,000,000, more? What are > >> you doing with the info field? Are you aggregating something? > >> Transforming it? Writing one new model for each entity? > >> > >> What do you mean continue another day? Do you mean process some > >> entities then stop and continue processing on another day? > >> > >> > >> > >> Robert > >> > >> > >> > >> > >> > >> > >> > >> > >> On Thu, Sep 9, 2010 at 22:34, Sumer Cip <sum...@gmail.com> wrote: > >> > Hi, > >> > I have a very basic problem. Below is my model: > >> > > >> > class DummyModel: > >> > id:Integer > >> > info:Integer > >> > added:DateTime > >> > > >> > What I need to do is to have a cron job running per day to filter on > >> > added field and do something with the info field, but I have many > >> > entities and I want to make this process in batches maybe continue > >> > another day. How to do that efficiently? Cursors? Can we use them with > >> > cron jobs? > >> > > >> > Any example suggestion would be appreciated. > >> > > >> > Thanks, > >> > > >> > -- > >> > You received this message because you are subscribed to the Google > >> > Groups "Google App Engine" group. > >> > To post to this group, send email to > google-appeng...@googlegroups.com. > >> > To unsubscribe from this group, send email to > >> > google-appengine+unsubscr...@googlegroups.com<google-appengine%2bunsubscr...@googlegroups.com> > . > >> > For more options, visit this group at > >> > http://groups.google.com/group/google-appengine?hl=en. > >> > > >> > > >> > >> -- > >> You received this message because you are subscribed to the Google > Groups > >> "Google App Engine" group. > >> To post to this group, send email to google-appeng...@googlegroups.com. > >> To unsubscribe from this group, send email to > >> google-appengine+unsubscr...@googlegroups.com<google-appengine%2bunsubscr...@googlegroups.com> > . > >> For more options, visit this group at > >> http://groups.google.com/group/google-appengine?hl=en. > >> > > > > > > > > -- > > Sumer Cip > > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to google-appeng...@googlegroups.com. > > To unsubscribe from this group, send email to > > google-appengine+unsubscr...@googlegroups.com<google-appengine%2bunsubscr...@googlegroups.com> > . > > For more options, visit this group at > > http://groups.google.com/group/google-appengine?hl=en. > > > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com<google-appengine%2bunsubscr...@googlegroups.com> > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- Ikai Lan Developer Programs Engineer, Google App Engine Blog: http://googleappengine.blogspot.com Twitter: http://twitter.com/app_engine Reddit: http://www.reddit.com/r/appengine -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
