Hi I wrote importing geoip database in JSF. It has like 100k entries. Besides appengine and datastore performance is awful and importing took around 1,5h of total CPU its fairly easy to use TaskQueue. If there is above average processing to be done then i suggest separating task into smaller batches and process them asynchronously.
Margus On Thu, Feb 25, 2010 at 3:52 AM, Jeff Schnitzer <j...@infohazard.org> wrote: > On Wed, Feb 24, 2010 at 1:06 PM, Ikai L (Google) <ika...@google.com> > wrote: > > My point wasn't necessarily that it wasn't possible. makePersistentAll > does > > use a batch write, and there are definitely sites that can do 12,000+ > writes > > a second (and well above that), but I don't know of any that will attempt > to > > do that in a single request. While it's an interesting thought exercise > to > > see if BigTable can do it through App Engine's interface (hint: it can, > > globally, easily), I can't think of a single use case for a site to need > to > > do this all the time and with the sub-second requirement. I think it's > > reasonable to ask why this design exists and why the requirements exist > and > > rethink one or the other. > > It does seem to be a pretty extreme case, but it's not all that far > fetched. It's possible for a Facebook user to have 5,000 friends. > Perhaps a user wants to message all 5k of them. > > I could actually use this right ability now. I would like to add a > game mechanic which, when you score some points, you also credit a > portion of that to all of a user's friends. Worst case scenario is a > 5,000 element read followed by a 5,000 element write. I'm probably > going to skip this mechanic for now because I can't afford it - even > with the average 200 or so friends. If I want it badly enough, I may > ultimately need to move my scoring system offsite. > > Jeff > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine for Java" group. > To post to this group, send email to > google-appengine-j...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine-java+unsubscr...@googlegroups.com<google-appengine-java%2bunsubscr...@googlegroups.com> > . > For more options, visit this group at > http://groups.google.com/group/google-appengine-java?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
