Collect offline the Facebook data with cron and taskqueue requests. Setup the taskqueue to request only 1 every x seconds, to be sure the previous one terminated. Cache the results of these collects in datastore objects. When the user requests, show the datastore object.
2010/1/13 Tom <sharpbla...@gmail.com>: > Hello! > I recently wrote an app running on GAE. It is a facebook statistics > gatherer, which relies upon the Facebook API to function. > > Recently the app got a lot more users, and the requests per second > "shot" to 4, which is the highest it has ever been. However AppEngine > did not scale well at all, with over 25% of my requests being shot > down because of the "Max Simultaneous request limit", which is said to > be always caused by "Bad coding and long page request times", which is > fair enough. > > The problem is my app relies on a Facebook API call called "feed.get > ()", which returns a (long) JSON response detailing all the users > actions in the last X days, from which my app generates statistics > from. The getting of this data, along with the users friends and the > users groups is very expensive (Takes up 99.9% of the request time). > The facebook API fails as well sometimes as well, as shown by these > logs: > > Request recieved > Stream.get() failed us > Stream.get() ran fine > get_add_url() ran fine > Friends.get() ran fine > Groups.get() ran fine > Network done in 9.25191187859 > Loop and stats created in 0.000972032546997 seconds > Created render info in 0.00014591217041 > Rendered in 0.0445511341095 seconds > Request finished in 9.29821896553 seconds > > > As you can see 9.25 seconds is spent on getting the data only only > 0.04 on processing it. The time taken varies greatly, sometimes it > takes only 2 seconds in total and other times it hits up to 15 (With > multiple API failures). > > My question is how can I try and avoid hitting the max request limit > when it is a third party that is causing my long response times.? > I could use some Ajax calls to get the stats, but that would double > the requests needed (1 request to get the page, another to get the > data). > > Is there any way to increase my request limit? > > Regards > ~Tom > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > > >
-- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
