Pervasive use of memcache + exponential backoff retries on most operations solved it for me.
On Sun, Apr 12, 2009 at 11:55 PM, Paul Kinlan <paul.kin...@gmail.com> wrote: > Hi Guys, > > My app is Twitterautofollow. I have a question about the quota, basically > my app was serving between 6-13 requests a second and jumped up to 32 > requests per-second and subsequently went over the quota. I am not sure > where the 32 requests a second are comming from although some of them might > come from my ping service that I am running to regularly perform some tasks > - I wouldn't be suprised if it was a bug I created > > Additionally the DataStore CPU Time is Limited even though it is only at 3% > of quota. > > Its starting to get a bit frustrating at the moment because I am having > Data Store Timeouts very often on reads and puts. Nothing in my model is in > an EntityGroup, that is, there is no use of parent, however there are many > RefernceProperties. > > The general process I have that is causing the process goes as follows > > > 1. Get the user (User Entity) from the datastore > 2. Get the current search term (Search Entity) for the user - I don't > use the refernce propery set from the user because I need to filter it > 1. Query Twitter > 2. For up to 3 search results add a new entity of type "Follow" and > reference the search and user > 1. For each result check to see if the "Follow" entity already > exists for the user - if it does we ignore the result > 3. update the search entity with some basic stats > > Overall there are, with 5 (1 user, 1 search and 3 reads of Follow) reads > and up to 4 puts (3 for new entities 1 for the "Search" entity). I don't > think this is too heavy, but it might be. > > So my question is, am I being too excessive, why would this cause a lot of > datastore timeouts in both the reads and puts? What tips do people have for > DataStore performance? > > Thanks, > Paul > > > > -- Alkis --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~----------~----~----~----~------~----~------~--~---
