I forgot to include this in the previous message.
I am reticent to do retries because the cases where I see a datastore
timeout is when it has taken 8000ms to do a put.
However, I might as well give it a go.
Paul
On 13 Apr 2009, at 13:13, Alkis Evlogimenos ('Αλκης Ευλογημένος) <evlog
ime...@gmail.com> wrote:
> A batch put does one round trip to the datastore and if your
> entities end up stored on different machines it will perform those
> in parallel.
>
> memcache alone won't save you from timeouts. If you timeout one per
> hundred db ops and you do 100 ops per sec you will see a timeout one
> per sec. If memcache reduces your db ops to 10 per sec you will see
> a timeout once per 10 secs. To eliminate the timeouts you need to
> wrap your db ops with retries.
>
> On Mon, Apr 13, 2009 at 10:52 AM, Paul Kinlan
> <paul.kin...@gmail.com> wrote:
> Hi,
>
> I have added some use of memcache, however I am still seeing
> timeouts at the moment - although it has just said I need a
> composite index for another part of my application.
>
> What I don't quite understand is that it appears to be "puts" of new
> entities that are timing out and I can't work out why, I do
> individual writes of 3 entities and I was thinking of batching these
> together in a db.put, but from what I remember that is only a
> convienince method and essentially performs the same task.
>
> Paul
>
> 2009/4/13 Paul Kinlan <paul.kin...@gmail.com>
>
> Hi,
>
> I will look at using memcache again, it might help a little as I do
> perform some http requests against a user sequentially so it might
> help a bit. The problem is that I see a lot of timeouts on puts at
> the moment and I am not sure how to remedy them.
>
> Paul.
>
> 2009/4/12 Alkis Evlogimenos ('Αλκης Ευλογημένος) <evlog
> ime...@gmail.com>
>
> Pervasive use of memcache + exponential backoff retries on most
> operations solved it for me.
>
> On Sun, Apr 12, 2009 at 11:55 PM, Paul Kinlan
> <paul.kin...@gmail.com> wrote:
> Hi Guys,
>
> My app is Twitterautofollow. I have a question about the quota,
> basically my app was serving between 6-13 requests a second and
> jumped up to 32 requests per-second and subsequently went over the
> quota. I am not sure where the 32 requests a second are comming
> from although some of them might come from my ping service that I am
> running to regularly perform some tasks - I wouldn't be suprised if
> it was a bug I created
>
> Additionally the DataStore CPU Time is Limited even though it is
> only at 3% of quota.
>
> Its starting to get a bit frustrating at the moment because I am
> having Data Store Timeouts very often on reads and puts. Nothing in
> my model is in an EntityGroup, that is, there is no use of parent,
> however there are many RefernceProperties.
>
> The general process I have that is causing the process goes as follows
>
> Get the user (User Entity) from the datastore
> Get the current search term (Search Entity) for the user - I don't
> use the refernce propery set from the user because I need to filter it
> Query Twitter
> For up to 3 search results add a new entity of type "Follow" and
> reference the search and user
> For each result check to see if the "Follow" entity already exists
> for the user - if it does we ignore the result
> update the search entity with some basic stats
> Overall there are, with 5 (1 user, 1 search and 3 reads of Follow)
> reads and up to 4 puts (3 for new entities 1 for the "Search"
> entity). I don't think this is too heavy, but it might be.
>
> So my question is, am I being too excessive, why would this cause a
> lot of datastore timeouts in both the reads and puts? What tips do
> people have for DataStore performance?
>
> Thanks,
> Paul
>
>
>
>
>
> --
>
> Alkis
>
>
>
>
>
>
>
>
>
> --
>
> Alkis
>
> >
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~----------~----~----~----~------~----~------~--~---
