A batch put does one round trip to the datastore and if your entities end up
stored on different machines it will perform those in parallel.
memcache alone won't save you from timeouts. If you timeout one per hundred
db ops and you do 100 ops per sec you will see a timeout one per sec. If
memcache reduces your db ops to 10 per sec you will see a timeout once per
10 secs. To eliminate the timeouts you need to wrap your db ops with
retries.
On Mon, Apr 13, 2009 at 10:52 AM, Paul Kinlan <paul.kin...@gmail.com> wrote:
> Hi,
>
> I have added some use of memcache, however I am still seeing timeouts at
> the moment - although it has just said I need a composite index for another
> part of my application.
>
> What I don't quite understand is that it appears to be "puts" of new
> entities that are timing out and I can't work out why, I do individual
> writes of 3 entities and I was thinking of batching these together in a
> db.put, but from what I remember that is only a convienince method and
> essentially performs the same task.
>
> Paul
>
> 2009/4/13 Paul Kinlan <paul.kin...@gmail.com>
>
> Hi,
>>
>> I will look at using memcache again, it might help a little as I do
>> perform some http requests against a user sequentially so it might help a
>> bit. The problem is that I see a lot of timeouts on puts at the moment and
>> I am not sure how to remedy them.
>>
>> Paul.
>>
>> 2009/4/12 Alkis Evlogimenos ('Αλκης Ευλογημένος) <evlogime...@gmail.com>
>>
>> Pervasive use of memcache + exponential backoff retries on most operations
>>> solved it for me.
>>>
>>> On Sun, Apr 12, 2009 at 11:55 PM, Paul Kinlan <paul.kin...@gmail.com>wrote:
>>>
>>>> Hi Guys,
>>>>
>>>> My app is Twitterautofollow. I have a question about the quota,
>>>> basically my app was serving between 6-13 requests a second and jumped up
>>>> to
>>>> 32 requests per-second and subsequently went over the quota. I am not sure
>>>> where the 32 requests a second are comming from although some of them might
>>>> come from my ping service that I am running to regularly perform some tasks
>>>> - I wouldn't be suprised if it was a bug I created
>>>>
>>>> Additionally the DataStore CPU Time is Limited even though it is only at
>>>> 3% of quota.
>>>>
>>>> Its starting to get a bit frustrating at the moment because I am having
>>>> Data Store Timeouts very often on reads and puts. Nothing in my model is
>>>> in
>>>> an EntityGroup, that is, there is no use of parent, however there are many
>>>> RefernceProperties.
>>>>
>>>> The general process I have that is causing the process goes as follows
>>>>
>>>>
>>>> 1. Get the user (User Entity) from the datastore
>>>> 2. Get the current search term (Search Entity) for the user - I
>>>> don't use the refernce propery set from the user because I need to
>>>> filter it
>>>> 1. Query Twitter
>>>> 2. For up to 3 search results add a new entity of type "Follow"
>>>> and reference the search and user
>>>> 1. For each result check to see if the "Follow" entity already
>>>> exists for the user - if it does we ignore the result
>>>> 3. update the search entity with some basic stats
>>>>
>>>> Overall there are, with 5 (1 user, 1 search and 3 reads of Follow) reads
>>>> and up to 4 puts (3 for new entities 1 for the "Search" entity). I don't
>>>> think this is too heavy, but it might be.
>>>>
>>>> So my question is, am I being too excessive, why would this cause a lot
>>>> of datastore timeouts in both the reads and puts? What tips do people have
>>>> for DataStore performance?
>>>>
>>>> Thanks,
>>>> Paul
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>>
>>> Alkis
>>>
>>>
>>>
>>
>
> >
>
--
Alkis
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~----------~----~----~----~------~----~------~--~---
