Re: [google-appengine] Re: Large difference between datastore quota detail and appstat.

2011-11-18 Thread djidjadji
Does ndb support lists in get_by_id()? This could then be executed in parallel.
You should then check with appstats on the GAE hardware if it works in parallel.

db_keys = EMail.query(EMail.Status != 'Complete').fetch(keys_only=True)
res = EMail.get_by_id( [k.id() for k in db_keys] )
list comprehensions are faster then append() calls

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.



[google-appengine] Re: Large difference between datastore quota detail and appstat.

2011-11-17 Thread Simon Knott
How many entities are you pulling back in your queries?  Each query costs 1 
read for the query, plus one read for each entity returned (I believe).

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/4NruFnQsNmAJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.



Re: [google-appengine] Re: Large difference between datastore quota detail and appstat.

2011-11-17 Thread Timofey Koolin
...
Othen request can read up to 30 entity it execute about 350 times for
screenshots in first email.

2011/11/18 Timofey Koolin timo...@koolin.ru

 In query I always fetch only keys (about 40-60), then I fetch every entity
 by get_by_id, and almost all fetched from memcache (I use ndb).
 My code:
 db_keys = EMail.query(EMail.Status != 'Complete').fetch(keys_only=True)
 keys = []
 for key in db_keys:
 keys.append(key.id())
 res = []
 for key in keys:
   res.append(EMail.get_by_id(key))

 app-stat timeline in attached screenshot.
 It sync request execute every one minute.

 Othen request can read up to 30 entity it execute about 350 times for this
 screenshot. Code:
 channels = ChannelInfo.query()
 for channel in channels:
   ...

 2011/11/18 Simon Knott knott.si...@gmail.com

 How many entities are you pulling back in your queries?  Each query costs
 1 read for the query, plus one read for each entity returned (I believe).

 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To view this discussion on the web visit
 https://groups.google.com/d/msg/google-appengine/-/4NruFnQsNmAJ.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.




 --
 Blog: www.rekby.ru




-- 
Blog: www.rekby.ru

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.