[google-appengine] Re: GQL performance slow

2011-03-23 Thread adhi
Hi Ikai see the a href=http://i52.tinypic.com/28hndxk.jpg; target=_blankapp stats/a @905ms datastore_v3.RunQuery real=1090ms api=8225ms @2504ms datastore_v3.Next real=339ms api=4711ms my q.fetch(2000) method takes ~3.5 seconds On Mar 23, 12:42 am, Ikai Lan (Google) ika...@google.com wrote: I

[google-appengine] Re: GQL performance slow

2011-03-22 Thread adhi
Hi Ikai, is there a measure to say the entities are large? is it about the number of columns or blob or text? if so in my entities I have string and number properties only. Yes. I need all the 2000 (some times bit more) entities at once for reporting. Currently I'm not seeing a way where I can

Re: [google-appengine] Re: GQL performance slow

2011-03-22 Thread Ikai Lan (Google)
I think you should run AppStats and see what's going on. 2000 entities is never going to be particularly fast, but 3 seconds might be a bit on the high side. Run the tool to be sure. Ikai Lan Developer Programs Engineer, Google App Engine Blog: http://googleappengine.blogspot.com Twitter:

[google-appengine] Re: GQL performance slow

2011-03-21 Thread adhi
q = db.Query(PrimaryData) q.filter('SheetMetadataId', metadata.getSheetId()) return q.fetch(2000) Hi Ikai, even the above query takes 3.3 ~ 3.6 seconds. The number of entities returned by the query is 1261. I'm using Expando model, the total number columns for this particular set of entities are

Re: [google-appengine] Re: GQL performance slow

2011-03-21 Thread Ikai Lan (Google)
If that's the performance, then I'm inclined to say the entities are large and that's about the performance you can expect. One more thing to keep in mind: the cost of deserializing each entity is non-trivial. Is there a reason you need all 2000 entities? Perhaps there is a different way to

[google-appengine] Re: GQL performance slow

2011-03-17 Thread adhi
Hi Robert, I've not used reference properties. Its a simple query and it'll be like this. q = db.Query(PrimaryData) q.filter('SheetMetadataId', metadata.getSheetId()) q.filter('InstanceAssignedTo IN', [u'User_c42e8919_448e_11e0_b87b_f58d20c6e2c3', u'User_1fd87ac5_073d_11e0_8ba1_c122a5867c4a'])

Re: [google-appengine] Re: GQL performance slow

2011-03-17 Thread Ikai Lan (Google)
If you have a composite index, the performance of this query should be mostly linear (find start location of index and just return min(2000, number of entities that satisfy query from there on out) number of keys and items. If you only have individual indexes for SheetMetadataId as well as