If that's the performance, then I'm inclined to say the entities are large and that's about the performance you can expect. One more thing to keep in mind: the cost of deserializing each entity is non-trivial.
Is there a reason you need all 2000 entities? Perhaps there is a different way to accomplish what you want without querying a (relatively) large number of entities at once. One other possible solution that will at least make your user facing code run faster is to split the query up by key, perform multiple async queries and join them at the end. It'll still consume the same (or more) CPU time, but the user ms should be lower since you are doing things in parallel. The tradeoff here, of course, is that you'd need to know how you can split up the keyspace. Ikai Lan Developer Programs Engineer, Google App Engine Blog: http://googleappengine.blogspot.com Twitter: http://twitter.com/app_engine Reddit: http://www.reddit.com/r/appengine On Mon, Mar 21, 2011 at 12:17 AM, adhi <adhinaraya...@gmail.com> wrote: > q = db.Query(PrimaryData) > q.filter('SheetMetadataId', metadata.getSheetId()) > return q.fetch(2000) > > Hi Ikai, even the above query takes 3.3 ~ 3.6 seconds. The number of > entities returned by the query is 1261. > I'm using Expando model, the total number columns for this particular > set of entities are 25. > Yes. I've created composite index. Here is the index definition. > > - kind: PrimaryData > properties: > - name: SheetMetadataId > - name: InstanceAssignedTo > > please let me know if you need more information. > > > On Mar 17, 11:50 pm, "Ikai Lan (Google)" <ika...@google.com> wrote: > > If you have a composite index, the performance of this query should be > > mostly linear (find start location of index and just return min(2000, > number > > of entities that satisfy query from there on out) number of keys and > items. > > If you only have individual indexes for SheetMetadataId as well as > > InstanceAssignedTo, this uses zigzag merge join, where performance can > > greatly vary depending on the shape of the data. > > > > Just out of curiosity, what is the performance of this query? > > > > q = db.Query(PrimaryData) > > q.filter('SheetMetadataId', metadata.getSheetId()) > > return q.fetch(2000) > > > > If it's in a similar ballpark, my other guess is that the query takes > that > > amount of time because the entities are large. > > > > You mentioned created a composite index. Can you post that? > > > > I'll talk to the datastore team to see what we can do to provide > something > > similar to EXPLAIN plans for queries. > > > > Ikai Lan > > Developer Programs Engineer, Google App Engine > > Blog:http://googleappengine.blogspot.com > > Twitter:http://twitter.com/app_engine > > Reddit:http://www.reddit.com/r/appengine > > > > > > > > > > > > > > > > On Thu, Mar 17, 2011 at 10:07 AM, adhi <adhinaraya...@gmail.com> wrote: > > > Hi Robert, > > > I've not used reference properties. Its a simple query and it'll be > > > like this. > > > > > q = db.Query(PrimaryData) > > > q.filter('SheetMetadataId', metadata.getSheetId()) > > > q.filter('InstanceAssignedTo IN', > > > [u'User_c42e8919_448e_11e0_b87b_f58d20c6e2c3', > > > u'User_1fd87ac5_073d_11e0_8ba1_c122a5867c4a']) > > > return q.fetch(2000) > > > > > and I replaced the IN filter to '=' with single value. Still its > > > taking same time. > > > > > On Mar 17, 6:27 am, Robert Kluin <robert.kl...@gmail.com> wrote: > > > > Use Appstats. It may not be the query that is slow. If you are using > > > > reference properties, perhaps you are dereferencing them. If you > > > > should us the query and how you're using the results we might be abel > > > > to give more suggestions. > > > > > >http://code.google.com/appengine/docs/python/tools/appstats.html > > > > > > Robert > > > > > > On Mon, Mar 14, 2011 at 10:54, adhi <adhinaraya...@gmail.com> wrote: > > > > > Hi, I'm running a query in appengine which is returning just 1200 > > > > > entities, but its taking 3.5 seconds. The query doesn't contains > > > > > inequality filters, but anyway I added index for that. Can anyone > tell > > > > > me how to analyse this and improve theperformance? > > > > > > > -- > > > > > You received this message because you are subscribed to the Google > > > Groups "Google App Engine" group. > > > > > To post to this group, send email to > google-appengine@googlegroups.com > > > . > > > > > To unsubscribe from this group, send email to > > > google-appengine+unsubscr...@googlegroups.com. > > > > > For more options, visit this group athttp:// > > > groups.google.com/group/google-appengine?hl=en. > > > > > -- > > > You received this message because you are subscribed to the Google > Groups > > > "Google App Engine" group. > > > To post to this group, send email to google-appengine@googlegroups.com > . > > > To unsubscribe from this group, send email to > > > google-appengine+unsubscr...@googlegroups.com. > > > For more options, visit this group at > > >http://groups.google.com/group/google-appengine?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
