Hi Alexander, I'm sorry to disagree with Robert, but the zig zag merge-join doesn't return the various rows for each index back to the application code to combine them in the application. It's a difficult algorithm to explain in an email but basically the problem looks to be that with the 90000 entities and < 500 entites case you have too few overlapping entites in the two sets. Basically, think of it as you start off with 90000 possible candidates that could be part of the result set. Now, the zig zag merge join goes and checks the other list (the less than 500 list), but it basically does this one at a time. Every time it doesn't find a match it throws that candidate away and goes on to the next. It has to do this for each equality filter so if you have 4 then it has to check for indexes for each candidate. You can see that this can take some time. Now if it finds enough matches fast enough to satisfy the query limit that is requested to be returned then all is fine, but if it has to keep throwing candidates away and trying new ones than it times out and reports that it needs an index. So with your queries that have large individual results (90000, 50000 and 50000) you obviously have a large overlapping set and it finds enough matches very quickly, but when you have large sets combined with small sets (90000 and <500) then you have few overlaps but lots of checking to do. I hope this helps and if I misinterpreted Robert's post then I apologize to him. Stephen
On Thu, Apr 14, 2011 at 8:36 AM, Alexandru Farcaş < alex.far...@expert-group.biz> wrote: > Hi Robert, > Thanks for your answer. > If the number of results returned by each filter individually is > smaller than, let's say 1000, the "final" number of results will be > returned correctly? > I want to know if there are some known limits for this type of query > on a single list property (maybe for the size of the list, for the > number of the results returned by each filter, for the number of > filters, etc...). I didn't find the answer for this question. > In all queries that returns datastore need index exception I have one > filter that returns almost 90000 entities and one that returns < 500 > results. > But I have also many queries with 3 filters (90000 results, 50000 > results, 50000 results) that returns the correct results. > > > Hi Andrei, > Yes, almost the same queries: the difference is that in the second > table I have a single list property. This list contains the rest of > fields from the first table. > I tried to limit the number of the results from datastore viewer but > with the same result :(. > > > On Apr 14, 5:03 pm, Andrei Cosmin Fifiiţă <andrei.fifi...@gmail.com> > wrote: > > And also, i limit the number of results (for ex when implementing > > pagination)... Does that affect the success of the query ? > > > > 2011/4/14 Andrei Cosmin Fifiiţă <andrei.fifi...@gmail.com>: > > > > > > > > > > > > > > > > > I don't want to create custom indices cause they will fail... but does > the > > > order of the equality filters matter? > > > > > On Apr 14, 2011 3:38 AM, "Robert Kluin" <robert.kl...@gmail.com> > wrote: > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
