The issue here is zig zag merge join. It's a bit more complex subject than I
can easily explain in email, but to sum things up, what's happening is that
we take multiple indexes and zig-zag between them to generate your result
set. When this takes too long, that causes that exception to be thrown. Your
query is using the searchables index multiple times. The way an AND query
works is by making multiple queries and merging the results by zig-zagging
between them. The query engine is doing this:
- Go to index pointing to an entity where searchables = hidden:::false
- Now go to index pointing to an entity where searchables = sex
- Now go to index pointing to an entity where searchables = repeat
...
- Is there an entity that matches all these properties? No. Repeat the
process.
After traversing a large amount of zig/zags, the query engine will determine
that you've attempted a query that is too complex and needs composite
indices. This is much more likely to happen if the combination of the two
facts is true:
1. You have a large amount of entities that do NOT match
2. You have a large amount of properties to match
The reason is that most of your index traversals are going to result in
nothing being returned.
If this sounds complex, it *is*, but some visuals may help. You'll want to
look at the links below:
http://code.google.com/appengine/articles/datastore/overview.html
http://www.youtube.com/watch?v=AgaL6NGpkB8
A solution to your problem may be simply to whittle down the result set to
as small as you can reasonably get it, then traverse the entities in
memory.
We're currently working on a solution that would at least return a subset of
the results so this exception isn't thrown, and you'll be able to resume the
query from the returned cursor, but this won't be available for a few
releases.
On Thu, Sep 16, 2010 at 10:17 AM, Nickolay Tzvetinov wrote:
> An exact example is "SELECT __key__ FROM SearchIndex WHERE searchables
> = ^_hidden:::false_^ AND searchables = sex AND searchables = repeat
> AND searchables = mate AND searchables = feed AND searchables = sugar
> AND searchables = rating:1".
>
> Executing this query I get the exception when the entities in which I
> search are about 5000...
>
> Best regarts,
> Nickolay Georgiev (Meddle)
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine for Java" group.
> To post to this group, send email to
> google-appengine-j...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine-java+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/google-appengine-java?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
