If your list property is long and you include it multiple times in the same index, it will result in lots and lots of index rows. I suggested one *possible* method you could use to avoid needing to add that index, that might allow the query to run. http://code.google.com/appengine/docs/python/datastore/queries.html#Big_Entities_and_Exploding_Indexes
The *final* number of results returned by the query may be 10 or 100, but how many results are returned by each filter individually? Those results get sent back to your app and combined in code. If lots and lots of entities need fetched to build that final list of 10 results, the query could fail. Robert On Thu, Apr 14, 2011 at 09:28, Ice13ill <andrei.fifi...@gmail.com> wrote: > So, that technique will not work on lists with more than 10-20 words ? > And also, when i have that error too, the number of results in that > query is between 10 and 100 most of the cases... > > On Apr 13, 8:27 pm, Robert Kluin <robert.kl...@gmail.com> wrote: >> Hi Alexandru, >> I suspect some of those key_words are common across many entities. >> You do not have an explicit index defined, so queries are done using >> merge-join. That means a lot of records need returned and processed >> to fullfil your query. You're storing a lot of keywords on each >> entity, so adding an explicit index is going to result in an >> 'explosion' of index rows, and would likely noticeably impact your >> write performance. >> >> I have no idea if you could make it work, but perhaps you could keep >> track of the frequency of your tags, select the most restrictive ones >> in a particular query, use those to execute the query, then filter the >> returned rows in memory with the 'common' search terms. Since you >> know your data and use-cases, perhaps you could optimize this process >> further with some clever 'index' entities of your own. >> >> Robert >> >> >> >> >> >> >> >> On Tue, Apr 12, 2011 at 20:54, Alexandru Farcas <alexfarca...@gmail.com> >> wrote: >> > Hello, i have a problem with default indexes when executing a query >> > with equality filters on a list of properties. >> >> > When querying a table with about 30000 entities I get the response >> > without errors, but after the table reaches 90000 entities the query >> > returns the error "The built-in indices are not efficient enough for >> > this query and your data. ...". >> > The entities have a list of about 30-40 keywords, and the queries have >> > about 3-4 eq filters. >> >> > For example: SELECT __key__ FROM MyKeywords WHERE keyWordsList = 'leg' >> > AND keyWordsList = '53' AND keyWordsList = 'cod' AND entityName = Law >> >> > I migrated from Master/Slave to HR. The error still appeared on the >> > SAME queries. >> > I created another table for the keywords (where i introduced the >> > "entityName" field inside the list of properties): exactly the same >> > behaviour... >> > Can someone tell me what i'm doing wrong ? >> >> > -- >> > You received this message because you are subscribed to the Google Groups >> > "Google App Engine" group. >> > To post to this group, send email to google-appengine@googlegroups.com. >> > To unsubscribe from this group, send email to >> > google-appengine+unsubscr...@googlegroups.com. >> > For more options, visit this group >> > athttp://groups.google.com/group/google-appengine?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
