Hi Alfred Fuller, Thanks for your information, videos gave me lot of information. As per your suggestion I have added few composite index to avoid this exception still index are getting built. As we get exception on some queries, will add those to composite index. But this will eat my lot of CPU quota. Could you please tell me when will this restriction on query will be removed because we will plan our product enhancement accordingly.
Thanks, Ravi On Jun 15, 4:57 am, Alfred Fuller <arfuller+appeng...@google.com> wrote: > Yes, you can create composite indexes that contain columns that are often > queried together. The merge join algorithm will use these indexes to make > queries that include those specific columns more efficient. Or if you see > this error more often on some queries than others, create composite indexes > for those queries (or the common columns between those queries) > > http://www.youtube.com/watch?v=ofhEyDBpngMat time 30:30 gives an example of > this. > > > > On Mon, Jun 14, 2010 at 7:58 AM, Ravi <msr...@gmail.com> wrote: > > Hi Geoffrey Spear, > > Thanks for your clarification. > > We suspected the same reason. Last few days back our data was less > > around 20,000 rows there was no problem. > > Only when more data is added we are facing this problem. > > > Our table is too large which is around 50 fields/columns and around > > 50,000 rows. Query filter is also dynamically generated (user selects > > few fields and submits). > > Our servlet will query database depending on the user selected fields. > > In this scenario we dont know which fields will be selected by user > > and certainly I cant have composite index for 50 combinations. > > > Is there any work around for this problem?? > > > Thanks, > > Ravi > > > On Jun 14, 5:59 pm, Geoffrey Spear <geoffsp...@gmail.com> wrote: > > > It's not a problem with indexing, it's a problem with your data. If > > > you don't add composite indexes, the system will attempt to serve > > > queries that contain only equality filters using a merge-join > > > strategy, which is less efficient than a simple index lookup (with the > > > benefit of not needing to add the indexes, which use space and write- > > > time CPU). If your data becomes such that the query can not be done > > > efficiently with the merge-join anymore, you'll start to see that > > > exception and you'll need to add a composite index. > > > > On Jun 12, 2:00 am, Ravi <msr...@gmail.com> wrote: > > > > > Hi Google Team, > > > > > All these days our application was working fine and we have not > > > > changed any datastore index or codebase of our application. > > > > Everyday we upload new set of data to our application. Today also we > > > > have uploaded new set of data but today after uploading the data we > > > > are getting very strange exception as below: > > > > > com.google.appengine.api.datastore.DatastoreNeedIndexException: The > > > > built-in indices are not efficient enough for this query and your > > > > data. Please add a composite index for this query.. An index is > > > > missing but we are unable to tell you which one due to a bug in the > > > > App Engine SDK. If your query only contains equality filters you most > > > > likely need a composite index on all the properties referenced in > > > > those filters. > > > > > We are worried after seeing the above exception, we dont have any > > > > complex queries, all the queries contain equals to filter (==). > > > > We tested the same set of data and query in our development > > > > environment, it is working fine. > > > > For testing purpose, we uploaded the same set of data to our other > > > > testing appengine applcation, it is working fine there also. > > > > > Only in our production system, we are getting > > > > DatastoreNeedIndexException. > > > > > Could you please confirm is there any problem with the datastore > > > > indexing. > > > > Data stored in our production datastore is more than 0.5 GB, will > > > > thee be any problem in indexing if there is high amount of data. > > > > > Application ID: bigpaisa2.appspot.com > > > > Latest application version: > >http://061102.latest.bigpaisa2.appspot.com > > > > - In this version DatastoreNeedIndexException is occuring. > > > > > Please help us as soon as possible. > > > > > Thanks, > > > > Ravi > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to google-appeng...@googlegroups.com. > > To unsubscribe from this group, send email to > > google-appengine+unsubscr...@googlegroups.com<google-appengine%2Bunsubscrib > > e...@googlegroups.com> > > . > > For more options, visit this group at > >http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
