Hi Jason, Thanks for filing this issue. These are all good suggestions. You can do a large portion of this analysis today by collecting performance metrics. I would suggest statistically sampling a % of your queries in the following way:
eq_properties = a set of the property names that have equality filters on them sort_order = a ordered list of the sort orders has_ancestor = if the query has an ancestor filter key = (has_ancestor, eq_properties, sort_order) then sample: key -> occurrences key -> time / results This way you can find and optimize slow frequently used queries (and monitor the performance over time). - Alfred On Sep 26, 8:08 am, Jason Collins <jason.a.coll...@gmail.com> wrote: > We've been working on a "billing optimization project" for our large > app, steprep. Right now, we are tackling index optimizations - > datastore writes form the largest part of our bill. > > Zig zag merge has let us do some really crazy and unexpected things - > in a good way. Thanks! > > However, we're somewhat concerned with the long term health of these > queries. That is, today, they work and will offer us substantial > savings over previous index approaches. But algorithm changes and data > distribution changes may make our queries perform quite differently > over time. (We have the additional difficulty of our test deployment > having a different data distribution than our prod deployment, so we > kind of have to cross our fingers and hope for the best when dropping > indexes on prod...) > > It would be great to get some meta information back from a query to > help track the efficiency of our zig zag merge queries over time > (i.e., as data distributions change). E.g.: > > - which indexes were used to generate the results > - how much zig-zag "thrashing" occurred (perhaps how many entries you > needed to consider to find the results) > - tell the query planner to _not_ use a particular index, so that we > can simulate the drop of an optimal index to test if zig zag will be > efficient > > I'm really excited about what we can do with zig zag, but we have > concerns about how we will keep it healthy over time. > > I've openedhttp://code.google.com/p/googleappengine/issues/detail?id=5978 > for this request; please star it if you feel it would be helpful. > > Thanks, > j -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
