hello everyone, i want to provide detailed reports in my app. those reports require data aggregated across full tables. in the client i want to plot the change of values over time. this then requires to select very huge data sets. however, in the app engine there is the restriction to get 1000 items in one request. so what is the best approach to do queries with big result sets?
i can imagine that the client could split the big request into 10 smaller ones to collect, merge and display the data. but i think this is complex, error prone and slow. i would rather like to do this on the server side. but do i have to split the requests to work around the 1000 items limit or is there another pattern that developers should use to handle huge datasets. basically i want to know two things: 1) how can i retrieve huge amounts of data regarding the get 1000 items limit? 2) is the app engine performance good enough to allow showing reports on the client side i.e. can app engine retrieve and analyse (avg,min,max,grouping) huge amounts of data (10k - 1million rows) fast enough? one more thing: is it possible to reduce the amount of data that is returned dramatically by doing all the aggregations (e.g. by grouping and summing) completely within the datastore? currently i am experimenting with doing full table scans (but i can only retrieve 1000 items in one request..) and aggregating the data within the application (on server side). kind regards, ingo -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
