The query volume doubled since I first started this thread and, at the current rate, should double again by the end of next week. I definitely need a solution that can handle in the thousand QPS because that's where we're heading. I'm currently running this without consistency using a dedicated memcache.
On Thu, Nov 21, 2013 at 2:00 PM, Jim <jeb62...@gmail.com> wrote: > If data points for each entity are not coming too fast, you could use > blobstore/gcs to store your time series for each entity in a blob, then > store a pointer to that blob in your entity in the data store. updating is > expensive but can run off a task queue. retrieval of the blobs is very > fast, and then you can quicky parse the blob into memory and compute your > stats on a given entity. cross entity stats are trickier and require some > map-reduce-esque processing. > > we use this approach for smart-meter analytics where data points for a > given entity (meter) don't come any faster than once every 15 min... not > sure if it would work for you. > > > > On Thursday, November 21, 2013 8:12:15 AM UTC-6, Mathieu Simard wrote: >> >> I need the median value for multiple entities but only compared to >> themselves. >> In the future I will probably create the media across entities by doing a >> median average. >> >> On Tuesday, November 19, 2013 9:23:59 PM UTC-5, Jim wrote: >>> >>> Are you doing a time-series type analysis where you need the rolling >>> median value for a specific entity, or do you need the median value across >>> a range of entities? >>> >>> >>> >>> On Tuesday, November 12, 2013 2:07:34 PM UTC-6, Mathieu Simard wrote: >>>> >>>> Since there is no appengine solution available such as the Redis atomic >>>> list, I'm left wondering how to implement a cost effective rolling median. >>>> Has anyone come up with a solution that would be more convenient than >>>> running a redis instance on Google Compute Engine? >>>> >>> -- > You received this message because you are subscribed to a topic in the > Google Groups "Google App Engine" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/google-appengine/VMG96Xzvsok/unsubscribe > . > To unsubscribe from this group and all its topics, send an email to > google-appengine+unsubscr...@googlegroups.com. > To post to this group, send email to google-appengine@googlegroups.com. > Visit this group at http://groups.google.com/group/google-appengine. > For more options, visit https://groups.google.com/groups/opt_out. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
