Yeh that's exactly what I am doing right now (forgot to attach the query to the email).
insert into table CORE_USER_ROLE_YEARLY_ACTIVITY_SUMMERY select y.dateHourly as date, count(distinct userid) as activity, appKey as appKey, role as role from (select clientUserId as userid, timestamp as date, dateFormatter(timestamp, "yyyy-MM", "yyyy") as dateHourly, appKey as appKey, role as role from eventStatData) as y group by y.dateHourly, y.appKey, y.role; I was wondering whether this was the right step for scenarios where data is summarized using time stamp. While looking into this I found out another interesting project which deals with time series data [1]. [1] - https://github.com/prometheus/prometheus On Thu, May 19, 2016 at 10:00 PM, Srinath Perera <[email protected]> wrote: > schedule a spark query that aggregate the data per year and write to new > table, and you can query from that table? > > On Fri, May 20, 2016 at 12:06 AM, Dulitha Wijewantha <[email protected]> > wrote: > >> Hi guys, >> I have a bunch of data streams that have time stamps (like event logs) - >> currently in DAS 3.0.1 - is there a way to efficiently query these? >> >> Right now I am summarizing them based on string manipulation to the >> timestamp. An example query I have is - >> >> 1) How many unique user events do we have in the last year. >> >> Cheers~ >> >> -- >> Dulitha Wijewantha (Chan) >> Software Engineer - Mobile Development >> WSO2 Inc >> Lean.Enterprise.Middleware >> * ~Email [email protected] <[email protected]>* >> * ~Mobile +94712112165 <%2B94712112165>* >> * ~Website dulitha.me <http://dulitha.me>* >> * ~Twitter @dulitharw <https://twitter.com/dulitharw>* >> *~Github @dulichan <https://github.com/dulichan>* >> *~SO @chan <http://stackoverflow.com/users/813471/chan>* >> > > > > -- > ============================ > Blog: http://srinathsview.blogspot.com twitter:@srinath_perera > Site: http://home.apache.org/~hemapani/ > Photos: http://www.flickr.com/photos/hemapani/ > Phone: 0772360902 > -- Dulitha Wijewantha (Chan) Software Engineer - Mobile Development WSO2 Inc Lean.Enterprise.Middleware * ~Email [email protected] <[email protected]>* * ~Mobile +94712112165* * ~Website dulitha.me <http://dulitha.me>* * ~Twitter @dulitharw <https://twitter.com/dulitharw>* *~Github @dulichan <https://github.com/dulichan>* *~SO @chan <http://stackoverflow.com/users/813471/chan>*
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
