updateStateByKey and invFunction

2015-02-24 Thread Ashish Sharma
So say I want to calculate top K users visiting a page in the past 2 hours updated every 5 mins. so here I want to maintain something like this Page_01 = {user_01:32, user_02:3, user_03:7...} ... Basically a count of number of times a user visited a page. Here my key is page name/id and state

Re: updateStateByKey and invFunction

2015-02-24 Thread Arush Kharbanda
You can use a reduceByKeyAndWindow with your specific time window. You can specify the inverse function in reduceByKeyAndWindow. On Tue, Feb 24, 2015 at 1:36 PM, Ashish Sharma ashishonl...@gmail.com wrote: So say I want to calculate top K users visiting a page in the past 2 hours updated every

Re: updateStateByKey and invFunction

2015-02-24 Thread Ashish Sharma
But how will I specify my state there? On Tue, Feb 24, 2015 at 12:50 AM Arush Kharbanda ar...@sigmoidanalytics.com wrote: You can use a reduceByKeyAndWindow with your specific time window. You can specify the inverse function in reduceByKeyAndWindow. On Tue, Feb 24, 2015 at 1:36 PM, Ashish