Mine was not really a moving average problem. It was more like partitioning on some keys and sorting(on different keys) and then running a sliding window through the partition. I reverted back to map-reduce for that(I needed secondary sort, which is not very mature in Spark right now).
But, as far as I understand your problem, you should be able to handle it by converting your RDD to key-value RDDs which I think will be automatically partitioned on the key and then use *mapPartitions *to run your logic. On Mon, Feb 2, 2015 at 1:20 AM, ashu [via Apache Spark User List] < ml-node+s1001560n21458...@n3.nabble.com> wrote: > Hi, > I want to know about your moving avg implementation. I am also doing some > time-series analysis about CPU performance. So I tried simple regression > but result is not good. rmse is 10 but when I extrapolate it just shoot up > linearly. I think I should first smoothed out the data then try regression > to forecast. > i am thinking of moving avg as an option,tried it out according to this > http://stackoverflow.com/questions/23402303/apache-spark-moving-average > > but "partitionBy" is giving me error, I am building with Spark 1.2.0. > Can you share your ARIMA implementation if it is open source, else can you > give me hints about it > > Will really appreciate the help. > Thanks > > ------------------------------ > If you reply to this email, your message will be added to the discussion > below: > > http://apache-spark-user-list.1001560.n3.nabble.com/Window-comparison-matching-using-the-sliding-window-functionality-feasibility-tp15352p21458.html > To unsubscribe from Window comparison matching using the sliding window > functionality: feasibility, click here > <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=15352&code=bml0aW5rYWswMDFAZ21haWwuY29tfDE1MzUyfDEyMjcwMjA2NQ==> > . > NAML > <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Window-comparison-matching-using-the-sliding-window-functionality-feasibility-tp15352p21467.html Sent from the Apache Spark User List mailing list archive at Nabble.com.
