Would partitioning your data based on the key and then running mapPartitions help?
Best Regards, Sonal Founder, Nube Technologies <http://www.nubetech.co> <http://in.linkedin.com/in/sonalgoyal> On Thu, May 21, 2015 at 4:33 AM, roy <rp...@njit.edu> wrote: > I have a key-value RDD, key is a timestamp (femto-second resolution, so > grouping buys me nothing) and I want to reduce it in the chronological > order. > > How do I do that in spark? > > I am fine with reducing contiguous sections of the set separately and then > aggregating the resulting objects locally. > > Thanks > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/How-to-process-data-in-chronological-order-tp22966.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > >
