Would partitioning your data based on the key and then running
mapPartitions help?
Best Regards,
Sonal
Founder, Nube Technologies http://www.nubetech.co
http://in.linkedin.com/in/sonalgoyal
On Thu, May 21, 2015 at 4:33 AM, roy rp...@njit.edu wrote:
I have a key-value RDD, key is a timestamp (femto-second resolution, so
grouping buys me nothing) and I want to reduce it in the chronological
order.
How do I do that in spark?
I am fine with reducing contiguous sections of the set separately and then
aggregating the resulting objects locally.
Thanks
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/How-to-process-data-in-chronological-order-tp22966.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org
