also, multiple calls to mapPartitions() will be pipelined by the spark execution engine into a single stage, so the overhead is minimal.
On Fri, Jun 13, 2014 at 9:28 PM, zhen <z...@latrobe.edu.au> wrote: > Thank you for your suggestion. We will try it out and see how it performs. > We > think the single call to mapPartitions will be faster but we could be > wrong. > It would be nice to have a "clone method" on the iterator. > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/multiple-passes-in-mapPartitions-tp7555p7616.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. >
