the single call to mapPartitions will be faster but we could be
wrong.
It would be nice to have a clone method on the iterator.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/multiple-passes-in-mapPartitions-tp7555p7616.html
Sent from the Apache Spark User
in context:
http://apache-spark-user-list.1001560.n3.nabble.com/multiple-passes-in-mapPartitions-tp7555p7616.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
.1001560.n3.nabble.com/multiple-passes-in-mapPartitions-tp7555.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
.nabble.com/multiple-passes-in-mapPartitions-tp7555p7616.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
the memory.
Which is also bad in terms of more GC.
Is there a faster/better way of taking multiple passes without copying all
the data?
Thank you,
Zhen
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/multiple-passes-in-mapPartitions-tp7555.html
Sent from