Not sure I understand correctly, but it sounds like you're looking for mapPartitions(). -Sven
On Mon, Jan 12, 2015 at 10:17 AM, maherrt <mahe...@hotmail.com> wrote: > Dear All > > what i want to do is : > as the data is partitioned on many worker nodes I want to be able to > process > this partition of data as a whole on each partition and then produce my > output using flatMap for example. > so can I loads all of the input records on one worker node and emitting any > output using map function? > > Thanks > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/can-I-buffer-flatMap-input-at-each-worker-node-tp21106.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > > -- http://sites.google.com/site/krasser/?utm_source=sig
