-dev
If you want to guarantee the side effects happen you should use foreach or
foreachPartitions. A `take`, for example, might only evaluate a subset of
the partitions until it find enough results.
On Wed, Aug 12, 2015 at 7:06 AM, Eugene Morozov fathers...@list.ru wrote:
Hi!
I’d like to
Hi!
I’d like to complete action (store / print smth) inside of transformation (map
or mapPartitions). This approach has some flaws, but there is a question. Might
it happen that Spark will optimise (RDD or DataFrame) processing so that my
mapPartitions simply won’t happen?
--
Eugene Morozov
