-dev If you want to guarantee the side effects happen you should use foreach or foreachPartitions. A `take`, for example, might only evaluate a subset of the partitions until it find enough results.
On Wed, Aug 12, 2015 at 7:06 AM, Eugene Morozov <fathers...@list.ru> wrote: > Hi! > > I’d like to complete action (store / print smth) inside of transformation > (map or mapPartitions). This approach has some flaws, but there is a > question. Might it happen that Spark will optimise (RDD or DataFrame) > processing so that my mapPartitions simply won’t happen? > > -- > Eugene Morozov > fathers...@list.ru > > > > >
