It's smart. Have a look at https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/rdd/CoalescedRDD.scala#L123
On Thu, Oct 8, 2015 at 4:00 AM, Cesar Flores <ces...@gmail.com> wrote: > It is my understanding that the default behavior of coalesce function when > the user reduce the number of partitions is to only merge them without > executing shuffle. > > My question is: Is this merging smart? For example does spark try to merge > the small partitions first or the election of partitions to merge is random? > > > Thanks > -- > Cesar Flores > -- -- Iulian Dragos ------ Reactive Apps on the JVM www.typesafe.com
