When exactly does AQE coalesce small partitions or break up large partitions? Say, I do df.repartition(2), and the data in this df is huge, will spark first repartition into 2 partitions (which may cause an OOM) and then break them up? Or will it make the necessary number of partitions according to the advisoryPartitionSizeInBytes setting right away, without repartitioning into 2 partitions first?
Thanks!
