Hi Community, Is it possible to set no of shuffle partitions per exchange ?
My spark query contains a lot of joins/aggregations involving big tables and small tables, so keeping a high value of spark.sql.shuffle.partitions helps with big tables, for small tables it creates a lot of overhead on the scheduler. Cheers, Vibhor
