In addition,you can also set spark.sql.adaptive.enabled=true (default=false)
,enable adaptive query execution。
--
View this message in context:
http://apache-spark-developers-list.1001551.n3.nabble.com/Resultant-RDD-after-a-group-by-query-always-returns-200-partitions-tp18647p18650.html
Sent
That's the default shuffle partitions with Spark, You can tune it using
spark.sql.shuffle.partitions.
Regards,
Rishitesh Mishra,
SnappyData . (http://www.snappydata.io/)
https://in.linkedin.com/in/rishiteshmishra
On Tue, Aug 16, 2016 at 11:31 AM, Niranda Perera
wrote:
> Hi,
>
> I ran the follo