Re: Resultant RDD after a group by query always returns 200 partitions

2016-08-16 Thread colzer
In addition,you can also set spark.sql.adaptive.enabled=true (default=false) ,enable adaptive query execution。 -- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/Resultant-RDD-after-a-group-by-query-always-returns-200-partitions-tp18647p18650.html Sent

Re: Resultant RDD after a group by query always returns 200 partitions

2016-08-16 Thread Rishi Mishra
That's the default shuffle partitions with Spark, You can tune it using spark.sql.shuffle.partitions. Regards, Rishitesh Mishra, SnappyData . (http://www.snappydata.io/) https://in.linkedin.com/in/rishiteshmishra On Tue, Aug 16, 2016 at 11:31 AM, Niranda Perera wrote: > Hi, > > I ran the follo