Hi Team, How do we increase the parallelism in Spark SQL. In Spark Core, we can re-partition or pass extra arguments part of the transformation.
I am trying the below example,
val df1 = sqlContext.read.format("jdbc").options(Map(...)).load
val df2= df1.cache
val df2.count
Here count operation using only one task. I couldn't increase the
parallelism.
Thanks in advance
Thanks
Siva
