Thanks for your reply, Vipin!
I am using spark-perf benchmark. The command to create RDD is :
val data: RDD[Vector] = RandomRDDs.normalVectorRDD(sc, m, n, numPartitions,
seed)
after I set the numPartitions, for example 40 partitions, I think spark core
code will allocate those partitions to
Hi, I have a cluster with 4 nodes (12 cores/ node). I want to distribute my
dataset to 24 partitions and allocate 6 partitions/ node. However, i found i
got 12 partitions with 2 nodes and 0 partition with the other 2 nodes.
Anyone has idea of how to set 6 partitions/node? is that possible to do
