You can use a custom partitioner if your need is specific in any way.
If you care about ordering then you can zipWithIndex your rdd and decide
based on the sequence of the message.
The following partitioner should work for you.
class ExactPartitioner[V](
partitions: Int,
elements: Int)
extends Partitioner {
def getPartition(key: Any): Int = {
val k = key.asInstanceOf[Int]
// `k` is assumed to go continuously from 0 to elements-1.
return k * partitions / elements
}
}
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/partition-RDD-of-images-tp25515p25592.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org
