I am trying to implement top-k in scala within apache spark. I am aware that spark has a top action. But, top() uses reduce(). Instead, I would like to use treeReduce(). I am trying to compare the performance of reduce() and treeReduce().
The main issue I have is that I cannot use these 2 lines of code which are used in the top() action within my Spark application. val queue = new BoundedPriorityQueue[T](num)(ord.reverse) queue ++= util.collection.Utils.takeOrdered(items, num)(ord) How can I go about implementing top() using treeReduce()? -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Implementing-top-using-treeReduce-tp23227.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
