specifying sort order for sort by value
Hi, I am using rdd.sortBy(_._2) to get an RDD sorted by value. The default order is ascending order. How can I get it sorted in descending order? I could not find an option to specify the order. I need to get the top K elements of the list sorted in descending order. If there is no option to get the descending order, I would like to know if there is a way to get the last K elements of the list sorted in ascending order. take(k) gets the first k elements, is there an option to get the last K elements of an RDD ? thanks -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/specifying-sort-order-for-sort-by-value-tp18289.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
Re: specifying sort order for sort by value
Yes you can sort it in desc, you simply specify a boolean value in the second argument to the sortBy function. Default is ascending. So it will look like: rdd.sortBy(_._2, false) Read more over here http://spark.apache.org/docs/latest/api/scala/#org.apache.spark.rdd.RDD Thanks Best Regards On Fri, Nov 7, 2014 at 12:20 AM, SK skrishna...@gmail.com wrote: Hi, I am using rdd.sortBy(_._2) to get an RDD sorted by value. The default order is ascending order. How can I get it sorted in descending order? I could not find an option to specify the order. I need to get the top K elements of the list sorted in descending order. If there is no option to get the descending order, I would like to know if there is a way to get the last K elements of the list sorted in ascending order. take(k) gets the first k elements, is there an option to get the last K elements of an RDD ? thanks -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/specifying-sort-order-for-sort-by-value-tp18289.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
Re: specifying sort order for sort by value
Thanks. I was looking at an older RDD documentation that did not specify the ordering option. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/specifying-sort-order-for-sort-by-value-tp18289p18292.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
