Did a quick test: rdd2: org.apache.spark.rdd.RDD[(Int, Int)] = MapPartitionsRDD[1] at map at <console>:29
I think sales_map is MapPartitionsRDD FYI On Fri, Dec 4, 2015 at 6:18 AM, Gokula Krishnan D <email2...@gmail.com> wrote: > Thanks Ayan for the updates. > > But in my example, I hope "sales_map" is a pair_RDD , isn't it?. > > Thanks & Regards, > Gokula Krishnan* (Gokul)* > > > On Fri, Dec 4, 2015 at 8:16 AM, ayan guha <guha.a...@gmail.com> wrote: > >> sortByKey() is a property of pairRDD as it requires key value pair to >> work. I think in scala their are transformation such as .toPairRDD(). >> >> On Sat, Dec 5, 2015 at 12:01 AM, Gokula Krishnan D <email2...@gmail.com> >> wrote: >> >>> Hello All - >>> >>> In spark-shell when we press tab after <RDD_Name>. ; we could see the >>> possible list of transformations and actions. >>> >>> But unable to see all the list. is there any other way to get the rest >>> of the list. I'm mainly looking for sortByKey() >>> >>> val sales_RDD = sc.textFile("Data/Scala/phone_sales.txt") >>> val sales_map = sales_RDD.map(sales=>{val x=sales.split(","); >>> (x(0),x(1))}) >>> >>> Layout of phone_sales.txt is (Brand, #.of Phones sold) >>> >>> I am mainly looking for SortByKey() but when I do Sales_map or >>> sales_RDD, I could see only sortBy() but not SortByKey(). >>> >>> By the way, I am using spark 1.3.0 with CDH 5.4 >>> >>> [image: Inline image 1] >>> >>> >>> >>> Thanks >>> Gokul >>> >>> >> >> >> -- >> Best Regards, >> Ayan Guha >> > >