Have you tried the following construct ? new OrderedRDDFunctions[K, V, (K, V)](rdd).sortByKey()
See core/src/main/scala/org/apache/spark/api/java/JavaPairRDD.scala On Wed, Mar 30, 2016 at 5:20 AM, Nirav Patel <npa...@xactlycorp.com> wrote: > Hi, I am trying to use filterByRange feature of spark OrderedRDDFunctions > in a hope that it will speed up filtering by scanning only required > partitions. > I have created Paired RDD with a RangePartitioner in one scala class and > in another class I am trying to access this RDD and do following: > > In first scala class called RDDInitializer I do: > > implicit val rowKeyOrdering = rowKeyOrd > > val repartitionRdd = rowdataRdd.partitionBy(new RangePartitioner( > minPartitions.toInt, dataRdd, true)) > > dataRdd = repartitionRdd.sortByKey() > > > In second scala class I do: > > import org.apache.spark.SparkContext._ > > RDDInitializer.dataRdd.filterByRange(myKeyFirst, myKeyLast) > But I am getting following compile error: > > "value filterByRange is not a member of org.apache.spark.rdd.RDD[(myKey, > myData)]" > > > Looks like I can use all methods of OrderedRDDFunctions inside first > scala class where implicit rowKeyOrdering is defined but not in second > class. > > > Please help me resolve this compile error. > > > Thanks > > Nirav > > > > > > [image: What's New with Xactly] <http://www.xactlycorp.com/email-click/> > > <https://www.nyse.com/quote/XNYS:XTLY> [image: LinkedIn] > <https://www.linkedin.com/company/xactly-corporation> [image: Twitter] > <https://twitter.com/Xactly> [image: Facebook] > <https://www.facebook.com/XactlyCorp> [image: YouTube] > <http://www.youtube.com/xactlycorporation>
