how to convert RDD to PairRDDFunctions ?

2014-07-08 Thread Konstantin Kudryavtsev
Hi all, sorry for fooly question, but how can I get PairRDDFunctions RDD? I'm doing it to perform leftOuterJoin aftewards currently I do in this was (it seems incorrect): val parRDD = new PairRDDFunctions( oldRdd.map(i = (i.key, i)) ) I guess this constructor is definitely wrong... Thank you,

Re: how to convert RDD to PairRDDFunctions ?

2014-07-08 Thread Sean Owen
If your RDD contains pairs, like an RDD[(String,Integer)] or something, then you get to use the functions in PairRDDFunctions as if they were declared on RDD. On Tue, Jul 8, 2014 at 6:25 PM, Konstantin Kudryavtsev kudryavtsev.konstan...@gmail.com wrote: Hi all, sorry for fooly question, but

Re: how to convert RDD to PairRDDFunctions ?

2014-07-08 Thread Mark Hamstra
See Working with Key-Value Pairs http://spark.apache.org/docs/latest/programming-guide.html. In particular: In Scala, these operations are automatically available on RDDs containing Tuple2 objects (the built-in tuples in the language, created by simply writing (a, b)), as long as you import