Re: Change RDDs using map()

2014-09-17 Thread Mark Hamstra
You don't. That's what filter or the partial function version of collect are for: val transformedRDD = yourRDD.collect { case (k, v) if k == 1 = v } On Wed, Sep 17, 2014 at 3:24 AM, Deep Pradhan pradhandeep1...@gmail.com wrote: Hi, I want to make the following changes in the RDD (create new

RE: Change RDDs using map()

2014-09-17 Thread qihong
if you want the result as RDD of (key, 1) new_rdd = rdd.filter(x = x._2 == 1) if you want result as RDD of keys (since you know the values are 1), then new_rdd = rdd.filter(x = x._2 == 1).map(x = x._1) x._1 and x._2 are the way of scala to access the key and value from key/value pair.