Re: RDD of Iterable[String]

2015-06-15 Thread nir 
Have you found answer to this? I am also looking for exact same solution.



--
View this message in context: 
http://apache-spark-user-list.1001560.n3.nabble.com/RDD-of-Iterable-String-tp15016p23329.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org

Re: RDD of Iterable[String]

2014-09-25 Thread Liquan Pei 
Hi Deep,

I believe that you are referring to the map for Iterable[String]

suppose you have
iter:Iterable[String]
you can do

newIter = iter.map(item = Item + a )

which will create an new Iterable[String] with each element appending  an
a to all string in iter.

Does this answer your question?

Liquan

On Thu, Sep 25, 2014 at 12:43 AM, Deep Pradhan pradhandeep1...@gmail.com
wrote:

 what should come in the map??
 Thanks Liquan for answering me...
 I really need some help..I am stuck in some thing.


 On Thu, Sep 25, 2014 at 12:57 PM, Deep Pradhan pradhandeep1...@gmail.com
 wrote:

 what should come in the map??

 On Wed, Sep 24, 2014 at 10:52 PM, Liquan Pei liquan...@gmail.com wrote:

 Hi Deep,

 The Iterable trait in scala has methods like map and reduce that you can
 use to iterate elements of Iterable[String]. You can also create an
 Iterator from the Iterable. For example, suppose you have

 val rdd: RDD[Iterable[String]]

 you can do

 rdd.map { x = //x has type Iterable[String]
x.map(...) // Process elements in iterable[String]
val iter:Iterator[String] = x.iterator
while(iter.hasNext) {
 iter.next()
}
 }

 Hope this helps!

 Liquan

 On Wed, Sep 24, 2014 at 8:21 AM, Deep Pradhan pradhandeep1...@gmail.com
  wrote:

 Can we iterate over RDD of Iterable[String]? How do we do that?
 Because the entire Iterable[String] seems to be a single element in the
 RDD.

 Thank You




 --
 Liquan Pei
 Department of Physics
 University of Massachusetts Amherst






-- 
Liquan Pei
Department of Physics
University of Massachusetts Amherst

RDD of Iterable[String]

2014-09-24 Thread Deep Pradhan 
Can we iterate over RDD of Iterable[String]? How do we do that?
Because the entire Iterable[String] seems to be a single element in the RDD.

Thank You

Re: RDD of Iterable[String]

2014-09-24 Thread Liquan Pei 
Hi Deep,

The Iterable trait in scala has methods like map and reduce that you can
use to iterate elements of Iterable[String]. You can also create an
Iterator from the Iterable. For example, suppose you have

val rdd: RDD[Iterable[String]]

you can do

rdd.map { x = //x has type Iterable[String]
   x.map(...) // Process elements in iterable[String]
   val iter:Iterator[String] = x.iterator
   while(iter.hasNext) {
iter.next()
   }
}

Hope this helps!

Liquan

On Wed, Sep 24, 2014 at 8:21 AM, Deep Pradhan pradhandeep1...@gmail.com
wrote:

 Can we iterate over RDD of Iterable[String]? How do we do that?
 Because the entire Iterable[String] seems to be a single element in the
 RDD.

 Thank You




-- 
Liquan Pei
Department of Physics
University of Massachusetts Amherst