Thanks for the replies. We decided to use concurrency in Scala to do the two mappings using the same source RDD in parallel. So far, it seems to be working. Any comments?
On Wednesday, May 6, 2015, Evo Eftimov <evo.efti...@isecc.com> wrote: > RDD1 = RDD.filter() > > RDD2 = RDD.filter() > > > > *From:* Bill Q [mailto:bill.q....@gmail.com > <javascript:_e(%7B%7D,'cvml','bill.q....@gmail.com');>] > *Sent:* Tuesday, May 5, 2015 10:42 PM > *To:* user@spark.apache.org > <javascript:_e(%7B%7D,'cvml','user@spark.apache.org');> > *Subject:* Map one RDD into two RDD > > > > Hi all, > > I have a large RDD that I map a function to it. Based on the nature of > each record in the input RDD, I will generate two types of data. I would > like to save each type into its own RDD. But I can't seem to find an > efficient way to do it. Any suggestions? > > > > Many thanks. > > > > > > Bill > > > > -- > > Many thanks. > > Bill > > > -- Many thanks. Bill
