It seems like the union function on RDDs might be what you are looking
for, or was there something else you were trying to achieve?
On Thursday, April 7, 2016, Tenghuan He <tenghua...@gmail.com> wrote:
> Hi all,
>
> I know that nested RDDs are not possible like linke rdd1.map(x => x +
> rdd2.count())
> I tried to create a custome RDD like following
>
> class MyRDD(base: RDD, part: Partitioner) extends RDD[(K, V)] {
>
> var rdds = new ArrayBuffer.empty[RDD[(K, (V, Int))]]
> def update(rdd: RDD[_]) {
> udds += rdd
> }
> def comput ...
> def getPartitions ...
> }
>
> In the compute method I call the internal rdds' iterators and got
> NullPointerException
> Is this also a form of nested RDDs and how do I get rid of this?
>
> Thanks.
>
>
> Tenghuan
>
--
Cell : 425-233-8271
Twitter: https://twitter.com/holdenkarau
