About nested RDD

[Tenghuan He]

Hi all,

I know that nested RDDs are not possible like linke rdd1.map(x => x +
rdd2.count())
I tried to create a custome RDD like following

class MyRDD(base: RDD, part: Partitioner) extends RDD[(K, V)] {

var rdds = new  ArrayBuffer.empty[RDD[(K, (V, Int))]]
def update(rdd: RDD[_]) {
  udds += rdd
}
def comput ...
def getPartitions ...
}

In the compute method I call the internal rdds' iterators and got
NullPointerException
Is this also a form of nested RDDs and how do I get rid of this?

Thanks.


Tenghuan