It is really nice that Spark RDD's provide functions that are often
equivalent to functions found in Scala collections. For example, I can
call:
myArray.map(myFx)
and equivalently
myRdd.map(myFx)
Awesome!
My question is this. Is it possible to write code that works on either
an RDD or a local collection without having to have parallel
implementations? I can't tell that RDD or Array share any supertypes or
traits by looking at the respective scaladocs. Perhaps implicit
conversions could be used here. What I would like to do is have a
single function whose body is like this:
myData.map(myFx)
where myData could be an RDD[Array[String]] (for example) or an
Array[Array[String]].
Has anyone had success doing this?
Thanks,
Philip
