Hello, I am wondering what will happen if I use a reference for transforming
rdd, for example:
def func1(rdd: RDD[Int]): RDD[Int] = {
rdd.map(x => x * 2) // example transformation, but I am using a more
complex function
}
def main() {
.....
val myrdd = sc.parallelize(1 to 1000000)
val myrdd2 = func1(myrdd)
myrdd2.count()
}
The above is just an example. I am wondering if it is safe to use reference
like this. Thank you.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Using-reference-for-RDD-is-safe-tp23843.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org
