rdd.foreach return value

code snippet ​ the 'print' actually print info on the worker node, but I feel confused where the 'return' value goes to. for I get nothing on the driver node. -- *--* a spark lover, a quant, a developer and a good man. http://github.com/litaotao

Re: rdd.foreach return value

The foreach operation on RDD has a void (Unit) return type. See attached. So there is no return value to the driver. David "All that is gold does not glitter, Not all those who wander are lost." Original Message Subject: rdd.foreach return value Local Time: Janua

Re: rdd.foreach return value

Here is signature for foreach: def foreach(f: T => Unit): Unit = withScope { I don't think you can return element in the way shown in the snippet. On Mon, Jan 18, 2016 at 7:34 PM, charles li wrote: > code snippet > > > ​ > the 'print' actually print info on the worker

Re: rdd.foreach return value

hi, great thanks to david and ted, I know that the content of RDD can be returned to driver using 'collect' method. but my question is: 1. cause we can write any code we like in the function put into 'foreach', so what happened when we actually write a 'return' sentence in the foreach function?

Re: rdd.foreach return value

For #2, RDD is immutable. > On Jan 18, 2016, at 8:10 PM, charles li wrote: > > > hi, great thanks to david and ted, I know that the content of RDD can be > returned to driver using 'collect' method. > > but my question is: > > > 1. cause we can write any code we

Re: rdd.foreach return value

1. foreach doesn't expect any value from function being passed (in your func_foreach). so nothing happens. The return values are just lost. it's like calling a function without saving return value to another var. foreach also doesn't return anything so you don't get modified RDD (like map*). 2.

Re: rdd.foreach return value

got it, great thanks, Vishal, Ted and David On Tue, Jan 19, 2016 at 1:10 PM, Vishal Maru wrote: > 1. foreach doesn't expect any value from function being passed (in your > func_foreach). so nothing happens. The return values are just lost. it's > like calling a function

Re: rdd.foreach return value

thanks, david and ted, I know that the content of RDD can be returned to driver using `collect ​ On Tue, Jan 19, 2016 at 11:44 AM, Ted Yu wrote: > Here is signature for foreach: > def foreach(f: T => Unit): Unit = withScope { > > I don't think you can return element in

Re: rdd.foreach return value

-05:00) To: charles li <charles.up...@gmail.com> Cc: user@spark.apache.org Subject: Re: rdd.foreach return value The foreach operation on RDD has a void (Unit) return type. See attached. So there is no return value to the driver. David "All that is gold does not glitter, Not all those wh