The callback function f only accept 2 arguments, if you want to pass
another objects to it, you need closure, such as:
foo=xxx
def f(index, iterator, foo):
yield (index, foo)
rdd.mapPartitionsWithIndex(lambda index, it: f(index, it, foo))
also you can make f become `closure`:
def f2(index, iterator):
yield (index, foo)
rdd.mapPartitionsWithIndex(f2)
On Sun, Aug 17, 2014 at 10:25 AM, Chengi Liu <[email protected]> wrote:
> Hi,
> In this example:
> http://www.cs.berkeley.edu/~pwendell/strataconf/api/pyspark/pyspark.rdd.RDD-class.html#mapPartitionsWithSplit
> Let say, f takes three arguments:
> def f(splitIndex, iterator, foo): yield splitIndex
> Now, how do i send this foo parameter to this method?
> rdd.mapPartitionsWithSplit(f)
> Thanks
>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
