Hi,
The reduce lambda accepts as its first argument the return value of the
previous execution. The first time, it is invoked with:
x = ("a", 1), y = ("b", 2)
And returns 1+2=3
Second time, it is invoked with
x = 3, y = ("c", 3)
so you can see why it raises the error that you are seeing.
There are several ways you could fix it. One way is to use a map before the
reduce, e.g.
rdd..map(lambda x: x[1]).reduce(lambda x,y: x + y)
Hope that's helpful,
Chris
-----Original Message-----
From: capitnfrak...@free.fr <capitnfrak...@free.fr>
Sent: 19 January 2022 02:41
To: user@spark.apache.org
Subject: newbie question for reduce
Hello
Please help take a look why my this simple reduce doesn't work?
>>> rdd = sc.parallelize([("a",1),("b",2),("c",3)])
>>>
>>> rdd.reduce(lambda x,y: x[1]+y[1])
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/opt/spark/python/pyspark/rdd.py", line 1001, in reduce
return reduce(f, vals)
File "/opt/spark/python/pyspark/util.py", line 74, in wrapper
return f(*args, **kwargs)
File "<stdin>", line 1, in <lambda>
TypeError: 'int' object is not subscriptable
>>>
spark 3.2.0
Thank you.
---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscr...@spark.apache.org
---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscr...@spark.apache.org
