Can you isolate the row that is causing the problem? I mean start using
show(31) up to show(60).
Perhaps this will help you to understand the problem.
regards,
Apostolos
On 07/09/2018 01:11 πμ, dimitris plakas wrote:
Hello everyone, I am new in Pyspark and i am facing an issue. Let me
explain what exactly is the problem.
I have a dataframe and i apply on this a map() function
(dataframe2=datframe1.rdd.map(custom_function())
dataframe = sqlContext.createDataframe(dataframe2)
when i have
dataframe.show(30,True) it shows the result,
when i am using dataframe.show(60, True) i get the error. The Error is
in the attachement Pyspark_Error.txt.
Could you please explain me what is this error and how to overpass it?
---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscr...@spark.apache.org
--
Apostolos N. Papadopoulos, Associate Professor
Department of Informatics
Aristotle University of Thessaloniki
Thessaloniki, GREECE
tel: ++0030312310991918
email: papad...@csd.auth.gr
twitter: @papadopoulos_ap
web: http://datalab.csd.auth.gr/~apostol
