Hi Crakjie,
Did you find the solution for the below problems?
Regards,
Hokam
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Stream-are-not-serializable-tp25185p26630.html
Sent from the Apache Spark User List mailing list archive at Nabble.com
)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
But this exception is not very clear about what part of my stream is not
serializable.
I try to add
.set("spark.driver.extraJavaOptions","-Dsun.io.serialization.extendedDebugInfo=true")
.set("spark.executor.extraJavaOptions","-Dsun
t;
> .set("spark.executor.extraJavaOptions","-Dsun.io.serialization.extendedDebugInfo=true")
>
> to my spark conf to have more information, but it changes nothing ( it
> should )
>
> So how can I find which function or part of my stream is not serializabl
park.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:664)
>> at
>> org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:169)
>> at
>> org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:192)
>>
