This is more a curiosity than an immediate problem.
Here is my question: I ran into this easily solved issue
http://stackoverflow.com/questions/22592811/task-not-serializable-java-io-notserializableexception-when-calling-function-ou
recently. The solution was to replace my class with a scala
That's a great idea and it is also a pain point for some users. However, it
is not possible to solve this problem at compile time, because the content
of serialization can only be determined at runtime.
There are some efforts in Scala to help users avoid mistakes like this. One
example project
Hi Jay,
I just came across SPARK-720 Statically guarantee serialization will succeed
https://issues.apache.org/jira/browse/SPARK-720 which sounds like exactly
what you're referring to. Like Reynold I think it's not possible at this
time but it would be good to get your feedback on that ticket.
