This is more a curiosity than an immediate problem. Here is my question: I ran into this easily solved issue http://stackoverflow.com/questions/22592811/task-not-serializable-java-io-notserializableexception-when-calling-function-ou recently. The solution was to replace my "class" with a scala singleton, which i guess is readily serializable.
So its clear that spark needs to serialize objects which carry the driver methods for an app, in order to run... but I'm wondering,,, maybe there is a way to change or update the spark API to catch unserializable spark apps at compile time? -- jay vyas
