Re: SparkException: Task not serializable - Jackson Json

2015-02-14 Thread mickdelaney
to get past this you can move the mapper creation code down into the closure. its then created on the worker node so it doesnt need to be serialized. // Parse it into a specific case class. We use flatMap to handle errors // by returning an empty list (None) if we encounter an issue and a //

SparkException: Task not serializable - Jackson Json

2015-01-24 Thread mickdelaney
Hi, I'm getting an exception using the Jackson with Spark Scala. I'm using Scala Case Classes to represent the Json that i'm trying to parse. Its being caused by: *Caused by: java.io.NotSerializableException: com.fasterxml.jackson.module.scala.modifiers.SetTypeModifier$ * I using a