I have a small snippet of code which relays on argonaut <http://argonaut.io/> for JSON serialization which is ran from a `PairRDDFunctions.mapWithState` once a session is completed.
This is the code snippet (not that important): override def sendMessage(pageView: PageView): Unit = { Future { LogHolder.logger.info(s"Sending pageview: ${pageView.id} to automation") try { Http(url) .postData(pageView.asJson.toString) .option(HttpOptions.connTimeout(timeOutMilliseconds)) .asString .throwError } catch { case NonFatal(e) => LogHolder.logger.error("Failed to send pageview", e) } } } argonaut relys on a user implementation of a trait called `EncodeJson[T]`, which tells argonaut how to serialize and deserialize the object. The problem is, that the trait `EncodeJson[T]` is not serializable, thus throwing a NotSerializableException: Caused by: java.io.NotSerializableException: argonaut.EncodeJson$$anon$2 Serialization stack: - object not serializable (class: argonaut.EncodeJson$$anon$2, value: argonaut.EncodeJson$$anon$2@6415f61e) This is obvious and understandable. The question I have is - What possible ways are there to work around this? I'm currently depended on a third-party library which I can't control of change to implement Serializable in anyway. I've seen this this StackOverflow answer <http://stackoverflow.com/questions/22592811/task-not-serializable-java-io-notserializableexception-when-calling-function-ou> but couldn't implement any reasonable workaround. Anyone have any ideas? -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Using-a-non-serializable-third-party-JSON-serializable-on-a-spark-worker-node-throws-NotSerializablen-tp26372.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org