I am currently using a third party library(Lucene) with Spark that is not serializable. Due to that reason, it generates the following exception :
Job aborted due to stage failure: Task 144.0 in stage 25.0 (TID 2122) had a not serializable result: org.apache.lucene.facet.FacetsConfig Serialization stack: - object not serializable (class: org.apache.lucene.facet.FacetsConfig, value: org.apache.lucene.facet.FacetsConfg While it would be ideal if this class was serializable, there is really nothing I can do to change this third party library in order to add serialization to it. What options do I have, and what's the recommended option to handle this problem? Thank you!
