Hi, there, I am using Spark 1.4.1. The protobuf 2.5 is included by Spark 1.4.1 by default. However, I would like to use Protobuf 3 in my spark application so that I can use some new features such as Map support. Is there anyway to do that?
Right now if I build a uber.jar with dependencies including protobuf 3 classes and pass to spark-shell through --jars option, during the execution, I got the error *java.lang.NoSuchFieldError: unknownFields. * Is there anyway to use a different version of Protobuf other than the default one included in the Spark distribution? I guess I can generalize and extend the question to any third party libraries. How to deal with version conflict for any third party libraries included in the Spark distribution? Thanks! Lan
