Right problem solved in a most disgraceful manner. Just add a package relocation in maven shade config. The downside is that it is not compatible with my IDE (IntelliJ IDEA), will cause:
Error:scala.reflect.internal.MissingRequirementError: object scala.runtime in compiler mirror not found.: object scala.runtime in compiler mirror not found. and all scala object inspection fail and marked as error. So I'm still looking for an alternative -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-throws-NoSuchFieldError-when-testing-on-cluster-mode-tp8064p8088.html Sent from the Apache Spark User List mailing list archive at Nabble.com.
