Re: Missing dependencies when submitting scala app
Hi Jeff, The problem was I was pulling json4s 3.3.0 which seems to have some problem with spark, I switched to 3.2.11 and everything is fine now On Tue, Dec 22, 2015 at 5:36 PM, Jeff Zhang wrote: > It might be jar conflict issue. Spark has dependency org.json4s.jackson, > do you also specify org.json4s.jackson in your sbt dependency but with a > different version ? > > On Wed, Dec 23, 2015 at 6:15 AM, Daniel Valdivia > wrote: > >> Hi, >> >> I'm trying to figure out how to bundle dependendies with a scala >> application, so far my code was tested successfully on the spark-shell >> however now that I'm trying to run it as a stand alone application which >> I'm compilin with sbt is yielding me the error: >> >> >> *java.lang.NoSuchMethodError: >> org.json4s.jackson.JsonMethods$.parse$default$3()Z at >> ClusterIncidents$$anonfun$1.apply(ClusterInciden* >> >> I'm doing "sbt clean package" and then spark-submit of the resulting jar, >> however seems like either my driver or workers don't have the json4s >> dependency, therefor can't find the parse method >> >> Any idea on how to solve this depdendency problem? >> >> thanks in advance >> > > > > -- > Best Regards > > Jeff Zhang >
Re: Missing dependencies when submitting scala app
It might be jar conflict issue. Spark has dependency org.json4s.jackson, do you also specify org.json4s.jackson in your sbt dependency but with a different version ? On Wed, Dec 23, 2015 at 6:15 AM, Daniel Valdivia wrote: > Hi, > > I'm trying to figure out how to bundle dependendies with a scala > application, so far my code was tested successfully on the spark-shell > however now that I'm trying to run it as a stand alone application which > I'm compilin with sbt is yielding me the error: > > > *java.lang.NoSuchMethodError: > org.json4s.jackson.JsonMethods$.parse$default$3()Z at > ClusterIncidents$$anonfun$1.apply(ClusterInciden* > > I'm doing "sbt clean package" and then spark-submit of the resulting jar, > however seems like either my driver or workers don't have the json4s > dependency, therefor can't find the parse method > > Any idea on how to solve this depdendency problem? > > thanks in advance > -- Best Regards Jeff Zhang
Missing dependencies when submitting scala app
Hi, I'm trying to figure out how to bundle dependendies with a scala application, so far my code was tested successfully on the spark-shell however now that I'm trying to run it as a stand alone application which I'm compilin with sbt is yielding me the error: java.lang.NoSuchMethodError: org.json4s.jackson.JsonMethods$.parse$default$3()Z at ClusterIncidents$$anonfun$1.apply(ClusterInciden I'm doing "sbt clean package" and then spark-submit of the resulting jar, however seems like either my driver or workers don't have the json4s dependency, therefor can't find the parse method Any idea on how to solve this depdendency problem? thanks in advance