I have the same issue. For me to resolve the issue, I did what Patrick suggested here, but I had to also remove the old jars from the default spark jars directory. Could anyone suggest a way to avoid that?
On Wednesday, August 9, 2017 at 4:25:35 AM UTC-4, Patrick McGeever wrote: > > Actually we managed to resolve the problem so just in case someone else > stumbles upon this thread I will describe how we resolved it. > > In our case we were depending on a jar in our Spark project (Google > Adwords ads-lib) which in turn depended on guava-20.0 and > commons-configuration-1.10. > When we launched our Spark job we were getting the "NoSuchMethodError" > reported above for MapConfiguration in commons-configuration. > We tried to package the commons-configuration lib in our fat jar but this > did not work. > After some investigation we found that Spark includes a bunch of libs in > its build in addition to the provided Spark libs. > It contained older versions of both commons-configuration and guava, of > which the older version of commons-configuration did not contain the > function which was being used by ads-lib (actually I think it was a > constructor with a different signature). > > This was in my local Spark installation on my dev machine... > ls -alrh /usr/local/Cellar/apache- > spark/2.2.0/libexec/jars | grep -E 'commons-configuration|\sguava' > -rw-r--r-- 1 pmcgeever admin 2.1M Jul 1 00:09 guava-14.0.1.jar > -rw-r--r-- 1 pmcgeever admin 292K Jul 1 00:09 > commons-configuration-1.6.jar > > So we found that no matter what we did this older version of > commons-configuration provided by Spark was being used. > After we resolved this problem we had the same issue with a method not > being found in Guava because the older version provided by Spark was being > used. > > The fix... > > We provide the jars for commons-configuration-1.10 and guava-20.0 to > spark-submit and instruct it to load them instead of the versions it has in > its build. > > e.g. > > spark-submit --deploy-mode cluster \ > --jars ./lib/guava-20.0.jar,./lib/commons-configuration-1.10.jar \ > --conf > "spark.driver.extraClassPath=guava-20.0.jar:commons-configuration-1.10.jar" \ > --conf > "spark.executor.extraClassPath=guava-20.0.jar:commons-configuration-1.10.jar" > \ > ./some_spark_application.jar > > > I hope this helps someone. > > > On Friday, 28 July 2017 04:06:34 UTC+1, Peter Oliquino wrote: >> >> Hi Patrick, >> >> This appears to be a very old thread. This said, and also for better >> tracking, could you create a new one? We'll continue our discussion on the >> issue you are encountering from there. >> >> Best regards, >> Peter >> AdWords API Team >> > -- -- =~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~ Also find us on our blog: https://googleadsdeveloper.blogspot.com/ =~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~ You received this message because you are subscribed to the Google Groups "AdWords API and Google Ads API Forum" group. To post to this group, send email to adwords-api@googlegroups.com To unsubscribe from this group, send email to adwords-api+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/adwords-api?hl=en --- You received this message because you are subscribed to the Google Groups "AdWords API and Google Ads API Forum" group. To unsubscribe from this group and stop receiving emails from it, send an email to adwords-api+unsubscr...@googlegroups.com. Visit this group at https://groups.google.com/group/adwords-api. To view this discussion on the web visit https://groups.google.com/d/msgid/adwords-api/c89848eb-ded0-4592-9495-c41413aa188b%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
