Hello,
I am using Spark 1.6.1 distribution over Cloudera CDH 5.7.0 cluster.
When I am running my fatJar - spark jar and when it is making a call to
HttpClient it is getting classic NoSuchField Error : INSTANCE. Which
happens usually when httrpclient in classpath is older than anticipated
httpclient jar version (should be 4.5.2).
Can someone help me how do I overcome this, I tried multiple options
1. specifying --conf spark.files={path-to-httpclient-4.5.2.jar} and then
using --conf spark.driver.extraClassPath=./httpclient-4.5.2.jar and
--conf spark.executor.extraClassPath=./httpclient-4.5.2.jar
2. I specified the jar in --jar option
3. I specified jar in hdfs://master:8020/path/to/jar/httpclient.4.5.2.jar
In each case I had error.
I also would like to know if
--conf spark.files={path-to-httpclient-4.5.2.jar},
is path-to-httpclient-4.5.2.jar can be giving to some local path from where
I am issuing spark-submit?
And also are there any other suggestions how should I resolve this conflict?Any clues? -Regards Sagar
