I have had the same issue while using HttpClient from AWS EMR Spark Streaming
to post to a nodejs server.
I have found ... using
Classloder.getResource('org/apache/http/client/HttpClient") .... that the
class
Is being loaded front the spark-assembly-1.1.0-hadoop2.4.0.jar.
That in itself is not the issue because the version is 4.2.5 .... the same
version I am using on my local machine with success .... using Hadoop cdh 5.
The issue is that HttpClient relies on Httpcore .... and there is an old
commons-httpcore-1.3.jar as well as httpcore-4.5.2 in the spark-assembly
jar.
It looks like the old one is getting loaded first.
So the fix might be to build the Spark jar myself without the httpcore-1.3
and replace it on bootstrap.
I will keep you posted on the outcome.
--
View this message in context:
http://apache-spark-developers-list.1001551.n3.nabble.com/spark-1-1-0-w-hadoop-2-4-vs-aws-java-sdk-1-7-2-tp8481p10250.html
Sent from the Apache Spark Developers List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
