[ https://issues.apache.org/jira/browse/SPARK-3638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14231086#comment-14231086 ]
Aniket Bhatnagar commented on SPARK-3638: ----------------------------------------- [~ashrafuzzaman] did you build using using kinesis-asl profile as described in https://github.com/apache/spark/blob/master/docs/streaming-kinesis-integration.md (mvn -Pkinesis-asl -DskipTests clean package)? None of the pre build spark distributions have the profile enabled. I just pulled down 1.1.1 source code and did a build as mentioned in the Kinesis integration documentation and I can see the class the in the built assembly. > Commons HTTP client dependency conflict in extras/kinesis-asl module > -------------------------------------------------------------------- > > Key: SPARK-3638 > URL: https://issues.apache.org/jira/browse/SPARK-3638 > Project: Spark > Issue Type: Bug > Components: Examples, Streaming > Affects Versions: 1.1.0 > Reporter: Aniket Bhatnagar > Labels: dependencies > Fix For: 1.1.1, 1.2.0 > > > Followed instructions as mentioned @ > https://github.com/apache/spark/blob/master/docs/streaming-kinesis-integration.md > and when running the example, I get the following error: > {code} > Caused by: java.lang.NoSuchMethodError: > org.apache.http.impl.conn.DefaultClientConnectionOperator.<init>(Lorg/apache/http/conn/scheme/SchemeRegistry;Lorg/apache/http/conn/DnsResolver;)V > at > org.apache.http.impl.conn.PoolingClientConnectionManager.createConnectionOperator(PoolingClientConnectionManager.java:140) > at > org.apache.http.impl.conn.PoolingClientConnectionManager.<init>(PoolingClientConnectionManager.java:114) > at > org.apache.http.impl.conn.PoolingClientConnectionManager.<init>(PoolingClientConnectionManager.java:99) > at > com.amazonaws.http.ConnectionManagerFactory.createPoolingClientConnManager(ConnectionManagerFactory.java:29) > at > com.amazonaws.http.HttpClientFactory.createHttpClient(HttpClientFactory.java:97) > at > com.amazonaws.http.AmazonHttpClient.<init>(AmazonHttpClient.java:181) > at > com.amazonaws.AmazonWebServiceClient.<init>(AmazonWebServiceClient.java:119) > at > com.amazonaws.AmazonWebServiceClient.<init>(AmazonWebServiceClient.java:103) > at > com.amazonaws.services.kinesis.AmazonKinesisClient.<init>(AmazonKinesisClient.java:136) > at > com.amazonaws.services.kinesis.AmazonKinesisClient.<init>(AmazonKinesisClient.java:117) > at > com.amazonaws.services.kinesis.AmazonKinesisAsyncClient.<init>(AmazonKinesisAsyncClient.java:132) > {code} > I believe this is due to the dependency conflict as described @ > http://mail-archives.apache.org/mod_mbox/spark-dev/201409.mbox/%3ccajob8btdxks-7-spjj5jmnw0xsnrjwdpcqqtjht1hun6j4z...@mail.gmail.com%3E -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org