[ https://issues.apache.org/jira/browse/SPARK-12269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15051741#comment-15051741 ]
Sean Owen commented on SPARK-12269: ----------------------------------- Ah, I realize Spark is on Jackson 2.4, not 2.5. If that difference has been a problem elsewhere, I guess I'm surprised if it's not a problem here. The AWS SDK will be using 2.4.4 then effectively. {{mvn dependency:tree}} can help diff the dependency tree before and after to understand exactly what's changed. > Update aws-java-sdk version > --------------------------- > > Key: SPARK-12269 > URL: https://issues.apache.org/jira/browse/SPARK-12269 > Project: Spark > Issue Type: Improvement > Components: Streaming > Reporter: Brian London > Priority: Minor > > The current Spark Streaming kinesis connector references a quite old version > 1.9.40 of the AWS Java SDK (1.10.40 is current). Numerous AWS features > including Kinesis Firehose are unavailable in 1.9. Those two versions of > the AWS SDK in turn require conflicting versions of Jackson (2.4.4 and 2.5.3 > respectively) such that one cannot include the current AWS SDK in a project > that also uses the Spark Streaming Kinesis ASL. > Bumping the version of Jackson and the AWS library solves this problem and > will allow Firehose integrations. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org