[
https://issues.apache.org/jira/browse/SPARK-12269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15051733#comment-15051733
]
Brian London commented on SPARK-12269:
--------------------------------------
The jackson 2.4 and 2.5 incompatibility has appeared elsewhere. See the
following for example:
https://github.com/FasterXML/jackson-module-scala/issues/177
http://stackoverflow.com/questions/31039367/spark-parallelize-could-not-find-creator-property-with-name-id
All tests pass after changing those two versions. Could you elaborate what you
mean by research conflicts beyond that?
> Update aws-java-sdk version
> ---------------------------
>
> Key: SPARK-12269
> URL: https://issues.apache.org/jira/browse/SPARK-12269
> Project: Spark
> Issue Type: Improvement
> Components: Streaming
> Reporter: Brian London
> Priority: Minor
>
> The current Spark Streaming kinesis connector references a quite old version
> 1.9.40 of the AWS Java SDK (1.10.40 is current). Numerous AWS features
> including Kinesis Firehose are unavailable in 1.9. Those two versions of
> the AWS SDK in turn require conflicting versions of Jackson (2.4.4 and 2.5.3
> respectively) such that one cannot include the current AWS SDK in a project
> that also uses the Spark Streaming Kinesis ASL.
> Bumping the version of Jackson and the AWS library solves this problem and
> will allow Firehose integrations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
