[
https://issues.apache.org/jira/browse/FLUME-3174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Denes Arvay resolved FLUME-3174.
--------------------------------
Resolution: Fixed
Fix Version/s: 1.8.0
I'm marking this resolved as the joda-time version has been bumped in
FLUME-3173.
> HdfsSink AWS S3A authentication does not work on JDK 8
> ------------------------------------------------------
>
> Key: FLUME-3174
> URL: https://issues.apache.org/jira/browse/FLUME-3174
> Project: Flume
> Issue Type: Bug
> Reporter: Marcell Hegedus
> Priority: Minor
> Fix For: 1.8.0
>
>
> Flume writing to S3A with the following Hdfs Sink configuration fails with
> AmazonS3Exception: Forbidden (Service: Amazon S3; Status Code: 403; Error
> Code: 403 Forbidden...
> {code}
> a1.sinks = k1
> a1.sinks.k1.channel = c1
> a1.sinks.k1.type = hdfs
> a1.sinks.k1.hdfs.path = s3a://testflume/logs
> {code}
> AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY are provided either in flume-env
> or in core-site.xml and running "hdfs dfs -ls s3a://testflume/logs" works
> properly.
> The cause and the fix is documented in
> [hadoop-aws/index.md|https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/index.md#authentication-failures-when-running-on-java-8u60]
> {quote}A change in the Java 8 JVM broke some of the toString() string
> generation of Joda Time 2.8.0, which stopped the Amazon S3 client from being
> able to generate authentication headers suitable for validation by S3.
> Fix: Make sure that the version of Joda Time is 2.8.1 or later, or use a new
> version of Java 8.{quote}
> Tested that authentication is successful with
> * JDK 7
> * JDK 8 + joda-time updated to v2.9.6.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
