[ https://issues.apache.org/jira/browse/HADOOP-14237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15966314#comment-15966314 ]
Mingliang Liu commented on HADOOP-14237: ---------------------------------------- You may need to rebase the patch as [HADOOP-14248] and/or [HADOOP-14301]. Thanks, > S3A Support Shared Instance Profile Credentials Across All Hadoop Nodes > ----------------------------------------------------------------------- > > Key: HADOOP-14237 > URL: https://issues.apache.org/jira/browse/HADOOP-14237 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 > Affects Versions: 2.8.0, 3.0.0-alpha1, 3.0.0-alpha2, 2.8.1 > Environment: EC2, AWS > Reporter: Kazuyuki Tanimura > Assignee: Kazuyuki Tanimura > > When I run a large Hadoop cluster on EC2 instances with IAM Role, it fails > getting the instance profile credentials, eventually all jobs on the cluster > fail. Since a number of S3A clients (all mappers and reducers) try to get the > credentials, the AWS credential endpoint starts responding 5xx and 4xx error > codes. > SharedInstanceProfileCredentialsProvider.java is sort of trying to solve it, > but it still does not share the credentials with other EC2 nodes / JVM > processes. > This issue prevents users from creating Hadoop clusters on EC2 -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org