[
https://issues.apache.org/jira/browse/HADOOP-18481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17617560#comment-17617560
]
ASF GitHub Bot commented on HADOOP-18481:
-----------------------------------------
ahmarsuhail commented on PR #4973:
URL: https://github.com/apache/hadoop/pull/4973#issuecomment-1278675683
Yes, sorry should have mentioned! Tested by setting
`fs.s3a.aws.credentials.provider` to
`com.amazonaws.auth.AWSStaticCredentialsProvider`, and ran an Integration test
(`ITestPrefetchingInputStream`) which logs warning. Then tested with
`com.amazonaws.auth.EnvironmentVariableCredentialsProvider`,
`com.amazonaws.auth.EC2ContainerCredentialsProviderWrapper` and
`com.amazonaws.auth.InstanceProfileCredentialsProvider` which do not log
warning.
> AWS v2 SDK warning to skip warning of EnvironmentVariableCredentialsProvider
> ----------------------------------------------------------------------------
>
> Key: HADOOP-18481
> URL: https://issues.apache.org/jira/browse/HADOOP-18481
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.4.0
> Reporter: Steve Loughran
> Assignee: Ahmar Suhail
> Priority: Major
> Labels: pull-request-available
>
> looking at test output with the sdk warnings enabled, it is now always
> warning of a v1 provider reference, even if the user hasn't set any
> fs.s3a.credential.provider option
> {code}
> 2022-10-05 14:09:09,733 [setup] DEBUG s3a.S3AUtils
> (S3AUtils.java:createAWSCredentialProvider(691)) - Credential provider class
> is org.apache.hadoop.fs.s3a.TemporaryAWSCredentialsProvider
> 2022-10-05 14:09:09,733 [setup] DEBUG s3a.S3AUtils
> (S3AUtils.java:createAWSCredentialProvider(691)) - Credential provider class
> is org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider
> 2022-10-05 14:09:09,734 [setup] WARN s3a.SDKV2Upgrade
> (LogExactlyOnce.java:warn(39)) - Directly referencing AWS SDK V1 credential
> provider com.amazonaws.auth.EnvironmentVariableCredentialsProvider. AWS SDK
> V1 credential providers will be removed once S3A is upgraded to SDK V2
> 2022-10-05 14:09:09,734 [setup] DEBUG s3a.S3AUtils
> (S3AUtils.java:createAWSCredentialProvider(691)) - Credential provider class
> is com.amazonaws.auth.EnvironmentVariableCredentialsProvider
> 2022-10-05 14:09:09,734 [setup] DEBUG s3a.S3AUtils
> (S3AUtils.java:createAWSCredentialProvider(691)) - Credential provider class
> is org.apache.hadoop.fs.s3a.auth.IAMInstanceCredentialsProvider
> {code}
> This is because the EnvironmentVariableCredentialsProvider provider is on the
> default list of providers.
> Everybody who is using the S3 a connector and who has not explicitly declared
> a new set of providers excluding this one will be seeing the error message.
> Proposed:
> Don't warn on this provider. Instead with the v2 move the classname can be
> patched to switch to a modified one.
> The alternative would be to provide an s3a specific env var provider subclass
> of this; and while that is potentially good in future it is a bit more effort
> for the forthcoming 3.3.5 release.
> And especially because and it will not be in previous versions people cannot
> explicitly switch to it in their configs and be confident it will always be
> there,
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
