[ https://issues.apache.org/jira/browse/HADOOP-18382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17576941#comment-17576941 ]
ASF GitHub Bot commented on HADOOP-18382: ----------------------------------------- steveloughran commented on code in PR #4698: URL: https://github.com/apache/hadoop/pull/4698#discussion_r940562613 ########## hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/aws_sdk_upgrade.md: ########## @@ -0,0 +1,76 @@ +<!--- + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. See accompanying LICENSE file. +--> + +# Upgrading S3A to AWS SDK V2 + +This document explains the upcoming work for upgrading S3A to AWS SDK V2. +This work is tracked in [HADOOP-18073](https://issues.apache.org/jira/browse/HADOOP-18073). + +## Why do we want to upgrade? Review Comment: prefer "why the upgrade" ########## hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java: ########## @@ -849,6 +849,10 @@ private void bindAWSClient(URI name, boolean dtEnabled) throws IOException { // with it if so. LOG.debug("Using delegation tokens"); + LOG.warn( Review Comment: you need to use a LogOnce; we should consider having a special log just for v2 warnings, so people can turn it off entirely (i would for example) ########## hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AUtils.java: ########## @@ -637,6 +637,11 @@ public static AWSCredentialProviderList buildAWSProviderList( AWSCredentialProviderList providers = new AWSCredentialProviderList(); for (Class<?> aClass : awsClasses) { + if (aClass.getName().contains(AWS_AUTH_CLASS_PREFIX)) { Review Comment: again, log through the proposed LogOnce log > Upgrade AWS SDK to V2 - Prerequisites > -------------------------------------- > > Key: HADOOP-18382 > URL: https://issues.apache.org/jira/browse/HADOOP-18382 > Project: Hadoop Common > Issue Type: Sub-task > Reporter: Ahmar Suhail > Assignee: Ahmar Suhail > Priority: Minor > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > We want to update the AWS SDK to V2, before we do this we should warn on > things that will no longer supported. The following changes should be made: > > * > [getAmazonS3Client()|https://github.com/apache/hadoop/blob/221eb2d68d5b52e4394fd36cb30d5ee9ffeea7f0/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java#L1174] > - Warn that this method will be removed > * > [initCustomSigners()|https://github.com/apache/hadoop/blob/03cfc852791c14fad39db4e5b14104a276c08e59/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/auth/SignerManager.java#L65] > - Warn that the interface is changing, any custom signers will need to be > updated > * > [bindAWSClient|https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java#L840] > - If DT is enabled, warn that credential providers interface is changing, > any custom cred providers used in binding classes will need to be updated > * > [buildAWSProviderList|https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AUtils.java#L618] > - if any SDK V1 cred providers are in this list, warn that these will be > removed > * > [S3ClientFactory|https://github.com/apache/hadoop/blob/221eb2d68d5b52e4394fd36cb30d5ee9ffeea7f0/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3ClientFactory.java] > - Update javadocs to say this interface will be replaced by a V2 client > factory, mark interface deprecated? -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org