This is an automated email from the ASF dual-hosted git repository.
slfan1989 pushed a commit to branch branch-3.4.0
in repository https://gitbox.apache.org/repos/asf/hadoop.git
The following commit(s) were added to refs/heads/branch-3.4.0 by this push:
new f01d33d4539f HADOOP-19046. S3A: update AWS V2 SDK to 2.23.5; v1 to
1.12.599 (#6467)
f01d33d4539f is described below
commit f01d33d4539f971bd7b1c04594c2294f148b4249
Author: Steve Loughran <ste...@cloudera.com>
AuthorDate: Sun Jan 21 19:00:34 2024 +0000
HADOOP-19046. S3A: update AWS V2 SDK to 2.23.5; v1 to 1.12.599 (#6467)
This update ensures that the timeout set in
fs.s3a.connection.request.timeout is passed down
to calls to CreateSession made in the AWS SDK to get S3 Express session
tokens.
Contributed by Steve Loughran
---
LICENSE-binary | 4 ++--
NOTICE-binary | 2 +-
hadoop-project/pom.xml | 4 ++--
.../src/site/markdown/tools/hadoop-aws/testing.md | 13 ++++++++++---
4 files changed, 15 insertions(+), 8 deletions(-)
diff --git a/LICENSE-binary b/LICENSE-binary
index 3720a7809563..93e38cc34ee4 100644
--- a/LICENSE-binary
+++ b/LICENSE-binary
@@ -215,7 +215,7 @@ com.aliyun:aliyun-java-sdk-kms:2.11.0
com.aliyun:aliyun-java-sdk-ram:3.1.0
com.aliyun:aliyun-java-sdk-sts:3.0.0
com.aliyun.oss:aliyun-sdk-oss:3.13.2
-com.amazonaws:aws-java-sdk-bundle:1.12.565
+com.amazonaws:aws-java-sdk-bundle:1.12.599
com.cedarsoftware:java-util:1.9.0
com.cedarsoftware:json-io:2.5.1
com.fasterxml.jackson.core:jackson-annotations:2.12.7
@@ -363,7 +363,7 @@ org.objenesis:objenesis:2.6
org.xerial.snappy:snappy-java:1.1.10.4
org.yaml:snakeyaml:2.0
org.wildfly.openssl:wildfly-openssl:1.1.3.Final
-software.amazon.awssdk:bundle:jar:2.21.41
+software.amazon.awssdk:bundle:jar:2.23.5
--------------------------------------------------------------------------------
diff --git a/NOTICE-binary b/NOTICE-binary
index 6db51d08b42f..7389a31fd5a1 100644
--- a/NOTICE-binary
+++ b/NOTICE-binary
@@ -66,7 +66,7 @@ available from http://www.digip.org/jansson/.
AWS SDK for Java
-Copyright 2010-2022 Amazon.com, Inc. or its affiliates. All Rights Reserved.
+Copyright 2010-2024 Amazon.com, Inc. or its affiliates. All Rights Reserved.
This product includes software developed by
Amazon Technologies, Inc (http://www.amazon.com/).
diff --git a/hadoop-project/pom.xml b/hadoop-project/pom.xml
index 2c0cc3edcb6c..b79fdd576109 100644
--- a/hadoop-project/pom.xml
+++ b/hadoop-project/pom.xml
@@ -186,8 +186,8 @@
<exec-maven-plugin.version>1.3.1</exec-maven-plugin.version>
<make-maven-plugin.version>1.0-beta-1</make-maven-plugin.version>
<surefire.fork.timeout>900</surefire.fork.timeout>
- <aws-java-sdk.version>1.12.565</aws-java-sdk.version>
- <aws-java-sdk-v2.version>2.21.41</aws-java-sdk-v2.version>
+ <aws-java-sdk.version>1.12.599</aws-java-sdk.version>
+ <aws-java-sdk-v2.version>2.23.5</aws-java-sdk-v2.version>
<aws.eventstream.version>1.0.1</aws.eventstream.version>
<hsqldb.version>2.7.1</hsqldb.version>
<frontend-maven-plugin.version>1.11.2</frontend-maven-plugin.version>
diff --git
a/hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/testing.md
b/hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/testing.md
index c2eafbcb8de2..62d449daeea5 100644
--- a/hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/testing.md
+++ b/hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/testing.md
@@ -1142,7 +1142,7 @@ as it may take a couple of SDK updates before it is ready.
1. Identify the latest AWS SDK [available for
download](https://aws.amazon.com/sdk-for-java/).
1. Create a private git branch of trunk for JIRA, and in
`hadoop-project/pom.xml` update the `aws-java-sdk.version` to the new SDK
version.
-1. Update AWS SDK versions in NOTICE.txt.
+1. Update AWS SDK versions in NOTICE.txt and LICENSE.binary
1. Do a clean build and rerun all the `hadoop-aws` tests.
This includes the `-Pscale` set, with a role defined for the assumed role
tests.
in `fs.s3a.assumed.role.arn` for testing assumed roles,
@@ -1164,11 +1164,18 @@ your IDE or via maven.
`mvn dependency:tree -Dverbose > target/dependencies.txt`.
Examine the `target/dependencies.txt` file to verify that no new
artifacts have unintentionally been declared as dependencies
- of the shaded `aws-java-sdk-bundle` artifact.
+ of the shaded `software.amazon.awssdk:bundle:jar` artifact.
1. Run a full AWS-test suite with S3 client-side encryption enabled by
setting `fs.s3a.encryption.algorithm` to 'CSE-KMS' and setting up AWS-KMS
Key ID in `fs.s3a.encryption.key`.
+The dependency chain of the `hadoop-aws` module should be similar to this,
albeit
+with different version numbers:
+```
+[INFO] +- org.apache.hadoop:hadoop-aws:jar:3.4.0-SNAPSHOT:compile
+[INFO] | +- software.amazon.awssdk:bundle:jar:2.23.5:compile
+[INFO] | \- org.wildfly.openssl:wildfly-openssl:jar:1.1.3.Final:compile
+```
### Basic command line regression testing
We need a run through of the CLI to see if there have been changes there
@@ -1365,5 +1372,5 @@ Don't be surprised if this happens, don't worry too much,
and,
while that rollback option is there to be used, ideally try to work forwards.
If the problem is with the SDK, file issues with the
- [AWS SDK Bug tracker](https://github.com/aws/aws-sdk-java/issues).
+ [AWS V2 SDK Bug tracker](https://github.com/aws/aws-sdk-java-v2/issues).
If the problem can be fixed or worked around in the Hadoop code, do it there
too.
---------------------------------------------------------------------
To unsubscribe, e-mail: common-commits-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-commits-h...@hadoop.apache.org
