[
https://issues.apache.org/jira/browse/HADOOP-18820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17749241#comment-17749241
]
ASF GitHub Bot commented on HADOOP-18820:
-----------------------------------------
steveloughran commented on code in PR #5872:
URL: https://github.com/apache/hadoop/pull/5872#discussion_r1279491510
##########
hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/troubleshooting_s3a.md:
##########
@@ -70,14 +70,45 @@ These are Hadoop filesystem client classes, found in the
`hadoop-aws` JAR.
An exception reporting this class as missing means that this JAR is not on
the classpath.
-### `ClassNotFoundException: com.amazonaws.services.s3.AmazonS3Client`
-(or other `com.amazonaws` class.)
+### `NoClassDefFoundError: software/amazon/awssdk/crt/s3/S3MetaRequest`
+
+The library `aws-crt.jar` is not on the classpath. Its classes
+are not in the AWS `bundle.jar` file, yet are needed for uploading
+and renaming objects.
+
+Fix: add.
+
+```
+java.lang.BootstrapMethodError: java.lang.NoClassDefFoundError:
software/amazon/awssdk/crt/s3/S3MetaRequest
+at
software.amazon.awssdk.services.s3.internal.crt.S3MetaRequestPauseObservable.<init>(S3MetaRequestPauseObservable.java:33)
+at
software.amazon.awssdk.transfer.s3.internal.DefaultS3TransferManager.uploadFile(DefaultS3TransferManager.java:205)
+at org.apache.hadoop.fs.s3a.S3AFileSystem.putObject(S3AFileSystem.java:3064)
+at org.apache.hadoop.fs.s3a.S3AFileSystem.executePut(S3AFileSystem.java:4054)
+
+```
+### `ClassNotFoundException: software.amazon.awssdk.services.s3.S3Client`
-This means that the `aws-java-sdk-bundle.jar` JAR is not on the classpath:
+(or other `software.amazon` class.)
+
+This means that the AWS V2 SDK `bundle.jar` JAR is not on the classpath:
add it.
Review Comment:
yes, because people forget to include them when manually setting up their
spark installations, where they just drop in a random pair of hadoop-aws and
aws-sdk JARs
> AWS SDK v2: make the v1 bridging support optional
> -------------------------------------------------
>
> Key: HADOOP-18820
> URL: https://issues.apache.org/jira/browse/HADOOP-18820
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.4.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Major
> Labels: pull-request-available
>
> The AWS SDK v2 code includes the v1 sdk core for plugin support of
> * existing credential providers
> * delegation token binding
> I propose we break #2 and rely on those who have implemented to to upgrade.
> apart from all the needless changes the v2 SDK did to the api (why?) this is
> fairly straighforward
> for #1: fix through reflection, retaining a v1 sdk dependency at test time so
> we can verify that the binder works.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
