yandrey321 opened a new pull request, #11939:
URL: https://github.com/apache/ozone/pull/11939
## What changes were proposed in this pull request?
**Problem.** AWS SDK for Java v1 is end of life, and Ozone still used it in
two places:
- **Freon:** the S3 generators `ozone freon s3kg` and `s3bg`. Because of
freon, the distribution ships
`aws-java-sdk-core`, `aws-java-sdk-s3` and `aws-java-sdk-kms`, plus
`jmespath-java` and `jackson-dataformat-cbor`.
- **Tests:** a v1 copy of the S3 SDK integration suite,
`AbstractS3SDKV1Tests`, about 3,200 lines.
**Change.**
- **Freon moves to SDK v2.**
- `S3EntityGenerator` builds an `S3Client` with the same settings as
before: environment-variable credentials, and
path-style access with region `us-east-1` for an explicit endpoint,
otherwise `us-west-2` (v1's default region).
- `s3kg` (put and multipart upload) and `s3bg` (create bucket) use the v2
API.
- The W3C trace-context `RequestHandler2` is now
`FreonS3TraceContextInterceptor`, a v2 `ExecutionInterceptor`.
- **The v1 test suite is removed.** Every v1 test was mapped to the v2 suite
by what it asserts:
- 59 were already covered;
- 22 had missing or partial coverage, and are ported to
`AbstractS3SDKV2Tests`, in the areas listed under Testing.
- **v1 is removed from the build:** the root pom version property and
managed dependencies, freon,
integration-test-s3, and the `com.amazonaws` logger entries in both
`log4j.properties` files.
- **v1 can't come back.** The root pom adds two enforcer rules:
- `ban-imports` bans `com.amazonaws.**` imports, in main and test code;
- a new `ban-aws-sdk-v1` execution (`bannedDependencies`) fails a module
that depends on any `com.amazonaws`
artifact, even transitively.
- **Distribution licensing:** `jar-report.txt`, `LICENSE.txt` and
`NOTICE.txt` are updated (see Side effects).
**Key decisions.**
- **Freon uses v2's `apache-client`.** It runs on the Apache HttpClient 4
that Ozone already ships, and that v1 used.
`s3` would otherwise also bring `apache5-client` (HttpClient 5) and
`netty-nio-client`, so freon excludes both.
The distribution gains no new third-party jars, only SDK v2's own.
- **Checksums only when required:** `requestChecksumCalculation` and
`responseChecksumValidation` are `WHEN_REQUIRED`.
This replaces v1's `disablePutObjectMD5Validation`: a load generator
should not spend its time on checksums.
- **The v1 tests were ported by behaviour, not by name.** Three v2 tests
have the same name as a v1 test but check
something else (the expected-bucket-owner header), so their v1
counterparts were ported too:
`testDeleteBucket`, `testListMultipartUploads` and `testListParts`.
- **Not ported:** v1 checks that only test the v1 SDK itself:
- `ErrorType`, and `null` returns where v2 already asserts the 412 or 404;
- presigned-URL workarounds for APIs v1 lacked;
- `doesBucketExist*`.
- **The multitenancy-ranger exclusion of `aws-java-sdk-bundle` stays.** It
keeps Ranger's transitive v1 out, which
the new dependency ban now requires anyway.
**Side effects.**
- **Distribution jars:**
- removed: 5 jars, the three v1 jars, `jmespath-java` and
`jackson-dataformat-cbor`;
- added: 31 SDK v2 jars and `software.amazon.eventstream:eventstream`,
about 8.4 MB, all Apache 2.0;
- unchanged: `joda-time`, still used by other dependencies.
- **`NOTICE.txt`** gets the AWS SDK v2 notice. The v2 jars carry one, the v1
jars did not.
- **`ozone freon s3kg` / `s3bg`:** the same options and credentials
(`AWS_ACCESS_KEY_ID` and
`AWS_SECRET_ACCESS_KEY`), and the same requests.
Co-authored-by: Claude Opus 5.5 <[email protected]>
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-17707
## How was this patch tested?
Unit + integration tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
