[GitHub] [hadoop] virajjasani commented on pull request #5908: HADOOP-18832. Upgrade aws-java-sdk to 1.12.499
virajjasani commented on PR #5908: URL: https://github.com/apache/hadoop/pull/5908#issuecomment-1676685837 i have repeated above steps for both `-Dscale -Dprefetch` as well as `-Dscale` and confirmed above 3 points for both rounds -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] virajjasani commented on pull request #5908: HADOOP-18832. Upgrade aws-java-sdk to 1.12.499
virajjasani commented on PR #5908: URL: https://github.com/apache/hadoop/pull/5908#issuecomment-1666112780 > [ ] run the CLI tests on a full build done > [ ] looked in all the logs of the test runs for new messages no surprises so far, scanned logs for many of the long running scale tests > [ ] reviewed the dependencies of the bundle to see if something else is now referenced which we need to exclude? this was pending until now, we are good w.r.t dependencies from the bundle, no need to exclude anything other than `io.netty:*` that is already done ``` com.amazonaws:aws-java-sdk-bundle:jar:1.12.499 \- com.amazonaws:aws-java-sdk:jar:1.12.499:compile +- com.amazonaws:aws-java-sdk-appfabric:jar:1.12.499:compile | \- com.amazonaws:jmespath-java:jar:1.12.499:compile +- com.amazonaws:aws-java-sdk-verifiedpermissions:jar:1.12.499:compile +- com.amazonaws:aws-java-sdk-codegurusecurity:jar:1.12.499:compile ... ... ... +- com.amazonaws:aws-java-sdk-iotjobsdataplane:jar:1.12.499:compile +- com.amazonaws:aws-java-sdk-sagemakerruntime:jar:1.12.499:compile +- com.amazonaws:aws-java-sdk-kinesisvideo:jar:1.12.499:compile | +- io.netty:netty-codec-http:jar:4.1.94.Final:compile | | +- io.netty:netty-common:jar:4.1.94.Final:compile | | +- io.netty:netty-buffer:jar:4.1.94.Final:compile | | +- io.netty:netty-transport:jar:4.1.94.Final:compile | | \- io.netty:netty-codec:jar:4.1.94.Final:compile | \- io.netty:netty-handler:jar:4.1.94.Final:compile | +- io.netty:netty-resolver:jar:4.1.94.Final:compile | \- io.netty:netty-transport-native-unix-common:jar:4.1.94.Final:compile +- com.amazonaws:aws-java-sdk-appsync:jar:1.12.499:compile +- com.amazonaws:aws-java-sdk-guardduty:jar:1.12.499:compile ... ... ... +- com.amazonaws:aws-java-sdk-workdocs:jar:1.12.499:compile +- com.amazonaws:aws-java-sdk-core:jar:1.12.499:compile | +- commons-logging:commons-logging:jar:1.1.3:compile | +- commons-codec:commons-codec:jar:1.15:compile | +- org.apache.httpcomponents:httpclient:jar:4.5.13:compile | | \- org.apache.httpcomponents:httpcore:jar:4.4.13:compile | +- software.amazon.ion:ion-java:jar:1.0.2:compile | +- com.fasterxml.jackson.core:jackson-databind:jar:2.12.7.1:compile | | +- com.fasterxml.jackson.core:jackson-annotations:jar:2.12.7:compile | | \- com.fasterxml.jackson.core:jackson-core:jar:2.12.7:compile | +- com.fasterxml.jackson.dataformat:jackson-dataformat-cbor:jar:2.12.6:compile | \- joda-time:joda-time:jar:2.8.1:compile +- com.amazonaws:aws-java-sdk-models:jar:1.12.499:compile \- com.amazonaws:aws-java-sdk-swf-libraries:jar:1.11.22:compile ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] virajjasani commented on pull request #5908: HADOOP-18832. Upgrade aws-java-sdk to 1.12.499
virajjasani commented on PR #5908: URL: https://github.com/apache/hadoop/pull/5908#issuecomment-1660890327 load tests are passing: ``` org.apache.hadoop.fs.s3a.scale.ILoadTestS3ABulkDeleteThrottling org.apache.hadoop.fs.s3a.auth.delegation.ILoadTestSessionCredentials org.apache.hadoop.fs.s3a.auth.delegation.ILoadTestRoleCredentials ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] virajjasani commented on pull request #5908: HADOOP-18832. Upgrade aws-java-sdk to 1.12.499
virajjasani commented on PR #5908:
URL: https://github.com/apache/hadoop/pull/5908#issuecomment-1659539707
re-ran the tests with assume role and encryption enabled:
the only tests that are getting ignored are:
- contract tests that don't apply to s3a (e.g.
`fs.capability.etags.preserved.in.rename`)
- `ITestS3AContractSeek`: `Tests run: 72, Failures: 0, Errors: 0, Skipped:
24, Time elapsed: 198.155 s - in
org.apache.hadoop.fs.contract.s3a.ITestS3AContractSeek` 24 tests getting
skipped because they need native hadoop lib:
```
if (this.sslChannelMode == OpenSSL) {
assumeTrue(NativeCodeLoader.isNativeCodeLoaded() &&
NativeCodeLoader.buildSupportsOpenssl());
}
```
everything else is passing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] virajjasani commented on pull request #5908: HADOOP-18832. Upgrade aws-java-sdk to 1.12.499
virajjasani commented on PR #5908: URL: https://github.com/apache/hadoop/pull/5908#issuecomment-1658954373 `AbstractSTestS3AHugeFiles` is successful with `SSE-KMS`, will run assumed role tests now -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] virajjasani commented on pull request #5908: HADOOP-18832. Upgrade aws-java-sdk to 1.12.499
virajjasani commented on PR #5908: URL: https://github.com/apache/hadoop/pull/5908#issuecomment-1658777571 yes, assumed role test coverage is pending -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] virajjasani commented on pull request #5908: HADOOP-18832. Upgrade aws-java-sdk to 1.12.499
virajjasani commented on PR #5908: URL: https://github.com/apache/hadoop/pull/5908#issuecomment-1657726850 With encryption enabled: ``` $ mvn clean verify -Dparallel-tests -DtestsThreadCount=8 -Dscale -Dprefetch [INFO] --- maven-failsafe-plugin:3.0.0-M1:integration-test (default-integration-test) @ hadoop-aws --- [INFO] [INFO] --- [INFO] T E S T S [INFO] --- [INFO] Results: [INFO] [WARNING] Tests run: 1170, Failures: 0, Errors: 0, Skipped: 143 [INFO] --- maven-failsafe-plugin:3.0.0-M1:integration-test (sequential-integration-tests) @ hadoop-aws --- [INFO] [INFO] --- [INFO] T E S T S [INFO] --- [INFO] Results: [INFO] [ERROR] Errors: [ERROR] ITestS3AContractRootDir>AbstractContractRootDirectoryTest.testRecursiveRootListing:266 » TestTimedOut [INFO] [ERROR] Tests run: 135, Failures: 0, Errors: 1, Skipped: 10 [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 33:28 min [INFO] Finished at: 2023-07-30T16:57:05-07:00 [INFO] ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[GitHub] [hadoop] virajjasani commented on pull request #5908: HADOOP-18832. Upgrade aws-java-sdk to 1.12.499
virajjasani commented on PR #5908: URL: https://github.com/apache/hadoop/pull/5908#issuecomment-1657720539 us-west-2 ``` $ mvn clean verify -Dparallel-tests -DtestsThreadCount=8 -Dscale -Dprefetch [INFO] --- maven-failsafe-plugin:3.0.0-M1:integration-test (default-integration-test) @ hadoop-aws --- [INFO] [INFO] --- [INFO] T E S T S [INFO] --- [INFO] Results: [INFO] [WARNING] Tests run: 1170, Failures: 0, Errors: 0, Skipped: 148 [INFO] --- maven-failsafe-plugin:3.0.0-M1:integration-test (sequential-integration-tests) @ hadoop-aws --- [INFO] [INFO] --- [INFO] T E S T S [INFO] --- [INFO] Results: [INFO] [ERROR] Errors: [ERROR] ITestS3AContractRootDir>AbstractContractRootDirectoryTest.testRecursiveRootListing:266 » TestTimedOut [INFO] [ERROR] Tests run: 135, Failures: 0, Errors: 1, Skipped: 10 [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 29:35 min [INFO] Finished at: 2023-07-30T14:54:18-07:00 [INFO] ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
