[ https://issues.apache.org/jira/browse/HADOOP-18853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17755901#comment-17755901 ]
ASF GitHub Bot commented on HADOOP-18853: ----------------------------------------- ahmarsuhail opened a new pull request, #5960: URL: https://github.com/apache/hadoop/pull/5960 ### Description of PR * Upgrades SDK Version so it has the new Java Async client with MPU support. * Configures java async client with MPU threshold and part size * Removes CRT dependency Since Multipart operations are in the java async client now, and not the TM, we could consider removing the TM in a follow up PR. We'd lose out on the transfer listener..but we don't really use any of the other additional functionality the TM provides us. ### How was this patch tested? Testing in progress in eu-west-1 with `mvn -Dparallel-tests -DtestsThreadCount=16 clean verify` Also ran checked output of ITestS3HugeFileArrayBlocks.test_100_renameHugeFile(), time taken to rename a 256MB file is at par with V1, around ~1s from my m4.2xlarge EC2. > AWS SDK V2 - Integrate new transfer manager > ------------------------------------------- > > Key: HADOOP-18853 > URL: https://issues.apache.org/jira/browse/HADOOP-18853 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 > Affects Versions: 3.4.0 > Reporter: Ahmar Suhail > Priority: Major > > With 2.20.121, the TM has MPU functionality. Upgrading to to this version > will also solve the issue with needing to include the CRT dependency. -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org