[ https://issues.apache.org/jira/browse/HADOOP-18435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17599164#comment-17599164 ]
Viraj Jasani commented on HADOOP-18435: --------------------------------------- The way I came to know about this issue is by testing hbase on s3 (hbase WAL on HDFS and actual HFiles on S3, accessed using S3A). I just wanted to see how much perf improvement could be achieved by tuning fs.s3a.executor.capacity. I tried with small bump, and then eventually very high bump but no significant difference was observed. When I looked into the usages of fs.s3a.executor.capacity, realized that it's not being used anywhere as per the above comment. Just to confirm that I have not missed any usecase, I provided value 0 to fs.s3a.executor.capacity (adjusted min value from 1 to 0), and ran all S3A tests with -Dscale, and not a single test failed. On the other hand, HBase doesn't face any issues either. > Remove usage of fs.s3a.executor.capacity > ---------------------------------------- > > Key: HADOOP-18435 > URL: https://issues.apache.org/jira/browse/HADOOP-18435 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 > Reporter: Viraj Jasani > Assignee: Viraj Jasani > Priority: Major > > When s3guard was part of s3a, DynamoDBMetadataStore was the only consumer of > StoreContext that used throttled executor provided by StoreContext, which > internally uses fs.s3a.executor.capacity to determine executor capacity for > SemaphoredDelegatingExecutor. With the removal of s3guard from s3a, we should > also remove fs.s3a.executor.capacity and it's usages as it's no longer being > used by any StoreContext consumers. The config's existence and its > description can be really confusing for the users. -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org