[ https://issues.apache.org/jira/browse/HADOOP-19093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17829910#comment-17829910 ]
ASF GitHub Bot commented on HADOOP-19093: ----------------------------------------- steveloughran commented on PR #6596: URL: https://github.com/apache/hadoop/pull/6596#issuecomment-2015343365 Latest PR adds op-specific capacity multiplication to abfs store; numbers are pretty arbitrary as this is only used in the committer. could be tuned in future. > Improve rate limiting through ABFS in Manifest Committer > -------------------------------------------------------- > > Key: HADOOP-19093 > URL: https://issues.apache.org/jira/browse/HADOOP-19093 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/azure, test > Affects Versions: 3.4.0 > Reporter: Steve Loughran > Assignee: Steve Loughran > Priority: Major > Labels: pull-request-available > > I need a load test to verify that the rename resilience of the manifest > committer actually works as intended > * test suite with name ILoadTest* prefix (as with s3) > * parallel test running with many threads doing many renames > * verify that rename recovery should be detected > * and that all renames MUST NOT fail. > maybe also: metrics for this in fs and doc update. > Possibly; LogExactlyOnce to warn of load issues -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org