[ https://issues.apache.org/jira/browse/HADOOP-16415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17258415#comment-17258415 ]
Steve Loughran commented on HADOOP-16415: ----------------------------------------- h3. Huge tests we have too many of the Huge upload tests, one for each buffer mechanism. Proposed: * only test disk buffering * make sure we have unit tests for the others for large buffers which verify we can mark/reset back to the beginning, which is what the aws sdk needs h3. Surprisingly slow 85.901 s - in org.apache.hadoop.fs.s3a.fileContext.ITestS3AFileContextURI -- too many exists/isFile/isDir checks. Best to only do isDir > Speed up S3A test runs > ---------------------- > > Key: HADOOP-16415 > URL: https://issues.apache.org/jira/browse/HADOOP-16415 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 > Affects Versions: 3.3.0 > Reporter: Steve Loughran > Priority: Major > > S3A Test runs are way too slow. > Speed them by > * reducing test setup/teardown costs > * eliminating obsolete test cases > * merge small tests into larger ones. > One thing i see is that the main S3A test cases create and destroy new FS > instances; There's both a setup and teardown cost there, but it does > guarantee better isolation. > Maybe if we know all test cases in a specific suite need the same options, we > can manage that better; demand create the FS but only delete it in an > @Afterclass method. That'd give us the OO-inheritance based setup of tests, > but mean only one instance is done per suite -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org