[ https://issues.apache.org/jira/browse/HADOOP-16415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16883813#comment-16883813 ]
Steve Loughran commented on HADOOP-16415: ----------------------------------------- Thinking for of this We're running the same operation (MR or terasort sequence) with different cluster configs, which is exactly what parameterized test runs can do. So we just need some parameterization which declares everything a specific test run can do: * config options * extra callbacks on validation * expected outcomes then we have a test which brings up the mini yarn cluster in static setup, destroys it in teardown, and has the sets run parameterized The only thing we'd need to do is implement a significantly more complex parameterization than normal, with each one being a class declaring all that is needed for each one. Ideally, one which we could share between the Terasort and the TestMRJob tests > Speed up S3A test runs > ---------------------- > > Key: HADOOP-16415 > URL: https://issues.apache.org/jira/browse/HADOOP-16415 > Project: Hadoop Common > Issue Type: Sub-task > Components: fs/s3 > Affects Versions: 3.3.0 > Reporter: Steve Loughran > Priority: Major > > S3A Test runs are way too slow. > Speed them by > * reducing test setup/teardown costs > * eliminating obsolete test cases > * merge small tests into larger ones. > One thing i see is that the main S3A test cases create and destroy new FS > instances; There's both a setup and teardown cost there, but it does > guarantee better isolation. > Maybe if we know all test cases in a specific suite need the same options, we > can manage that better; demand create the FS but only delete it in an > @Afterclass method. That'd give us the OO-inheritance based setup of tests, > but mean only one instance is done per suite -- This message was sent by Atlassian JIRA (v7.6.14#76016) --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org