[
https://issues.apache.org/jira/browse/HADOOP-16415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17258420#comment-17258420
]
Steve Loughran commented on HADOOP-16415:
-----------------------------------------
v1 list API calls are 100s. Do we need these? Better: a minimal set of tests
102.663 s - in org.apache.hadoop.fs.s3a.ITestS3AContractGetFileStatusV1List
ITestS3ARemoteFileChanged is 800s, because it is so parameterized, including on
change detection policy on open streams.
Not all tests change behaviour on those options, especially the rename ones.
Better: split into tests which read file data, and tests which just manipulate
files.
With S3Guard off, we should still need to test what happens when a file is
changed while open. We shouldn't need to worry about mismatch between listing
and opened/renamed files.
> Speed up S3A test runs
> ----------------------
>
> Key: HADOOP-16415
> URL: https://issues.apache.org/jira/browse/HADOOP-16415
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.3.0
> Reporter: Steve Loughran
> Priority: Major
>
> S3A Test runs are way too slow.
> Speed them by
> * reducing test setup/teardown costs
> * eliminating obsolete test cases
> * merge small tests into larger ones.
> One thing i see is that the main S3A test cases create and destroy new FS
> instances; There's both a setup and teardown cost there, but it does
> guarantee better isolation.
> Maybe if we know all test cases in a specific suite need the same options, we
> can manage that better; demand create the FS but only delete it in an
> @Afterclass method. That'd give us the OO-inheritance based setup of tests,
> but mean only one instance is done per suite
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
