GitHub user gengliangwang opened a pull request:
https://github.com/apache/spark/pull/22643
[SPARK-25630][TEST] Reduce test time of HadoopFsRelationTest
## What changes were proposed in this pull request?
There was 5 suites extends `HadoopFsRelationTest`, for testing
"orc"/"parquet"/"text"/"json" data sources.
This PR refactor the base trait `HadoopFsRelationTest`:
1. Rename unnecessary loop for setting parquet conf
2. The test case `SPARK-8406: Avoids name collision while writing files`
takes about 14 to 20 seconds. As now all the file format data source are using
common code, for creating result files, we can test one data source(Parquet)
only to reduce test time.
The total test run time is reduce from 6 minutes to 4.5 minutes.
## How was this patch tested?
Unit test
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/gengliangwang/spark
refactorHadoopFsRelationTest
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/22643.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #22643
----
commit 9a74db0195ca775878b5fec65fe38928c09c1792
Author: Gengliang Wang <gengliang.wang@...>
Date: 2018-10-05T15:53:33Z
refactor HadoopFsRelationTest
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
