[GitHub] spark pull request #23033: [SPARK-26036][PYTHON] Break large tests.py files ...

[HyukjinKwon]

GitHub user HyukjinKwon opened a pull request:

    https://github.com/apache/spark/pull/23033

    [SPARK-26036][PYTHON] Break large tests.py files into smaller files

    ## What changes were proposed in this pull request?
    
    This PR continues to break down a big large file into smaller files. See 
https://github.com/apache/spark/pull/23021. It targets to follow 
https://github.com/numpy/numpy/tree/master/numpy.
    
    Basically this PR proposes to break down `pyspark/tests.py` into ...:
    
    ```
    pyspark
    ...
    ├── testing
    ...
    │   └── utils.py
    ├── tests
    │   ├── __init__.py
    │   ├── test_appsubmit.py
    │   ├── test_broadcast.py
    │   ├── test_conf.py
    │   ├── test_context.py
    │   ├── test_daemon.py
    │   ├── test_join.py
    │   ├── test_profiler.py
    │   ├── test_rdd.py
    │   ├── test_readwrite.py
    │   ├── test_serializers.py
    │   ├── test_shuffle.py
    │   ├── test_taskcontext.py
    │   ├── test_util.py
    │   └── test_worker.py
    ...
    ```
    
    
    ## How was this patch tested?
    
    Existing tests should cover.
    
    `cd python` and .`/run-tests-with-coverage`. Manually checked they are 
actually being ran.
    
    Each test (not officially) can be ran via:
    
    ```bash
    SPARK_TESTING=1 ./bin/pyspark pyspark.tests.test_context
    ```
    
    Note that if you're using Mac and Python 3, you might have to 
`OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES`.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/HyukjinKwon/spark SPARK-26036

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/23033.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #23033
    
----
commit 08cb59eeda3fd3b1042013f72f6fc45ea1146bd1
Author: hyukjinkwon <gurwls223@...>
Date:   2018-11-14T10:16:13Z

    Break large tests.py files into smaller files

----


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org