[GitHub] spark pull request #23034: [WIP][SPARK-26035][PYTHON] Break large streaming/...

[HyukjinKwon]

GitHub user HyukjinKwon opened a pull request:

    https://github.com/apache/spark/pull/23034

    [WIP][SPARK-26035][PYTHON] Break large streaming/tests.py files into 
smaller files

    ## What changes were proposed in this pull request?
    
    This PR continues to break down a big large file into smaller files. See 
https://github.com/apache/spark/pull/23021. It targets to follow 
https://github.com/numpy/numpy/tree/master/numpy.
    
    Basically this PR proposes to break down `pyspark/streaming/tests.py` into 
...:
    
    ```
    pyspark
    ├── __init__.py
    ...
    ├── streaming
    │   ├── __init__.py
    ...
    │   ├── tests
    │   │   ├── __init__.py
    │   │   ├── test_context.py
    │   │   ├── test_dstream.py
    │   │   ├── test_kinesis.py
    │   │   └── test_listener.py
    ...
    ├── testing
    ...
    │   ├── streamingutils.py
    ...
    ```
    
    
    ## How was this patch tested?
    
    Existing tests should cover.
    
    `cd python` and .`/run-tests-with-coverage`. Manually checked they are 
actually being ran.
    
    Each test (not officially) can be ran via:
    
    ```bash
    SPARK_TESTING=1 ./bin/pyspark pyspark.tests.test_context
    ```
    
    Note that if you're using Mac and Python 3, you might have to 
`OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES`.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/HyukjinKwon/spark SPARK-26035

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/23034.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #23034
    
----
commit 6c02003b947a97944f6a48e57be5b7c98dbda896
Author: hyukjinkwon <gurwls223@...>
Date:   2018-11-14T15:15:56Z

    [SPARK-26035][PYTHON] Break large streaming/tests.py files into smaller 
files

----


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org