[spark] branch master updated: [SPARK-28582][PYTHON] Fix flaky test DaemonTests.do_termination_test which fail on Python 3.7

[gurwls223]

This is an automated email from the ASF dual-hosted git repository.

gurwls223 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/master by this push:
     new b3394db  [SPARK-28582][PYTHON] Fix flaky test 
DaemonTests.do_termination_test which fail on Python 3.7
b3394db is described below

commit b3394db1930b3c9f55438cb27bb2c584bf041f8e
Author: WeichenXu <weichen...@databricks.com>
AuthorDate: Sat Aug 3 10:31:15 2019 +0900

    [SPARK-28582][PYTHON] Fix flaky test DaemonTests.do_termination_test which 
fail on Python 3.7
    
    ## What changes were proposed in this pull request?
    
    This PR picks up https://github.com/apache/spark/pull/25315 back after 
removing `Popen.wait` usage which exists in Python 3 only. I saw the last test 
results wrongly and thought it was passed.
    
    Fix flaky test DaemonTests.do_termination_test which fail on Python 3.7. I 
add a sleep after the test connection to daemon.
    
    ## How was this patch tested?
    
    Run test
    ```
    python/run-tests --python-executables=python3.7 --testname 
"pyspark.tests.test_daemon DaemonTests"
    ```
    **Before**
    Fail on test "test_termination_sigterm". And we can see daemon process do 
not exit.
    **After**
    Test passed
    
    Closes #25343 from HyukjinKwon/SPARK-28582.
    
    Authored-by: WeichenXu <weichen...@databricks.com>
    Signed-off-by: HyukjinKwon <gurwls...@apache.org>
---
 python/pyspark/tests/test_daemon.py | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/python/pyspark/tests/test_daemon.py 
b/python/pyspark/tests/test_daemon.py
index 2cdc16c..898fb39 100644
--- a/python/pyspark/tests/test_daemon.py
+++ b/python/pyspark/tests/test_daemon.py
@@ -47,6 +47,9 @@ class DaemonTests(unittest.TestCase):
         # daemon should accept connections
         self.assertTrue(self.connect(port))
 
+        # wait worker process spawned from daemon exit.
+        time.sleep(1)
+
         # request shutdown
         terminator(daemon)
         time.sleep(1)


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org