This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch branch-2.4
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-2.4 by this push:
new dc09a02 [SPARK-28582][PYSPARK] Fix flaky test
DaemonTests.do_termination_test which fail on Python 3.7
dc09a02 is described below
commit dc09a02c142d3787e728c8b25eb8417649d98e9f
Author: WeichenXu
AuthorDate: Fri Aug 2 22:07:06 2019 +0900
[SPARK-28582][PYSPARK] Fix flaky test DaemonTests.do_termination_test which
fail on Python 3.7
Fix flaky test DaemonTests.do_termination_test which fail on Python 3.7. I
add a sleep after the test connection to daemon.
Run test
```
python/run-tests --python-executables=python3.7 --testname
"pyspark.tests.test_daemon DaemonTests"
```
**Before**
Fail on test "test_termination_sigterm". And we can see daemon process do
not exit.
**After**
Test passed
Closes #25315 from WeichenXu123/fix_py37_daemon.
Authored-by: WeichenXu
Signed-off-by: HyukjinKwon
(cherry picked from commit fbeee0c5bcea32346b2279c5b67044f12e5faf7d)
Signed-off-by: HyukjinKwon
---
python/pyspark/tests.py | 7 +--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/python/pyspark/tests.py b/python/pyspark/tests.py
index a2d825b..fef2959 100644
--- a/python/pyspark/tests.py
+++ b/python/pyspark/tests.py
@@ -1916,7 +1916,7 @@ class OutputFormatTests(ReusedPySparkTestCase):
class DaemonTests(unittest.TestCase):
def connect(self, port):
-from socket import socket, AF_INET, SOCK_STREAM
+from socket import socket, AF_INET, SOCK_STREAM# request shutdown
sock = socket(AF_INET, SOCK_STREAM)
sock.connect(('127.0.0.1', port))
# send a split index of -1 to shutdown the worker
@@ -1939,9 +1939,12 @@ class DaemonTests(unittest.TestCase):
# daemon should accept connections
self.assertTrue(self.connect(port))
+# wait worker process spawned from daemon exit.
+time.sleep(1)
+
# request shutdown
terminator(daemon)
-time.sleep(1)
+daemon.wait(5)
# daemon should no longer accept connections
try:
-
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org
