Repository: spark Updated Branches: refs/heads/branch-1.6 649a0a357 -> 85bc72908
[SPARK-11463] [PYSPARK] only install signal in main thread Only install signal in main thread, or it will fail to create context in not-main thread. Author: Davies Liu <dav...@databricks.com> Closes #9574 from davies/python_signal. (cherry picked from commit bd70244b3cda62cc447fd4cc343d4eb5ddaec893) Signed-off-by: Davies Liu <davies....@gmail.com> Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/85bc7290 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/85bc7290 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/85bc7290 Branch: refs/heads/branch-1.6 Commit: 85bc7290881bee4f02833f0802571ef589e96856 Parents: 649a0a3 Author: Davies Liu <dav...@databricks.com> Authored: Tue Nov 10 22:46:17 2015 -0800 Committer: Davies Liu <davies....@gmail.com> Committed: Tue Nov 10 22:46:28 2015 -0800 ---------------------------------------------------------------------- python/pyspark/context.py | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/85bc7290/python/pyspark/context.py ---------------------------------------------------------------------- diff --git a/python/pyspark/context.py b/python/pyspark/context.py index afd74d9..77710a1 100644 --- a/python/pyspark/context.py +++ b/python/pyspark/context.py @@ -21,6 +21,7 @@ import os import shutil import signal import sys +import threading from threading import RLock from tempfile import NamedTemporaryFile @@ -222,7 +223,9 @@ class SparkContext(object): def signal_handler(signal, frame): self.cancelAllJobs() - signal.signal(signal.SIGINT, signal_handler) + # see http://stackoverflow.com/questions/23206787/ + if isinstance(threading.current_thread(), threading._MainThread): + signal.signal(signal.SIGINT, signal_handler) def _initialize_context(self, jconf): """ --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
