Repository: spark Updated Branches: refs/heads/master 1a8e0468a -> bd70244b3
[SPARK-11463] [PYSPARK] only install signal in main thread Only install signal in main thread, or it will fail to create context in not-main thread. Author: Davies Liu <dav...@databricks.com> Closes #9574 from davies/python_signal. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/bd70244b Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/bd70244b Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/bd70244b Branch: refs/heads/master Commit: bd70244b3cda62cc447fd4cc343d4eb5ddaec893 Parents: 1a8e046 Author: Davies Liu <dav...@databricks.com> Authored: Tue Nov 10 22:46:17 2015 -0800 Committer: Davies Liu <davies....@gmail.com> Committed: Tue Nov 10 22:46:17 2015 -0800 ---------------------------------------------------------------------- python/pyspark/context.py | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/bd70244b/python/pyspark/context.py ---------------------------------------------------------------------- diff --git a/python/pyspark/context.py b/python/pyspark/context.py index afd74d9..77710a1 100644 --- a/python/pyspark/context.py +++ b/python/pyspark/context.py @@ -21,6 +21,7 @@ import os import shutil import signal import sys +import threading from threading import RLock from tempfile import NamedTemporaryFile @@ -222,7 +223,9 @@ class SparkContext(object): def signal_handler(signal, frame): self.cancelAllJobs() - signal.signal(signal.SIGINT, signal_handler) + # see http://stackoverflow.com/questions/23206787/ + if isinstance(threading.current_thread(), threading._MainThread): + signal.signal(signal.SIGINT, signal_handler) def _initialize_context(self, jconf): """ --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
