Repository: spark Updated Branches: refs/heads/master bb2b9d0a4 -> 801e4d097
[SPARK-16606][CORE] Misleading warning for SparkContext.getOrCreate "WARN SparkContext: Use an existing SparkContext, some configuration may not take effect." ## What changes were proposed in this pull request? SparkContext.getOrCreate shouldn't warn about ignored config if - it wasn't ignored because a new context is created with it or - no config was actually provided ## How was this patch tested? Jenkins + existing tests. Author: Sean Owen <so...@cloudera.com> Closes #14533 from srowen/SPARK-16606. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/801e4d09 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/801e4d09 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/801e4d09 Branch: refs/heads/master Commit: 801e4d097f45b269a9c6b25723d925f3e24ba498 Parents: bb2b9d0 Author: Sean Owen <so...@cloudera.com> Authored: Tue Aug 9 09:38:12 2016 +0100 Committer: Sean Owen <so...@cloudera.com> Committed: Tue Aug 9 09:38:12 2016 +0100 ---------------------------------------------------------------------- .../main/scala/org/apache/spark/SparkContext.scala | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/801e4d09/core/src/main/scala/org/apache/spark/SparkContext.scala ---------------------------------------------------------------------- diff --git a/core/src/main/scala/org/apache/spark/SparkContext.scala b/core/src/main/scala/org/apache/spark/SparkContext.scala index fc7ea51..4f3bb1c 100644 --- a/core/src/main/scala/org/apache/spark/SparkContext.scala +++ b/core/src/main/scala/org/apache/spark/SparkContext.scala @@ -2262,9 +2262,10 @@ object SparkContext extends Logging { SPARK_CONTEXT_CONSTRUCTOR_LOCK.synchronized { if (activeContext.get() == null) { setActiveContext(new SparkContext(config), allowMultipleContexts = false) - } - if (config.getAll.nonEmpty) { - logWarning("Use an existing SparkContext, some configuration may not take effect.") + } else { + if (config.getAll.nonEmpty) { + logWarning("Using an existing SparkContext; some configuration may not take effect.") + } } activeContext.get() } @@ -2281,7 +2282,12 @@ object SparkContext extends Logging { * even if multiple contexts are allowed. */ def getOrCreate(): SparkContext = { - getOrCreate(new SparkConf()) + SPARK_CONTEXT_CONSTRUCTOR_LOCK.synchronized { + if (activeContext.get() == null) { + setActiveContext(new SparkContext(), allowMultipleContexts = false) + } + activeContext.get() + } } /** --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
