[
https://issues.apache.org/jira/browse/SPARK-17535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15490737#comment-15490737
]
Sean Owen commented on SPARK-17535:
-----------------------------------
I think this happens to work out because the context is held in an
AtomicReference, and it's set as the very last statement in all paths. I am not
sure it's worth changing and making this more complex unless there's a
compelling reason though, like evidence this is heavily contended. For example,
if later something else occurred in the synchronized block after the context
was set, you'd be able to get into strange situations where the reference is
available but the synchronized init hasn't completed.
> Performance Improvement of Signleton pattern in SparkContext
> ------------------------------------------------------------
>
> Key: SPARK-17535
> URL: https://issues.apache.org/jira/browse/SPARK-17535
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 2.0.0
> Reporter: WangJianfei
> Labels: easyfix, performance
>
> I think the singleton pattern of SparkContext is inefficient if there are
> many request to get the SparkContext,
> So we can write the singleton pattern as below,The second way if more
> efficient when there are many request to get the SparkContext.
> {code}
> // the current version
> def getOrCreate(): SparkContext = {
> SPARK_CONTEXT_CONSTRUCTOR_LOCK.synchronized {
> if (activeContext.get() == null) {
> setActiveContext(new SparkContext(), allowMultipleContexts = false)
> }
> activeContext.get()
> }
> }
> // by myself
> def getOrCreate(): SparkContext = {
> if (activeContext.get() == null) {
> SPARK_CONTEXT_CONSTRUCTOR_LOCK.synchronized {
> if(activeContext.get == null) {
> @volatile val sparkContext = new SparkContext()
> setActiveContext(sparkContext, allowMultipleContexts = false)
> }
> }
> activeContext.get()
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
