[
https://issues.apache.org/jira/browse/SPARK-7478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tathagata Das updated SPARK-7478:
---------------------------------
Description:
Having a SQLContext singleton would make it easier for applications to use a
lazily instantiated single shared instance of SQLContext when needed. It would
avoid problems like
1. In REPL/notebook environment, rerunning the line "val sqlContext = new
SQLContext" multiple times created different contexts while overriding the
reference to previous context, leading to issues like registered temp tables
going missing.
2. In Streaming, creating SQLContext directly leads to
serialization/deserialization issues when attempting to recover from DStream
checkpoints. See [SPARK-6770](https://issues.apache.org/jira/browse/SPARK-6770)
was:
Having a SQLContext singleton would make it easier for applications to use a
lazily instantiated single shared instance of SQLContext when needed. It would
avoid problems like
1. In REPL/notebook environment, rerunning the line "val sqlContext = new
SQLContext" multiple times created different contexts while overriding the
reference to previous context, leading to issues like registered temp tables
going missing.
2. In Streaming, creating SQLContext directly leads to
serialization/deserialization issues when attempting to recover from DStream
checkpoints. See
> Add a SQLContext.getOrCreate to maintain a singleton instance of SQLContext
> ---------------------------------------------------------------------------
>
> Key: SPARK-7478
> URL: https://issues.apache.org/jira/browse/SPARK-7478
> Project: Spark
> Issue Type: New Feature
> Components: SQL
> Reporter: Tathagata Das
> Assignee: Tathagata Das
> Priority: Blocker
>
> Having a SQLContext singleton would make it easier for applications to use a
> lazily instantiated single shared instance of SQLContext when needed. It
> would avoid problems like
> 1. In REPL/notebook environment, rerunning the line "val sqlContext = new
> SQLContext" multiple times created different contexts while overriding the
> reference to previous context, leading to issues like registered temp tables
> going missing.
> 2. In Streaming, creating SQLContext directly leads to
> serialization/deserialization issues when attempting to recover from DStream
> checkpoints. See
> [SPARK-6770](https://issues.apache.org/jira/browse/SPARK-6770)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
