[jira] [Comment Edited] (SPARK-7478) Add a SQLContext.getOrCreate to maintain a singleton instance of SQLContext
[
https://issues.apache.org/jira/browse/SPARK-7478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14534094#comment-14534094
]
Tathagata Das edited comment on SPARK-7478 at 5/8/15 8:24 AM:
--
[~rxin] [~marmbrus] Thoughts?
was (Author: tdas):
[~rxin][~marmbrus] Thoughts?
Add a SQLContext.getOrCreate to maintain a singleton instance of SQLContext
---
Key: SPARK-7478
URL: https://issues.apache.org/jira/browse/SPARK-7478
Project: Spark
Issue Type: New Feature
Components: SQL
Reporter: Tathagata Das
Assignee: Tathagata Das
Priority: Blocker
Having a SQLContext singleton would make it easier for applications to use a
lazily instantiated single shared instance of SQLContext when needed. It
would avoid problems like
1. In REPL/notebook environment, rerunning the line {{val sqlContext = new
SQLContext}} multiple times created different contexts while overriding the
reference to previous context, leading to issues like registered temp tables
going missing.
2. In Streaming, creating SQLContext directly leads to
serialization/deserialization issues when attempting to recover from DStream
checkpoints. See [SPARK-6770].
This can be solved by {{SQLContext.getOrCreate}} which get or creates a new
singleton instance of SQLContext using either a given SparkContext or a given
SparkConf
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Comment Edited] (SPARK-7478) Add a SQLContext.getOrCreate to maintain a singleton instance of SQLContext
[
https://issues.apache.org/jira/browse/SPARK-7478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14534094#comment-14534094
]
Tathagata Das edited comment on SPARK-7478 at 5/8/15 8:24 AM:
--
[~rxin][~marmbrus] Thoughts?
was (Author: tdas):
[~rxin] Thoughts?
Add a SQLContext.getOrCreate to maintain a singleton instance of SQLContext
---
Key: SPARK-7478
URL: https://issues.apache.org/jira/browse/SPARK-7478
Project: Spark
Issue Type: New Feature
Components: SQL
Reporter: Tathagata Das
Assignee: Tathagata Das
Priority: Blocker
Having a SQLContext singleton would make it easier for applications to use a
lazily instantiated single shared instance of SQLContext when needed. It
would avoid problems like
1. In REPL/notebook environment, rerunning the line {{val sqlContext = new
SQLContext}} multiple times created different contexts while overriding the
reference to previous context, leading to issues like registered temp tables
going missing.
2. In Streaming, creating SQLContext directly leads to
serialization/deserialization issues when attempting to recover from DStream
checkpoints. See [SPARK-6770].
This can be solved by {{SQLContext.getOrCreate}} which get or creates a new
singleton instance of SQLContext using either a given SparkContext or a given
SparkConf
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
