[
https://issues.apache.org/jira/browse/SPARK-7553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tathagata Das updated SPARK-7553:
---------------------------------
Description:
In a REPL/notebook environment, its very easy to lose a reference to a
StreamingContext by overriding the variable name. So if you happen to execute
the following commands
{{
val ssc = new StreamingContext(...) // cmd 1
ssc.start() // cmd 2
...
val ssc = new StreamingContext(...) // accidentally run cmd 1 again
}}
The value of ssc will be overwritten. Now you can neither start the new context
(as only one context can be started), nor stop the previous context (as the
reference is lost).
Hence its best to maintain a singleton reference to the active context, so that
we never loose reference for the active context.
Since this is useful in REPL environments, its best to add this as an
Experimental support in the Scala API only.
was:
In a REPL/notebook environment, its very easy to lose a reference to a
StreamingContext by overriding the variable name. So if you happen to execute
the following commands
{{
val ssc = new StreamingContext(...) // cmd 1
ssc.start() // cmd 2
...
val ssc = new StreamingContext(...) // accidentally run cmd 1 again
}}
The value of ssc will be overwritten. Now you can neither start the new context
(as only one context can be started), nor stop the previous context (as the
reference is lost).
Hence its best to maintain a singleton reference to the active context, so that
we never loose reference for the active context.
> Add methods to maintain a singleton StreamingContext
> -----------------------------------------------------
>
> Key: SPARK-7553
> URL: https://issues.apache.org/jira/browse/SPARK-7553
> Project: Spark
> Issue Type: New Feature
> Components: Streaming
> Reporter: Tathagata Das
> Assignee: Tathagata Das
> Priority: Blocker
>
> In a REPL/notebook environment, its very easy to lose a reference to a
> StreamingContext by overriding the variable name. So if you happen to execute
> the following commands
> {{
> val ssc = new StreamingContext(...) // cmd 1
> ssc.start() // cmd 2
> ...
> val ssc = new StreamingContext(...) // accidentally run cmd 1 again
> }}
> The value of ssc will be overwritten. Now you can neither start the new
> context (as only one context can be started), nor stop the previous context
> (as the reference is lost).
> Hence its best to maintain a singleton reference to the active context, so
> that we never loose reference for the active context.
> Since this is useful in REPL environments, its best to add this as an
> Experimental support in the Scala API only.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
