[
https://issues.apache.org/jira/browse/SPARK-10649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-10649:
------------------------------------
Assignee: Tathagata Das (was: Apache Spark)
> Streaming jobs unexpectedly inherits job group, job descriptions from context
> starting thread
> ---------------------------------------------------------------------------------------------
>
> Key: SPARK-10649
> URL: https://issues.apache.org/jira/browse/SPARK-10649
> Project: Spark
> Issue Type: Bug
> Components: Streaming
> Affects Versions: 1.3.1, 1.4.1, 1.5.0
> Reporter: Tathagata Das
> Assignee: Tathagata Das
>
> The job group, job descriptions and scheduler pool information is passed
> through thread local properties, and get inherited by child threads. In case
> of spark streaming, the streaming jobs inherit these properties from the
> thread that called streamingContext.start(). This may not make sense.
> 1. Job group: This is mainly used for cancelling a group of jobs together. It
> does not make sense to cancel streaming jobs like this, as the effect will be
> unpredictable. And its not a valid usecase any way, to cancel a streaming
> context, call streamingContext.stop()
> 2. Job description: This is used to pass on nice text descriptions for jobs
> to show up in the UI. The job description of the thread that calls
> streamingContext.start() is not useful for all the streaming jobs, as it does
> not make sense for all of the streaming jobs to have the same description,
> and the description may or may not be related to streaming.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
