[jira] [Commented] (SPARK-4276) Spark streaming requires at least two working thread
[
https://issues.apache.org/jira/browse/SPARK-4276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224566#comment-14224566
]
Tathagata Das commented on SPARK-4276:
--
Since this is not an issue, I am closing this JIRA.
> Spark streaming requires at least two working thread
>
>
> Key: SPARK-4276
> URL: https://issues.apache.org/jira/browse/SPARK-4276
> Project: Spark
> Issue Type: Bug
> Components: Streaming
>Affects Versions: 1.1.0
>Reporter: varun sharma
> Fix For: 1.1.0
>
>
> Spark streaming requires at least two working threads.But example in
> spark/examples/src/main/scala/org/apache/spark/examples/streaming/NetworkWordCount.scala
>
> has
> // Create the context with a 1 second batch size
> val sparkConf = new SparkConf().setAppName("NetworkWordCount")
> val ssc = new StreamingContext(sparkConf, Seconds(1))
> which creates only 1 thread.
> It should have atleast 2 threads:
> http://spark.apache.org/docs/latest/streaming-programming-guide.html
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-4276) Spark streaming requires at least two working thread
[
https://issues.apache.org/jira/browse/SPARK-4276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14224565#comment-14224565
]
Tathagata Das commented on SPARK-4276:
--
This issue is resolved by this JIRA, the user will get warned clearly if local
or local[1] mode is used.
> Spark streaming requires at least two working thread
>
>
> Key: SPARK-4276
> URL: https://issues.apache.org/jira/browse/SPARK-4276
> Project: Spark
> Issue Type: Bug
> Components: Streaming
>Affects Versions: 1.1.0
>Reporter: varun sharma
> Fix For: 1.1.0
>
>
> Spark streaming requires at least two working threads.But example in
> spark/examples/src/main/scala/org/apache/spark/examples/streaming/NetworkWordCount.scala
>
> has
> // Create the context with a 1 second batch size
> val sparkConf = new SparkConf().setAppName("NetworkWordCount")
> val ssc = new StreamingContext(sparkConf, Seconds(1))
> which creates only 1 thread.
> It should have atleast 2 threads:
> http://spark.apache.org/docs/latest/streaming-programming-guide.html
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-4276) Spark streaming requires at least two working thread
[
https://issues.apache.org/jira/browse/SPARK-4276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200571#comment-14200571
]
varun sharma commented on SPARK-4276:
-
[~srowen] Yeah right. I spent lot of time to figure this out as example in repo
and in documentation are different.
I now understood the concept of resource starvation here.
Thanks for pointing it out and sorry for the inconvenience.
> Spark streaming requires at least two working thread
>
>
> Key: SPARK-4276
> URL: https://issues.apache.org/jira/browse/SPARK-4276
> Project: Spark
> Issue Type: Bug
> Components: Streaming
>Affects Versions: 1.1.0
>Reporter: varun sharma
> Fix For: 1.1.0
>
>
> Spark streaming requires at least two working threads.But example in
> spark/examples/src/main/scala/org/apache/spark/examples/streaming/NetworkWordCount.scala
>
> has
> // Create the context with a 1 second batch size
> val sparkConf = new SparkConf().setAppName("NetworkWordCount")
> val ssc = new StreamingContext(sparkConf, Seconds(1))
> which creates only 1 thread.
> It should have atleast 2 threads:
> http://spark.apache.org/docs/latest/streaming-programming-guide.html
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-4276) Spark streaming requires at least two working thread
[
https://issues.apache.org/jira/browse/SPARK-4276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200540#comment-14200540
]
Apache Spark commented on SPARK-4276:
-
User 'svar29' has created a pull request for this issue:
https://github.com/apache/spark/pull/3141
> Spark streaming requires at least two working thread
>
>
> Key: SPARK-4276
> URL: https://issues.apache.org/jira/browse/SPARK-4276
> Project: Spark
> Issue Type: Bug
> Components: Streaming
>Affects Versions: 1.1.0
>Reporter: varun sharma
> Fix For: 1.1.0
>
>
> Spark streaming requires at least two working threads.But example in
> spark/examples/src/main/scala/org/apache/spark/examples/streaming/NetworkWordCount.scala
>
> has
> // Create the context with a 1 second batch size
> val sparkConf = new SparkConf().setAppName("NetworkWordCount")
> val ssc = new StreamingContext(sparkConf, Seconds(1))
> which creates only 1 thread.
> It should have atleast 2 threads:
> http://spark.apache.org/docs/latest/streaming-programming-guide.html
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-4276) Spark streaming requires at least two working thread
[
https://issues.apache.org/jira/browse/SPARK-4276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14200539#comment-14200539
]
Sean Owen commented on SPARK-4276:
--
This is basically the same concern addressed already by
https://issues.apache.org/jira/browse/SPARK-4040 no?
This code can set a master of, say, "local[2]", but it was my understanding
that all examples don't set a master and this is supplied by spark-submit.
Then again SPARK-4040 changed the doc example to set a local[2] master. hm.
> Spark streaming requires at least two working thread
>
>
> Key: SPARK-4276
> URL: https://issues.apache.org/jira/browse/SPARK-4276
> Project: Spark
> Issue Type: Bug
> Components: Streaming
>Affects Versions: 1.1.0
>Reporter: varun sharma
> Fix For: 1.1.0
>
>
> Spark streaming requires at least two working threads.But example in
> spark/examples/src/main/scala/org/apache/spark/examples/streaming/NetworkWordCount.scala
>
> has
> // Create the context with a 1 second batch size
> val sparkConf = new SparkConf().setAppName("NetworkWordCount")
> val ssc = new StreamingContext(sparkConf, Seconds(1))
> which creates only 1 thread.
> It should have atleast 2 threads:
> http://spark.apache.org/docs/latest/streaming-programming-guide.html
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
