[GitHub] spark pull request: [SPARK-12001] Allow partially-stopped Streamin...

2015-12-02 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/9982#issuecomment-161442938 Alright, I'm going to merge this into master but _not_ into branch-1.6 out of a desire to minimize risk/churn in that branch during the RC period. --- If your project

[GitHub] spark pull request: [SPARK-12001] Allow partially-stopped Streamin...

2015-12-02 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/9982#issuecomment-161444022 @JoshRosen thanks for doing this! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not h

[GitHub] spark pull request: [SPARK-12001] Allow partially-stopped Streamin...

2015-12-02 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/9982 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-12001] Allow partially-stopped Streamin...

2015-12-02 Thread zsxwing
Github user zsxwing commented on the pull request: https://github.com/apache/spark/pull/9982#issuecomment-161440667 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled a

[GitHub] spark pull request: [SPARK-12001] Allow partially-stopped Streamin...

2015-12-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9982#issuecomment-161147183 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/4

[GitHub] spark pull request: [SPARK-12001] Allow partially-stopped Streamin...

2015-12-01 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9982#issuecomment-161147182 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-12001] Allow partially-stopped Streamin...

2015-12-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9982#issuecomment-161147028 **[Test build #47013 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/47013/consoleFull)** for PR 9982 at commit [`787acc4`](https://git

[GitHub] spark pull request: [SPARK-12001] Allow partially-stopped Streamin...

2015-12-01 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9982#issuecomment-161136075 **[Test build #47013 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/47013/consoleFull)** for PR 9982 at commit [`787acc4`](https://gith

[GitHub] spark pull request: [SPARK-12001] Allow partially-stopped Streamin...

2015-12-01 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/9982#issuecomment-161132945 Actually, here's a report which gives a more complete sense about the frequency of this failing: https://spark-tests.appspot.com/tests/org.apache.spark.streaming.Basic

[GitHub] spark pull request: [SPARK-12001] Allow partially-stopped Streamin...

2015-11-29 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/9982#issuecomment-160513571 By the way, it looks like the test failure which motivated this patch occurs more frequently than I originally thought. It seems to happen about once a day, give or ta

[GitHub] spark pull request: [SPARK-12001] Allow partially-stopped Streamin...

2015-11-28 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/9982#issuecomment-160368342 @tdas, my idea behind adding a new state was to give better error messages in the case where a stop gets interrupted and then the user calls `getOrCreate()` and contin

[GitHub] spark pull request: [SPARK-12001] Allow partially-stopped Streamin...

2015-11-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9982#issuecomment-159803737 **[Test build #46721 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46721/consoleFull)** for PR 9982 at commit [`77df0ce`](https://git

[GitHub] spark pull request: [SPARK-12001] Allow partially-stopped Streamin...

2015-11-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9982#issuecomment-159803869 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-12001] Allow partially-stopped Streamin...

2015-11-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/9982#issuecomment-159803871 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/4

[GitHub] spark pull request: [SPARK-12001] Allow partially-stopped Streamin...

2015-11-25 Thread tdas
Github user tdas commented on the pull request: https://github.com/apache/spark/pull/9982#issuecomment-159783735 rather than introducing another state which needs to be reasoned about, isnt it easier to just make sure that state is set to STOPPED only when the stop has completed. If t

[GitHub] spark pull request: [SPARK-12001] Allow partially-stopped Streamin...

2015-11-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/9982#issuecomment-159766486 **[Test build #46721 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/46721/consoleFull)** for PR 9982 at commit [`77df0ce`](https://gith

[GitHub] spark pull request: [SPARK-12001] Allow partially-stopped Streamin...

2015-11-25 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/9982#issuecomment-159764666 /cc @tdas @zsxwing --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this f

[GitHub] spark pull request: [SPARK-12001] Allow partially-stopped Streamin...

2015-11-25 Thread JoshRosen
GitHub user JoshRosen opened a pull request: https://github.com/apache/spark/pull/9982 [SPARK-12001] Allow partially-stopped StreamingContext to be completely stopped If `StreamingContext.stop()` is interrupted midway through the call, the context will be marked as stopped but cert