[GitHub] spark pull request: SPARK-1407 drain event queue before stopping e...

2014-04-13 Thread kanzhang
Github user kanzhang commented on the pull request: https://github.com/apache/spark/pull/366#issuecomment-40319077 @pwendell @andrewor14 , I think the unit test depends on a race condition that the stopper thread is run before the listener thread does (but we don't know the actual ord

[GitHub] spark pull request: SPARK-1407 drain event queue before stopping e...

2014-04-11 Thread kanzhang
Github user kanzhang commented on the pull request: https://github.com/apache/spark/pull/366#issuecomment-40262882 I opened SPARK-1475 to track this PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: SPARK-1407 drain event queue before stopping e...

2014-04-10 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/366#issuecomment-40121574 Yes, could you open one that describes the listener bus draining part of this PR specifically? Thanks. --- If your project is set up for it, you can reply to this emai

[GitHub] spark pull request: SPARK-1407 drain event queue before stopping e...

2014-04-09 Thread kanzhang
Github user kanzhang commented on the pull request: https://github.com/apache/spark/pull/366#issuecomment-40039003 Or I can open a separate JIRA to track it? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: SPARK-1407 drain event queue before stopping e...

2014-04-09 Thread kanzhang
Github user kanzhang commented on the pull request: https://github.com/apache/spark/pull/366#issuecomment-40031940 Thanks for getting rid of the 100ms wait. I noticed it but didn't have better idea. I think this patch addresses SPARK-1407. Without this patch, even if sc.stop(

[GitHub] spark pull request: SPARK-1407 drain event queue before stopping e...

2014-04-09 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/366 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabl

[GitHub] spark pull request: SPARK-1407 drain event queue before stopping e...

2014-04-09 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/366#issuecomment-40024403 Thanks - I've merged this. I think SPARK-1407 is an incorrect label btw. I also re-wrote the unit test with @andrewor14 on merge to make it a bit simpler. --- If your pr

[GitHub] spark pull request: SPARK-1407 drain event queue before stopping e...

2014-04-09 Thread kanzhang
Github user kanzhang commented on the pull request: https://github.com/apache/spark/pull/366#issuecomment-40019447 thanks for your feedback. Made changes based on reviewer comments and merged test from #251. --- If your project is set up for it, you can reply to this email and have y

[GitHub] spark pull request: SPARK-1407 drain event queue before stopping e...

2014-04-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/366#issuecomment-39939019 All automated tests passed. Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13940/ --- If your project

[GitHub] spark pull request: SPARK-1407 drain event queue before stopping e...

2014-04-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/366#issuecomment-39939018 Merged build finished. All automated tests passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: SPARK-1407 drain event queue before stopping e...

2014-04-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/366#issuecomment-39936421 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1407 drain event queue before stopping e...

2014-04-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/366#issuecomment-39936412 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] spark pull request: SPARK-1407 drain event queue before stopping e...

2014-04-09 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/366#issuecomment-39936400 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have t

[GitHub] spark pull request: SPARK-1407 drain event queue before stopping e...

2014-04-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/366#issuecomment-39935453 Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/13935/ --- If your project is set up for it, you can r

[GitHub] spark pull request: SPARK-1407 drain event queue before stopping e...

2014-04-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/366#issuecomment-39935451 Merged build finished. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1407 drain event queue before stopping e...

2014-04-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/366#issuecomment-39931899 Merged build started. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: SPARK-1407 drain event queue before stopping e...

2014-04-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/366#issuecomment-39931893 Merged build triggered. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] spark pull request: SPARK-1407 drain event queue before stopping e...

2014-04-08 Thread concretevitamin
Github user concretevitamin commented on the pull request: https://github.com/apache/spark/pull/366#issuecomment-39931839 This does look much simpler. Also, free free to use the test there ;) --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: SPARK-1407 drain event queue before stopping e...

2014-04-08 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/366#issuecomment-39931703 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have t

[GitHub] spark pull request: SPARK-1407 drain event queue before stopping e...

2014-04-08 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/366#issuecomment-39931680 Hi @kanzhang, we have a long-running discussion about this at #251. Joining on the thread is a simple alternative to what is proposed in the other PR, but at the same t

[GitHub] spark pull request: SPARK-1407 drain event queue before stopping e...

2014-04-08 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/366#discussion_r11423726 --- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala --- @@ -884,6 +883,7 @@ class SparkContext( cleaner.foreach(_.stop())

[GitHub] spark pull request: SPARK-1407 drain event queue before stopping e...

2014-04-08 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/366#discussion_r11423705 --- Diff: examples/src/main/scala/org/apache/spark/examples/SparkHdfsLR.scala --- @@ -73,6 +73,7 @@ object SparkHdfsLR { } println("

[GitHub] spark pull request: SPARK-1407 drain event queue before stopping e...

2014-04-08 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/366#discussion_r11423681 --- Diff: core/src/main/scala/org/apache/spark/scheduler/LiveListenerBus.scala --- @@ -36,6 +36,7 @@ private[spark] class LiveListenerBus extends SparkList

[GitHub] spark pull request: SPARK-1407 drain event queue before stopping e...

2014-04-08 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/366#discussion_r11423695 --- Diff: examples/src/main/scala/org/apache/spark/examples/SparkHdfsLR.scala --- @@ -73,6 +73,7 @@ object SparkHdfsLR { } println("

[GitHub] spark pull request: SPARK-1407 drain event queue before stopping e...

2014-04-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/366#issuecomment-39928894 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your proj

[GitHub] spark pull request: SPARK-1407 drain event queue before stopping e...

2014-04-08 Thread kanzhang
GitHub user kanzhang opened a pull request: https://github.com/apache/spark/pull/366 SPARK-1407 drain event queue before stopping event logger You can merge this pull request into a Git repository by running: $ git pull https://github.com/kanzhang/spark SPARK-1407 Alternative