[GitHub] spark pull request: [SPARK-12265][Mesos] Spark calls System.exit i...

2016-01-27 Thread nraychaudhuri
Github user nraychaudhuri commented on the pull request: https://github.com/apache/spark/pull/10921#issuecomment-175780122 Thanks for taking this up @dragos --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12265][Mesos] Spark calls System.exit i...

2016-01-27 Thread nraychaudhuri
Github user nraychaudhuri closed the pull request at: https://github.com/apache/spark/pull/10729 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark pull request: [SPARK-12265][Mesos] Spark calls System.exit i...

2016-01-25 Thread nraychaudhuri
Github user nraychaudhuri commented on the pull request: https://github.com/apache/spark/pull/10729#issuecomment-174522212 @dragos Thanks for looking into it. I will not have time to work on this. Please take it up --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: [SPARK-12265][Mesos] Spark calls System.exit i...

2016-01-14 Thread nraychaudhuri
Github user nraychaudhuri commented on a diff in the pull request: https://github.com/apache/spark/pull/10729#discussion_r49761842 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerUtils.scala --- @@ -109,20 +109,17 @@ private[mesos] trait

[GitHub] spark pull request: [SPARK-7831][Mesos] Added flag to shutdown dri...

2016-01-14 Thread nraychaudhuri
Github user nraychaudhuri commented on the pull request: https://github.com/apache/spark/pull/10701#issuecomment-171725744 Yes. I have tested this and it seems to work. I will make the necessary changes --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-7831][Mesos] Added flag to shutdown dri...

2016-01-14 Thread nraychaudhuri
Github user nraychaudhuri commented on a diff in the pull request: https://github.com/apache/spark/pull/10701#discussion_r49776110 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosClusterScheduler.scala --- @@ -318,7 +319,7 @@ private[spark] class

[GitHub] spark pull request: [SPARK-12265][Mesos] Spark calls System.exit i...

2016-01-14 Thread nraychaudhuri
Github user nraychaudhuri commented on a diff in the pull request: https://github.com/apache/spark/pull/10729#discussion_r49737434 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerUtils.scala --- @@ -109,20 +109,17 @@ private[mesos] trait

[GitHub] spark pull request: [SPARK-12265][Mesos] Spark calls System.exit i...

2016-01-14 Thread nraychaudhuri
Github user nraychaudhuri commented on a diff in the pull request: https://github.com/apache/spark/pull/10729#discussion_r49748427 --- Diff: core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerUtils.scala --- @@ -109,20 +109,17 @@ private[mesos] trait

[GitHub] spark pull request: [SPARK-12265][Mesos] Spark calls System.exit i...

2016-01-13 Thread nraychaudhuri
Github user nraychaudhuri commented on the pull request: https://github.com/apache/spark/pull/10729#issuecomment-171280201 @dragos do you see it hanging? I introduced markErr method to take care of the countdown latch. This is called from the error handler. --- If your project

[GitHub] spark pull request: [SPARK-12265][Mesos] Spark calls System.exit i...

2016-01-13 Thread nraychaudhuri
Github user nraychaudhuri commented on the pull request: https://github.com/apache/spark/pull/10729#issuecomment-171437903 @dragos @srowen Took care of the latch countdown. The process should not hang anymore. --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: [SPARK-7831][Mesos] Added flag to shutdown dri...

2016-01-13 Thread nraychaudhuri
Github user nraychaudhuri commented on the pull request: https://github.com/apache/spark/pull/10701#issuecomment-171302887 @dragos @tnachen @skyluc Could you please take a look at this one? --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [SPARK-12265][Mesos] Spark calls System.exit i...

2016-01-13 Thread nraychaudhuri
Github user nraychaudhuri commented on the pull request: https://github.com/apache/spark/pull/10729#issuecomment-171302545 @dragos good catch. I guess I only tested in coarse grain mode. I will push a fix. Thx --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: [SPARK-12265][Mesos] Remove System.exit

2016-01-12 Thread nraychaudhuri
GitHub user nraychaudhuri opened a pull request: https://github.com/apache/spark/pull/10729 [SPARK-12265][Mesos] Remove System.exit You can merge this pull request into a Git repository by running: $ git pull https://github.com/nraychaudhuri/spark SPARK-12265 Alternatively

[GitHub] spark pull request: [SPARK-12265][Mesos] Spark calls System.exit i...

2016-01-12 Thread nraychaudhuri
Github user nraychaudhuri commented on the pull request: https://github.com/apache/spark/pull/10729#issuecomment-17400 Updated the commit message & description --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-7831][Mesos] Added flag to shutdown dri...

2016-01-11 Thread nraychaudhuri
GitHub user nraychaudhuri opened a pull request: https://github.com/apache/spark/pull/10701 [SPARK-7831][Mesos] Added flag to shutdown driver when mesos dispatch… Fix for SPARK-7831 You can merge this pull request into a Git repository by running: $ git pull https

[GitHub] spark pull request: [SPARK-12413] Fix Mesos ZK persistence

2015-12-17 Thread nraychaudhuri
Github user nraychaudhuri commented on the pull request: https://github.com/apache/spark/pull/10366#issuecomment-165659685 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-7874][MESOS] Don’t allocate more than...

2015-10-21 Thread nraychaudhuri
Github user nraychaudhuri commented on the pull request: https://github.com/apache/spark/pull/9027#issuecomment-149896275 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: testing

2015-08-26 Thread nraychaudhuri
Github user nraychaudhuri closed the pull request at: https://github.com/apache/spark/pull/8463 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request: testing

2015-08-26 Thread nraychaudhuri
GitHub user nraychaudhuri opened a pull request: https://github.com/apache/spark/pull/8463 testing You can merge this pull request into a Git repository by running: $ git pull https://github.com/typesafehub/spark ghpr_validator_branch Alternatively you can review and apply

[GitHub] spark pull request: testing

2015-08-26 Thread nraychaudhuri
Github user nraychaudhuri commented on the pull request: https://github.com/apache/spark/pull/8463#issuecomment-135069699 ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-8978][Streaming] Implements the DirectK...

2015-08-05 Thread nraychaudhuri
Github user nraychaudhuri commented on a diff in the pull request: https://github.com/apache/spark/pull/7796#discussion_r36272684 --- Diff: external/kafka/src/test/scala/org/apache/spark/streaming/kafka/DirectKafkaStreamSuite.scala --- @@ -381,3 +447,20 @@ object

[GitHub] spark pull request: [SPARK-8978][Streaming] Implements the DirectK...

2015-08-05 Thread nraychaudhuri
Github user nraychaudhuri commented on a diff in the pull request: https://github.com/apache/spark/pull/7796#discussion_r36272732 --- Diff: external/kafka/src/test/scala/org/apache/spark/streaming/kafka/DirectKafkaStreamSuite.scala --- @@ -381,3 +447,20 @@ object

[GitHub] spark pull request: [SPARK-8978][Streaming] Implements the DirectK...

2015-08-05 Thread nraychaudhuri
Github user nraychaudhuri commented on the pull request: https://github.com/apache/spark/pull/7796#issuecomment-127919507 Fixed the doc. Thanks @dragos for fixing the title --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] spark pull request: Update sbt-launch-lib.bash

2015-08-05 Thread nraychaudhuri
Github user nraychaudhuri closed the pull request at: https://github.com/apache/spark/pull/7792 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature

[GitHub] spark pull request: Update sbt-launch-lib.bash

2015-08-05 Thread nraychaudhuri
Github user nraychaudhuri commented on the pull request: https://github.com/apache/spark/pull/7792#issuecomment-128012923 @srowen Sorry got busy with things. Ok to close this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request: [SPARK-8978][Streaming] Implements the DirectK...

2015-08-05 Thread nraychaudhuri
Github user nraychaudhuri commented on the pull request: https://github.com/apache/spark/pull/7796#issuecomment-128156363 @tdas Took care of the remaining nits. Hope this makes the code clear /cc @deanwampler --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: [SPARK-8978][Streaming] Implements the DirectK...

2015-08-05 Thread nraychaudhuri
Github user nraychaudhuri commented on the pull request: https://github.com/apache/spark/pull/7796#issuecomment-128157075 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-8978][Streaming] Implements the DirectK...

2015-08-05 Thread nraychaudhuri
Github user nraychaudhuri commented on the pull request: https://github.com/apache/spark/pull/7796#issuecomment-128250793 Not sure why pySpark tests passed and failed for the same commit. Trying to build again --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: [SPARK-8978][Streaming] Implements the DirectK...

2015-08-05 Thread nraychaudhuri
Github user nraychaudhuri commented on the pull request: https://github.com/apache/spark/pull/7796#issuecomment-128250803 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-8978][Streaming] Implements the DirectK...

2015-08-04 Thread nraychaudhuri
Github user nraychaudhuri commented on the pull request: https://github.com/apache/spark/pull/7796#issuecomment-127560630 @tdas Made the necessary change. Should I squash all the commits? --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [SPARK-8978][Streaming] Implements the DirectK...

2015-08-04 Thread nraychaudhuri
Github user nraychaudhuri commented on the pull request: https://github.com/apache/spark/pull/7796#issuecomment-127561465 @tdas I will add the test next. It might take a while as I am new this codebase --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-8978][Streaming] Implements the DirectK...

2015-08-04 Thread nraychaudhuri
Github user nraychaudhuri commented on the pull request: https://github.com/apache/spark/pull/7796#issuecomment-127531667 @tdas found the issue. My branch was not updated. I will fix the issues you pointed out. I am not sure what do you mean by update the title? I will try

[GitHub] spark pull request: [SPARK-8978][Streaming] Implements the DirectK...

2015-08-04 Thread nraychaudhuri
Github user nraychaudhuri commented on the pull request: https://github.com/apache/spark/pull/7796#issuecomment-127511409 @tdas Added the missing rate controller.I hope this makes this PR complete now. --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-8978][Streaming] Implements the DirectK...

2015-08-04 Thread nraychaudhuri
Github user nraychaudhuri commented on the pull request: https://github.com/apache/spark/pull/7796#issuecomment-127522870 @tdas Just noticed. Didn't get that in local build. Investigating --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark pull request: [SPARK-8978][Streaming] Implements the DirectK...

2015-08-04 Thread nraychaudhuri
Github user nraychaudhuri commented on the pull request: https://github.com/apache/spark/pull/7796#issuecomment-127656907 @tdas Tried my best to come up with a testcase for the DirectKafkaInputDStream. Please take a look. I hope this PR is good for merge now. --- If your project

[GitHub] spark pull request: [SPARK-8979] Add a PID based rate estimator

2015-07-31 Thread nraychaudhuri
Github user nraychaudhuri commented on the pull request: https://github.com/apache/spark/pull/7648#issuecomment-126670962 @tdas I will try to take care of this by early next week. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request: Update sbt-launch-lib.bash

2015-07-30 Thread nraychaudhuri
Github user nraychaudhuri commented on the pull request: https://github.com/apache/spark/pull/7792#issuecomment-126569434 @srowen Sounds good. I will use the guidelines. Thanks --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request: Update sbt-launch-lib.bash

2015-07-30 Thread nraychaudhuri
GitHub user nraychaudhuri opened a pull request: https://github.com/apache/spark/pull/7792 Update sbt-launch-lib.bash typesafe.artifactoryonline.com is no longer available (it was retired recently) so switched to the new repo location You can merge this pull request into a Git