[GitHub] spark pull request: [SPARK-2996] Implement userClassPathFirst for ...

2014-12-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3233#issuecomment-66360529 [Test build #24262 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24262/consoleFull) for PR 3233 at commit [`44010b6`](https://githu

[GitHub] spark pull request: [SPARK-4765] Make GC time always shown in UI.

2014-12-09 Thread ksakellis
Github user ksakellis commented on the pull request: https://github.com/apache/spark/pull/3622#issuecomment-66360558 @kayousterhout I ran into this issue with a private case class too on: https://github.com/apache/spark/pull/3486 and added exclusions. I can try adding private[spark] a

[GitHub] spark pull request: [SPARK-4765] Make GC time always shown in UI.

2014-12-09 Thread kayousterhout
Github user kayousterhout commented on the pull request: https://github.com/apache/spark/pull/3622#issuecomment-66360193 Ok well suspiciously making the class private[spark] now makes all of the MiMA tests pass... --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request: [WIP] SPARK-2450 Adds exeuctor log links to We...

2014-12-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/3486#discussion_r21563407 --- Diff: core/src/main/scala/org/apache/spark/ui/exec/ExecutorsPage.scala --- @@ -138,6 +140,13 @@ private[ui] class ExecutorsPage(

[GitHub] spark pull request: [WIP] SPARK-2450 Adds exeuctor log links to We...

2014-12-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/3486#discussion_r21563357 --- Diff: core/src/main/scala/org/apache/spark/ui/exec/ExecutorsPage.scala --- @@ -79,6 +80,7 @@ private[ui] class ExecutorsPage( Shuffl

[GitHub] spark pull request: [WIP] SPARK-2450 Adds exeuctor log links to We...

2014-12-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/3486#discussion_r21563321 --- Diff: core/src/main/scala/org/apache/spark/scheduler/SparkListener.scala --- @@ -84,6 +86,14 @@ case class SparkListenerBlockManagerRemoved(time: Long,

[GitHub] spark pull request: [SPARK-4006] Block Manager - Double Register C...

2014-12-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2854#issuecomment-66359732 [Test build #24261 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24261/consoleFull) for PR 2854 at commit [`95ae4db`](https://githu

[GitHub] spark pull request: [WIP] SPARK-2450 Adds exeuctor log links to We...

2014-12-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/3486#discussion_r21563053 --- Diff: core/src/main/scala/org/apache/spark/scheduler/SparkListener.scala --- @@ -84,6 +86,14 @@ case class SparkListenerBlockManagerRemoved(time: Long,

[GitHub] spark pull request: [SPARK-4006] In long running contexts, we enco...

2014-12-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2914#issuecomment-66359393 [Test build #24260 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24260/consoleFull) for PR 2914 at commit [`1014493`](https://githu

[GitHub] spark pull request: [SPARK-4006] In long running contexts, we enco...

2014-12-09 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/2914#issuecomment-66359278 I think this might be an issue with the Jenkins pull request builder and pull requests that are opened against non-master backport branches. Once this latest test run

[GitHub] spark pull request: [SPARK-4803] Remove duplicate RegisterReceiver...

2014-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3648#issuecomment-66359249 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[GitHub] spark pull request: [WIP] SPARK-2450 Adds exeuctor log links to We...

2014-12-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/3486#discussion_r21562933 --- Diff: core/src/main/scala/org/apache/spark/executor/CoarseGrainedExecutorBackend.scala --- @@ -50,10 +50,16 @@ private[spark] class CoarseGrainedExecu

[GitHub] spark pull request: [SPARK-4006] In long running contexts, we enco...

2014-12-09 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/2914#issuecomment-66359052 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this f

[GitHub] spark pull request: [SPARK-4205][SQL] Timestamp and Date with comp...

2014-12-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3158#issuecomment-66358965 [Test build #24259 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24259/consoleFull) for PR 3158 at commit [`4cfb864`](https://githu

[GitHub] spark pull request: [SPARK-4006] Block Manager - Double Register C...

2014-12-09 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/2854#issuecomment-66359025 Yeah the PR seems fine. But let's retest this please so we can look at the Jenkins logs, which have now been deleted. --- If your project is set up for it, you can re

[GitHub] spark pull request: [SPARK-4803] Remove duplicate RegisterReceiver...

2014-12-09 Thread ilayaperumalg
GitHub user ilayaperumalg opened a pull request: https://github.com/apache/spark/pull/3648 [SPARK-4803] Remove duplicate RegisterReceiver message - The ReceiverTracker receivers `RegisterReceiver` messages two times 1) When the actor at `ReceiverSupervisorImpl`'s preStart

[GitHub] spark pull request: [SPARK-4771][Docs] Document standalone cluster...

2014-12-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/3627#discussion_r21562819 --- Diff: docs/spark-standalone.md --- @@ -272,6 +272,15 @@ should specify them through the `--jars` flag using comma as a delimiter (e.g. ` To contro

[GitHub] spark pull request: [SPARK-4691][shuffle] Restructure a few lines ...

2014-12-09 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/3553 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-4691][shuffle] Restructure a few lines ...

2014-12-09 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/3553#issuecomment-66358597 @aarondav I wanted to back port this into branch-1.2 as well. It would be good to minimize the divergence between master and 1.2 if possible. I'm merging this

[GitHub] spark pull request: [SPARK-4771][Docs] Document standalone cluster...

2014-12-09 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/3627#discussion_r21562628 --- Diff: docs/spark-standalone.md --- @@ -272,6 +272,15 @@ should specify them through the `--jars` flag using comma as a delimiter (e.g. ` To control the

[GitHub] spark pull request: [SPARK-4765] Make GC time always shown in UI.

2014-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3622#issuecomment-66358495 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24

[GitHub] spark pull request: [SPARK-4765] Make GC time always shown in UI.

2014-12-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3622#issuecomment-66358486 [Test build #24255 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24255/consoleFull) for PR 3622 at commit [`15ac242`](https://gith

[GitHub] spark pull request: SPARK-3779. yarn spark.yarn.applicationMaster....

2014-12-09 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/3471#issuecomment-66358381 LGTM, but do we need to add backward compatibility? This patch seems to get rid of the old config altogether and Spark will silently ignore the setting if an old appli

[GitHub] spark pull request: Add example that reads a local file, writes to...

2014-12-09 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/3347#issuecomment-66358073 I think the latest changes look fine. @pwendell any thoughts? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request: [SPARK-4737] Task set manager properly handles...

2014-12-09 Thread mccheah
Github user mccheah commented on the pull request: https://github.com/apache/spark/pull/3638#issuecomment-66357595 Anyone have any comment on this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-4772] Clear local copies of accumulator...

2014-12-09 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/3570#issuecomment-66357628 LGTM, nice catch. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this fe

[GitHub] spark pull request: [SPARK-4789] [mllib] Standardize ML Prediction...

2014-12-09 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/3637#discussion_r21562163 --- Diff: mllib/src/main/scala/org/apache/spark/ml/LabeledPoint.scala --- @@ -0,0 +1,52 @@ +/* + * Licensed to the Apache Software Foundation (ASF) unde

[GitHub] spark pull request: [WIP] SPARK-2450 Adds exeuctor log links to We...

2014-12-09 Thread ksakellis
Github user ksakellis commented on the pull request: https://github.com/apache/spark/pull/3486#issuecomment-66356776 @andrewor14 Yes you can start doing a review. I wanted to post this just to get sign off on the strategy i'm using. If people feel good about this, i'll post more unit

[GitHub] spark pull request: [SPARK-4792] Add error message when making loc...

2014-12-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3635#issuecomment-66356636 [Test build #24254 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24254/consoleFull) for PR 3635 at commit [`2a55bc2`](https://gith

[GitHub] spark pull request: [SPARK-4792] Add error message when making loc...

2014-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3635#issuecomment-66356650 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24

[GitHub] spark pull request: [SPARK-4771][Docs] Document standalone cluster...

2014-12-09 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/3627#discussion_r21561337 --- Diff: docs/spark-standalone.md --- @@ -272,6 +272,15 @@ should specify them through the `--jars` flag using comma as a delimiter (e.g. ` To control the

[GitHub] spark pull request: [SPARK-4791] [sql] Infer schema from case clas...

2014-12-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3646#issuecomment-66355649 [Test build #543 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/543/consoleFull) for PR 3646 at commit [`796b2e4`](https://githu

[GitHub] spark pull request: [SPARK-4771][Docs] Document standalone cluster...

2014-12-09 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/3627#discussion_r21561116 --- Diff: docs/spark-standalone.md --- @@ -272,6 +272,15 @@ should specify them through the `--jars` flag using comma as a delimiter (e.g. ` To control the

[GitHub] spark pull request: [SPARK-2261] Make event logger use a single fi...

2014-12-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1222#issuecomment-66355094 [Test build #24258 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24258/consoleFull) for PR 1222 at commit [`ed0023e`](https://githu

[GitHub] spark pull request: [SPARK-4789] [mllib] Standardize ML Prediction...

2014-12-09 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/3637#discussion_r21559541 --- Diff: mllib/src/main/scala/org/apache/spark/ml/LabeledPoint.scala --- @@ -0,0 +1,52 @@ +/* + * Licensed to the Apache Software Foundation (ASF) u

[GitHub] spark pull request: Improved GraphX PageRank Test Coverage

2014-12-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/1228#issuecomment-66351203 [Test build #24257 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24257/consoleFull) for PR 1228 at commit [`8498483`](https://githu

[GitHub] spark pull request: [SPARK-874] adding a --wait flag

2014-12-09 Thread jbencook
Github user jbencook commented on the pull request: https://github.com/apache/spark/pull/3567#issuecomment-66350319 Awesome! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabl

[GitHub] spark pull request: [SPARK-4802] Remove receiverInfo once receiver...

2014-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3647#issuecomment-66349914 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[GitHub] spark pull request: [SPARK-874] adding a --wait flag

2014-12-09 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/3567#issuecomment-66349965 Thanks Ben - I merged this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-874] adding a --wait flag

2014-12-09 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/3567 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-4802] Remove receiverInfo once receiver...

2014-12-09 Thread ilayaperumalg
GitHub user ilayaperumalg opened a pull request: https://github.com/apache/spark/pull/3647 [SPARK-4802] Remove receiverInfo once receiver is de-registered Once the streaming receiver is de-registered at executor, the `ReceiverTrackerActor` needs to remove the corresponding rev

[GitHub] spark pull request: [SPARK-4736][mllib] [random forest] functions ...

2014-12-09 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/3583#issuecomment-66348700 @dikejiang Thanks for the PR! I'm wondering if you'd be interested in a more general API. In the new experimental ML package, I have a PR [https://www.github.com/apac

[GitHub] spark pull request: [SPARK-874] adding a --wait flag

2014-12-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3567#issuecomment-66348310 [Test build #24252 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24252/consoleFull) for PR 3567 at commit [`d05c5bb`](https://gith

[GitHub] spark pull request: [SPARK-874] adding a --wait flag

2014-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3567#issuecomment-66348321 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24

[GitHub] spark pull request: [SPARK-4789] [mllib] Standardize ML Prediction...

2014-12-09 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/3637#issuecomment-66346654 Question: Do people have preferences for the name of what is currently "predictRaw?" Possibilities are: ``` predictRaw() predictConfidence() confidences

[GitHub] spark pull request: [SPARK-3967] don’t redundantly overwrite exe...

2014-12-09 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/2848#discussion_r21557323 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -412,6 +408,85 @@ private[spark] object Utils extends Logging { }

[GitHub] spark pull request: [WIP] SPARK-2450 Adds exeuctor log links to We...

2014-12-09 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/3486#issuecomment-66346418 Hey @ksakellis is this still WIP? Should I start doing a detailed review yet? --- If your project is set up for it, you can reply to this email and have your reply ap

[GitHub] spark pull request: [SPARK-4772] Clear local copies of accumulator...

2014-12-09 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3570#issuecomment-66346390 This looks good to me. If you don't mind, could you update the pull request description to more accurately describe the change that we're actually committing? This is

[GitHub] spark pull request: [SPARK-3967] don’t redundantly overwrite exe...

2014-12-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/2848#discussion_r21557057 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -412,6 +408,85 @@ private[spark] object Utils extends Logging { }

[GitHub] spark pull request: [SPARK-4772] Clear local copies of accumulator...

2014-12-09 Thread JoshRosen
Github user JoshRosen commented on a diff in the pull request: https://github.com/apache/spark/pull/3570#discussion_r21557013 --- Diff: core/src/main/scala/org/apache/spark/Accumulators.scala --- @@ -281,7 +282,9 @@ object AccumulatorParam { private object Accumulators {

[GitHub] spark pull request: [SPARK-3967] don’t redundantly overwrite exe...

2014-12-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/2848#discussion_r21556861 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -412,6 +408,85 @@ private[spark] object Utils extends Logging { }

[GitHub] spark pull request: [SPARK-4765] Make GC time always shown in UI.

2014-12-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3622#issuecomment-66345669 [Test build #24255 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24255/consoleFull) for PR 3622 at commit [`15ac242`](https://githu

[GitHub] spark pull request: [SPARK-4048] Enhance and extend hadoop-provide...

2014-12-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/2982#issuecomment-66345643 [Test build #24256 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24256/consoleFull) for PR 2982 at commit [`7377e7b`](https://githu

[GitHub] spark pull request: [SPARK-3967] don’t redundantly overwrite exe...

2014-12-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/2848#discussion_r21556751 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -412,6 +408,85 @@ private[spark] object Utils extends Logging { }

[GitHub] spark pull request: [SPARK-3967] don’t redundantly overwrite exe...

2014-12-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/2848#discussion_r21556716 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -412,6 +408,85 @@ private[spark] object Utils extends Logging { }

[GitHub] spark pull request: SPARK-4338. [YARN] Ditch yarn-alpha.

2014-12-09 Thread sryza
Github user sryza commented on the pull request: https://github.com/apache/spark/pull/3215#issuecomment-66345411 Woot! I'll make that fix as part of SPARK-4447. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[GitHub] spark pull request: [SPARK-3967] don’t redundantly overwrite exe...

2014-12-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/2848#discussion_r21556581 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -412,6 +408,85 @@ private[spark] object Utils extends Logging { }

[GitHub] spark pull request: [SPARK-3967] don’t redundantly overwrite exe...

2014-12-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/2848#discussion_r21556631 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -412,6 +408,85 @@ private[spark] object Utils extends Logging { }

[GitHub] spark pull request: [SPARK-3382] GradientDescent convergence toler...

2014-12-09 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/3636#issuecomment-66345003 @Lewuathe Thanks for the updates! I just saw a couple more things, but I think it's almost ready. --- If your project is set up for it, you can reply to this email an

[GitHub] spark pull request: [SPARK-4791] [sql] Infer schema from case clas...

2014-12-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3646#issuecomment-66345006 [Test build #543 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/543/consoleFull) for PR 3646 at commit [`796b2e4`](https://github

[GitHub] spark pull request: [SPARK-3382] GradientDescent convergence toler...

2014-12-09 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/3636#discussion_r21556490 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/optimization/GradientDescentSuite.scala --- @@ -138,6 +138,45 @@ class GradientDescentSuite extends F

[GitHub] spark pull request: [SPARK-3382] GradientDescent convergence toler...

2014-12-09 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/3636#discussion_r21556494 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/optimization/GradientDescentSuite.scala --- @@ -138,6 +138,45 @@ class GradientDescentSuite extends F

[GitHub] spark pull request: [SPARK-3382] GradientDescent convergence toler...

2014-12-09 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/3636#discussion_r21556486 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/optimization/GradientDescent.scala --- @@ -182,34 +202,40 @@ object GradientDescent extends Logging {

[GitHub] spark pull request: [SPARK-3382] GradientDescent convergence toler...

2014-12-09 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/3636#discussion_r21556482 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/optimization/GradientDescent.scala --- @@ -155,7 +169,13 @@ object GradientDescent extends Logging {

[GitHub] spark pull request: [SPARK-3382] GradientDescent convergence toler...

2014-12-09 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/3636#discussion_r21556478 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/optimization/GradientDescent.scala --- @@ -142,7 +154,9 @@ object GradientDescent extends Logging {

[GitHub] spark pull request: [SPARK-4765] Make GC time always shown in UI.

2014-12-09 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3622#issuecomment-66344482 @kayousterhout Yeah, there must be something about `private object` where it behaves differently than `private class` because we ran into the same problem over at #3570

[GitHub] spark pull request: [SPARK-3967] don’t redundantly overwrite exe...

2014-12-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/2848#discussion_r21556089 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -412,6 +408,85 @@ private[spark] object Utils extends Logging { }

[GitHub] spark pull request: [SPARK-4792] Add error message when making loc...

2014-12-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3635#issuecomment-66344031 [Test build #24254 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24254/consoleFull) for PR 3635 at commit [`2a55bc2`](https://githu

[GitHub] spark pull request: [SPARK-4461][YARN] pass extra java options to ...

2014-12-09 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/3409#issuecomment-66343795 the spark.yarn.applicationMaster.waitTries is being changed in a pull request to change it from tries to time and there is another one to add memory overhead param. I

[GitHub] spark pull request: [SPARK-4797] Replace breezeSquaredDistance

2014-12-09 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/3643#issuecomment-66342802 Hi, it looks like this may be faster for dense vectors but not for sparse. SparseVector.toArray will create a dense vector, making it much slower if the vector is very

[GitHub] spark pull request: [SPARK-4792] Add error message when making loc...

2014-12-09 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/3635#issuecomment-66342602 Jenkins, this is ok to test. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] spark pull request: [SPARK-4461][YARN] pass extra java options to ...

2014-12-09 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/3409#issuecomment-66342475 By the way, I just dug into the codebase and found that we already have the following: ``` spark.yarn.appMasterEnv spark.yarn.applicationMaster.waitTries

[GitHub] spark pull request: [SPARK-4461][YARN] pass extra java options to ...

2014-12-09 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/3409#issuecomment-66341713 `clientam` is fine, though maybe we should camel case it `clientAm`. Though I still prefer keeping it terse as `spark.yarn.am.*` and just throw an exception or log a w

[GitHub] spark pull request: [SPARK-4461][YARN] pass extra java options to ...

2014-12-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/3409#discussion_r21554124 --- Diff: yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ClientBase.scala --- @@ -360,6 +360,10 @@ private[spark] trait ClientBase extends Logging

[GitHub] spark pull request: [SPARK-4461][YARN] pass extra java options to ...

2014-12-09 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/3409#issuecomment-66340028 How about a compromise: `spark.yarn.clientam.*`? I don't like just "am" because, while terse, it's ambiguous. --- If your project is set up for it, you can reply to this

[GitHub] spark pull request: [SPARK-4461][YARN] pass extra java options to ...

2014-12-09 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/3409#issuecomment-66339412 By the way you will need to rebase this to master because we just removed support for yarn alpha. --- If your project is set up for it, you can reply to this email an

[GitHub] spark pull request: [SPARK-4461][YARN] pass extra java options to ...

2014-12-09 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/3409#issuecomment-66339357 Hey regarding the naming, I would actually prefer `spark.yarn.am.*` over `spark.yarn.clientmode.am.*`. Although I agree that it's clearer to have clientmode in there,

[GitHub] spark pull request: [SPARK-1953][YARN]yarn client mode Application...

2014-12-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/3607#discussion_r21553460 --- Diff: yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ClientArguments.scala --- @@ -39,23 +39,34 @@ private[spark] class ClientArguments(args:

[GitHub] spark pull request: [SPARK-1953][YARN]yarn client mode Application...

2014-12-09 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/3607#issuecomment-66338560 By the way you will need to rebase to master since we just removed the support for yarn alpha. --- If your project is set up for it, you can reply to this email and h

[GitHub] spark pull request: SPARK-4338. [YARN] Ditch yarn-alpha.

2014-12-09 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/3215 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: SPARK-4338. [YARN] Ditch yarn-alpha.

2014-12-09 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/3215#issuecomment-66337338 Hey this LGTM. I'm just gonna merge this. My comments are minor enough that we can slip them in later. --- If your project is set up for it, you can reply to this ema

[GitHub] spark pull request: SPARK-4338. [YARN] Ditch yarn-alpha.

2014-12-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/3215#discussion_r21552652 --- Diff: yarn/alpha/src/main/scala/org/apache/spark/deploy/yarn/YarnRMClientImpl.scala --- @@ -1,118 +0,0 @@ -/* - * Licensed to the Apache Softw

[GitHub] spark pull request: SPARK-4338. [YARN] Ditch yarn-alpha.

2014-12-09 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/3215#discussion_r21552599 --- Diff: yarn/src/main/scala/org/apache/spark/deploy/yarn/ClientArguments.scala --- @@ -178,21 +178,25 @@ private[spark] class ClientArguments(args: Arra

[GitHub] spark pull request: [SPARK-4791] [sql] Infer schema from case clas...

2014-12-09 Thread KirthiRaman
Github user KirthiRaman commented on the pull request: https://github.com/apache/spark/pull/3646#issuecomment-66336768 unsubscribe On Tue, Dec 9, 2014 at 1:57 PM, UCB AMPLab wrote: > Test FAILed. > Refer to this link for build results (access rights to CI server n

[GitHub] spark pull request: [SPARK-4791] [sql] Infer schema from case clas...

2014-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3646#issuecomment-66336585 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24

[GitHub] spark pull request: SPARK-4338. [YARN] Ditch yarn-alpha.

2014-12-09 Thread andrewor14
Github user andrewor14 commented on the pull request: https://github.com/apache/spark/pull/3215#issuecomment-66336371 We actually don't need to wait on the RC to merge this since this is only going into master. I'll take a quick look and will likely merge it after that since this is p

[GitHub] spark pull request: [SPARK-4798][SQL] A new set of Parquet testing...

2014-12-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3644#issuecomment-66336134 [Test build #24251 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24251/consoleFull) for PR 3644 at commit [`ee17d7b`](https://gith

[GitHub] spark pull request: [SPARK-4798][SQL] A new set of Parquet testing...

2014-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3644#issuecomment-66336150 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/24

[GitHub] spark pull request: [SPARK-2309][MLlib] Generalize the binary logi...

2014-12-09 Thread dbtsai
Github user dbtsai commented on the pull request: https://github.com/apache/spark/pull/1379#issuecomment-66336110 @avulanov 1. I did the same optimization for MLlib in [my recently PRs](https://github.com/apache/spark/commits/master?author=dbtsai). * Accessing the va

[GitHub] spark pull request: [SPARK-4791] [sql] Infer schema from case clas...

2014-12-09 Thread jkbradley
GitHub user jkbradley opened a pull request: https://github.com/apache/spark/pull/3646 [SPARK-4791] [sql] Infer schema from case class with multiple constructors Modified ScalaReflection.schemaFor to take primary constructor of Product when there are multiple constructors. Added te

[GitHub] spark pull request: [SPARK-874] adding a --wait flag

2014-12-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3567#issuecomment-66334417 [Test build #24252 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24252/consoleFull) for PR 3567 at commit [`d05c5bb`](https://githu

[GitHub] spark pull request: [SPARK-874] adding a --wait flag

2014-12-09 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/3567#issuecomment-66333686 Jenkins, test this please. LGTM pending tests. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pr

[GitHub] spark pull request: [SPARK-4785] [SQL] Support udf instance ser/de...

2014-12-09 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/3640#issuecomment-66332833 Thanks a lot guys for digging into this! Merged to master and branch 1.2 --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: [SPARK-4785] [SQL] Support udf instance ser/de...

2014-12-09 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/3640 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [MLLIB][SPARK-4675] Find similar products and ...

2014-12-09 Thread MLnick
Github user MLnick commented on the pull request: https://github.com/apache/spark/pull/3536#issuecomment-66331985 I'd agree that cosine similarity is preferred. Can't really think of a case where I've *not* used cosine sim for a similar items or similar users computation. Of course, i

[GitHub] spark pull request: [SPARK-2199] [mllib] topic modeling

2014-12-09 Thread akopich
Github user akopich commented on the pull request: https://github.com/apache/spark/pull/1269#issuecomment-66325635 @chazchandler, thank you very much for your help. I shouldn't have rebase on master. Rebase on 1.2 was successful. --- If your project is set up for it, you can reply t

[GitHub] spark pull request: [SPARK-4798][SQL] A new set of Parquet testing...

2014-12-09 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3644#issuecomment-66325520 [Test build #24251 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/24251/consoleFull) for PR 3644 at commit [`ee17d7b`](https://githu

[GitHub] spark pull request: [SPARK-4799] Use IP address instead of local h...

2014-12-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3645#issuecomment-66324983 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[GitHub] spark pull request: [SPARK-4799] Use IP address instead of local h...

2014-12-09 Thread smola
GitHub user smola opened a pull request: https://github.com/apache/spark/pull/3645 [SPARK-4799] Use IP address instead of local hostname in ConnectionManager See https://issues.apache.org/jira/browse/SPARK-4799 Spark fails when a node hostname is not resolvable by

[GitHub] spark pull request: [SPARK-4798][SQL] A new set of Parquet testing...

2014-12-09 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/3644#issuecomment-66324222 retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this fe

<    1   2   3   4   5   >