[GitHub] spark pull request: [SPARK-5154] [PySpark] [Streaming] Kafka strea...

2015-01-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3715#issuecomment-71722042 [Test build #26177 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26177/consoleFull) for PR 3715 at commit [`31e2317`](https://gith

[GitHub] spark pull request: [SPARK-3381] [MLlib] Eliminate bins for unorde...

2015-01-27 Thread MechCoder
GitHub user MechCoder opened a pull request: https://github.com/apache/spark/pull/4231 [SPARK-3381] [MLlib] Eliminate bins for unordered features For unordered features, it is sufficient to use splits since the threshold of the split corresponds to the the threshold of the HighSplit

[GitHub] spark pull request: [SPARK-3381] [MLlib] Eliminate bins for unorde...

2015-01-27 Thread MechCoder
Github user MechCoder commented on the pull request: https://github.com/apache/spark/pull/4231#issuecomment-71724940 ping @jkbradley . Would be great if you could have a look. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as we

[GitHub] spark pull request: [SPARK-3381] [MLlib] Eliminate bins for unorde...

2015-01-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4231#issuecomment-71725517 [Test build #26183 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26183/consoleFull) for PR 4231 at commit [`6dfaa28`](https://githu

[GitHub] spark pull request: [SPARK-3974][MLlib] Distributed Block Matrix A...

2015-01-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3200#issuecomment-71725650 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26

[GitHub] spark pull request: [SPARK-3974][MLlib] Distributed Block Matrix A...

2015-01-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3200#issuecomment-71725639 [Test build #26178 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26178/consoleFull) for PR 3200 at commit [`5eecd48`](https://gith

[GitHub] spark pull request: [SPARK-5430] move treeReduce and treeAggregate...

2015-01-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4228#issuecomment-71726359 [Test build #26184 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26184/consoleFull) for PR 4228 at commit [`3ae1a4b`](https://githu

[GitHub] spark pull request: [SPARK-5430] move treeReduce and treeAggregate...

2015-01-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4228#issuecomment-71726735 [Test build #26184 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26184/consoleFull) for PR 4228 at commit [`3ae1a4b`](https://gith

[GitHub] spark pull request: [SPARK-5097][SQL] DataFrame

2015-01-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4173#issuecomment-71726744 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26

[GitHub] spark pull request: [SPARK-5430] move treeReduce and treeAggregate...

2015-01-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4228#issuecomment-71726740 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26

[GitHub] spark pull request: [SPARK-5097][SQL] DataFrame

2015-01-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4173#issuecomment-71726729 [Test build #26175 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26175/consoleFull) for PR 4173 at commit [`828f70d`](https://gith

[GitHub] spark pull request: [SPARK-4879] Use the Spark driver to authorize...

2015-01-27 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/4155#issuecomment-71727094 > I don't exactly know how to test the full stack (as now this requires actual executors to throw the TaskEndReason back to the driver) while having the scheduler someh

[GitHub] spark pull request: [SPARK-5432] DriverSuite and SparkSubmitSuite ...

2015-01-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4230#issuecomment-71728069 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26

[GitHub] spark pull request: [SPARK-5432] DriverSuite and SparkSubmitSuite ...

2015-01-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4230#issuecomment-71728056 [Test build #26181 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26181/consoleFull) for PR 4230 at commit [`8092c36`](https://gith

[GitHub] spark pull request: [WIP][SPARK-5388] Provide a stable application...

2015-01-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4216#issuecomment-71728944 [Test build #26186 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26186/consoleFull) for PR 4216 at commit [`d8d3717`](https://githu

[GitHub] spark pull request: [SPARK-5430] move treeReduce and treeAggregate...

2015-01-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4228#issuecomment-71728939 [Test build #26185 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26185/consoleFull) for PR 4228 at commit [`e89a43e`](https://githu

[GitHub] spark pull request: [WIP][SPARK-4586][MLLIB] Python API for ML pip...

2015-01-27 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/4151#discussion_r23643515 --- Diff: python/pyspark/ml/util.py --- @@ -0,0 +1,35 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor lic

[GitHub] spark pull request: [WIP][SPARK-5388] Provide a stable application...

2015-01-27 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/4216#issuecomment-71729792 Hi @andrewor14 , I see the bug is targeted at 1.3. This seems like a pretty considerable change to come so code to the branching point. Should we target it at 1.4 instead?

[GitHub] spark pull request: [WIP][SPARK-4586][MLLIB] Python API for ML pip...

2015-01-27 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/4151#discussion_r23643966 --- Diff: python/pyspark/ml/util.py --- @@ -0,0 +1,35 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor lic

[GitHub] spark pull request: [SPARK-5094][MLlib] Add Python API for Gradien...

2015-01-27 Thread cthom
Github user cthom commented on the pull request: https://github.com/apache/spark/pull/3951#issuecomment-71731512 Is there anyway to maintain some kind state about the model as it's being built? For GBT models, one usually sees a plot of the error vs number of trees in the model. If th

[GitHub] spark pull request: [SPARK-5428]: Declare the 'assembly' module at...

2015-01-27 Thread tzolov
GitHub user tzolov opened a pull request: https://github.com/apache/spark/pull/4232 [SPARK-5428]: Declare the 'assembly' module at the bottom of the element in the parent POM - Reorder the pom's element list to put the assembly module at the bottom. - Add a comment that remi

[GitHub] spark pull request: [SPARK-5428]: Declare the 'assembly' module at...

2015-01-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4232#issuecomment-71734526 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pro

[GitHub] spark pull request: [SPARK-5097][SQL] DataFrame

2015-01-27 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/4173#discussion_r23645877 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Column.scala --- @@ -0,0 +1,528 @@ +/* +* Licensed to the Apache Software Foundation (ASF) unde

[GitHub] spark pull request: [SPARK-5135][SQL] Add support for describe [ex...

2015-01-27 Thread rxin
Github user rxin closed the pull request at: https://github.com/apache/spark/pull/4127 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enable

[GitHub] spark pull request: [SPARK-5135][SQL] Add support for describe [ex...

2015-01-27 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/4127#issuecomment-71734642 Ah never mind this is my own pull request. Closing it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If you

[GitHub] spark pull request: [SPARK-5097][SQL] DataFrame

2015-01-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4173#issuecomment-71734627 [Test build #26187 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26187/consoleFull) for PR 4173 at commit [`0a1a73b`](https://githu

[GitHub] spark pull request: [SPARK-5094][MLlib] Add Python API for Gradien...

2015-01-27 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/3951#issuecomment-71734912 @cthom Validation on-the-fly during training would be great to have. Let's discuss it in a separate JIRA; I just created one: [https://issues.apache.org/jira/browse/S

[GitHub] spark pull request: [SPARK-5097][SQL] DataFrame

2015-01-27 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/4173#discussion_r23646110 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrame.scala --- @@ -0,0 +1,596 @@ +/* +* Licensed to the Apache Software Foundation (ASF) u

[GitHub] spark pull request: [SPARK-5097][SQL] DataFrame

2015-01-27 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/4173#discussion_r23646135 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Column.scala --- @@ -0,0 +1,528 @@ +/* +* Licensed to the Apache Software Foundation (ASF) unde

[GitHub] spark pull request: [SPARK-4879] Use the Spark driver to authorize...

2015-01-27 Thread aarondav
Github user aarondav commented on a diff in the pull request: https://github.com/apache/spark/pull/4155#discussion_r23646271 --- Diff: core/src/main/scala/org/apache/spark/scheduler/OutputCommitCoordinator.scala --- @@ -0,0 +1,258 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request: [SPARK-4879] Use the Spark driver to authorize...

2015-01-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4155#issuecomment-71735460 **[Test build #26179 timed out](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26179/consoleFull)** for PR 4155 at commit [`d63f63f`](https://git

[GitHub] spark pull request: [SPARK-4879] Use the Spark driver to authorize...

2015-01-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4155#issuecomment-71735472 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26

[GitHub] spark pull request: [SPARK-5097][SQL] DataFrame

2015-01-27 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/4173#discussion_r23646548 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrame.scala --- @@ -0,0 +1,596 @@ +/* +* Licensed to the Apache Software Foundation (ASF) u

[GitHub] spark pull request: [SPARK-5097][SQL] DataFrame

2015-01-27 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/4173#discussion_r23646583 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrame.scala --- @@ -0,0 +1,596 @@ +/* +* Licensed to the Apache Software Foundation (ASF) u

[GitHub] spark pull request: [SPARK-5097][SQL] DataFrame

2015-01-27 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/4173#discussion_r23646640 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrame.scala --- @@ -0,0 +1,596 @@ +/* +* Licensed to the Apache Software Foundation (ASF) u

[GitHub] spark pull request: [SPARK-3381] [MLlib] Eliminate bins for unorde...

2015-01-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4231#issuecomment-71737068 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26

[GitHub] spark pull request: [SPARK-3381] [MLlib] Eliminate bins for unorde...

2015-01-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4231#issuecomment-71737055 [Test build #26183 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26183/consoleFull) for PR 4231 at commit [`6dfaa28`](https://gith

[GitHub] spark pull request: SPARK-2450 Adds executor log links to Web UI

2015-01-27 Thread ksakellis
Github user ksakellis commented on the pull request: https://github.com/apache/spark/pull/3486#issuecomment-71739273 @andrewor14 @JoshRosen Ping. Can you guys review this. I'd like to get it in Spark 1.3 --- If your project is set up for it, you can reply to this email and have your

[GitHub] spark pull request: [SPARK-4879] Use the Spark driver to authorize...

2015-01-27 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/4155#discussion_r23648248 --- Diff: core/src/main/scala/org/apache/spark/scheduler/OutputCommitCoordinator.scala --- @@ -0,0 +1,258 @@ +/* + * Licensed to the Apache Software Fo

[GitHub] spark pull request: SPARK-2450 Adds executor log links to Web UI

2015-01-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3486#issuecomment-71739952 [Test build #26189 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26189/consoleFull) for PR 3486 at commit [`33d4dc0`](https://githu

[GitHub] spark pull request: [SPARK-5437] Fix DriverSuite and SparkSubmitSu...

2015-01-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4230#issuecomment-71739945 [Test build #26188 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26188/consoleFull) for PR 4230 at commit [`f5c80fd`](https://githu

[GitHub] spark pull request: [SPARK-5416] init Executor.threadPool before E...

2015-01-27 Thread sryza
Github user sryza commented on the pull request: https://github.com/apache/spark/pull/4212#issuecomment-71740268 This looks reasonable. Was initially worried that changing the order might mess with the effect of classloader instantiation on the thread pool but on deeper inspection th

[GitHub] spark pull request: SPARK-5300 Add LocalFileSystem which will retu...

2015-01-27 Thread ehiggs
Github user ehiggs commented on the pull request: https://github.com/apache/spark/pull/4204#issuecomment-71740260 Thanks for your feedback. So the `FileInputFormat` is responsible for sorting the file pieces. I think this means any file format that one expects `sortByKey` to

[GitHub] spark pull request: [WIP][SPARK-5388] Provide a stable application...

2015-01-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4216#issuecomment-71740548 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26

[GitHub] spark pull request: [WIP][SPARK-5388] Provide a stable application...

2015-01-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4216#issuecomment-71740533 [Test build #26186 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26186/consoleFull) for PR 4216 at commit [`d8d3717`](https://gith

[GitHub] spark pull request: [SPARK-5430] move treeReduce and treeAggregate...

2015-01-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4228#issuecomment-71740589 [Test build #26185 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26185/consoleFull) for PR 4228 at commit [`e89a43e`](https://gith

[GitHub] spark pull request: [SPARK-5430] move treeReduce and treeAggregate...

2015-01-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4228#issuecomment-71740598 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26

[GitHub] spark pull request: [WIP] [SPARK-4587] [mllib] ML model import/exp...

2015-01-27 Thread jkbradley
GitHub user jkbradley opened a pull request: https://github.com/apache/spark/pull/4233 [WIP] [SPARK-4587] [mllib] ML model import/export This is a WIP PR for Parquet-based model import/export. Please see the design doc on [the JIRA](https://issues.apache.org/jira/browse/SPARK-4587)

[GitHub] spark pull request: [SPARK-5416] init Executor.threadPool before E...

2015-01-27 Thread ryan-williams
Github user ryan-williams commented on the pull request: https://github.com/apache/spark/pull/4212#issuecomment-71741357 cool, thanks Sandy, I was also a little unsure whether any crazy side effects could result from this but couldn't detect any. --- If your project is set up for it,

[GitHub] spark pull request: [WIP] [SPARK-4587] [mllib] ML model import/exp...

2015-01-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4233#issuecomment-71741610 [Test build #26190 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26190/consoleFull) for PR 4233 at commit [`14711b7`](https://githu

[GitHub] spark pull request: SPARK-5300 Add LocalFileSystem which will retu...

2015-01-27 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/4204#issuecomment-71741379 Yes, or override `listStatus` I think. I suppose that also has the problem of not being universal. I wonder how often it's necessary to ensure that partitions are preserve

[GitHub] spark pull request: [SPARK-3562]Periodic cleanup event logs

2015-01-27 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/4214#discussion_r23649826 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -53,8 +79,10 @@ private[history] class FsHistoryProvider(conf: Sp

[GitHub] spark pull request: [SPARK-3562]Periodic cleanup event logs

2015-01-27 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/4214#discussion_r23649795 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -43,9 +47,31 @@ private[history] class FsHistoryProvider(conf: Sp

[GitHub] spark pull request: [SPARK-4879] Use the Spark driver to authorize...

2015-01-27 Thread mccheah
Github user mccheah commented on the pull request: https://github.com/apache/spark/pull/4155#issuecomment-71743261 Jenkins, retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-3562]Periodic cleanup event logs

2015-01-27 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/4214#discussion_r23649845 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -53,8 +79,10 @@ private[history] class FsHistoryProvider(conf: Sp

[GitHub] spark pull request: [SPARK-3562]Periodic cleanup event logs

2015-01-27 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/4214#issuecomment-71743602 LGTM aside from some very minor things. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project doe

[GitHub] spark pull request: [WIP][SPARK-5341] Use maven coordinates as dep...

2015-01-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4215#issuecomment-71744089 [Test build #26191 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26191/consoleFull) for PR 4215 at commit [`2edc9b5`](https://githu

[GitHub] spark pull request: [SPARK-4879] Use the Spark driver to authorize...

2015-01-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4155#issuecomment-71744093 [Test build #26192 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26192/consoleFull) for PR 4155 at commit [`d63f63f`](https://githu

[GitHub] spark pull request: [SPARK-5361]python tuple not supported while c...

2015-01-27 Thread wingchen
Github user wingchen commented on the pull request: https://github.com/apache/spark/pull/4146#issuecomment-71744301 @JoshRosen Can we have this in the next release? We will have to use our own fork if it's not in. Thanks --- If your project is set up for it, you can reply to this ema

[GitHub] spark pull request: [SPARK-4879] Use the Spark driver to authorize...

2015-01-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4155#issuecomment-71744862 [Test build #26193 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26193/consoleFull) for PR 4155 at commit [`d63f63f`](https://githu

[GitHub] spark pull request: [SPARK-4809] Rework Guava library shading.

2015-01-27 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/3658#issuecomment-71747360 @vanzin yeah this is a fair point, this would mean that network/common would expose the (un-shaded) Guava classes... a bit clunky. If those classes change in the future

[GitHub] spark pull request: [SPARK-5097][SQL] DataFrame

2015-01-27 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/4173#discussion_r23651932 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/api.scala --- @@ -0,0 +1,289 @@ +/* +* Licensed to the Apache Software Foundation (ASF) under o

[GitHub] spark pull request: SPARK-3290 [GRAPHX] No unpersist callls in SVD...

2015-01-27 Thread srowen
GitHub user srowen opened a pull request: https://github.com/apache/spark/pull/4234 SPARK-3290 [GRAPHX] No unpersist callls in SVDPlusPlus This just unpersist()s each RDD in this code that was cache()ed. You can merge this pull request into a Git repository by running: $ git pu

[GitHub] spark pull request: SPARK-3290 [GRAPHX] No unpersist callls in SVD...

2015-01-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4234#issuecomment-71749497 [Test build #26194 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26194/consoleFull) for PR 4234 at commit [`c0311bb`](https://githu

[GitHub] spark pull request: [SPARK-3562]Periodic cleanup event logs

2015-01-27 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/4214#discussion_r23652649 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -163,9 +179,6 @@ private[history] class FsHistoryProvider(conf: S

[GitHub] spark pull request: [SPARK-3562]Periodic cleanup event logs

2015-01-27 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/4214#discussion_r23652781 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -113,12 +129,12 @@ private[history] class FsHistoryProvider(conf:

[GitHub] spark pull request: [SPARK-3562]Periodic cleanup event logs

2015-01-27 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/4214#discussion_r23653061 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -163,9 +179,6 @@ private[history] class FsHistoryProvider(conf: S

[GitHub] spark pull request: [SPARK-5097][SQL] DataFrame

2015-01-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4173#issuecomment-71750590 [Test build #26187 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26187/consoleFull) for PR 4173 at commit [`0a1a73b`](https://gith

[GitHub] spark pull request: [SPARK-5097][SQL] DataFrame

2015-01-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4173#issuecomment-71750600 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26

[GitHub] spark pull request: [SPARK-5437] Fix DriverSuite and SparkSubmitSu...

2015-01-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4230#issuecomment-71750882 [Test build #26188 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26188/consoleFull) for PR 4230 at commit [`f5c80fd`](https://gith

[GitHub] spark pull request: [SPARK-4586][MLLIB] Python API for ML pipeline...

2015-01-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4151#issuecomment-71750939 [Test build #26195 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26195/consoleFull) for PR 4151 at commit [`a4f4dbf`](https://githu

[GitHub] spark pull request: SPARK-2450 Adds executor log links to Web UI

2015-01-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/3486#issuecomment-71750978 [Test build #26189 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26189/consoleFull) for PR 3486 at commit [`33d4dc0`](https://gith

[GitHub] spark pull request: [SPARK-5437] Fix DriverSuite and SparkSubmitSu...

2015-01-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4230#issuecomment-71750891 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26

[GitHub] spark pull request: SPARK-2450 Adds executor log links to Web UI

2015-01-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/3486#issuecomment-71750987 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26

[GitHub] spark pull request: [SPARK-5430] move treeReduce and treeAggregate...

2015-01-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4228#issuecomment-71751603 [Test build #26196 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26196/consoleFull) for PR 4228 at commit [`20ad40d`](https://githu

[GitHub] spark pull request: [MLlib] fix python example of ALS in guide

2015-01-27 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4226 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [MLlib] fix python example of ALS in guide

2015-01-27 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/4226#issuecomment-71751797 Merged into master and branch-1.2. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [WIP] [SPARK-4587] [mllib] ML model import/exp...

2015-01-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4233#issuecomment-71752235 [Test build #26190 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26190/consoleFull) for PR 4233 at commit [`14711b7`](https://gith

[GitHub] spark pull request: [WIP] [SPARK-4587] [mllib] ML model import/exp...

2015-01-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4233#issuecomment-71752240 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26

[GitHub] spark pull request: SPARK-3290 [GRAPHX] No unpersist callls in SVD...

2015-01-27 Thread ankurdave
Github user ankurdave commented on the pull request: https://github.com/apache/spark/pull/4234#issuecomment-71752716 I don't think this will always have the desired effect. In the cases where you unpersist upstream RDDs before materializing their results, it's equivalent to never cach

[GitHub] spark pull request: SPARK-5199. FS read metrics should support Com...

2015-01-27 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/4050#issuecomment-71752915 Cool - thanks Sandy! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this f

[GitHub] spark pull request: SPARK-5199. FS read metrics should support Com...

2015-01-27 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4050 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: SPARK-1934 [CORE] "this" reference escape to "...

2015-01-27 Thread pwendell
Github user pwendell commented on the pull request: https://github.com/apache/spark/pull/4225#issuecomment-71753401 @zxwing - want to take a look at this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project do

[GitHub] spark pull request: [WIP][SPARK-5341] Use maven coordinates as dep...

2015-01-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4215#issuecomment-71754108 [Test build #26191 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26191/consoleFull) for PR 4215 at commit [`2edc9b5`](https://gith

[GitHub] spark pull request: [SPARK-4879] Use the Spark driver to authorize...

2015-01-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4155#issuecomment-71754191 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26

[GitHub] spark pull request: [WIP][SPARK-5341] Use maven coordinates as dep...

2015-01-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4215#issuecomment-71754113 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26

[GitHub] spark pull request: [SPARK-4879] Use the Spark driver to authorize...

2015-01-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4155#issuecomment-71754184 [Test build #26192 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26192/consoleFull) for PR 4155 at commit [`d63f63f`](https://gith

[GitHub] spark pull request: [SPARK-4879] Use the Spark driver to authorize...

2015-01-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4155#issuecomment-71754843 [Test build #26193 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26193/consoleFull) for PR 4155 at commit [`d63f63f`](https://gith

[GitHub] spark pull request: [SPARK-4879] Use the Spark driver to authorize...

2015-01-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/4155#issuecomment-71754854 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26

[GitHub] spark pull request: [MLLIB][SPARK-3278] Monotone (Isotonic) regres...

2015-01-27 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/3519#issuecomment-71754871 Btw, the citations should go to the Scala doc, so they appear in the generated API docs. --- If your project is set up for it, you can reply to this email and have your r

[GitHub] spark pull request: [SPARK-4586][MLLIB] Python API for ML pipeline...

2015-01-27 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/4151#issuecomment-71754975 [Test build #26197 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/26197/consoleFull) for PR 4151 at commit [`0882513`](https://githu

[GitHub] spark pull request: [SPARK-4586][MLLIB] Python API for ML pipeline...

2015-01-27 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/4151#discussion_r23655542 --- Diff: examples/src/main/python/ml/simple_text_classification_pipeline.py --- @@ -0,0 +1,79 @@ +# +# Licensed to the Apache Software Foundation (ASF

[GitHub] spark pull request: [SPARK-5430] move treeReduce and treeAggregate...

2015-01-27 Thread sryza
Github user sryza commented on the pull request: https://github.com/apache/spark/pull/4228#issuecomment-71756156 I think this would be a great API to add. Have you weighed adding a numLevels argument to `reduce` itself instead of a new method? --- If your project is set up for it, y

[GitHub] spark pull request: [SPARK-5097][SQL] DataFrame

2015-01-27 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/4173#issuecomment-71756241 Alright I'm going to merge this one since it touches too many moving parts. I will submit another PR later today to update documentation and address Michael's comments. It w

[GitHub] spark pull request: [SPARK-5097][SQL] DataFrame

2015-01-27 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/4173 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enab

[GitHub] spark pull request: [SPARK-4586][MLLIB] Python API for ML pipeline...

2015-01-27 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/4151#discussion_r23655833 --- Diff: python/docs/pyspark.ml.rst --- @@ -0,0 +1,41 @@ +pyspark.ml package += --- End diff -- This package should b

[GitHub] spark pull request: [SPARK-4586][MLLIB] Python API for ML pipeline...

2015-01-27 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/4151#discussion_r23655870 --- Diff: python/docs/pyspark.ml.rst --- @@ -0,0 +1,41 @@ +pyspark.ml package += + +Submodules +-- + +py

[GitHub] spark pull request: [SPARK-5097][SQL] Test cases for DataFrame exp...

2015-01-27 Thread rxin
GitHub user rxin opened a pull request: https://github.com/apache/spark/pull/4235 [SPARK-5097][SQL] Test cases for DataFrame expressions. You can merge this pull request into a Git repository by running: $ git pull https://github.com/rxin/spark df-tests1 Alternatively you can

[GitHub] spark pull request: [SPARK-4586][MLLIB] Python API for ML pipeline...

2015-01-27 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/4151#discussion_r23655941 --- Diff: python/pyspark/ml/__init__.py --- @@ -0,0 +1,324 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributo

[GitHub] spark pull request: [SPARK-4586][MLLIB] Python API for ML pipeline...

2015-01-27 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/4151#discussion_r23655975 --- Diff: python/pyspark/ml/__init__.py --- @@ -0,0 +1,324 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributo

<    1   2   3   4   5   >