[GitHub] spark pull request: [SPARK-14984][ML] Deprecated model field in Li...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12763#issuecomment-215551667 **[Test build #57270 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57270/consoleFull)** for PR 12763 at commit

[GitHub] spark pull request: [SPARK-11171][SPARK-11237][SPARK-11241][ML] Tr...

2016-04-28 Thread vruusmann
Github user vruusmann commented on the pull request: https://github.com/apache/spark/pull/9207#issuecomment-215551480 The main difference between PMML and PFA is the abstraction level. PMML is a high-level language (more similar to modeling languages such as UML), where you're

[GitHub] spark pull request: [SPARK-14938][ML] replace RDD.map with Dataset...

2016-04-28 Thread jaceklaskowski
Github user jaceklaskowski commented on the pull request: https://github.com/apache/spark/pull/12718#issuecomment-215550190 Other than the few places where you could use symbols not string literals LGTM. Excellent job! Thanks. --- If your project is set up for it, you can reply to

[GitHub] spark pull request: [SPARK-14830][SQL] Add RemoveRepetitionFromGro...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12590#issuecomment-215549270 **[Test build #57274 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57274/consoleFull)** for PR 12590 at commit

[GitHub] spark pull request: [SPARK-4224] [Core][YARN] - Support group acls...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12760#issuecomment-215549032 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-4224] [Core][YARN] - Support group acls...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12760#issuecomment-215549034 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-4224] [Core][YARN] - Support group acls...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12760#issuecomment-215548834 **[Test build #57266 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57266/consoleFull)** for PR 12760 at commit

[GitHub] spark pull request: [SPARK-14931][ML][PySpark] Mismatched default ...

2016-04-28 Thread yinxusen
Github user yinxusen commented on the pull request: https://github.com/apache/spark/pull/12738#issuecomment-215548179 @holdenk It seems not good enough, I'll add a unit test to check the mismatch, then it will be easy to fix the mismatch default values issue. --- If your project is

[GitHub] spark pull request: [SPARK-14613][ML] Add @Since into the matrix a...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12416#issuecomment-215547993 **[Test build #57273 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57273/consoleFull)** for PR 12416 at commit

[GitHub] spark pull request: [SPARK-14939][SQL] Improve EliminateSorts opti...

2016-04-28 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-215547799 Thank you so much, @gatorsmile ! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-14938][ML] replace RDD.map with Dataset...

2016-04-28 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/12718#discussion_r61494448 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/ChiSqSelector.scala --- @@ -79,11 +79,12 @@ final class ChiSqSelector(override val uid:

[GitHub] spark pull request: [SPARK-4224] [Core][YARN] - Support group acls...

2016-04-28 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/12760#discussion_r61494141 --- Diff: docs/configuration.md --- @@ -1249,8 +1249,32 @@ Apart from these, the following properties are also available, and may be useful

[GitHub] spark pull request: [SPARK-14891][ML] Add schema validation for AL...

2016-04-28 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/12762#discussion_r61494019 --- Diff: mllib/src/main/scala/org/apache/spark/ml/recommendation/ALS.scala --- @@ -53,24 +53,43 @@ import org.apache.spark.util.random.XORShiftRandom

[GitHub] spark pull request: [SPARK-14939][SQL] Improve EliminateSorts opti...

2016-04-28 Thread gatorsmile
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-215546800 Sure, will do it tomorrow. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-14939][SQL] Improve EliminateSorts opti...

2016-04-28 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-215545987 Hi, @gatorsmile and @cloud-fan . Could you take a look at this PR when you have some time? I think you are the best persons to review this PR. --- If

[GitHub] spark pull request: [SPARK-14613][ML] Add @Since into the matrix a...

2016-04-28 Thread dbtsai
Github user dbtsai commented on the pull request: https://github.com/apache/spark/pull/12416#issuecomment-215545617 @pravingadakh Yes. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-14464] [MLLIB] Better support for logis...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12761#issuecomment-215545319 **[Test build #57272 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57272/consoleFull)** for PR 12761 at commit

[GitHub] spark pull request: [SPARK-14970][SQL] Prevent DataSource from enu...

2016-04-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/12748 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-14599][ML] BaggedPoint should support s...

2016-04-28 Thread sethah
Github user sethah commented on the pull request: https://github.com/apache/spark/pull/12370#issuecomment-215544854 ping @jkbradley @MLnick I created this PR and [#12374](https://github.com/apache/spark/pull/12374) to make

[GitHub] spark pull request: [SPARK-14613][ML] Add @Since into the matrix a...

2016-04-28 Thread pravingadakh
Github user pravingadakh commented on the pull request: https://github.com/apache/spark/pull/12416#issuecomment-215543380 @dbtsai Is this something you had in mind? Zip file contains the patch for modules.

[GitHub] spark pull request: [SPARK-4224] [Core][YARN] - Support group acls...

2016-04-28 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/12760#discussion_r61492158 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -2179,6 +2179,16 @@ private[spark] object Utils extends Logging {

[GitHub] spark pull request: [SPARK-14984][ML] Deprecated model field in Li...

2016-04-28 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/12763#issuecomment-215541758 CC: @yanboliang could you please take a look since this modifies the code you added? Thanks! --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request: [SPARK-14965] [SQL] Indicate an exception is t...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12758#issuecomment-215541630 **[Test build #2902 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2902/consoleFull)** for PR 12758 at commit

[GitHub] spark pull request: [SPARK-14891][ML] Add schema validation for AL...

2016-04-28 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/12762#discussion_r61491217 --- Diff: mllib/src/main/scala/org/apache/spark/ml/recommendation/ALS.scala --- @@ -53,24 +53,43 @@ import org.apache.spark.util.random.XORShiftRandom

[GitHub] spark pull request: [SPARK-12810][PySpark] PySpark CrossValidatorM...

2016-04-28 Thread vectorijk
Github user vectorijk commented on a diff in the pull request: https://github.com/apache/spark/pull/12464#discussion_r61491169 --- Diff: python/pyspark/ml/tests.py --- @@ -461,6 +461,31 @@ def _fit(self, dataset): class CrossValidatorTests(PySparkTestCase): +

[GitHub] spark pull request: [SPARK-14984][ML] Deprecated model field in Li...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12763#issuecomment-215541439 **[Test build #57270 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57270/consoleFull)** for PR 12763 at commit

[GitHub] spark pull request: [SPARK-14939][SQL] Improve EliminateSorts opti...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12719#issuecomment-215541446 **[Test build #57271 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57271/consoleFull)** for PR 12719 at commit

[GitHub] spark pull request: [SPARK-14984][ML] Deprecated model field in Li...

2016-04-28 Thread jkbradley
GitHub user jkbradley opened a pull request: https://github.com/apache/spark/pull/12763 [SPARK-14984][ML] Deprecated model field in LinearRegressionSummary ## What changes were proposed in this pull request? Deprecated model field in LinearRegressionSummary Removed

[GitHub] spark pull request: [SPARK-14891][ML] Add schema validation for AL...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12762#issuecomment-215539505 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-14464] [MLLIB] Better support for logis...

2016-04-28 Thread dbtsai
Github user dbtsai commented on the pull request: https://github.com/apache/spark/pull/12761#issuecomment-215539469 Make it build first, and then we can start to review the code. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: [SPARK-14891][ML] Add schema validation for AL...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12762#issuecomment-215539507 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-14891][ML] Add schema validation for AL...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12762#issuecomment-215539378 **[Test build #57267 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57267/consoleFull)** for PR 12762 at commit

[GitHub] spark pull request: [SPARK-14464] [MLLIB] Better support for logis...

2016-04-28 Thread dbtsai
Github user dbtsai commented on the pull request: https://github.com/apache/spark/pull/12761#issuecomment-215539228 You need to manually add it into MiMa exclude. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-14891][ML] Add schema validation for AL...

2016-04-28 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/12762#discussion_r61489680 --- Diff: mllib/src/main/scala/org/apache/spark/ml/recommendation/ALS.scala --- @@ -53,24 +53,43 @@ import org.apache.spark.util.random.XORShiftRandom

[GitHub] spark pull request: [SPARK-11940][PYSPARK][ML] Python API for ml.c...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12723#issuecomment-215538648 **[Test build #57268 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57268/consoleFull)** for PR 12723 at commit

[GitHub] spark pull request: [SPARK-11940][PYSPARK][ML] Python API for ml.c...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12723#issuecomment-215538699 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-11940][PYSPARK][ML] Python API for ml.c...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12723#issuecomment-215538704 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-14585][ML][WIP] Provide accessor method...

2016-04-28 Thread pravingadakh
Github user pravingadakh commented on the pull request: https://github.com/apache/spark/pull/12420#issuecomment-215538279 @jkbradley I'll update it soon. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-14464] [MLLIB] Better support for logis...

2016-04-28 Thread daniel-siegmann-aol
Github user daniel-siegmann-aol commented on the pull request: https://github.com/apache/spark/pull/12761#issuecomment-215536649 Binary compatibility check failed: method this()Unit in class org.apache.spark.mllib.stat.MultivariateOnlineSummarizer does not have a correspondent in

[GitHub] spark pull request: [SPARK-14467][SQL] Experiments: Async I/O in F...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12667#issuecomment-215536529 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-12919][SPARKR] Implement dapply() on Da...

2016-04-28 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/12493#issuecomment-215536504 LGTM over all. There are still a few of the change that are not needed by this PR (for example, SERIALIZED_R_DATA_SCHEMA), are these kept for future? --- If your

[GitHub] spark pull request: [SPARK-14467][SQL] Experiments: Async I/O in F...

2016-04-28 Thread sameeragarwal
Github user sameeragarwal closed the pull request at: https://github.com/apache/spark/pull/12667 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark pull request: [SPARK-14467][SQL] Experiments: Async I/O in F...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12667#issuecomment-215536531 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-14467][SQL] Experiments: Async I/O in F...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12667#issuecomment-215536188 **[Test build #57265 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57265/consoleFull)** for PR 12667 at commit

[GitHub] spark pull request: [SPARK-14886] [MLLIB] RankingMetrics.ndcgAt th...

2016-04-28 Thread MLnick
Github user MLnick commented on the pull request: https://github.com/apache/spark/pull/12756#issuecomment-215535868 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request: [SPARK-14891][ML] Add schema validation for AL...

2016-04-28 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/12762#discussion_r61487974 --- Diff: mllib/src/main/scala/org/apache/spark/ml/recommendation/ALS.scala --- @@ -53,24 +53,43 @@ import org.apache.spark.util.random.XORShiftRandom

[GitHub] spark pull request: [SPARK-12919][SPARKR] Implement dapply() on Da...

2016-04-28 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/12493#discussion_r61487677 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -158,10 +158,15 @@ object EliminateSerialization

[GitHub] spark pull request: [SPARK-14931][ML][PySpark] Mismatched default ...

2016-04-28 Thread yinxusen
Github user yinxusen commented on the pull request: https://github.com/apache/spark/pull/12738#issuecomment-215534356 @holdenk Yes this PR is just a quick fix. I'll create a new JIRA for auditing default values. --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request: [SPARK-14464] [MLLIB] Better support for logis...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12761#issuecomment-215534202 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-4224] [Core][YARN] - Support group acls...

2016-04-28 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/12760#discussion_r61487225 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -2179,6 +2179,16 @@ private[spark] object Utils extends Logging {

[GitHub] spark pull request: [SPARK-14464] [MLLIB] Better support for logis...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12761#issuecomment-215534169 **[Test build #57269 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57269/consoleFull)** for PR 12761 at commit

[GitHub] spark pull request: [SPARK-14464] [MLLIB] Better support for logis...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12761#issuecomment-215534206 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-12919][SPARKR] Implement dapply() on Da...

2016-04-28 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/12493#discussion_r61486972 --- Diff: R/pkg/inst/worker/worker.R --- @@ -84,6 +84,15 @@ broadcastElap <- elapsedSecs() # as number of partitions to create. numPartitions <-

[GitHub] spark pull request: [SPARK-12919][SPARKR] Implement dapply() on Da...

2016-04-28 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/12493#discussion_r61486517 --- Diff: R/pkg/R/DataFrame.R --- @@ -1125,6 +1126,66 @@ setMethod("summarize", agg(x, ...) }) +#' dapply +#'

[GitHub] spark pull request: [SPARK-4224] [Core][YARN] - Support group acls...

2016-04-28 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/12760#discussion_r61486445 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -2179,6 +2179,16 @@ private[spark] object Utils extends Logging {

[GitHub] spark pull request: [SPARK-14850][ML] convert primitive array from...

2016-04-28 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/12640#discussion_r61486213 --- Diff: sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/UnsafeArrayData.java --- @@ -336,4 +336,62 @@ public UnsafeArrayData copy() {

[GitHub] spark pull request: [SPARK-14464] [MLLIB] Better support for logis...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12761#issuecomment-215532308 **[Test build #57269 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57269/consoleFull)** for PR 12761 at commit

[GitHub] spark pull request: [SPARK-14464] [MLLIB] Better support for logis...

2016-04-28 Thread dbtsai
Github user dbtsai commented on the pull request: https://github.com/apache/spark/pull/12761#issuecomment-215532091 Jenkins, ok to test. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-4224] [Core][YARN] - Support group acls...

2016-04-28 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/12760#discussion_r61485864 --- Diff: core/src/main/scala/org/apache/spark/security/ShellBasedGroupsMappingProvider.scala --- @@ -0,0 +1,53 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-14850][ML] convert primitive array from...

2016-04-28 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/12640#discussion_r61485789 --- Diff: mllib/src/test/scala/org/apache/spark/mllib/linalg/UDTSerializationBenchmark.scala --- @@ -0,0 +1,70 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-14891][ML] Add schema validation for AL...

2016-04-28 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/12762#discussion_r61485772 --- Diff: mllib/src/main/scala/org/apache/spark/ml/recommendation/ALS.scala --- @@ -53,24 +53,43 @@ import org.apache.spark.util.random.XORShiftRandom

[GitHub] spark pull request: [SPARK-14613][ML] Add @Since into the matrix a...

2016-04-28 Thread dbtsai
Github user dbtsai commented on the pull request: https://github.com/apache/spark/pull/12416#issuecomment-215531609 @pravingadakh I think you need to update `dev/sparktestsupport/modules.py`. See https://github.com/apache/spark/commit/efaf7d18205f5ae3a1c767942ee7d7320f7410de

[GitHub] spark pull request: [SPARK-14464] [MLLIB] Better support for logis...

2016-04-28 Thread daniel-siegmann-aol
Github user daniel-siegmann-aol commented on the pull request: https://github.com/apache/spark/pull/12761#issuecomment-215531118 I should mention Nick Pentreath pointed me in the right direction on the dev list. So, credit to him for the help! --- If your project is set up for it,

[GitHub] spark pull request: [SPARK-11940][PYSPARK][ML] Python API for ml.c...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12723#issuecomment-215531053 **[Test build #57268 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57268/consoleFull)** for PR 12723 at commit

[GitHub] spark pull request: [SPARK-14098][SQL] Generate Java code that get...

2016-04-28 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/11956#issuecomment-215530899 @kiszk Can we target this for 2.1? We may switch to use Apache Arrow or other in-memory representation instead of current one. --- If your project is set up for it,

[GitHub] spark pull request: [SPARK-14891][ML] Add schema validation for AL...

2016-04-28 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/12762#discussion_r61485103 --- Diff: mllib/src/main/scala/org/apache/spark/ml/recommendation/ALS.scala --- @@ -53,24 +53,43 @@ import org.apache.spark.util.random.XORShiftRandom

[GitHub] spark pull request: [SPARK-4224] [Core][YARN] - Support group acls...

2016-04-28 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/12760#discussion_r61485159 --- Diff: core/src/main/scala/org/apache/spark/security/ShellBasedGroupsMappingProvider.scala --- @@ -0,0 +1,53 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request: [SPARK-11940][PYSPARK][ML] Python API for ml.c...

2016-04-28 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/12723#issuecomment-215530890 Updated! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-14970][SQL] Prevent DataSource from enu...

2016-04-28 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/12748#issuecomment-215530907 We might still disable schema inference, but until then this LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: [SPARK-1989] [CORE] Exit executors faster if t...

2016-04-28 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/12571#issuecomment-215530798 thanks for pointing that out, I made similar comments on the jira in case others don't agree with me we can discuss more. --- If your project is set up for it, you

[GitHub] spark pull request: [SPARK-14511][Build] Upgrade genjavadoc to lat...

2016-04-28 Thread jodersky
Github user jodersky commented on the pull request: https://github.com/apache/spark/pull/12707#issuecomment-215530295 @srowen, I did quite some digging through the genjavadoc codebase when re-implementing @mengxr's initial fix. I can try to fix the group warnings and object privacy

[GitHub] spark pull request: [SPARK-14891][ML] Add schema validation for AL...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12762#issuecomment-215529657 **[Test build #57267 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57267/consoleFull)** for PR 12762 at commit

[GitHub] spark pull request: [SPARK-14894][PySpark] Add result summary api ...

2016-04-28 Thread GayathriMurali
Github user GayathriMurali commented on the pull request: https://github.com/apache/spark/pull/12675#issuecomment-215529653 @jkbradley Can you please ok to test this --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-14981][SQL] Throws exception if DESC is...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12759#issuecomment-215527193 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request: [SPARK-14891][ML] Add schema validation for AL...

2016-04-28 Thread MLnick
Github user MLnick commented on the pull request: https://github.com/apache/spark/pull/12762#issuecomment-215528792 cc @sethah @BenFradet @srowen @jkbradley @yanboliang --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request: [SPARK-14891][ML] Add schema validation for AL...

2016-04-28 Thread MLnick
GitHub user MLnick opened a pull request: https://github.com/apache/spark/pull/12762 [SPARK-14891][ML] Add schema validation for ALS This PR adds schema validation to `ml`'s ALS and ALSModel. Currently, no schema validation was performed as `transformSchema` was never called in

[GitHub] spark pull request: [SPARK-14916] [MLlib] A more friendly tostring...

2016-04-28 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/12698 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request: [SPARK-14981][SQL] Throws exception if DESC is...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12759#issuecomment-215527190 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-14916] [MLlib] A more friendly tostring...

2016-04-28 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/12698#issuecomment-215526823 Merged to master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-14981][SQL] Throws exception if DESC is...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12759#issuecomment-215527012 **[Test build #57264 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57264/consoleFull)** for PR 12759 at commit

[GitHub] spark pull request: [SPARK-14464] [MLLIB] Better support for logis...

2016-04-28 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12761#issuecomment-215526701 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-4224] [Core][YARN] - Support group acls...

2016-04-28 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/12760#discussion_r61482000 --- Diff: core/src/main/scala/org/apache/spark/SecurityManager.scala --- @@ -303,16 +321,33 @@ private[spark] class SecurityManager(sparkConf: SparkConf)

[GitHub] spark pull request: [SPARK-14882] [DOCS] Clarify that Spark can be...

2016-04-28 Thread benmccann
Github user benmccann commented on a diff in the pull request: https://github.com/apache/spark/pull/12757#discussion_r61480996 --- Diff: docs/programming-guide.md --- @@ -24,7 +24,8 @@ along with if you launch Spark's interactive shell -- either `bin/spark-shell` f

[GitHub] spark pull request: [SPARK-11321] [SQL] Python non null udfs

2016-04-28 Thread kevincox
Github user kevincox commented on the pull request: https://github.com/apache/spark/pull/12335#issuecomment-215525434 @davies You mean to support non-null return values? I don't think I know enough scala to automatically infer that. --- If your project is set up for it, you can

[GitHub] spark pull request: [SPARK-14464] [MLLIB] Better support for logis...

2016-04-28 Thread daniel-siegmann-aol
GitHub user daniel-siegmann-aol opened a pull request: https://github.com/apache/spark/pull/12761 [SPARK-14464] [MLLIB] Better support for logistic regression when features are sparse ## What changes were proposed in this pull request? Where aggregations were being done

[GitHub] spark pull request: [SPARK-14882] [DOCS] Clarify that Spark can be...

2016-04-28 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/12757#discussion_r61481646 --- Diff: docs/programming-guide.md --- @@ -24,7 +24,8 @@ along with if you launch Spark's interactive shell -- either `bin/spark-shell` f

[GitHub] spark pull request: [SPARK-11321] [SQL] Python non null udfs

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12335#issuecomment-215524881 **[Test build #2903 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2903/consoleFull)** for PR 12335 at commit

[GitHub] spark pull request: [SPARK-11321] [SQL] Python non null udfs

2016-04-28 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/12335#issuecomment-215524585 @kevincox Could you also Update the Scala UDF? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request: [SPARK-14098][SQL] Generate Java code that get...

2016-04-28 Thread kiszk
Github user kiszk commented on the pull request: https://github.com/apache/spark/pull/11956#issuecomment-215524446 We can shrink the size of this PR (reducing the number of lines from 1800 to 700) by dropping test suites (1450 lines) and a benchmark program (250 lines). Is it better

[GitHub] spark pull request: [SPARK-11321] [SQL] Python non null udfs

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12335#issuecomment-215524403 **[Test build #2903 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2903/consoleFull)** for PR 12335 at commit

[GitHub] spark pull request: [SPARK-14315][SparkR]Add model persistence to ...

2016-04-28 Thread GayathriMurali
Github user GayathriMurali commented on a diff in the pull request: https://github.com/apache/spark/pull/12683#discussion_r61481184 --- Diff: R/pkg/inst/tests/testthat/test_mllib.R --- @@ -71,7 +71,25 @@ test_that("glm and predict", { data = iris, family =

[GitHub] spark pull request: [SPARK-14945][PYTHON] SparkSession Python API

2016-04-28 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/12746#issuecomment-215523928 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request: [SPARK-1989] [CORE] Exit executors faster if t...

2016-04-28 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/12571#issuecomment-215521904 I'm OK with not doing this, I think the contributor was just following up on an old idea from Matei. --- If your project is set up for it, you can reply to this email

[GitHub] spark pull request: [SPARK-1989] [CORE] Exit executors faster if t...

2016-04-28 Thread tgravescs
Github user tgravescs commented on the pull request: https://github.com/apache/spark/pull/12571#issuecomment-215521675 I understand the argument of we want the best user experience and I'm not against the settings themselves, I just think the benefit isn't worth the cost here.

[GitHub] spark pull request: [SPARK-4224] [Core, YARN] - Support group acls...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12760#issuecomment-215519529 **[Test build #57266 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/57266/consoleFull)** for PR 12760 at commit

[GitHub] spark pull request: [SPARK-11171][SPARK-11237][SPARK-11241][ML] Tr...

2016-04-28 Thread mengxr
Github user mengxr commented on the pull request: https://github.com/apache/spark/pull/9207#issuecomment-215519584 I want to hold PMML support in spark.ml for Spark 2.1. We might need to discuss more about the potential coverage we can get, especially on pipeline support and feature

[GitHub] spark pull request: [SPARK-14965] [SQL] Indicate an exception is t...

2016-04-28 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12758#issuecomment-215519366 **[Test build #2901 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2901/consoleFull)** for PR 12758 at commit

[GitHub] spark pull request: [SPARK-12154] Upgrade to Jersey 2

2016-04-28 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/12715#issuecomment-215519249 LGTM, I'll leave this for @srowen since he had a comment about licenses. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request: [SPARK-12154] Upgrade to Jersey 2

2016-04-28 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/12715#discussion_r61477968 --- Diff: dev/deps/spark-deps-hadoop-2.3 --- @@ -73,32 +74,41 @@ hadoop-yarn-client-2.3.0.jar hadoop-yarn-common-2.3.0.jar

[GitHub] spark pull request: [SPARK-10931][PYSPARK][ML] PySpark ML Models s...

2016-04-28 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/10270#issuecomment-215519160 OK, thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

<    1   2   3   4   5   6   7   8   9   >