[GitHub] spark pull request: [MINOR][ML][PYTHON][DOC] Remove use of JavaMLW...

2016-04-25 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/12542#issuecomment-214464015 @yanboliang Thanks! Merging with master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your pr

[GitHub] spark pull request: [SPARK-14732][ML] spark.ml GaussianMixture sho...

2016-04-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12593#issuecomment-214464159 **[Test build #56906 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56906/consoleFull)** for PR 12593 at commit [`3d310f2`](https://gi

[GitHub] spark pull request: [MINOR][ML][PYTHON][DOC] Remove use of JavaMLW...

2016-04-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/12542 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request: [SPARK-14721][SQL] Remove HiveContext (part 2)

2016-04-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12585#issuecomment-214465486 **[Test build #56907 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56907/consoleFull)** for PR 12585 at commit [`f59b869`](https://gi

[GitHub] spark pull request: [SPARK-14878][SQL] Trim characters string func...

2016-04-25 Thread kevinyu98
Github user kevinyu98 commented on the pull request: https://github.com/apache/spark/pull/12646#issuecomment-214465479 Hello Dongjoon: Thanks for your comments, I will make changes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request: [SPARK-13857][ML][WIP] Add "recommend all" fun...

2016-04-25 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/12574#discussion_r60959688 --- Diff: mllib/src/main/scala/org/apache/spark/ml/recommendation/ALS.scala --- @@ -200,8 +240,42 @@ class ALSModel private[ml] ( @Since("1.3.0")

[GitHub] spark pull request: [SPARK-14862][ML] Updated Classifiers to not r...

2016-04-25 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/12663#discussion_r60959821 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/GBTClassifier.scala --- @@ -126,16 +126,8 @@ final class GBTClassifier @Since("1.4.0") (

[GitHub] spark pull request: [SPARK-14346][SQL] Show Create Table (Native)

2016-04-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12579#issuecomment-214466180 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-13857][ML][WIP] Add "recommend all" fun...

2016-04-25 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/12574#discussion_r60960027 --- Diff: mllib/src/main/scala/org/apache/spark/ml/recommendation/ALS.scala --- @@ -374,9 +599,26 @@ class ALS(@Since("1.4.0") override val uid: String)

[GitHub] spark pull request: [SPARK-14346][SQL] Show Create Table (Native)

2016-04-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12579#issuecomment-214466178 Build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request: [SPARK-14346][SQL] Show Create Table (Native)

2016-04-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12579#issuecomment-214465973 **[Test build #56899 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56899/consoleFull)** for PR 12579 at commit [`13e9775`](https://g

[GitHub] spark pull request: [SPARK-14721][SQL] Remove HiveContext (part 2)

2016-04-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12585#issuecomment-214466313 **[Test build #2868 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2868/consoleFull)** for PR 12585 at commit [`f59b869`](https://g

[GitHub] spark pull request: [SPARK-14721][SQL] Remove HiveContext (part 2)

2016-04-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12585#issuecomment-214466387 **[Test build #2870 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2870/consoleFull)** for PR 12585 at commit [`f59b869`](https://g

[GitHub] spark pull request: [SPARK-14721][SQL] Remove HiveContext (part 2)

2016-04-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12585#issuecomment-214466355 **[Test build #2869 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2869/consoleFull)** for PR 12585 at commit [`f59b869`](https://g

[GitHub] spark pull request: [SPARK-14892] [SQL] [TEST] Disable the HiveCom...

2016-04-25 Thread gatorsmile
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/12662#issuecomment-214467967 @rxin @yhuai @hvanhovell --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] spark pull request: Fix dynamic allocation docs to address cached ...

2016-04-25 Thread mgummelt
GitHub user mgummelt opened a pull request: https://github.com/apache/spark/pull/12664 Fix dynamic allocation docs to address cached data. ## What changes were proposed in this pull request? Documentation changes ## How was this patch tested? No tests

[GitHub] spark pull request: Fix dynamic allocation docs to address cached ...

2016-04-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12664#issuecomment-214469702 **[Test build #56908 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56908/consoleFull)** for PR 12664 at commit [`05a3f45`](https://gi

[GitHub] spark pull request: [SPARK-14716][SQL] Added support for partition...

2016-04-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12409#issuecomment-214469692 **[Test build #2871 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2871/consoleFull)** for PR 12409 at commit [`fc6a325`](https://g

[GitHub] spark pull request: [SPARK-14862][ML] Updated Classifiers to not r...

2016-04-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12663#issuecomment-214469708 **[Test build #56904 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56904/consoleFull)** for PR 12663 at commit [`94e206f`](https://g

[GitHub] spark pull request: [SPARK-14862][ML] Updated Classifiers to not r...

2016-04-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12663#issuecomment-214469877 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14888][SQL] UnresolvedFunction should u...

2016-04-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12659#issuecomment-214469887 **[Test build #56902 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56902/consoleFull)** for PR 12659 at commit [`b87838b`](https://g

[GitHub] spark pull request: [SPARK-14862][ML] Updated Classifiers to not r...

2016-04-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12663#issuecomment-214469882 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14888][SQL] UnresolvedFunction should u...

2016-04-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12659#issuecomment-214470077 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14888][SQL] UnresolvedFunction should u...

2016-04-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12659#issuecomment-214470079 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-12919][SPARKR] Implement dapply() on Da...

2016-04-25 Thread shaneknapp
Github user shaneknapp commented on the pull request: https://github.com/apache/spark/pull/12493#issuecomment-214470548 it's because the github pull request builder generally sucks. :) anyways, looking in to it now. --- If your project is set up for it, you can reply to this

[GitHub] spark pull request: [SPARK-14716][SQL] Added support for partition...

2016-04-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12409#issuecomment-214470777 **[Test build #2871 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2871/consoleFull)** for PR 12409 at commit [`fc6a325`](https://

[GitHub] spark pull request: [SPARK-12919][SPARKR] Implement dapply() on Da...

2016-04-25 Thread shaneknapp
Github user shaneknapp commented on the pull request: https://github.com/apache/spark/pull/12493#issuecomment-214470976 jenkins, test this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not ha

[GitHub] spark pull request: [SPARK-14732][ML] spark.ml GaussianMixture sho...

2016-04-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12593#issuecomment-214471592 **[Test build #56906 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56906/consoleFull)** for PR 12593 at commit [`3d310f2`](https://g

[GitHub] spark pull request: Fix dynamic allocation docs to address cached ...

2016-04-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12664#issuecomment-214471598 **[Test build #56908 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56908/consoleFull)** for PR 12664 at commit [`05a3f45`](https://g

[GitHub] spark pull request: [SPARK-14732][ML] spark.ml GaussianMixture sho...

2016-04-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12593#issuecomment-214471646 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14433][PySpark][ML]:PySpark ml Gaussian...

2016-04-25 Thread wangmiao1981
Github user wangmiao1981 commented on a diff in the pull request: https://github.com/apache/spark/pull/12402#discussion_r60963521 --- Diff: python/pyspark/ml/clustering.py --- @@ -22,7 +22,151 @@ from pyspark.mllib.common import inherit_doc __all__ = ['BisectingKMean

[GitHub] spark pull request: [SPARK-14732][ML] spark.ml GaussianMixture sho...

2016-04-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12593#issuecomment-214471645 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: Fix dynamic allocation docs to address cached ...

2016-04-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12664#issuecomment-214471719 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: Fix dynamic allocation docs to address cached ...

2016-04-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12664#issuecomment-214471725 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14346][SQL] Show Create Table (Native)

2016-04-25 Thread xwu0226
Github user xwu0226 commented on the pull request: https://github.com/apache/spark/pull/12579#issuecomment-214472079 @liancheng Thanks for triggering the test! I am looking into the test failure. --- If your project is set up for it, you can reply to this email and have your reply a

[GitHub] spark pull request: Fix dynamic allocation docs to address cached ...

2016-04-25 Thread srowen
Github user srowen commented on the pull request: https://github.com/apache/spark/pull/12664#issuecomment-214472099 OK by me. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enab

[GitHub] spark pull request: [SPARK-11399] Add label support in include_exa...

2016-04-25 Thread yinxusen
Github user yinxusen commented on the pull request: https://github.com/apache/spark/pull/9430#issuecomment-214472349 I'll close this one according to https://github.com/apache/spark/pull/11128 for now --- If your project is set up for it, you can reply to this email and have your rep

[GitHub] spark pull request: [SPARK-11399] Add label support in include_exa...

2016-04-25 Thread yinxusen
Github user yinxusen closed the pull request at: https://github.com/apache/spark/pull/9430 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is en

[GitHub] spark pull request: [SPARK-12919][SPARKR] Implement dapply() on Da...

2016-04-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12493#issuecomment-214472422 **[Test build #56909 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56909/consoleFull)** for PR 12493 at commit [`8aaa91d`](https://gi

[GitHub] spark pull request: [SPARK-14756][Core] Use parseLong instead of v...

2016-04-25 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/12520#discussion_r60964326 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningUtils.scala --- @@ -441,7 +441,7 @@ private[sql] object Partitioning

[GitHub] spark pull request: [SPARK-12919][SPARKR] Implement dapply() on Da...

2016-04-25 Thread shaneknapp
Github user shaneknapp commented on the pull request: https://github.com/apache/spark/pull/12493#issuecomment-214473135 i guess it just likes me more than you both. ;) i'll triple-check the whitelist and see if something broke there. as usual, there's nothing in the logs exc

[GitHub] spark pull request: [SPARK-14571][ML]Log instrumentation in ALS

2016-04-25 Thread wangmiao1981
Github user wangmiao1981 commented on the pull request: https://github.com/apache/spark/pull/12560#issuecomment-214474647 @MLnick Yanbo does not like the change of train() API. The new parameter is optional, so the user of train should not be aware of this change. In addition, I think

[GitHub] spark pull request: [SPARK-14861][SQL] Replace internal usages of ...

2016-04-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12625#issuecomment-214475204 **[Test build #56910 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56910/consoleFull)** for PR 12625 at commit [`2c88b4e`](https://gi

[GitHub] spark pull request: [SPARK-14861][SQL] Replace internal usages of ...

2016-04-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12625#issuecomment-214476630 **[Test build #56911 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56911/consoleFull)** for PR 12625 at commit [`521292e`](https://gi

[GitHub] spark pull request: [SPARK-14732][ML] spark.ml GaussianMixture sho...

2016-04-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12593#issuecomment-214478040 **[Test build #56912 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56912/consoleFull)** for PR 12593 at commit [`98f398c`](https://gi

[GitHub] spark pull request: [SPARK-14862][ML] Updated Classifiers to not r...

2016-04-25 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/12663#discussion_r60967938 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/Classifier.scala --- @@ -62,6 +65,76 @@ abstract class Classifier[ def setRawPredicti

[GitHub] spark pull request: [MINOR][DOCS] Update license header of LZ4Bloc...

2016-04-25 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/12665 [MINOR][DOCS] Update license header of LZ4BlockInputStream.java and remove it from RAT exclusion list. ## What changes were proposed in this pull request? `LZ4BlockInputStream.java`

[GitHub] spark pull request: [SPARK-6717][ML] Clear shuffle files after che...

2016-04-25 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/11919#discussion_r60968178 --- Diff: mllib/src/main/scala/org/apache/spark/ml/recommendation/ALS.scala --- @@ -656,13 +656,15 @@ object ALS extends DefaultParamsReadable[ALS] with Log

[GitHub] spark pull request: [SPARK-13962][ML] spark.ml Evaluators should s...

2016-04-25 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/12500#discussion_r60968226 --- Diff: mllib/src/test/scala/org/apache/spark/ml/util/MLTestingUtils.scala --- @@ -47,14 +48,32 @@ object MLTestingUtils extends SparkFunSuite { va

[GitHub] spark pull request: [MINOR][DOCS] Update license header of LZ4Bloc...

2016-04-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12665#issuecomment-214479554 **[Test build #56913 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56913/consoleFull)** for PR 12665 at commit [`879fda9`](https://gi

[GitHub] spark pull request: [MINOR][DOCS] Update license header of LZ4Bloc...

2016-04-25 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12665#issuecomment-214479711 Hi, @davies . I made this PR to ask your opinion. - Can we remove `LZ4BlockInputStream.java` from RAT exclusion file like other files? - Can we remove

[GitHub] spark pull request: [SPARK-14862][ML] Updated Classifiers to not r...

2016-04-25 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/12663#discussion_r60969068 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/GBTClassifier.scala --- @@ -126,16 +126,8 @@ final class GBTClassifier @Since("1.4.0") (

[GitHub] spark pull request: [SPARK-14862][ML] Updated Classifiers to not r...

2016-04-25 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/12663#discussion_r60969026 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/Classifier.scala --- @@ -62,6 +65,76 @@ abstract class Classifier[ def setRawPredicti

[GitHub] spark pull request: [SPARK-14892] [SQL] [TEST] Disable the HiveCom...

2016-04-25 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/12662#issuecomment-214481012 LGTM Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-13962][ML] spark.ml Evaluators should s...

2016-04-25 Thread MLnick
Github user MLnick commented on the pull request: https://github.com/apache/spark/pull/12500#issuecomment-214481330 @BenFradet just one more minor style comment, then I think this is ready to merge. --- If your project is set up for it, you can reply to this email and have your repl

[GitHub] spark pull request: [SPARK-7264][ML] Parallel lapply for sparkR

2016-04-25 Thread shivaram
Github user shivaram commented on the pull request: https://github.com/apache/spark/pull/12426#issuecomment-214481734 FWIW the error from Jenkins is `Error in namespaceExport(ns, exports) : undefined exports: sparkLapply` --- If your project is set up for it, you can reply to this em

[GitHub] spark pull request: [SPARK-14747][SQL] Add assertStreaming/assertN...

2016-04-25 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/12521#discussion_r60969585 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/DataFrameReaderWriterSuite.scala --- @@ -368,4 +368,79 @@ class DataFrameReaderWriterSuite

[GitHub] spark pull request: [SPARK-14892] [SQL] [TEST] Disable the HiveCom...

2016-04-25 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/12662#issuecomment-214481742 Just a note. The test in that file is ``` create table test (a int) stored as inputformat 'org.apache.hadoop.hive.ql.io.RCFileInputFormat' outputformat 'org.apac

[GitHub] spark pull request: [SPARK-6717][ML] Clear shuffle files after che...

2016-04-25 Thread MLnick
Github user MLnick commented on a diff in the pull request: https://github.com/apache/spark/pull/11919#discussion_r60969566 --- Diff: mllib/src/main/scala/org/apache/spark/ml/recommendation/ALS.scala --- @@ -656,13 +656,15 @@ object ALS extends DefaultParamsReadable[ALS] with Logg

[GitHub] spark pull request: [SPARK-14892] [SQL] [TEST] Disable the HiveCom...

2016-04-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/12662 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request: [SPARK-14747][SQL] Add assertStreaming/assertN...

2016-04-25 Thread marmbrus
Github user marmbrus commented on the pull request: https://github.com/apache/spark/pull/12521#issuecomment-214482001 /cc @tdas --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature e

[GitHub] spark pull request: [SPARK-10574] [ML] [MLlib] HashingTF supports ...

2016-04-25 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/12498#issuecomment-214482165 This LGTM. We'll still need to update the pyspark.mllib implementation. @yanboliang I'll merge this with master, but could you please do that in a follow-up PR?

[GitHub] spark pull request: [SPARK-6717][ML] Clear shuffle files after che...

2016-04-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11919#issuecomment-214482289 **[Test build #56915 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56915/consoleFull)** for PR 11919 at commit [`e980596`](https://gi

[GitHub] spark pull request: [SPARK-14862][ML] Updated Classifiers to not r...

2016-04-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12663#issuecomment-214482287 **[Test build #56914 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56914/consoleFull)** for PR 12663 at commit [`657faef`](https://gi

[GitHub] spark pull request: [SPARK-14889][Spark Core] scala.MatchError: NO...

2016-04-25 Thread sbcd90
GitHub user sbcd90 opened a pull request: https://github.com/apache/spark/pull/12666 [SPARK-14889][Spark Core] scala.MatchError: NONE (of class scala.Enumeration) when spark.scheduler.mode=NONE ## What changes were proposed in this pull request? Handling exception for the b

[GitHub] spark pull request: [MINOR][DOCS] Update license header of LZ4Bloc...

2016-04-25 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/12665#issuecomment-214483662 This PR contains only comment updates and moves, and RAT check is passed. ``` R

[GitHub] spark pull request: [SPARK-10574] [ML] [MLlib] HashingTF supports ...

2016-04-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/12498 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request: [SPARK-14889][Spark Core] scala.MatchError: NO...

2016-04-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12666#issuecomment-214484848 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your p

[GitHub] spark pull request: [SPARK-10574] [ML] [MLlib] HashingTF supports ...

2016-04-25 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/12498#issuecomment-214485807 Update after speaking with @mengxr Since we're breaking behavior anyways with with PR, perhaps we should try to enforce good practice by: * Removing "native"

[GitHub] spark pull request: [SPARK-10574] [ML] [MLlib] HashingTF supports ...

2016-04-25 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/12498#issuecomment-214486402 Here it is: https://issues.apache.org/jira/browse/SPARK-14899 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub a

[GitHub] spark pull request: [SPARK-14888][SQL] UnresolvedFunction should u...

2016-04-25 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/12659#discussion_r60971154 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -622,40 +620,44 @@ class SessionCatalog( * bas

[GitHub] spark pull request: [SPARK-14862][ML] Updated Classifiers to not r...

2016-04-25 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/12663#discussion_r60971502 --- Diff: mllib/src/main/scala/org/apache/spark/ml/classification/Classifier.scala --- @@ -62,6 +65,76 @@ abstract class Classifier[ def setRawPredi

[GitHub] spark pull request: [SPARK-14888][SQL] UnresolvedFunction should u...

2016-04-25 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/12659#issuecomment-214488360 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled an

[GitHub] spark pull request: [SPARK-14571][ML]Log instrumentation in ALS

2016-04-25 Thread MLnick
Github user MLnick commented on the pull request: https://github.com/apache/spark/pull/12560#issuecomment-214490075 @wangmiao1981 I tend to agree with @yanboliang. I don't think adding instrumentation is critical enough to break the `train` method signature, even if it is `DeveloperAp

[GitHub] spark pull request: [SPARK-14731][shuffle]Revert SPARK-12130 to ma...

2016-04-25 Thread vanzin
Github user vanzin commented on the pull request: https://github.com/apache/spark/pull/12568#issuecomment-214492945 LGTM. Merging to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have th

[GitHub] spark pull request: [SPARK-13962][ML] spark.ml Evaluators should s...

2016-04-25 Thread BenFradet
Github user BenFradet commented on the pull request: https://github.com/apache/spark/pull/12500#issuecomment-214493135 @MLnick will do --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this fea

[GitHub] spark pull request: [SPARK-14828][SQL] Start SparkSession in REPL ...

2016-04-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12589#issuecomment-214493425 **[Test build #56905 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56905/consoleFull)** for PR 12589 at commit [`e69d7cf`](https://g

[GitHub] spark pull request: [SPARK-14828][SQL] Start SparkSession in REPL ...

2016-04-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12589#issuecomment-214493611 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14731][shuffle]Revert SPARK-12130 to ma...

2016-04-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/12568 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request: [SPARK-14828][SQL] Start SparkSession in REPL ...

2016-04-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12589#issuecomment-214493617 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14862][ML] Updated Classifiers to not r...

2016-04-25 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/12663#issuecomment-214493865 I'll wait to update this until your reviews are done. Thanks for taking a look! --- If your project is set up for it, you can reply to this email and have your repl

[GitHub] spark pull request: [SPARK-14732][ML] spark.ml GaussianMixture sho...

2016-04-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12593#issuecomment-214494351 **[Test build #56912 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56912/consoleFull)** for PR 12593 at commit [`98f398c`](https://g

[GitHub] spark pull request: [SPARK-14732][ML] spark.ml GaussianMixture sho...

2016-04-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12593#issuecomment-214494500 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14732][ML] spark.ml GaussianMixture sho...

2016-04-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12593#issuecomment-214494498 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14829][MLLIB] Deprecate GLM APIs using ...

2016-04-25 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/12596#discussion_r60974980 --- Diff: python/pyspark/mllib/classification.py --- @@ -312,6 +313,10 @@ def train(cls, data, iterations=100, step=1.0, miniBatchFraction=1.0,

[GitHub] spark pull request: [SPARK-14829][MLLIB] Deprecate GLM APIs using ...

2016-04-25 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/12596#issuecomment-214495321 Thanks for the PR. This should also include: * LassoWithSGD (recommend using LinearRegression with elasticNetParam = 1.0) * RidgeRegressionWithSGD (recommend u

[GitHub] spark pull request: [SPARK-14433][PySpark][ML]:PySpark ml Gaussian...

2016-04-25 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/12402#discussion_r60975535 --- Diff: python/pyspark/ml/clustering.py --- @@ -22,7 +22,151 @@ from pyspark.mllib.common import inherit_doc __all__ = ['BisectingKMeans',

[GitHub] spark pull request: [SPARK-14862][ML] Updated Classifiers to not r...

2016-04-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12663#issuecomment-214496772 **[Test build #56914 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56914/consoleFull)** for PR 12663 at commit [`657faef`](https://g

[GitHub] spark pull request: [SPARK-14862][ML] Updated Classifiers to not r...

2016-04-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12663#issuecomment-214496922 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14862][ML] Updated Classifiers to not r...

2016-04-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12663#issuecomment-214496921 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-14721][SQL] Remove HiveContext (part 2)

2016-04-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12585#issuecomment-214497498 **[Test build #2868 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2868/consoleFull)** for PR 12585 at commit [`f59b869`](https://

[GitHub] spark pull request: [SPARK-13962][ML] spark.ml Evaluators should s...

2016-04-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12500#issuecomment-214498270 **[Test build #56916 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56916/consoleFull)** for PR 12500 at commit [`6c66068`](https://gi

[GitHub] spark pull request: [SPARK-14721][SQL] Remove HiveContext (part 2)

2016-04-25 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/12585#issuecomment-214498735 **[Test build #56907 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/56907/consoleFull)** for PR 12585 at commit [`f59b869`](https://g

[GitHub] spark pull request: [SPARK-14721][SQL] Remove HiveContext (part 2)

2016-04-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12585#issuecomment-214499333 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-14721][SQL] Remove HiveContext (part 2)

2016-04-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/12585#issuecomment-214499329 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-7264][ML] Parallel lapply for sparkR

2016-04-25 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/12426#discussion_r60977638 --- Diff: R/pkg/NAMESPACE --- @@ -287,6 +287,7 @@ export("as.DataFrame", "read.json", "read.parquet", "read.text",

[GitHub] spark pull request: [SPARK-7264][ML] Parallel lapply for sparkR

2016-04-25 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/12426#discussion_r60977738 --- Diff: R/pkg/R/context.R --- @@ -226,6 +226,47 @@ setCheckpointDir <- function(sc, dirName) { invisible(callJMethod(sc, "setCheckpointDir", s

[GitHub] spark pull request: [SPARK-7264][ML] Parallel lapply for sparkR

2016-04-25 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/12426#discussion_r60977845 --- Diff: R/pkg/inst/tests/testthat/test_context.R --- @@ -141,3 +141,8 @@ test_that("sparkJars sparkPackages as comma-separated strings", { exp

[GitHub] spark pull request: Fix dynamic allocation docs to address cached ...

2016-04-25 Thread Mpn177
Github user Mpn177 commented on the pull request: https://github.com/apache/spark/pull/12664#issuecomment-214501325 Don't see any problems with this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does no

[GitHub] spark pull request: [MINOR][DOCS] Update license header of LZ4Bloc...

2016-04-25 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/12665#issuecomment-214502099 LZ4BlockInputStream.java is NOT licensed to Apache Software Foundation (ASF), even it's licensed under Apache License. --- If your project is set up for it, you can rep

<    1   2   3   4   5   6   7   8   >